After 20+ years of immersion in ECM and KM, I still find myself in awe and amazed. As I have commented before, ECM and KM are forever changing in ways that astonish. Managing “stuff” that seems to defy management, and mimicking the human brain’s ability to infer meaning, spot associations and discover insights locked away in words and pictures, is not an easy thing to do. We have come light years from the 1960s when IDI’s BASIS and IBM’s STAIRS heralded the ability of software to mine content from a library of documents. But, excitingly so, we have a long way to go. Perhaps the greatest opportunity and challenge is associated with retrieval/search tools. Not only in how they work, but how they are leveraged.
My position as KM Director at FSG, juxtaposes this fascination of mine with an organization that focuses on issues even greater than “how to build the smartest search tool.” The work of FSG, to put it succinctly, is to solve the world’s greatest problems (e.g., world hunger). So I find myself in a perfect storm, awash in an invigorating monsoon of innovation and creativity - great thinkers and creative problem solvers who can benefit from creative and powerful approaches to KM. OK – I’ll get to the point, why am I so excited right now?
A few weeks back I saw a phenomenal show on Nova about the IBM project that led to the creation of Watson – the search engine that became famous when it was a challenger on the popular TV show Jeopardy. If you are the least bit interested in how search, heuristic learning and lexical analysis/meaning inference and content mining work, I highly recommend you watch the Nova Watson special. It is AMAZING. (My family had to put up with me in the living room periodically murmuring “Wow”, “that’s so cool”, and “I knew it.”, (confirming in their minds what an ECM geek I am.)
What made the Nova show so fascinating was not watching Watson eventually win at Jeopardy, but the process the developers at IBM went through to bring the search engine to this state. It was a powerful example of what it takes to fine tune a software system to interact with questions and a body of content in order to, not just retrieve content, but bubble up answers. FASCINATING.
Serendipitously, I am at a stage in the development of the FSG KM system, where we are revisiting the search function. As I have said time and again, search is pivotal to the value of an ECM/KM system, and search is not a one-size fits all technology. (Watson is but one example of the richness that can be embodied in “search”). So yes, we are “revisting” search, with eyes set at something that goes beyond our current word and phrase searching capability. Now, I readily admit that my talents and my arsenal are nowhere near those of the team at IBM that built Watson, but I drew great inspiration from them. They demonstrated how lexical analysis, clustering, word pattern recognition and heuristics executed at the speed of a computer, tailored to a particular situation, can provide impressive recall and precision, as well as inference and “discovery”. (I cannot help but wax nostalgic a time when explanation of a search tool was met with blurred eyes. Now virtually everyone gets it; Google is a household word. We are fortunate to be living in a time when many “off the shelf” search tools provide rich functionality. We are planning on starting our re-visit with a trial of Vivisimo.)
Watson can beat the best of the best Jeopardy champions. It deciphers questions posed in natural language, and looks at possible answers based on experience, patterns and connections. It does not retrieve a collection of content, but pinpoints potential answers. My point is “search” has become increasingly powerful. I often recall conversations I had with Dr. Matthew Koll many years ago, who used to enjoy posing to audiences the question “What if you could obtain perfect precision and recall, what would you do with it?”.
Therein lies the real challenge - right? Watson demonstrates the possibility of achieving Koll’s dream – perfect recall with instinctive precision. But, then what? It’s fun to watch Watson beat humans at answering trivia questions – but that is not the goal of this technology, is it? FSG’s KM system is light years behind Watson, but, if it can achieve 1% the impact that Watson achieved, it will matter. FSGers are faced not with answering challenging trivia questions, but finding solutions to much more complex issues – and they so with creative approaches and determination. ECMers such as myself, are faced with daunting challenges of a different sort. Together, there is that perfect storm I mentioned earlier; that mission that stokes my fire. I will be sure to revisit this issue based on our experience. Stay tuned …
