In the days leading to the 10th anniversary of what is now simply known as "9/11", it is difficult not to encounter media reliving that moment and/or looking back on its aftermath. The focus of this blog has never been political, and I will not start now. But, in today's Metro, I read an article that, while focused on 9/11 had a ECM undertone to it, perhaps only picked up by enterprise content management (ECM) geeks such as myself. The article, "The US Lexicon: New Words for a Changed World", provides an interesting overview of the effects of 9/11 on the American vocabulary. 9/11: now not just a date or series of numbers, but a label that evokes a day in history; a turning point. "War on Terror" a phrase and concept that did not exist prior. "Ground Zero", no longer the area closest to a bomb explosion, but a specific place in lower Manhattan.
Some three to four decades ago when I spent much time enlightening business and technical audiences on the capabilities of search engines (then full-text retrieval; yes even ECM has an evolving lexicon), the ability for an engine to keep pace with a language over time, or to adapt to a lexicon belonging to a specific "Community of practice", was a critical concept to appreciate and leverage.
Text retrieval grew from simple word-based queries into various intelligent approaches to understanding the meaning contained in words and phrases, to not just search on exact text strings, but to intelligently process the written word, the language and expand retrieval to intelligent adaptive concept-based retrieval. Foundation to this is customizable and/or adaptive lexicon management.
Search technology has come a long long way. But still retrieval tools, especially some of the more powerful were acquired and bundled into larger ECM platforms - e.g. FAST acquisition by Microsoft. For the most part lexical capabilities, or intelligent adaptive search technology is only available inside of large ECM platforms. Six months ago I blogged about my experience with small-medium ECM platfomrs that bundle retrieval, but typically in the form of more simple tools, e.g., Lucene. These tools do not adapt, comprehend, morph or expand a search into its various permutations, to treat retrieval conceptually, and/or support customized lexicons.
For those who have never experienced an "intelligent search tool", it is easy to be led to believe that simpler search engines such as Lucene are the state of the industry. But, standalone search tools, such as Vivisimo, Coveo and Endeca still do exist, and it is prudent that every ECM manager or knowledge manager, who finds themselves using a ECM platform or KM platform that does not provide an "intelligent search engine" to ask themslevs what value would be achieved by integrating such a tool.
In one of the last gigs I had as a knowledge management/ECM, we brought in just such a search tool, (Vivisimo) - in this case to a very large corporation, for whom "intellignet" search, for a variety of reasons had escaped them. In one demonstration of "effective" search, they were literally blown away. Prior to this, they had undertaken a handful of attempts, some costing in the millions of dollars to develop and train a search engine to provide concept clusters, heuristic taxonomies and conceptual searching. Here it was - "Out out ofthe box". More importantly, seeing "it work", they readily appreciated the power od conpet-based, heuristic clustering as a way to effectivley mine intelligence from contyent - not just locate words and phrases in text.
In my current position, as I previously blogged about, we began with a simple search tool, embedded in an ECM platform. But our plan is very much focused on expanding this to include high end search. Size of a community and corpus of content somewhat effects the need for "intelligent search", but more improtantly, in virtually every situation you will need it becasue, as the article in today's Metro that insppired this post pointed out, langauge is always changing, across communities, industries and subject matter.