Mining the Talk: Unlocking the Business Value in Unstructured Information by Jeffrey Kreulen,Scott Spangler

There's currently a lot of noise about "supercrunching" -- continually analyzing customer data and "thinking with numbers." Business has been mining its databases for decades with increasing sophistication. In many ways, however, analyzing the data you already have simply tells you more about what you already know, largely because the decision about what types of data to collect has already limited the opportunities for exploration. The processes and tools described by these authors open the door to a completely new and largely untapped source of information about your customers, your products, your competition, and your reputation. Furthermore, its a source of information that hasn't previously been filtered by the need to limit data storage or structure data into relational databases. What's surprising, or perhaps exciting, is that the authors have managed to explain the value of mining unstructured data and the processes for doing so in language that is not only easy to understand but also easy to read. If your job is to understand the behavior of "the crowd" and predict how they will respond to your advertising and promotion, then this book is a must. If your job is trying to find a needle in the proverbial haystack, then this book is essential. If your interest is in classifying the masses of information being spewed forth on the Internet, then this book is a place to start. And, if you are fascinated with advances in our ability to put metrics on an increasingly fractured world, then this book will tweak your imagination.

This is a good overview of text mining. It is very readable and easy to understand.

So how many of you IT professionals have faced this particular question from someone in your IT organization? "We have so much data locked up in (email/forums/wikis/etc.), and we can't analyze it".

Thought so...

I had the opportunity to read and review the book Mining the Talk: Unlocking the Business Value in Unstructured Information, and I think it's one of those books that can change the way your organization views this type of data.

Table of Contents:
1. Introduction
2. Mining Customer Interactions
3. Mining the Voice of the Customer
4. Mining to Improve Innovation
5. Mining to See the Future
6. Future Applications
7. The IBM Unstructured Information Modeler Users Manual
Index

Spangler and Kreulen are IBMers who have spent a lot of time working with the concepts behind mining data that resides in unstructured formats. This could be web pages, help desk tickets, discussion database, or what have you. Once you step outside the structured world of relational databases and official taxonomies, it's often thought that the only way to get a feel for what's happening is to have someone responsible for reading it all. But there *is* a better way...

Using a tool named The IBM Unstructured Information Modeler, they show how you can use software to analyze unformatted data, cluster the results and keywords, and develop a real taxonomy that covers what is truly happening in your business. For instance, your help desk system probably has categorization fields. But you'll never get all the combinations that could happen, nor will every last help desk person categorize things in the same way. Using this tool, you can pass in the unstructured data (comment fields) and start to get a feel for what really is happening. Then by refining and grouping the initial results, the taxonomy for the target data can be accurately grouped for even larger datasets. The net result is a true view into what issues are occurring most often, in what combinations, as well as where most of the issues come from. These combinations might remain hidden if you rely on typical structured reporting based on static categorization. But with a little effort, you can start to gain insights not readily available by other means.

Even better... There's a version of the software tool that is downloadable from the AlphaWorks site, so you can start experimenting with your own data mining efforts. The install is straight-forward and easy, so it's not as if you'll spend two days trying to get a proof-of-concept going. You could easily pull off an initial test in a couple of hours, and then use that to decide if you need to expand your efforts to a larger dataset and the licensed version of the software.

Think of what this means for all your discussion and document databases out there. No longer would you need to "apologize" for not having the capabilities to analyze and mine the information stored out there. A 15 minute rollout of a discussion database can start paying immediate communication dividends, and then the mining tool could be used to spot trends and hot points. That's some powerful abilities coupled with immediate ROI to the business. Can't beat that combination...

It's well worth the time to pick up a copy of this book and try out the mining software. You might even want to run it against your mail file to see where and who is taking most of *your* time and attention.

Mining the Talk: Unlocking the Business Value in Unstructured Information (IBM Press)
This really does get down the whole philosophy and all that you need to know about the practise of text mining with state of the art tools. In the age of consumer generated media and the internet the world is generating more than we can ever handle and understand without help. Mining the Talk explains how a combination of people and computer technology can help us deal with the information overload. Computers are fast and stupid and we are smart but slow. Thus is about getting fast and smart.

Mining the Talk introduces the reader to the different ways of thinking needed to work with unstructured text, gives them just the right amount of theory, goes onto the case studies and finally end by putting the tools in the hands of the readers. (Software that performs the text mining operations described can be downloaded from a site mentioned in the book).

The book is recommended reading for anyone who is involved in the analysis of documents, interet, news etc.

NB. The reviewer has met the authors of the book.

The authors have created a highly readable, yet technically comprehensive, work about a topic of critical importance in the current marketplace. As an IT professional frequently faced with convincing non-IT colleagues of the importance of technology in industry, this book is just the kind of tool needed. It presents a solution that is ingenious and innovative, and has direct practical application to current business problems. I can think of many immediate uses in my industry and the straightforward presentation, with clear examples, practically sells the concept of mining unstructured data by itself. It makes the job of communicating important, complex concepts to decision-makers who often have no IT background almost easy! Especially for large to medium-sized corporations struggling with basic data management, and missing huge opportunities for discovering the value hidden in their data, this book will be enormously important.

Linda J. Joseph, PhD
ME Technology
General Motors Corporation