Wednesday, March 26, 2008

Text Analytics - Attensity

I was at a client's yesterday and saw a demo Attensity - a (newish) text analytics tool.

I have seen a few of these over the last couple of years. Mainly in the context of blog/message board/Web analysis - basically pulling text off the Internet and analyzing it for trends. This typically involved asking the software to look for something, which, as the Attensity guys made clear, is so 'text analytics 1.0'.

Attensity is a 'next generation' tool that does a thing called 'exhaustive extraction' (I think that was the term they used). It looks at everything and pulls out all relevant trends - not just the ones you were looking for. That is new. And it means that you're not guessing at what is and isn't important. Attensity understands the context in which certain topics are discussed and can roll up these contexts into trends - that is far from just collecting word or phrase frequencies.

It looks to be great for companies with a lot of data stored as text - emails, product feedback, maintenance records, etc. Text analysis really is one of those hairy problems few people have worked out. These guys look to have a good head-start.

It was also interesting to sit in a software demonstration when I'm involved in building a software application that will ultimately be put to a similar test (selling itself). This is maybe a separate post, but a few things I noticed in the pitch were:

  • A technology story is not always compelling to a non-tech audience - the Attensity guys made a bit of a song and dance about how their product evolved, how it fitted into the whole Business Intelligence space, etc. Not interesting to people who have had nothing to do with the industry - this is the sort of thing that gets Gartner analysts excited, but means nothing to most other people.
  • Ditch PowerPoint - or at least keep it to 20 slides, max. When trying to sell something to a business audience, you just have to answer one question, 'how will this tool make my life easier?' That's it. You don't need a long PP to do that, just a few slides and a product demo. When they get it, they will get it.
  • Get strategic - this is a knowledge tool, its value resides in the decisions you make off the back of the information it provides. Even though as a salesman for the software company you might not know a lot about the business you are pitching to, you have to get to the decision point - somehow. That's where the pain is, and it's the pain you are trying to chase down and relieve.
Despite not doing some of these things though, they sold me on the tool. Whether or not they sold my client we'll have to wait and see.

Digg this
Sphere: Related Content


Falafulu Fisi said...

a (newish) text analytics tool

No, it is not newish. It is probably to you, but there has been researches in this area and accelerated over the last few years. The domain is really called text-mining. There are some links to other projects (some open source) on text-mining from the link shown above. There are many commercial vendors who have developed products in text-mining, such as SPSS, SAS and others. This is a huge domain and it is one of the area that I am interested in, algorithmwise.

I noted that you're interested in business intelligence. I believe to crack it in New York, you have to go above than just traditional BI, since commercial BI tools are so advanced, especially in place like the US which is very competitive. You might try to learn about about the domain of machine learning (ML) for your product development, because any tool that involves ML has a chance of competing in a dense BI market as the US. If you don't want to learn it or perhaps, it is a bit difficult, then hire someone who can. You have a better chance with a product with ML algorithms incorporated than one that using traditional OLAP. ML + OLAP, is cutting-edge. Note that Hyperion is developing their suites of BI tools with both traditional OLAP with ML into their apps.

Here is the best open source project in Machine learning & data-mining locally developed in Java from our own NZ University of Waikato, called WEKA. You might want to download it and play with it.

Paul Soldera said...

Indeed, the text-mining industry has been around for a long while, it was the attensity tool itself that I was talking about - I think they launched early 2000s.

If you're interested in text-mining I would check them out. They do seem to be best-of-breed up here at the moment due to sheer amount of R&D invested in their tool - it has been used by the US government for text analysis for a while.

To be honest, commercial BI tools are kind of a joke - and I have seen many of them. Their complexity largely lies in the data management process - which is the work-horse area of BI. An inordinate amount of time is spent trying to pull everything together rather than worry about what is finally delivered. Which hasn't really changed much at a working level for a while - analysts who use programs to spit out report after report.

ML is great for a different set of data to the one we are interested in. But I agree, it's a nice technology, and I've seen some good applications of it.

Sid Banerjee said...

I read your comments regarding the positioning of text mining/text analytics vendors like Attensity with intereset. I am at a company, Clarabridge (, that also plays in the Text Analytics space, and in fact has competed against and been selected against Attensity at a number of Fortune 1000 customers over the past few years.

I agree with you that the presentation and solution space needs to be defined crisply, and concisely, and that the solution needs to be offered to support a very business specific problem.

Text Analytics has been around for at least 10 years, but until recently it was truly a solution in search of a problem. Early vendors were focused on abstract goals like "semantically mapping documents to concepts, tags, and person/place/thing mappings," and in fact early interest grew out of the government, specifically the intelligence world - because a technology that can extract people/places/things, and map words to codewords, and vice versa, can be useful for finding "bad guys" and relating bad guys to bad things, to other bad people, etc...

The government sector interest in text mining has been especially high since 9/11, for obvious reasons.

Commercial applications of text mining are aimed at a very different business problem, however -- not finding bad guys, so much as quantifying good and bad experiences, and linking experiences to real world business challenges like customer profitability, customer retention, campaign effectivness, new product launch success, and bottom line returns on investments in sales, services, marketing, and retention programs.

Another way to think about the problem -- government text mining is about finding the "needle in the haystack" - ie the bad guy, planning the bad thing. Commercial text analytics is about transforming customer feedback, about customer experiences with products and services, into "customer experience intelligence."

What is Customer Experience Management, and why is it important? Consider two companies in a competing space (perhaps mobile telecom vendors, or hardware retailers, or restaurant chains with similar product lines). It’s likely that those two companies offer the same (or very similar) products, sourced from the same (or very similar) suppliers, at the same (or very similar) prices, and market with the same (or very similar) marketing campaigns. The brand is a commodity, and loyalty is more fleeting now than ever.

Why, then, do consumers select one over the other? Increasingly, it’s not because the product is different, or better, or priced more attractively, it’s because the experience at one vendor, as perceived by the consumer, is qualitatively preferred over the other.

Understanding the experience, and the preference, or passion, for the experience is not something obtainable from a sales transaction, or inventory forecast, or financial report. It requires qualitative insight from the consumer.

A concise vendor presentation, presuming the prospect is someone who cares about "customer experience intelligence" should cover the following:

- what is the problem - getting into the heads of your customers, knowing what they like, what they experience that makes them happy, upset, loyal, prone to churn, etc.

- what is the challenge - traditional qualitative market research is (take your pick) expensive, time consuming, subjective, difficult to "code" consistently, and not responsive to fast changing, often fleeting customer challenges and opportunities.

- what is the solution: Text Analytics, specifically solutions optimized to create a quick, scalable, enterprise class analysis framework that "transforms feedback into customer experience intelligence."

- What are the business applications of the solution:
- to quantify the impact of a new product marketing campaign on customer perceptions
- to identify problems with new products
- to continuously monitor service levels across a wide range of customer interactions (email, phone calls, surveys
- to perform passive market research across a wide range of web based listening posts - forums, web 2.0 sites, blogs, review sites, etc.

Like every vendor, Attensity has its technical calling cards (exhaustive extraction). It's safe to say that a number of vendors have specific technical capabilities that make them interesting, unique, and useful. Clarabridge, for instance, has:
- the first (and most actively deployed) Software as a Service based Customer Experience Intelligence solution, deployed to over half of its customers.

- active and successful Business Intelligence enabled solutions, making use of all three leading BI vendors Cognos, MicroStrategy, and Business Objects (making it easy for customers to view insights using frameworks and interfaces they're used to).

- a rich "navigation" interface that quickly exposes themes, experiences, and issues that require active review by the business.

- packaged applications tailored for a wide range of of industries, that quickly enable customers to achieve business value with the solution.

In short - the space is interesting, growing agressively, and adding a lot of value to corporations who increasintly need to quantify the qualitative -- to make sure they're creating loyalty enabling experiences for their customers.

Paul Soldera said...

Hi Sid, thanks for the comment. I am going to make a post in response as I was also contacted by Attensity after I made the original post. Lots of interest in this area from clients of mine so I think it deserves a bit more attention.


Fern Halper said...

Hi Paul:

Welcome to the world of text analytics! I've been following the space for the past few years and recently starting blogging on it, myself. Check out There's info there on Clarabridge, Attensity, and SAS to name a few.

Fern Halper
Hurwitz & Associates

Paul Soldera said...

Hi Fern, thanks for stopping by. Already added you to my feed-reader! Looking forward to following this space as well.



Anonymous said...

wow! thanks for another great blog post!
Customized application development

price per head said...

Its so interesting article..Thanks for sharing..

price per head said...

Its so interesting article..Thanks for sharing..