Phil Archer

phil@philarcher.org

Sentiment Analysis: We Need Some New Terms

When I meet new people in a professional or potentially professional situation, like anyone else, I need to explain what I do. Since I do lots of different things, that can take a while. Let me try:

Hmmm… Now, depending who I talk to, the term Sentiment Analysis, let alone Custom Sentiment Analysis, either means nothing at all or it means something that may or may not be close to what I hope it means. Never yet has it actually meant what I want it to mean.

It's like saying "I like music." Well, most people do. But when I say that I like music, you're probably thinking about whatever music you like. You may like the same music as me but if I tell you that I'm writing this while listening to Franz Waxman's Tristan and Isolde Fantasia on BBC Radio 3, you get a much clearer idea of what music I like.

I'd like a way to encapsulate what I mean by "Sentiment Analysis" that recognises what other people may mean by the same term but that makes it clear precisely what I mean.

Philip Lambert of Investor Dynamics put me on to an article in New Scientist this week that defined it thus:

Lexicon of tomorrow: SENTIMENT ANALYSIS
-noun
A method for gauging the real-time emotions and opinions of a demographic or region using text mining. The political analyst performed a sentiment analysis of tweets in Virginia ahead of the election

Well, that's one (accurate) definition although I'd use the phrase "Real Time Sentiment Analysis". But it's not what I mean.

Wikipedia offers this definition:

Sentiment analysis or opinion mining refers to the application of natural language processing, computational linguistics, and text analytics to identify and extract subjective information in source materials.

Absolutely right. But it tells you nothing about what the company I represent in this field does and why it's different from others who offer services with the same name.

What follows is a look at different meanings of Sentiment Analysis, inspired in part by an article by Brandwatch [broken link removed] that Emma Jell put me on to. In brief, I'm suggesting we divide the field into;

An attempt at defining new terms

One approach to sentiment analysis is to analyse individual sentences such as:

Gone With the Wind is one of the most memorable Hollywood films ever. Based on the 1936 novel by Margaret Mitchell, its dismal backdrop is the American Civil War but essentially it's an uplifting romantic epic that generations have loved.

We can break this down in various ways. For example, using the online demo of part-of-speech tagging offered by the University of Illinois we can see that the adjectives and adverbs used are memorable, ever, dismal, essentially and romantic. By looking at the polarity of these terms, we can assess the overall sentiment expressed: positive (memorable, romantic), negative (dismal) and neutral (essentially, ever). A very simplistic approach would be to take the average of these and therefore conclude that the sentiment expressed here is broadly positive. We can assume that companies offering services based on this aproach are more sophisticated than that but even so, this is wide open to problems.

Try this:

Generations have loved Gone With the Wind, but for me it's one of the most dismal Hollywood films ever. Based on the 1936 novel by Margaret Mitchell it's essentially a romantic epic set against the memorable backdrop of the American Civil War.

The adjectives and adverbs are exactly the same so the same over-simplistic analysis would lead to the same result, i.e. that this is broadly positive, even though a human can see immediately that the reverse is true. You can go on playing this game: rearranging the words to see whether you can swing between positive and negative but the key point is that a very small change in the natural language can imply a very different sentiment.

This approach relies on having a look up table that assigns a numerical value to the sentiment-bearing words with bigger values for stronger words (so 'like' might be assigned a value of 1 and 'hate' -4 or whatever).

A more sophisticated version of this looks at phrases rather than individual words but you're still left with a problem. Take this sentence:

They have not succeeded, and will never succeed, in breaking the will of this valiant people.

In a much quoted paper, Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis Wilson et al classify this as positive since the people are valiant and therefore the people's will is a positive thing. Breaking that will would be negative but 'they' have not succeeded in doing so. Therefore, overall, this is a positive statement.

But ask yourself:

Imagine a dictator saying that gunning down a protest march through his capital's main square was justified because they, the protestors, will never succeed in breaking the will of his valiant people.

Or a democratically elected leader saying that the perpetrators of a terrorist attack have not succeeded, and never will succeed, in breaking the will of his valiant people.

What's missing here is context! How many times have you heard someone say I was quoted out of context? Even at its most sophisticated, phrase level sentiment analysis lacks context. Incidentally, this is why you need to be careful of analysis of Twitter as I described recently.

Notably, it is Wilson et al's paper that introduced the notion that human beings can't agree on the polarity of statements more than 82% of the time. This is used by some sentiment analysis companies to explain why their algorithms achieve a classification accuracy of between 70 and 75%. They say "look, our machines are nearly as good as humans, impressive eh?" I'll leave it to you to decide.

Notice that this is still reliant on look up tables of words and so companies using these techniques are only able to work in languages for which they have linguistic resources. If a sentiment analysis company tells you that they work in x number of languages, or just one language, at heart this is what they're doing.

They look for the keywords you give them in sentences they've harvested from a pre-defined list of Web sites and when they get a match, they assess the polarity of the sentences and present it as a bunch of (usually very impressive) graphics. It's easy to do, looks very good and can be offered as a low-price service. Many successful sentiment analysis business are built on this model and it provides exactly what some marketeers are looking for especially if they want instant results.

I suggest we call it Sentiment Polarity Analysis.

Machine Learning

Machine learning is a more robust approach that doesn't depend on pre-defined lists of words. Essentially you give a machine the ability to recognise patterns, the frequency with which certain words appear, word orders, even accuracy of grammar. For example, hate speech (pro-Nazi, white supremacist stuff etc.) includes a lot of BOLD words in CAPITALS and is almost always full of grammatical errors (hardly surprising of people ignorant enough to spout such vile nonsense).

Training a machine means identifying a bunch of examples that have something in common such as being a positive film review, a complaint about insurance or expressing a worry about a health issue.

Language independence is a great advantage here since the machine doesn't need to include any knowledge of the words themselves. All you need are people who speak different languages who can gather the original training corpus.

But there's a limiting factor here — the business model. Companies that offer Sentiment Polarity Analysis give clients a dashboard that they can log in to, type in some key words and instantly see graphics representing a variety of things such as changes in sentiment over different time periods. It looks really good in a presentation and is very cost effective. To compete with this, companies that use machine learning have a bunch of classifiers that work in different sectors and so ultimately it's much the same process. The client enters a keyword, the machine looks for the keyword, and then runs the appropriate classification engine to work out whether a relevant comment is positive or negative.

You can add bells and whistles to this of course and provide richer services for clients but the key aspect here is that the classifiers exist before the client asks a question.

I'm having difficulty thinking of a terms for this. Generic Sentiment Analysis? ML Sentiment Analysis? Domain Sentiment Analysis? None of them sound good. In the end I think this approach probably could be the one that gets the actual title of simply "Sentiment Analysis."

Custom Sentiment Analysis from i-sieve

And so to what the company I represent in Britain does which I'm suggesting we call Custom Sentiment Analysis.

Our system is based on machine learning but it's what we teach the machines that makes us stand out. For each customer we teach machines to look for answers to the specific question they asked. So rather than simple positive and negative polarity around a given set of keywords, we build machines that identify relevant comments and then classify them according to why they feel positive or negative towards something. What are they unhappy about? Why do they prefer the competitor? and so on.

Another difference is that we don't analyse the content we've harvested from a pre-defined list of Web sites. Once we've trained the machine to answer the question, we crawl the Web itself and then give the client the data: the URL of the comment, the comment itself and other information, notably about a given comment's impact, a measure of how widely it was read.

It's much more of a strategic tool than a tactical quick assessment of public perception of a brand.

I guess the elevator reached your floor a long time ago but, well, that's why I have difficulty explaining what I do.