What is sentiment analysis and how to do it by yourself

Tom
February 13, 2021 ・ 9 min read

We can definitely tell that with the development of e-commerce, SaaS tools, and digital technologies, sentiment analysis is becoming more and more popular. Welcome to the guide to sentiment analysis.

In this article, you’ll learn all you need about:

An illustration of a woman with thumbs up and down

What is sentiment analysis?

Sentiment analysis, also known as opinion mining or emotion AI, boils down to one thing:

It’s the process of analyzing online pieces of writing to determine the emotional tone they carry, whether they’re positive, negative, or neutral.

In simple words, sentiment analysis helps to find the author’s attitude towards a topic. Sentiment analysis tools categorize pieces of writing as positive, neutral, or negative.

Here’s an example of a negative sentiment piece of writing because it contains hate.

What is sentiment score?

Sentiment score is a scaling system that reflects the emotional depth of emotions in a piece of text.

Sentiment score detects emotions and assigns them sentiment scores, for example, from 0 up to 10 – from the most negative to most positive sentiment. Sentiment score makes it simpler to understand how customers feel.

Why sentiment analysis is important?

First and foremost, it saves time and effort because the process of sentiment extraction is fully automated – it’s the algorithm that analyses the sentiment analysis datasets, and so human participation is sparse.

Can you imagine browsing the Web, finding relevant texts, reading them, and assessing the tone they carry manually?

It’s doable, but completely ineffective.

Secondly, sentiment analysis is important because emotions and attitudes towards a topic can become actionable pieces of information useful in numerous areas of business and research.

Thirdly, it’s becoming a more and more popular topic as artificial intelligence, deep learning, machine learning techniques and natural language processing technologies that are booming these days and this will go on.

Fourthly, as the technology develops, sentiment analysis will be more accessible and affordable for the public and smaller companies as well.

And lastly, the tools are becoming smarter every day. The more they’re fed with data, the smarter and more accurate they become in sentiment extraction.

What is sentiment analysis used for?

Text analytics and opinion mining find numerous applications in e-commerce, marketing, advertising, politics, market research and any other research.

Let’s have a closer look at how text analysis benefits these areas.

Brand reputation management

The internet is where consumers talk about brands, products, services, share their experiences and recommendations. Social platforms, product reviews, blog posts and discussion forums are boiling with opinions and comments which, if collected and analyzed, are a source of business information.

When it comes to brand reputation management, sentiment analysis can be used for brand monitoring to analyze the web and social media buzz about a product, a service, a brand, or a marketing campaign.

Online analysis helps to gauge brand reputation and its perception by consumers.

An example of social mention from Brand24 illustrating sentiment analysis► Try sentiment analysis

This is how businesses can discover consumer, media and expert attitudes towards their products, services, marketing campaigns and brands expressed on discussion forums, online review sites, news sites, blogs, Twitter and other publicly available online sources.

Brand monitoring is an important area of business for PR specialists and sentiment analysis should be one of their tools for everyday use.

Customer feedback

Companies use sentiment analysis to analyze customers’ opinions.

These days, consumers use their social profiles to share both their positive and negative experiences with brands.

A sentiment analysis tool can identify mentions conveying positive pieces of content showing strengths, as well as negative mentions, showing bad reviews and problems users face and write about online.

An example of social mention from Brand24 illustrating sentiment analysis► Try sentiment analysis

In some cases, this makes customer service far more attentive and responsive, as customer support team is informed in real time about any negative comments. The support folks need to know about any blunders as quickly as possible. Because the mentions get detected extremely quickly, customer service has the advantage of rapid reaction time. This makes the customer experience management much more seamless and enjoyable.

Our wonderful content manager, Chia, made a video that sums up how analyzing the sentiment of your customer feedback lets you discover what your customers like and dislike about your company and products.

Here are some use cases:

UBER

Some time ago UBER used social media monitoring and text analytics tools to discover if users like the new version of their app.

It’s a pretty good case study that illustrates the use of sentiment analysis in social media.

“At Uber, we use social listening on a daily basis, which allows us to understand how our users feel about the changes we’re implementing. As soon as we introduce a modification, we know which parts of it are greeted with enthusiasm, and which need more work. We’re happy that the new app was received so well because we’ve put a lot of work into it”, says Krzysiek Radoszewski, Marketing Lead for central and eastern Europe at Uber.

Case study how UBER uses sentiment analysis► Try sentiment analysis

Market research

Sentiment analysis offers a vast set of data, making it an excellent addition to any type of market research.

Whether you’re analyzing entire markets, niches, segments, products, their specific features, or assessing any market buzz, sentiment analysis provides you with tremendous amounts of invaluable information: what consumers like, dislike, or what their expectations are.

All of this data allows you to conduct relatively specific market investigations, making the decision-making process better.

Crisis prevention

Sentiment analysis tools, for example Brand24, are also media monitoring tools. They collect mentions of predefined keywords in real-time from websites, news sites, discussion forums,

Using such a tool, PR specialists can receive real time notifications about any negative piece of content that appeared online. On seeing a negative customer sentiment mention, a company can quickly react and nip the problem in the bud before it escalates into a social media crisis.

United Airlines

Last year, United Airlines experienced an image crisis. Using a social media monitoring tool, we analyzed the sentiment of #UnitedAirlines hashtag. I wonder if they used a sentiment analysis model at that time.

Here’s what popped out:

Sentiment analysis for United Airlines

Politics

Political scientists have also found a great use for sentiment analysis.

In 2012, using sentiment analysis, the Obama administration investigated the reception of policy announcements during the 2012 presidential election.

During the last presidential election in the US, some organizations analyzed, for example, how many negative mentions about particular candidates appeared in the media and news articles.

There have been at least a few academic papers examining sentiment analysis in relation to politics.

  • Prediction of Indian election on the basis of Twitter sentiment analysis
  • Political Data Science: Analyzing Trump, Clinton and Sanders Tweets and Sentiment
  • Analysis of political sentiment in presidential elections in Egypt using Twitter data

It shows another application of sentiment analysis – research. It can used to measure emotional polarisation in any topic.

How is sentiment analysis done?

The science behind the process is based on algorithms of natural language processing and machine learning to categorize pieces of writing as positive, neutral, or negative.

Sentiment analysis might use various types of algorithms.

Automatic

Such an algorithm relies exclusively on machine learning techniques and learns on received data. Machine learning is the most fundamental aspect of artificial intelligence.

Automatic sentiment analysis starts with creating a dataset that contains a set of texts classified either as positive, negative, or neutral.

With this in place, learning begins and continues as a semi-automatic process. This algorithm learns on data until the system achieves some level of independence, sufficient enough to correctly assess the sentiment of new, unknown texts. It’s then utterly important what data the algorithm is fed with.

If the algorithm hadn’t come across a particular example earlier, it won’t perform an accurate analysis.

One of the biggest advantages of this algorithm is the quantity of data it can analyze – way, way more than the rule-based algorithm.

When it comes to disadvantages, the algorithm makes it difficult to explain decisions behind text analyses, meaning, it’s impossible to tell why it classified a particular text as positive or negative.

Rule-based

This algorithm is based on manually created lexicons that define positive and negative strings of words. The algorithm then analyzes the amounts of positive and negative words to see which ones dominate.

Rules can be set around other aspects of text, for example, part of speech, syntax, and more.

This approach is easy to implement and transparent when it comes to rules standing behind analyses.

Hybrid

This one combines both of the above mentioned algorithms and seem to be the most effective solution.

It’s because it combine high accuracy provided by machine learning and stability from the rule-based, lexicon-based approach.

Sentiment analysis challenges

Due to language complexity, sentiment analysis has to face at least a couple of issues. In some cases, it gets difficult to assign a sentiment classification to a phrase. That’s where the natural language processing-based sentiment analysis comes in handy, as the algorithm makes an effort to mimic regular human language.

Contrastive conjunction

One problem a sentiment analysis system has to face is contrastive conjunctions — they happen when one piece of writing (a sentence) consists of two contradictory words (both positive and negative).

  • Example sentence: “The weather was terrible, but the hike was amazing!”

Named-entity recognition

Another big problem algorithms face is named-entity recognition. Words in context have different meanings.

  • Does “Everest” refer to the mountain or to the movie?

Anaphora resolution

Also known as pronoun resolution, describes the problem of references within a sentence: what a pronoun, or a noun refers to.

  • Example sentence: “We went to the theater and went for a dinner. It was awful.”

Sarcasm

Is there any sentiment analysis system detecting sarcasm? Please recommend one!

  • Example sentence: “I’m so happy the plane is delayed.”

The Internet

It just so happens that any language used online takes its own form. The economy of language and the Internet as a medium result in poor spelling, abbreviations, acronyms, lack of capitals and poor grammar. Analyzing such pieces of writing may cause problems for sentiment analysis algorithms.

How to do sentiment analysis?

Now let’s say what’s the best way to do sentiment analysis.

To get started, there are a couple of sentiment analysis tools on the market. What’s interesting, most of media monitoring can perform such an analysis.

One of the most affordable and effective tools that offers high monitoring capabilities is Brand24. It offers trial account is completely free of any cost.

Still, you can extract valuable insights and make sense out of them. It’s worth trying as the trial account is completely free of any cost.

If you decide you want to try it out, here’s how to find your way inside the tool.

  1. Create a free account here. Then, provide keyword(s), or hashtag(s) you want to collect mentions of from social networks, discussion forums, websites, blogs, news sites and elsewhere. It can be, for example your company name.
  2. Next, the tool will take you to the main dashboard. In there, you can see all mentions of predefined keywords that appear in real time.`Among sources, volume charts and filters, there is a Sentiment filter. By default, it’s set to display both negative and positive mentions. You can change it as you wish using the slider.
  3. In the Analysis tab, apart from other data, you can get a numerical and percentile summary of negative and positive mentions.
  4. To stay always on top of your mentions, you can set up email alerts that will inform you about, for example, negative mentions the moment they appear online.

Why would you do it all?

To sum up, from the marketing standpoint, sentiment analysis helps with:

  • Identifying negative mentions about a business, a service, a company, a marketing campaign, and events on social platforms
  • Spotting angry customers on the verge of starting a social media crisis
  • Analyzing how your customers react to product changes
  • Spotting super happy users who, for example, are more likely to become your brand ambassadors

Do you have any experiences with sentiment analysis tools?

Read more