What Is Sentiment Analysis in AI? Methods, Code and Limitations

Sentiment analysis is the use of AI to work out whether a piece of text expresses a positive, negative or neutral opinion, and sometimes how strongly or about what. Businesses use it to read product reviews, support tickets and social media at a scale no team could.

Oct 11, 2026 - 07:29
Updated: 
103.7k

Machine Learning in AI & Data · Explainer

What Is Sentiment Analysis in Artificial Intelligence?

Quick answer: Sentiment analysis is the use of AI to work out whether a piece of text expresses a positive, negative or neutral opinion, and sometimes how strongly or about what. Businesses use it to read product reviews, support tickets and social media at a scale no team could. It works well on clear opinions in the domain it was built for, and struggles with sarcasm, mixed opinions, slang and code-mixed language such as Hinglish. Treat its output as a trend indicator across many texts, not a verdict on any single one.

Key takeaways

  • Three main approaches: word lists (lexicons), trained classifiers, and transformer models.
  • Transformers handle context and negation far better than word lists.
  • Accuracy depends on domain. A model trained on movie reviews may misread banking complaints.
  • Sarcasm and mixed languages are still hard for every approach.
  • Always measure on a labelled sample of your own data before trusting the numbers.

Table of Contents

What Is Sentiment Analysis?

Also called opinion mining, sentiment analysis is a natural language processing task. The input is text; the output is a label such as positive, negative or neutral, often with a confidence score. More detailed systems estimate intensity, detect emotions such as anger or joy, or attach sentiment to specific topics within the text.

What Are the Levels of Sentiment Analysis?

  • Document level: one label for a whole review or ticket.
  • Sentence level: a label for each sentence, which helps when opinions are mixed.
  • Aspect level: sentiment about specific features. In "the camera is excellent but the battery dies by noon", the camera is positive and the battery negative. This is usually what product teams actually need.

How Does Sentiment Analysis Work?

ApproachHow it worksStrengthsWeaknesses
Lexicon-basedScores words from a sentiment dictionary, with rules for negation and emphasisFast, no training data, explainableMisses context, domain-specific meaning
Classical machine learningLearns from labelled examples using word counts as featuresAdapts to your domain, cheap to runNeeds labelled data, weak on context
Transformer modelsPretrained language models fine-tuned for sentimentHandles context and negation wellHeavier to run, can still miss sarcasm
Large language model promptingAsks a general model to classify the textNo training, handles nuance and explanationsCost per call, variable output, privacy

What Does a Lexicon Approach Look Like in Code?

VADER is a lexicon and rule-based analyser tuned for social media text. It returns a compound score from -1 to 1; its documentation treats 0.05 and above as positive and -0.05 and below as negative.

pip install vaderSentiment

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
analyzer = SentimentIntensityAnalyzer()

for text in ['The delivery was fast and the product is great!',
             'Worst support experience ever.',
             'Oh great, another update that breaks everything.']:
    print(round(analyzer.polarity_scores(text)['compound'], 2), text)

The first two come out clearly positive and negative. The third is sarcasm, and a word-based scorer sees the word great and scores it positive. That single example is the clearest lesson in the field's limits.

What Does a Transformer Approach Look Like?

A model fine-tuned for sentiment reads words in context. With Hugging Face Transformers, a pretrained English model downloads on first use:

pip install transformers torch

from transformers import pipeline
clf = pipeline('sentiment-analysis',
               model='distilbert/distilbert-base-uncased-finetuned-sst-2-english')
print(clf(['The battery is not bad at all.',
           'I would not recommend this to anyone.']))

Negation such as not bad at all is handled far better than by word lists. This model was trained on English movie reviews and has only positive and negative labels, so check it on your own domain before relying on it.

How Do You Train and Evaluate Your Own Classifier?

When your texts differ from public data, such as banking complaints or Indian e-commerce reviews, training on your own labelled examples usually beats an off-the-shelf model. The important part is honest evaluation on data the model never saw, using scikit-learn:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

# texts and labels come from your own labelled sample, ideally hundreds or more
X_train, X_test, y_train, y_test = train_test_split(texts, labels, test_size=0.2,
                                                    stratify=labels, random_state=42)
vec = TfidfVectorizer(ngram_range=(1, 2), min_df=2)
model = LogisticRegression(max_iter=1000).fit(vec.fit_transform(X_train), y_train)
print(classification_report(y_test, model.predict(vec.transform(X_test))))

Read the per-class precision and recall, not just overall accuracy. If 90 percent of your tickets are neutral, a model that always says neutral scores 90 percent accuracy and is useless. Including two-word phrases, as ngram_range=(1, 2) does, helps with simple negation such as not good.

Why Does Sentiment Analysis Get Things Wrong?

  • Sarcasm and irony: the words say one thing and mean the opposite.
  • Domain shift: unpredictable is good for a film plot and bad for a car's brakes.
  • Mixed opinions: one review praising and criticising different features.
  • Code-mixed language: Hinglish and other mixes such as bahut accha but delivery slow are common in India and poorly covered by English-only models.
  • Emoji, slang and spelling that change faster than models are retrained.
  • Comparisons: better than the last one says little about how good either is.

Where Is Sentiment Analysis Used?

  • Product reviews: tracking what customers like and dislike by feature.
  • Customer support: routing angry tickets for faster handling.
  • Brand monitoring: watching social media for sudden shifts.
  • Employee surveys: spotting themes in free-text answers.
  • Security and fraud teams sometimes use it alongside other signals to prioritise complaints mentioning scams.

How Should Results Be Interpreted and Reported?

  • Report trends over many texts, with counts, not single scores.
  • State the method, the model and its measured accuracy on your data.
  • Show examples of both correct and incorrect classifications.
  • Have people read a sample regularly, because language and products change.

For the underlying ideas, see machine learning basics and the difference between AI and machine learning.

Talk to a WebAsha training advisor about AI and data science courses, batches and current fees.

References

Frequently Asked Questions

The use of AI to decide whether text expresses a positive, negative or neutral opinion, sometimes with intensity, emotion or the specific topic the opinion is about.

Lexicon-based scoring with word lists, classical machine learning trained on labelled examples, fine-tuned transformer models, and prompting large language models.

A lexicon and rule-based sentiment analyser tuned for social media. It returns a compound score from -1 to 1, with 0.05 and above treated as positive and -0.05 and below as negative.

Because the words say the opposite of what is meant. Word-based methods score the literal words, and even context-aware models often miss irony without more context.

Finding sentiment about specific features within a text, such as a positive view of a camera and a negative view of the battery in the same review.

It depends on the domain and data. Measure it on a labelled sample of your own texts, using per-class precision and recall rather than overall accuracy alone.

English-only models handle code-mixed text such as Hinglish poorly. Better results need models trained on code-mixed data or your own labelled examples.

Not always. A lexicon tool such as VADER needs no training and works for quick checks, but machine learning usually performs better on your own domain.

VADER and NLTK for lexicon methods, scikit-learn for classical machine learning, and Hugging Face Transformers for pretrained deep learning models.

As trends across many texts with counts, stating the method and its measured accuracy on your data, and with examples of both correct and incorrect classifications.

What's Your Reaction?

Like Like 0
Dislike Dislike 0
Love Love 0
Funny Funny 0
Wow Wow 0
Sad Sad 0
Angry Angry 0
Priyanka

I am dedicated to staying up-to-date on the most current technology trends and like to craft content that informs and inspires. Whether through detailed analysis, informative guides, or exact opinion articles, I desire to create engaging content for both technology lovers and industry experts.