Imagine you’re running a business and you want to know what your customers think about your product. Or maybe you’re a movie director wanting to gauge the public reaction to your latest release.

This is where sentiment analysis comes into play.

Sentiment analysis is a technique used in text analysis that helps in identifying and categorizing opinions expressed in a piece of text.

Sentiment analysis determines whether the expressed opinion in a document, a sentence or an entity feature/aspect is positive, negative, or neutral.

In a world where data is king, sentiment analysis is a crown jewel. It’s like having a superpower to understand the emotional tone behind words at scale.

Companies use it to understand customer feedback on products and services. Governments and organizations use it to get a sense of public opinion.

In social media management, sentiment analysis is used for brand monitoring, customer service, and market research.

It’s not just about understanding how many people are talking about your brand or product, but how they feel about it.

What is Hugging Face?

Now, let’s talk about Hugging Face. No, it’s not what you think. You don’t go around hugging faces.

In the world of AI, Hugging Face is quite the star. It’s an AI community and platform that provides state-of-the-art tools and models for Natural Language Processing (NLP).

Think of it as a toolbox that gives you the power to understand and generate human language. It’s like having a linguistic wizard by your side.

Hugging Face’s most popular offering is the ‘Transformers’ library. The Transformers library comes packed with APIs and tools that let you easily grab and train top-notch pre-trained models.

When you pick these pre-trained models, you’re cutting down on compute costs and carbon footprint. Plus, you save loads of time and resources that you’d otherwise spend training a model from scratch.

These models solve common tasks across various domains, like:

• Natural Language Processing (NLP): Here, you can do a bunch of cool stuff like text classification, spotting names or entities in text, answering questions, language modelling, summarizing, translating, handling multiple-choice questions, and even generating text.
• Computer Vision: This involves image classification, spotting and outlining objects in images, and more.
• Audio: You can work on recognizing speech automatically and classifying different types of sounds.
• Multimodal Tasks: These are tasks that mix it up, like answering questions based on tables, recognizing text in images (like scanned documents), pulling out information from these documents, classifying videos, and answering questions based on images.

The neat thing about Transformers is that they’re flexible with different frameworks. Whether you’re into PyTorch, TensorFlow, or JAX, Transformers has got you covered.

Its ease of use and comprehensive nature make it a go-to for researchers, developers, and businesses alike.

Code for Sentiment Analysis

Now that you know what sentiment analysis and Hugging Face are, let’s write some code. We’ll use Python and the Hugging Face transformers library to build a simple sentiment analyzer.

You can either use your terminal, install Python and run the code, or use a Google Colab notebook. I would recommend the latter since it comes pre-installed with Python.

Install the transformerslibrary with this command:

pip install transformers

If you are using a Colab notebook, use a ! symbol before the command for the notebook to treat it as a shell command (Colab executes code as Python by default).

!pip install transfomers

Once the installation is complete, you can start using the library. First, let's import pipeline from the transformers library.

from transformers import pipeline

In Hugging Face, a “pipeline” is like a tool that helps you perform a series of steps to change data into the form you want. The pipeline makes it simple to use these tools for different jobs, without needing to know all the complex details about how these tools work on the inside.

Now let’s load the sentiment-analysis pipeline.

sentiment_pipeline = pipeline("sentiment-analysis")

Now would you believe me if I said we are pretty much done? Our sentiment analysis model is ready and we can pass text to the pipeline and get the label as well as a sentiment score.

# Run sentiment analysis
result = sentiment_pipeline("Every new day brings a chance to create joyful memories and embrace new opportunities.")

# Print the result
print(result)

This is the output of the above code:

[{'label': 'POSITIVE', 'score': 0.9998821020126343}]

If you want to pass multiple sentences, pass an array of inputs to the pipeline.

result = sentiment_pipeline(["Every new day brings a chance to create joyful memories and embrace new opportunities.","Despite the effort, the project failed to meet expectations, leading to disappointment and frustration among the team."])
print(result)

Following will be the output of the above code:

[{'label': 'POSITIVE', 'score': 0.9998821020126343}, {'label': 'NEGATIVE', 'score': 0.9997937083244324}]

I hope you understand how powerful the Hugging Face Transformer library is. This is just a sample of the many pre-trained models that Hugging Face provides. Unless you are working on a unique problem, you should find a pre-trained model in Hugging Face available for you to work with.

Summary

In this article, we’ve learned about sentiment analysis and Hugging Face, a powerful tool in the world of NLP. Most importantly, you’ve taken your first steps in performing sentiment analysis by using the Hugging Face Transformers library.

Remember, what we’ve covered is just the tip of the iceberg. The field of NLP is vast and constantly evolving. The Hugging Face Transformers library is a powerful ally in your journey through AI. It simplifies complex tasks and gives you access to pre-trained models, saving you time and resources.

Hope you enjoyed this article. Find more beginner-friendly articles on AI at turingtalks.ai

Turning machine learning models into actual applications other people can use is not something that is covered in most AI and Machine Learning Tutorials.

In this article, we are going to create an end-to-end AI Sentiment Analysis web application using Gradio and Hugging face transformers.

What is Sentiment Analysis?

According to Wikipedia,

Sentiment analysis is the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information.

In simple words, Sentiment Analysis is the ability of Artificial Intelligence to analyze a sentence or block of text and get the emotions behind that sentence or block of text.

What is Gradio?

Gradio is an open-source Python library that you can use to quickly create and customize easy-to-use UI components for your ML model, any API, or any arbitrary function in just a few lines of code.

Gradio makes it very easy for you to build Graphical User Interfaces and deploy machine learning models.

What is Hugging Face?

Hugging Face is a library that provides pre-trained and open-sourced Natural Language Processing models and datasets for machine learning engineers.

It is an open-source Machine Learning community where you can download pre-trained machine learning models and use them in your own projects.

Time to Build our Project

Prerequisites

• Have Python installed
• Have an IDE / text editor (like Visual Studio, PyCharm, or Jupyter Notebook )
• Have an internet connection

Here is the GitHub Repository for the project.

Install Dependencies

Here we are going to install the libraries needed to build the Sentiment Analysis app.

How to install Transformers

Here we are going to install the transformers library. This library will give us access to the hugging face API.

#In a jupyter notebook
!pip install transformers

#In terminal
pip install transformers

How to install PyTorch

We are going to install the PyTorch deep learning library. Visit the PyTorch Website and install your specialized version.

Below is my installed version of PyTorch.

#install in jupyter notebook
!pip3 install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio===0.9.1 -f https://download.pytorch.org/whl/torch_stable.html

#Install in Terminal
pip3 install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio===0.9.1 -f https://download.pytorch.org/whl/torch_stable.html

Import and Set up Pipeline

Here we are going to import and set up our sentiment analysis model using a Hugging Face pipeline.

Hugging Face provides an automatic pipeline that helps handle things like tokenizing, pre-processing, encoding, and decoding for you and lets you focus on core things like model optimization.

#setting up hugging face pipeline
from transformers import pipeline
classifier = pipeline("sentiment-analysis")

Above we imported and instantiated the pipeline object, and then we passed the sentiment-analysis models an argument.

How to Define the Gradio Function

We are going to define a Gadio function that will help us provide the sentiment analysis functionality for our web app.

If you read my past article on building graphical user interfaces (GUI) for machine learning models using Gradio, you'll know that Gradio allows you to build graphical components for your models and they provide the model's prediction functionality through functions.

#model function for gradio

def func(utterance):
  return classifier(utterance)

Above we created a function called func and added utterance (that is, the word to be analyzed by the model for sentiments) as an argument for our function. We then make our function return the sentiment analysis of the utterance earlier passed and this takes us to the next step.

How to Build our Gradio Interface

Here we are going to create our Gradio web app, add graphical components to it, then we are going to launch the app.

#getting gradio library
import gradio as gr
descriptions = "This is an AI sentiment analyzer which checks and gets the emotions in a particular utterance. Just put in a sentence and you'll get the probable emotions behind that sentence"

app = gr.Interface(fn=func, inputs="text", outputs="text", title="Sentiment Analayser", description=descriptions)
app.launch()

Above we imported the Gradio library, and then we added a description of our project which will be then passed on to our web app.

We then created a Gradio interface instance where we are going to provide details about our web app. We passed the model's function into the fn parameter, we then provided the type of input.

Gradio allows you to create any form of input of your choice be it text, radios, checkboxes, numbers, and so on. But here we are going to use our input as text.

Next, we provided the output format, the same way Gradio allows you to pick your input format (that is text, numbers, checkbox, and so on). It also allows you to pick your output format.

In this case, we are going to use text, too. After passing the output parameter we gave a title to our web app.

Lastly, we launch the app.

Now you can use your new sentiment analysis tool!

Thank you for reading.

Important resources

• Gradio Official Website
• Gradio Documentation
• Gradio GitHub Repo

What is Sentiment Analysis?

Sentiment analysis is a technique through which you can analyze a piece of text to determine the sentiment behind it. It combines machine learning and natural language processing (NLP) to achieve this.

Using basic Sentiment analysis, a program can understand whether the sentiment behind a piece of text is positive, negative, or neutral.

It is a powerful technique in Artificial intelligence that has important business applications.

For example, you can use sentiment analysis to analyze customer feedback. After collecting that feedback through various mediums like Twitter and Facebook, you can run sentiment analysis algorithms on those text snippets to understand your customers' attitude towards your product.

How Sentiment Analysis Works

The simplest implementation of sentiment analysis is using a scored word list.

For example, AFINN is a list of words scored with numbers between minus five and plus five. You can split a piece of text into individual words and compare them with the word list to come up with the final sentiment score.

Let's say we had the phrase, "I love cats, but I am allergic to them".

In the AFINN word list, you can find two words, “love” and “allergic” with their respective scores of +3 and -2. You can ignore the rest of the words (again, this is very basic sentiment analysis).

By combining these two, you get a total score of +1. So you can classify this sentence as mildly positive.

There are complex implementations of sentiment analysis used in the industry today. Those algorithms can provide you with accurate scores for long pieces of text. Besides that, we have reinforcement learning models that keep getting better over time.

For complex models, you can use a combination of NLP and machine learning algorithms. There are three major types of algorithms used in sentiment analysis. Let's take a look at them.

Automated Systems

Automatic approaches to sentiment analysis rely on machine learning models like clustering.

Long pieces of text are fed into the classifier, and it returns the results as negative, neutral, or positive. Automatic systems are composed of two basic processes, which we'll look at now.

Rule-based Systems

Unlike automated models, rule-based approaches are dependent on custom rules to classify data. Popular techniques include tokenization, parsing, stemming, and a few others. You can consider the example we looked at earlier to be a rule-based approach.

A good thing about rule-based systems is the ability to customize them. These algorithms can be tailor-made based on context by developing smarter rules.

Just keep in mind that you will have to regularly maintain these types of rule-based models to ensure consistent and improved results.

Hybrid Systems

Hybrid techniques are the most modern, efficient, and widely-used approach for sentiment analysis. Well-designed hybrid systems can provide the benefits of both automatic and rule-based systems.

Hybrid models enjoy the power of machine learning along with the flexibility of customization. An example of a hybrid model would be a self-updating wordlist based on Word2Vec. You can track these wordlists and update them based on your business needs.

Use Cases for Sentiment Analysis

Analyzing Customer Feedback

Customer feedback analysis is the most widespread application of sentiment analysis. Direct customer feedback is gold for businesses, especially startups. Accurate audience targeting is essential for the success of any type of business.

Well-made sentiment analysis algorithms can capture the core market sentiment towards a product.

You can also extend this use case for smaller sub-sections, like analyzing product reviews on your Amazon store. The more customer-driven a company is, the better sentiment analysis can be of service.

Campaign Monitoring

Manipulating voter emotions is a reality now, thanks to the Cambridge Analytica Scandal.

Another use-case of sentiment analysis is a measure of influence. Taking the 2016 US Elections as an example, many polls concluded that Donald Trump was going to lose.

But experts had noted that people were generally disappointed with the current system. They backed their claims with strong evidence through sentiment analysis.

I worked on a tool called Sentiments (Duh!) that monitored the US elections during my time as a Software Engineer at my former company. We noticed trends that pointed out that Mr. Trump was gaining strong traction with voters.

This should be evidence that the right data combined with AI can produce accurate results, even when it goes against popular opinion.

Brand Monitoring

Brand monitoring is another great use-case for sentiment analysis. Companies can use sentiment analysis to check the social media sentiments around their brand from their audience.

KFC is a perfect example of a business that uses sentiment analysis to track, build, and enhance its brand. KFC’s social media campaigns are a great contributing factor to its success. They tailor their marketing campaigns to appeal to the young crowd and to be “present” in social media.

Tools like Brandwatch can tell you if something negative about your brand is going viral. Other brands that use social media to promote a positive brand sentiment include Amazon, Netflix, and Dominoes.

Stock Market Analysis

If you are a trader or an investor, you understand the impact news can have on the stock market. Whenever a major story breaks, it is bound to have a strong positive or negative impact on the stock market.

Sentiment analysis is a powerful tool for traders. You can analyze the market sentiment towards a stock in real-time, usually in a matter of minutes. This can help you plan your long or short positions for a particular stock.

Recently, Moderna announced the completion of phase I of its COVID-19 vaccine clinical trials. This news resulted in a strong rise in the stock price of Moderna.

But today, Moderna’s stock stumbled after losing a patent. Using sentiment analysis, you can analyze these types of news in realtime and use them to influence your trading decisions.

Compliance Monitoring

Regulatory and legal compliance can make or break large organizations. Often, these compliance documents are stashed into large websites like Financial Conduct Authority.

Large organizations spend a good chunk of their budgets on regulatory compliance. In these cases, traditional data analytics cannot offer a complete solution.

Tools like ScrapingHub can help fetch documents from these websites. But companies need intelligent classification to find the right content among millions of web pages.

Sentiment analysis can make compliance monitoring easier and more cost-efficient. It can help build tagging engines, analyze changes over time, and provide a 24/7 watchdog for your organization.

Conclusion

Sentiment analysis is a powerful tool that you can use to solve problems from brand influence to market monitoring. New tools are built around sentiment analysis to help businesses become more efficient.

And by the way, if you love Grammarly, you can go ahead and thank sentiment analysis.

Loved this article? Join my Newsletter and get a summary of my articles and videos every Monday.

This weekend I had some time on my hands and decided to build a Twitter sentiment analysis tool.

The idea is that you enter a search term and the tool will search recent tweets. It will then use sentiment analysis to determine how positive or negative Twitter is about the subject.

For example, you could search "Donald Trump" to get Twitter's sentiment on the president.

Let's dive in!

Getting a Twitter API key

The very first thing we need to do is create a Twitter application in order to get an API key.

Head over to the Twitter apps page to create a new application. You must have a developer account to be able to create an application.

If you don't have a developer account you can apply for one. Most requests are granted instantly. ?

Copy down the API Key and API Key Secret that you find in your Twitter application.

Creating a NodeJS project

I'm going to use NodeJS to create this application.

To create a new project I run:

npm init
npm install twitter-lite

This will create a new NodeJS project and install the twitter-lite package. This package makes interacting with the Twitter API super easy.

To authenticate our requests we are going to use an OAuth2.0 bearer token. The twitter-lite package has an easy way of handling the Twitter authentication.

Let's create a new index.js file and add the following code to it:

const Twitter = require('twitter-lite');

const user = new Twitter({
    consumer_key: "YOUR_API_KEY",
    consumer_secret: "YOUR_API_SECRET",
});

// Wrap the following code in an async function that is called
// immediately so that we can use "await" statements.
(async function() {
    try {
        // Retrieve the bearer token from twitter.
        const response = await user.getBearerToken();
        console.log(`Got the following Bearer token from Twitter: ${response.access_token}`);

        // Construct our API client with the bearer token.
        const app = new Twitter({
            bearer_token: response.access_token,
        });
    } catch(e) {
        console.log("There was an error calling the Twitter API.");
        console.dir(e);
    }
})();

When running this the console outputs the following:

Got the following Bearer token from Twitter: THE_TWITTER_BEARER_TOKEN

Awesome, so far everything works. ?

Getting recent tweets

The next part is retrieving recent tweets from the Twitter API.

On the Twitter documentation you can see that there is an endpoint to search for recent tweets.

To implement this I add the following code to the index.js file:

const Twitter = require('twitter-lite');

(async function() {
    const user = new Twitter({
        consumer_key: "YOUR_API_KEY",
        consumer_secret: "YOUR_API_SECRET",
    });

    try {
        let response = await user.getBearerToken();
        const app = new Twitter({
            bearer_token: response.access_token,
        });

        // Search for recent tweets from the twitter API
        response = await app.get(`/search/tweets`, {
            q: "Lionel Messi", // The search term
            lang: "en",        // Let's only get English tweets
            count: 100,        // Limit the results to 100 tweets
        });

        // Loop over all the tweets and print the text
        for (tweet of response.statuses) {
            console.dir(tweet.text);
        }
    } catch(e) {
        console.log("There was an error calling the Twitter API");
        console.dir(e);
    }
})();

When running this you can see a lot of twitter comments about Lionel Messi, meaning that it works perfectly! ⚽

"RT @TheFutbolPage: Some of Lionel Messi's best dribbles."

"RT @MagufuliMugabe: Lionel Messi ? didn't just wake up one day  and become the best player in the world no  HE trained. So if your girl is…"

""RT @goal: The boy who would be King ? Is Ansu Fati the heir to Lionel Messi's throne?"

and many more...

Performing sentiment analysis

To perform the sentiment analysis I'm going to use Google Cloud's Natural Language API. With this API you can get the sentiment score of a text with a simple API call.

First, head over to the Google Cloud Console to create a new cloud project.

Next, head over to the Natural Language API and enable it for the project.

Finally, we need to create a service account to authenticate ourselves. Head over to the create a service account page to create a service account.

When creating a service account you will need to download the json file containing the private key of that service account. Store this file in the project folder.

Google has a NodeJS package to interact with the Natural Language API so let's use that. To install it run:

npm install @google-cloud/language

In order for the language package to work, it needs to know where the private key file is.

The package will attempt to read a GOOGLE_APPLICATION_CREDENTIALS environment variable that should point to this file.

To set this environment variable I update the script key in the package.json file.

"scripts": {
  "start": "GOOGLE_APPLICATION_CREDENTIALS='./gcloud-private-key.json' node index.js"
}

Note that in order for this to work you must start the script by running npm run start.

With all that set up we can finally start coding.

I add a new getSentiment function to the index.js file:

const language = require('@google-cloud/language');
const languageClient = new language.LanguageServiceClient();

async function getSentiment(text) {
    const document = {
        content: text,
        type: 'PLAIN_TEXT',
    };

    // Detects the sentiment of the text
    const [result] = await languageClient.analyzeSentiment({document: document});
    const sentiment = result.documentSentiment;

    return sentiment.score;
}

This function calls the Google Natural Language API and returns a sentiment score between -1 and 1.

Let's test it out with a few examples:

getSentiment("I HATE MESSI");

Returns the following.

The sentiment score is -0.40

Similarly:

getSentiment("I LOVE MESSI");

Returns a higher sentiment. ?

The sentiment score is 0.89

Bringing it all together

The final thing to do is calling the getSetiment function with the text from the tweets.

There is a catch though: only the first 5,000 API requests are free, after that Google will charge you for subsequent API requests.

To minimise the amount of API calls I'm going to combine all the tweets into one single string like so:

let allTweets = "";
for (tweet of response.statuses) {
    allTweets += tweet.text + "\n";
}

const sentimentScore = await getSentimentScore(allTweets);
console.log(`The sentiment about ${query} is: ${sentimentScore}`);

Now I only have to call the API once instead of 100 times.

The final question is of course: what does Twitter think about Lionel Messi? When running the program it gives the following output:

The sentiment about Lionel Messi is: 0.2

So, Twitter is lightly positive about Lionel Messi.

Conclusion

We've created a NodeJS program that interacts with the Twitter API to get recent tweets. It then sends these tweets to the Google Cloud Natural Language API to perform a sentiment analysis.

You can find a live version of this sentiment analysis here.

You can also view the completed code here on Github.

When applying one-hot encoding to words, we end up with sparse (containing many zeros) vectors of high dimensionality. On large data sets, this could cause performance issues.

Additionally, one-hot encoding does not take into account the semantics of the words. So words like airplane and aircraft are considered to be two different features while we know that they have a very similar meaning. Word embeddings address these two issues.

Word embeddings are dense vectors with much lower dimensionality. Secondly, the semantic relationships between words are reflected in the distance and direction of the vectors.

We will work with the TwitterAirlineSentiment data set on Kaggle. This data set contains roughly 15K tweets with 3 possible classes for the sentiment (positive, negative and neutral). In my previous post, we tried to classify the tweets by tokenizing the words and applying two classifiers. Let’s see if word embeddings can outperform that.

After reading this tutorial you will know how to compute task-specific word embeddings with the Embedding layer of Keras. Secondly, we will investigate whether word embeddings trained on a larger corpus can improve the accuracy of our model.

The structure of this tutorial is:

• Intuition behind word embeddings
• Project set-up
• Data preparation
• Keras and its Embedding layer
• Pre-trained word embeddings — GloVe
• Training word embeddings with more dimensions

Intuition behind word embeddings

Before we can use words in a classifier, we need to convert them into numbers. One way to do that is to simply map words to integers. Another way is to one-hot encode words. Each tweet could then be represented as a vector with a dimension equal to (a limited set of) the words in the corpus. The words occurring in the tweet have a value of 1 in the vector. All other vector values equal zero.

Word embeddings are computed differently. Each word is positioned into a multi-dimensional space. The number of dimensions in this space is chosen by the data scientist. You can experiment with different dimensions and see what provides the best result.

The vector values for a word represent its position in this embedding space. Synonyms are found close to each other while words with opposite meanings have a large distance between them. You can also apply mathematical operations on the vectors which should produce semantically correct results. A typical example is that the sum of the word embeddings of king and female produces the word embedding of queen.

Project set-up

Let’s start by importing all packages for this project.

import pandas as pd
import numpy as np
import re
import collections
import matplotlib.pyplot as plt
from pathlib import Path
from sklearn.model_selection import train_test_split
from nltk.corpus import stopwords
from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
from keras.utils.np_utils import to_categorical
from sklearn.preprocessing import LabelEncoder
from keras import models
from keras import layers

We define some parameters and paths used throughout the project. Most of them are self-explanatory. But others will be explained further in the code.

NB_WORDS = 10000  # Parameter indicating the number of words we'll put in the dictionary
VAL_SIZE = 1000  # Size of the validation set
NB_START_EPOCHS = 10  # Number of epochs we usually start to train with
BATCH_SIZE = 512  # Size of the batches used in the mini-batch gradient descent
MAX_LEN = 24  # Maximum number of words in a sequence
GLOVE_DIM = 100  # Number of dimensions of the GloVe word embeddings
root = Path('../')
input_path = root / 'input/'
ouput_path = root / 'output/'
source_path = root / 'source/'

Throughout this code, we will also use some helper functions for data preparation, modeling and visualization. These function definitions are not shown here to keep the blog post clutter free. You can always refer to the notebook in Github to look at the code.

Data preparation

Reading the data and cleaning

We read in the CSV file with the tweets and apply a random shuffle on its indexes. After that, we remove stop words and @ mentions. A test set of 10% is split off to evaluate the model on new data.

df = pd.read_csv(input_path / 'Tweets.csv')
df = df.reindex(np.random.permutation(df.index))
df = df[['text', 'airline_sentiment']]
df.text = df.text.apply(remove_stopwords).apply(remove_mentions)
X_train, X_test, y_train, y_test = train_test_split(df.text, df.airline_sentiment, test_size=0.1, random_state=37)

Convert words into integers

With the Tokenizer from Keras, we convert the tweets into sequences of integers. We limit the number of words to the _NBWORDS most frequent words. Additionally, the tweets are cleaned with some filters, set to lowercase and split on spaces.

tk = Tokenizer(num_words=NB_WORDS,
filters='!"#$%&()*+,-./:;<=>?@[\]^_`{"}~\t\n',lower=True, split=" ")
tk.fit_on_texts(X_train)
X_train_seq = tk.texts_to_sequences(X_train)
X_test_seq = tk.texts_to_sequences(X_test)

Equal length of sequences

Each batch needs to provide sequences of equal length. We achieve this with the _padsequences method. By specifying maxlen, the sequences or padded with zeros or truncated.

X_train_seq_trunc = pad_sequences(X_train_seq, maxlen=MAX_LEN)
X_test_seq_trunc = pad_sequences(X_test_seq, maxlen=MAX_LEN)

Encoding the target variable

The target classes are strings which need to be converted into numeric vectors. This is done with the LabelEncoder from Sklearn and the _tocategorical method from Keras.

le = LabelEncoder()
y_train_le = le.fit_transform(y_train)
y_test_le = le.transform(y_test)
y_train_oh = to_categorical(y_train_le)
y_test_oh = to_categorical(y_test_le)

Splitting off the validation set

From the training data, we split off a validation set of 10% to use during training.

X_train_emb, X_valid_emb, y_train_emb, y_valid_emb = train_test_split(X_train_seq_trunc, y_train_oh, test_size=0.1, random_state=37)

Modeling

Keras and the Embedding layer

Keras provides a convenient way to convert each word into a multi-dimensional vector. This can be done with the Embedding layer. It will compute the word embeddings (or use pre-trained embeddings) and look up each word in a dictionary to find its vector representation. Here we will train word embeddings with 8 dimensions.

emb_model = models.Sequential()
emb_model.add(layers.Embedding(NB_WORDS, 8, input_length=MAX_LEN))
emb_model.add(layers.Flatten())
emb_model.add(layers.Dense(3, activation='softmax'))
emb_history = deep_model(emb_model, X_train_emb, y_train_emb, X_valid_emb, y_valid_emb)

We have a validation accuracy of about 74%. The number of words in the tweets is rather low, so this result is quite good. By comparing the training and validation loss, we see that the model starts overfitting from epoch 6.

In a previous article, I discussed how we can avoid overfitting. You might want to read that if you want to deep dive on that topic.

When we train the model on all data (including the validation data, but excluding the test data) and set the number of epochs to 6, we get a test accuracy of 78%. This test result is OK, but let’s see if we can improve with pre-trained word embeddings.

emb_results = test_model(emb_model, X_train_seq_trunc, y_train_oh, X_test_seq_trunc, y_test_oh, 6)
print('/n')
print('Test accuracy of word embeddings model: {0:.2f}%'.format(emb_results[1]*100))

Pre-trained word embeddings — Glove

Because the training data is not so large, the model might not be able to learn good embeddings for the sentiment analysis. Alternatively, we can load pre-trained word embeddings built on a much larger training data.

The GloVe database contains multiple pre-trained word embeddings, and more specific embeddings trained on tweets. So this might be useful for the task at hand.

First, we put the word embeddings in a dictionary where the keys are the words and the values the word embeddings.

glove_file = 'glove.twitter.27B.' + str(GLOVE_DIM) + 'd.txt'
emb_dict = {}
glove = open(input_path / glove_file)
for line in glove:
    values = line.split()
    word = values[0]
    vector = np.asarray(values[1:], dtype='float32')
    emb_dict[word] = vector
glove.close()

With the GloVe embeddings loaded in a dictionary, we can look up the embedding for each word in the corpus of the airline tweets. These will be stored in a matrix with a shape of _NBWORDS and _GLOVEDIM. If a word is not found in the GloVe dictionary, the word embedding values for the word are zero.

emb_matrix = np.zeros((NB_WORDS, GLOVE_DIM))
for w, i in tk.word_index.items():
    if i < NB_WORDS:
        vect = emb_dict.get(w)
        if vect is not None:
        emb_matrix[i] = vect
    else:
        break

Then we specify the model just like we did with the model above.

glove_model = models.Sequential()
glove_model.add(layers.Embedding(NB_WORDS, GLOVE_DIM, input_length=MAX_LEN))
glove_model.add(layers.Flatten())
glove_model.add(layers.Dense(3, activation='softmax'))

In the Embedding layer (which is layer 0 here) we set the weights for the words to those found in the GloVe word embeddings. By setting trainable to False we make sure that the GloVe word embeddings cannot be changed. After that, we run the model.

glove_model.layers[0].set_weights([emb_matrix])
glove_model.layers[0].trainable = False
glove_history = deep_model(glove_model, X_train_emb, y_train_emb, X_valid_emb, y_valid_emb)

The model overfits fast after 3 epochs. Furthermore, the validation accuracy is lower compared to the embeddings trained on the training data.

glove_results = test_model(glove_model, X_train_seq_trunc, y_train_oh, X_test_seq_trunc, y_test_oh, 3)
print('/n')
print('Test accuracy of word glove model: {0:.2f}%'.format(glove_results[1]*100))

As a final exercise, let’s see what results we get when we train the embeddings with the same number of dimensions as the GloVe data.

Training word embeddings with more dimensions

We will train the word embeddings with the same number of dimensions as the GloVe embeddings (i.e. GLOVE_DIM).

emb_model2 = models.Sequential()
emb_model2.add(layers.Embedding(NB_WORDS, GLOVE_DIM, input_length=MAX_LEN))
emb_model2.add(layers.Flatten())
emb_model2.add(layers.Dense(3, activation='softmax'))
emb_history2 = deep_model(emb_model2, X_train_emb, y_train_emb, X_valid_emb, y_valid_emb)

emb_results2 = test_model(emb_model2, X_train_seq_trunc, y_train_oh, X_test_seq_trunc, y_test_oh, 3)
print('/n')
print('Test accuracy of word embedding model 2: {0:.2f}%'.format(emb_results2[1]*100))

On the test data we get good results, but we do not outperform the LogisticRegression with the CountVectorizer. So there is still room for improvement.

Conclusion

The best result is achieved with 100-dimensional word embeddings that are trained on the available data. This even outperforms the use of word embeddings that were trained on a much larger Twitter corpus.

Until now we have just put a Dense layer on the flattened embeddings. By doing this, we do not take into account the relationships between the words in the tweet. This can be achieved with a recurrent neural network or a 1D convolutional network. But that’s something for a future post :)

In this tutorial, I will explore some text mining techniques for sentiment analysis. We'll look at how to prepare textual data. After that we will try two different classifiers to infer the tweets' sentiment. We will tune the hyperparameters of both classifiers with grid search. Finally, we evaluate the performance on a set of metrics like precision, recall and the F1 score.

For this project, we'll be working with the Twitter US Airline Sentiment data set on Kaggle. It contains the tweet’s text and one variable with three possible sentiment values. Let's start by importing the packages and configuring some settings.

import numpy as np 
import pandas as pd 
pd.set_option('display.max_colwidth', -1)
from time import time
import re
import string
import os
import emoji
from pprint import pprint
import collections
import matplotlib.pyplot as plt
import seaborn as sns
sns.set(style="darkgrid")
sns.set(font_scale=1.3)
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline, FeatureUnion
from sklearn.metrics import classification_report
from sklearn.naive_bayes import MultinomialNB
from sklearn.linear_model import LogisticRegression
from sklearn.externals import joblib
import gensim
from nltk.corpus import stopwords
from nltk.stem import PorterStemmer
from nltk.tokenize import word_tokenize
import warnings
warnings.filterwarnings('ignore')
np.random.seed(37)

Loading the data

We read in the comma separated file we downloaded from the Kaggle Datasets. We shuffle the data frame in case the classes are sorted. Applying the reindex method on the permutation of the original indices is good for that. In this notebook, we will work with the text variable and the airline_sentiment variable.

df = pd.read_csv('../input/Tweets.csv')
df = df.reindex(np.random.permutation(df.index))
df = df[['text', 'airline_sentiment']]

Exploratory Data Analysis

Target variable

There are three class labels we will predict: negative, neutral or positive.

The class labels are imbalanced as we can see below in the chart. This is something that we should keep in mind during the model training phase. With the factorplot of the seaborn package, we can visualize the distribution of the target variable.

sns.factorplot(x="airline_sentiment", data=df, kind="count", size=6, aspect=1.5, palette="PuBuGn_d")
plt.show();

Imbalanced distribution of the target class labels

Input variable

To analyze the text variable we create a class TextCounts. In this class we compute some basic statistics on the text variable.

• count_words: number of words in the tweet
• count_mentions: referrals to other Twitter accounts start with a @
• count_hashtags: number of tag words, preceded by a #
• count_capital_words: number of uppercase words are sometimes used to “shout” and express (negative) emotions
• count_excl_quest_marks: number of question or exclamation marks
• count_urls: number of links in the tweet, preceded by http(s)
• count_emojis: number of emoji, which might be a good sign of the sentiment

class TextCounts(BaseEstimator, TransformerMixin):

    def count_regex(self, pattern, tweet):
        return len(re.findall(pattern, tweet))

    def fit(self, X, y=None, **fit_params):
        # fit method is used when specific operations need to be done on the train data, but not on the test data
        return self

    def transform(self, X, **transform_params):
        count_words = X.apply(lambda x: self.count_regex(r'\w+', x)) 
        count_mentions = X.apply(lambda x: self.count_regex(r'@\w+', x))
        count_hashtags = X.apply(lambda x: self.count_regex(r'#\w+', x))
        count_capital_words = X.apply(lambda x: self.count_regex(r'\b[A-Z]{2,}\b', x))
        count_excl_quest_marks = X.apply(lambda x: self.count_regex(r'!|\?', x))
        count_urls = X.apply(lambda x: self.count_regex(r'http.?://[^\s]+[\s]?', x))
        # We will replace the emoji symbols with a description, which makes using a regex for counting easier
        # Moreover, it will result in having more words in the tweet
        count_emojis = X.apply(lambda x: emoji.demojize(x)).apply(lambda x: self.count_regex(r':[a-z_&]+:', x))

        df = pd.DataFrame({'count_words': count_words
                           , 'count_mentions': count_mentions
                           , 'count_hashtags': count_hashtags
                           , 'count_capital_words': count_capital_words
                           , 'count_excl_quest_marks': count_excl_quest_marks
                           , 'count_urls': count_urls
                           , 'count_emojis': count_emojis
                          })

        return df
tc = TextCounts()
df_eda = tc.fit_transform(df.text)
df_eda['airline_sentiment'] = df.airline_sentiment

It could be interesting to see how the TextStats variables relate to the class variable. So we write a function show_dist that provides descriptive statistics and a plot per target class.

def show_dist(df, col):
    print('Descriptive stats for {}'.format(col))
    print('-'*(len(col)+22))
    print(df.groupby('airline_sentiment')[col].describe())
    bins = np.arange(df[col].min(), df[col].max() + 1)
    g = sns.FacetGrid(df, col='airline_sentiment', size=5, hue='airline_sentiment', palette="PuBuGn_d")
    g = g.map(sns.distplot, col, kde=False, norm_hist=True, bins=bins)
    plt.show()

Below you can find the distribution of the number of words in a tweet per target class. For brevity, we will limit us to only this variable. The charts for all TextCounts variables are in the notebook on Github.

• The number of words used in the tweets is rather low. The largest number of words is 36 and there are even tweets with only 2 words. So we’ll have to be careful during data cleaning not to remove too many words. But the text processing will be faster. Negative tweets contain more words than neutral or positive tweets.
• All tweets have at least one mention. This is the result of extracting the tweets based on mentions in the Twitter data. There seems to be no difference in the number of mentions with regard to the sentiment.
• Most of the tweets do not contain hash tags. So this variable will not be retained during model training. Again, no difference in the number of hash tags with regard to the sentiment.
• Most of the tweets do not contain capitalized words and we do not see a difference in distribution between the sentiments.
• The positive tweets seem to be using a bit more exclamation or question marks.
• Most tweets do not contain a URL.
• Most tweets do not use emojis.

Text Cleaning

Before we start using the tweets’ text we need to clean it. We’ll do the this in the class CleanText. With this class we’ll perform the following actions:

• remove the mentions, as we want to generalize to tweets of other airline companies too.
• remove the hash tag sign (#) but not the actual tag as this may contain information
• set all words to lowercase
• remove all punctuations, including the question and exclamation marks
• remove the URLs as they do not contain useful information. We did not notice a difference in the number of URLs used between the sentiment classes
• make sure to convert the emojis into one word.
• remove digits
• remove stopwords
• apply the PorterStemmer to keep the stem of the words

class CleanText(BaseEstimator, TransformerMixin):
    def remove_mentions(self, input_text):
        return re.sub(r'@\w+', '', input_text)

    def remove_urls(self, input_text):
        return re.sub(r'http.?://[^\s]+[\s]?', '', input_text)

    def emoji_oneword(self, input_text):
        # By compressing the underscore, the emoji is kept as one word
        return input_text.replace('_','')

    def remove_punctuation(self, input_text):
        # Make translation table
        punct = string.punctuation
        trantab = str.maketrans(punct, len(punct)*' ')  # Every punctuation symbol will be replaced by a space
        return input_text.translate(trantab)
    def remove_digits(self, input_text):
        return re.sub('\d+', '', input_text)

    def to_lower(self, input_text):
        return input_text.lower()

    def remove_stopwords(self, input_text):
        stopwords_list = stopwords.words('english')
        # Some words which might indicate a certain sentiment are kept via a whitelist
        whitelist = ["n't", "not", "no"]
        words = input_text.split() 
        clean_words = [word for word in words if (word not in stopwords_list or word in whitelist) and len(word) > 1] 
        return " ".join(clean_words) 

    def stemming(self, input_text):
        porter = PorterStemmer()
        words = input_text.split() 
        stemmed_words = [porter.stem(word) for word in words]
        return " ".join(stemmed_words)

    def fit(self, X, y=None, **fit_params):
        return self

    def transform(self, X, **transform_params):
        clean_X = X.apply(self.remove_mentions).apply(self.remove_urls).apply(self.emoji_oneword).apply(self.remove_punctuation).apply(self.remove_digits).apply(self.to_lower).apply(self.remove_stopwords).apply(self.stemming)
        return clean_X

To show how the cleaned text variable will look like, here’s a sample.

ct = CleanText()
sr_clean = ct.fit_transform(df.text)
sr_clean.sample(5)

glad rt bet bird wish flown south winter
point upc code check baggag tell luggag vacat day tri swimsuit
vx jfk la dirti plane not standard
tell mean work need estim time arriv pleas need laptop work thank
sure busi go els airlin travel name kathryn sotelo

One side-effect of text cleaning is that some rows do not have any words left in their text. For the CountVectorizer and TfIdfVectorizer this does not pose a problem. Yet, for the Word2Vec algorithm this causes an error. There are different strategies to deal with these missing values.

• Remove the complete row, but in a production environment this is not desirable.
• Impute the missing value with some placeholder text like [no_text]
• When applying Word2Vec: use the average of all vectors

Here we will impute with placeholder text.

empty_clean = sr_clean == ''
print('{} records have no words left after text cleaning'.format(sr_clean[empty_clean].count()))
sr_clean.loc[empty_clean] = '[no_text]'

Now that we have the cleaned text of the tweets, we can have a look at what are the most frequent words. Below we’ll show the top 20 words. The most frequent word is “flight”.

cv = CountVectorizer()
bow = cv.fit_transform(sr_clean)
word_freq = dict(zip(cv.get_feature_names(), np.asarray(bow.sum(axis=0)).ravel()))
word_counter = collections.Counter(word_freq)
word_counter_df = pd.DataFrame(word_counter.most_common(20), columns = ['word', 'freq'])
fig, ax = plt.subplots(figsize=(12, 10))
sns.barplot(x="word", y="freq", data=word_counter_df, palette="PuBuGn_d", ax=ax)
plt.show();

Creating test data

To check the performance of the models we’ll need a test set. Evaluating on the train data would not be correct. You should not test on the same data used for training the model.

First, we combine the TextCounts variables with the CleanText variable. Initially, I made the mistake to execute TextCounts and CleanText in the GridSearchCV. This took too long as it applies these functions each run of the GridSearch. It suffices to run them only once.

df_model = df_eda
df_model['clean_text'] = sr_clean
df_model.columns.tolist()

So df_model now contains several variables. But our vectorizers (see below) will only need the clean_text variable. The TextCountsvariables can be added as such. To select columns, I wrote the class ColumnExtractor below.

class ColumnExtractor(TransformerMixin, BaseEstimator):
    def __init__(self, cols):
        self.cols = cols
    def transform(self, X, **transform_params):
        return X[self.cols]
    def fit(self, X, y=None, **fit_params):
        return self
X_train, X_test, y_train, y_test = train_test_split(df_model.drop('airline_sentiment', axis=1), df_model.airline_sentiment, test_size=0.1, random_state=37)

Hyperparameter tuning and cross-validation

As we will see below, the vectorizers and classifiers all have configurable parameters. To choose the best parameters, we need to test on a separate validation set. This validation set was not used during the training. Yet, using only one validation set may not produce reliable validation results. Due to chance, you might have a good model performance on the validation set. If you would split the data otherwise, you might end up with other results. To get a more accurate estimation, we perform cross-validation.

With cross-validation we split the data into a train and validation set many times. The evaluation metric is then averaged over the different folds. Luckily, GridSearchCV applies cross-validation out-of-the-box.

To find the best parameters for both a vectorizer and classifier, we create a Pipeline.

Evaluation metrics

By default GridSearchCV uses the default scorer to compute the best_score_. For both the MultiNomialNb and LogisticRegression this default scoring metric is accuracy.

In our function grid_vectwe additionally generate the classification_report on the test data. This provides some interesting metrics per target class. This might be more appropriate here. These metrics are the precision, recall and F1 score.

• Precision: Of all rows we predicted to be a certain class, how many did we correctly predict?
• Recall: Of all rows of a certain class, how many did we correctly predict?
• F1 score: Harmonic mean of Precision and Recall.

With the elements of the confusion matrix we can calculate Precision and Recall.

# Based on http://scikit-learn.org/stable/auto_examples/model_selection/grid_search_text_feature_extraction.html
def grid_vect(clf, parameters_clf, X_train, X_test, parameters_text=None, vect=None, is_w2v=False):

    textcountscols = ['count_capital_words','count_emojis','count_excl_quest_marks','count_hashtags'
                      ,'count_mentions','count_urls','count_words']

    if is_w2v:
        w2vcols = []
        for i in range(SIZE):
            w2vcols.append(i)
        features = FeatureUnion([('textcounts', ColumnExtractor(cols=textcountscols))
                                 , ('w2v', ColumnExtractor(cols=w2vcols))]
                                , n_jobs=-1)
    else:
        features = FeatureUnion([('textcounts', ColumnExtractor(cols=textcountscols))
                                 , ('pipe', Pipeline([('cleantext', ColumnExtractor(cols='clean_text')), ('vect', vect)]))]
                                , n_jobs=-1)

    pipeline = Pipeline([
        ('features', features)
        , ('clf', clf)
    ])

    # Join the parameters dictionaries together
    parameters = dict()
    if parameters_text:
        parameters.update(parameters_text)
    parameters.update(parameters_clf)
    # Make sure you have scikit-learn version 0.19 or higher to use multiple scoring metrics
    grid_search = GridSearchCV(pipeline, parameters, n_jobs=-1, verbose=1, cv=5)

    print("Performing grid search...")
    print("pipeline:", [name for name, _ in pipeline.steps])
    print("parameters:")
    pprint(parameters)
    t0 = time()
    grid_search.fit(X_train, y_train)
    print("done in %0.3fs" % (time() - t0))
    print()
    print("Best CV score: %0.3f" % grid_search.best_score_)
    print("Best parameters set:")
    best_parameters = grid_search.best_estimator_.get_params()
    for param_name in sorted(parameters.keys()):
        print("\t%s: %r" % (param_name, best_parameters[param_name]))

    print("Test score with best_estimator_: %0.3f" % grid_search.best_estimator_.score(X_test, y_test))
    print("\n")
    print("Classification Report Test Data")
    print(classification_report(y_test, grid_search.best_estimator_.predict(X_test)))

    return grid_search

Parameter grids for GridSearchCV

In the grid search, we will investigate the performance of the classifier. The set of parameters used to test the performance are specified below.

# Parameter grid settings for the vectorizers (Count and TFIDF)
parameters_vect = {
    'features__pipe__vect__max_df': (0.25, 0.5, 0.75),
    'features__pipe__vect__ngram_range': ((1, 1), (1, 2)),
    'features__pipe__vect__min_df': (1,2)
}

# Parameter grid settings for MultinomialNB
parameters_mnb = {
    'clf__alpha': (0.25, 0.5, 0.75)
}

# Parameter grid settings for LogisticRegression
parameters_logreg = {
    'clf__C': (0.25, 0.5, 1.0),
    'clf__penalty': ('l1', 'l2')
}

Classifiers

Here we will compare the performance of a MultinomialNBand LogisticRegression.

mnb = MultinomialNB()
logreg = LogisticRegression()

CountVectorizer

To use words in a classifier, we need to convert the words to numbers. Sklearn’s CountVectorizer takes all words in all tweets, assigns an ID and counts the frequency of the word per tweet. We then use this bag of words as input for a classifier. This bag of words is a sparse data set. This means that each record will have many zeroes for the words not occurring in the tweet.

countvect = CountVectorizer()
# MultinomialNB
best_mnb_countvect = grid_vect(mnb, parameters_mnb, X_train, X_test, parameters_text=parameters_vect, vect=countvect)
joblib.dump(best_mnb_countvect, '../output/best_mnb_countvect.pkl')
# LogisticRegression
best_logreg_countvect = grid_vect(logreg, parameters_logreg, X_train, X_test, parameters_text=parameters_vect, vect=countvect)
joblib.dump(best_logreg_countvect, '../output/best_logreg_countvect.pkl')

TF-IDF Vectorizer

One issue with CountVectorizer is that there might be words that occur frequently. These words might not have discriminatory information. Thus they can be removed. TF-IDF (term frequency — inverse document frequency)can be used to down-weight these frequent words.

tfidfvect = TfidfVectorizer()
# MultinomialNB
best_mnb_tfidf = grid_vect(mnb, parameters_mnb, X_train, X_test, parameters_text=parameters_vect, vect=tfidfvect)
joblib.dump(best_mnb_tfidf, '../output/best_mnb_tfidf.pkl')
# LogisticRegression
best_logreg_tfidf = grid_vect(logreg, parameters_mnb, X_train, X_test, parameters_text=parameters_vect, vect=tfidfvect)
joblib.dump(best_logreg_tfidf, '../output/best_logreg_tfidf.pkl')

Word2Vec

Another way of converting the words to numerical values is to use Word2Vec. Word2Vec maps each word in a multi-dimensional space. It does this by taking into account the context in which a word appears in the tweets. As a result, words that are similar are also close to each other in the multi-dimensional space.

The Word2Vec algorithm is part of the gensim package.

The Word2Vec algorithm uses lists of words as input. For that purpose, we use the word_tokenize method of the the nltk package.

SIZE = 50
X_train['clean_text_wordlist'] = X_train.clean_text.apply(lambda x : word_tokenize(x))
X_test['clean_text_wordlist'] = X_test.clean_text.apply(lambda x : word_tokenize(x))
model = gensim.models.Word2Vec(X_train.clean_text_wordlist
, min_count=1
, size=SIZE
, window=5
, workers=4)
model.most_similar('plane', topn=3)

The Word2Vec model provides a vocabulary of the words in all the tweets. For each word you also have its vector values. The number of vector values is equal to the chosen size. These are the dimensions on which each word is mapped in the multi-dimensional space. Words with an occurrence less than min_count are not kept in the vocabulary.

A side effect of the min_count parameter is that some tweets could have no vector values. This is would be the case when the word(s) in the tweet occur in less than min_count tweets. Due to the small corpus of tweets, there is a risk of this happening in our case. Thus we set the min_count value equal to 1.

The tweets can have a different number of vectors, depending on the number of words it contains. To use this output for modeling we will calculate the average of all vectors per tweet. As such we will have the same number (i.e. size) of input variables per tweet.

We do this with the function compute_avg_w2v_vector. In this function we also check whether the words in the tweet occur in the vocabulary of the Word2Vec model. If not, a list filled with 0.0 is returned. Else the average of the word vectors.

def compute_avg_w2v_vector(w2v_dict, tweet):
    list_of_word_vectors = [w2v_dict[w] for w in tweet if w in w2v_dict.vocab.keys()]

    if len(list_of_word_vectors) == 0:
        result = [0.0]*SIZE
    else:
        result = np.sum(list_of_word_vectors, axis=0) / len(list_of_word_vectors)

    return result
X_train_w2v = X_train['clean_text_wordlist'].apply(lambda x: compute_avg_w2v_vector(model.wv, x))
X_test_w2v = X_test['clean_text_wordlist'].apply(lambda x: compute_avg_w2v_vector(model.wv, x))

This gives us a Series with a vector of dimension equal to SIZE. Now we will split this vector and create a DataFrame with each vector value in a separate column. That way we can concatenate the Word2Vec variables to the other TextCounts variables. We need to reuse the index of X_train and X_test. Otherwise this will give issues (duplicates) in the concatenation later on.

X_train_w2v = pd.DataFrame(X_train_w2v.values.tolist(), index= X_train.index)
X_test_w2v = pd.DataFrame(X_test_w2v.values.tolist(), index= X_test.index)
# Concatenate with the TextCounts variables
X_train_w2v = pd.concat([X_train_w2v, X_train.drop(['clean_text', 'clean_text_wordlist'], axis=1)], axis=1)
X_test_w2v = pd.concat([X_test_w2v, X_test.drop(['clean_text', 'clean_text_wordlist'], axis=1)], axis=1)

We only consider LogisticRegression as we have negative values in the Word2Vec vectors. MultinomialNB assumes that the variables have a multinomial distribution. So they cannot contain negative values.

best_logreg_w2v = grid_vect(logreg, parameters_logreg, X_train_w2v, X_test_w2v, is_w2v=True)
joblib.dump(best_logreg_w2v, '../output/best_logreg_w2v.pkl')

Conclusion

• Both classifiers achieve the best results when using the features of the CountVectorizer
• Logistic Regression outperforms the Multinomial Naive Bayes classifier
• The best performance on the test set comes from the LogisticRegression with features from CountVectorizer.

Best parameters

• C value of 1
• L2 regularization
• max_df: 0.5 or maximum document frequency of 50%.
• min_df: 1 or the words need to appear in at least 2 tweets
• ngram_range: (1, 2), both single words as bi-grams are used

Evaluation metrics

• A test accuracy of 81,3%. This is better than a baseline performance of predicting the majority class (here a negative sentiment) for all observations. The baseline would give 63% accuracy.
• The Precision is rather high for all three classes. For instance, of all cases that we predict as negative, 80% is negative.
• The Recall for the neutral class is low. Of all neutral cases in our test data, we only predict 48% as being neutral.

Apply the best model on new tweets

For the fun, we will use the best model and apply it to some new tweets that contain @VirginAmerica. I selected 3 negative and 3 positive tweets by hand.

Thanks to the GridSearchCV, we now know what are the best hyperparameters. So now we can train the best model on all training data, including the test data that we split off before.

textcountscols = ['count_capital_words','count_emojis','count_excl_quest_marks','count_hashtags'
,'count_mentions','count_urls','count_words']
features = FeatureUnion([('textcounts', ColumnExtractor(cols=textcountscols))
, ('pipe', Pipeline([('cleantext', ColumnExtractor(cols='clean_text'))
, ('vect', CountVectorizer(max_df=0.5, min_df=1, ngram_range=(1,2)))]))]
, n_jobs=-1)
pipeline = Pipeline([
('features', features)
, ('clf', LogisticRegression(C=1.0, penalty='l2'))
])
best_model = pipeline.fit(df_model.drop('airline_sentiment', axis=1), df_model.airline_sentiment)
# Applying on new positive tweets
new_positive_tweets = pd.Series(["Thank you @VirginAmerica for you amazing customer support team on Tuesday 11/28 at @EWRairport and returning my lost bag in less than 24h! #efficiencyiskey #virginamerica"
,"Love flying with you guys ask these years. Sad that this will be the last trip ? @VirginAmerica #LuxuryTravel"
,"Wow @VirginAmerica main cabin select is the way to fly!! This plane is nice and clean & I have tons of legroom! Wahoo! NYC bound! ✈️"])
df_counts_pos = tc.transform(new_positive_tweets)
df_clean_pos = ct.transform(new_positive_tweets)
df_model_pos = df_counts_pos
df_model_pos['clean_text'] = df_clean_pos
best_model.predict(df_model_pos).tolist()
# Applying on new negative tweets
new_negative_tweets = pd.Series(["@VirginAmerica shocked my initially with the service, but then went on to shock me further with no response to what my complaint was. #unacceptable @Delta @richardbranson"
,"@VirginAmerica this morning I was forced to repack a suitcase w a medical device because it was barely overweight - wasn't even given an option to pay extra. My spouses suitcase then burst at the seam with the added device and had to be taped shut. Awful experience so far!"
,"Board airplane home. Computer issue. Get off plane, traverse airport to gate on opp side. Get on new plane hour later. Plane too heavy. 8 volunteers get off plane. Ohhh the adventure of travel ✈️ @VirginAmerica"])
df_counts_neg = tc.transform(new_negative_tweets)
df_clean_neg = ct.transform(new_negative_tweets)
df_model_neg = df_counts_neg
df_model_neg['clean_text'] = df_clean_neg
best_model.predict(df_model_neg).tolist()

The model classifies all tweets correctly. A larger test set should be used to assess the model’s performance. But on this small data set it does what we are aiming for. I hope you enjoyed reading this story. If you did, feel free to share it.

This blog is based on the video Twitter Sentiment Analysis — Learn Python for Data Science #2 by Siraj Raval. In this challenge, we will be building a sentiment analyzer that checks whether tweets about a subject are negative or positive. We will be making use of the Python library textblob for this.

image from google

Sentiment Analysis, also called opinion mining or emotion AI, is the process of determining whether a piece of writing is positive, negative, or neutral. A common use case for this technology is to discover how people feel about a particular topic. Sentiment analysis is widely applied to reviews and social media for a variety of applications.

Sentiment analysis can be performed in many different ways. Many brands and marketers use keyword-based tools that classify data (i.e. social, news, review, blog, etc.) as positive/negative/neutral.

Automated sentiment tagging is usually achieved through word lists. For example, mentions of ‘hate’ would be tagged negatively.

There can be two approaches to sentiment analysis.

1. Lexicon-based methods
2. Machine Learning-based methods.

In this problem, we will be using a Lexicon-based method.

Lexicon based methods define a list of positive and negative words, with a valence — (eg ‘nice’: +2, ‘good’: +1, ‘terrible’: -1.5 etc). The algorithm looks up a text to find all known words. It then combines their individual results by summing or averaging. Some extensions can check some grammatical rules, like negation or sentiment modifier (like the word “but”, which weights sentiment values in text differently, to emphasize the end of text).

Let’s build the analyzer now.

Twitter API

Before we start coding, we need to register for the Twitter API https://apps.twitter.com/. Here we need to register an app to generate various keys associated with our API. The Twitter API can be used to perform many actions like create and search.

Now after creating the app we can start coding.

We need to install two packages:

pip install tweepy

This package will be used for handling the Twitter API.

pip install textblob

This package will be used for the sentiment analysis.

sentiment_analyzer.py

import tweepyfrom textblob import TextBlob

We need to declare the variables to store the various keys associated with the Twitter API.

consumer_key = ‘[consumer_key]’

consumer_key_secret = ‘[consumer_key_secret]’

access_token = ‘[access_token]’

access_token_secret = ‘[access_token_secret]’

The next step is to create a connection with the Twitter API using tweepy with these tokens.

Tweepy

Tweepy supports OAuth authentication. Authentication is handled by the tweepy.OAuthHandler class.

An OAuthHandler instance must be created by passing a consumer token and secret.

On this auth instance, we will call a function set_access_token by passing the access_token and access_token_secret.

Finally, we create our tweepy API instance by passing this auth instance into the API function of tweepy.

auth = tweepy.OAuthHandler(consumer_key, consumer_key_secret)

auth.set_access_token(access_token, access_token_secret)

api = tweepy.API(auth)

We can now search Twitter for any topic using the search method of the API.

public_tweets = api.search(‘Dogs’)

Now we will be getting all the tweets related to the topic ‘Dogs’. We can perform sentiment analysis using the library textblob.

TextBlob

TextBlob is a Python (2 and 3) library for processing textual data. It provides a simple API for diving into common natural language processing (NLP) tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis, classification, translation, and more.

A textblob can be created in the following way (example, and not part of the original code):

example = TextBlob("Python is a high-level, general-purpose programming language.")

And tokenization can be performed by the following methods:

words: returns the words of text

usage:

example.words

sentences: returns the sentences of text

usage:

example.sentences

Part-of-speech Tagging

Part-of-speech tags can be accessed through the tags property.

wiki.tags[('Python', 'NNP'), ('is', 'VBZ'), ('a', 'DT'), ('high-level', 'JJ'), ('general-purpose', 'JJ'), ('programming', 'NN'), ('language', 'NN')]

Sentiment Analysis

The sentiment property returns a named tuple of the form Sentiment (polarity, subjectivity). The polarity score is a float within the range [-1.0, 1.0]. The subjectivity is a float within the range [0.0, 1.0] where 0.0 is very objective and 1.0 is very subjective.

Now back to the code.

We can iterate the publice_tweets array, and check the sentiment of the text of each tweet based on the polarity.

for tweet in public_tweets:    print(tweet.text)    analysis = TextBlob(tweet.text)    print(analysis.sentiment)    if analysis.sentiment[0]>0:       print 'Positive'    elif analysis.sentiment[0]<0:       print 'Negative'    else:       print 'Neutral'

Now we run the code using the following:

python sentiment_analyzer.py

and we get the output:

We can see that the sentiment of the tweet is displayed.

This is an example of how sentiment analysis can be done on data from social media like Twitter. I hope you find it useful!

Find the code at https://github.com/amkurian/twitter_sentiment_challenge

Write your own sentiment checker in 5 minutes.

Sentiment Analysis is the process of determining whether a piece of text is positive, negative, or neutral.

Real world applications for Sentiment Analysis

The goal of this article is to get you up and running using the Google Natural Language API with Laravel. You’ll be using this API to perform sentiment analysis on text.

Using these techniques, you can build some great functionality into existing applications. Some ideas include:

• detecting sentiment in comments or reviews
• forecasting market movements based on social media activity
• ascertaining the effectiveness of a marketing campaign by observing sentiment before and after

Interpreting Sentiment Analysis values

The Google API takes the provided text, analyzes it, and determines the prevailing emotional opinion. It determines whether the writing is positive, negative, or neutral.

The sentiment is represented by numerical score and magnitude values.

• The score ranges between -1.0 (negative) and 1.0 (positive).
• The magnitude indicates the strength of emotion (both positive and negative). The range spans from 0.0 to infinity. The magnitude is not normalized, so longer passages of text will always have a larger magnitude.

The values above are guides only, and you’ll need to adjust according to your specific environment.

Google Cloud Platform setup

The first step involves creating a new project in the Google Cloud Platform console.

Head over to the dashboard and create a new project.

Once your project is created, keep the Project ID handy.

• Once you have your project, go to the Create service account key page.
• Ensure your Sample project is selected at the top.
• Under Service account, select New service account.
• Enter a name in the Service account name field.
• Under Role, select Project > Owner.
• Finally, click Create to have the JSON credentials file downloaded automatically.

You may also need to enable the Cloud Natural Language API via the API Library section.

Laravel project setup

The next step involves setting up a new Laravel project. If you already have an existing Laravel project, you can skip this step.

I’m using Laravel 5.5 LTS for this article. In the command line, run the following Composer command to create a new project (you can also use the Laravel installer):

composer create-project --prefer-dist laravel/laravel sample "5.5.*"

If you used Composer, rename the .env.example file to .env and run the following command afterwards to set the application key:

php artisan key:generate

Add the Google “cloud-language” package

Run the following command to add the Google Cloud Natural Language package to your project:

composer require google/cloud-language

You may go ahead and place the downloaded JSON credentials file in your application root (NOT in your public directory). Feel free to rename it. Never commit this file to your code repo — the same goes for any sensitive settings. One option is to add it to the server manually after initial deployment.

The main event: adding the actual code to your project

I’ll be adding the following route to my routes/web.php file:

<?php

Route::get('/', 'SampleController@sentiment');

I’ve created a simple controller to house the code. I’ll be adding all the code within the controller. In a production application, I strongly suggest using separate service classes for any business logic. This way controllers are lean and stick to their original intention: controlling the input/output.

We’ll start with a simple controller, adding a use statement to include the Google Cloud ServiceBuilder class:

<?php

namespace App\Http\Controllers;

use Google\Cloud\Core\ServiceBuilder;

class SampleController extends Controller{    public function sentiment()    {        // Code will be added here    }}

The first thing we’ll do is create an instance of the ServiceBuilder class so we can specify our Project ID and JSON credentials.

$cloud = new ServiceBuilder([    'keyFilePath' => base_path('gc.json'),    'projectId' => 'sample-207012']);

You specify the location of the JSON file using the keyFilePath option. I’ve used the Laravel base_path() helper to refer to the fully qualified app root path.

The next option is the projectId. This is the value you grabbed when you created the project in the GCP console.

Next, we’ll create an instance of the LanguageClient class. The ServiceBuilder class makes it easy by exposing various factory methods which grant access to services in the API.

$language = $cloud->language();

Now that we have an instance of the class, we can start making use of the Natural Language API. We’ll declare a variable with some text, analyze the sentiment, and output the results:

// The text to analyse$text = 'I hate this - why did they not make provisions?';

// Detect the sentiment of the text$annotation = $language->analyzeSentiment($text);$sentiment = $annotation->sentiment();

echo 'Sentiment Score: ' . $sentiment['score'] . ', Magnitude: ' . $sentiment['magnitude'];

And that’s all there is to it!

Output for the code above.

Here is the final controller class code:

<?php

namespace App\Http\Controllers;

use Google\Cloud\Core\ServiceBuilder;

class SampleController extends Controller{    public function sentiment()    {        $cloud = new ServiceBuilder([            'keyFilePath' => base_path('gc.json'),            'projectId' => 'sample-207012'        ]);

        $language = $cloud->language();

        // The text to analyse        $text = 'I hate this - why did they not make provisions?';

        // Detect the sentiment of the text        $annotation = $language->analyzeSentiment($text);        $sentiment = $annotation->sentiment();

        echo 'Sentiment Score: ' . $sentiment['score'] . ', Magnitude: ' . $sentiment['magnitude'];    }}

Conclusion

We’ve only scratched the surface of what the Google Natural Language API has to offer. Once you’ve come to grips with this, I suggest checking out the following additional services available in this API:

• Entity Analysis: analyze entities like landmarks and public figures.
• Content Classification: analyze text and return a list of categories that apply to the content.

If you have any questions — please feel free to make contact!

Originally published at www.chowles.com on June 13, 2018.