It involves creating new features (columns), transforming existing ones, and selecting the most relevant attributes to improve the performance and accuracy of machine learning models.

Feature engineering helps the model understand the data’s underlying patterns, relationships, and nuances. It plays a crucial role in the success of a machine learning project, as it can turn a good model into a great one by optimizing the input features. Effective feature engineering also leads to faster training times, and more interpretable results.

Before delving into the realm of feature engineering, you need to recognize the pivotal role of data cleaning in ensuring the success of Feature Engineering. Addressing missing values, handling outliers, and resolving inconsistencies not only improves the integrity of the data but also establishes a solid groundwork for subsequent feature extraction and transformation.

In this tutorial, we'll explore the concept of feature engineering for structured data. We'll cover some techniques such as feature scaling, feature creation, feature selection, and binning. We'll use a hypothetical dataset to demonstrate these various feature engineering techniques.

Let's start by creating some dummy data.

How to Create Dummy Data

We'll use Python and the pandas library to create a dummy dataset for our feature engineering examples. Here's how you can generate dummy data:

import pandas as pd
import numpy as np
# Create a sample dataframe
data = {
'Age': [25, 30, 35, 40, 45],
'Income': [50000, 60000, 75000, 90000, 80000],
'Education': ['High School', 'Bachelor', 'Master', 'Ph.D.', 'Bachelor'],
'City': ['New York', 'San Francisco', 'Chicago', 'Los Angeles', 'Miami'],
'Gender': ['Male', 'Female', 'Male', 'Female', 'Male'],
'Productivity': [5, 4, 3, 2, 4]
}
df = pd.DataFrame(data)

Code Output

As seen above, the dummy dataset created consists of six features: 'Age,' 'Income,' 'Education,' 'City,' 'Gender,' and 'Productivity.'

Now, let's explore different feature engineering techniques.

One-Hot Encoding

One-hot encoding is a method used to convert categorical variables into a binary matrix. Our dataset has some columns filled with words such as gender, education, and cities.

But machines like numbers, not words. So we’ll use a technique called one-hot encoding. It’s like giving each category its own switch – either it’s on (1) or off (0). It creates a binary column for each category within a categorical feature.

This technique is essential because many machine learning algorithms cannot work directly with categorical data.

One-Hot Encoding Example

Let's apply one-hot encoding to the 'Education' , 'City' and 'Gender' columns in our dataset. We'll use the get_dummies function from pandas for this purpose:

## Perform one-hot encoding
df_encoded = pd.get_dummies(df, columns=['Education', 'City', 'Gender'])

After one-hot encoding, our dataset will now have binary columns for each category within 'Education' 'City' and 'Gender'. This transformation allows machine learning models to understand and utilize these categorical variables effectively.

Code Output

As you can see in the image above, the categorical columns are now being transformed into a binary matrix.

Feature Scaling

Feature scaling is crucial when working with numerical features that have different scales. Scaling ensures that all features contribute equally to the model, preventing any one feature from dominating the others.

In our data, ‘Age’ and ‘Income’ might have different scales. ‘Age’ might go from 0 to 100, while ‘Income’ could go from 20,000 to 200,000. Feature scaling makes them play nicely together by putting them on the same scale.

Feature Scaling Example

In our dataset, 'Age' and 'Income' have different scales. To scale these features, we'll use the StandardScaler from the sklearn.preprocessing module:

from sklearn.preprocessing import StandardScaler
# Initialize the StandardScaler
scaler = StandardScaler()
# Fit and transform the selected columns
df_encoded[['Age', 'Income']] = scaler.fit_transform(df_encoded[['Age', 'Income']])

Code Output

From the code output above, now we can see that the 'Age' and 'Income' have been scaled to have a mean of 0 and a standard deviation of 1. This makes them compatible for modeling and helps algorithms that rely on distances or gradients perform better.

Feature Creation

Feature creation is one of the most pivotal techniques in Feature Engineering. It involves generating new features (columns) from existing ones to provide additional information to the model. It can help uncover complex relationships and patterns within the data.

Let’s say we mix ‘Age’ and ‘Income’ to create a new feature called ‘Income per Age’. This new feature might help our model understand how money and age are related.

Feature Creation Example

In our dataset, we can create a new feature, 'Income per Age,' to capture the relationship between income and age. This can be a useful feature for certain prediction tasks:

# Create a new feature 'Income per Age'
df_encoded['Income per Age'] = df_encoded['Income'] / df_encoded['Age']

Code Output

The 'Income per Age' feature provides a measure of income relative to age, which could be valuable in various modeling scenarios. With this an additional column has been created.

Feature Selection

Feature selection is the process of choosing the most relevant features for a given modeling task. Reducing the number of features can improve model performance, reduce overfitting, and speed up training. It makes your model’s job easier because it doesn’t have to deal with unnecessary things.

Feature Selection Example

Let's assume we want to select the most relevant features for predicting 'Productivity.' We can use feature selection techniques like Recursive Feature Elimination (RFE) with a linear regression model:

from sklearn.feature_selection import RFE
from sklearn.linear_model import LinearRegression

# Separate the target variable and the features
X = df_encoded.drop('Productivity', axis=1)
y = df_encoded['Productivity']

# Initialize the linear regression model
model = LinearRegression()

# Perform RFE with cross-validation
rfe = RFE(model, n_features_to_select=3)  # Select the top 3 features
fit = rfe.fit(X, y)  # Corrected: use X instead of x

# Print the selected features
selected_features = X.columns[fit.support_]
print('Selected Features:', selected_features)

From our data, we used RFE to select the top 3 features that are most relevant for predicting 'Productivity.' This reduces the dimensionality of the data and can improve model performance.

Code Output

This output received indicates that, according to the Recursive Feature Elimination (RFE) process, the columns: 'Income', 'City_Miami', and 'Gender_Male' are considered the most important or informative for predicting the target variable 'Productivity' using a linear regression model.

Let's break down the interpretation:

1. Income: The 'Income' feature is likely identified as an important predictor. It suggests that, according to the RFE process, changes in income have a significant impact on predicting 'Productivity' based on the given linear regression model.
2. City_Miami: The presence of 'City_Miami' as an important feature suggests that, in the context of our data, whether an individual is from Miami or not contributes significantly to predicting 'Productivity.'
3. Gender_Male: Similarly, the 'Gender_Male' feature is considered important. This implies that, according to the model, the gender of an individual, specifically being male, is informative for predicting 'Productivity.'

It's important to note that the interpretation of "importance" here is specific to the linear regression model used in combination with the RFE feature selection technique. RFE ranks features based on their contribution to the model's performance in predicting the target variable.

Binning and Bucketing

Binning or bucketing involves grouping continuous numerical data into discrete intervals. It can help capture non-linear relationships and make modeling more robust.

Let’s consider a practical example. Based on our dummy data, Instead of treating the "Age" column as a continuous variable, you decide to bin it into categories like "Young," "Mid-career," and "Senior" to better understand how age relates to productivity.

After your analysis, you might discover that employees in the "Mid-career" category tend to have the highest productivity levels. This doesn't mean that age directly causes productivity, but it suggests a potential relationship that could be influenced by factors such as experience, skill development, and familiarity with the job.

By binning age into categories, you've made the age-productivity relationship more interpretable, which can provide your model with more detailed information.

Binning Example

Let's say we want to create bins for the 'Age' feature. We can use the cut function from pandas to define bin boundaries:

# Create bins and labels for the single column
bins = [-float('inf'), -0.5, 0.5, float('inf')]  # Adjust the bin edges as needed
labels = ['Young', 'Mid_Career', 'Senior']

# Bin the Age column
df_encoded['Binned_Column'] = pd.cut(df_encoded['Age'], bins=bins, labels=labels, right=False)

# Print the updated DataFrame
df_encoded

Binning 'Age' into discrete intervals allows the model to consider the age groups as a categorical feature, capturing non-linear relationships.

Code Output

As you can see above, the 'Age' column has been transformed into a binned column, which further enhances the information given to the model.

Conclusion

Feature engineering techniques enables data scientists and machine learning practitioners to create more informative and relevant features for modeling.

In this tutorial, we explored various feature engineering techniques, including one-hot encoding, feature scaling, feature creation, feature selection, and binning.

Experiment with these techniques to enhance your data analysis and modeling capabilities. Remember that the choice of feature engineering techniques should be driven by the specific requirements of your project and the nature of your data.

There’s an important thing to remember though, especially if your customer is some type of small business which deals with the public (such as a restaurant, a bakery, and so on). That thing to remember is that the customer probably doesn’t care about HTML, CSS, or JavaScript. They care about whether the website performs well in search and whether customers come into their business as a result.

This means that you need to build the website with search engine optimization (SEO) in mind. One of the bigger developments recently in the area of SEO is Google’s increasing use of “structured data” in its analysis of websites. This trend made me decide to write this article on the use of structured data in your various web projects.

I’m going to divide this discussion up into four sections. The areas I’m going to delve into include:

1. What is structured data and why does Google care about it?

2. How to use structured data in a website.

3. How to test your structured data and monitor for errors after the site launches.

4. The need to keep your structured data up to date after the website launches.

This is an important discussion to have. I find that many, many, many, many……(many) people who hold themselves out as web developers don’t actually know much (if anything) about SEO.

I also find that many people who hold themselves out as assisting with SEO don’t actually know anything about web development (and often can barely code at all).

Someone who can actually code, and who understands what the search engines are looking for, can provide a great deal of value to their customers. This value, in turn, helps you to make more money as a developer. In other words, pairing an understanding of SEO with your newly learned web development skills can help you go from looking like this:

To looking like this:

So let’s get to it and discuss why structured data is important to a website’s SEO.

What is structured data and why does Google care about it?

Structured data is form of markup you can apply to your website’s content. This markup allows you to provide information to search engines about your web pages and the information they contain.

This markup is important because search engines, while getting better at understanding natural language, can struggle with understanding the wording or other content contained within a web page.

For example, if someone is searching for “professional to help with investing,” search engines may struggle to distinguish between sites which belong to investment managers and sites which discuss how to pick an investment manager in general (this is a very generalized example).

By using structured data, you can help Google know that your website actually belongs to an investment manager.

Another purpose of structured data is that it helps search engines identify who is who on the web. Suppose, for example, you're doing a website for a political candidate. The candidate, in addition to an official campaign website, has an official Facebook page for the campaign.

For obvious reasons, there may be people who start false Facebook pages about the candidate. By including certain structured data in the website, you can create a relational link between the official Facebook page and the campaign’s website. This helps the search engines to know which Facebook page is legit and which one is bogus.

Google has been working to identify who is who on the web for roughly a decade. This started with their now defunct social network, Google+ (you know…..that social network that you may have tried but none of your friends were on).

Back in the early part of this decade, if a website included a link to a Google+ profile, and the link included the “rel=author” attribute, the link informed Google that the website belonged to the holder of the Google+ profile. This was something Google intended to ratchet up in search, as one of their executives explained in this 2013 video:

Google abandoned this approach, mainly due to the struggles of Google+, in August of 2014. Since that time, Google has been increasing its emphasis on structured data as a way of annotating information in search results and identifying who is who on the web.

So, in short, structured data is something you should be including to add value to any website you build for your clients.

How to use structured data in a website

Structured data can be used in a number of ways. In addition to using it to help identify the individual or entity operating the site, you can use it to help Google better understand a page’s content.

If, for example, you’ve built a website for a bakery, then there are types of markup you can use to highlight the business’ good reviews, to highlight upcoming events, and so on. This markup can lead to highlights in search results which, in turn, will make your customer (the bakery owner) happy.

Let’s look at a few examples of what this looks like in real life, using a real estate agent website which I recently built (I'm including the link in case you want to take a look at the code).

The realtor I built the website for focuses her business on dealing with investors. The website includes structured data which informs Google that the site belongs to an actual real estate agent. When I perform a Google search for “Dayton realtor for investors,” the top three organic results I receive are as shown in this photo:

The first result is the website I built. The latter two are websites which do not belong to actual real estate agents, even though that is what I was clearly looking for with my search term. In fact, only one other realtor website even appears on the first page of the search results. Now, I’m not saying that this is entirely due to the structured data, but it certainly helps.

The markup used for structured data is generated/governed by Schema.org. When you’re marking up a site, an individual page, an event, or a product, it’s important to use as much markup as is reasonably possible in order to provide the search engines with relevant information.

Schema.org's website often provides examples of what your markup should look like. The start of the markup that I used for the realtor site involved informing Google that the site belonged to a real estate agent by placing the following inside of a

:

<itemscope itemtype=http://schema.org/RealEstateAgent>

This tells the search engines that I am relying on schema’s markup to identify and describe the site and that the site belongs to a real estate agent, as defined by schema here.

I was also able to use structured data to tell Google that this realtor has good online reviews and that they work on commission. This, in turn, is now showing up in search results.

For example, the page at the following link is on the first page of search when I look for Dayton apartments for sale. (Again, I'm including this link in case you want to look at the site's source code).

Notice that the search results include the fact that this is a five-star rated realtor and that the professional works on commission? In other words, the use of structured data helps the site to stand out more in the search results. This information was added to the site with the following markup:

<div itemprop="aggregateRating" itemscope itemtype="http://schema.org/AggregateRating">
Rated <span itemprop="ratingValue">Actual Rating of Realtor</span> out of <span itemprop="bestRating">Highest Possible Rating of Realtor</span> by <span itemprop="ratingCount">Number of Total Reviews</span> clients at <a href="URL of Website Where Reviews Are Located" target="_blank" rel="noopener">Name of Website Where Reviews Are Located</a>
Fee Structure: <span itemprop="priceRange">Commission</span>
</div>

Figuring out what structured data to use in a site, or on a particular page, can be difficult. Fortunately, Google gives a few tools that help with this. Let’s look at those tools in the following section of this article.

Testing your structured data and monitoring it after your site launches

The first step in adding structured data to your content is to figure out the category it falls into. You can do this by researching the Schema.org website, looking at the data from other websites, or a combination of both. Once you find the category you fall into, the rest is pretty easy. Let’s stick with the real estate agent example from above.

The first step is to add the realtor markup, from Schema, the site. Then enter your url into Google’s structured data testing tool. The tool will tell you what structured data has been found on your site, what errors you may have, and what structured data is “suggested” for the category you’ve selected. Once you start using this tool, you’ll find that making sure you have the right information in the website becomes fairly simple and straightforward.

Also, I heavily rely on the examples which Schema provides for various categories and data types. Using Google’s testing tool can help you to make sure that you have the correct data in your site from the get go.

Another important tool, in monitoring for errors, is Google Search Console. This is another developer tool, from Google, that will let you know when structured data errors appear on your website after launch. This is an incredibly useful tool and if you’re supporting your client’s website on an ongoing basis, after launch, then you need to be using it to monitor things.

The need to keep your structured data up to date after a website launches

It is important to understand that you may need to go back and edit a site’s older structured data after a site launches. This is because, like many other things, the standards for structured data change over time. As an example, I build and maintain websites for law firms as part of my primary business. Under the prior structured data standards, these websites were marked up with the following:

In later revisions to the structured data standards, however, the “Attorney” classification was deprecated and changed to “LegalService” – as such, I had to change the markup on each website I manage. I tell you this because it’s important to realize that these standards change somewhat often. It’s important that you keep up with the changes.

Conclusion

Web results are becoming increasingly rich in that they provide more information than just a link to a website. It’s important for your clients that you markup your pages accordingly. Doing so is important to your SEO efforts and to providing value to your customers. This is why it’s important to include structured data in your projects.

About Me

I am a web developer who primarily provides various types of services to law firms. I am also a co-founder of Modern Website Design. I enjoy writing on issues which help freelance developers and small businesses to grow their revenue.