Do you want to access your database, control your system, or execute some code from another website? An API can do all of this for you, and they’re surprisingly easy to set up.

An API is a URL that you can perform GET, PUT, POST, and DELETE requests on to control another service. If you make one yourself, then you can build these APIs to do whatever you want behind the scenes. Common uses are providing database control, performing actions on third-party APIs (API-ception), or controlling another service.

Why use an API?

You may ask why we need an API when we can access the database directly or run the code on the website. There are a few massive advantages to APIs over running the code in your website.

Hide your access keys and tokens

This is possibly the most important reason to use an API. If you are accessing a database, then you’re going to need the database details as well as user and access token/key data.

If you access the database from the website, then you’re going to have all of these details in the source code of your site. This is really bad practice, as anyone can look into the source control and steal your details. This doesn’t sound too bad, but what if these are your AWS or Google Cloud Compute credentials? The people accessing your site could then use these to run whatever they want on your account, leaving you with a huge bill.

Running these processes from behind an API means no one can see any of the private details — they can’t steal them to use in their own projects. If you store your website code in GitHub or another public source control, then it also means that they aren’t visible there, either.

Run the code elsewhere

What if you aren’t using any other services and aren’t using any secret keys? If you are running a large or complex bit of code, or if you don’t want anyone else reading your code and discovering how it works, then you can still use an API.

Control who has access

_[Locked, closed green door](https://www.pexels.com/photo/door-green-closed-lock-4291/" rel="noopener" target="blank" title=").

Providing an API also allows you to restrict who is able to access the database or run the code. You can do this by requiring an API key. This key is used to identify the user making the request, and then allowing or rejecting the request.

This can be used to allow only a few people to access the service, or even to create a tier system. This is how a lot of paid APIs work. Everyone is given free but limited access, and then you allow payment for access to superior parts of the service or just a higher rate of requests.

Building the API

Now that we know some reasons why we might want to create an API, let’s do just that. We’re going to use API Gateway and AWS Lambdas, because it’s simpler than running a server. Make sure you have an AWS account and are logged in.

Setting up an API Gateway

We’ll start by opening the API Gateway service and clicking “Get Started”. On the next page, we need to select the “New API” option. Then we’ll give our API a name and description, and click “Create API”.

Clicking “Create API” will get us into the configuration page for the API.

The first thing we need to do is to add a resource onto the API. Using resources allows us to group similar API calls together using nested slashes. We are going to create an API that we can use to make recommendations on what to watch. Therefore we can have /tv-shows and /movies as two base methods.

Click the “Actions” dropdown and select “Create Resource”. Name your resources, making sure that they are both in the “/” path.

We want users to be able to go to “/movies/horror”or “/tv-shows/comedy”, and we can do this by adding path parameters. These are variables that we can access inside the API. To create one of these, we need to set the resource to {resourceName} as shown below. This can be done for “tv-shows”and “movies”.

Now that we have length and genre, we can create methods for getting and adding data to a table. Select one of the {genre} resources, click “Actions”, and then _“Create Method”._ This will create a small grey box below the resource which we can click. We are going to start with a GET request, so select that and click the tick button.

This is where we get to decide how to handle the request. We are going to use AWS Lambdas, but we need to create them before we can finish setting up the methods.

Creating the Lambdas

We are able to respond to these API requests using Lambdas, which is great as they only run when we need them to. They are also really easy to create, so that’s what we’ll do now.

In the Lambda console, click “Create function”. Then we can name our first API function movieAPI, set it to run Node 8.10, and “Create new role from template(s)”. We’ll name our new role “tableAPI” and add “Simple Microservice permissions” as the only template.

All code can be found at https://github.com/SamWSoftware/Projects/tree/master/movieAPI

Clicking “Create function” will send us into the Lambda window. Scroll down to the “Function code” section and we’ll change the code. The first thing we’re going to do is to check what request method was used.

exports.handler = async (event) => {
    console.log(event);
    if (event.httpMethod === 'PUT'){
        let response = putMovie(event)
        return done(response);
    } else if (event.httpMethod === 'GET'){
        let response = getMovie(event);
        return done(response);
    }
};

We’re going to start by writing the getMoviefunction. This function will start by getting the genrefrom the path parameters. This is where using path parameters can make this process easy.

const getMovie = event => {
    let genre = event.pathParameters.genre;
    return;
}

With the genre that the user requested, we are going to get a recommended movie for them. I copied these from 25 Top Films From Each Genre and added them to an object with the genre as the key. We can then get the film by getting the value of the genre requested.

const movies = {
    action: 'Desperado (1995)',
    fantasy: 'Inception (2010)',
    ...
    horror: 'Black Swan (2010)'
}

const getMovie = event => {
    let genre = event.pathParameters.genre;
    return movies[genre];
}

This means that the title of the movie is being passed into the done function. This function is used, as API Gateway expects the data to come back in a very specific format. This function turns a string into that required format.

const done = response => {
    return {
        statusCode: '200',
        body: JSON.stringify(response),
        headers: {
            'Content-Type': 'application/json',
            'Access-Control-Allow-Methods': '*',
            'Access-Control-Allow-Origin': '*'
        }
    }
}

We can do a very similar thing for a tv-showsAPI function, reusing most of the code. Just change the function names and the movie suggestions to tv show.

Connecting the Lambdas to API Gateway

Back in API Gateway, we can add our new Lambdas to the methods we created earlier. We need to make sure that “Use Lambda Proxy integration” is selected and that we are pointing at the correct Lambda. Clicking “Save” will ask you for permissions to access this Lambda, to which we can give the “OK”.

Do this for the GET methods on both resources and we can start to test. Selecting the methods should now show a method execution diagram. This sounds complicated but the only bit we need is the “TEST” section.

Clicking “TEST” will open a new section where we can try out the API. There are lots of things you can set here, but the only one we care about is the Path {genre}. We need to set this to the genre we’re requesting. Entering “western” as the genre and hitting the “Test” button gets a response like this:

We got our API working! Now we need to make sure that other people can access it. There are two steps to this.

1. We enable CORS — Select the “{genre}” resource and then click “Actions” and “Enable CORS”. Leave everything as defaults and, when asked, click “Yes, replace existing values”.
2. Deploy our API — Click on “Actions” and _“Deploy API”. Set the deployment stage to “[New Stage]” and then give your stage a name like “production” or “public”._

Once your API has deployed you should get a URL like this. This is the base of your API. You can add /movies/western to access your API.

https://{uniqueCode}.execute-api.eu-west-1.amazonaws.com/production

Your API URL might end up something like:

https://fds1fe31fes476s.execute-api.eu-west-1.amazonaws.com/production/movies/western

That’s all for this article. In the next one we’ll connect this API to Dynamo tables and let users vote on their favourite movies in each genre. You can read that article below.

Building an API with Lambdas and API Gateway — Part 2
In the first part we created an API which passed requests through to a Lambda which returned the top tv show or movie…
medium.com

Movie trailers in 2016 promised viewers so many strong female characters. Jyn Erso. Dory. Harley Quinn. Judy Hopps. Wonder Woman. I felt like this could be the year for gender equality in Hollywood’s biggest films.

I was wrong.

And I don’t make this statement lightly.

As a scientist, I turn to data to answer questions I have about the world. And I’ve got the data to back up my claim. In fact, you can have the data, code, and resulting data visualization that I made trying to better understand this topic. But first, let me tell you how I became so interested.

It all started when I went to see Rogue One: A Star Wars Story. All promotional materials for the movie indicated that Jyn Erso (played by Felicity Jones) was the main character. I mean, just look at the poster.

When your picture is several times larger than everyone else’s, you’re probably the main character.

What I didn’t notice at first was that Jyn is the only woman on that poster.

I went into the movie theater expecting to see men and women fighting side by side. I left feeling certain that I could count every female character from the movie on one hand. While Jyn was the main character, I was profoundly aware that she was often the only woman in any scene.

It felt strangely familiar to have a lead female character be so outnumbered. Then I realized that Jyn and Princess Leia suffered the same inequality 39 years apart. I was overwhelmed with a need to know exactly how female representation in Star Wars movies has changed. But it seemed unfair to compare movies made today with movies made decades ago.

So instead, I decided to look for female equality across the Top 10 Worldwide Highest Grossing Films of 2016. They were:

• Captain America: Civil War
• Finding Dory
• Zootopia
• The Jungle Book
• The Secret Life of Pets
• Batman V. Superman: Dawn of Justice
• Rogue One: A Star Wars Story
• Deadpool
• Fantastic Beasts and Where to Find Them
• Suicide Squad

With so many powerful women in these films, some of them must be gender-equal, right?

The Data

Now that I decided what I wanted to investigate, I needed to figure out how to do it. Similar data exploration projects have focused on dialogue or screen-time equality. Both seemed like good options, but I wanted the ability to report on equality at the movie and character level.

In the end, I decided to explore the movies’ dialogue. This choice gave me the ability to focus on characters with an active role in the story and to cut non-speaking characters from my analysis.

Luckily for me, dedicated movie fans often transcribe a movie’s dialogue and make it freely available online. If I couldn’t find a transcript, I used closed-caption files instead. For those, I re-watched the movie and manually assigned characters to their spoken lines.

This process was a labor of love. It was time consuming, but I have no regrets.

Analysis

Once I had all of the transcripts, I just needed to read the .txt files into R and separate the characters from their lines. For the Rogue One transcript, that process looked like this:

Now that I had a data frame with both Character and Words columns, I had to assign genders to each Character. To remain consistent with my categorizations, I came up with a few simple rules:

1. When possible, assign gender according to the pronouns that other characters use. For example, if a character is referred to by others as “he” or “him”, then he is categorized as “male”.
2. If there is no pronoun used throughout the movie but the character is named or credited (on IMDB), use the gender of the actor or actress. Note that the gender of an actor or actress was assumed based on publicly available information as of January 2017.
3. If no pronoun is used for the character and the character is not named or credited, refer to the closed captions. Sometimes they will identify the character that spoke.
4. If all else fails, make an educated guess based on the character’s voice.

I’ll be the first to say that these methods are not perfect. In fact, here are some caveats:

1. If a male character was voiced by a female actress (or vice versa) and the character was never addressed by other characters using pronouns, he may be incorrectly labelled. (I don’t think this happened, but anything is possible.)
2. Voices that are not associated with a physical embodiment of a character (e.g., the voice of a computer) were categorized according to the gender of their voice actor/actress.
3. I can never really know the gender of any character, but I’m using the cues and information that I have at my disposal.

Again, I am far from infallible, so if you caught a mistake on my part, please let me know.

So now I just needed to count the number of words spoken by each character. Again, I was able to do this in R using the dplyr and stringi packages.

It’s worth noting that I included every speaking character in this analysis. So yes, every stormtrooper who shouts a simple “Wait, stop!” before getting shot is included.

Spoiler Alert: The stormtroopers in Rogue One are all voiced by men.

Data Visualization

I had my data. Unfortunately, tables upon tables of word counts and character names don’t give anyone much insight. Like any good data exploration project, it was time to visualize my results. I had to work through a few iterations before I found the best one.

Scatterplots and bar charts both masked characters with small roles.

A simple bubble chart was better but it became difficult to identify individual characters. It was also challenging to understand movie-level statistics.

Which bubble is which?!

In the end, I decided to learn enough d3.js to make an interactive graphic. Here, each bubble represents a character, and the bubble’s area is scaled based on the number of words spoken. Female and male bubbles can be separated for better insight. The stacked bars below indicate movie-level information.

_Full interactive version [here](https://proquestionasker.github.io/projects/MovieDialogueInteractive/" rel="noopener" target="blank" title=")

Go ahead, check out the full interactive version.

Interested in exploring the raw word-count data for yourself? I’ve made all of the data and code used to generate these visualizations open source. It’s available here:

ProQuestionAsker/2016MovieDialogue
_Contribute to 2016MovieDialogue development by creating an account on GitHub._github.com

Takeaways

Ok, so the analysis is done. I’ve got a fancy (and fun-to-play-with) visualization. What did I find?

I recommend taking a quick second to look at something “a-Dory-ble” before going on, because this post is about to get real depressing real fast.

Aw, so cute. Feeling good?

All right, here we go.

This is a static version of what the visualization for all 10 movies looks like:

(If you’d like to check out the interactive visualization, go here.)

_The interactive version of this visualization can be found [here](https://proquestionasker.github.io/projects/MovieDialogueInteractive/" rel="noopener" target="blank" title=").

There are a couple of things here that I need to point out:

Not one of the top 10 movies of 2016 had a 50% speaking, female cast.

Finding Dory was the closest to this level of equality with 43% female characters. To be equal, the movie would have needed 8 more speaking, female roles.

Rogue One was the worst. Only 9% of its speaking characters were female. Of those 10 characters, 1 was a computer voice, 1 appeared on screen for no more than 5 seconds, and 1 was a CGI cameo that said 1 word.

Only 1 of 2016’s top 10 movies had 50% dialogue by a female character.

Finding Dory comes out on top here too with 53% female dialogue. But, 76% of that dialogue came from Dory alone.

Trailing at the end was The Jungle Book with only 10% of its dialogue spoken by a female character. Keep in mind, this is after casting Scarlett Johansson as the voice of the historically-male snake, Kaa.

We’re gender equal….Trusssssssst in me….

Here’s a few more:

• Finding Dory and Zootopia were the only 2 movies in 2016’s top 10 in which a female character had the most dialogue.
• Female characters were outnumbered in Captain America: Civil War’s final battle 5:1. Throughout the movie, they only contributed 16% of the dialogue.
• Batman spoke 2.4 times more than Superman and 6 times more than Wonder Woman in Batman V. Superman.
• 78% of the female-spoken lines in Rogue One came from Jyn Erso.
• While Harley Quinn was a highly advertised character in Suicide Squad, she only spoke 42% as many words as Floyd/Deadshot (played by Will Smith). Notably, Amanda Waller (played by Viola Davis) spoke frequently, totaling just 222 words (16%) short of Deadshot’s word count.

I started this project because I had a feeling that Rogue One’s cast and dialogue were not equally divided between male and female characters. I was shocked (and saddened) to find that almost none of the top 10 movies from last year were gender equal.

We can do better.

Added: If you’re looking for more studies and data explorations like this, check out:

• Inequality in 800 popular films from 2007–2015 (includes gender, race/ethnicity, sexual orientation, and disability)
• This exploration of 2000 randomly selected movie scripts from 1980’s — 2010's
• This research on 200 biggest movies from 2014 & 2015
• Female representations in 2014’s biggest movies
• This Twitter thread about gender equality in 2016’s animated films

TL;DR Version: Women represent (on average) 30–35% of speaking roles across each of these investigations.

Added: Have questions or comments about my methodology or conclusions? Check out my follow-up article featuring the most frequently asked questions.

I analyzed the dialogue in 2016’s biggest movies and it started a lot of conversations.
_A few weeks ago I published a story about my analysis of the dialogue in 2016’s 10 Highest Grossing Films. I am so…_medium.com

If you liked this article and want to see more like it, please click the green heart below and share away on your social media network of choice.

I am currently spending my time working on personal projects and data visualizations like this while I look for a data science job. So, if you have a fun project idea (or a job inquiry) you’d like to discuss with me, please reach out to me on Twitter or by email.

Thank you!