In this article, I’ll show you some exciting facts about American cities, the value of bubble plots in deciding which city to live in, and how to create those plots.

Are you thinking about investing in real estate in 2018? Moving to a new city? When considering these decisions, you need to weigh in different factors like unemployment rate, housing price, the size of the city, safety and so on. Even with all that data and four corresponding bar charts, you’ll still be clueless staring at that table. You’ll try to find the best candidates, but those factors are telling different stories… Sounds like a complex problem.

So, is there a way we can visualize all these factors in 1 chart and compare them ALL? Yes, we can use a bubble plot!

What’s a bubble plot?

A bubble plot is a type of chart that displays more than two dimensions of data (compared to traditional scatter plots). In addition to plotting a dot on an X-Y plane, it uses the size, color, or shape of the point to display more dimensions.

We use unemployment rate as the X-axis, median home price as the Y-axis, and the population of the cities as the size of the dots. This makes a good third dimension. Color is randomly assigned to each city.

The best city in the US to live in is…(wait for it)

Winner: Nashville!

Other recommendations: Austin, Omaha, Milwaukee, Dallas, Minneapolis, Denver and Aurora.

They have low unemployment (and therefore there’s higher chance of finding a job), and low home price, because they are on the lower left side of the plot. What does that mean?

It means you can make your choices based on this plot.

For example, if you consider unemployment rate to be more important and don’t mind the higher home prices, then Honolulu, Oakland, Boston, and San Diego are strong candidates

What about adding safety as another factor?

Sure. Let’s add safety as a fourth factor (the other three factors are still home price, unemployment rate, and population). Instead of randomly assigning a color for a city, we use the color scale for crime (crime rate per 100,000 people). Red means more crime and blue means less.

Does the result change?

It did! If safety is very important for you, then Milwaukee might not be such a great choice among the previous recommendations (even though it’s at the lower left side of the graph).

Now you see the power of a bubble plot: the ability to demonstrate multiple factors in one 2-D plot. If you only have bar charts for those factors, it’s hard for you to identify the cities with an ideal combination of factors. The bubble plot basically created a “visual objective function” for you to optimize a multi-variable decision-making problem.

How do unemployment rate and home price change over time?

We can create an interactive motion chart to add time as a dimension (2013 to 2017) to see how the factors change for these cities over time.

To avoid too much visual information, I didn’t use crime data and used the different colors to represent a few selected cities.

The good news is that the unemployment rate for almost all cities decreased significantly (moving from right to left). But the bad news is that the housing prices are going up pretty fast (especially for San Francisco, San Jose, Los Angles, New York, and Seattle).

Want to create the charts yourself? Here is my code for the bubble plots and the motion chart in R. Have fun playing with the plots :)

################ Bubble Plot ################library(data.table)library(ggplot2)library(ggrepel)

bubble_data <-fread("https://raw.githubusercontent.com/zhendata/Medium_Posts/c007346db1575aca391a6623c87bb5a31a60b365/bubble_plot_merged_city_data.csv",sep=",")

bubble_plot <- ggplot(bubble_data,                aes(x = Unemployment_Rate, y = Home_Price/1000)) +

geom_point(aes(size = Population, fill = Total_Crime),shape=21) +# Create 'Bubble' by assigning size a variable #

scale_fill_continuous(low = "#33FFFF", high ="#FF6699" ) +scale_size_area(max_size = 20)+# Select bubble color scale and bubble maximum size #

geom_text_repel(          aes(label = City),nudge_x = 0,nudge_y = 0.75,size = 6) +# Use geom_text_repel to repel the labels away from each other #

theme_bw()+# Use white background instead of the default grey one #

ggtitle("Best Cities in US to Live in") +labs(x = "Unemployment Rate%", y = "Home Price",       size = "Population",fill="Crime") +theme(plot.title = element_text(size=25, hjust = 0.5),        axis.title=element_text(size=20, face = "bold"),        axis.text=element_text(size=15)) +# Style title and axis #

scale_y_continuous(name="Home Price", breaks = seq(0, 1500, by=250),                       labels=c("0", "250K", "500K", "750K", "1000k",    "1250k", "1500K"))# Make y-axis more readable by replacing scientific number by "K" #

print(bubble_plot)

################# Motion Chart #################library(data.table)library(googleVis)

motion_data <-fread("https://raw.githubusercontent.com/zhendata/Medium_Posts/c007346db1575aca391a6623c87bb5a31a60b365/motion_chart_merged_city_data.csv",sep=",")

motion_chart <- gvisMotionChart(motion_data, idvar = "City", timevar = "Year",xvar = "Unemployment Rate",yvar= "Home Price",sizevar="Population")

plot(motion_chart)# R automatically opens a tab in the browser for you# The flash player needs to be enabled in browser

Click the “⌽” icon to enable Flash

######### Data #########"""The datasets I used are from Zillow (medium housing), FBI’s UCR program, census.gov (population), Bureau of Labor (unemployment). I did some data cleaning and joining for the format I needed in this article, and you can click the links below to download."""bubble_plot_merged_city_data.csv, motion_chart_merged_city_data.csv

Follow me and give me a few claps if you found this helpful!

You can also read my previous articles on data science, real estate, and decision making:

How to analyze seasonality and trends to save money on your apartment lease.
_When I was looking for a new apartment to rent, I started to wonder: is there a data-driven decision making strategy…_medium.freecodecamp.orgHow to Use Data to Predict Rent and Optimize Your Lease Duration So You Can Save Money
_In my last post, we talked about how to pick the best month to sign the lease based on seasonality. Now, how long…_medium.freecodecamp.org

In my last post, we talked about how to pick the best month to sign the lease based on seasonality. Now, how long should you sign the lease for when facing different options like 12-month, 15-month, 18-month or longer? Is there any strategy in selecting the best option to save money?

To analyze this, I modelled 353 cities’ rent data from Zillow (one-bedroom, city-level data). In this article, I will show you how to make time series predictions, and which cities are predicted to increase the most in rent!

First, how does lease duration help you save money?

As shown below, you can save money by signing a longer lease if you predict the rent will increase in your city. If the monthly rent increases $100 in the next year, you’ll save $1,200 by signing a 2-year lease, then renew it year-by-year.

How do you predict if rent will increase?

We observed that rent is an additive time series with a combination of seasonality, trend and some random noise.

Additive model: Y(t) =Seasonality(t) + Trend(t) + Randomness(t)

We can decompose a time series into the right hand side of the equation above by applying R’s stl() function (stl stands for "seasonal and trend decomposition using locally weighted scatterplot smoothing”).

# Decompose the additive time seriesdecomposed_rent <- stl(rent.series, s.window="periodic") #periodic means the seasonality factor is same for every year

# Extract the components from time seriesseasonal   <- decomposed_rent$time.series[,1]trend       <- decomposed_rent$time.series[,2]random     <- decomposed_rent$time.series[,3]

The plot of the decomposed time series of rent verified that the components are additive, where rent = Seasonality + Trend + Randomness

You can simply apply the st() function in R on the time series format of rent data to predict rent in the next 2 years.

# Forecast rent for the next 24 months with 95% Confidence Intervalfore_rent<-stlf(rent.series, s.window="period",h=24, level = 95)

Which cities have the predicted increase of rent?

How to read the plots: The light green band area after 2018 is the 95% Confidence Interval of the rent prediction. The *text in purple tells you how much you can save if you sign a 2-year rent vs 1-year rent, according to the purple rectangular area outlined. I used ggplot2 for all the plots.

1. Bay Area

Sunnyvale’s predicted monthly rent increase is the greatest among all 246 cities I analyzed, which is $165 (comparing 2018–01’s rent to the predicted rent in 2019–01). So signing a 2-year lease in 2018 Jan can save you 16512= $ 1980 on the second year; signing a 18-month lease can save 1656 = $990. Given the seasonality effect in Sunnyvale, you should also try to avoid renewing the lease around July.

2.Denver

3.Southern California

4. Seattle Area

5. Florida

6. Texas

For the 11 cities above, if a 2-year lease isn’t an option, 18-months can still save a lot compared to an yearly updated increasing rent.

Which other cities show a huge leap in rent? I plotted the 20 cities total (including the cities mentioned above) to show you a comparison of rent as well as the increase of rent among more cities.

The length of line segment of each city is the increase of the rent where the red dot is the rent in 2018–01 and the green is the predicted rent in 2019 -01.

From the plot above, Lakewood (Denver Metro in CO) and El Cajon (San Diego Metro in CA)’s rents are not that high among the 20 cities, but the “step” of increase is bigger compared to other cities with similar range of rent.

The cities with rent >$2000 and significant predicted increase are all in CA (Top 4 of the plot). The rent there is already expensive, and they are getting more expensive, faster.

Among the top 20, there are 8 in CA, 6 in FL, 2 in WA, 2 in TX, 1 in NY and 1 in CO.

Are there any cities that don’t show much trend in rent?

For the cities above, there’s no predicted increase. So for cities with very significant seasonality effect like Boston and Wilmington, it doesn’t really matter how long you sign the lease; but which month you sign.

The month with the highest rent in Boston is November, while it’s April in Wilmington.

If you are curious about what are other cities like this, read more about cities with seasonality in my last post!

_Find the R code for time series models and visualization with ggplot2 here._

Give me a few claps and follow me here if you find it helpful!

When I was looking for a new apartment to rent, I started to wonder: is there any seasonality impact? Is there a month when the rent is lowest so I can save money when I start my lease?

To tackle this question, I used Zillow’s public data here. I analyzed their one-bedroom rental data from January, 2011 to September, 2017 for the top 100 US cities ranked by size.

The short answer is YES. You can save from $1000 to $2000 if you pick the right month to start renting in certain cities. By simply fitting a linear regression model using time and month to estimate rent, I found some interesting seasonality patterns for a few cities.

Methodology:

On the high level, rent = trend + seasonality. I fit a linear regression model for each city to breakdown trend and seasonality (using a cycle of 12 months).

Model: estimated rent(for a specific month)=t+t²+m1+m2+m3+…+m12

Variables: t and t² are continuous variables to estimate trend; t is the count of months from the beginning month in a city. I added t² to adjust for quadratic trend, and you’ll see some clear curves in the plot of Philadelphia below.

m1, m2, … , m12 are binary variables (0 or 1) that indicate to which month one data point (rent) belongs. Each rent data point can only be assigned one of the monthly variable (as 1). The rest will be 0.

After fitting the model above for all cities, I counted how many months’ coefficients were statistically significantly higher than the month estimated to have the lowest rent. I considered cities with a count ≥3 to have a potentially large seasonality effect.

Then I examined the the overall model fitting to filter out cities with a lot of noise, and came up with a final list of the six most representative cities.

Now I’m going to show you these cities so you can see the sweetest month for you to start renting. I plotted the simulated rent against the actual rent below. You can see the pure seasonal difference (adjusted by each city’s trend) for each month on the lower right corner. Here’s how to read the plots:

Black line: actual rent data

Green line: simulated rent by regression model given month and year

Green bar plot on the right corner: pure seasonal effect estimated by model

Grey line: estimated trend by regression model

Seasonal gap: highest rent minus lowest rent (the difference estimated between the highest and lowest point from the regression model without trend effect)

Numerical labels: represent the months estimated to have highest (red) and lowest (blue) rent

Six Cities with Significant Seasonality Effect

You’ll definitely save money if you start renting on a “low” month in these cities.

1. Boston

If you start renting in June, you’ll save about $2484 a year (207*12) compared to starting a lease in November. The grey line shows a slight trend in Boston, but it’s not very significant compared to the strong seasonal factor.

2. Minneapolis

There is a slight upward trend, but the seasonality effect is more significant than the trend. Your yearly savings, if renting from December, can be as high as $1896 (158*12). In reality, this number is likely to be slightly lower, because the upward trend tends to shrink the difference a bit.

3. Philadelphia

After the regression model’s adjustment for the curve-shaped trend, the estimated yearly saving on rent is $1404 (117*12). This number is greater during the period with a downward trend: you can see that the distance between January and May’s rent is stretched further prior to 2014. The estimated savings are smaller when the overall rent increased during recent years.

4. Chicago

The overall trend in Chicago is actually the opposite of Philadelphia’s — it went up and then down. But the seasonality effect is still significant after adjusting for trend. The estimated yearly saving is $1248 (104*12). If the downward trend continues, the saving will be greater — the rent distance between November and April is stretched further as plotted in recent years.

5. Columbus

There is a noticeable upward trend in Columbus’s rent, but the seasonality effect is also quite significant. The estimated yearly savings are smaller after adjusting the pure seasonal gap ($89) by the upward trend, so you’d save around $720 (60*12). But you should still consider starting your lease in November and avoiding August.

6. Woodbridge

If you start renting in December, you’ll save about $948 (79*12) a year compared to renting from July. The trend isn’t very significant here, so it’s still seasonality that drives the rent price in Woodbridge.

What about Seattle, the city where I live?

The seasonality effect also exists in Seattle, and it shows significance in the regression model. However, the trend is so big that the seasonality almost doesn’t matter.

Even so, understanding the seasonality for cities like Seattle can be helpful. While you might not able to negotiate the rent down that much in a less busy season, you could ask that the application fee be waived or something like that.

My current apartment waived mine when I started my lease in January — December has the lowest rent, followed by January. But they might not offer this perk in the busiest months with higher rents, like May and June.

Another city where the trend outweighs the seasonality is Omaha.

Knowing your city’s seasonality in rent can help you save thousands if you know the pattern. I did my analysis and plots using R, but you can simply plot your city’s data in Excel if you just want to see if there are any noticeable trend and seasonality. Using open source data to hack your life decisions and save money is actually pretty simple.

Now, how long should you sign your lease?

Say you are offered a few different options for the length of your lease. Usually it’s nine months to 18 months. Do you know what’s the best length to choose when you sign your lease? There is actually another trick to save money when you pick the duration, and I’ll show you the trick and the math behind it in my next post.

Give me a few claps and share this with friends who might find it useful!

_You can find my code here_.