We just published a full course on the freeCodeCamp.org YouTube channel that teaches how to implement various data visualization techniques with D3.js.

Curran Kelleher teaches this course. He is one of the top D3 instructors in the world. He has a Ph.D. in Computer Science, and has taught at universities including MIT. This is the third time Curran has taught this course, and it gets better every time.

Curran originally taught this course in a series of live streams. Now, for the first time, you can watch the entire course in a single video.

In this course, the code is organized in such a way that the rendering logic (e.g. DOM manipulation) is decoupled from the data transformation logic (e.g. transform rows into objects that represent graphical marks). This approach makes it straightforward to adapt the core of the visualization technique to work with various frameworks like React, Vue, Svelte, and others.

Here are the sections covered in this course:

• SVG Fundamentals
• Pseudo Visualizations
• Sol LeWitt in Vanilla JS
• Sol LeWitt in D3
• D3 Selections
• JavaScript Modules & Build Tools
• Productizing a Codebase
• Scatter Plot
• Reusable Charts
• Dynamic Charts
• Charts with Menus
• Quantitative and Categorical
• Closing the Series

Watch the full course below or on the freeCodeCamp.org YouTube channel (20-hour watch).

We'll talk about what D3.js is, how it works, and we'll create some basic visualizations to add transitions, interactions, and zooming.

Table of Contents

• Getting Started What is D3.js? How to Set Up a D3.js Environment

• Selections How to Select Elements in D3 How to Modify Elements in D3

• D3 is Data Driven Data Join in D3 Data Loading in D3

• Scales in D3

• Create a bar chart with d3.js Axis Component in D3 D3 Margin Convention How to Style it With CSS in D3

• Create a World Map with d3.js How to Use Multiple Datasets in D3.js Map with city names Event Handling with D3.js Map with Panning and Zooming Programmatic zooming Adding ToolTips

• Conclusion

Who is this article for?

This article is aimed at developers who already have a basic knowledge of HTML, CSS, SVG and JavaScript who want to learn how to visualize data with D3.js.

This article is suitable for both complete beginners and those who already have some experience working with D3.js.

By the end of this article, you should understand how D3.js works and how to create visualizations with your data.

Getting Started with D3.js

D3.js is a JavaScript library for creating visualizations like charts, maps, and more on the web.

D3.js (also known as D3, short for Data-Driven Documents) is a JavaScript library for producing dynamic, interactive data visualizations in web browsers. It makes use of Scalable Vector Graphics (SVG), HTML5, and Cascading Style Sheets (CSS) standards. – Wikipedia

Unlike many other data visualization libraries that provide ready made charts, D3 gives you lots of creative freedom as you have total control over the visualizations you create. D3 also uses web technologies like HTML, CSS, SVG and JavaScript.

In addition to the fact that D3 uses these familiar technologies, it has several other benefits:

• D3 is extremely fast,

• It encourages code reusability

• It supports large datasets and provides an easy way of loading and transforming data

• It's good for creating visualizations with rich interactions

How to Set Up a D3 Environment

D3 works in all modern browsers, and at the time of writing this article, D3.js is on version 7 (v7).

To use the latest version of D3 you have to link to it on your web page like this:

<script src="https://d3js.org/d3.v7.min.js"></script>

However for the purposes of teaching, all examples in this article are on Codepen, so you can edit the live examples.

How to Select Elements in D3

When you're coding in JavaScript and you need to modify elements on a page, you need to select those elements. D3.js works the same way, and provides us with two methods to select DOM elements:

• d3.select()

• d3.selectAll()

Both of this selector methods will take in any CSS selector and return the element that matches the specified selector. If no element matches the selector it will return an empty selection.

The d3.select() method will select the first element that matches in the DOM (from top to bottom).

d3.select("#d3_p").style("color", "blue");

<div class="d3_example">
    <cite class="d3_example-text">Example output</cite>
<p id="d3_p"> hello world 1</p>
</div>

If there are multiple elements that match the specified selector, d3.select() will match the first one it finds.

<div class="d3_example">
    <cite class="d3_example-text">Example output</cite>
<p id="d3_p-all"> hello world 1</p>
<p id="d3_p-all"> hello world 2</p>
<p id="d3_p-all"> hello world 3</p>
<p id="d3_p-all"> hello world 4</p>
</div>

The d3.selectAll() method works very similarly to d3.select() – but instead it selects ALL elements that match the selector:

d3.selectAll
(".d3_p").style("color", "blue");

<div class="d3_example">
    <cite class="d3_example-text">Example output</cite>
<p class="d3_p"> hello world 1</p>
<p class="d3_p"> hello world 2</p>
<p class="d3_p"> hello world 3</p>
<p class="d3_p"> hello world 4</p>
</div>

How to Modify Elements in D3

After you have selected your DOM elements, D3 provides the following methods to modify them:

| Method   |      Usage      |  Example |  
|---|---|---|
| `.attr()` |  Update selected element attribute | `d3.select("p").attr("name", "fred")` | 
| `..classed()` | Assigns or unassigns the specified CSS class names on the selected elements   |   `d3.select("p").classed("radio", true);` |
| `.style()`  | Updates the style property |    `d3.select("p").style("color", "blue");` |
| `.property()` | Used to set an element property |    `d3.select('input').property('value', 'hello world')` |
| `.text()`  | Updates selected element text content |    `d3.select('h1').text('Learning d3.js')` |
| `.html()` | Sets the inner HTML to the specified value on all selected elements |    `d3.select('div').html('h1>learning d3.js</h1>')` |
| `.append()`  | Appends a new element as the last child of the selected element |    `d3.select("div").append("p")` |
| `.insert()`  | Works the same as the `.append()` method, except you can specify another element to insert before |    `d3.select("div").insert("p", "h1")` |
| `.remove()`  | Removes selected element from the DOM  | `d3.select("div").remove("p")` |

Don't worry if all these doesn't make sense right away – we will soon be using all these methods in our examples.

Each of the above DOM manipulation methods takes in a constant value or a function as a parameter which gives rise to creating Dynamic Properties.

The function takes in two properties: the first is the data which is conventionally called d in d3.js, and the other is the index.

d3.selectAll("circle").attr('cx', ((d, i) => i * 100))

<div class="d3_example">
    <cite class="d3_example-text">Example output</cite>
<svg id="d3_svg_demo1">
  <g transform="translate(50, 40)">
    <circle r="30" />
    <circle r="30" />
    <circle r="30" />
    </g>
</svg>
</div>

As you can see above, within this function we can apply any logic to manipulate the data and output.

D3 is Data Driven

D3.js itself is data-driven, which means it gets its super powers from data. D3 supports different types of data like arrays, CSV, XML, TSV, JSON, and so on.

This data can come from a local file in your working directory or can be fetched from an API.

Data Joins in D3

D3's data join lets us join the specified data to the selected element(s). To create a data join, you can use the .data() method:

let fruits = ['Apple', 'Orange', 'Mango']

d3.selectAll(".d3_fruit").data(fruits).text((d) => d)

<p class="d3_fruit"></p>

<div class="d3_example">
    <cite class="d3_example-text">Example output</cite>
<p class="d3_fruit"></p>
</div>

Let's see what's going on here and why we got only one output instead of three.

So far, we have:

1. 3 data points in our Fruits Array

2. 1 p element in our selection

D3 just assigns the first fruit (Apple) in our array to the only selection p it got and forgets about the rest.

A quick fix for this is to manually create the other 2 p elements and just move on with your life. But most of the time you don't actually know how many items are in your array of data that is fetched from an external API.

To solve this problem, the latest versions of D3 provides us with a .join() method. It appends, removes, and reorders elements as necessary to match the specified data. Let's try it with our previous example to see what happens:

let fruits = ['Apple', 'Orange', 'Mango']

d3.select(".d3_fruit")
    .selectAll("p")
    .data(fruits)
    .join("p") // the join method
        .attr("class", "d3_fruit")
        .text((d) => d)

<div class="d3_fruit"></div>

<div class="d3_example">
    <cite class="d3_example-text">Example output</cite>
<div class="d3_fruits"></div>
</div>

Let's break this down a bit:

1. Select the div wrapper d3_fruit

2. Select all the p elements even when there are no p elements in the div - this returns an empty selection

3. .data(fruits) - Binds the fruits array to the empty selection

4. .join("p") - This methods creates all the p elements for each item in our Array

5. .attr("class", "d3_fruit") - We set a class for each p element that was created

6. .text((d) => d) - Sets the text of each created p based on the fruits Array

Data Loading in D3

We have seen what data is to D3 and how to join data to our selections. But so far we have only used our own self-created data let fruits = ['Apple', 'Orange', 'Mango'].

In a real world scenario this is not usually the case – you sometimes have to fetch data from an API or a local file.

D3 has some methods to load various types of files:

• d3.json

• d3.csv

• d3.xml

• d3.tsv

• d3.text

When using any of these methods, the syntax is generally the same:

// async await
const data = await d3.csv("/path/to/file.csv");
console.log(data);

// or
d3.json("/path/to/file.json").then((data) => {  console.log(data); })

Let's see this in action by loading data from an actual external JSON file.

For this example I have a JSON file that contains all the info about Nigeria and all its states:

const el = d3.select("#d3_svg_demo2");

d3.json("https://raw.githubusercontent.com/iamspruce/intro-d3/main/nigeria-states.json").then(({data}) => {
    el
     .selectAll("p")
     .data(data)
     .join("p")
      .text((d) => d.Name)
});



<div class="d3_example" id="d3_svg_demo2">
    <cite class="d3_example-text">Example output</cite>
    <p></p>
    <p></p>    
    <p></p>    
    <p></p>    
    <p></p>    
    ... + 31 others
</div>

Using the above method you can fetch any data in D3.

Scales in D3

Thus far, you've learned how to load and use data in D3.js. Now we need to learn about Scales. This can be the most confusing part to learn for most folks and it's also the most important concept of D3.

In the last example we just looked at above, we loaded JSON data from an API and for each State in Nigeria we appended the name to a p element. That JSON file also contains the population of each state and some other info.

The population of each state ranges from the lowest at 2 million to the highest at 16 million. To correctly represent that data on a bar chart, for example, you need to create a bar chart with a height of 16000000px.

Just imagining that, you'd probably agree that it would be a very long bar chart. That's where d3.scale comes in.

The d3.scale function takes in data as input and returns a visual value in pixels. d3.scale needs to be set with a domain and a range. The domain sets a LIMIT for the data we are trying to represent visually.

const x_scale = d3.scaleLinear()
    .domain([10, 500])
    .range([2000000, 16000000]);

Let's break this down a bit:

• d3.scaleLinear() - we tell D3 we are going to use the scaleLinear

• .domain([10, 500]) - we set the domain (Limit) from 10 to 500

• .range([2000000, 16000000]) - we set our minimum value to 2 million and maximum to 16 million which means we map out 2 million to 10px and 16 million to 500px

Now if we have a city with a population of about 8000000(half of 15 million) it would map out to a pixel value of 250px(half of 500).

It is important to point out that D3 has various form of Scales. The one you decide to use we be determined by the type of data you are trying to represent.

• When you're working with data that represents dates, use d3.scaleTime

• When you're creating bar charts, use d3.scaleBand

• For other scales, refer to d3.scale

How to Create a Bar Chart with D3.js

Now let's apply everything we've learned to create a real world bar chart with D3.

For this example we are going to continue building from the example code in the data loading section of this tutorial:

const el = d3.select("#d3_svg_demo2");

d3.json("https://raw.githubusercontent.com/iamspruce/intro-d3/main/nigeria-states.json").then(({data}) => {
    el
     .selectAll("p")
     .data(data)
     .join("p")
      .text((d) => d.Name)
});

First let's create the scales for our bar chart:

const width = 960, height = 500;
const x_scale = d3.scaleBand().range([0, width])
const y_scale = d3.scaleLinear().range([height, 0])

What's going on here:

• First we defined our x scale (horizantal scale) with a minimum of 0 and maximum of our SVG width

• Secondly we set our y scale (vertical scale) to range from 0 to our SVG height

Next we need to select our SVG element in the document:

const svg = d3.select("#d3_demo")
    .attr("width", width)
    .attr("height", height)

Here we selected our SVG element and set the height and width to our specified height and width. Next let's fetch the JSON data from our API:

d3.json("https://raw.githubusercontent.com/iamspruce/intro-d3/main/nigeria-states.json").then(({ data }) => {
    data.forEach((d) => (d.Population = +d.info.Population))
})

If this doesn't look familiar please re-read the data loading section. Because of the way our JSON data is structured I destructed { data } from the API.

The fetched data comes in as a string but we need the Population field to be a number. So using the JavaScript + operator we convert each Population field to a number:

data.forEach((d) => (d.Population = +d.info.Population))

Next we need to set the domain of our scales – and now that we have fetched our data we can do that:

x_scale.domain(data.map((d) => d.Name);
y_scale.domain([0, d3.max(data, (d) => d.Population)]);

Let's see what's going on here:

• x_scale.domain(data.map((d) => d.Name) - The x scale is a band scale so we set the domain to the name of states (36 states)

• y_scale.domain([0, d3.max(data, (d) => d.Population)]) - The y scale is a linear scale so we set the minimum value to 0. And rather than setting the maximum value ourselves, we let D3 do that for us by using the d3.max() method.

NOTE: with the d3.max() method we loop through the provided data and always return the maximum value of the specified field (Population in our case).

Lastly we need to add the rectangles so we can see our bar chart:

svg
 .selectAll("rect")
 .data(data)
 .join("rect")
  .attr("class", "bar")
  .attr("x", (d) => x_scale(d.Name))
  .attr("y", (d) => y_scale(d.Population))
  .attr("width", x_scale.bandwidth())
  .attr("height", (d) => height - y_scale(d.Population));

Okay this isn't something new right? If this is still new please re-read the data join section of this tutorial. But there are some things we are seeing for the first time:

• .attr("x", (d) => x_scale(d.Name)) - We set the x (horizontal) position of each rect created according to the generated scale. Same for the y (vertical position .attr("y", (d) => y_scale(d.Population)).

• .attr("width", x_scale.bandwidth()) - here we set the width of each rect. Of course we can set this to any number we like, but using x_scale.bandwidth() D3 automatically sizes the rect for us to match the width of our SVG.

• .attr("height", (d) => height - y_scale(d.Population)) - lastly we set the height of each rect to the SVG height and then subtract the height generated by the y_scale(d.Population), making sure each rect is represented correctly.

Here is the full code put together in one place:

const width = 960, height = 500;

const x_scale = d3.scaleBand().range([0, width]).padding(0.1);
const y_scale = d3.scaleLinear().range([height, 0]);

const svg = d3.select("#d3_demo")
    .attr("width", width)
    .attr("height", height);

d3.json("https://raw.githubusercontent.com/iamspruce/intro-d3/main/nigeria-states.json")
    .then(({ data }) => {

     data.forEach((d) => (d.Population = +d.info.Population));

     // Scale the Domain
     x_scale.domain(data.map((d) => d.Name));
     y_scale.domain([0, d3.max(data, (d) => d.Population)]);

     // add the rectangles for the bar chart
     svg
      .selectAll("rect")
      .data(data)
      .join("rect")
      .attr("class", "bar")
      .attr("x", (d) => x_scale(d.Name))
      .attr("y", (d) => y_scale(d.Population))
      .attr("width", x_scale.bandwidth())
      .attr("height", (d) => height - y_scale(d.Population));
    });

And here's the output:

%[https://codepen.io/Spruce_khalifa/pen/porxVVd]

And there you have it, a very basic D3.js bar chart. But if you showed that bar chart to a colleague or friend, they would probably ask you "what's going on here, what are we looking at?" That would lead us to another topic – the Axis.

Axis Component in D3

The axis component renders human-readable reference marks for scales. – D3 docs

To create these human readable reference marks, the d3.axis makes uses the d3.scale function to determine the number of ticks to generate.

To create different orientations for our axis, D3 provides four methods:

• d3.axisTop

• d3.axisBottom

• d3.axisLeft

• d3.axisRight

Let's see an example of these:

let svg = d3.select("#d3_demo8").attr('width', 200).attr('height', 200)
let scale = d3.scaleLinear().domain([0, 100]).range([0, 200]);


let bottom_axis = d3.axisBottom(scale);

svg.append("g").call(bottom_axis);

<svg id="d3_demo">
</svg>

<div class="d3_example d3_no_pad" id="d3_svg_demo2">
    <cite class="d3_example-text">Example output</cite>
<svg id="d3_demo8"></svg>
</div>

To make all this work, you only need to pass in your existing d3.scale function. Let's apply this to our previous example.

The first thing we need to do is to set up the D3 margin convention.

D3 Margin Convention

The margin convention is just a way of adding margins to our graphics so as to have space to add our Axis.

To create the margin, first create an object with a property for each of the four sides:

const margin = { top: 20, right: 30, bottom: 55, left: 70 }

Then you need to define the width and height for our SVG. For a responsive graphic we set the width to the document body:

const width = document.querySelector("body").clientWidth;
const height = 500;

Next we need to apply this width as a view box to our SVG element:

const svg = d3.select("#d3_demo").attr("viewBox", [0, 0, width, height])

Next we need to set the x_scale and y_scale to work with our new margins:

const x_scale = d3
    .scaleBand()
    .range([margin.left, width - margin.right])
    .padding(0.1);

const y_scale = d3.scaleLinear()
    .range([height - margin.bottom, margin.top]);

Next let's define our left and bottom axis – remember we only need to pass in our existing scale (the ones above):

let x_axis = d3.axisBottom(x_scale);

let y_axis = d3.axisLeft(y_scale);

Everything else is the same as our previous example except the last part where we add the Axis:

// append x axis
svg
 .append("g")
  .attr("transform", `translate(0,${height - margin.bottom})`)
  .call(x_axis)
  .selectAll("text") // everything from this point is optional
  .style("text-anchor", "end")
  .attr("dx", "-.8em")
  .attr("dy", ".15em")
  .attr("transform", "rotate(-65)");

// add y axis
svg
 .append("g")
  .attr("transform", `translate(${margin.left},0)`)
  .call(y_axis);

You can view the output and full code on Codepen.

How to Style it With CSS in D3

You'll notice that our bar chart is green in color – how come? Well, we added a class of bar to each bar in the chart:

.attr("class", "bar")

We can use that class to style our bar chart with CSS:

.bar {
  fill: green;
}

How to Create a World Map With D3.js

One of the things I personally love about D3 is its ability to handle geographic data. Unlike our former examples, which used JSON data format maps, now we'll use a special form of JSON data called GeoJSON.

You can find the GeoJSON data we are going to use here.

Like in our other examples, let's first select our SVG element in the document and also set up the margin convention:

const margin = { top: 5, right: 5, bottom: 5, left: 5 },
    width = document.querySelector("body").clientWidth,
    height = 500;

const svg = d3.select("#d3_demo").attr("viewBox", [0, 0, width, height]);

<body>
<svg id="d3_demo"></svg>
</body>

Next, to generate our map we'll need a projection to render spherical coordinates (in our data file) and a Path Generator to convert the projected coordinates to a SVG path which is then rendered on the screen:

let projection = d3.geoEquirectangular().center([0, 0]);

D3 provides a lot of projections (I only used this one because I like it). Now that we have chosen our projection, let's convert it to an SVG path. D3 handles the conversion for us when we use the d3.geoPath() method. This method takes in a projection (the one we defined above):

const pathGenerator = d3.geoPath().projection(projection);

We don't want to draw the map directly on the SVG because we are going to be adding animations and zooming later on. So we append a g element to the selected SVG:

let g = svg.append("g");

Then we'll load our data for the map:

d3.json("https://raw.githubusercontent.com/iamspruce/intro-d3/main/data/countries-110m.geojson")
  .then((data) => {
      console.log(data)
  });

If this doesn't make sense I suggest you re-read the data loading section.

Lastly let's use our pathGenerator to generate our paths:

 g.selectAll("path")
    .data(data.features)
    .join("path")
    .attr("d", pathGenerator);

Above we used D3 data join to append a path for each country and then set the d attribute to our pathGenerator:

.attr("d", pathGenerator);

And in case that isn't clear, it's the equivalent of writing this:

.attr('d', (d) => pathGenerator(d))

You can find the final code and live preview on Codepen.

How to Use Multiple Datasets in D3.js

Sometimes you'll want to visualize two datasets from different sources. For example I have a data file that contains the geographic data of Nigeria and another file that contains information about states in Nigeria.

In the data loading section of this tutorial, we only covered loading a single data set. Loading multiple datasets in D3 looks like this:

Promise.all([
    d3.json("https://raw.githubusercontent.com/iamspruce/intro-d3/main/data/nigeria_state_boundaries.geojson"),
    d3.json("https://raw.githubusercontent.com/iamspruce/intro-d3/main/data/nigeria-states.json")
]).then(([geoJSONdata, countryData]) => {
    console.log(geoJSONdata)
    console.log(countryData)
});

By adding all the D3 data loading methods d3.json() inside the Promise.all, the .then() callback will only get called when all the data have finished loading, although if one of the data file fails to load the callback will not be called and would result in an error.

Map with City Names

Now let's use the loading multiple datasets idea to create a map with city names.

For simplicity we are going to leave out the part of creating the map because we already covered that above. Now, we'll only focus on adding the cities names:

Once we have loaded the data we need to format it:

countryData.data.forEach((d) => {
 d.info.Longitude = +d.info.Longitude;
 d.info.Latitude = +d.info.Latitude;
});

Above we converted the longitudes and latitudes. Next we need to fit our map to our container. To do that you'll use the d3.fitSize() method:

projection.fitSize([width, height], geoJSONdata);

Lastly we need to add the city names:

g.selectAll("text")
 .data(countryData.data)
 .join("text")
  .attr("x", (d) => projection([d.info.Longitude, d.info.Latitude])[0])
  .attr("y", (d) => projection([d.info.Longitude, d.info.Latitude])[1])
  .attr("dy", -7)
  .style("fill", "black")
  .attr("text-anchor", "middle")
  .text((d) => d.Name);

And that's it! We have a map with name of cities (I might have added circles too, because I think it's cool). The full code is on Codepen.

Event Handling with D3.js

At the beginning of this tutorial we talked about selections, but one thing we didn't cover was event handling.

In D3 we can add or remove event handlers to or from selected document elements using the .on() method.

The .on() method accepts two arguments:

1. Event type (usually a string)

2. A callback function that is called when our event is fired

Event Types in D3

The D3 .on() event type can be any DOM event type, but the most common events with D3 are:

| Event Type  | Description  |
|---|---|
| zoom  | selection is being panned and zoomed  |
| click  | selection got clicked  |
| mouseover  | mouse pointer moves over a selection  |
| mouseout  | mouse pointer leaves a slection  |

Map with Panning and Zooming

To see how D3 event handling works let's add pan and zoom to our previously created map.

The first thing we need to do is to define the zoom function:

let zooming = d3
  .zoom()
  .scaleExtent([1, 8])
  .on("zoom", (event) => {
   console.log(event)
  })

Then the next thing we need to do is use the d3.zoom() method. We also set the scaleExtent([1,8]). We do this to set the limit of the zoom, otherwise you'll keep zooming to infinity. Now let's add the transformation to our map paths in the callback function:

.on("zoom", (event) => {
  // transform paths when zoomed
  g.selectAll("path").attr("transform", event.transform);

  // transform circles when zoomed
  g.selectAll("circle")
    .attr("transform", event.transform)
    .attr("r", 5 / event.transform.k);

  // transform text when zoomed
  g.selectAll("text")
    .attr("transform", event.transform)
    .style("font-size", `${18 / event.transform.k}`)
    .attr("dy", -7 / event.transform.k);
});

NOTE: The event.transform is a short hand for setting the translate('x','y') and scale (event.transform.k).

Lastly let's call the zooming function on our SVG selection:

svg.call(zooming)

You can find the full code and preview on Codepen.

Programmatic zooming in D3

Turns out in D3 we can control zooming programmatically, this let us create buttons that can be used to control the zoom behavior:

Let's add those buttons to our previous map:

<body>
 <div class="btn-group-vertical" role="group" aria-label="..." id="float-button-group">
  <button class="btn-default" id="zoomIn">
   <svg class="svg-icon" viewBox="0 0 20 20">
    <title>Zoom In</title>
    ...svg icon
    </svg>
  </button>
  <button class="btn-default" id="zoomOut">
   <svg class="svg-icon" viewBox="0 0 20 20">
  <title>Zoom Out</title>
  ...svg icon
    </svg>
  </button>
  <button class="btn-default" id="resetZoom">
   <svg class="svg-icon" viewBox="0 0 20 20">
    <title>Reset Zoom</title>
    ...svg icon
    </svg>
  </button>
</div>

<svg id="d3_demo"></svg>
</body>

The next step is to select those buttons and control the zoom behavior:

d3.select("#zoomIn").on("click", () => {
  svg.transition().call(zooming.scaleBy, 2);
});
d3.select("#zoomOut").on("click", () => {
  svg.transition().call(zooming.scaleBy, 0.5);
});
d3.select("#resetZoom").on("click", () => {
  svg.transition().call(zooming.scaleTo, 1);
});

What is scaleBy and scaleTo? scaleBy multiplies the current scale by our given value (2), while scaleTo sets the scale factor to our given value (1) which resets the zoom.

You can find the preview and full code on Codepen.

How to Add ToolTips in D3

Let's add ToolTips to our map. A tooltip shows more information about an item when the user hovers over that item.

Let's first create the tooltip:

let tooltip = d3
  .select("body")
  .append("div")
  .attr("class", "tooltip")
  .style("opacity", 0);

Next let's add the tooltip when the circle is hovered over, and remove it when the mouse pointer leaves the circle:

g.selectAll("circle")
  ...
  .style("fill", "green")
  .on("mouseover", (event, d) => {
    tooltip.transition().duration(200).style("opacity", 0.9);
    tooltip.html(`<p>Population: ${d.info.Population}</a>` + `<p>Name: ${d.Name}</p>`)
    .style("left", event.pageX + "px")
    .style("top", event.pageY - 28 + "px");
  })
  .on("mouseout", (d) => {
    tooltip.transition().duration(500).style("opacity", 0);
  });

Here's the final code and preview (try hovering on the circles):

Conclusion

Congratulations D3 Ninja! You've made it this far. Hopefully you have learnt the basics of Data Visualization with D3.

Here are some next steps:

• Check out the freeCodeCamp Data Visualization certification

• Check out the docs and more on D3's officail website

If you created something wonderful with this, please feel free to tweet about it and tag me @sprucekhalifa. And don't forget to hit the follow button.

Oh and happy coding!

We've released a comprehensive 17-hour course on data visualization that is perfect for anyone new to the field. The course focusses on visualizing data using D3.js, JavaScript, and React.

Curran Kelleher developed this course. Curran previously created one of the most popular data visualization courses on the internet and now he has completely redone and improved the course. This course comes out of his extensive experience developing data visualizations and teaching new developers.

You will learn how to conceptualize, design, and build interactive data visualizations with Web technologies.

Here is the list of topics covered in this thorough course:

• What is Data Visualization
• Finding Visualizations
• Introduction to Web Technologies
• Let's Make a Face Part I (HTML, CSS, SVG)
• Let's Make a Face Part II (React)
• Let's Make a Face Part III (React & D3)
• Let's Make a Face Part IV (React Components & ES6)
• Let's Make a Face Part V multiple files with ES6 modules
• Let's Make a Face Part VI (Compartmentalizing Complexity)
• Preparing Data for Visualization
• Loading Data with Fetch, Promises, Aync & Await
• Interaction With React
• Parsing CSV Data with D3
• Loading Data in React
• Inputs for Visualization: Data & Tasks
• Iterating Fast with Vega Lite API
• Marks & Channels
• Visualizing Data with React & D3
• Making a Bar Chart with React & D3
• Margins and Axes
• Refactoring a Bar Chart
• Stylized Bar Chart
• Making a Scatter Plot
• Working with Time
• Making a Line Chart
• Preparing Geospatial Data
• Making a World Map with React & D3
• Menus with React
• Scatter Plot with Menus
• Polished Scatter Plot with Menus
• Scatter Plot with Color
• Making a Color Legend
• Interactive Color Legend
• Points on a Map
• Using Size
• Missing Migrants
• Missing Migrants Part II: Aggregation by Month
• Missing Migrants Part III: On a Map
• Multiple Views
• Why Use Modules
• Multiple Views Cleanup
• Multiple Views with Brushing
• React Performance Optimization
• Choropleth Map
• Log Scales
• Making a Choropleth Map
• Basic Line Chart Project
• Line Chart with Marker Lines
• React & D3 Axes with useRef and D3 axis
• Line Chart with Log Scale and Labels
• Line Chart with Multiple Colored Lines
• Line Chart with Voronoi Overlay
• Line Chart with Voronoi Tooltips
• Organization Graph Diagram

Watch the full course on the freeCodeCamp.org YouTube channel (17-hour watch).

You'll learn the basics of data visualization using D3 through this project.

What is D3.js?

For those who are not familiar with D3, D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG and CSS.

D3.js helps you attach your data to DOM (Document Object Model) elements. Then you can use CSS3, HTML, and/or SVG showcase this data. Finally, you can make the data interactive through the use of D3.js data-driven transformations and transitions.

Project Explanation:

Scenario:

There”s a class of Online students interacting with different topics of the subject and gives quiz on those topics.

There are 15 topics and on each topic, we have a numbers of student who have taken quiz and have scored in three categories: Low, Mid and High

For example (Given data ):

Topic: “1”, low: 4, mid:13, high: 18

Topic: “2”, low: 11, mid:12, high: 6

Topic: “3”, low: 12, mid:24, high: 6 and so on.

Note that Topic 1 has4 + 13 + 8 = 35 student has taken the quiz and Topic 2 has 11+12+6 = 29 student and topic 3 has 42 students and so on.

We want to make interactive Bar and Pie charts. For example, a mouse hover on one of the bars will change the pie chart accordingly and vice-versa.

With Interactive combination of bar chart and pie chart, where bar chart shows number of students who have interacted with particular topic(taken the quiz) and pie chart showing classification of that students performance in categories of “low, mid, high”, we can visualize our data and get more data analysis from it.

Hints and Resources:

Step by Step instructions:

D3 Introduction: where you can learn about Selections , Dynamic properties and Transitions in D3.js.

In the main javascript function, write a function to handle Histogram (Bar chart) - Histogram will show the total number of students who has taken the quiz (interacted with that topic) for 15 topics.

• (Here”s the Tutorial where you can learn how to make a bar chart using the D3 JavaScript library. The first tutorial teaches how to make a bare-bones version in HTML, then a more complete chart in Scalable Vector Graphics (SVG), and lastly animated transitions between views.)
• Create SVG for histogram
• Create function for x-axis mapping and add x-axis to histogram SVG
• Create function for y-axis mapping and Create bars for histogram to contain rectangles and topic labels.
• Create the rectangles and topic labels
• Create function to update the bars. This will be used by pie-chart

Write a function to handle pieChart. – pie chart will have three slices – Low , Mid and High to represent scores.

• Tutorial where you can learn how to make a pie chart, then transitions between views and how to create a legend.
• Create svg for pie chart.
• Create function to draw the arcs of the pie slices - pie slices will be Low , Mid and high
• Create a function to compute the pie slice angles.
• Draw the pie slices.
• Create function to update pie-chart. This will be used by histogram.
• Calculate total frequency by segment for all topic.
• Calculate total frequency by state for all segment.

Result of Data Analysis and what can we infer from Visualization:

• Initial pie chart shows aggregate classification of all student score on all topic combined into three categories “low, mid ,high”
• Initial bar chart showing number of students who have interacted on that particular topic
• Any Selected category from pie chart will update bar chart, showing number of students who have interacted on various topics having score belonging to that particular category .
• Screenshots below shows the mouse hover on “Mid” slice and “High” slice of the pie chart respectively and for that particular slice , topic-wise bar charts and number of students.

• Any selected bar chart”s bar will update pie chart showing classification of all students score on that particular topic into three categories Low , Mid and High. Screenshot below shows the mouse hover on Topic 2 and for that particular topic , how many student are in categories Low , Mid and High

Besides teaching all about D3, the course also covers the basics of JavaScript, HTML, CSS, and SVG so you will have all the prerequisite knowledge to create stunning data visualizations.

This comprehensive course covers the following topics:

• Why Visualize Data?
• Inputs for Data Visualization: Data & Tasks
• Intro to Javascript
• Intro to HTML, CSS & SVG
• Intro to D3.js - Let's Make a Face!
• Making a Bar Chart with D3.js and SVG
• Customizing Axes of a Bar Chart with D3.js
• Making a Scatter Plot with D3.js
• Making Line and Area Charts with D3.js
• The General Update Pattern of D3.js
• Marks & Channels in Data Visualization
• Interaction with Unidirectional Data Flow using D3.js
• Making a World Map with D3
• Cheap Tricks for Interaction on a D3.js World Map
• Blank Canvas
• Building a Tree Visualization of World Countries with D3.js
• Color and Size Legends with D3.js
• Choropleth Map with D3.js
• Interactive Filtering on a Choropleth Map
• Using Color in Visualization
• Scatter Plot with Menus
• Circles on a Map
• Line Chart with Multiple Lines
• Melting and Munging Data with JavaScript
• Selecting a Year on a Line Chart

You can watch the full video course on the freeCodeCamp.org YouTube channel (13 hour watch).

I’ve been using D3 quite a bit recently to build an analytics dashboard. The main reason I went with D3 was because I got tired of dealing with the limitations of the various charting libraries and trying to understand their documentation.

I found myself in a situation where I was spending more time trying to figure out whether the library I was using could do what I wanted rather than actually trying to figure out how to do what I wanted.

With D3, there is almost never a case of whether you can do something. It’s always about how you can do something.

Granted, getting to grips with D3’s API is a mammoth challenge but it’s worth the hassle.

In this article, I’m going to try to give you a visual representation of my understanding of D3 after working with it for a while now.

Main Ideas Behind D3

The Ups And Downs Of D3

The thing I love about D3 is that its built on top of the smallest primitives, and you have access to all of those primitives.

If you were to use a typical charting library for React, you’d probably do something like this

import { BarChart } from 'generic-charting-library'

export default class Chart() {
  render() {
    return (
      <BarChart data={data} /
    )
  }
}

The above is perfectly fine for most use cases. However, the moment you want to do anything slightly more complicated like interacting with the bar chart in unique ways (think beyond just displaying a tool tip), my personal experience has been that it turns into a war with the documentation of the library.

First, you have to figure out if it is even possible to do this with the library, and then you have to figure out how to do it.

The first part, in my experience, is the hard bit. It is quite frustrating to look for something that you’re not sure exists. However, with D3, that isn’t the case. Pretty much anything you want to do can be done with D3. It is simply a matter of figuring out how to do it.

The downside of course is that D3’s API and documentation is so vast and expansive, that you end up having disparate pieces of knowledge of how things work. This is also a consequence of how I chose to learn D3, that is by building something with it. When you choose to build something with a technology, you only look up the pieces relevant to what you’re currently building.

Say, for instance, you wanted to build a bar chart. Well, you’re probably to going look up something like how to define and place axes on a web page. Then, you’ll probably look up how to define the actual bars of the chart themselves. These are well defined problems and have straightforward solutions.

I took pretty much the same route and ended up in a frustrating place where I could get things to work but I couldn’t quite understand how it was all coming together.

I’m going to explain my thought process of how I put together all the different pieces.

The Big Picture

The following image is that of a simplistic bar chart (something we’ll try to recreate for this post)

Typical Bar Chart

Now, here’s the same bar chart with the different components being marked out. By components, I mean the different things we will need to worry about when creating the bar chart using D3.

Labelled Bar Chart

1. The X-axis
2. The Y-axis
3. The title
4. The bars (I’m counting all of them as one)
5. The spacing between the ticks on the X axis
6. The actual chart area itself
7. The spacing between the ticks on the Y axis

We’ll explore everything except the spacing between ticks in detail. The spacing between ticks will take care of itself when figure out how to construct the axes.

The Small Things That Make Up The Big Picture

Before we can even start with composing the small pieces that make up the big picture, we need to understand the one element that is a part of every component in the chart above: the SVG element.

The SVG Element

A D3 chart is primarily composed of SVG elements. In the bar chart above, the x-axis, the y-axis, each individual bar are all instances of SVG elements.

I recommend reading this page to get a better understanding of what an SVG is. SVG’s are essentially how you describe 2D graphics on a web page.

The most basic example of an SVG element is the circle element.

See the Pen qGMxzO by Zaid Humayun (@redixhumayun) on CodePen.

Take a look at the codepen above and you should see the definition for the circle SVG inside the HTML file.

I’d advise going through the SVG documentation on MDN (linked above), and familiarizing yourself with it. D3 makes extensive use of SVG’s.

The G in SVG

There is a specific kind of SVG element called a G element. Similar to the way we defined the circle element above, we define this with

<svg viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg">
  <!-- Using g to inherit presentation attributes -->
  <g fill="white" stroke="green" stroke-width="5">
    <circle cx="40" cy="40" r="25" />
    <circle cx="60" cy="60" r="25" /> </g>
</svg>

Think of a g element as similar to a div element that is used as a container in HTML. They are both used to group certain elements.

Read the MDN documentation for g here

If you want to understand why people make use of the g element, read this

The Data We Need

Before we continue any further, let’s quickly create the data we’ll be using. Here is a sample JSON file we can use to create a bar chart.

[
  {
    "key": "A",
    "value": 20
  },
  {
    "key": "B",
    "value": 40
  },
  {
    "key": "C",
    "value": 80
  },
  {
    "key": "D",
    "value": 55
  },
  {
    "key": "E",
    "value": 70
  }
]

The Chart Area

Let’s start off with creating a simple chart area first. The way to create a chart area is to set up a basic SVG element and then assign a viewBox attribute to it.

Just ignore what the viewBox attribute is for now. It isn’t relevant to the discussion of this post.

<svg class="chart" viewBox="0 0 800 600">

You won’t see anything on the screen yet because the chart is transparent at this point. If you use the browser inspector, however, you will see the SVG element.

We’ll also define some dimensions for our chart area like the height, width and margins.

const height = 600
const width = 800
const margin = { top: 15, right: 30, bottom: 15, left: 30 }

Now that we’ve defined the dimensions we need to actually create the area for our chart in the DOM. To do that, we need to use something called d3.select

Think of this as exactly the same as the set of document.getElementBy[X]commands the DOM offers natively.

When you use something like d3.select('.chart'), you are asking D3 to select an element with a class named chart.

Note that we’re saving the selection inside of a variable. This will be important later.

When you select something with d3.select, D3 allows you to use method chaining to alter attributes like width and height, like I've done here.

const chart = d3.select(".chart")  
                .attr("width", width)  
                .attr("height", height)

What we’ll end up with is something like the following image

Chart Display With Margins

Don’t worry about the margins for now. We’ll take care of that later.

Defining The Axes

Now, we start with the meaty part of D3: creating and placing our axes.

Before we can start, we need to understand something fundamental about the way D3 axes work: they are essentially a mapping from one set of values to another set of values.

The two sets of values are called domain and range. D3’s mapping works from the domain onto the range.

I have defined two really simple number lines to illustrate the domain and range. The range is the exact same as the domain with double the number of markings.

In this example its really easy to see how the domain can be mapped to the range. You just need to multiply the value by 2 since the range has double the number of ticks and has the same starting tick value of 0.

I’ve drawn two dashed lines to show the following mappings

2 -> 4
5.5 -> 11

Now, D3 is not limited only to having real numbers (or even just numbers) to define scales. You can even use characters to define your scales.

The Y Scale

We’ll start with the Y scale.

D3 has different kinds of scales but the one we’ll be using is called the linear scale.

To define the scale we need two things: the domain and the range.

We’ll use a simple, stupid rule to define our domain. We’ll assume that the minimum value we can have for one of our categories is 0 and the max value is 100. No negative numbers. The domain then becomes [0, 100]

const y = d3.scaleLinear()            
            .domain([0, 100])            
            .range([height - margin.bottom, margin.top])

One thing we need to examine here is the range. It took me a little while to understand why the range seems to be in “reverse”. My initial thought was that the range should be [margin.top, height - margin.bottom]. But, we want our Y axis for the chart to start at the bottom and moves vertically upwards.

We’ll consider the following two scenarios in a subsequent diagram to examine this.

1. .range([height - margin.bottom, margin.top])
2. .range([margin.top, height - margin.bottom])

The important difference between the two scenarios is that in the first scenario we are treating the value of height as our ‘zero’ value. In the second scenario, we are treating the margin.top value as our 'zero' value.

One thing to remember before we proceed further: the point of origin of every SVG coordinate system is at the top left corner.

Interpreted another way, the bottom of the Y-axis is our ‘zero’ value in the first scenario and the top of the Y-axis is our ‘zero’ value in the second scenario.

The Two Scenarios Represented Visually

In the image above, scenario 1 is on the left and scenario 2 is on the right. You can see the direction of movement for the domain in each image.

In scenario 1, the domain grows upwards from the bottom, which is what we want. In scenario 2, the domain grows downwards from the top, which is what we don’t want.

I appreciate that I might have made things more confusing for those of you who managed to grab the above intuitively but this is something that took me a while to figure out. If you understand intuitively, don’t worry about the above. If you still don’t get it, you will by the end of this post.

The X Scale

The X scale is a little easier to figure out. We need the X scale to grow from left to right keeping in mind the width of our chart area and also the margins on the left and right.

The domain on this scale is a little more confusing though because we aren’t dealing with numbers anymore. We are dealing with the letters of our categories instead.

To figure out how to construct this scale, we first need to understand something called the ordinal scale. The quickest way to understand the ordinal scale is to consider the differences between the linear and ordinal scales.

The Linear And Ordinal Scale

In the image above, you can see a poor drawing of the two scales. The important difference to note is that the linear scale is a continuous scale and the ordinal scale is a discrete scale.

In the example of the linear scale, if you were to provide a value of 5.5, it would be mapped to the midway point between 5 and 6. However, if you were to provide a value of a letter somewhere between C and D (which doesn’t exist), D3 would have no idea how to map it. As far as D3 is concerned, there is no way to map that value because you have stated that all those values are discrete. That is, there are no connecting values in between.

Now, let’s construct the X axis.

function getKeys(array) {
  return array.map(arrObj = {
    return arrObj.category;
  });
}
const keys = getKeys(data)

const x = d3.scaleOrdinal()
            .domain([...keys])
            .range([margin.left, width - margin.right])

If you’re wondering about the function in there and the variable keys, that is to extract all the categories present in our data and provide it to the domain function as an array.

I could just as easily have written .domain(['A', 'B', 'C', 'D', 'E']) but then I would have had to manually update that every time my data changed.

The range, as I have already mentioned, needs to grow from left to right. So, we leave out the margin on the left, move the length of the width and leave out the margin on the right.

Creating The Actual Axes

Now, we have the chart area and the scales defined, we need to set up the axes themselves. Here is how we do that.

const xAxis = d3.axisBottom(x)

Here, we are creating a function called xAxis which uses the d3.axisBottomfunction with our x scale provided as a parameter.

To actually display the X-axis on our chart, we need to do the following

chart.append('g')
      .attr('transform', `translate(0, ${height})`)
      .call(xAxis)

Two things to examine here.

We’re appending a g element to our chart. We discussed the g element in an earlier section. We then apply a transform to our g element. This transform is something that comes up in D3 all the time.

SVG’s have what are called transform functions. There are multiple kinds of transform functions, but the one we care about here is translate. Translateaccepts two parameters an x and y co-ordinate. This signifies how many units of pixels to move the g element either in the X or the Y direction.

You can read more about transforms here.

The two parameters we provide to the translate function are 0 and height. Remember that the point of origin of our SVG chart is at the top left corner. Since, we already know this is a horizontal axis that begins at the point of origin, we need to move it vertically down by height number of units.

If you didn’t provide the transform attribute, the X-axis would be situated at the top of your chart.

The last part of the method chain is a call function where the xAxis is provided as a parameter. This is probably the most confusing aspect so far because of the poor choice of terminology.

We’ll examine just these two lines first.

.append('g')
.attr('transform', `translate(0, ${height})`)

What you need to understand is that when you do something like chart.append('g'), this appends a g element onto the chart element, selects the g element and then returns it. You can test this by doing the following

const test = chart.append('g')      
            .attr('transform', `translate(0, ${height})`)          
            .call(xAxis)console.log(test)

When the result of the log shows up, you’ll see a g element under a Selection object. This is actually what enables us to do method chaining on the append method. Since it returns the g element, we can transform as part of the same method chain.

Let’s go to the last line now

.call(xAxis)

Here’s what D3’s documentation says about call

Invokes the specified function exactly once, passing in this selection along with any optional arguments. Returns this selection.

So, we know we utilize call as a function and we have to pass it a function as a parameter. We know this because the documentation says, it invokes the specific function exactly once. Now, the other thing to realize is that xAxis is also a function. You can verify this again by logging xAxis.

But, if xAxis is also a function then that needs a parameter passed to it as well. Read the documentation for call again and you'll notice it says "passes in this selection...". This means that the xAxis function is being implicitly called with the g selection returned from calling chart.append('g')

Having to explain how call works is precisely why I don't like it. There's too much implicitly happening that just seems like black magic.

If you’re still confused about how call works hopefully the following graphic clears it up for you.

How Call Works

Creating the Y axis now that we know how the X axis works is far simpler. We use the same principles but swap out axisBottom for axisLeft and change the translate function slightly.

const yAxis = d3.axisLeft(y);

chart
  .append("g")
  .attr("transform", `translate(${margin.left}, ${margin.bottom})`)
  .call(yAxis);

You’ll notice that the transform attribute has a translate function where the y attribute is set to margin.bottom. If you go back to the range we set for the y scale, you'll notice we set it to height - margin.bottom.

When we call D3’s axisBottom function, D3 will place this at height - margin.bottom, but the bottom of the chart is actually at height, so we add the margin.bottom offset.

Placing The Bars

This is the most visually important part of the chart because this is where the user actually gets to see the data.

First, let me just show you the code that will create the bars for us and then step through it.

chart.selectAll('rect')
    .data(data)
    .join('rect')
    .attr('x', d => x(d.category))
    .attr('y', d => y(d.value))
    .attr('width', x.bandwidth())
    .attr('height', height - y(d.value))
    .style('fill', 'steelblue')

The first two lines are straightforward. selectAll works the same as selectexcept it returns all possible selections of a specific elemenet.

Calling .data allows you to define the data you want to associate with the DOM elements.

Now, the .join is where the crux of D3 comes in. This is what makes D3 unbelievably powerful to create visualizations with.

If you want to read what Mike Bostock (the creator of D3) has to say on data joins, you can find that here.

What follows is my attempt at explaining what the .join function does in the context of a bar chart.

So, if you go back and look at the data we defined earlier in this post, you’ll notice that it is an array. The reason is because this is the data structure D3 expects.

The .join function then takes every element of the array and constructs a corresponding DOM element with this data point attached.

Note: The _.join_ function used to earlier be separate functions called _.enter_and _.append_. However this syntax is a lot cleaner. Here is the GitHub issue where Mike Bostock first suggested it.

How .join() works visually

Note: In the graphic above, it should read _.join('rect')_ not _.join('bar')_

The graphic above illustrates what is going on when you do a data join. If you take an array of 5 elements and perform a .join('rect') on it, what D3 will do is create a rect SVG element for each of those elements.

Another thing D3 will do is associate each data point from your array to its respective rect element.

const data = [1, 2, 3, 4, 5]

const selection = d3.selectAll('rect')
                    .data(data)
                    .join('rect)

selection.each(function(d, i) {
  console.log(d)
})

//1, 2, 3, 4, 5

The above code snippet shows you how to do the logging of each individual data point to satisfy your own curiosity.

You could, of course, replace the rect above with any other SVG element and you would have the same result.

Great, now we know how to create our bars but we still need to figure out how to place them. Before continuing, I recommend you read this MDN articleabout rects.

One thing that tripped me up a lot about working with D3 intially was trying to figure out how the SVG coordinate system works.

If you want a deeper understanding of how SVG coordinate systems work, check out this article

The graphic above shows you how different measurements would impact the placement of a rect in the SVG coordinate space.

A rect SVG element has four main attributes we’ll be concerned with: x, y, width and height.

You can see how each of them relate to the SVG coordinate space in the image.

Let’s translate the above into code.

chart
  .selectAll("rect")
  .data(data)
  .join("rect")
  .attr("x", d => return x(d.category))
  .attr("y", d => return y(d.value))
  .attr("width", x.bandwidth())
  .attr("height", d => height - y(d.value))
  .style("fill", "steelblue");

Let’s step through the bits of the code after the .join call.

When we set the x and y attributes, we make a call to the respective scales we defined earlier. Remember when we defined the scales, we said that each of them would be functions that could be called with a value to map it from the domain to the range. That's precisely what we're doing here.

Now, to understand the width attribute, we first need to go back to the ordinalScale we defined. D3 has a function associated with each scale called the bandwidth function. This returns the width of each band defined. D3 internally does this by dividing the range equally among each element of the domain.

So, we provided an array of 5 characters as the domain of the x axis and we set the range to [margin.left, width - margin.right], where width = 800and margin = { left: 60, right: 60 }

So, we have

(800 - 60 - 60) / 5 = 136
All units are in pixels.

Now, the height attribute is another thing that tripped me up for a long time because I couldn’t quite figure out why we were doing height - y(d.value)to represent the height of the rect. Surely, it should have just been y(d.value)?

This is answered again by remembering that the SVG coordinate has its point of origin in the top left corner and the +ve Y-axis goes downwards.

Measurements For A Bar in SVG Coordinate System

In the graphic above, I’ve presented my understanding of how the height of the bar is calculated. Again, if the calculation for the height of the bar makes intuitive sense for you, feel free to skip this.

The main thing to notice in the visual is that there is a difference between the axes of the SVG coordinate system and the axes of our chart. The Y axis for the SVG coordinate system is positive downwards but the Y axis for our chart is positive upwards.

This is the reason I’ve drawn two separate sets of axes for both X and Y. Technically, the two Y axes should be superimposed on top of one another but that would make it hard to visually see it. But, you can assume that they are overlayed on top of each other.

When we call the y scale function with y(d.value), we get a value that counts down the +ve Y-axis of the SVG coordinate system starting from the top. The height is shown on the side which is the entire length of the Y axis and then what remains is height - y(d.value), which is the height we are assigning to the bar.

Adding Titles And Labels

Now, we get to the easy bit. It’s only easy because of everything we’ve covered so far!

Similar to how we’ve appended rects to our SVG so far, we can also append text as an SVG element like below:

chart.append('text')
      .attr('x', width / 2)
      .attr('y', margin.top)
      .style('font-size', 32px)
      .style('text-anchor', 'middle')
      .text('Distribution Among Categories')

The text SVG element also has an x and y attribute that work very similarly to how the x and y attributes of the rect work.

You can set different style attributes to the text element and you set the text itself using the .text attribute.

Now, let’s place the Y-axis label

chart
  .append("text")
  .attr("transform", "rotate(-90)")
  .attr("x", -height / 2)
  .attr("y", margin.left / 4)
  .text("Values")

Okay, this one is a little confusing, so let’s step through it.

First, we apply a transform to the element and set that value to rotate(-90). What this does is rotate the SVG coordinate system itself by -90 degrees.

Note: Everything that follows is my attempt to reverse engineer how the rotate function works. If I turn out to be wrong, please excuse me.

Axis Rotation

The graphic above shows what happens to the coordinate system on applying rotate(-90). Now, you're probably even more confused because a negative rotation typically means a clockwise rotation. Yet, it looks like I've rotated anti-clockwise here.

Well, remember that a typical coordinate system has the Y-axis pointing positively upwards. We have it pointing positively downwards. Therefore, our rotations are reversed.

Now, our new X axis points in the opposite direction of the old Y axis and our new Y axis points in the direction of the old X axis.

Now, in the context of this new information, looking at the values of the xand y attributes makes more sense. Since our new X points opposite to the direction of the old Y, we set a negative value to the x attribute.

Conclusion

Okay, that was quite the post. I wasn’t envisioning it becoming quite so massive but we did cover a lot in detail. I hope you enjoyed going through this post and more than anything, I hope you have a better grasp of how D3 works. This is a truly wonderful library that provides you with a set of very powerful tools.

I’ve created a Code Sanbox here with a working version of the code from this post. Feel free to fork it and play around with it!

If you want to follow me, you can do so on GitHub or Twitter. If you have any questions, please don’t hesitate to ask.

How to go from a tentative to a confident user

d3.js is a fantastic library — but I can honestly tell you that for a long time I used it without fully understanding it. Lacking any solid mental model of what I was working with, I would admittedly tend to copy/paste pieces of code from various snippets over at bl.ocks.org, cross my fingers and hope for the best.

Getting Started with D3

When I started to learn D3, nothing made sense to me. Things only became more clear when I started to learn about reusable charts.

In this article, I’ll show you how to create a reusable bubble chart and give you a gentle introduction to D3 along the way. The dataset we’ll use is composed of stories published in the freeCodeCamp in January 2017.

This is the chart you’re going to build

About D3

D3 is a JavaScript library for data visualization. It brings data to life using HTML, SVG and CSS.

We often need to reuse a chart in another project, or event share the chart with others. For this, Mike Bostock (the creator of D3) proposed a model called reusable charts. We’ll use his approach with some small modifications, as presented by Pablo Navarro Castillo in the book Mastering D3.js.

We are using D3 version 4.6.0 here.

?Reusable charts

Graphs following the reusable chart pattern have two characteristics:

• Configurability. We want to modify the appearance and behavior of the graph without having to modify the code itself.
• Ability to be built in an independent way. We want every graph element associated with a data point of our dataset independently. This has to do with the way D3 associates data instances with DOM elements. This will become more clear in a minute.

“To sum up: implement charts as closures with getter-setter methods.” — Mike Bostock

The bubble chart

You first need to define which elements of the chart can be customized:

• The size of the chart
• The input dataset

Defining the size of the chart

Let's start by creating a function to encapsulate all variables of the graph and set the default values. This structure is called a closure.

// bubble_graph.js

var bubbleChart = function () {    var width = 600,    height = 400;    function chart(selection){        // you gonna get here    }    return chart;}

You want to create charts of different sizes without having to change the code. For this you will create charts as follows:

// bubble_graph.html

var chart = bubbleChart().width(300).height(200);

To do that, now you will define accessors for the width and height variables.

// bubble_graph.js

var bubbleChart = function () {    var width = 600    height = 400;

    function chart(selection){        // we gonna get here    }    chart.width = function(value) {        if (!arguments.length) { return width; }        width = value;        return chart;    }

    chart.height = function(value) {        if (!arguments.length) { return height; }        height = value;        return chart;    }    return chart;}

If you call bubbleChart() (without width or height attributes) the graph is created with the default width and heights values you defined inside the closure. If called without arguments, the method returns the variable value.

// bubble_graph.html

var chart = bubbleChart();bubbleChart().width(); // returns 600

You may be wondering why the methods return the chart function. This is a JavaScript pattern used to simplify the code. It's called method chaining. With this pattern you can create new objects like this:

// bubble_graph.html

var chart = bubbleChart().width(600).height(400);

instead of:

// bubble_graph.html

var chart = bubbleChart(); chart.setWidth(600); chart.setHeight(400);

Joining data with our chart

Now let’s learn how to join data with chart elements. Here’s how the chart is structured: the div with the graph has an SVG element, and each data point corresponds to a circle in the chart.

// bubble_graph.html, after the bubbleChart() function is called

<svg width="600" height="400">;    <circle></circle> // a story from data    <circle></circle&gt; // another story from data    ...</svg>

? d3.data()

The d3.selection.**data**([data[,key]]) function returns a new selection that represents an element successfully bound to data. To do that, you first need to load the data from the .csv file. You will use the d3.**csv**(_url_[[, _row_], _callback_]) function.

// bubble_graph.html

d3.csv('file.csv', function(error, our_data) {    var data = our_data; //here you can do what you want with the data}

// medium_january.csv|                title                 |   category   | hearts ||--------------------------------------|--------------|--------|| Nobody wants to use software         | Development  |  2700  |  | Lossless Web Navigation with Trails  |    Design    |  688   |   | The Rise of the Data Engineer        | Data Science |  862   |

? d3-selection

You will use the d3-select() and the data() functions to pass our data to the chart.

Selections allow powerful data-driven transformation of the document object model (DOM): set attributes, styles, properties, HTML or text content, and more. — D3 documentation

// bubble_graph.html

<div class="chart-example" id="chart"><svg></svg></div>

d3.csv('medium_january.csv', function(error, our_data) {    if (error) {        console.error('Error getting or parsing the data.');        throw error;    }

    var chart = bubbleChart().width(600).height(400);    d3.select('#chart').data(our_data).call(chart);

 });

Another important selector is d3.selectAll(). Let's say you have the following structure:

<body>    <div></div>    <div></div>    <div></div></body>

d3.select("body").selectAll("div") selects all those divs for us.

?? d3.enter()

And now you’re going to learn about an important D3 function: d3.enter(). Let's say you have an empty body tag and an array with data. You want to go through each element of the array and create a new div for each element. You can do this with the following code:

<!-- before --><body> //empty</body>

----// js script

var our_data = [1, 2, 3]var div = d3.select("body") .selectAll("div") .data(our_data) .enter() .append("div");---

<!-- after --><body>    <div></div>    <div></div>    <div></div></body>

Why do you need selectAll("div") if the the divs don't even exist yet? Because in D3 instead of telling how to do something, we tell what we want.

In this case, you want to associate each div with a element of the array. That's what you are saying with the selectAll("div").

var div = d3.select("body") .selectAll("div") // here you are saying 'hey d3, each data element      of the array that comes next will be bound to a div' .data(our_data) .enter().append("div");

The enter() returns the selection with the data bound to the element of the array. You then finally add this selection to the DOM with the .append("div")

?d3.forceSimulation()

You need something to simulate the physics of the circles. For this you will use d3.forceSimulation([nodes]). You also need to tell what kind of force will change the position or the velocity of the nodes.

In our case, we’ll use the d3.forceManyBody().

// bubble_chart.js

var simulation = d3.forceSimulation(data) .force("charge", d3.forceManyBody().strength([-50])) .force("x", d3.forceX()) .force("y", d3.forceY()) .on("tick", ticked);

A positive strength value causes the nodes to attract each other, while a negative strength value causes them to repel each other.

The strength() effect

We don't want the nodes spreading out through the whole SVG space, though, so we use d3.forceX(0) andd3.forceY(0). This "drags" the circles to the 0 position. Go ahead and try removing this from the code to see what happens.

When you refresh the page, you can see that the circles adjust until they finally stabilize. The ticked() function updates the positions of the circles. The d3.forceManyBody() keeps updating the x and y position of each node, and the the ticked() function updates the DOM with these values (the cx and cy attributes).

// bubble_graph.js

function ticked(e) {    node.attr("cx", function(d) { return d.x; })        .attr("cy", function(d) { return d.y; });    // 'node' is each circle of the bubble chart

 }

Here's the code with everything together:

var simulation = d3.forceSimulation(data)     .force("charge", d3.forceManyBody().strength([-50]))     .force("x", d3.forceX())     .force("y", d3.forceY())     .on("tick", ticked);

function ticked(e) {     node.attr("cx", function(d) { return d.x; })         .attr("cy", function(d) { return d.y; }); }

To sum up, all this simulation does is give each circle an x and y position.

? d3.scales

Here comes the most exciting part: actually adding the circles. Remember the enter() function? You will use it now. In our chart the radius of each circle is proportional to the number of recommendations of each story. To do that you will use a linear scale: d3.scaleLinear()

To use scales you need to define two things:

• Domain: the minimum and maximum values of the input data (in our case, the minimum and maximum number of recommendations). To get the minimum and maximum values, you’ll use the d3.min() and d3.max() functions.
• Range: the minimum and maximum output values of the scale. In our case, we want the smallest radius of size 5 and the biggest radius of size 18.

// bubble_graph.js

var scaleRadius = d3.scaleLinear()            .domain([d3.min(data, function(d) { return +d.views; }),                     d3.max(data, function(d) { return +d.views; })])            .range([5,18]);

And then you finally create the circles:

// bubble_graph.js

var node = svg.selectAll("circle")   .data(data)   .enter()   .append("circle")   .attr('r', function(d) { return scaleRadius(d.views)})});

To color the circles, you’ll use a categorical scale: d3.scaleOrdinal(). This scale returns discrete values.

Our dataset has 3 categories: Design, Development and Data Science. You will map each of these categories to a color. d3.schemeCategory10 gives us a list of 10 colors, which is enough for us.

// bubble_graph.js

var colorCircles = d3.scaleOrdinal(d3.schemeCategory10);var node = svg.selectAll("circle")    .data(data)    .enter()    .append("circle")    .attr('r', function(d) { return scaleRadius(d.views)})    .style("fill", function(d) { return colorCircles(d.category)});

You want the circles drawn in the middle of the SVG, so you’ll move each circle to the middle (half the width and half the height). Go ahead and remove this from the code to see what happens.

// bubble_graph.js

var node = svg.selectAll("circle") .data(data) .enter() .append("circle") .attr('r', function(d) { return scaleRadius(d.views)}) .style("fill", function(d) {return colorCircles(d.category)}) .attr('transform', 'translate(' + [width / 2, height / 2] + ')');

Now you’ll add tooltips to the chart. They need to appear whenever we place the mouse over the circles.

var tooltip = selection .append("div") .style("position", "absolute") .style("visibility", "hidden") .style("color", "white") .style("padding", "8px") .style("background-color", "#626D71") .style("border-radius", "6px") .style("text-align", "center") .style("font-family", "monospace") .style("width", "400px") .text("");

var node = svg.selectAll("circle") .data(data) .enter() .append("circle") .attr('r', function(d) { return scaleRadius(d.views)}) .style("fill", function(d) {return colorCircles(d.category)}) .attr('transform', 'translate(' + [width / 2, height / 2] + ')') .on("mouseover", function(d){     tooltip.html(d.category +"<br>"+ d.title+"<br>"+d.views);      return tooltip.style("visibility", "visible");}) .on("mousemove", function(){   return tooltip.style("top", (d3.event.pageY-       10)+"px").style("left",(d3.event.pageX+10)+"px");}) .on("mouseout", function(){return tooltip.style("visibility", "hidden");});

The mousemove follows the cursor when the mouse is moving. d3.event.pageX and d3.event.pageY return the mouse coordinates.

And that's it! You can see the final code here.

You can play with the bubble chart here.

Did you found this article helpful? I try my best to write a deep dive article each month, you can receive an email when I publish a new one.

Any questions or suggestions? Leave them in the comments. Thanks for reading! ?

Special thanks to John Carmichael and Alexandre Cisneiros.

The bind, the draw and the interactivity

Let’s say you’re building a data visualization using D3 and SVG. You may hit a ceiling when you try to display several thousand elements at the same time. Your browser may start to puff under the weight of all those DOM elements.

Well here comes HTML5 Canvas to the rescue! It’s much faster, so it can solve your browser’s puffing problems.

But you may quickly find yourself daunted. Because D3 and Canvas works a bit differently from D3 and SVG — especially when it comes to drawing and adding interactivity.

But fear not — it’s not that complicated. Any experience you’ve had with building visuals with D3 and SVG — or approaching D3 with a different renderer — will help you tremendously.

This tutorial built on the shoulders of giants who have already covered Canvas well. I learned these three tutorials by heart and I recommend you do, too:

• Working with D3.js and Canvas: When and How from Irene Ros
• Needles, Haystacks, and the Canvas API from Yannick Assogba
• Learnings from a D3.js addict on starting with Canvas from Nadieh Bremer

So why continue reading this, then? Well, when I want to learn something new, it helps me a great deal to look at the same subject from slightly different angles. And this tutorial is a slightly different angle.

Also, this tutorial covers the three key steps: binding data, drawing elements, and adding interactivity — and it does all this in one go, with an added step-by-step manual to set you up.

What do we build?

A grid of pretty colours

A grid of (many) squares. Their colours aren’t of any deep meaning but don’t they look pretty? The important bit is that you can update it (to cover binding and updating data), that it has many elements (up to 10,000 in order for canvas to pay out), and that you can hover over each square to show square-specific information (interactivity). You can play with it here on a full screen or here with all the code

The mental model

Before we actually dive in, let’s quickly step back and grasp conceptually what we do when we create elements with D3 to draw them to the screen. Skip this if you just want to make things.

The first step when using D3 usually doesn’t involve drawing — it involves preparing all your elements you want to draw. It’s a bit like building some LEGO. You can rip open the box and start building something or you can look at the manual first and build it according to the blueprint. The manual is your mental model, a blueprint or recipe of what you want to build.

_A mental model turned material (Mike, 2009 [https://creativecommons.org/licenses/by/2.0/](https://creativecommons.org/licenses/by/2.0/" rel="noopener" target="blank" title="))

What is D3’s model? Apart from the large number of helpful functions and methods that calculate positions, re-shape datasets (the layouts) and generate functions that draw, for example, paths for us, D3 has a model for how the elements’ lives should evolve on the screen. It has a certain way to think about the lifecycle of each element.

Less ethereally, you inject data into a yet non-existent DOM, and D3 creates new elements of your choice as per the data you inject. Usually one element per datapoint. If you want to inject new data into the DOM you can do so and D3 identifies which elements have to be newly created, which elements are allowed to stay and which elements should pack up and leave the screen.

D3 is usually used in conjunction with SVG or sometimes with HTML-elements. In this orthodox case, you can see the data in the DOM when you choose to look at it through the console, for example. You can grab it, you can move it up or down the DOM and you can — importantly — add interactivity to each element you like to show, for example, a tooltip.

But — on the downside — you can’t show a lot of elements. Why? Because the more elements you push into the DOM, the harder the browser has to work to display them all. Let them also move around and the browser needs to re-calculate them constantly. The more knackered the browser gets the lower your frame rate or FPS (Frames Per Second), which measures how many frames the browser can paint each second. A frame rate of 60 is good and enables a fluid experience as long as no frames are missed — a frame rate of anything under 30 can equal a choppy ride. So when you want to show more elements, you can revert to canvas.

Why canvas? Canvas is an HTML5 element which comes with its own API to paint on it. All elements drawn on the canvas element won’t manifest in the DOM and save a lot of work for the browser. They are drawn in immediate mode. This means the rendered elements won’t get saved in the DOM but your instructions draw them directly to a particular frame. The DOM only knows the one canvas element; everything on it is only in memory. If you want to change your canvas elements you have to redraw the scene for the next frame.

The problem with this is of course that you can’t communicate directly with these non-material elements living in memory. You have to find a way to talk to them indirectly. This is where the D3 model comes in as well as custom or ‘virtual’ DOM-elements. What you’ll do in principal is:

1. Bind your data to custom DOM elements. They don’t live in the DOM but only in memory (in a ‘virtual’ DOM) and describe the life-cycle of these elements in a known D3 way.
2. Use canvas to draw these elements.
3. Add interactivity with a technique called ‘picking’.

Let’s do it.

The data

Before we start to code, let’s produce some data. Let’s say you want 5,000 datapoints. So let’s create an array with 5,000 elements, each of which is an object with just a single property value carrying the element’s index. Here’s how you create it with d3.range(). [d3.range()](https://github.com/d3/d3-array/blob/master/README.md#range) is a D3 utility function, that creates an array based on its argument:

var data = [];

d3.range(5000).forEach(function(el) {

  data.push({ value: el });

});

Here’s how the data looks in the console

Thrills!

The canvas container and its tools

The canvas element is an HTML element. It’s conceptually very much like any SVG-parent-element, which I at least usually add to a simple container div as in:

<div id=“container”></div>

So, let’s add it to your container with D3 as in…

var width = 750, height = 400;

var canvas = d3.select('#container')  .append('canvas')  .attr('width', width)  .attr('height', height);

var context = canvas.node().getContext('2d');

You also need to add the context, which is the canvas toolbox. The context variable is from now on the object carrying all the properties and methods, the brushes and colours you need to draw on the canvas. Without the context, the canvas element would remain empty and white. That’s all you need to setup — a canvas and its tools…

_Base by Stilfehler — Own work, CC BY-SA 3.0, [https://creativecommons.org/licenses/by/2.0/](https://commons.wikimedia.org/w/index.php?curid=5899171" rel="noopener" target="_blank" title="">https://commons.wikimedia.org/w/index.php?curid=5899171; blue Lego by David Lofink, 2008 <a href="https://creativecommons.org/licenses/by/2.0/" rel="noopener" target="blank" title=")

The HTML

…is simple. The main HTML structure of your site will be:

<!-- A title --><h3>Coloured grids</h3>

<!-- An input field with a default value. --> <input type="text" id="text-input" value="5000">

<!-- An explanation... --> <div id="text-explain">...takes numbers between 1 and 10k</div>

<!-- ...and a container for the canvas element. --> <div id="container"></div>

The Javascript structure

On a top level you only need 2 functions:

databind(data) {

  // Bind data to custom elements.

}

draw() {

  // Draw the elements on the canvas.

}

Pretty straight forward so far.

Bind the elements

To bind data to the elements you first create a base element for all your custom elements you will produce and draw. If you know D3 well, think of it as a replacement to the SVG element:

var customBase = document.createElement('custom');

var custom = d3.select(customBase); // This is your SVG replacement and the parent of all other elements

Then you add some settings for your grid. In short, these settings allow you to draw a grid of squares. 100 squares build a ‘parcel’ and there is a line break after 10 parcels (or after 1,000 squares). You can adjust this for different ‘parceling’ of the squares or different line-breaking. Or just not worry about it. I suggest the latter…

// Settings for a grid with 10 cells in a row, // 100 cells in a block and 1000 cells in a row.

var groupSpacing = 4; var cellSpacing = 2; var offsetTop = height / 5; var cellSize = Math.floor((width - 11 * groupSpacing) / 100) - cellSpacing;

Now let’s start the data-binding mission. Let’s get the necessities out of the way first and create a colour scale you will apply to your squares a little later.

function databind(data) {

// Get a scale for the colours - not essential but nice.

colourScale = d3.scaleSequential(d3.interpolateSpectral)                      .domain(d3.extent(data, function(d) { return d; }));

Now let’s join your data to the ‘replacement-SVG’ you called custom above and add yet non-existing custom elements with the class .rect

var join = custom.selectAll('custom.rect')  .data(data);

You enter the custom elements (remember nothing enters the DOM, this is all in memory).

var enterSel = join.enter()  .append('custom')  .attr('class', 'rect')  .attr("x", function(d, i) {    var x0 = Math.floor(i / 100) % 10, x1 = Math.floor(i % 10);         return groupSpacing * x0 + (cellSpacing + cellSize) * (x1 + x0 * 10); })  .attr("y", function(d, i) {  var y0 = Math.floor(i / 1000), y1 = Math.floor(i % 100 / 10);   return groupSpacing * y0 + (cellSpacing + cellSize) * (y1 + y0 * 10); })  .attr('width', 0)  .attr('height', 0);

When an element enters your model, you just give it an x and a y position as well as a width and a height of 0, which you’ll change in the upcoming update selection…

You merge the enter selection into the update selection and define all attributes for the update and enter selection. This includes a width and a height value as well as a colour from the colour scale you built earlier:

join   .merge(enterSel)  .transition()  .attr('width', cellSize)  .attr('height', cellSize)  .attr('fillStyle', function(d) { return colourScale(d); });

Two things of note about this last line. When you work with SVG this line would be

.style('color', function(d) { return colourScale(d); })

But with canvas you use .attr(). Why? Your main interest here is to find a pain-free way to transfer some element-specific information. Here you want to transfer a colour-string from the databind() to the draw() function. You use the element simply as a vessel to transport your data over to where it is being rendered to the canvas.

That's a very important distinction: when working with SVG or HTML you can bind data to elements and draw or apply styles to the elements in one step. In canvas you need two steps. First you bind the data then you draw the data. You can't style the elements while binding. They only exist in memory and canvas can't be styled via CSS style properties, which is exactly what you access when using .style().

At first, this might seem limiting as you can do less in one step, but it’s conceptually almost cleaner and also gives you some freedom. .attr() allows us to send any key-value pairs on the journey. You could use other methods like the HTML .dataset property for example, but .attr() will do just fine.

Notice we don't say color but fillStyle. To be honest, you could use color or you could use chooChooTrain here. You would only need to remember this when you fetch the information later during drawing. However, as canvas uses a property called fillStyle to style elements, it seems more appropriate in this case.

Finally, you also define the exit selection, deciding what should happen to exiting elements.

var exitSel = join.exit()  .transition()  .attr('width', 0)  .attr('height', 0)  .remove();

That’s it! You can close your databind() function and move on...

} // databind()

This is not really scary coming from D3 as it’s pretty much exactly the same. You have now successfully created your data model, the way the application will think about data. Each element will get the properties it needs to be drawn via the .attr() functions and each element will be assigned a life-cycle state depending on the injected data. Our standard D3 model.

Drawing the elements

_By Kristina Alexanderson, 2011 [https://creativecommons.org/licenses/by-nc-nd/2.0/](https://creativecommons.org/licenses/by-nc-nd/2.0/" rel="noopener" target="blank" title=")

Now you need to write the draw function to get the elements on screen. Let’s just note here that nothing has happened yet. You haven’t called databind() yet because you need to find a way to draw it to the canvas first. So here we go... The draw() function doesn't need to take any arguments in this case:

function draw() {

As mentioned fleetingly above, you need to take care of cleaning the canvas every time you draw afresh. The DOM is material, in that when you draw a rect-element on it and you change its x value, it will move in the x-direction and the DOM will take care of this move (or the re-paint) automatically.

If you move a rect from x = 0 to x = 1 at a certain point in time (after a button press for example) the browser will move the rect from 0 to 1 within one tick or frame-paint (which is roughly 16ms long). If you move it from 0 to 10, it will do so in a time depending on the duration you asked this transition to happen, maybe 1 pixel per tick maybe 8 pixel per tick (for more read this blog post).

But it will tell the pixel at 0 that the rect has disappeared and the pixel at 1 that there is a rect now. Canvas doesn’t do this. You need to tell canvas what to paint, and if you paint something new, you need to tell it to remove the previous paint.

So let’s start with cleaning up anything that might be on the canvas before you draw. Here’s how:

context.clearRect(0, 0, width, height); // Clear the canvas.

Simple.

Now you…

1. …get hold of all elements in order to
2. loop through all elements and
3. take the information you have stored in the databind() function to draw the element:

// Draw each individual custom element with their properties.

var elements = custom.selectAll('custom.rect');// Grab all elements you bound data to in the databind() function.

elements.each(function(d,i) { // For each virtual/custom element...

  var node = d3.select(this);   // This is each individual element in the loop.     context.fillStyle = node.attr('fillStyle');   // Here you retrieve the colour from the individual in-memory node and set the fillStyle for the canvas paint

  context.fillRect(node.attr('x'), node.attr('y'), node.attr('width'), node.attr('height'));  // Here you retrieve the position of the node and apply it to the fillRect context function which will fill and paint the square.

}); // Loop through each element.

And that’s it! You can close the draw() function

} // draw()

When I started with canvas after a while of wanting to dive into it, this simplicity really upped my spirits.

However, nothing has happened in the browser yet. We have the tools in the databind() and the draw() function, but nothing has been drawn yet. How do you do this? If you just wanted to draw a static visual or image, you just call:

databind(data);

draw();

This would bind the data to the custom elements, which would live in memory and then draw it — once!

But you have transitions. Remember above: when you wrote the databind() function you transitioned the cell width and height from 0 to their size as well as the colour from black (the default) to the respective element’s colour. A default D3 transition lasts 250 milliseconds, so you need to redraw the squares many times in these 250 ms in order to get a smooth transition. How do you do this?

It’s again simple. You just call databind(data) to create our custom elements before you repeatedly call draw() for as long as it takes the transition to run. So in our case at least 250 ms. You could use setInterval() for this but we really should use requestAnimationFrame() in order to be as performant as possible (for more read this). There are a few ways to use it, but keeping within the D3 spirit, I suggest using d3.timer() which implements requestAnimationFrame() as well as being straight forward to use. So here we go:

// === First call === //

databind(d3.range(value)); // Build the custom elements in memory.

var t = d3.timer(function(elapsed) {

  draw();

  if (elapsed > 300) t.stop();

}); // Timer running the draw function repeatedly for 300 ms.

d3.timer() calls the callback repeatedly until elapsed (which is the passed time in milliseconds from instantiation) is past 300 and then the timer is stopped. In these 300 milliseconds, it runs the draw() at each tick (roughly each 16ms). draw() then looks at each element's attributes and draws them accordingly.

This is how a transition works in canvas. You call the drawing function right after the binding function many times. Whatever your D3-model is set up to transition (positions, colours, sizes) will be re-drawn many times with small incremental changes for each draw

Note that draw() needs to come right after the databind() function. You couldn't ask the machine to run databind(), then do something else for a second and then call draw(). Because after 1 second the transitioned states calculated by your databind() function have all transitioned already. Done, dusted and forgotten.

That’s it! You’ve bound data to custom elements and you’ve drawn it to the canvas.

Let the user update the number of squares

To give the user the chance to repeat this feat with a custom number of elements (ok, semi-custom with a maximum of 10,000) you add the following listener and handler to your text-input box:

// === Listeners/handlers === //

d3.select('#text-input').on('keydown', function() {

if (d3.event.keyCode === 13) { // Only do something if the user hits return (keycode 13).

  if (+this.value < 1 || +this.value > 10000) {   // If the user goes lower than 1 or higher than 10k...         d3.select('#text-explain').classed('alert', true);     // ... highlight the note about the range and return.

    return;

  } else {   // If the user types in a sensible number...

    d3.select('#text-explain').classed('alert', false);     // ...remove potential alert colours from the note...

    value = +this.value; // ...set the value...

    databind(d3.range(value)); // ...and bind the data.

    var t = d3.timer(function(elapsed) {

      draw();        if (elapsed > 300) t.stop();

    }); // Timer running the draw function repeatedly for 300 ms.     } // If user hits return.

}); // Text input listener/handler

Here it is again, our colourful grid of canvas squares, ready to be updated and redrawn:

Interactivity

The biggest ‘pain’ with canvas in comparison to SVG or HTML is that there are no material elements living in the DOM. If there were you could just register listeners to the elements and add handlers to the listeners. For example you can trigger a mouse-over on an SVG rect element and whenever the listener triggers, you could do something to the rect. Like showing data values stored with the rect in a tooltip.

With canvas you have to find another way to make an event heard on our canvas elements. Luckily there are a number of clever people who thought of an indirect but logical way.

So what interactivity do we want? As said above let’s go for a tooltip and let’s assume you want to show the index of the square in a tooltip as soon as you hover over the element. Not very thrilling, but the key is that you can access the data bound to the element by hovering over it.

Picking

There are a few steps involved (all logical though). But in short you will build two canvases to achieve this. One main canvas that produces our visual and one hidden canvas (as in we can’t see it) that produces the same visual. The key here is that all elements on the second canvas will be at the exact same position in relation to the canvas origin compared to the first canvas. So square 1 starts on 0,0 on the main canvas as well as on the hidden canvas. Square 2 starts on 8,0 on the main canvas as well as on the hidden canvas and so on.

There is only one important difference. Each element on the hidden canvas will get a unique colour. We will create an object (or rather an associative array or map for brevity) that links each unique colour to each element’s data.

Why? Because next we attach a mouse-move listener to the main-canvas to retrieve a stream of mouse-positions. At each mouse-position we can use a canvas-own method to “pick” the colour at this exact position. Then we just look up the colour in our associative array and we have the data ! And we’re flying…

_By Kenny Louie, 2010 [https://creativecommons.org/licenses/by/2.0/](https://creativecommons.org/licenses/by/2.0/" rel="noopener" target="blank" title=")

You could say “well, my squares have already got a unique colour, I can use those?” And indeed, you could use them. However, your interactivity would go out of the window as soon as you decide to bereft your squares from the colours. So you should make sure to always have one canvas — the hidden canvas — that has a guaranteed set of unique colours for the squares.

Let’s apply this technique step by step. The code you’ve built so far can stay as it is — you just add to it as you go along.

1. Prepare the hidden canvas

First let’s create the hidden canvas that will harbour our visual with a unique colour per square.

1.1 Create hidden canvas element and set its CSS to { display: none; }.

// Rename the main canvas and add a 'mainCanvas' class to it.

var mainCanvas = d3.select('#container')  .append('canvas')  .classed('mainCanvas', true)  .attr('width', width) .attr('height', height); // new -----------------------------------

// Add the hidden canvas and give it the 'hiddenCanvas' class.

var hiddenCanvas = d3.select('#container')  .append('canvas')  .classed('hiddenCanvas', true)   .attr('width', width)   .attr('height', height);

In fact, I won’t set the canvas to hidden in this example to show what is going on. But to do so, just add .hiddenCanvas { display: none; } to your CSS and the deed is done.

1.2 Build the context variable in the draw() function and pass two arguments to the function: the canvas as well as a boolean called 'hidden' determining which canvas we build (hidden = true || false) as in:

function draw(canvas, hidden) {

1.3 You now need to adapt all draw functions to include the two new draw() arguments. So from now on, you don't just call draw() you call either draw(mainCanvas, false) or draw(hiddenCanvas, true)

2. Apply unique colours to the hidden elements and map them

Here, dear reader, comes the key part of our operation, the engine of our truck, the spice in our soup.

_By Andrew Becraft, 2007 [https://creativecommons.org/licenses/by-nc-sa/2.0/](https://creativecommons.org/licenses/by-nc-sa/2.0/" rel="noopener" target="blank" title=")

2.1 Include a function to generate a new unique colour every time it gets called (via Stack Overflow)

// Function to create new colours for the picking.

var nextCol = 1;

function genColor(){     var ret = [];

  if(nextCol < 16777215){         ret.push(nextCol & 0xff); // R     ret.push((nextCol & 0xff00) >> 8); // G     ret.push((nextCol & 0xff0000) >;> 16); // B

    nextCol += 1;     }

var col = "rgb(" + ret.join(',') + ")";

return col;

}

genColour() produces a colour defining string in the form rgb(0,0,0). Every time it's called it increments the R value by one. Once it reaches 255, it increments the G value by 1 and resets the R value to 0. Once it reaches r(255,255,0) it increments the B value by 1 resetting the R and the G to 0 and so on.

So in total you can have 256256256 = 16.777.216 elements to retain a unique colour. However, I can assure you your browser will die beforehand. Even with canvas (webGL tutorial to follow).

2.2 Create the map-object that will keep track of which custom element has which unique colour:

var colourToNode = {}; // Map to track the colour of nodes.

You can add the genColour() function wherever you want in your script, as long as it's outside the databind() and draw() function scope. But note that your map variable needs to be created before and beyond the scope of the databind() function.

2.3 Add a a unique colour to each custom element as for example .attr('fillStyleHidden') and
2.4 build the map-object during element creation

Here you’ll use your ‘colour-canon’ genColour() in our databind() function when assigning the fillStyle to our elements. As you also have access to each datapoint while it's being bound to each element, you can bring colour and data together in your colourToNode map.

join   .merge(enterSel)   .transition()   .attr('width', cellSize)   .attr('height', cellSize)   .attr('fillStyle', function(d) {     return colorScale(d.value);   });

  // new -----------------------------------------------------         .attr('fillStyleHidden', function(d) {

    if (!d.hiddenCol) {

      d.hiddenCol = genColor();       colourToNode[d.hiddenCol] = d;

    }

    // Here you (1) add a unique colour as property to each element     // and(2) map the colour to the node in the colourToNode-map.

    return d.hiddenCol;

});

2.5 You can now colour the elements according to the canvas the draw() function is rendering. You add a conditional on the fillStyle in the draw() function applying the colours for our visual to the main canvas and the unique colours to the hidden canvas. It's a simple one-liner:

context.fillStyle = hidden ? node.attr('fillStyleHidden') : node.attr('fillStyle');

// The node colour depends on the canvas you draw.

The main canvas still looks the same of course:

Let’s finally add some interactivity and start with drawing the hidden canvas whenever we move the mouse onto our main canvas.

3. Pick up the colours with the mouse

3.1 First, simply register a listener to the main canvas, listening to mouse-move events.

d3.select('.mainCanvas').on('mousemove', function() {

});

Why mousemove? As you can’t register listeners with individual squares but have to use the entire canvas you won’t be able to work with mouseover or -out events as they will only trigger when entering the canvas not the elements. In order to get the mouse position on your canvas you can do mousemove or click/mousedown.

d3.select('.mainCanvas').on('mousemove', function() {

  draw(hiddenCanvas, true); // Draw the hidden canvas.

});

This way, the first thing our user triggers when mousing over the main canvas is to unknowingly create the hidden canvas. As said, in production this canvas would be hidden, but for our educational purposes we want to see it and indeed, trigger the hidden canvas to be drawn when the mouse moves over the main canvas like so:

The colours on the main canvas range from black to red, from rgb(0,0,0) to rgb(255,0,0) and then it looks as if the same range from black to red is repeated. However, now the colour ranges from a slightly greener black, precisely from rgb(0,1,0) to rgb(255,1,0):

Zooming into the first couple of hundred squares, here are the colours of the first, the 256th and the 257th square:

3.3 As our hidden canvas is structurally a carbon copy of our main canvas, all the hidden canvas elements will be at the same position as the elements on our main canvas. So, you can now use the mouse’s x and y positions you are collecting from the listener on the main canvas to establish the same location on the hidden canvas. Back in the listener, you add:

d3.select('.mainCanvas').on('mousemove', function() {       // Draw the hidden canvas.  draw(hiddenCanvas, true);

  // Get mouse positions from the main canvas.  var mouseX = d3.event.layerX || d3.event.offsetX;   var mouseY = d3.event.layerY || d3.event.offsetY; });

Note here we take the event.layerX and event.layerY properties which return the mouse position including scrolling. This can break so use offsetX as a fallback (or just use offsetX).

3.4 The picking: Canvas greatly allows access to the pixel-data the mouse is hovering over with the getImageData() function and its .data property. In full bloom this will look like:

getImageData(posX, posY, 1, 1).data .

It will return an array with four numbers: the R, the G, the B and the alpha value. As you diligently built the colourToNode map assigning the element data to each of its hidden colours, you can now access this element's data simply by looking up the colour in the map!

d3.select('.mainCanvas').on('mousemove', function() {

  // Draw the hidden canvas.  draw(hiddenCanvas, true);

  // Get mouse positions from the main canvas.  var mouseX = d3.event.layerX || d3.event.offsetX;   var mouseY = d3.event.layerY || d3.event.offsetY;

// new -----------------------------------------------

  // Get the toolbox for the hidden canvas.  var hiddenCtx = hiddenCanvas.node().getContext('2d');

  // Pick the colour from the mouse position.   var col = hiddenCtx.getImageData(mouseX, mouseY, 1, 1).data;

  // Then stringify the values in a way our map-object can read it.  var colKey = 'rgb(' + col[0] + ',' + col[1] + ',' + col[2] + ')';

  // Get the data from our map!   var nodeData = colourToNode[colKey];

  console.log(nodeData);

});

And indeed, logging the nodeData to the console returns an object every time you hover over a square:

The data per node now shows the value which constitutes the original data as well as the key hiddenCol showing this node's colour for the hidden canvas:

3.5 Finally — and that’s a formality — you add the tooltip

d3.select('.mainCanvas').on('mousemove', function() {

  // Draw the hidden canvas.  draw(hiddenCanvas, true);

  // Get mouse positions from the main canvas.  var mouseX = d3.event.layerX || d3.event.offsetX;   var mouseY = d3.event.layerY || d3.event.offsetY;

  // Get the toolbox for the hidden canvas.  var hiddenCtx = hiddenCanvas.node().getContext('2d');

  // Pick the colour from the mouse position.   var col = hiddenCtx.getImageData(mouseX, mouseY, 1, 1).data;

  // Then stringify the values in a way our map-object can read it.  var colKey = 'rgb(' + col[0] + ',' + col[1] + ',' + col[2] + ')';

  // Get the data from our map!   var nodeData = colourToNode[colKey];     console.log(nodeData);

  // new -----------------------------------------------

  if (nodeData) {   // Show the tooltip only when there is nodeData found by the mouse

    d3.select('#tooltip')       .style('opacity', 0.8)       .style('top', d3.event.pageY + 5 + 'px')       .style('left', d3.event.pageX + 5 + 'px')         .html(nodeData.value);

  } else {   // Hide the tooltip when the mouse doesn't find nodeData.      d3.select('#tooltip').style('opacity', 0);     }

}); // canvas listener/handler

That’s it! You’ve visualised a large number of elements on canvas — more than you would’ve been able to enjoy problem-free with SVG. You still used D3’s lifecycle model and you added some interactivity to access the data attached to each element. These three steps should enable you to do pretty much anything or at least more than what you’re used to when working with D3 and SVG.

There’s a step-by-step manual from scratch to interactive D3/canvas on my blog which allows internal page links. This way you can see the whole process in one view and click your way through it with ease:

Click to get to the manual

…and here’s the full code again.

I hope you enjoyed reading this and please do say hello and/or …

_lars verspohl www.datamake.io @larsvers https://www.linkedin.com/in/larsverspohl

…is always grateful for a like? or a follow he can return.

I joined Free Code Camp’s community 18 months ago as a data scientist who was curious about web development. I quickly discovered that not only did they take data science seriously — they already had plans to release their datasets to the public as open data.

After some discussion, Quincy Larson promoted me to Free Code Camp’s core team and gave me the green light to create the Data Science Room. This would serve as an official chat room for discussing data science, data visualization, and open data.

All of this happened at the end of July 2015 — just over one year ago.

Since that time, my experience in the Data Science Room has been rich and motivating. As of August 2016, the Data Science Room is composed of more than 700 moderately active users. Its logs contains a wealth of useful resources. And many of its participants have built projects involving real Free Code Camp data.

This article is a brief overview of some of the more interesting developments over the past year in our Data Science Room.

The Open Data Vision

The first message in our Data Science Room came from Quincy Larson, who summarized Free Code Camp’s interest in Open Data as natural extension of the Open Source movement:

“Our near-term plans are to open up our entire anonymized dataset for academic study.” - Quincy Larson in July 2015

Six month later, Free Code Camp conducted its first open data experiment by releasing a large open dataset containing the progress of more than 100,000 campers who’d opted in, on Christmas Eve:

Free Code Camp Christmas Special: Giving the Gift of Open Data
_To the countless researchers and data scientists who have politely asked for our data over the past few months: Merry…_medium.freecodecamp.com

Then a few months later, the Data Science Room released survey results from the 2016 New Coder Survey. This included data from 15,000 respondents, each of whom answered up to 48 questions:

We asked 15,000 people who they are, and how they’re learning to code
_More than 15,000 people responded to the 2016 New Coder Survey, granting researchers an unprecedented glimpse into how…_medium.freecodecamp.com

The results of the survey are currently one of the most popular datasets on Kaggle, and are also available in raw and normalized form on GitHub.

Members of our Data Science Room have also joined in on the data analysis, focusing their energies on datasets generated from the many platforms used by the Free Code Camp community.

Gitter chat rooms

Gitter has been a core social hub for Free Code Camp since the beginning, barring a short “Slack interlude”:

So Yeah We Tried Slack… and We Deeply Regretted It
_Back in April, all was well with our community of busy adults learning to code. We were communicating using Gitter.im…_medium.freecodecamp.com

Gitter chat rooms have been the main venue for welcoming new campers, asking for programming help, and coordinating contributions to open source projects. And Gitter has become quite popular.

There were close to 2,000,000 messages posted in the Free Code Camp main chat room alone between January 1, 2015 and May 31, 2016.

Between all the Free Code Camp chat rooms, the main one (FreeCodeCamp/FreeCodeCamp) is the most popular. It bottoms out at around 1 message per minute, and jumps to 6–9 messages per minute during peak time.

A community-driven effort to help newcomers.

Since Free Code Camp is a volunteer-driven open source community, a big part of its success depends upon campers who are willing to share their time and expertise. Our community is designed to keep that engine of helpfulness always running.

Every month, hundreds of campers are able to land their first developer jobs, and as a result become less active in chat. But new generations of campers rise up to fill the gap they leave behind. That’s how the community sustains itself.

When one camper helps another, they are rewarded by “brownie points” within Free Code Camp’s platform. Want to see which campers have been the most helpful lately? Check out the unofficial Free Code Camp top 100 leaderboard (built by Roel Verbunt using in React, MongoDB, and node.js).

Local study groups

Free Code Camp Mexico City.

Campers also meet in-person through Free Code Camp’s 1,500+ local study groups.

These groups organize events through Facebook groups. The average group has 40 campers, and major cities like San Francisco, London, Toronto, and Delhi have more than 1,000 members each.

Alice Jiang, Aleksandar B, and myself ran an analysis and found that around 45% of registered study groups had future events scheduled.

Here are some other community statistics:

• By July 2016 there were at least 430 people who have made at least one commit to Free Code Camps’ core open source repository
• Free Code Camp has the 13th most popular publication on Medium
• Our YouTube channel has more than 40,000 followers
• We recently introduced a forum, which now has more than 30,000 users and 1,000 visitors per day.

I also analyzed the words in Free Code Camp’s 222 most recent tweets:

(‘code’, 44),
(‘freecodecamp’, 40),
(‘time’, 35),
(‘design’, 28),
(‘make’, 24),
(‘medium’, 24),
(‘trying’, 21),
(‘write’, 20),
(‘browser’, 20),
(‘history’, 19),
(‘future’, 19),
(‘developer’, 18),
(‘inspiration’, 17),
(‘programming’, 16),
(‘read’, 16),
(‘javascript’, 16),
(‘stories’, 13),
(‘coding’, 13),
(‘learn’, 12),
(‘spend’, 12),
(‘declarative’, 11),
(‘imperative’, 11),
(‘work’, 11),
(‘people’, 11),
(‘using’, 11),
(‘become’, 11),
(‘perfect’, 10),
(‘finding’, 10),
(‘better’, 10)

In short — coding, learning, and hard work.

A huge thanks to everyone in the Data Science Room who helped with the data analysis and visualization necessary to create this article.

Until next time, stay social! And happy coding!

In the past year, the grid map style visualization has gained a lot of popularity in the US [2,4]. It has a quite a few nice properties, which inspired me to create one for Thailand.

The rest of this article will explain grid maps, and how I created the grid map for Thailand map you see above.

But if you are interested in using the map already without reading the details of how to create one yourself, jump straight to my gridmap-layout-thailand github repository.

What is a Grid Map, and what are its properties?

In a grid map, each region is represented as a tile of the equal shape and size. The tiles are placed to fit within a grid at positions that approximate real geographic position. If the tile is a square, the grid is rectangular. If the tile is a hexagon, then the grid is like a honeycomb. See the US maps below:

_Maps of the US: (left) Hexagon Tile Grid Map (center) Square Tile Grid Map (right) Geographic Map. Source: [NPR Blog](http://blog.apps.npr.org/2015/05/11/hex-tile-maps.html" rel="noopener" target="blank" title=")

Most of grid map visualizations are choropleth maps that use colors to encode values, such as amount of rainfall, average population age, or political alignment. Creating choropleth maps from geographic maps has some disadvantages, because the different sizes of each region introduce biases while interpreting the results. Also, many small regions are often too small to be notice on a map.

Pros: grid maps do not introduce biases due to different sizes of the regions, making it an interesting choice when creating a choropleth map. Small regions are now guaranteed to be big enough to be seen. It’s also easier to implement and faster to load than a geographic map.

Cons: The positions of the regions are only approximations, and could be inaccurate. For example, some adjacent regions may not appear adjacent on the grid map. It also distorts the distance between regions.

Creating Grid Map for Thailand

Here are the requirements:

• The overall map still looks similar to the geographical shape of Thailand. The neighboring regions should appear adjacent or close by.
• Use square tiles for ease of curation and application. For example, this map can be drawn in Excel [3] or Google Sheets easily.
• The map does not have any holes in the middle to avoid confusion.

Of course, the most straightforward approach to create this map is to draw the map manually from scratch. However, I would like to save some energy with a semi-automatic approach:

1. For each province, create one rectangle centered at its centroid.
2. Use a force-directed simulation to detect collisions between rectangles and remove overlaps. Each rectangle has its own center of gravity at the centroid of its province (multi-foci force-directed layout). The solid lines in the figure below show displacements of the rectangles from their ideal positions.
3. Snap the rectangles to a rectangular grid.
4. Export results from Step 3 as CSV file and manually curate in Google Sheets or Excel. The goal of this step is to remove unnecessary gaps, connect the map into one contiguous piece, and adjust the positions of some provinces.
5. Done! The new map only requires 40% of the original space.

Usage

The output from this process can be used as JS, CSV or JSON files. Please see gridmap-layout-thailand on GitHub for instructions. The code I used for steps 1–5 above are also in that repo.

Lastly, this method still depends a lot on manual curation at the end, which may not produce the best optimization. If you have any suggestions or would like to recommend a better approach, the comment box below is yours. :)

References

[1] Mike Bostock. “Multi-Foci Force Layout” bl.ocks.org Published 11 February 2012
[2] Danny DeBelius. “Let’s Tesselate: Hexagons For Tile Grid Maps” NPR Blog. Published 11 May 2015
[3] Caitlin Dempsey Morais. “How to Make a Tile Grid Map Using Excel” GIS Lounge. Published 10 November 2015
[4] Nathan Yau. “The Great Grid Map Debate of 2015” Flowing Data. Published 12 May 2015