I am sure you have heard this many times
A picture is worth a thousand words.
I think with the proliferation of data, this statement can easily be modified to
A picture is worth thousand(s) of data points.
If you are not convinced, look at the example below.
Let’s look at the following statement:
“In 2013, Gun Deaths Claimed 11,419 lives in the U.S.”
What comes to your mind first?
Without visualisation the above are just a set of statistics. Though this stat is useful, it still falls short in conveying the bigger picture. You don’t get the complete context – What kind of people were involved? How much was the nation’s loss? etc.
Let’s see what happens when we visualise it:
Periscope’s US Gun Deaths Visualisation (takes some time to load) – Periscope has made this visualisation that shows the number of people died due to the gun violence in the U.S. in 2013. The orange line denotes the age at which they died and the white line denotes the number of years those people could have lived had they died of natural causes. This visualisation shows what 11,419 lost lives looks like and just how many years of life was stolen from the victims. It is a good example of how we can express the data in a more sublime and impactful manner by using basic animations and interactivity.
“When we present the data as a story, it gives a range of new perspectives and a much more holistic view of the situation which helps policy makers in making informed decisions.”
The web is becoming more accessible day by day and with advancements in browser technology, it is now possible to render complex visualisations on the fly across a variety of devices. This combination of accessibility and complexity makes the web an apt platform to reach out to large audiences.
A lot of organisations are already using web / mobile applications showing dashboards to your mobile as well at your laptop and computer.
Enter D3.js, a powerful library that enables you to build customized visualisations for any kind of storytelling you could imagine for the web.
Let’s look at another amazing visualisation :
Hans Rosling’s 200 Countries, 200 Years, 4 Minutes – Hans tries to explain the change in health and income in the last 200 years across 200 countries of the world in just 4 minutes by utilising the power of effective visualisation.
The properties of this graph are as follows:
Notice how by using simple visual encodings like shape, size, colour, etc. such complex information(200 years of data) is conveyed across. We will talk about what is visual encoding and this visualisation in detail in the upcoming articles.
What if I tell you the above visualisation is created using D3.js? That’s the potential of D3.js! Hopefully, we will learn to create magic of our own too 🙂
We would be needing a few things before starting:
Note: Since D3.js is designed to make visualisations on the web, it can fetch data over the internet from a web server to show visualisations. Hence we will also be working with a small server of our own.
Step 1: Go to the directory you want to keep your D3.js project.
Step 2: Python 2.7.x users can start the server by typing the code in the command line:
python -m SimpleHTTPServer
Python 3.x users can do so by type the code in command line:
python -m http.server
You can check your python version by:
python --version
Note: For windows users, please install anaconda and then open anaconda prompt.
This should give an output similar to the following:
Step 3: Here “0.0.0.0” is your server’s IP address and “8000” is the port number. Imagine you have ordered your favorite pizza and you need to give your house address to complete the order. You will first give the Street Name and then the House Number, here IP address is analogous to street name (i.e. It identifies your computer) and Port number is analogous to the house number that helps you reach a particular house (i.e. identifies the process on your computer) in a building.
Step 4: To check if your server is working, head to your browser and type the link : http://0.0.0.0:8000/
You will get an output like this:
Congratulations, you are now on your way to build your first d3.js visualisation!
Before diving into D3, we need to know some web development basics. The web is built on three main pillars: HTML, CSS and JavaScript. Also, for a clearer understanding of D3.js, you should know the basics of SVG. We will have a crash course on these topics with some task based questions.
Note: You can skip this section if you are familiar with HTML, CSS and JavaScript. In case you have some doubts or need help, each task is followed by a stepwise guide 🙂
We have a list of four tasks to complete before we can get started with D3. Tasks 1 to 3 are based on HTML,CSS and JavaScript. Task 4 is based on SVG and related topics.
Let’s start with our first task!
For this task, you would need to :
Try doing this task on your own before checking the solution 🙂
Solution:
HTML or Hypertext Markup Language describes the structure of your web pages. HTML uses “tags”, a kind of syntax to form the skeletal structure of a web page. Specific text enclosed between angular brackets form a tag.
Let’s create our first HTML !
<!DOCTYPE html> <html> <head> <title>D3 Tutorial</title> </head> <body> <p>Hello World</p> </body> </html>
python -m SimpleHTTPServer 8000
Congratulations! You just made your first web page. Let’s get into some details:
For example :
For this task you would need to :
Try doing this task on your own before checking the solution 🙂
Solution:
CSS or Cascading Style Sheets is used to add styling and formatting to web pages. In other words, it helps to set a variety of rules(like: position, color, formatting etc.) that make the HTML look pretty. Let’s make our own web page pretty!
text-align: center; /*align our text in center*/ color: teal; /*write your favourite colour here */ font-family: Tahoma; /*change font family*/ font-size: 66px; /*set font size to 66 pixels*/
Note: The text between /**/ are comments and are ignored by the browser. They are for the programmer to keep notes of his / her code.
<p id=”myText”>Hello World</p>
<!DOCTYPE html> <html> <head> <title>D3 Tutorial</title> <style type=”text/css”> #myText{ text-align: center; /*align our text in center*/ color: teal; /*set color to teal*/ font-family: Tahoma; /*change font family*/ font-size: 66px; /*set font size to 66 pixels*/ } </style> </head> <body> <p id=”myText”>Hello World</p> </body> </html>
Note: that we use the # symbol to call id in our CSS. This tells CSS to look for an element with the id = “myText”.
CSS Selectors – Selectors are way our CSS is able to know which HTML elements does the rule applies to? We have already learned about the id selector.
Now imagine, that if we have multiple paragraphs and we want to do the same styling to each? Surely we can repeat the ids for all elements but an id is supposed to be unique. We can use a class selector to group the elements that belong to the same class of styling.
.text-beautify{ text-align: center; /*align our text in center*/ color: teal; /*set color to teal*/ font-family: Tahoma; /*change font family */ font-size: 66px; /*set font size to 66 pixels*/ }
Note: We use “.” (dot) to tell our CSS we mean a class.
<p class="text-beautify">Hello World</p> <p class="text-beautify"> My First Web Page!</p>
For this task you would need to:
Try doing this task on your own before checking the solution 🙂
Solution:
JavaScript is a programming language that adds immense functionality to a basic HTML web page. Almost all the “cool” things that you see in a website are because of the amazing power and flexibility JavaScript provides. It runs in the browser, unlike python and other programming languages you don’t need to install JavaScript, as long as you have a browser you can easily run it. Let’s add some special effects to impress our friend on his / her birthday!
We will first show a generic message to our friend and when he /she clicks on the text, surprise! Happy Birthday! Let’s code it up :
<body> <p onclick="show()" id="myText">Hey Buddy! Click here!</p> <img src="happybday.gif" id="image"> </body>
<p onclick=”show()” id=”myText”>Hello Buddy! Click here!</p>
#image{ visibility: hidden;/* hide the image */ width: 100%; /* set width of image to 100% of body */ height: 50%; /* set height of image to 50% of body */ }
<script type="text/javascript"> var show = function(){ var txt = document.getElementById("myText"), img = document.getElementById("image"); txt.innerHTML = "Happy Birthday! :D"; img.style.visibility = "visible"; } </script>
Isn’t it amazing what 5 lines of JavaScript can achieve? Let us dive into the code and understand what each line does:
var functionname = function(){ //code}
Or
functionname(){ //code}
img = document.getElementById("image");
elementvariable.style.propertyname = propertyvalue
Congratulations! So far so good, you have successfully learnt the building blocks of web! We will be learning more about them as we go ahead with our D3 journey 🙂
If you want to dig deeper into Web Technologies, you can look at W3Schools . For D3.js only basic understanding is required.
In order to build with D3 we also need a basic knowledge of SVG. The following tasks are designed to give you a brief intro into the world of SVG:
For this task you would need to :
Try doing this task on your own before checking the solution 🙂
Solution:
SVG or Scalable Vector Graphics is a format used to draw xml based graphics and animations. SVG graphics do NOT lose any quality if they are zoomed or resized.
SVG provides some basic shapes like lines, rectangles, circle, ellipse, polygon to work with. You can also create custom shapes by combining or tweaking these shapes. For extremely customisable graphics(like country maps, etc.) we use paths. A lot of these will refresh your high school geometry 🙂 Let’s go through each one:
Line
One of the simplest graphic. A line is formed when we join two points separated by a distance. There can be many such lines joining two points but we are interested in a straight line. For example, you and your friend are standing some distance apart, if you walk straight to him the path you’ll follow will make a straight line.
<!DOCTYPE html> <html> <head> <title>D3 Tutorial</title> </head> <body> <p>Hello World</p> </body> </html>
<svg width=”500” height=”500”></svg>
<svg width=”500” height=”500”> <line x1="10" y1="10" x2="140" y2="140" stroke="black"></line> </svg>
A Side Note – Coordinate space in SVG:
Those of you who know a little about coordinate system in geometry would be finding something strange with the line. The line should be like this :
This is because the origin(0,0) in SVG is at the “top-left” corner as opposed to conventional “bottom-left”.
As we move towards the right the x value increases and on moving down y value increases.
Enough of serious talk, let’s draw !
Circle
<circle r=”50” cx=”100” cy=”100” fill=”blue”>
<circle cx="100" cy="100" fill="blue" r="50" stroke="red" stroke-width="5"></circle>
<circle cx="200" cy="200" r="50" ></circle>
Groups
<svg width="500" height="500"> <g fill="blue" stroke="red" stroke-width="5"> <circle cx="100" cy="100" r="50"></circle> <circle cx="200" cy="200" r="50" ></circle> </g> </svg>
You can read more about svg here, for using D3, only basic understanding of svg is required.
D3.js is a javascript library written by Mike Bostock. It takes advantage of already established web technologies like canvas, svg to make out of the world visualisations.
D3’s name comes from the fact that it is designed to act as a driver for web documents(web page) based on the available data (json, csv, tsv etc.). In other words,
“D3’s magic helps you to create beautiful, interactive visualisations from seemingly boring data(json,csv,etc.)”
We will get to know more about it as we go deeper into D3. You can also look at some of the awesome visualisations created using D3.js 🙂
Let’s get our hands dirty with D3!
We will create a new html file , with name “barchart.html” in our D3 folder. Let us also put some basic html in our file :
<!DOCTYPE html>
<html>
<head>
<title>D3 Tutorial</title>
</head>
<body></body>
</html>
In order to use D3 we have to include it’s library, There are many ways to do so but we will stick to loading it from a url. D3 is completely written in javascript so we can include just like we include our javascript code with <script></script> tags like this:
<script src="https://d3js.org/d3.v3.js"></script>
Note: This article uses version 3 of D3.js . You can find the documentation here.
This line tells your browser to load the d3 file from the given URL. We will place this tag inside our <head> tag just below the <title> tags. We will also create another <script> tag inside our body for us to write javascript code. Our final HTML code should look like this :
<!DOCTYPE html> <html> <head> <title>D3 Tutorial</title> <script src="https://d3js.org/d3.v3.js"></script> </head> <body> <script type="text/javascript"> //We will write our D3 code here </script> </body> </html>
Our webpage looks empty, let’s add a paragraph here with D3. Write the below line between the <script> tags inside <body>:
d3.select(“body”).append(“p”).text(“Our First Paragraph using D3!”);
Save the page and head to your browser. This time you will have to use the url http://0.0.0.0:8000/barchart.html because your html file is named barchart.html. Let’s see what happened in our page:
Wasn’t that easy? Let’s try to understand our D3 code:
d3.select(“body”).append(“p”).text(“Our First Paragraph using D3!”);
When we include D3’s library, it gives us a global object d3 that we can use to call various D3 functions. Here we are asking d3 to select <body> of our DOM and append a paragraph there with the text content “Our First Paragraph using D3!”.
Let’s walk through what just happened. In sequence, we:
All of those crazy dots are just part of D3’s chain syntax. The chain syntax is possible because every time you call a D3 function on an object it performs some operations on it and returns a reference to the new object, which in turn gets picked up by the next function in the chain.This is called method chaining. Note that the above task could have also achieved by calling each function separately like we do conventionally.
Our text looks boring let us add some css styling to it and see what happens:
d3.select("body").append("p").text("Our First Paragraph using D3!").style({"font-size":"40px","font-family":"arial"});
Much better! We use .style(….) to add css styling in D3. Our chain has become too long to fit in a single line, let’s us arrange it :
var p = d3.select(“body”).append(“p”).text(“Our First Paragraph using D3!”); p.style({“font-size”:”40px”,”font-family”:”arial”});
Notice how we first created a <p> element with some text and stored in a variable p and then later used the same variable to add css styling to it. We can now use this variable whenever we want to make changes or edits to the paragraph.
What is data binding and why should I do it?
Hence you need to bind your data to the DOM elements in order to represent it as a visualisation. D3.js provides powerful ways to bind data in different forms(csv,tsv, json etc.) to the DOM.
Let’s create a data array :
var data_values = [5,10,30,8,45,24,16,55,60];
Note: In javascript an array is an equivalent of collections of R or lists of Python.
Now that we have our data, let’s bind our data to the DOM. We want to create <p> elements per data value and display it. Our new code will be like:
var p = d3.selectAll("p") .data(data_values).enter().append("p").text("hello world"); p.style({"font-size":"20px","font-family":"arial"});
Let’s see what does our browser show:
Whoa that was a lot of method chaining. We have got our text nine times on the page each as a paragraph. Let us break down the code:
Since we don’t have any paragraphs it will return an empty selection. Think of this as the selection of paragraphs that will soon exist.
This is how the entire process looked like:
First select body → from body select all the <p> elements that we are soon going to add → read the data values and count it, the following lines will be executed once for each value → create placeholders for the elements going to be added and return references to them → append elements based on the references returned earlier → add text content to each element.
We follow the similar process all throughout our code in D3. Hence it is important that you understand it thoroughly.
Did you notice that even though we got nine paragraphs, but we don’t see any data? Where did it go?
Remember the line where I said that every line following data(..) will be executed nine times, once for each value? D3 provides us a way to access each value while it iterates over them. Let’s check that value, edit the code to:
var p = d3.selectAll("p").data(data_values).enter().append("p").text(function(value){ return value; }); p.style({"font-size":"20px","font-family":"arial"});
This time instead of directly providing text content we create a function inside text(..). D3 executes this function every time it goes over our data. Each time this function is called, it is passed the value for which it is called. That means if we are on the first element of the array this function will be called with 5. We return the same value to our text(..) function so that it can add it to the text content of the paragraph. Here’s our data:
Just like html elements, creating svg with D3 is quite easy(one line easy!). For example we want to create an svg with width 500px and height 500px :
var svg = d3.select(“body”).append(“svg”).attr({“width”:450,”height”:400});
We append an <svg> like did for <p> element. The attr(..) function is used to set attributes of the svg. Here we want to set width and height of the svg.
Let us draw a circle in this svg. We want to create a circle of radius 30 and centre at (50,50):
svg.append(“circle”).attr({“r”:”30px”, “cx”:”50px”, “cy”:”50px”});
Notice how we used append(..) to add a circle to our svg.
Let us add fill to our circle and increase the radius:
svg.append(“circle”).attr({“r”:”50px”, ”fill”: “red” , “cx”:”50px”, “cy”:”50px”});
You can see that the same attributes we learnt in the svg work here. D3 provides an easy interface to create SVGs programmatically.
Simple Barchart
A bar chart is a graphic that presents grouped data with rectangular bars. Here lengths of bars are proportional to the values they represent.
Drawing bars
We will be taking advantage of SVG rectangle to make rectangles for the bar chart. A rectangle has four properties : Height, Width, x start position, y start position. Let’s create a rectangle using D3:
svg.append("rect").attr({“x”:”30px”, “y”:”30px”, "width":"30px", "height":"100px"});
Notice how the (x,y) control the starting point of the rectangle. Let us add colour to our bar but before we will store our rectangle in a variable so that we can reuse it again. Let’s do that:
var bars = svg.append(“rect”).attr({“x”:”30px”, “y”:”30px”, "width":"30px", "height":"150px"}); bars.attr(“fill”, ”blue”); //fill the rect with blue
Let us create bars based on our data values. We will use the same process that we used to bind data earlier. Our bars variable will change to:
var bars = svg.selectAll("rect").data(data_values).enter().append("rect").attr("width","25px").attr("height", function(d){ return d; });
We have chosen a fixed width of 25 pixels for each bar and we are setting height based our data_values array. Let us see how the graphic looks:
Weren’t we supposed to get nine rectangles? We have nine rectangles but since we have not given different x values they all lie on the same position. Let’s add different x values to our bars:
bars.attr(“x”, function(d,i){ return i*30; });
We already know D3 passes the current data value while iterating through our data. Along with the current value it also passes the current value’s index. This index helps us keep track of the iteration count. What we are doing is we are giving i*30 as x value for our bars. This is because the i will denote the index of the current value and 30 is there because we know each bar’s width is 25 pixels so we give a padding of 5 pixels between each bar.
Let’s refresh our page and we got our 9 data bars !:
Isn’t our bars a little funky? Let’s scale our heights a little bit. Let’s multiply d by 5 in our height function:
.attr(“height”, function(d){ return d*5; });
Much better, but why are the bars upside down?
Remember that SVG Coordinate space is different as compared to conventional? Here Origin is at the top left corner so our y-axis starts from the top. Let’s adjust this so that we get a proper bar chart, we can change this by subtracting the height of each bar from the total height of the svg(400px).
bars.attr(“y”, function(d){ return 400-d*5; });
Let’s give a nice color to our bars:
bars.attr(“fill”, “steelblue”);
Let’s add some more values to our data_values array , it should now look like this:
var data_values = [5,10,30,8,45,24,16,55,60,45,32,18,11,3];
Do you think this visualisation is efficiently conveying the information? No, there are many things that can be improved so that we are able to convey inferences in as efficient manner as possible. One such improvement can be :
“Highlighting min and max bars so that as soon as we put our gaze on the visualisation we get an idea of the two extremes, we won’t have to go through each bars and hence this will save our time.”
Let’s do that!
We first need to find the two extremes in our dataset. Lucky for us, D3 provides d3.max(..) and d3.min(..) functions that we can use:
var max = d3.max(data_values); var min = d3.min(data_values);
We will have to now select those two bars two have max and min value respectively , for that we’ll use D3’s filter(..) method. What this will do is select only those bars that match the filter. In this case we will have filter for maximum value. We will add “green” colour fill to the bar:
bars.filter(function(d){ return d==max; }).attr(“fill”,”green”);
Similarly for the smallest bar, we will add “red” fill colour:
bars.filter(function(d){ return d==min; }).attr(“fill”,”red”);
There we have our simple barchart in less than 20 lines of code!
Congratulations for coming this far, we have covered a lot of D3 basics these will help you pursue your own adventures someday! D3.js is a very powerful library and we have just scratched the surface. There is still a complex but immensely useful functionality it has to offer; that is making better charts, visualisations loaded with animations and interactivity.
I have included all the code of this article is available on github. Also check out these small visualisations that I created for fun to get a glimpse of some cool stuff we can do by knowing just the basics of D3 :
I urge you to try them out on your own. Good luck!
You can check out my next article on D3.js here
Great article, Rizvi Machine learning is the great for modern time and giving the information in this article is great thanks for sharing and keep posting
Great work Sanad,
Very nicely written with lots of graphics for better understanding.
Thanks for your comment!