Hello there

I am Roberto.

Here my websites, if you wanna have a look at my work

otlabs.net
rob.otlabs.net

Started working with these guys

Then i moved to London, 15 years ago. ish

And it was mainly advertisement, gaming, campaigns. A lot of fun

Now I do dataviz for Pirelli.

Not so fun

Well, at least not so fun like this

What is D3

D3.js is a set of tools, in JavaScript for manipulating documents based on data.

D3 comes from Data Driven Documents

3 D :)

D3 helps you bring data to life using HTML, SVG and CSS. and Canvas.

the author: Mike Bostock

HOW

HTML

SVG

Scalable Vector Graphics

SVG are described by functions

size agnostic where raster ( jpeg ) are size dependant

SVG are one of the best solution for responsive design

Canvas. what is it

HTML Canvas is a tag, first introduced by Apple, in 2004

the Canvas tag heps to draw graphics, on the fly, via scripting (usually JavaScript).

What can you do

D3 Gallery

Why D3

It works on the web
It is extremely flexible

(data manipulation, binding, render)

It is not opinionated (compared to highcharts etc)

From v4 is modular, so you can use just what you need

It is lightweight compared to plotly (220kb vs 2.5Mb)

It is well documented and great community

Ah, forgot. it is open source

quoting mr Mike Bostock about D3:

d3 espouses abstractions that are useful for any visualization application and rejects the tyranny of charts.

So far

What is D3

How to use D3

What can you do with D3

Why use

The Three Pillars

Data manipulation

Data Binding

Data Rendering

codePen

d3 draw a circle

						
let svg = d3.select("#d3-container").append("svg")
 .append("circle")
  .attr("cx", (width * .5) / 2 )
  .attr("cy", height * .5)
  .attr("r", 200)
  .style("fill", "steelblue");
codePen

Binding Data

						
const mydata = [5, 3, 2, 8];
svg.selectAll('circle')
 .data(mydata)
 .enter()
 .append("circle")
 .attr("cx", (width * .5) / 2)
 .attr("cy", (d, i) => {
	 return i * 100 + 100
 })
 .attr("r", (d) => {
	 return 10 * d;
 })
 .style("fill", "yellow");

Before you can change or modify any elements in d3 you must first select them.

you can use d3.select() or d3.selectAll()

An array will be returned
Natural transition

Brain use more energy if things just appear on screen, without a natural explanation

So we need to move things following natural process
codePen

Animate Data

						
let t = d3.transition()
 .ease(d3.easeCubic)
 .duration(1500);
...
** bind data
 .data(mydata)
 .enter()
 .append("circle")
 ...
 .attr("r", (d) => {
	 return 0 * d;
 })
 .transition(t)
 .attr("r", (d) => {
	 return 10 * d;
 })
We apply a transition to the animation. We set an initial state for r and then we apply the transition and the final r
d3.ease

Data fill space (I)

						
							// random array of numbers.
							// random array of colors
const mydata = [25, 25, 40, 10];
const mycolor = ['yellow', 'red', 'green', 'black'];

Data fill space (II)

						
							// we bind data
svg.selectAll('rect')
	.data(mydata)
	.enter()
	.append("rect")
We bind data

Data fill space (III)

						
	.attr("height", 0)
	.style("fill", (d, i) => {
		return mycolor[i];
	})
	.transition(t)
	.attr("height", (d, i) => {
		let perc = (mydata[i] * 1) / sum;
		return height * perc;
	});
We dynamically assign a percentage to fill the entire screen, in height
codePen

Data fill space 2

						
.attr("y", (d, i) => {
	let returnY = 0;
	let percTotal = 0;
	for (let j = i - 1; j > -1; j--) {
		percTotal += (mydata[j]) / sum;
	}
	returnY = height * percTotal;
	return returnY
})
.attr("width", (d) => {
	return width;
})
We fill the entire available space
codePen

Data fill space 3

						
.transition()
	.delay((d, i) => {
		return 300 * i;
	})
	.duration(1000)
	.attr("width", (d) => {
		return width;
	})
Here we fill the space and animate width a bit more organically

Let's interact with data

First, let's see how to write text using d3
codePen

Text using d3

						
svg
	.append("text")
	.attr('id', 'id-used-to-retrieve-the-text')
	.text('CIAO BELLA')
	.style('fill', 'black')
	.style('opacity', 1);
codePen

Mouse Text Interaction

						
.on('mouseover', (d, i, nodes) => {
 let rect = nodes[i].getBoundingClientRect();
 let text = d3.select(nodes[i])
  .append("text")
  .attr('id', 't' + i)
  .text(d)
  .style('opacity', 1);

})
.on('mouseout', (d, i, nodes) => {
	d3.select('#t' + i).remove();
})
on mouse over we show the value that gives the percentage of the screen fill
codePen

Update Data (I)

						
const mycolor = ['yellow', 'red', 'green', 'pink', '#ff0099', '#ffff99', '#DDCCFF', '#c9fa05'];
setInterval(() => {
 let mydata = [];
 for(let i = 0; i < 10; i++) {
	 mydata.push(3 + Math.floor(Math.random() * 40));
 }
 renderChart(mydata);
}, 2000);
codePen

Update Data (||)

						
	let rects = svg.selectAll('rect')
		.data(mydata)

	//**REMOVE

	//** UPDATE

	//**ADD
codePen

Update Data: #REMOVE

						
  let rects = svg.selectAll('rect')
    .data(mydata)

  //**REMOVE
  rects
    .exit()
    .remove()

  //** UPDATE

  //** ADD
codePen

Update Data

						
  let rects = svg.selectAll('rect')
    .data(mydata)

  //**REMOVE
	...

  //** UPDATE
  rects
    .attr("x", 0)
    .attr("y", (d, i) => {
      let returnY = 0;
      let percTotal = 0;
      for (let j = i - 1; j > -1; j--) {
        percTotal += (mydata[j]) / sum;
      }
      returnY = height * percTotal;
      return returnY
    })
    .attr("width", (d) => {
      return width;
    })
    .attr("height", (d, i) => {
      let perc = (mydata[i] * 1) / sum;
      console.log('update ?perc ', perc);
      return height * perc;
    })

  //**ADD
	...
codePen

Update Data: #ADD 

						
let rects = svg.selectAll('rect')
 .data(mydata)

//**REMOVE
...

//** UPDATE
...

//**ADD
rects
 .enter()
 .append("rect")
 .attr("x", 0)
 .attr("y", (d, i) => {
	let returnY = (calc y based on perc)...
	return returnY
 })
 .attr("height", (d, i) => {
	let perc = (mydata[i] * 1) / sum;
	console.log('add ?perc ', perc);
	return height * perc;
 })
 .attr("width", (d) => {
 	return width;
 })
}

So far we have seen

Basic rendering in d3, and binding data

animate data

d3 data model: add, update, remove data

Now let's talk about d3 scale

“Scales are functions that map from an input domain to an output range.”

We think about dimensions as spatial and quantitative

But when working with abstract data

there are ordinal data or categorical

Let's consider fish for example.

The order could be:

by weight (quantitative)

by quality (good very good)

or categorical (type of fish)

So scales are functions that take abstract values and convert to a visual value (like x, y position)

d3.scale expl
d3.scale

Test scale

							

let x = d3.scaleLinear()
 .domain([100, 500])
 .range([0, 960]);

 console.log(x(100));//0
 console.log(x(250)); // 360

We declare the type of scale (linear in this case)

We set the input (Domain)

We set the the output (range)

codePen

Test Scale

						
const mydata = [15, 3, 2, 8];
let xScale = d3.scaleLinear()
 .domain([d3.min(mydata), d3.max(mydata)])
 .range([0 + 100, height - 100]);
 ...
 .attr("cy", (d, i) => {
	 return xScale(d)
 })
We map an unordered array, and we display content, based on their value
codePen

Let's scale to a circle!

						
let scale = d3.scaleLinear()
 .domain([d3.min(mydata), d3.max(mydata)])
 .range([0, Math.PI*2]);

svg.selectAll('circle')
	.data(mydata)
	.enter()
	.append("circle")
	.attr("cx", (d, i) => {
		return Math.cos(scale(i)) * r + c.x;
	})
	.attr("cy", (d, i) => {
		return Math.sin(scale(i)) * r + c.y;
	})
We map an array to a circle

Scales and colors

							
let color = d3.scaleLinear()
  .domain([10, 100])
  .range(["yellow", "pink"]);

We will scale colors from yellow to pink

codePen

scale colors

						
let scaleColor = d3.scaleLinear()
 .domain([d3.min(mydata), d3.max(mydata)])
 .range(['yellow', 'red']);
 ...
 .style("fill", (d) => {
	 return scaleColor(d);
 })
We map an array to a circle, using scale color

Ok. now let's apply scale to a bar chart

						
const data = [
  {key: "porsche",     value: 132},
  {key: "ferrari",      value: 71},
  {key: "lambo",      value: 337},
  {key: "lotus",  value: 93},
  {key: "alfaromeo",      value: 78}
];

let xScale = d3.scaleBand()
.domain(data.map(function(d){
      return d.key;
  }))
.range([0, 200]);

let yScale = d3.scaleLinear()
.domain([
  d3.min(data, (d) => {
    return d.value
  }),
  d3.max(data, (d) => {
    return d.value
  })
])
.range([0, 500]);

console.log('xScale ', xScale('ferrari'));
console.log('yScale ', yScale(337));
We have a scaleBand for xAxis, using ordinal data (map keys to range 0 - 200)
and linear for the height (map values to a range 0 - 500)

codePen

Barchart and scale

						
let xScale = d3.scaleBand()
  .domain(data.map(function(d){
        return d.key;
    }))
  .range([0, width / 2])
  .padding([0.1]);

So, here we create the scale for x ( scaleBand)

codePen

Barchart and scale

						

let yScale = d3.scaleLinear()
 .domain([
 d3.min(data, (d) => {
  return d.value
 }),
 d3.max(data, (d) => {
  return d.value
 })
])
.range([0, height - 100]);
the scale for y ( scaleLinear)
codePen

Barchart and scale

						
let scaleColor = d3.scaleLinear()
 .domain([
 d3.min(data, (d) => {
  return d.value
 }),
 d3.max(data, (d) => {
  return d.value
 })
])
.range(['green', 'red']);
And a scale for color, to highlight, in red, the highest

D3 offers a set of functions to create axis

Axis

Or Tooltip

Grouping data

one of the most powerful things, in d3, is the tools to group and nest data

Let's consider this json

							
const data = [
  {
    "car": "Porsche",
    "model": "P1",
    "maxSpeed": "300",
    "color": "black",
  },
  {
    "car": "Ferrari",
    "model": "testa-rossa",
    "maxSpeed": "320",
    "color": "red",
  },
  {
    "car": "Ferrari",
    "model": "testa-nera",
    "maxSpeed": "320",
    "color": "red",
  },
  {
    "car": "Fiat",
    "model": "Uno",
    "maxSpeed": "120",
    "color": "white",
  },
  {
    "car": "Fiat",
    "model": "Uno-turbo",
    "maxSpeed": "122",
    "color": "red",
  },
  {
    "car": "Fiat",
    "model": "Uno-turbo",
    "maxSpeed": "125",
    "color": "black",
  },
  {
    "car": "Porsche",
    "model": "PP",
    "maxSpeed": "290",
    "color": "black",
  }
]

d3.nest

						
let parsedResults = d3.nest()
	.key(function(d) { return d.car ; })
	.entries(dataNesting);
By Car
By Color
By Model

So, this is great for visualize data

But we can do more

d3.nest + d3.rollup

						
let parsedResults = d3.nest()
	.key((d) => {
		return d[value] ;
	})
	.rollup((v) => {
		return v.length;
	})
	.entries(dataNesting);
By Car
By Color
By Model

So far we have seen

Scales (for colors, for positions)

Bar chart example

Array manipulation (nest, rolling)

So, Performance.

Things can go really nasty if we have to plot a lot of data on screen.
Worst if we need to animate them

Try to plot that

in html...
or even worst, try to animate it

Your framerate per second will go close to zero.

Something from uber

here the live link
deck.gl

variable.io

here the link to the project

So, what's the secret here?

canvas / WebGL, (that run on canvas)

WebGL enables web content to use an API based on OpenGL ES 2.0 to perform 3D rendering in an HTML, using Canvas

codePen

Pixi simple render

						
d3.selectAll('circle')
 .data(mydata)
 .enter()
 .append('custom')
 .each((d,i) => {
  let semicircle = new PIXI.Graphics();
	...
});

Let's go a bit crazy

						
if (times) {
 intervalId = setInterval(() => {
 let dd = [];
 for (let i = 0; i < 10; i++) {
  let rn = Math.floor(Math.random() * 50 + 10);
  dd.push(rn);
 }
 executeSimpleRender(dd, 1);
 }, 500);
}

running timeline

						
d3.selectAll('circle')
 .data(data)
 .enter()
 .append('custom')
 .each(function(d,i) {
  ... pixi render ...
}
example with topojson
example with Three.js

Here a few example we coded with this approach

airbnb
frame
Thanks for your patience