D3.js – Data-Driven Documents – Web Standards

View Github Repository Open presentation in a new window

Yonet

See all presentation from Yonet

D3.js – Data-Driven Documents – Web Standards

3 3

d3-codeclass

D3.js workshop

On Github Yonet / d3-codeclass

D3.js

Data-Driven Documents

Slides : bit.ly/d3-codeclass

Aysegul Yonet / @AysegulYonet

AnnieCannons.com

What is D3?

D3 allows you to bind data to the DOM, and then apply data-driven transformations to the document. Manupulates attributes of DOM elements with the attributes of data items.

What d3 is not?

Google Charts Example

D3 Scatterplot Example

Not a prototyping tool. For ex: Google charts has chart.draw(data, options). With d3, you have to define how the chart is rendered with the use of d3 methods.

Examples

D3 is very Flexible, It allows you to not just create basic charts but create anything you like with the use of svg. Open Sourced, Great community.

What you will learn

  • SVG
  • Selections
  • Update Pattern
  • Scales
  • Axis
  • Loading External Data
  • Working with Data methods
  • Working with Geospatial data
  • Creating maps
  • Angular and d3.js

I will talk to you about...

Web Standards

  • HTML
  • CSS
  • SVG

SVG

  • Another HTML element, except you can not put another HTML element inside an SVG.
  • Does not support common HTML attributes or styles such as position, left, top, right, bottom or float.
  • Default svg size is browser-dependant.
d3 works with html and css and shines through with the use of SVGs. What is an SVG? SVG default size is implemented differently by various browsers. Make sure to define the Width and Height when you are creating an SVG.

Circle

<circle cx="250" cy="100" r="30" fill="#25B0B0">
                            </circle>
Let's take a look at circle. Notice we are calculating an area of a circle each time we draw a circle using these attributes. That means we have to be aware of the performance issues.

Rectangle

<rect x="200" y="50" width="100" height="100" rx="10" ry="0" fill="#25B0B0">
                            </rect>
Interesting fact: if a properly specified value is provided for rx but not for ry (or the opposite), then the browser will consider the missing value equal to the defined one.

Group

<g transform="translate(100, 100)">
                                <circle cx="20" cy="10" r="30" fill="rgb(255, 0, 0)">
                                </circle>
                                <circle cx="100" cy="10" r="30" fill="#25B0B0">
                                </circle>
                            </g>

jsFiddle

Any transformations (positioning, scaling, rotating) that are applied to the group element will also be applied to each of the child elements.

Line

<line x1="10" y1="10" x2="100" y2="100" stroke="blue" stroke-width="100">
                            </line>

Path

<path d="
	M50,100
	L131,66
	L259,115
	L339,50
	L400,98
	M350,150
	L100,150" fill="none" stroke="#25B0B0">
                            </path>
  • M - move.
  • L - line.
  • z - close path.

Text

<svg>
                                <text x="100" y="100" fill="red" font-family="arial" font-size="16">
                                    Look at me, I am a text.
                                </text>
                            </svg>
Unless you set the style of the SVG text, it inherits font-styles from CSS.

Selection

jQuery

var paragraphs = $("div p");

D3

var paragraphs = d3.select("div").selectAll("p");
D3 uses css selectors to define a selection. The selection can be based on tag, class, identifier, attribute, or place in the hierarchy. Child nodes might not exist at the time of our selection. One thing to note, D3 selector object only contains the elements that matched the selection rule when the selection was first created. 
var p = d3.select("body").append('p');
                        p.html('Hello CodeClass!').attr('style', 'color: red;');

jsFiddle

Once we have a selection, we can operate on our nodes.

Data

d3.selectAll("p")
                        .data([0, 1, 1, 2, 3, 5, 8, 13, 21, 34])

JSFiddle

Most importantly we can bind data to our selections. When data is bound to a selection, each element in the data array is paired with the corresponding DOM node in the selection. We can use the data to define the attributes on that node

Enter()

d3.select("body").selectAll("p")
                        .data([3, 7, 21, 31, 35, 42])
                        .enter();
If there are fewer nodes than data, the extra data elements form the enter selection, which you can instantiate by appending to the enter selection.

Append()

d3.select("body").selectAll("p")
                        .data([3, 7, 21, 31, 35, 42])
                        .enter().append("p")
                        .text(function(d) { return "I’m number " + d + "!"; })
                        .style("color", "black");

D3 methods returns the selection they act upon, and we can apply multiple operations to the same selection. Here we are able to add text to our newly appended selection.

JSFiddle

Our new paraghraph's text is created using the data bound data points.

Exercise

http://jsfiddle.net/w5qvpy7v/150/ solution: http://jsfiddle.net/yonet3d/w319gwyw/

Exit() and Remove()

var bars = d3.select('body').selectAll('p').data(newData);

                        //if we have more data, add new 'p' for those data items
                        bars.enter().append('p')

                        //if we have less data points, remove the 'p' that no longer have a data pairing.
                        bars.exit().remove();
What if we have less data points? Using D3’s exit and remove, you can get rid of outgoing nodes that are no longer needed. Like the enter method, exit selects the extra nodes that doesn't have data matching and remove deletes the nodes from our DOM.

Update Pattern

// Update…
                        var p = d3.select("body").selectAll("p")
                        .data([3, 7, 21, 31, 35, 42]);

                        // Enter…
                        p.enter().append("p")

                        // Exit…
                        p.exit().remove();

JSFiddle

This is what we call the update pattern. If you forget about the enter and exit selections, you will automatically select only the elements for which there is corresponding data. By handling these three cases separately, you specify precisely which operations run on which nodes.

Chaining

d3.select("body").selectAll("p")
        .data([3, 7, 21, 31, 35, 42])
        .enter().append("p")
        .text(function(d){return d;})
        .exit().remove();

jsFiddle

If you have been to the d3's website, I am sure you came accross the chained methods. Every chained method operates on the previous selection, and most operations return the same selection. Some methods return a new one! For example, selection.append returns a new selection containing the new elements. This conveniently allows you to chain operations into the new elements. Since method chaining can only be used to descend into the document hierarchy, use var to keep references to selections and go back up.

Operating on selection

  • Setting attributes or styles
  • Registering event listeners
  • Adding, removing or sorting nodes
  • Changing HTML or text content

JSFiddle

You can operate over the nodes in a declarative way using selector methods.

Transition

circles.attr("r", "0")
                        .attr({
                        'cx': function(d) {return (d * 2); },
                        'cy': 10
                        })
                        .transition()
                        .duration(750)
                        .delay(function(d, i) { return i * 100; })
                        .attr("r", 10);

JSFiddle

Transition will happen between what you have assigned before and after the transion is called.

Exercise

  • Update the bar graph with new data
  • Add transitions to see the the enter and exit phases.
  • Refactor your code to have an update method that you can call with the new data.
solution 1: http://jsfiddle.net/ubmk6j27/130/ solution 2: http://jsfiddle.net/ubmk6j27/131/ solution 3: http://jsfiddle.net/ubmk6j27/132/

Scales

var scale = d3.scale.linear();

Examples we saw so far calculates the position of our SVG elements by using data. D3 comes with scale methods to make these mappings easy. Scales transform a dataset in a certain interval (called the domain) into a number in another interval (called the range). For ex, transforms a domain in miles to a range in pixels.

Quantitative Scale

  • Linear scales
  • Logarithmic scales
  • Power scales
var xScale = d3.scale.linear()
                        .domain([0, d3.max(data)])// your data minimum and maximum
                        .range([0, 420]);//the pixels to map to
                        d3.select(".chart")
                        .selectAll("div")
                        .data([3, 7, 21, 31, 35, 42])
                        .enter().append("div")
                        .style("width", function(d) { return xScale(d) + "px"; })
JSFiddle Scale func creates a new instance of a scale. On our new scale, we can create new properties that persists. Quantitative: numarical. 
var xScale = d3.scale.linear()
                        .domain([0, 1000])
                        .range([0, 200]);
                        console.log(xScale(500)); //100
                        console.log(xScale.domain());//[0, 1000]
Domain and range is a setter and getter. If you don't pass any arguments it is returning the values. 

var xScale = d3.scale.linear()
                        .domain([2, 3, 5, 7, 11, 13, 17, 19, 23, 29])
                        .range([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);

JsFiddle

Clamp

var xScale = d3.scale.linear()
                        .domain([2, 3, 5, 7, 11, 13, 17, 19, 23, 29])
                        .range([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
                        .clamp(true);

jsFiddle

Defaults to false.

Ordinal Scale

Ordinal Scale have a discrete domain, such as a set of names or categories.

var xScale = d3.scale.ordinal()
                        .domain(["Bob", "Stuart", "Kevin", "Scarlet"])
                        .rangePoints([0, 100]);
                        console.log(xScale("Stuart"));//33.333333333333336

                        xScale.range(); // [0, 33.333333333333336, 66.66666666666667, 100]

JSFiddle

You need to have the same number of arguments. If you have more in range, that is OK. RangePoints subdivides everything in between the values you give to it.

RangeBands 

var o = d3.scale.ordinal()
                        .domain([1, 2, 3])
                        .rangeBands([0, 100]);

                        o.rangeBand(); // 33.333333333333336
                        o.range(); // [0, 33.333333333333336, 66.66666666666667]
                        o.rangeExtent(); // [0, 100]
Sets the output range from the specified continuous interval. This interval is subdivided into n evenly-spaced bands, where n is the number of (unique) values in the input domain. The bands may be offset from the edge of the interval and other bands according to the specified padding, which defaults to zero. 

var x = d3.scale.ordinal()
                        .domain(["A", "B", "C", "D", "E", "F"])
                        .rangeRoundBands([0, width], .1);

Like rangeBands, except guarantees that range values and band width are integers so as to avoid antialiasing artifacts.

Color Categories

d3.scale.category10()

JSFiddle

Constructs a new ordinal scale with a range of ten categorical colors:

Time Scales

d3.time.scale()
                        .domain([dates])
                        .range([values]);
Constructs a new time scale with the default domain and range; the ticks and tick format are configured for local time. TODO: Dates and times can be represented using numbers for days, years,... or specific date-time encoding. ISO 8601.

d3.format

d3.time.format("%Y-%m-%d"); //2015-08-26
                                d3.format("+,%"); //+2,400%
JSFiddle d3.time.format - creates a new local time formatter for a given specifier.

d3.time.scale - constructs a linear time scale.

var xScale = d3.time.scale()
                            .domain([2009-07-13T00:02, 2009-07-13T23:48])
                            .rangeRound([0, width]);
                            //rangeRound does the same thing as range but rounds the values to integers or begining of
                            dates.
JSFiddle

d3.time.intervals - a time interval in local time.

  • d3.time.hour
  • d3.time.week
  • d3.time.monday
  • d3.time.year

Example

Scatterplot

var svg = d3.select('body').append('svg')
                            .attr({
                            'width': 250,
                            'height': 250
                            });

                            var xScale = d3.scale.linear()
                            .domain([0, 5])
                            .range([0, 200]);

var render = function(data, r, color){
                            //Bind data
                            var circles = svg.selectAll('circle').data(data);

                            //Enter and update
                            circles.enter().append('circle')
                            .attr({
                            'cx': function(d){return xScale(d)},
                            'cy': 30,
                            'r': r,
                            'fill': function(){return color || 'red'}
                            });
                            }
var oldData = [1, 2, 3];
                            var newData = [1, 2, 3, 4, 5];
                            render(oldData, 10);
                            render(newData, 15, 'blue');

How many people think the circles all should be blue?

jsFiddle

var render = function(data, r, color){
                            //Bind Data
                            var circles = svg.selectAll('circle').data(data);
                            //Enter
                            circles.enter().append('circle')

                            //Update
                            circles.attr({
                            'cx': function(d){return xScale(d)},
                            'cy': 30,
                            'r': r,
                            'fill': function(){return color || 'red'}
                            })
                            }
I am using a function to return a default value if the color is not defined.

jsFiddle

var oldData = [1, 2, 3, 4, 5];
                            var newData = [1, 2, 3];
                            render(oldData, 10);
                            render(newData, 15, 'blue');

jsFiddle

var render = function(data, r, color){
                            //Bind Data
                            var circles = svg.selectAll('circle').data(data);

                            //Enter
                            circles.enter().append('circle')

                            //Update
                            circles.attr({
                            'cx': function(d){return xScale(d)},
                            'cy': 30,
                            'r': r,
                            'fill': function(){return color || 'red'}
                            })

                            //Remove
                            circles.exit().remove();
                            }

jsFiddle

Recap

  • Bind Data.
  • Enter and Append.
  • Update the attributes.
  • Exit and Remove.
TODO add scale to scatter plot.

Exercise

Solution

Chart Components

Generators: take Array Values and create SVG drawing code for the d attribute of path elements. Components:takes functions like Scale() and create element or event listeners. Layouts: takes whole datasets ==> New datasets with attributes for layout.

Line Graph

var line = d3.svg.line()
                            .interpolate("basis")
                            .x(function(d) {
                            return x(d.year);
                            })
                            .y(function(d) {
                            return y(d.stat);
                            });

jsFiddle

Axis

var xAxis = d3.svg.axis()
                            .scale(xScale)
                            .orient("bottom");

jsFiddle

Ticks

xAxis.ticks(5);
                            xAxis.tickValues([1,2,4]);
                            xAxis.ticks(d3.time.month, 2);

jsFiddle

jsFiddle

Loading External Resources

  • d3.json(url[, callback])
  • d3.csv(url[, accessor][, callback])
  • d3.tsv(url[, accessor][, callback])

CSV

day,donut
                            Monday,91
                            Tuesday,23
                            Wednesday,7
                            Thursday,4
                            Friday,82
                            Saturday,39
                            Sunday,27

var x = "day";
                            var y = "donut";

                            var accessor = function(d) {
                            d[y] = parseInt(d[y]);
                            return d;
                            }

                            d3.csv("data/data.csv", accessor, function(d) {
                            console.log(d);//Array of objects [{day: "Monday", donut: 91},...]
                            render(d);
                            });

jsFiddle

Exercise

Download Files Here

  • Read the census.csv file inside the data folder.
  • Display or console.log the data from the file.

Working with Arrays

  • d3.min/d3.max
  • d3.reduce
  • d3.keys - lists the keys of an associative array.
  • d3.merge - merges multiple arrays into one array.
  • d3.nest - groups array elements hierarchically.
  • d3.quantile(numbers, p)
  • d3.deviation(array[, accessor]) - returns the standard deviation

jsFiddle

Reduce

[0, 1, 2, 3].reduce(function(a, b) {
                            return a - b;
                            });//-6

                            [0, 1, 2, 3].reduceRight(function(a, b) {
                            return a - b;
                            });//0

left-to-right

d3.nest

var donutData = [
                            {day: 'Monday', donut: 34, variety: "plain"},
                            {day: 'Tuesday', donut: 41, variety: "glazed"}
                            ];

var nestedData = d3.nest()
                            .key(function(d){ return d.day;})
                            .entries(donutData);

                            [{
                            key: "Monday",
                            values: [
                            {day: 'Monday', donut: 34, variety: "plain"}
                            ]},
                            {
                            key: "Tuesday",
                            values: [
                            {day: 'Tuesday', donut: 41, variety: "glazed"}
                            ]}
                            ]

var nestedData = d3.nest()
                            .key(function(d){ return d.day;})
                            .key(function(d){ return d.variety;})
                            .entries(donutData);

                            [{
                            key: "Monday",
                            values: [
                            { key: "plain", values: [{day: 'Monday', donut: 34, variety: "plain"}]}
                            ]},
                            {
                            key:"Tuesday",
                            values: [
                            { key: "glazed", values: [{day: 'Tuesday', donut: 41, variety: "glazed"}]}
                            ]
                            }
                            ]

Exercise

TODO load data and change format. Find some data with time, so I can use it for time formatting in the next slides.

Maps

GeoJson

{ "type": "FeatureCollection",
                                "features": [
                                {
                                "type": "Feature",
                                "geometry": {
                                "type": "Polygon",
                                "coordinates": [[125.6, 10.1], [124.4, 9.2],...]
                                },
                                "properties": {
                                "name": "Dinagat Islands"
                                }
                                }
                                ]

                                }

Let's make a Map

GeoJSON is a format for encoding a variety of geographic data structures. It is designed to represent discrete geometry objects grouped into feature collections of name/value pairs. Each feature in a feature collection is a JSON object that stores the border data in a coordinates array. Also some metadata in a properties object.

TopoJson

<script src="http://d3js.org/topojson.v1.min.js"></script>

GeoJson to TopoJson convertor

TopoJSON is an extension of GeoJSON that encodes topology. Rather than representing geometries discretely, geometries in TopoJSON files are stitched together from shared line segments called arcs. TopoJSON eliminates redundancy, offering much more compact representations of geometry than with GeoJSON; typical TopoJSON files are 80% smaller than their GeoJSON equivalents. In addition, TopoJSON facilitates applications that use topology, such as topology-preserving shape simplification, automatic map coloring, and cartograms.

How to steal the data

Map Example

http://bl.ocks.org/mbostock/4060606

Projection

var projection = d3.geo.albersUsa()
                                .scale(500)
                                .translate([width / 2, height / 2]);

Scale and transform is different for every projections type. 

var worldProjection = d3.geo.mercator();
                                var geoPath = d3.geo.path().projection(worldProjection);

d3.select('svg').selectAll('path').data(countries.features)
                                .enter()
                                .append('path')
                                .attr('d', geoPath)
                                .attr('class', 'countries');

main.js 

d3.select('svg').selectAll('path').data(topojson.feature(us,
                                us.objects.states).features);
Returns the GeoJSON MultiLineString geometry object representing the mesh for the specified object in the given topology. This is useful for rendering strokes in complicated objects efficiently, as edges that are shared by multiple features are only stroked once.

Drawing on a map

<script src="http://d3js.org/queue.v1.min.js"></script>
                                queue()
                                .defer(d3.json, 'states.json')
                                .defer(d3.json, 'cities.json')
                                .await(makeMyMap);

Interactivity

var centered;
                                g.on("click", clicked);
                                function clicked(d) {
                                var x, y, k;

                                if (d && centered !== d) {
                                var centroid = path.centroid(d);
                                x = centroid[0];
                                y = centroid[1];
                                k = 4;
                                centered = d;
                                } else {
                                x = width / 2;
                                y = height / 2;
                                k = 1;
                                centered = null;
                                }

                                g.selectAll("path")
                                .classed("active", centered && function(d) { return d === centered; });

                                g.transition()
                                .duration(750)
                                .attr("transform", "translate(" + width / 2 + "," + height / 2 + ")scale(" + k +
                                ")translate(" + -x + "," + -y + ")")
                                .style("stroke-width", 1.5 / k + "px");
                                }

Example

TODO zooming in/out, panning.

Other mapping libraries

Good News!

Other resources

Angular and d3

bit.ly/AngularU-d3

THE END

BY Aysegul Yonet / aysegul.yonet@autodesk.com

Slides : bit.ly/d3-codeclass