D3.js Getting started (again..)

The same questions as some years back: How to get started with it ?

The learning curve for beginners is certainly a bit steep if you dont have a basic background in at least HTML, CSS and JavaScript, in this case you rather go for a visualization tool like Tableau if you look for fast results.
The good news: There is plenty of material out there about D3, books, websites, tutorials. I purchased a couple of books from Packt Books and O’Reilly as PDF version, I think they have the most books, though most of them based on earlier D3 versions.

Packt Books

Packt Books

O'Reilly Books

O’Reilly Books

I wont recommend a specific one, most follow the same principles, a short intro to the DOM, explain the D3 basics followed by more or less complex samples. As with any other technology or programming language you learn most hands-on, there so many examples and good tutorials available. I never attempted to understand everything, understand the concepts, dissect existing visualizations and get creative !

Continue reading

D3.js Data-Driven-Documents Revisited

I came across this fantastic JavaScript library for DOM manipulation in 2011, at that time it was still fairly new in the visualization space and I picked it up for a couple of visualizations moving away from prefuse and other thick client embeded libraries. Now 5 years later, it matured massively and got adopted at large scale. Just recently the version 4.0 was releasey. One of the main features certainly the modularit

D3.js Webesit

D3.js Website and

I will revisit some of the old visualizations I created and also try to convert some newer ideas into visualizations. Now tinkering and playing became much easier, I recommend JSFiddle if you want to avoid local web server setup, though JSFiddle does not support version 4.0 yet (at the time of writing this).

JSFiddle

JSFiddle

You also can run your visualization with the good old GitHub Gist and the bl.ocks site or the browser plugins.

I am working with D3 again after experimenting with visualization tools like Tableau and realizing quickly that outstanding visualizations only can be created with the power of D3, though you have to work with sourcecode and understand whats going on under the hood of a modern web browser. Please dont forget D3 is released under the BSD license !

 

d3.js – Available Books

D3 is my favourite visualization platform, though the learning curve is steeper because it is about selections, data mapping and transformation close to the DOM. D3 does not come with pre-defined visualizations like bar and piecharts. The website comes with lots of samples and tutorials are available as well. If you take the time to walk through them and experiment by yourself you will learn most. Still I enjoy reading books about technical topics with an end to end walk-through.

Currently there are 2 books about D3 both from O’Reilly and both have a similar introductory focus.

Getting Started with D3

d3a June 2012, 12.99 U$ (ebook)

The books does what its title promises, getting you started, It jumps right into D3 with sample applications and code. What I really like is the fact the author connects the visualizations to real life data (New York’s MTA transportation data) which makes the whole book more entertaining and tangible. It also provides a chapter about transition and interaction, even about layouts which make more exciting visualizations, like those we all know from the D3 websites sample page. Though it does not go into advanced details. At this reasonable price I would recommend the title.

Interactive Data Visualization for the Web

d3b

November 2012, 23.99 US (ebook)

This book is a bit more comprehensive than the first one, it starts with some more basic underlying technologies and provides the reader with an introduction to HTML, DOM, CSS and Javascripts. The chapters covering D3 are written lengthier providing slightly more details. It runs along the sample around a bar-charts and scatter-plots which turns dull after a while.  The early release I have seems to be incomplete, so I dont want to give a final verdict.

With D3 obviously getting more popular we will certainly see more books, hopefully covering advanced features and more visualization centric. I was asked if I like to write one but my D3 knowledge is way not comprehensive enough, I wish Mike Bostock would write one.

Post number 300 ! Thanks to the up to 1000 visitors a day.

d3.js – From tree to cluster and radial projection

Some visualizations seem to be more sophisticated (to implement) than they actual are, specifically the radial projections. Starting from a tree representation with nodes and links it is quite easy to get to the radial version.
Remark: Of course there are more challenging radial diagrams like the bundle, but lets get started with something simple first !

“Standard” Tree
(I added few more nodes to make the visual difference more obvious)

Tree Visualization

We change 1 line of our sourcecode (see previous tutorial for complete code):

var tree = d3.layout.tree().size([300,300]);
to
var tree = d3.layout.cluster().size([300,300]);

Continue reading

d3.js – Most simple stack layout with bars

In order to understand the physics of various visualizations I walk through the samples and gists I find, dissect them and strip them down to the bare minimum. From there I explore my own variations. I took Mike’s sample for a stacked bar chart and changed it to chart reading from a matrix instead of the csv file.

Stacked Bar Chart

The matrix object

            var matrix = [
                [ 1,  5871, 8916, 2868],
                [ 2, 10048, 2060, 6171],
                [ 3, 16145, 8090, 8045],
                [ 4,   990,  940, 6907],
                [ 5,   450,  430, 5000]
            ];

The stack layout expects an object with an array of objects, each member of the array represents the values of one value group (eg. “all dark blue values”), and each object contains x and y value. It will process theis array amnd a y0 value (the offset to the baseline).
You can see this if you run the below code with the console.log output.

Console

<!DOCTYPE html>
<html>
    <head>
        <title>Simple Stack</title>
        <script type="text/javascript" src="libnew/d3.v2.min.js"></script>
        <style>
            svg {
                border: solid 1px #ccc;
                font: 10px sans-serif;
                shape-rendering: crispEdges;
            }
        </style>
    </head>
    <body>

        <div id="viz"></div>

        <script type="text/javascript">

            var w = 960,
            h = 500

            // create canvas
            var svg = d3.select("#viz").append("svg:svg")
            .attr("class", "chart")
            .attr("width", w)
            .attr("height", h )
            .append("svg:g")
            .attr("transform", "translate(10,470)");

            x = d3.scale.ordinal().rangeRoundBands([0, w-50])
            y = d3.scale.linear().range([0, h-50])
            z = d3.scale.ordinal().range(["darkblue", "blue", "lightblue"])

            console.log("RAW MATRIX---------------------------");
	    // 4 columns: ID,c1,c2,c3
            var matrix = [
                [ 1,  5871, 8916, 2868],
                [ 2, 10048, 2060, 6171],
                [ 3, 16145, 8090, 8045],
                [ 4,   990,  940, 6907],
                [ 5,   450,  430, 5000]
            ];
            console.log(matrix)

            console.log("REMAP---------------------------");
            var remapped =["c1","c2","c3"].map(function(dat,i){
                return matrix.map(function(d,ii){
                    return {x: ii, y: d[i+1] };
                })
            });
            console.log(remapped)

            console.log("LAYOUT---------------------------");
            var stacked = d3.layout.stack()(remapped)
            console.log(stacked)

            x.domain(stacked[0].map(function(d) { return d.x; }));
            y.domain([0, d3.max(stacked[stacked.length - 1], function(d) { return d.y0 + d.y; })]);

            // show the domains of the scales
            console.log("x.domain(): " + x.domain())
            console.log("y.domain(): " + y.domain())
            console.log("------------------------------------------------------------------");

            // Add a group for each column.
            var valgroup = svg.selectAll("g.valgroup")
            .data(stacked)
            .enter().append("svg:g")
            .attr("class", "valgroup")
            .style("fill", function(d, i) { return z(i); })
            .style("stroke", function(d, i) { return d3.rgb(z(i)).darker(); });

            // Add a rect for each date.
            var rect = valgroup.selectAll("rect")
            .data(function(d){return d;})
            .enter().append("svg:rect")
            .attr("x", function(d) { return x(d.x); })
            .attr("y", function(d) { return -y(d.y0) - y(d.y); })
            .attr("height", function(d) { return y(d.y); })
            .attr("width", x.rangeBand());

        </script>
    </body>
</html>

My gist: bl.ocks.org/2940908

d3.js Playing with Selections II (flat vs. hierarchy)

Based on the previews experiments you will get a flat array with all selected elements.

d3.selectAll("tbody td");

Selection

Using a chained (nested) selection

d3.selectAll("tbody tr").selectAll("td");

we get matrix of cells

Nested Selection

Now we are in full control and can manipulate eg. the color of specific cells and rows

var td = d3.selectAll("tbody tr").selectAll("td");
td.style("color", function(d, i, j) { return (j<2 && i<2) ? null : "red"; });

Nested Selection

d3.js Playing with Selections

One should have a good understanding of selections, it is worth playing with a sandbox and explore the various options you have. I adopted the tutorial from Mike here.

Lets assume we have a simple HTML table:

<!DOCTYPE html>
<html>
    <head>
        <title>Selections</title>
        <script type="text/javascript" src="lib/d3.v2.min.js"></script>
    </head>
    <body>
        <table>
            <thead>
                <tr><td>  A</td><td>  B</td><td>  C</td><td>  D</td></tr>
            </thead>
            <tbody>
                <tr><td>  0</td><td>  1</td><td>  2</td><td>  3</td></tr>
                <tr><td>  4</td><td>  5</td><td>  6</td><td>  7</td></tr>
                <tr><td>  8</td><td>  9</td><td> 10</td><td> 11</td></tr>
                <tr><td> 12</td><td> 13</td><td> 14</td><td> 15</td></tr>
            </tbody>
        </table>
        <script type="text/javascript">
           //nothing yet
        </script>
    </body>
</html>

HTML table

Continue reading

d3.js with Dynamic Data II

A slightly adopted version using SVG. A form to control radius of randomly drawn circles.

<!DOCTYPE html>
<html>
    <head>
        <title>Hello, data!</title>
        <script type="text/javascript" src="lib/d3.js"></script>
    </head>
    <body>

        <form name="myform" onSubmit="return handleClick()">
            <input name="Submit"  type="submit" value="Radius" />
            <input type="text" id="myRadius"/>
        </form>

        <div id="viz"></div>

        <script type="text/javascript">

            var mySVG = d3.select("#viz")
            .append("svg:svg")
            .attr("width", 600)
            .attr("height", 400);

            function handleClick(event){
                console.log(document.getElementById("myRadius").value)
                draw(document.getElementById("myRadius").value)
                return false;
            }

            function draw(rad){
                mySVG.append("svg:circle")
                .style("stroke", "black")
                .style("fill", "grey")
                .attr("r", rad)
                .attr("cx", Math.round(Math.random()*600))
                .attr("cy", Math.round(Math.random()*400))
            }

        </script>
    </body>
</html>

SVG circles

d3.js with Dynamic Data

On the journey to learn d3.js it is most important to understand the basic principles first, but once you got your hands dirty, thanks to lots of samples, you want make data ‘speak’. The available samples refer to static data sitting in variables or coming from a json or csv files, sooner or later you want to feed data dynamic into your visualization. In this very simple sample we want values from a form sitting on the same html page to be added to a list, as simple as no svg involved.

<!DOCTYPE html>
<html>
    <head>
        <title>Hello, data!</title>
        <script type="text/javascript" src="lib/d3.js"></script>
    </head>
    <body>
        <form name="myform" onSubmit="return handleClick()">
            <input name="Submit"  type="submit" value="Add Value" />
            <input type="text" id="myVal"/>
        </form>

        <p></p>
        <p></p>
        <p></p>
        <p></p>
        <p></p>

        <script type="text/javascript">

            var dataset = [];
            for(i=0; i<5; i++){
                dataset.push(Math.round(Math.random()*50));
            }

            var p = d3.select("body").selectAll("p")
            .data(dataset)
            .text(function(d,i){return i + " : " + d;});

            function handleClick(event){
                console.log(document.getElementById("myVal").value)
                draw(document.getElementById("myVal").value)
                return false;
            }

            function draw(val){
                d3.select("body").append("p");
                dataset.push(val);
                var p = d3.select("body").selectAll("p")
                .data(dataset)
                .text(function(d,i){return i + " : " + d;})
            }

        </script>

    </body>
</html>

Dynamic Data

In the next step we will play with some svg, from there I will explore ZK feeding a visualization from a DB or any other feed.

ZK and the clipboard

 

Sounds too simple and it is present at a lot of websites: Copy to clipboard
I never had the requirement to copy text from a website to the clipboard, like you get the git URL when you are on a GIT project website and click the little icon. Same if you hover over the sourcecode below, you will get a copy option.

github.com

No, it is not possible to use simple javascript because it cant access your clipboard and you have no other chance then using one of the flash movie based solutions, like zeroclipboard, maybe one of the most popular ones.
The usual question: How do I get this embeded into my ZK web application ?

Short Tutorial:

  • Download the latest tar from code.google.com
  • Download the latest jQuery from jquery.com
  • Create a simple ZK web application
  • Place the jquery and zeroclipboard files into your Web Pagesfolder

    required files

  • Modify the index.zul file
    <?xml version="1.0" encoding="UTF-8"?>
    <zk xmlns="http://www.zkoss.org/2005/zul">
    
        <window id="info" apply="controller.indexController">
           <script src="jquery-1.6.4.js" type="text/javascript" />
           <script type="text/javascript" src="ZeroClipboard.js"/>
    
            <div id="d_clip_container">
                <image src="/Clipboard.png" />
            </div>
    
            <textbox id= "txtClipText"   cols="50" readonly="true" value="Some string for the clipboard"/>
    
            <script>
    		var clip = null;
    
    		function $(id) { return document.getElementById(id); }
    
                    function init(divId,clipText) {
                        ZeroClipboard.setMoviePath("ZeroClipboard.swf");
                        var clip = new ZeroClipboard.Client();
    
                        clip.addEventListener('mouseOver', function (client) {
                            clip.setText(zk.Widget.$(jq("$txtClipText")).getValue() );
                            //clip.setText(clipText);
                        });
    
                        clip.addEventListener('complete', function (client, text) {
                            alert("Copied text to clipboard: \n\n" + text );
                        });
    
                        clip.glue(divId);
    		}
            </script>
            <textbox value="" cols="30"/>
        </window>
    </zk>
    

    Remarks:

    • Using clip.setText(zk.Widget.$(jq(“$txtClipText”)).getValue() ); you can access a ZK textbox on the same page
    • With clip.setText(clipText); you can pass a text from the controller
  • Create a controller class
    package controller;
    
    import org.zkoss.zk.ui.Component;
    import org.zkoss.zk.ui.util.Clients;
    import org.zkoss.zk.ui.util.GenericForwardComposer;
    import org.zkoss.zul.Div;
    import org.zkoss.zul.Image;
    
    public class indexController extends GenericForwardComposer {
    
        Div d_clip_container;
        Image imgClipboard;
    
        @Override
        public void doAfterCompose(Component comp) throws Exception {
            super.doAfterCompose(comp);
    
            String command = "init('" + d_clip_container.getUuid() + "','some Text from controller');";
            Clients.evalJavaScript(command);
    
        }
    
    }
    
  • Run the application

    Running web application

 

Remarks:

  • Thanks to my team to get this running 🙂
  • Most of it we learned by tinkering with the sample code from zeroclipboard.
  • I dont favor Flash solutions because we depend on Adobe’s policies and changes. While Flash 9 allow reading and writing the clipboard easily, they consider it as a security risk in Flash 10. And only the trickery above makes it work. We wont know what Adobe will do in future versions, the above solution might break.