Build a RESTful Webservice in less than 5 minutes

There are quite some tutorials around about building and exposing a RESTful Webservice, but some of them are outdated and make you wade through complex dependencies and tinkering with deployment descriptors and web.xml files. But using RESTeasy, the JBoss implementation that is fully compliant with the JAX-RS 2.0 JCP specification, and Eclipse you can build a simple webservice (“hello world”) with less than 10 lines of sourceode with annotations and no web.xml used in a few minutes and run it on Wildfly.

Lets build a webservice that creates random numbers.

Continue reading

PostgreSQL Replication Express Setup

The system I work on we deploy almost solely on the Amazon AWS platform. Even I try to design the architecture in a way not to be locked-in too much into Amazon, I make use of the Amazon tools and products as much as possible (EC2, VPC, S3, SNS). PostgreSQL is our reference DB and the only DB product in production environments, still we run dedicated instances with PostgreSQL. I am quite delighted about AWS recent offering staring RDS with PostgreSQL. While is is still in BETA and I did not started yet with a conclusive test and migration plan, I need to maintain our existing instances.

There are plenty of books and tutorials about setting up PostgreSQL replication with on-board tools, without going into the details I share the express setup in this tutorial based on Streaming Replication which is part of PostgreSQL since version 9.0. I highly recommend to review the parameters and settings from the below tutorial as your project might have different requirements.

References

Tutorial

Remarks

  • The tutorial is based on PostgreSQL 9.2 running on Ubuntu Server
  • Paths and settings are all the PostgreSQL defaults.
  • This is async setup, the master will not wait for feedback from the salve and continue to work even the slave is not available

Prerequisite

  • 2 Server running the same PostgreSQL version (9.0+)
  • Backup your data or use a sandbox environment.
  • In the tutorial I refer to
    MASTER (ip: 0.0.0.1) and
    SLAVE (ip: 0.0.0.2)

Configuration

Master

  • Create a replicator user
    sudo -u postgres psql -c "CREATE USER replicator REPLICATION LOGIN ENCRYPTED PASSWORD 'mypassword';"
    
  • Add the slave ip to /etc/postgresql/9.2/main/pg_hba.conf
    host    replication     all      0.0.0.2/32         trust
    
  • Modify parameters in /etc/postgresql/9.2/main/postgresql.conf
    wal_level = hot_standby
    max_wal_senders = 3
    checkpoint_segments = 3
    wal_keep_segments = 3
    

    Review these parameters and set them up according to your requirements

  • Start the PostgreSQL instance
    service postgresql start
    

Slave

  • Modify parameters in /etc/postgresql/9.2/main/postgresql.conf
    wal_level = hot_standby
    max_wal_senders = 3
    checkpoint_segments = 3
    wal_keep_segments = 3
    hot_standby = on
    
  • Stop the PostgreSQL instance
    service postgresql stop
    
  • Clean up the old data directory
    sudo -u postgres rm -rf /var/lib/postgresql/9.2/main
    
  • Copy the database from the master with pg_basebackup
    sudo -u postgres pg_basebackup -h 0.0.0.1 -D /var/lib/postgresql/9.2/main -U replicator -v -P -x
    

    You can see the backup progress and it should result in something likes

    root@:/var/lib/postgresql/9.2/main# transaction log start point: 41/7D000020
    31524952/31524952 kB (100%), 2/2 tablespaces (/var/lib/postgresql/9.2/main/PG_9.)
    transaction log end point: 41/7D0002A8
    pg_basebackup: base backup completed
    
  • Create a recovery configuration file /var/lib/postgresql/9.2/main/recovery.conf
    standby_mode = 'on'
    primary_conninfo = 'host=0.0.0.1 port=5432 user=replicator password=mypassword sslmode=require'
    trigger_file = '/tmp/postgresql.trigger'
    
  • Start the PostgreSQL instance
    service postgresql start
    

    Check the pglogs.

Test the replication

  • Open any table with pgadmin on the master and apply a change, it should be reflected in the slave within short time.
  • Try to change data on the slave, it will fail due to the hot-standby mode

Monitor the replication

  • The master instance will not alert you when the replication is down. You can check by yourself or create a little cronjob to do it for you with this sql statement.
    sudo -u postgres psql -x -c "select * from pg_stat_replication;"
    

    You get the status back (if it is running, otherwise the statement will return ‘no rows’).

    Check replication

    Check replication

Glassfish V3.1.2 and SSL

After almost 3 years (see previous post) I revisit the topic this time using the latest version og Glassfish 3.1.2 and GoDaddy as certificate provider. I created a certificate for a sub-domain (sub.whateverdomain.com) this time and make use of the extremly cheap 5.99 U$/year offer (no wildcard included)

Let me summarize the key steps here: Continue reading

d3.js – From tree to cluster and radial projection

Some visualizations seem to be more sophisticated (to implement) than they actual are, specifically the radial projections. Starting from a tree representation with nodes and links it is quite easy to get to the radial version.
Remark: Of course there are more challenging radial diagrams like the bundle, but lets get started with something simple first !

“Standard” Tree
(I added few more nodes to make the visual difference more obvious)

Tree Visualization

We change 1 line of our sourcecode (see previous tutorial for complete code):

var tree = d3.layout.tree().size([300,300]);
to
var tree = d3.layout.cluster().size([300,300]);

Continue reading

d3.js – Most simple stack layout with bars

In order to understand the physics of various visualizations I walk through the samples and gists I find, dissect them and strip them down to the bare minimum. From there I explore my own variations. I took Mike’s sample for a stacked bar chart and changed it to chart reading from a matrix instead of the csv file.

Stacked Bar Chart

The matrix object

            var matrix = [
                [ 1,  5871, 8916, 2868],
                [ 2, 10048, 2060, 6171],
                [ 3, 16145, 8090, 8045],
                [ 4,   990,  940, 6907],
                [ 5,   450,  430, 5000]
            ];

The stack layout expects an object with an array of objects, each member of the array represents the values of one value group (eg. “all dark blue values”), and each object contains x and y value. It will process theis array amnd a y0 value (the offset to the baseline).
You can see this if you run the below code with the console.log output.

Console

<!DOCTYPE html>
<html>
    <head>
        <title>Simple Stack</title>
        <script type="text/javascript" src="libnew/d3.v2.min.js"></script>
        <style>
            svg {
                border: solid 1px #ccc;
                font: 10px sans-serif;
                shape-rendering: crispEdges;
            }
        </style>
    </head>
    <body>

        <div id="viz"></div>

        <script type="text/javascript">

            var w = 960,
            h = 500

            // create canvas
            var svg = d3.select("#viz").append("svg:svg")
            .attr("class", "chart")
            .attr("width", w)
            .attr("height", h )
            .append("svg:g")
            .attr("transform", "translate(10,470)");

            x = d3.scale.ordinal().rangeRoundBands([0, w-50])
            y = d3.scale.linear().range([0, h-50])
            z = d3.scale.ordinal().range(["darkblue", "blue", "lightblue"])

            console.log("RAW MATRIX---------------------------");
	    // 4 columns: ID,c1,c2,c3
            var matrix = [
                [ 1,  5871, 8916, 2868],
                [ 2, 10048, 2060, 6171],
                [ 3, 16145, 8090, 8045],
                [ 4,   990,  940, 6907],
                [ 5,   450,  430, 5000]
            ];
            console.log(matrix)

            console.log("REMAP---------------------------");
            var remapped =["c1","c2","c3"].map(function(dat,i){
                return matrix.map(function(d,ii){
                    return {x: ii, y: d[i+1] };
                })
            });
            console.log(remapped)

            console.log("LAYOUT---------------------------");
            var stacked = d3.layout.stack()(remapped)
            console.log(stacked)

            x.domain(stacked[0].map(function(d) { return d.x; }));
            y.domain([0, d3.max(stacked[stacked.length - 1], function(d) { return d.y0 + d.y; })]);

            // show the domains of the scales
            console.log("x.domain(): " + x.domain())
            console.log("y.domain(): " + y.domain())
            console.log("------------------------------------------------------------------");

            // Add a group for each column.
            var valgroup = svg.selectAll("g.valgroup")
            .data(stacked)
            .enter().append("svg:g")
            .attr("class", "valgroup")
            .style("fill", function(d, i) { return z(i); })
            .style("stroke", function(d, i) { return d3.rgb(z(i)).darker(); });

            // Add a rect for each date.
            var rect = valgroup.selectAll("rect")
            .data(function(d){return d;})
            .enter().append("svg:rect")
            .attr("x", function(d) { return x(d.x); })
            .attr("y", function(d) { return -y(d.y0) - y(d.y); })
            .attr("height", function(d) { return y(d.y); })
            .attr("width", x.rangeBand());

        </script>
    </body>
</html>

My gist: bl.ocks.org/2940908

d3.js Playing with Selections II (flat vs. hierarchy)

Based on the previews experiments you will get a flat array with all selected elements.

d3.selectAll("tbody td");

Selection

Using a chained (nested) selection

d3.selectAll("tbody tr").selectAll("td");

we get matrix of cells

Nested Selection

Now we are in full control and can manipulate eg. the color of specific cells and rows

var td = d3.selectAll("tbody tr").selectAll("td");
td.style("color", function(d, i, j) { return (j<2 && i<2) ? null : "red"; });

Nested Selection

d3.js Playing with Selections

One should have a good understanding of selections, it is worth playing with a sandbox and explore the various options you have. I adopted the tutorial from Mike here.

Lets assume we have a simple HTML table:

<!DOCTYPE html>
<html>
    <head>
        <title>Selections</title>
        <script type="text/javascript" src="lib/d3.v2.min.js"></script>
    </head>
    <body>
        <table>
            <thead>
                <tr><td>  A</td><td>  B</td><td>  C</td><td>  D</td></tr>
            </thead>
            <tbody>
                <tr><td>  0</td><td>  1</td><td>  2</td><td>  3</td></tr>
                <tr><td>  4</td><td>  5</td><td>  6</td><td>  7</td></tr>
                <tr><td>  8</td><td>  9</td><td> 10</td><td> 11</td></tr>
                <tr><td> 12</td><td> 13</td><td> 14</td><td> 15</td></tr>
            </tbody>
        </table>
        <script type="text/javascript">
           //nothing yet
        </script>
    </body>
</html>

HTML table

Continue reading