The JavaDude Weblog

First hands-on Oracle Solaris 11

devdude — Sun, 04 Dec 2011 01:46:32 +0000

After Oracle took over SUN and they dropped OpenSolaris I stopped using or even looking at it. Now a customer is keen on going the Solaris way and I have a chance to get my hands on it (again). I dont have a SUN SPARC sitting here but Oracle continues to support i386 architecture and they have both the iso file with the text installer and even a ready to run virtual appliance ready to download (link). Oracle released version 11 of Solaris on 2011 November 9. Did you know SUN’s first OS goes back to 1982 as Sun UNIX 0.7 ?

I am not sure what happen to Solaris Express 11 which disappeared completely from Oracle’s website. Will there be another Express version for “free”/development/evaluation use ?

Import Appliance

Imported Oracle Solaris 11

Booting Solaris 11 first time

Solaris 11 Login

Solaris 11 Desktop

First impressions

It starts up with 1GB memory and 1 CPU. On a Ubuntu 11.10 with 8GB running on AMD II X4 965 it takes about 2min to the login screen.
Using the Gnome desktop it is easy to navigate for an Ubuntu user.
MPO disabled because memory is interleaved. (Wiki explanation)
Changing the settings to 2 CPU makes it extreme sluggish slow or even dont start at all
After this experience I decided to install with 2 CPU’s and 4GB from the scratch using the iso image.
This failed first, after installation boot up takes some 5 minutes

I could fix this by
a) add a second virtualbox interface adapter (host-only)
b) modify /etc/hosts to something like 127.0.0.1 solaris name.domain.com loghost
Now the boot time to console takes 40 seconds.

Additional Tasks:
I need to install JDK 7, Glassfish 3.1.1 and Oracle 11g

JDK 7 (manual)

Download from Oracle

jdk-7u1-solaris-i586.tar.Z
jdk-7u1-solaris-x64.tar.Z
zcat jdk-7u1-solaris-x64.tar.Z |tar xf -
zcat jdk-7u1-solaris-i586.tar.Z |tar xf -
pkgadd -d . SUNWj7rt SUNWj7dev SUNWj7cfg SUNWj7man SUNWj7dmo SUNWjavadb-client SUNWjavadb-common SUNWjavadb-core SUNWjavadb-demo SUNWjavadb-docs SUNWjavadb-javadoc SUNWjavadb-service
pkgadd -d . SUNWj7rtx SUNWj7dvx SUNWj7dmx

Change the link /usr/java to jdk/latest !

JDK 7 (Package Manager)

Start the package manager

ZK goes EC2 (Part 3)

devdude — Fri, 25 Nov 2011 13:44:44 +0000

The third part of the tutorial where I improve a few things. I will not walk through the complete code but highlight a few important points and give you the complete sourcecode at the end.

To recap, my requirements:

I want to allow users in my company to start and stop instances on their own without them login to AWS console.
Only specific instances are available to them.
Avoid using elastic IP’s (you pay for them if they are not assigned)
Make it configurable

The improvements in this version:

Remove the hardcoded access keys and place them encrypted in a properties file.
Only instances that are not protected can be started or stopped.
Update DynDNS entries from the application
Some cosmetic cleanup of the control panel

Running Application

Prerequisites:

The project from part 1 and 2 (link)

Important points:

Read from and write to properties files
The file will be located in our domain folder. We encrypt the password and secret access key.

Properties File

properties
Encryption
Encryption
Update DynDNS
This we can avoid paying for unused elastic IP’s since our instances only run occasionally.
I use a tag in the instance properties for the dyndns name

DynDNS Hostname

AWS instance tag

Update Method

Complete sourcecode:

instances.zul

instancesController.java

package controller;

import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.ec2.AmazonEC2;
import com.amazonaws.services.ec2.AmazonEC2Client;
import com.amazonaws.services.ec2.model.DescribeInstancesResult;
import com.amazonaws.services.ec2.model.DescribeRegionsResult;
import com.amazonaws.services.ec2.model.Instance;
import com.amazonaws.services.ec2.model.InstanceState;
import com.amazonaws.services.ec2.model.Region;
import com.amazonaws.services.ec2.model.Reservation;
import com.amazonaws.services.ec2.model.StartInstancesRequest;
import com.amazonaws.services.ec2.model.StartInstancesResult;
import com.amazonaws.services.ec2.model.StopInstancesRequest;
import com.amazonaws.services.ec2.model.StopInstancesResult;
import com.amazonaws.services.ec2.model.Tag;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Properties;
import java.util.Set;
import javax.crypto.Cipher;
import javax.crypto.SecretKey;
import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.DESKeySpec;
import org.zkoss.zk.ui.Component;
import org.zkoss.zk.ui.event.Event;
import org.zkoss.zk.ui.event.EventListener;
import org.zkoss.zk.ui.event.Events;
import org.zkoss.zk.ui.util.Clients;
import org.zkoss.zk.ui.util.ComposerExt;
import org.zkoss.zk.ui.util.GenericForwardComposer;
import org.zkoss.zul.Button;
import org.zkoss.zul.Label;
import org.zkoss.zul.ListModelList;
import org.zkoss.zul.Listbox;
import org.zkoss.zul.Listcell;
import org.zkoss.zul.Listitem;
import org.zkoss.zul.ListitemRenderer;
import org.zkoss.zul.Messagebox;
import org.zkoss.zul.Popup;
import org.zkoss.zul.Textbox;
import org.zkoss.zul.Window;
import sun.misc.BASE64Decoder;
import sun.misc.BASE64Encoder;

public class instancesControllerX extends GenericForwardComposer implements ComposerExt {

    // ZK Variables
    private Listbox lstInstance;
    private Listbox lstRegion;
    private Label lblStatus;
    private Label lblEndpoint;
    private Button btnStart;
    private Button btnStop;
    private Button btnRefresh;
    private Button btnEndPoints;
    private Button btnDynDNS;
    private Textbox txtKey;
    private Textbox txtSkey;
    private Textbox txtDynPw;
    private Textbox txtDynUser;
    private Popup regions;
    private Popup ec2keys;
    private Window list;
    //Amazon Variables
    AmazonEC2 ec2;
    List listEC2Reservations = null;
    List listEC2Instances = null;
    List listEC2Regions = null;
    //Others
    String msgboxTitle = "ZKEC2CloudControl";
    String defEndpoint = "ec2.ap-southeast-1.amazonaws.com";
    static String crypKey = "somekey";
    static String propFile = "ec2cloudcontrol.properties";
    String accessKey = "";
    String secretAccessKey = "";
    String endPoint = "";
    String dynDNSuser = "";
    String dynDNSpw = "";
    boolean ec2Conn = false;

    private void initEC2() {

        try {
            BasicAWSCredentials ecProp = new BasicAWSCredentials(accessKey, secretAccessKey);
            ec2 = new AmazonEC2Client(ecProp);
            ec2.setEndpoint(endPoint);

            // Retrieve the available EC2 regions
            DescribeRegionsResult regionsResult = ec2.describeRegions();
            listEC2Regions = regionsResult.getRegions();
            lstRegion.setModel(new ListModelList(listEC2Regions));

            ec2Conn = true;
            lblStatus.setValue("Connected.");

            btnRefresh.setDisabled(false);
            btnEndPoints.setDisabled(false);

        } catch (Exception ex) {
            ec2Conn = false;
            lblStatus.setValue(ex.getMessage());
            btnRefresh.setDisabled(true);
            btnEndPoints.setDisabled(true);
        }
    }

    @Override
    public void doAfterCompose(Component comp) throws Exception {
        super.doAfterCompose(comp);

        btnStart.setDisabled(true);
        btnStop.setDisabled(true);
        btnRefresh.setDisabled(true);

        if (readProperties()) {
            if (endPoint.isEmpty()) {
                endPoint = defEndpoint;
            }
            txtKey.setValue(accessKey);
            txtSkey.setValue(secretAccessKey);
            txtDynPw.setValue(dynDNSpw);
            txtDynUser.setValue(dynDNSuser);
            initEC2();
        } else {
            lblStatus.setValue("Missing AWS keys.");
        }

        setupRenderer();

        lblEndpoint.setValue(endPoint);
        if (ec2Conn) {
            listReservationsInstances();
        }

    }

    private void listReservationsInstances() {

        btnStart.setDisabled(true);
        btnStop.setDisabled(true);
        btnDynDNS.setDisabled(true);
        Clients.showBusy("Retrieving instance information..");
        Events.echoEvent("onListReservationsInstances", list, null);
    }

    public void onListReservationsInstances(Event event) {

        DescribeInstancesResult describeInstancesRequest = ec2.describeInstances();
        listEC2Reservations = describeInstancesRequest.getReservations();

        Set instances = new HashSet();
        for (Reservation reservation : listEC2Reservations) {
            instances.addAll(reservation.getInstances());
        }

        listEC2Instances = new ArrayList(instances);

        lstInstance.setModel(new ListModelList(listEC2Instances));
        lblStatus.setValue("Loaded " + listEC2Instances.size() + " instances.");
        Clients.clearBusy();

    }

    public void onClick$btnRefresh(Event evt) throws InterruptedException {
        listReservationsInstances();
        lblStatus.setValue("");
    }

    public void onClick$btnReconnect(Event evt) throws InterruptedException {
        lblStatus.setValue("");
        listEC2Instances.clear();
        lstInstance.setModel(new ListModelList(listEC2Instances));
        initEC2();
    }

    public void onClick$btnSelectEndpoint(Event evt) throws InterruptedException {
        if (lstRegion.getSelectedIndex() > -1) {
            String newEndpoint = ((Region) lstRegion.getSelectedItem().getValue()).getEndpoint();
            ec2.setEndpoint(newEndpoint);
            lblEndpoint.setValue(newEndpoint);
            listReservationsInstances();
            regions.close();
        }
    }

    public void onClick$btnStart(Event evt) throws InterruptedException {
          if (lstInstance.getSelectedIndex() > -1) {
            Messagebox.show("Start instance <" + ((Instance) lstInstance.getSelectedItem().getValue()).getInstanceId() + "> ?", msgboxTitle, Messagebox.OK | Messagebox.CANCEL,
                    Messagebox.QUESTION,
                    new EventListener() {

                        public void onEvent(Event evt) {
                            switch (((Integer) evt.getData()).intValue()) {
                                case Messagebox.OK:
                                    doStart();
                                    break; //the Yes button is pressed
                            }
                        }
                    });
        }
    }

    private void doStart() {
        List startIDs = new ArrayList();
        startIDs.add(((Instance) lstInstance.getSelectedItem().getValue()).getInstanceId());
        StartInstancesRequest start = new StartInstancesRequest(startIDs);
        StartInstancesResult result = ec2.startInstances(start);
        lblStatus.setValue(result.toString() + " Refresh after 10 or more seconds.");

    }

    public void onClick$btnStop(Event evt) throws InterruptedException {

          if (lstInstance.getSelectedIndex() > -1) {
            Messagebox.show("Stop instance <" + ((Instance) lstInstance.getSelectedItem().getValue()).getInstanceId() + "> ?", msgboxTitle, Messagebox.OK | Messagebox.CANCEL,
                    Messagebox.QUESTION,
                    new EventListener() {

                        public void onEvent(Event evt) {
                            switch (((Integer) evt.getData()).intValue()) {
                                case Messagebox.OK:
                                    doStop();
                                    break; //the Yes button is pressed
                            }
                        }
                    });
        }
    }

    private void doStop() {
        List stopIDs = new ArrayList();
        stopIDs.add(((Instance) lstInstance.getSelectedItem().getValue()).getInstanceId());
        StopInstancesRequest stop = new StopInstancesRequest(stopIDs);
        StopInstancesResult result = ec2.stopInstances(stop);
        lblStatus.setValue(result.toString() + " Refresh after 10 or more seconds.");
    }

    public void onClick$btnSaveKeys(Event evt) throws InterruptedException {

        accessKey = txtKey.getValue();
        secretAccessKey = txtSkey.getValue();
        dynDNSpw = txtDynPw.getValue();
        dynDNSuser = txtDynUser.getValue();

        if (endPoint.isEmpty()) {
            endPoint = defEndpoint;
        }

        if (writeProperties()) {
            lblStatus.setValue("Configuration saved. Please reconnect.");
        }

        ec2keys.close();
    }

    public void onClick$btnDynDNS(Event evt) throws InterruptedException {

        if (lstInstance.getSelectedIndex() > -1) {
            String ip = ((Instance) lstInstance.getSelectedItem().getValue()).getPublicIpAddress();
            String dynDNShost = findTagValuebyKey(((Instance) lstInstance.getSelectedItem().getValue()).getTags(), "dyndns");

            // only instance with dns hostnames
            if (!dynDNShost.toUpperCase().equals("NA")) {

                    String feedback = updateDynDNS(dynDNShost, ip);
                    lblStatus.setValue(feedback);

            } else {
                lblStatus.setValue("No dynDNS hostname configured. Cannot be attached.");
            }
        }
    }

    public void onSelect$lstInstance(Event evt) throws InterruptedException {

        InstanceState state = ((Instance) lstInstance.getSelectedItem().getValue()).getState();
        String prot = findTagValuebyKey(((Instance) lstInstance.getSelectedItem().getValue()).getTags(), "protected");
        String dyndns = findTagValuebyKey(((Instance) lstInstance.getSelectedItem().getValue()).getTags(), "dyndns");

        btnStart.setDisabled(true);
        btnStop.setDisabled(true);
        btnDynDNS.setDisabled(true);

        if ((state.getCode() == 16) && (prot.equalsIgnoreCase("na"))) {
            btnStop.setDisabled(false);
            if (!dyndns.equalsIgnoreCase("na"))
                btnDynDNS.setDisabled(false);
        }

        if ((state.getCode() == 80) && (prot.equalsIgnoreCase("na"))) {
            btnStart.setDisabled(false);
        }
    }

    private void setupRenderer() {

        // EC2 Instances list renderer
        ListitemRenderer listRenderInstance = new ListitemRenderer() {

            @Override
            public void render(Listitem item, Object data) throws Exception {
                item.setValue(data);
                item.appendChild(new Listcell(((Instance) data).getInstanceId()));
                item.appendChild(new Listcell(findTagValuebyKey(((Instance) data).getTags(), "name")));
                item.appendChild(new Listcell(((Instance) data).getPublicIpAddress()));

                String dyndns = findTagValuebyKey(((Instance) data).getTags(), "dyndns");
                if (dyndns.equalsIgnoreCase("na")) {
                    item.appendChild(new Listcell(" "));
                } else {
                    item.appendChild(new Listcell(dyndns));
                }

                InstanceState state = ((Instance) data).getState();
                item.appendChild(new Listcell(state.getName()));

                String prot = findTagValuebyKey(((Instance) data).getTags(), "protected");
                if (prot.equalsIgnoreCase("yes")) {
                    item.appendChild(new Listcell("Yes"));
                } else {
                    item.appendChild(new Listcell(" "));
                }

                if (state.getCode() == 16) {
                    item.appendChild(new Listcell(((Instance) data).getLaunchTime().toString()));
                } else {
                    item.appendChild(new Listcell(" "));
                }

                Listcell listcell = new Listcell();
                item.appendChild(listcell);
            }
        };
        lstInstance.setItemRenderer(listRenderInstance);

        // EC2 Regions list renderer
        ListitemRenderer listitem = new ListitemRenderer() {

            @Override
            public void render(Listitem item, Object data) throws Exception {
                item.setValue(data);
                item.appendChild(new Listcell(((Region) data).getRegionName()));
                item.appendChild(new Listcell(((Region) data).getEndpoint()));
                Listcell listcell = new Listcell();
                item.appendChild(listcell);
            }
        };
        lstRegion.setItemRenderer(listitem);
    }

    // HELPER METHODS
    private String findTagValuebyKey(List tags, String key) {

        for (Tag tag : tags) {
            if (tag.getKey().toUpperCase().equals(key.toUpperCase())) {
                return tag.getValue();
            }
        }
        return "na";
    }

    private String encryptString(String inStr) {
        try {
            DESKeySpec keySpec = new DESKeySpec(crypKey.getBytes("UTF8"));
            SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("DES");
            SecretKey key = keyFactory.generateSecret(keySpec);
            sun.misc.BASE64Encoder base64encoder = new BASE64Encoder();

            byte[] cleartext = inStr.getBytes("UTF8");
            Cipher cipher = Cipher.getInstance("DES"); // cipher is not thread safe
            cipher.init(Cipher.ENCRYPT_MODE, key);
            String encrypted = base64encoder.encode(cipher.doFinal(cleartext));

            return encrypted;
        } catch (Exception ex) {
            System.out.println(ex.getMessage());
            return null;
        }
    }

    private String decryptString(String inStr) {
        try {
            DESKeySpec keySpec = new DESKeySpec(crypKey.getBytes("UTF8"));
            SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("DES");
            SecretKey key = keyFactory.generateSecret(keySpec);
            sun.misc.BASE64Decoder base64decoder = new BASE64Decoder();

            byte[] encrypedBytes = base64decoder.decodeBuffer(inStr);

            Cipher cipher = Cipher.getInstance("DES");
            cipher.init(Cipher.DECRYPT_MODE, key);
            byte[] plainText = (cipher.doFinal(encrypedBytes));

            return new String(plainText);
        } catch (Exception ex) {
            System.out.println(ex.getMessage());
            return null;
        }
    }

    private boolean readProperties() {
        Properties properties = new Properties();
        try {
            properties.load(new FileInputStream(propFile));
            accessKey = decryptString(properties.getProperty("accesskey"));
            secretAccessKey = decryptString(properties.getProperty("secretaccesskey"));
            endPoint = properties.getProperty("endpoint", "");
            dynDNSuser = properties.getProperty("dyndnsuser", "");
            dynDNSpw = decryptString(properties.getProperty("dyndnspw", ""));

            if ((accessKey.isEmpty()) || secretAccessKey.isEmpty()) {
                return false;
            } else {
                return true;
            }

        } catch (Exception ex) {
            System.out.println(ex.getMessage());
            return false;

        }
    }

    private boolean writeProperties() {

        Properties properties = new Properties();
        properties.setProperty("accesskey", encryptString(accessKey));
        properties.setProperty("secretaccesskey", encryptString(secretAccessKey));
        properties.setProperty("endpoint", endPoint);
        properties.setProperty("dyndnspw", encryptString(dynDNSpw));
        properties.setProperty("dyndnsuser", dynDNSuser);

        try {
            properties.store(new FileOutputStream(propFile), null);
            return true;
        } catch (Exception ex) {
            System.out.println(ex.getMessage());
            return false;
        }

    }

    private String updateDynDNS(String hostName, String hostIP) {

        String userName = dynDNSuser;
        String userPassword = dynDNSpw;
        int responseCode;
        String feedback = "";

        try {
            // Encode username and password
            BASE64Encoder enc = new sun.misc.BASE64Encoder();
            String userpassword = userName + ":" + userPassword;
            String encodedAuthorization = enc.encode(userpassword.getBytes());

            // Connect to DynDNS
            URL url = new URL("http://members.dyndns.org/nic/update?hostname=" + hostName + "&myip=" + hostIP);
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();
            connection.setRequestMethod("GET");
            connection.setRequestProperty("User-Agent", "DynDNS Updater");
            connection.setRequestProperty("Authorization", "Basic " + encodedAuthorization);

            // Execute GET
            responseCode = connection.getResponseCode();
            System.out.println(responseCode + ":" + connection.getResponseMessage());

            // Print feedback
            String line;
            InputStreamReader in = new InputStreamReader((InputStream) connection.getContent());
            BufferedReader buff = new BufferedReader(in);
            do {
                line = buff.readLine();
                if (line != null) {
                    feedback = line;
                }
                System.out.println(line);
            } while (line != null);

            connection.disconnect();

        } catch (Exception ex) {
            System.out.println(ex.getMessage());
            feedback = ex.getMessage();
        }
        return feedback;
    }
}

Remarks:

I still like to add a EJB timer to start and stop the instances automatically
The sorting is still random

Remote Glassfish V3.1 and the mystical JMX settings

devdude — Fri, 18 Nov 2011 08:12:51 +0000

Once you have any serious sized application running on Glassfish, you need to profile and tune your server settings. A good tool to look under the hood of a running Glassfish is to to connect jconsole (part of JDK) to its JVM. This works without problem for a local Glassfish but when it comes to a remote instance you cant connect to the default Glassfish setup.

Lets look at the default setup

Glassfish Admin Console

JMX Connector (default)

By changing the above default parameters I was not able to connect to remote Glassfish.
Only after adding these JVM parameters:

-Djava.rmi.server.hostname=yourhostname
-Dcom.sun.management.jmxremote.port=8686
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false

Basically we are controlling the underlying JVM not the Glassfish instance.

JVM Settings

Please beware, with these settings anyone can login. So you need to adjust the settings accordingly, btw if you change the Glassfish JMX connector settings, nothing will change (eg. enable security).

If you enable the

-Dcom.sun.management.jmxremote.authenticate=false

you will get this:

Waiting for domain1 to start .Error starting domain domain1.
The server exited prematurely with exit code 1.
Before it died, it produced the following output:

Error: Password file read access must be restricted: /usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/management/jmxremote.password

Proof, we are tinkering with the JVM not with Glassfish settings.

jconsole conneted to remote server

Netbeans + Visual Paradigm = EJB Tutorial (Part 3)

devdude — Fri, 04 Nov 2011 13:48:01 +0000

In part 1 of this tutorial we created an EJB using Netbeans and Visual Paradigm, in part 2 a little ZK application to read data using the EJB. In part 3 we will move away from the Derby DB to PostgreSQL and Oracle DB and challenge ourselves with identifier more than 30 characters, which is an issue for Oracle (yes, it is 2011). We will add columns with more than 30 characters and play with a few different column types (the ones showing up in a normal DB layout).

Add new fields to the ERD
this_is_a_very_long_remark_field
floatcol
numbercol

Updated ERD
Synchronize to class diagram
Class diagram
Update Netbeans Project
Since we only added columns, that does not create a problem.
Try:
Lets change the name of one column above and go through the same procedure again, no problem too.
Delete a column and same procedure again, no problem.
(Seems the VP team fixed this, in earlier version deleted columns would not disappear from the code.)
Setup the required DB’s in VP
You need to tick the DB’s otherwise you cant create a script.
If we dont connect directly to the DB instance we dont need ID, driver, etc

Database Configuration

Create scripts to create the DB
(I prefer this over the direct connection to a DB as VP feature)
Go to your ERD diagram, dont select anything, right-click, Utilities, Generate SQL.

Create SQL

Oracle Script

CREATE TABLE "customer_name" (
  "last_name"                            varchar2(255),
  "first_name"                           varchar2(255),
  "entrydate"                            date,
  "last_contact"                         timestamp,
  "customer_uuid"                        varchar2(36) NOT NULL,
  "this_is_a_very_long_remark_field_too" varchar2(255),
  "floatcol"                             float(10),
  "numbercol"                            number(19),
  CONSTRAINT "pk_customer_name"
    PRIMARY KEY ("customer_uuid"));
CREATE TABLE "customer_address" (
  "customer_city"                   char(255),
  "customer_street"                 char(255),
  "customeraddress_uuid"            char(36) NOT NULL,
  "fk_customername_customeraddress" varchar2(36) NOT NULL,
  CONSTRAINT "pl_customer_address"
    PRIMARY KEY ("customeraddress_uuid"));
ALTER TABLE "customer_address" ADD CONSTRAINT "rel_customername_customeraddress" FOREIGN KEY ("fk_customername_customeraddress") REFERENCES "customer_name" ("customer_uuid");

PostgreSQL Script

CREATE TABLE "customer_name" (
  "last_name"                            varchar(255),
  "first_name"                           varchar(255),
  "entrydate"                            date,
  "last_contact"                         timestamp,
  "customer_uuid"                        varchar(36) NOT NULL,
  "this_is_a_very_long_remark_field_too" varchar(255),
  "floatcol"                             float4,
  "numbercol"                            int8,
  CONSTRAINT "pk_customer_name"
    PRIMARY KEY ("customer_uuid"));
CREATE TABLE "customer_address" (
  "customer_city"                   char(255),
  "customer_street"                 char(255),
  "customeraddress_uuid"            char(36) NOT NULL,
  "fk_customername_customeraddress" varchar(36) NOT NULL,
  CONSTRAINT "pl_customer_address"
    PRIMARY KEY ("customeraddress_uuid"));
ALTER TABLE "customer_address" ADD CONSTRAINT "rel_customername_customeraddress" FOREIGN KEY ("fk_customername_customeraddress") REFERENCES "customer_name" ("customer_uuid");

On the first sight only the number and float columns differ in the script.

Add a schema
The schema is not relevant to the classdiagramm or the created sourcecode, but it is important for the setup of our DB

Schema

The resulting scripts are slightly changed to:
CREATE TABLE “myschema”.”customer_name” (
“last_name” varchar2(255),
…
Run the SQL scripts
(Assuming you have PostgreSQL and Oracle DB at hand, I use a Virtualbox image for this purpose, one running POstgreSQL 9.1 the other Oracle Express 10g.

PostgreSQL with the pgAdmin III tool
(We need to create the schema ‘myschema’ in the DB of our choice)

Create tables

Oracle using the APEX web admin tool
We need to create the user ‘myschema’ !

Create user
Create the tables
Create tables

Now we hit 2 problems:
- ORA-00972: identifier is too long
  Due to our too long identifier
- ORA-01918: user ‘myschema’ does not exist
  Due to Oracle creates the user in capital
Remember, we want to serve multiple DB’s with the same ERD without doing any modifications (other than selecting the DB type when creating the script)
Address the ORA-00972 problem
Oracle, even the latest version 11g, cannot handle identifier longer than 30 characters. VP helps on this when you select Oracle as DB and warns you if your name is getting too long. Since we started with PostgreSQL we didnt have to limit ourselves from creating very descriptive names though.
Our sample column and troublemaker ‘this_is_a_very_long_remark_field_too’shows up in
1. ERD diagram
2. Class diagram
3. Created sourcecode class
4. Meta class
5. Facade class
  This is the class where you would implement your business logic
While we can change easily the ERD to ‘this_is_a_short_remark_field’ and automatically get up to the meta class, our variable might be all over the place downstream the facade class.
- The code creation creates a cosmetic hickup. The getter and setter is OK, but the internal variable remains the old one, makes the code less consistent.
  
  Solution: Delete the old classes (demo.Customer_name) before rerun the sync between VP and NB.
- Our facade class is screwed
  (In a real sized application you have maybe dozens of references)
  
  Solution: None
  We cannot refactor because it is an unresolved variable.
  No choice, but to manually do it, or replace in the whole project, which only works if you use unique names and you are sure it is not used in any other context.
Address the ORA-01918 problem
Just change the schema name to uppercase.

Remarks:

Careful naming conventions help to avoid this additional work.
In the next and last part we will configure Glassfish for the 2 different datasources.

ZK and the clipboard

devdude — Fri, 04 Nov 2011 09:21:38 +0000

Sounds too simple and it is present at a lot of websites: Copy to clipboard
I never had the requirement to copy text from a website to the clipboard, like you get the git URL when you are on a GIT project website and click the little icon. Same if you hover over the sourcecode below, you will get a copy option.

github.com

No, it is not possible to use simple javascript because it cant access your clipboard and you have no other chance then using one of the flash movie based solutions, like zeroclipboard, maybe one of the most popular ones.
The usual question: How do I get this embeded into my ZK web application ?

Short Tutorial:

Download the latest tar from code.google.com
Download the latest jQuery from jquery.com
Create a simple ZK web application
Place the jquery and zeroclipboard files into your Web Pagesfolder
required files

Modify the index.zul file

Remarks:

Using clip.setText(zk.Widget.$(jq(“$txtClipText”)).getValue() ); you can access a ZK textbox on the same page
With clip.setText(clipText); you can pass a text from the controller

Create a controller class

package controller;

import org.zkoss.zk.ui.Component;
import org.zkoss.zk.ui.util.Clients;
import org.zkoss.zk.ui.util.GenericForwardComposer;
import org.zkoss.zul.Div;
import org.zkoss.zul.Image;

public class indexController extends GenericForwardComposer {

    Div d_clip_container;
    Image imgClipboard;

    @Override
    public void doAfterCompose(Component comp) throws Exception {
        super.doAfterCompose(comp);

        String command = "init('" + d_clip_container.getUuid() + "','some Text from controller');";
        Clients.evalJavaScript(command);

    }

}

Run the application
Running web application

Remarks:

Thanks to my team to get this running
Most of it we learned by tinkering with the sample code from zeroclipboard.
I dont favor Flash solutions because we depend on Adobe's policies and changes. While Flash 9 allow reading and writing the clipboard easily, they consider it as a security risk in Flash 10. And only the trickery above makes it work. We wont know what Adobe will do in future versions, the above solution might break.

Amazon S3 plugin for Jenkins CI

devdude — Tue, 01 Nov 2011 13:53:48 +0000

About one year back I found this plugin, that saves your build artifacts to Amazon S3 storage. Unfortunately the plugin does not properly work with the current Jenkins versions and the original plugin is not maintained. Luckily someone forked and updated the plugin at github. You need to build the plugin by yourself with the help of maven, you still can refer to my original post, just one change is required to make it work, the settings for your ~/.m2/settings/xml need to be changed according to the plugin page description. Please be patient, maven is going to download a lot !


 
 org.jenkins-ci.tools
 

 
 
 
 jenkins
 
 true 
 
 
 
 maven.jenkins-ci.org
 http://maven.jenkins-ci.org/content/groups/artifacts/
 
 
 
 
 maven.jenkins-ci.org
 http://maven.jenkins-ci.org/content/groups/artifacts/

Otherwise you get errors like

[INFO] Scanning for projects...
Downloading: http://download.java.net/maven/2/org/jenkins-ci/plugins/plugin/1.434/plugin-1.434.pom
[INFO] Unable to find resource 'org.jenkins-ci.plugins:plugin:pom:1.434' in repository java.net2 (http://download.java.net/maven/2)
Downloading: http://repo1.maven.org/maven2/org/jenkins-ci/plugins/plugin/1.434/plugin-1.434.pom
[INFO] Unable to find resource 'org.jenkins-ci.plugins:plugin:pom:1.434' in repository central (http://repo1.maven.org/maven2)
[INFO] ------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error building POM (may not be this project's POM).

Netbeans + Visual Paradigm = EJB Tutorial (Part 2)

devdude — Mon, 31 Oct 2011 08:00:35 +0000

In part 1 of this tutorial we walk through the modeling and code creation of an EJB using Netbeans and Visual Paradigm. In part 2 we will create a simple sample ZK web application that reads data using the EJB.

Prerequisites:

The project from completed tutorial part 1

Tutorial:

Create web application ‘demoejbweb’
Create web application

Create web application
Add the EJB from dem demoejb.jar
Note: We have 2 independent projects now, if you rebuild the ejb, you also need to rebuild the web application.

Add library

Add a listbox to the zul file

Add the controller class

Controller Class

package controller;

import demo.Customer_name;
import facade.Customer_nameFacade;
import java.util.List;
import javax.naming.Context;
import javax.naming.InitialContext;
import org.zkoss.zk.ui.Component;
import org.zkoss.zk.ui.util.GenericForwardComposer;
import org.zkoss.zul.ListModelList;
import org.zkoss.zul.Listbox;
import org.zkoss.zul.Listcell;
import org.zkoss.zul.Listitem;
import org.zkoss.zul.ListitemRenderer;

public class indexController extends GenericForwardComposer {

    private Listbox lstCustomer;
    Context context;
    Customer_nameFacade customer_nameFacade;
    List customers = null;

    @Override
    public void doAfterCompose(Component comp) throws Exception {
        super.doAfterCompose(comp);

        setupRenderer();

        context = new InitialContext();

        try {
            customer_nameFacade = (Customer_nameFacade) context.lookup("java:global/demoejbweb/Customer_nameFacade");
            customers = customer_nameFacade.findAll();
            lstCustomer.setModel(new ListModelList(customers));

        } catch (Exception ex) {
            System.out.println(ex.getMessage());
        }

    }

    private void setupRenderer() {
        ListitemRenderer listitem = new ListitemRenderer() {

            @Override
            public void render(Listitem item, Object data) throws Exception {
                item.setValue(data);

                item.appendChild(new Listcell(((Customer_name) data).getLast_name()));
                item.appendChild(new Listcell(((Customer_name) data).getFirst_name()));
                item.appendChild(new Listcell(((Customer_name) data).getEntrydate().toString()));

                Listcell listcell = new Listcell();
                item.appendChild(listcell);
            }
        };
        lstCustomer.setItemRenderer(listitem);
    }
}

Add some data using the “data explorer” in Netbeans
Note: If you chosed the strategy ‘drop and create‘ for your persistence unit, the data will be lost on the next deployment. After the tables have been created change to ‘Nothing‘.

Adding data
Run the web application
Running web application

Remarks:

No, you don’t need to use the context to access the EJB, you could use the annotation approach, provided both Netbeans projects are part of one enterprise application.
In part 3 we will tinker with real databases, PostgreSQL and Oracle.

ZK goes EC2 (Part 2)

devdude — Mon, 31 Oct 2011 02:25:51 +0000

Part 1 of this tutorial we concluded with an web application that displays all our instances and their status. In this part we will add some more features to control our instances.

Prerequisites:

The project and environment from part 1

Tutorial (complete sourcecode at the end):

Display the region endpoints and allow to select different one
We hardcoded our endpoint in the first version, if you run instances across the globe in the various AWS datacentres (US, Ireland, Singapore, Tokyo) we need to switch the endpoint easily.
Lets add one more listbox, that we hide in a ZK popup (we could use a combo listbox, but for the sake of playing with all the available ZK components I use the popup). Same concept add a listbox in the zul and a EC2 region list and a list model with customer renderer in our controller.

Retrieve regions

Custom renderer

Listbox in popup
Start and Stop function for the selected instance
We use the startInstances and stopInstances provided by the Amazon SDK, these functions expect a list of ID’s.
Add 2 additional buttons and related controller functions

Start and Stop instances

Note, these are async calls, we wont know how long it takes to perform the startup or shutdown, so we just throw the result message (JSON which needs to be beautified) as status at the user and ask him to refresh after a while.
Run the application
Running Web Application

Display Regions

Remarks:

The application is very basic, we still have the access keys hardcoded and no security like a login to protect this web application is in place. I recommend to implement this if you plan to use this on the public web
There is still some features to be added and improved, we will cover this in part 3.

Complete sourcecode:
instances.zul

instancesController.java

package controller;

import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.ec2.AmazonEC2;
import com.amazonaws.services.ec2.AmazonEC2Client;
import com.amazonaws.services.ec2.model.DescribeInstancesResult;
import com.amazonaws.services.ec2.model.DescribeRegionsResult;
import com.amazonaws.services.ec2.model.Instance;
import com.amazonaws.services.ec2.model.InstanceState;
import com.amazonaws.services.ec2.model.Region;
import com.amazonaws.services.ec2.model.Reservation;
import com.amazonaws.services.ec2.model.StartInstancesRequest;
import com.amazonaws.services.ec2.model.StartInstancesResult;
import com.amazonaws.services.ec2.model.StopInstancesRequest;
import com.amazonaws.services.ec2.model.StopInstancesResult;
import com.amazonaws.services.ec2.model.Tag;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import org.zkoss.zk.ui.Component;
import org.zkoss.zk.ui.event.Event;
import org.zkoss.zk.ui.util.ComposerExt;
import org.zkoss.zk.ui.util.GenericForwardComposer;
import org.zkoss.zul.Label;
import org.zkoss.zul.ListModelList;
import org.zkoss.zul.Listbox;
import org.zkoss.zul.Listcell;
import org.zkoss.zul.Listitem;
import org.zkoss.zul.ListitemRenderer;
import org.zkoss.zul.Messagebox;
import org.zkoss.zul.Popup;

public class instancesController extends GenericForwardComposer implements ComposerExt {

    // ZK Variables
    private Listbox lstInstance;
    private Listbox lstRegion;
    private Label lblStatus;
    private Label lblEndpoint;
    private Popup regions;

    //Amazon Variables
    AmazonEC2 ec2;
    List listEC2Reservations = null;
    List listEC2Instances = null;
    List listEC2Regions = null;

    String msgboxTitle = "ZKEC2CloudControl";

    String defEndpoint = "ec2.ap-southeast-1.amazonaws.com";

    private void initEC2() {
        BasicAWSCredentials ecProp = new BasicAWSCredentials("{YOURKEY}", "{YOURSCRETKEY});
        ec2 = new AmazonEC2Client(ecProp);
        ec2.setEndpoint(defEndpoint);
    }

    @Override
    public void doAfterCompose(Component comp) throws Exception {
        super.doAfterCompose(comp);

        initEC2();
        setupRenderer();

        lblEndpoint.setValue(defEndpoint);
        listReservationsInstances();

        // Retrieve the available EC2 regions
        DescribeRegionsResult regionsResult = ec2.describeRegions();
        listEC2Regions = regionsResult.getRegions();
        lstRegion.setModel(new ListModelList(listEC2Regions));

    }

    private void listReservationsInstances() {

        DescribeInstancesResult describeInstancesRequest = ec2.describeInstances();
        listEC2Reservations = describeInstancesRequest.getReservations();

        Set instances = new HashSet();
        for (Reservation reservation : listEC2Reservations) {
            instances.addAll(reservation.getInstances());
        }
        listEC2Instances = new ArrayList(instances);

        lstInstance.setModel(new ListModelList(listEC2Instances));

    }

    public void onClick$btnRefresh(Event evt) throws InterruptedException {
        listReservationsInstances();
        lblStatus.setValue("");
    }

    public void onClick$btnSelectEndpoint(Event evt) throws InterruptedException {
         if (lstRegion.getSelectedIndex() > -1) {
             String newEndpoint = ((Region)lstRegion.getSelectedItem().getValue()).getEndpoint();
             ec2.setEndpoint(newEndpoint);
             lblEndpoint.setValue(newEndpoint);
             listReservationsInstances();
             regions.close();
         }
    }

    public void onClick$btnStart(Event evt) throws InterruptedException {
        if (lstInstance.getSelectedIndex() > -1) {

            if (Messagebox.show("Start instance <" + ((Instance) lstInstance.getSelectedItem().getValue()).getInstanceId() + "> ?", msgboxTitle, Messagebox.OK | Messagebox.CANCEL, Messagebox.QUESTION, null) == Messagebox.OK) {
                List startIDs = new ArrayList();
                startIDs.add(((Instance) lstInstance.getSelectedItem().getValue()).getInstanceId());
                StartInstancesRequest start = new StartInstancesRequest(startIDs);
                StartInstancesResult result = ec2.startInstances(start);
                lblStatus.setValue(result.toString() + " Refresh after 10 or more seconds.");
            }
        }
    }

    public void onClick$btnStop(Event evt) throws InterruptedException {

        if (lstInstance.getSelectedIndex() > -1) {

            if (Messagebox.show("Stop instance <" + ((Instance) lstInstance.getSelectedItem().getValue()).getInstanceId() + "> ?", msgboxTitle, Messagebox.OK | Messagebox.CANCEL, Messagebox.QUESTION, null) == Messagebox.OK) {
                List stopIDs = new ArrayList();
                stopIDs.add(((Instance) lstInstance.getSelectedItem().getValue()).getInstanceId());
                StopInstancesRequest stop = new StopInstancesRequest(stopIDs);
                StopInstancesResult result = ec2.stopInstances(stop);
                lblStatus.setValue(result.toString() + " Refresh after 10 or more seconds.");
            }
        }
    }

    private void setupRenderer() {

        // EC2 Instances list renderer
        ListitemRenderer listRenderInstance = new ListitemRenderer() {

            @Override
            public void render(Listitem item, Object data) throws Exception {
                item.setValue(data);
                item.appendChild(new Listcell(((Instance) data).getInstanceId()));
                item.appendChild(new Listcell(findTagValuebyKey(((Instance) data).getTags(), "name")));
                item.appendChild(new Listcell(((Instance) data).getPublicIpAddress()));

                InstanceState state = ((Instance) data).getState();
                item.appendChild(new Listcell(state.getName()));

                item.appendChild(new Listcell(((Instance) data).getLaunchTime().toString()));

                Listcell listcell = new Listcell();
                item.appendChild(listcell);
            }
        };
        lstInstance.setItemRenderer(listRenderInstance);

        // EC2 Regions list renderer
        ListitemRenderer listitem = new ListitemRenderer() {

            @Override
            public void render(Listitem item, Object data) throws Exception {
                item.setValue(data);
                item.appendChild(new Listcell(((Region) data).getRegionName()));
                item.appendChild(new Listcell(((Region) data).getEndpoint()));
                Listcell listcell = new Listcell();
                item.appendChild(listcell);
            }
        };
        lstRegion.setItemRenderer(listitem);

    }

    private String findTagValuebyKey(List tags, String key) {

        for (Tag tag : tags) {
            if (tag.getKey().toUpperCase().equals(key.toUpperCase())) {
                return tag.getValue();
            }
        }
        return "na";
    }
}

Netbeans + Visual Paradigm = EJB Tutorial

devdude — Fri, 28 Oct 2011 08:50:04 +0000

The only commercial product in my toolbox is Visual Paradigm, a professional UML and modeling tool that integrates with the Netbeans IDE. If you are sincere about ERD, Use-Case and class models you should look at it. The available Netbeans plugins for UML could not convince me yet, too simple (but free). Visual Paradigm seems to operate at the other end, looks rather overloaded with functionality, but once you know what you want and where the function is located, you can create a nice workflow.

In this tutorial (created for internal purpose) I share the steps to create an EJB and the ERD model and let VP (Visual Paradigm) create the class source code for me.

Requirements:

Netbeans 7.x
Visual Paradigm (Professional Version)
You can download the 30 days trial, the free community edition is not sufficient.

Tutorial Part 1:

Create EJB module ‘demoejb’
New EJB module

New EJB module

New EJB module
Create the VP project file
If you have properly integrated NB and VP you should see this additional buttons

Integration

Select the file view, select the project and click the VP icon
(It makes a difference ! If you select the project in the projects explorer and click the VP icon the file gets created under /src/vpproject and the code creation fails later on, not sure if this is a feature or a bug as of VP version 5.2)

Starting VP
Create a ERD diagram with our DB table layout
For this sample we will use the simple name-address sample
customer_name and customer_address (the tables are not complete, rather a base for the demo walk-through)
We use a UUID in all tables as primary key.

ERD

customer_name
primary key: pk_customer_name

customer_name

customer_address
primary key: pl_customer_address

customer_address

Create a one-to-many relationship
Click the ‘one-to-many’ tool, then drag from customer_name to customer_address, and enter:
fk_customername_customeraddress as new column for the relationship

Relationship Mapping

Create column

Resulting ERD

ERD
Create classdiagramm
ORM

Class Diagram (default)

Change the stereotype to entity bean for both classes

Stereotype

Final Class Diagram

Class Diagram
Synchronize with Netbeans (code creation)
Sync with NB

Created package and classes

Code Creation

There is a bug in the code creation, the imports dont get properly created (as in current version 5.2)

Import Problem

Fix the problem manually by importing
import java.sql.Date;
import java.sql.Timestamp;

You can check the Netbeans Settings in VP

NB settings in VP
Add Eclipse library to project
(we need for JPA queries at a later stage)

Eclipse Libraries
Create Meta Model
Just clean and build will result in meta class created.

Meta class
Create Persistence Unit ‘demoejbPU’
For now we use the default sample DB

Create PU

Create PU
Create Session Bean for Entity Class
in package facade

Session Bean

Session Bean
Final project structure
Project Structure
Deploy the EJB
The tables get created (depending on your PU strategy)

Database

Remarks:

Even VP supports round-trips, means changes in the EJB class can be sync’d back to your VP project, I prefer to maintain all DB changes in VP and recreate class diagram and classes, you better dont touch the class asit gets overwritten.
Be careful with deletions and name changes of columns, they are not ‘forwarded’ to your already existing classes (source code).

In Part 2 we will create a ZK client web application and tinker with different DB’s (Oracle and PostgreSQL). Stay tuned.

d3.js Tree: most simple sample

devdude — Thu, 27 Oct 2011 06:41:11 +0000

Learning d3.js might not be as easy as some other tools because there are no step-by-step instructions to get started on a hello-world level. The samples on the d3.js site are bit too complex for a noob. But browsing through the example folder of the d3.js download helps, and usually you learn most dissecting by re-assembling sample code from other people. Another great source is gist.github.com, a code snippet repository. Look for d3.js ! With bl.ocks.org you can even “run” the snippets.

The treeview is a classic visualization but one of my favorites, specifically if you spice it up with interaction and allow the user to dive into a subtree of his choice. But what is the most simple tree layout ? I want to understand the physics behind it. So let me share my compiled (of different samples) most simple version without any fancy features or interaction.

Simlpe Tree Layout

Here my bl.ocks.org version with code or here to copy and paste:



  
    Simple Tree Demo

Interesting to have a look at the raw data and the nodes/link data array

Tree Data structure