Project: Visualizing Trac Wiki Pages (Part 2)

More information before we start with the implementation:

Graph requirements:

• Display all wikipages as nodes
• Show direction (links are uni-directional)

Option Requirements:

• Highlight direct connected wikipages
• Display content (in separate browser window)

Wiki Graph

The XML for prefuse graphs look like this sample (our groovy script needs to produce this):

<?xml version="1.0" encoding="UTF-8"?><graphml xmlns="http://graphml.graphdrawing.org/xmlns">
 <graph edgedefault="directed">
 <key id="wname" for="node" attr.name="wname" attr.type="string"/>
 <node id="1">
 <data key="wname">WikiStart</data>
 </node>
 <node id="2">
 <data key="wname">IdeOverview</data>
 </node>
 <node id="3">
 <data key="wname">JavaEE</data>
 </node>
[..]
 <edge source="1" target="3"/>
 <edge source="1" target="4"/>
 <edge source="1" target="5"/>
 <edge source="1" target="6"/>
 <edge source="1" target="7"/>
 <edge source="1" target="13"/>
 <edge source="3" target="4"/>
 [..]
 </graph>
</graphml>

The prefuse xml file contains the metadata for the nodes, all nodes and all edges linking the nodes.

Using the backup function of Trac we need to process input like this sample:

INSERT INTO "wiki" VALUES('wikiPage1',2,1246948793,'user','220.255.7.193',' * MySQL on Server xyz * Evaluate Firebird and Oracle. (schema and timestamp issue)','',0);

The SQL file contains the complete wiki content. Following the sample we need to filter with

• type: INSERT INTO ‘wiki’ (filter out all others)
• page title: ‘wikiPage1‘
• version: ‘2‘ (search for the highest version)
• wiki user: ‘user‘ (filter out all ‘trac’ entries)
• content: ‘* MySQL on Server xyz * Evaluate Firebird and Oracle. (schema and timestamp issue)‘

Project: Visualizing Trac Wiki Pages (Part 1)

We are using Trac as Wiki and after a couple of months using it as main know-how repository, we tend to loose the overview of the myriad of pages and how they are linked to each other. A graph representing the relation between the pages would be very useful.

I could not find any plugin or tool, so it would be a good exercise to create a tool that convert a trac wiki into a graph.

Options:

• Use Java or Groovy. I choose Groovy because less boilerplate for standard tasks, such as creating xml, reading text files, etc.
• XML-RPC to access the wiki directly. I tried to create a mockup in Java, but failed due to the cryptic error messages and hard to debug.
• Trac backup to retrieve wiki data: You can create a backup in form of an INSERT sql file containing the complete structure of the wiki (inclusive of non-user content and all revisions of the wiki. I choose this as a starting point until I get the XML-RPC problem solved (maybe in Groovy).
• Visualization in JGraph, InfoVis, Prefuse or other toolkits. I decide for Prefuse because it is feature complete, powerful and I created simple applications previously.

Current Implementation:

• A simple Groovy script that works in 2 stages, a) reading and processing the sql backup file and b) creating a xml file that my prefuse viewer can display.

Future Plans:

• Create a standalone SwingXBuilder application
• Create a Netbeans Plugin

How to get started with Prefuse and Netbeans

I was looking for a java based visualization toolkit for quite a while. Either they are very static (image rendered and delivered without any interaction) or cost too much money or badly documented or plain buggy and incomplete. I found a toolkit called prefuse (link) on sourceforge which is an amazing piece of work done by Jeffrey Heer (link) of Berkeley University Stanford. Thanks for this and to the people in the forum (link) to continue the effort to answer questions.

Though it’s last release if dating 2007.10 it is feature complete and not really need to carry the beta tag, but never mind. The project migrated to flare (link) based on Flash and AS, but unfortunately that is not always the first choice, even more powerful in terms of graphics. But I wonder what would be missing. It is open source, so with community effort things can be implemented or even added.
It is released under BSD and can be used freely for commercial and non-commercial applications.

I am a true Netbean nerd, aka NB user and spent some time to get started with prefuse.

Today I wan to share with you the steps (i did) to get a running with prefuse and Netbeans running Windows.
(Please note, it might not be the perfect or smartest approach. I dont claim its the right way or complete, but I got it to work this way. )
I am sure this is very basic information for the prefuse experts, but other beginners might need to go through the same steps.

1. Donwload the file prefuse-beta-20071021.zip (link).

2. Unzip the file to folder.

3. Check for a properly set JAVA_HOME path, like

E:\Documents and Settings\SAM>echo %JAVA_HOME%
E:\Program Files\Java\jdk1.6.0_10

4. Open the file build.bat with text editor of your choice and change the follwoing line
%JAVA_HOME%\bin\java.exe -Dant.home=…
to
java.exe -Dant.home=…
(I removed the JAVA_HOME because java already is in the system PATH)

5. Open a DOS-Box (Windows, aka run.. CMD) and navigate to the folder with the build.bat file.

6. Execute ‘build.bat all’. You should get something like this:

prefuse and Netbeans: Build the jarfiles

7. Start Netbeans and create a new project (New, Choose project, Java, Java Application). Lets call it PrefuseDemo.

8. Open the project properties (right click on the project name in the project explorer) and select properties. Go to the libraries sectio and add prefuse.jar and demo.jar (required for the sample code). You find these files in the folder ‘build’ created by our build.bat.

9. Now you change the packagename (with the initial created Main.Java if selected) to prefusedemo.

10. In the build folder you also find a src folder with various sample java code. For this tutorial I took GraphView.java.

11. Create a new javaclass in the prefuse package with the name GraphView.

12. Copy the complete sourcecode from GraphView.java into it and replace the basic skeleton.

13. Fix the package name to package prefusedemo;

14. You can delete the Main.java if NB created it for you.

15. Build the project.

16. Right click onGraphView.java and select Run File. Finished ! You should see this:

prefuse and Netbeans: Run the demo

Remarks:
1. The other files (demos that come with the prefuse package) you can create the same way and add them into your demo application.
2. I tried to import the whole prefuse source folder into Netbeans via Import, but it didnt work.

Conclusion:
Now we got started, lets move on to understand the prefuse concept and adapt some of the samples to our own applications or create new ones from the scratch ! Stay tuned for more prefuse !