Running JavaDoc/JUnit/Emma on Hudson using Buckminster

Last week I wrote about "Building Products with Buckminster/Hudson", today I want to share some of my experience about my JavaDoc job, running JUnit tests and Emma code coverage.

JavaDoc

My goal was to generate JavaDoc for our framework softmodeler and the server/client application.

A short overview about the checkout sources:
${workspace}/source/softmodeler/plugins
${workspace}/source/scodi-server/plugins
${workspace}/source/scodi-rcp/plugins

To prevent errors (some errors prevent JavaDoc from creating the index.html and related files) it's important to set the classpath, in my case the target platform. So I pass the location of the target platform and use a fileset to get all the jars together. This path is then referred using classpathref="files-classpath" in the javadoc call.

If you get errors about too long filenames and such, make sure you use useexternalfile="true", more information on that here.
For the actual Javadoc task I use a bunch of filesets, excluding some unwanted packages.

Here the ant "create.javadoc" target:

<target name="create.javadoc" description="Generate the JavaDoc for the sources">
 <echo message="javadoc source ${source}"></echo>
 <echo message="javadoc destination ${javadoc.output}"></echo>
 <echo message="target platform ${target.platform}"></echo>


 <!-- set target platform as classpath -->
 <path id="files-classpath">
  <fileset dir="${target.platform}">
   <include name="*.jar"/>
  </fileset>
 </path>


 <!-- clean and create output location -->
 <delete dir="${javadoc.output}"/>
 <mkdir dir="${javadoc.output}"/>


 <!-- generate the javadoc -->
 <javadoc
  destdir="${javadoc.output}"
  classpathref="files-classpath"
  maxmemory="1024m"
  source="1.6"
  useexternalfile="true"
  author="true"
  version="true"
  use="true"
  windowtitle="Scodi/Softmodeler Documentation">
   <!-- link external APIs -->
   <link offline="false" href="http://java.sun.com/javase/6/docs/api/"/>
   <link offline="false" href="http://www.osgi.org/javadoc/r4v42/"/>
   <link offline="false" href="http://help.eclipse.org/galileo/topic/org.eclipse.platform.doc.isv/reference/api/"/>
   <link offline="false" href="http://download.eclipse.org/modeling/emf/emf/javadoc/2.5.0/"/>
   <link offline="false" href="http://docs.huihoo.com/javadoc/jboss/jbpm/4.1/"/>
   <link offline="false" href="http://technology-related.com/javaee/5/docs/api/"/>
   <link offline="false" href="http://docs.jboss.org/hibernate/stable/core/api/"/>
   <link offline="false" href="http://docs.jboss.org/hibernate/stable/annotations/api/"/>
   <link offline="false" href="http://docs.jboss.org/hibernate/stable/entitymanager/api/"/>
   <link offline="false" href="http://jackrabbit.apache.org/api/1.4/"/>
   <link offline="false" href="http://www.day.com/maven/jsr170/javadocs/jcr-1.0/"/>

   <!-- softmodeler sources -->
   <fileset dir="${source}/softmodeler/plugins/" defaultexcludes="true">
    <include name="**/*.java"/>
    <exclude name="**/org/**"/>
    <exclude name="**/net/**"/>
    <exclude name="**/test/**"/>
   </fileset>

   <!-- scodi sources -->
   <fileset dir="${source}/scodi-server/plugins/">
    <include name="**/*.java"/>
    <exclude name="**/test/**"/>
   </fileset>
   <fileset dir="${source}/scodi-rcp/plugins/">
    <include name="**/*.java"/>
    <exclude name="**/test/**"/>
    <exclude name="ch.scodi.mig/**"/>
   </fileset>

   <bottom><![CDATA[<i>Copyright © 2007 henzler informatik gmbh, CH-4106 Therwil</i>]]></bottom>
 </javadoc>
</target>

To be able to launch the ant task from Buckminster I had to add the following action to buckminster.cspex:

<cs:public name="create.javadoc" actor="ant">
  <cs:actorproperties>
      <cs:property key="buildFile" value="build/javadoc.ant">
      <cs:property key="targets" value="create.javadoc">
  </cs:property>
  <cs:properties>
      <cs:property key="source" value="${workspace}source">
      <cs:property key="javadoc.output" value="${workspace}javadoc">
  </cs:property>
</cs:property>

The Hudson job then needs to checkout the source and run a build step "Run Buckminster":

import '${WORKSPACE}source/scodi-rcp/features/ch.scodi.client.site/site.cquery'

perform -D workspace=${WORKSPACE} -D target.platform=${WORKSPACE}../../target.platform/workspace/.metadata/.plugins/org.eclipse.pde.core/.bundle_pool/plugins/ ch.scodi.client.site#create.javadoc

I know the target.platform path is ugly, didn't find a pre-defined variable. Tried ${targetPlatformPath} but that somehow didn't work, any hints?

You then can publish the JavaDoc using the "Post-Build-Action".


Junit & Emma

Buckminster provides the command "junit" which allows you to launch "JUnit Plug-In Tests". This is a really great feature, because you can run tests in your eclipse environment very easy.
I ran into some problems because my launch file was not found, the launch needs to be within your workspace (not your checkout sources).
I imported my product site.query and did not realize that my test feature (containing the launch file) was not part of that. So additionally I had to import my test feature (see below) and it worked.

import '${WORKSPACE}source/scodi-server/features/ch.scodi.server.site/site.cquery'
import '${WORKSPACE}source/scodi-server/features/ch.scodi.server.test.site/site.cquery'

build

perform -D target.os=* -D target.ws=* -D target.arch=* -D qualifier.replacement.*=${version} ch.scodi.server.site#site.p2
perform -D target.os=win32 -D target.ws=win32 -D target.arch=x86 ch.scodi.server.site#create.product.zip

junit -l '/ch.scodi.server.test.site/ScodiServerTest.launch' -o '${WORKSPACE}output/junit_result.xml'


There is a "Post-Build-Action" to publish JUnit results. It somehow does not work with the generated output and caused my build to fail.
A great alternative is the "Performance Plugin", which publishes your result and also performance trends.

Martin Taal, founder and lead of the EMF Teneo project, wrote a very useful wiki article about Teneo building with Buckminster and Hudson.
Interesting for me was the Emma part.
To get Emma coverage reports do the following:
- Install the Emma Plug-In
- Install org.eclipse.buckminster.emma.headless.feature.feature.group to the Buckminster installation
- change the "junit" command to "emma"
- add an additional paramter for the coverage report
- Your done. Awesome!!!

emma -l '/ch.scodi.server.test.site/ScodiServerTest.launch' -o '${WORKSPACE}output/junit_result.xml' --xml '${WORKSPACE}/output/coverage_report.xml' --flatXML

The Emma "Post-Build-Action" then publishes your coverage report.

Buckminster and Hudson, a great combination which makes releng of eclipse based products so much easier. Thanks to the Buckminster team!!!

Building Products with Buckminster/Hudson

I just finished setting up our Buckminster/Hudson build server. Due to lack of documentation it was a real struggle, sharing some of my experience may help other developers.

I used Ralf's tutorial to get an good overview about the topic, great blog.

First up some information about the project I'm working on. It's a server-client application (two separated products) called scodi, which is based on our framework softmodeler. For a more detailed overview you can read my previous post, if your interested.

Target Platform

How to setup Hudson and Buckminster can be read in Ralf's tutorial. Little tip, to prevent OutOfMemoryErrors, add -Xmx1024m to the "additional parameters" of your Buckminster installation (see troubleshooting tip Hudson out of memory).

I have a separated free style job to publish my target platform for other jobs. In the "Source-Code-Management" section I checkout the feature which contains my target defintion (in my case ch.scodi.client.site).
To actually resolve the target definition, I added a build step "Run Buckminster" with the following command:
importtargetdefinition -A '${WORKSPACE}ch.scodi.client.site/TargetDefinition.target'

In the "Post-Build-Action" checked "Archive and publish an Eclipse Target Platform" and added ".metadata/.plugins/org.eclipse.pde.core/.bundle_pool" as path.

Consider that the TargetDefinition can not resolve directory locations. My target definition used to have a directory location containing bundles from the springsource repository.
I tried using the rmap file to get the bundles during materialization but had some trouble with that, so I decided to create an own update site for those bundles and add this site to the target definition. More on that can be found here: http://www.eclipse.org/forums/index.php?t=msg&th=164508&start=0&

Building the Product

After the target definition job is run, we can start building the products.
This is pretty straight forward, see Ralf's tutorial on how to checkout your source from SVN.
I have three different builds for each, server and client product: Integration, Nightly and Release.
For each build the plug-in qualifier should be different (e.g. I20100326-2, N20100326, R20100326-01).
To accomplish this I installed the flowing plug-in: http://wiki.hudson-ci.org/display/HUDSON/Version+Number+Plugin
In the integration job I choose "Create a formatted version number" name it "version" and use something like this "I${BUILD_YEAR, XXXX}${BUILD_MONTH, XX}${BUILD_DAY, XX}-${BUILDS_TODAY}" as format.

To finally build the client product I added a Buckminster build step, selected the previously published target platform and used the following as commands:
import '${WORKSPACE}source/scodi-rcp/features/ch.scodi.client.site/site.cquery'

build

perform -D target.os=* -D target.ws=* -D target.arch=* -D qualifier.replacement.*=${version} ch.scodi.client.site#site.p2.zip
perform -D target.os=win32 -D target.ws=win32 -D target.arch=x86 ch.scodi.client.site#create.product.zip
perform -D target.os=win32 -D target.ws=win32 -D target.arch=x86_64 ch.scodi.client.site#create.product.zip

Notice qualifier.replacement.*=${version}, this tells Buckminster/Eclipse to use my formated version as qualifier and results in plug-ins named like this "com.softmodeler.model_1.0.0.I20100325-3.jar", requires that Bundle-Version: 1.0.0.qualifier is defined in the bundle manifest.

Ok this post is getting long and I'm tired.
I will post some more next week about my JavaDoc build and running JUnit Tests.

2009

2009 is ending, time to look back at a productive and very interesting year.

As mentioned in earlier posts, I'm working on a software called Scodi which is currently released as version 3.4.x.
For the next version 4.0, we dumped all the old code and rebuild the whole software using new and "fancy" technologies.
Version 3.4 was a fat RCP client, version 4.0 will be a server-client application.

That said, I had the pleasure to integrate a lot of amazing frameworks and APIs into our new software.

EMF to model our business objects, Teneo and Hibernate to persist those objects.
The Spring framework to create our beans and register some of them as OSGi services, Riena to publish them for remote access.
Apache Jackrabbit to store and version some of our data.
jBPM to handle workflows (e.g. data that is being committed needs to be reviewed by certain users) and a Mylyn connector to handle those user tasks.

This November I took a look at GMF to build our Process Flow designer, the designer itself will be developed by another company.

Towards the end of this year I cleaned out our RCP client to make it RAP ready as we have a web part of our software.
The EMF based UI framework which I built last year, uses RCP and SWT per default. The framework works on the web now including databinding, field decorations and such. RAP is great and single sourcing a blessing :-).

Using all these technologies I reported about 30-40 bugs I hope to soon find more time to contribute back to the open source community. My project on sourceforge.net "workreporter" didn't get any commits this year, I was fully concentrating on Scodi.

It was a busy year and I'm looking forward to release version 4.0 in 2010.

Override Eclipse Key Binding

A few weeks back is was struggling with an Eclipse key binding problem. I had to override the CTRL+S binding. Since I had my own "org.eclipse.ui.bindings" extension, I keept on getting conflict messages (no wonder).
So I thought I'll make a post as personal reference and maybe it helps some other developer.

This is what I did in ApplicationActionBarAdvisor.makeActions().

IBindingService bindingService = (IBindingService) window.getService(IBindingService.class);
bindingService.addBindingManagerListener(new IBindingManagerListener() {
   @Override
   public void bindingManagerChanged(BindingManagerEvent event) {
       BindingManager manager = event.getManager();
       for (Binding binding : manager.getBindings()) {
           ParameterizedCommand command = binding.getParameterizedCommand();
           if (command != null && ActionFactory.SAVE.getCommandId().equals(command.getId())) {
               manager.removeBinding(binding);
           }
       }
   }
});

IWorkbenchAction saveDocumentAction = ActionFactory.SAVE.create(window);
saveDocumentAction.setId("saveDocument");
saveDocumentAction.setActionDefinitionId(SAVE_DOCUMENT_COMMAND);

register(saveDocumentAction);

SWT instead of AWT packages

Today I got annoyed using eclipse code completion to add a KeyAdapter on a SWT component. Instead of adding the SWT KeyAdapter (since it's a SWT widget), eclipse suggests the AWT KeyAdapter before the SWT KeyAdapter. The result of this is that I often choose the wrong one.
Same thing with other classes like KeyListener, MouseListener...

So I was looking for a solution, eclipse should ignore the java.awt package.
I have not found a way to handle this for the whole workspace using preferences.
I now exclude the java.awt package in all my UI bundles from their Java Build Path.

Maybe you have had the same "trouble" and never thought about getting rid of it :-)

Interview

At the beginning of this month, my employer (henzler informatik gmbh) sent out a newsletter to all our customers.
Part of it was an interview with myself, about the development I'm involved with and about my working day.
It's in German but if anybody is interested, here is the link:
SCODi 4P Newsletter September 2009: Interview mit Flavio Donzé

Haven't been blogging lately, I'm really busy working on this version 4 of SCODi 4P.

jBPM 4.0 is Out!

jBPM 4.0 was released last Friday, more infos on that can be found here.

As read in the readme.html file, the two bugs I reported were fixed:

[JBPM-2319] - Finished executions are not changed to "ended" (they stay in the "active" state)
[JBPM-2340] - TaskService getTaskComments throws NullPointerException if the task is not found

Thanks guys great work!