Running JavaDoc/JUnit/Emma on Hudson using Buckminster

Last week I wrote about "Building Products with Buckminster/Hudson", today I want to share some of my experience about my JavaDoc job, running JUnit tests and Emma code coverage.

JavaDoc

My goal was to generate JavaDoc for our framework softmodeler and the server/client application.

A short overview about the checkout sources:
${workspace}/source/softmodeler/plugins
${workspace}/source/scodi-server/plugins
${workspace}/source/scodi-rcp/plugins

To prevent errors (some errors prevent JavaDoc from creating the index.html and related files) it's important to set the classpath, in my case the target platform. So I pass the location of the target platform and use a fileset to get all the jars together. This path is then referred using classpathref="files-classpath" in the javadoc call.

If you get errors about too long filenames and such, make sure you use useexternalfile="true", more information on that here.
For the actual Javadoc task I use a bunch of filesets, excluding some unwanted packages.

Here the ant "create.javadoc" target:

<target name="create.javadoc" description="Generate the JavaDoc for the sources">
 <echo message="javadoc source ${source}"></echo>
 <echo message="javadoc destination ${javadoc.output}"></echo>
 <echo message="target platform ${target.platform}"></echo>


 <!-- set target platform as classpath -->
 <path id="files-classpath">
  <fileset dir="${target.platform}">
   <include name="*.jar"/>
  </fileset>
 </path>


 <!-- clean and create output location -->
 <delete dir="${javadoc.output}"/>
 <mkdir dir="${javadoc.output}"/>


 <!-- generate the javadoc -->
 <javadoc
  destdir="${javadoc.output}"
  classpathref="files-classpath"
  maxmemory="1024m"
  source="1.6"
  useexternalfile="true"
  author="true"
  version="true"
  use="true"
  windowtitle="Scodi/Softmodeler Documentation">
   <!-- link external APIs -->
   <link offline="false" href="http://java.sun.com/javase/6/docs/api/"/>
   <link offline="false" href="http://www.osgi.org/javadoc/r4v42/"/>
   <link offline="false" href="http://help.eclipse.org/galileo/topic/org.eclipse.platform.doc.isv/reference/api/"/>
   <link offline="false" href="http://download.eclipse.org/modeling/emf/emf/javadoc/2.5.0/"/>
   <link offline="false" href="http://docs.huihoo.com/javadoc/jboss/jbpm/4.1/"/>
   <link offline="false" href="http://technology-related.com/javaee/5/docs/api/"/>
   <link offline="false" href="http://docs.jboss.org/hibernate/stable/core/api/"/>
   <link offline="false" href="http://docs.jboss.org/hibernate/stable/annotations/api/"/>
   <link offline="false" href="http://docs.jboss.org/hibernate/stable/entitymanager/api/"/>
   <link offline="false" href="http://jackrabbit.apache.org/api/1.4/"/>
   <link offline="false" href="http://www.day.com/maven/jsr170/javadocs/jcr-1.0/"/>

   <!-- softmodeler sources -->
   <fileset dir="${source}/softmodeler/plugins/" defaultexcludes="true">
    <include name="**/*.java"/>
    <exclude name="**/org/**"/>
    <exclude name="**/net/**"/>
    <exclude name="**/test/**"/>
   </fileset>

   <!-- scodi sources -->
   <fileset dir="${source}/scodi-server/plugins/">
    <include name="**/*.java"/>
    <exclude name="**/test/**"/>
   </fileset>
   <fileset dir="${source}/scodi-rcp/plugins/">
    <include name="**/*.java"/>
    <exclude name="**/test/**"/>
    <exclude name="ch.scodi.mig/**"/>
   </fileset>

   <bottom><![CDATA[<i>Copyright © 2007 henzler informatik gmbh, CH-4106 Therwil</i>]]></bottom>
 </javadoc>
</target>

To be able to launch the ant task from Buckminster I had to add the following action to buckminster.cspex:

<cs:public name="create.javadoc" actor="ant">
  <cs:actorproperties>
      <cs:property key="buildFile" value="build/javadoc.ant">
      <cs:property key="targets" value="create.javadoc">
  </cs:property>
  <cs:properties>
      <cs:property key="source" value="${workspace}source">
      <cs:property key="javadoc.output" value="${workspace}javadoc">
  </cs:property>
</cs:property>

The Hudson job then needs to checkout the source and run a build step "Run Buckminster":

import '${WORKSPACE}source/scodi-rcp/features/ch.scodi.client.site/site.cquery'

perform -D workspace=${WORKSPACE} -D target.platform=${WORKSPACE}../../target.platform/workspace/.metadata/.plugins/org.eclipse.pde.core/.bundle_pool/plugins/ ch.scodi.client.site#create.javadoc

I know the target.platform path is ugly, didn't find a pre-defined variable. Tried ${targetPlatformPath} but that somehow didn't work, any hints?

You then can publish the JavaDoc using the "Post-Build-Action".


Junit & Emma

Buckminster provides the command "junit" which allows you to launch "JUnit Plug-In Tests". This is a really great feature, because you can run tests in your eclipse environment very easy.
I ran into some problems because my launch file was not found, the launch needs to be within your workspace (not your checkout sources).
I imported my product site.query and did not realize that my test feature (containing the launch file) was not part of that. So additionally I had to import my test feature (see below) and it worked.

import '${WORKSPACE}source/scodi-server/features/ch.scodi.server.site/site.cquery'
import '${WORKSPACE}source/scodi-server/features/ch.scodi.server.test.site/site.cquery'

build

perform -D target.os=* -D target.ws=* -D target.arch=* -D qualifier.replacement.*=${version} ch.scodi.server.site#site.p2
perform -D target.os=win32 -D target.ws=win32 -D target.arch=x86 ch.scodi.server.site#create.product.zip

junit -l '/ch.scodi.server.test.site/ScodiServerTest.launch' -o '${WORKSPACE}output/junit_result.xml'


There is a "Post-Build-Action" to publish JUnit results. It somehow does not work with the generated output and caused my build to fail.
A great alternative is the "Performance Plugin", which publishes your result and also performance trends.

Martin Taal, founder and lead of the EMF Teneo project, wrote a very useful wiki article about Teneo building with Buckminster and Hudson.
Interesting for me was the Emma part.
To get Emma coverage reports do the following:
- Install the Emma Plug-In
- Install org.eclipse.buckminster.emma.headless.feature.feature.group to the Buckminster installation
- change the "junit" command to "emma"
- add an additional paramter for the coverage report
- Your done. Awesome!!!

emma -l '/ch.scodi.server.test.site/ScodiServerTest.launch' -o '${WORKSPACE}output/junit_result.xml' --xml '${WORKSPACE}/output/coverage_report.xml' --flatXML

The Emma "Post-Build-Action" then publishes your coverage report.

Buckminster and Hudson, a great combination which makes releng of eclipse based products so much easier. Thanks to the Buckminster team!!!

Building Products with Buckminster/Hudson

I just finished setting up our Buckminster/Hudson build server. Due to lack of documentation it was a real struggle, sharing some of my experience may help other developers.

I used Ralf's tutorial to get an good overview about the topic, great blog.

First up some information about the project I'm working on. It's a server-client application (two separated products) called scodi, which is based on our framework softmodeler. For a more detailed overview you can read my previous post, if your interested.

Target Platform

How to setup Hudson and Buckminster can be read in Ralf's tutorial. Little tip, to prevent OutOfMemoryErrors, add -Xmx1024m to the "additional parameters" of your Buckminster installation (see troubleshooting tip Hudson out of memory).

I have a separated free style job to publish my target platform for other jobs. In the "Source-Code-Management" section I checkout the feature which contains my target defintion (in my case ch.scodi.client.site).
To actually resolve the target definition, I added a build step "Run Buckminster" with the following command:
importtargetdefinition -A '${WORKSPACE}ch.scodi.client.site/TargetDefinition.target'

In the "Post-Build-Action" checked "Archive and publish an Eclipse Target Platform" and added ".metadata/.plugins/org.eclipse.pde.core/.bundle_pool" as path.

Consider that the TargetDefinition can not resolve directory locations. My target definition used to have a directory location containing bundles from the springsource repository.
I tried using the rmap file to get the bundles during materialization but had some trouble with that, so I decided to create an own update site for those bundles and add this site to the target definition. More on that can be found here: http://www.eclipse.org/forums/index.php?t=msg&th=164508&start=0&

Building the Product

After the target definition job is run, we can start building the products.
This is pretty straight forward, see Ralf's tutorial on how to checkout your source from SVN.
I have three different builds for each, server and client product: Integration, Nightly and Release.
For each build the plug-in qualifier should be different (e.g. I20100326-2, N20100326, R20100326-01).
To accomplish this I installed the flowing plug-in: http://wiki.hudson-ci.org/display/HUDSON/Version+Number+Plugin
In the integration job I choose "Create a formatted version number" name it "version" and use something like this "I${BUILD_YEAR, XXXX}${BUILD_MONTH, XX}${BUILD_DAY, XX}-${BUILDS_TODAY}" as format.

To finally build the client product I added a Buckminster build step, selected the previously published target platform and used the following as commands:
import '${WORKSPACE}source/scodi-rcp/features/ch.scodi.client.site/site.cquery'

build

perform -D target.os=* -D target.ws=* -D target.arch=* -D qualifier.replacement.*=${version} ch.scodi.client.site#site.p2.zip
perform -D target.os=win32 -D target.ws=win32 -D target.arch=x86 ch.scodi.client.site#create.product.zip
perform -D target.os=win32 -D target.ws=win32 -D target.arch=x86_64 ch.scodi.client.site#create.product.zip

Notice qualifier.replacement.*=${version}, this tells Buckminster/Eclipse to use my formated version as qualifier and results in plug-ins named like this "com.softmodeler.model_1.0.0.I20100325-3.jar", requires that Bundle-Version: 1.0.0.qualifier is defined in the bundle manifest.

Ok this post is getting long and I'm tired.
I will post some more next week about my JavaDoc build and running JUnit Tests.