Overview

Lets you configure tools so that slaves will install them on demand whenever running a job that needs them. This could be especially useful when running a large farm of slaves in a cloud which all start with a minimal operating system configuration.

Several techniques are already supported: run arbitrary shell command; download & unpack archive from arbitrary URL; download and install official Sun JDK, Ant release, or Maven release.

The configuration is part of a tool; a project owner just picks a tool (JDK, Ant, Maven, ...) by name as usual, and may observe a build running some extra steps if it needs to perform an installation, but it is otherwise transparent. Installers can be limited to slaves marked with a label.

Demo Configuration

Master

Hudson 1.311 on localhost:8080 with no executors.

Node config

One JNLP slave node ubuntu-vbox with one executor and Remote FS root set to /hudson.

Tool Config

1. JDK installations has one entry with name JDK 6 and a JDK installer set to 6u13.
2. Ant installation has one entry with name Ant 1.7.1 and an Ant installer set to 1.7.1.

Test Project

Freestyle project, no SCM. Two build steps:

1. Execute shell with Command:

   echo "JDK: $JAVA_HOME"
   echo '<project/>' > build.xml
   

2. Invoke Ant with Targets set to -version.

Slave

An Ubuntu 8.10 server installation inside VirtualBox (NAT networking) with only default-jre-headless installed and user hudson added.

/etc/rc.local

#!/bin/sh -e
su - hudson -c /hudson/slave-jnlp 2>&1 | perl -p -e 's/\n/\r\n/g' 1>&2 &

/etc/sudoers

# ...
hudson ALL=NOPASSWD: ALL

/hudson/slave-jnlp

#!/bin/sh
dir=`dirname $0`
server=http://10.0.2.2:8080
node=ubuntu-vbox
while :
do
    wget -O $dir/slave.jar $server/jnlpJars/slave.jar && \
        java -jar $dir/slave.jar -jnlpUrl $server/computer/$node/slave-agent.jnlp
    sleep 15
done

Expected project console output

...
Building remotely on ubuntu-vbox
...
JDK: /hudson/tools/JDK_6
...
[...] $ /hudson/tools/Ant_1.7.1/bin/ant -version
Apache Ant version 1.7.1 compiled on June 27 2008
Finished: SUCCESS

Suggested features

1. Install tools from an SCM such as Subversion.
2. Upload a tool archive directly into Hudson's web GUI.
3. Cache downloaded tools on master to avoid excess network traffic. (Already done for JDKs.)

Related: Tool Environment Plugin

Adding new versions

Hudson gets the information where to download the versions via a JSON download from the internet.

The JSON data for the tools are stored in the files https://hudson.dev.java.net/svn/hudson/trunk/www/updates/, so adding a new version of an existing tool would require a path to the related file. The JSON data for some tools is created by backend/crawler in Hudson sources.

More tools needed!

More things should extend ToolInstallation for this system to be really useful. An incomplete list:

1. Languages & frameworks
   1. Ruby
   2. Groovy
   3. Rails
2. Builders
   1. Rake
   2. NAnt
   3. MSBuild: issue #4120
   4. Ant, Maven, Gradle, Buckminster, SCons, WAS Builder, RAD Builder, STAF seem done
3. SCMs
   1. CVS
   2. Git
   3. Mercurial available as of 1.18
   4. unnecessary for SVN, which Hudson runs from Java