DistFork Plugin

Plugin Information

Plugin ID distfork
Latest Release 1.1
Latest Release Date Nov 19, 2010
Sources Subversion
Support Eclipse Hudson Forum
Issue Tracking Eclipse Bugzilla

Turns a Hudson cluster into a general purpose batch job execution environment through an SSH-like CLI.
This plugin adds a new command 'distfork' to Hudson CLI, which can be used to execute arbitrary command on a slave of your choice. The distfork command is modeled after ssh, but it's Hudson aware — for example, instead of hardcoding a machine name, you can specify a label to let Hudson choose a slave. This opens up a Hudson cluster for doing all kinds of things without requiring a job/build notion.

Use Case

  • You have a test that involves multiple nodes. Your test script can use distfork to launch additional JVMs on the Hudson cluster to carry that out. Such test script can be run from both within Hudson or from developer's laptops.
  • You have some computation that is expensive or require specific environment (say, building JDK or building a virtual machine image.) You can submit such computation to Hudson from your shell.
  • As a basis for distributed scripting.

Usage

The following example runs "uname -a" command on a slave with the 'linux' label (if the -l option is optional.)

$ java -jar hudson-cli.jar -s http://server/hudson/ dist-fork -l linux uname -a
[distfork5384565281499633486.tmp] $ uname -a
Linux bear 2.6.28-13-generic #44-Ubuntu SMP Tue Jun 2 07:55:09 UTC 2009 x86_64 GNU/Linux

The following example starts a Tomcat somewhere in a Hudson cluster, with a port-forwarding from the port 9999 to the HTTP listener of Tomcat.

% java -jar hudson-cli.jar -s http://localhost:8080/ dist-fork -z ~/apache-tomcat-5.5.27.tar.gz \\
    -L 9999:localhost:8080 apache-tomcat-5.5.27/bin/catalina.sh run
[distfork6313354709312415597.tmp] $ apache-tomcat-5.5.27/bin/catalina.sh run
Jul 15, 2009 12:25:31 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: The Apache Tomcat Native library which allows optimal performance in production environments was not found on ...
Jul 15, 2009 12:25:31 PM org.apache.coyote.http11.Http11BaseProtocol init
...

Features

  • distfork executions are scheduled in the queue and run by an executor, just like normal builds. Thus it makes Hudson aware of what slaves are busy and what are not.
  • Each command execution gets a one-time temporary directory for its working directory. Hudson will clean this up after the command exits, as well as terminating any remaining background processes to maintain the cluster health.
  • Port-forwarding both ways so that you can talk to the remote process without knowing their actual IP addresses.
  • Ability to stage the files before the process execution, and ability to bring back files after it ends.

Available Command Line Arguments

 -F LOCAL=REMOTE   : Remote files to be copied back to local locations after
                     the execution of a task
 -L PORT:HOST:PORT : Local to remote port forwarding
 -R PORT:HOST:PORT : Remote to local port forwarding
 -Z FILE           : Bring back the newly added/updated files in the target
                     remote machine after the end of the command by creating a
                     zip/tgz bundle and place this in the local file system by
                     this name.
 -d N              : Estimated duration of this task in milliseconds, or -1 if
                     unknown
 -e NAME=VAL       : Environment variables to set to the launched process
 -f REMOTE=LOCAL   : Local files to be copied to remote locations before the
                     exeuction of a task
 -l VAL            : Label for controlling where to execute this command
 -n VAL            : Human readable name that describe this command. Used in
                     Hudson's UI.
 -z FILE           : Zip/tgz file to be extracted into the target remote
                     machine before execution of the command

Changelog

Version 1.1 (Nov 19 2010)

  • Update code for more recent Hudson
  • Cut off the random meaningless name from the archive

Version 1.0 (Jul 15 2009)

  • Initial release

Labels:

plugin-cluster plugin-cluster Delete
plugin-cli plugin-cli Delete
Enter labels to add to this page:
Wait Image 
Looking for a label? Just start typing.
  1. Jan 08, 2010

    J. Longman says:

    Where do I get cli.jar? Is it available as a link from the hudson instance?

    Where do I get cli.jar? Is it available as a link from the hudson instance?

    1. Jan 09, 2010

      J. Longman says:

      Found it - it's maybe worth linking the Hudson CLI wiki page, and of course the ...

      Found it - it's maybe worth linking the Hudson CLI wiki page, and of course the jar has changed name to hudson-cli.jar.

  2. Jan 25, 2010

    J. Longman says:

    This is really cool. I'm currently walking a series of git commits, farming out ...

    This is really cool. I'm currently walking a series of git commits, farming out to a hudson instance which is building each commit, 10 commits at a time.

    Command line
    git rev-list --reverse fc14157a9a5af323db34869..CORE-940_magic | xargs -R 3 -P 10  -tI % \
       java -jar ~/hudson-cli.jar -s http://192.168.1.35:8081/ dist-fork \
       -l BSD -n $USER"-adhoc"-% -e SHA1_ID=% -f remote-step.sh=remote-step.sh \
       sh remote-step.sh   >>/tmp/buildjava
    

    nb: Some of the xargs options may be FreeBSD specific.

    remote-step.sh
    WORKSPACE=$PWD
    echo $SHA1_ID
    # a local to the machine copy of the git repository - could use a git: or ssh: but this should be faster
    git clone /usr/home/hudson/jail/tmp/Splat.git work
    cd work
    git checkout $SHA1_ID
    # build commands go here
    # jexec is a BSD jail'ism, like a chroot - not necessary on most systems
    sudo -E jexec -U root `jid build.local`  scons -C $WORKSPACE/work/element -U . 
    if [ "$?"-ne 0]; then echo "command failed $SHA1_ID"; fi
    # clean up step - MANDATORY for BSD when using root permissions/sudo
    touch $WORKSPACE
    sudo -E chflags -R 0 $WORKSPACE
    sudo -E chown -R hudson $WORKSPACE
    

    The only thing is that right now my script is NOT printing the SHA1 ID of the failed build, probably the sudo and jexec are interfering with the detection of the build failure.

  3. Feb 28, 2013

    Damon says:

    I may be misinterpreting what this plug-in is meant to achieve. I may also be do...

    I may be misinterpreting what this plug-in is meant to achieve. I may also be doing something wrong with it. I'm assuming that this is a base for running work (a script file...anything really) in parallel on multiple jenkins nodes at once in the form of a job. I tried this two ways:

    1. I have two nodes called nodeA and nodeB. Both share a label expression called AB. I've set AB as the value of the -l paramenter. The behavior I see is that it runs on A and never runs on B at all.
    2. I although tried running this with two lines like this:
      java -jar $WORKSPACE/../../jenkins-cli.jar -s http://10.86.135.202:8080/ dist-fork -l nodeA -z job.zip -n TINE1 -d "-1" scripts/init.sh
      java -jar $WORKSPACE/../../jenkins-cli.jar -s http://10.86.135.202:8080/ dist-fork -l nodeB -z job.zip -n TINE2 -d "-1" scripts/init.sh

    I can watch and not exit the initiating job until the two don't exit. The problem is if I abort I end up with orphaned processes. I feel like it was intended to work the way I tried it in option 1 and maybe I'm doing something wrong. Feel free to redirect me if this is a more suited comment for another forum.