The Butler Did It

Having created the tools to script my Smalltalk builds, I was left with the question of how to make the builds happen. XD images can be built on any supported platform for any target environment, but, as far as I know, non-headless images have to be built in the environment in which they’re going to run. This meant that I had to package in both Windows and UNIX. I also wanted to be able to do this with a tool that would demonstrate to management that Smalltalk could coexist with and even be controlled by off-the-shelf tools.

Around that time, I attended a meeting of the Toronto Smalltalk User Group, and one of the members, Yanni Chiu, mentioned that they’d had some success in using the Hudson build integration server to automate Squeak builds. It turned out that some colleagues of mine at TAF were looking at Hudson to automate some Java builds, so it seemed a foregone conclusion that I adopt Hudson for my needs. Hudson had many nice features on his resume, including a web-based interface for the definition of jobs and monitoring of build status.

I started simply by installing Hudson on my local development machine and scheduling Windows builds at the start of the work day. This worked well enough that I then tackled the problem of doing UNIX builds. Our normal UNIX development environment is AIX, but we have no administrative rights on that machine. I wanted to be able to experiment with the installation of Hudson in the UNIX environment, so for research purposes, I installed VirtualBox, and created a Linux virtual machine in which to do the builds.

It became apparent that a Windows-based Hudson would have to control the process. While it was possible to have a Windows session log into UNIX and trigger a Smalltalk activity via SSH with the help of Xvfb, the opposite was not true; there was no good way I could find to control Windows applications from UNIX.

All of this was fine for a quick proof of concept, but wasn’t really suitable for long term. It was impossible to do anything on my machine while Hudson was building because Smalltalk windows kept popping up and disappearing, and it was taking the better part of the morning to run through the builds. TAF encourages us to turn off our machines at night for conservation purposes, and I also didn’t want to be the conservator of the “magic build box”.

Once I had the concepts well established, I had the Windows technical team create me a Windows virtual machine on their ESX server, and I installed Hudson there as a service. This was actually an interesting problem in itself. By default, Windows services run under the control of a user ID that has no network access. When performing build jobs, Hudson needed access to the Smalltalk repository to perform the Smalltalk build, to copy the completed images to a network drive and to send out problem notifications. He also needed to be able to SSH to the AIX boxes where UNIX builds would take place.

The solution was to create a Windows ID within the TAF domain that had access to the necessary network drive, and also had the right to run services on the Windows VM. The Hudson service was set up to run using that ID. Accessing UNIX provided its own challenges. To overcome that obstacle, I took advantage of PuTTY, and pre-shared SSH keys. This allowed me to execute shell scripts on UNIX from Windows without being bothered with password prompts.

Sadly, we had to let Hudson go earlier this year. There was considerable discord between Oracle and the original developer of Hudson. We decided to take on Jenkins, which is the project forked off of Hudson’s original code base. So far, Jenkins has worked out quite well, and we are constantly finding new duties for him.

Advertisements

4 Responses to “The Butler Did It”


  1. 1 marianopeck 2011-04-12 at 18:25

    Hi. You may also want to take a look to ci.pharo-project.org. We use Hudson to build not only the Pharo image but also all type of projects on top of Pharo. Tests are run, etc.

    In addition, in the last months, there are also builds for the VM. Each time a commit is done in the Git repository, a full set of Cog VMs are automatically build 🙂

    Cheers

  2. 2 Tom Koschate 2011-04-12 at 21:36

    At the height of our conversion effort, I had something like 94 images building every night, including packaged and development images. Having Jenkins run our testing as well is planned for the future, but there’s some technical roadblocks to overcome first. I’ll have more to say on that in future posts.

  3. 3 Thomas Holzer 2013-05-21 at 12:08

    Excellent stuff Tom!

    with your documentation and code form VASTGOODIES we have created an automated packaging of our app.

    How do you handle errors during loading and packaging? For Example “load failed” or any exception during packaging. Visual Age Smalltalk keeps running forever with an error dialog on the packaging server, for example “Load failed”. So the calling packaging application (here Hudson) does not get any return code.

    Greetings Thomas

    • 4 Tom Koschate 2013-05-21 at 12:16

      Glad you found it helpful. As for handling of packaging problems, that’s handled by Jenkins. I have a timeout set to fail the build if it takes too long (e.g. the load has failed), and I use the text search plugin to look for key phrases in the logs to pass or fail the build.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s





%d bloggers like this: