Archive for the 'Continuous Integration' Category

Where Was I?

After many months, once more in a position to think about VA Smalltalk upgrades and the automation of packaging again. When 8.5 came out, it broke the tools I developed during the upgrade process from 7.5 to 8.0.2. Fortunately, subsequent releases have fixed things, so I’m at a point where I can complete the upgrade to 8.5.2. I wrote about many of these issues in an earlier blog entry.

As is my usual custom, I’ve started by modifying my config maps to specify either 8.0.2 or 8.5.2 maps as prerequisites. The development environments had few problems, but packaging was a challenge. Instantiations has added some interesting new time zone support to the base image, but it requires one to include an additional application, EsTimeZone, as part of the runtime package. If one is creating a new package from scratch, the default rules will apparently pick up this application automatically. The rest of us are not so lucky – we have to modify our packaging instructions to include this new package. This new requirement isn’t currently mentioned in the upgrade notes.

In my case, because I’m building 8.0.2 and 8.5.2 images from the same code base, I’ve been forced to move the packaging rules class to a subapplication. A configuration expression based on the system version loads the appropriate subapplication. This is where I found one more little “gotcha”.

In 8.0.2 and earlier versions, the preferred mechanism for determining the Smalltalk version was to send #vaVersion to System. This has been deprecated in favour of #imageVersion, which answers the same thing. If you’re like me, and trying to maintain a couple of versions of Smalltalk, you’ll probably find it easier to continue to use #vaVersion for now, and switch to #imageVersion once you’ve retired anything prior to 8.5.

Other than these few minor annoyances, it looks like there haven’t been any significant changes introduced, unless you were an early adopter of the log4s framework. If you were, you’ll want to look things over a little more closely.


The Butler Did It

Having created the tools to script my Smalltalk builds, I was left with the question of how to make the builds happen. XD images can be built on any supported platform for any target environment, but, as far as I know, non-headless images have to be built in the environment in which they’re going to run. This meant that I had to package in both Windows and UNIX. I also wanted to be able to do this with a tool that would demonstrate to management that Smalltalk could coexist with and even be controlled by off-the-shelf tools.

Around that time, I attended a meeting of the Toronto Smalltalk User Group, and one of the members, Yanni Chiu, mentioned that they’d had some success in using the Hudson build integration server to automate Squeak builds. It turned out that some colleagues of mine at TAF were looking at Hudson to automate some Java builds, so it seemed a foregone conclusion that I adopt Hudson for my needs. Hudson had many nice features on his resume, including a web-based interface for the definition of jobs and monitoring of build status.

I started simply by installing Hudson on my local development machine and scheduling Windows builds at the start of the work day. This worked well enough that I then tackled the problem of doing UNIX builds. Our normal UNIX development environment is AIX, but we have no administrative rights on that machine. I wanted to be able to experiment with the installation of Hudson in the UNIX environment, so for research purposes, I installed VirtualBox, and created a Linux virtual machine in which to do the builds.

It became apparent that a Windows-based Hudson would have to control the process. While it was possible to have a Windows session log into UNIX and trigger a Smalltalk activity via SSH with the help of Xvfb, the opposite was not true; there was no good way I could find to control Windows applications from UNIX.

All of this was fine for a quick proof of concept, but wasn’t really suitable for long term. It was impossible to do anything on my machine while Hudson was building because Smalltalk windows kept popping up and disappearing, and it was taking the better part of the morning to run through the builds. TAF encourages us to turn off our machines at night for conservation purposes, and I also didn’t want to be the conservator of the “magic build box”.

Once I had the concepts well established, I had the Windows technical team create me a Windows virtual machine on their ESX server, and I installed Hudson there as a service. This was actually an interesting problem in itself. By default, Windows services run under the control of a user ID that has no network access. When performing build jobs, Hudson needed access to the Smalltalk repository to perform the Smalltalk build, to copy the completed images to a network drive and to send out problem notifications. He also needed to be able to SSH to the AIX boxes where UNIX builds would take place.

The solution was to create a Windows ID within the TAF domain that had access to the necessary network drive, and also had the right to run services on the Windows VM. The Hudson service was set up to run using that ID. Accessing UNIX provided its own challenges. To overcome that obstacle, I took advantage of PuTTY, and pre-shared SSH keys. This allowed me to execute shell scripts on UNIX from Windows without being bothered with password prompts.

Sadly, we had to let Hudson go earlier this year. There was considerable discord between Oracle and the original developer of Hudson. We decided to take on Jenkins, which is the project forked off of Hudson’s original code base. So far, Jenkins has worked out quite well, and we are constantly finding new duties for him.