Image by MrsWoman via Flickr

This week I've been working on creating a virtual appliance (VA) version of the Kynetx engine. This is a necessary step for customers who need a version of the Kynetx Rules Engine (KRE) running behind their firewall (think SAS70 compliance, for example).

This post documents some of what I discovered. (N.B. Since I'm working with Xen, much of what appears below could be Xen specific; I haven't taken the time to generalize it.)

First, and most important, there's no standard definition of what a virtual appliance is or how they are built. There are companies like rPath and others who have developed processes, but there are literally dozens of ways you might do it. They can be categorized into three camps:

• You can, in the spirit of an appliance, hand over a disk image (or images) and an XML machine descriptor which can be used, over and over again if properly set up, to reconstitute a machine (using virt-image for example).
• You can create a VA factory that when fired off creates a new machine from scratch, including the loading of the OS, configuration, and so on.
• You can do a hybrid of these two that includes a mostly configured image that then bootstraps itself to a final configuration.

Let's make a few assumptions:

1. You want to install multiple copies of the VA at the customer site.
2. The installation needs to be "appliance-like" implying that this isn't just a custom software installtion but is relatively independent of customer environment and is configurable.

Given these requirements and some choices we're already made--such as using a rather large puppet recipe collection to configure machines--I opted for for the VA factory method of deploying our appliances.

To make that all work, there are a number of pieces you need: DHCP, Kickstart files (if you are using RedHat, Centos, or Fedora), repositories, puppet recipes, and bootstrap scripts. Setting all these up individually can be done, but there's a better way: Cobbler.

Cobbler is a Linux installation server that takes care of putting all these together for you into a nice package that you configure and then it takes care of the details of making things work. It took me most of a day to play with it, get it set up how I wanted, and then see success in creating a usable image.

You could put Cobbler on Dom0 (the host machine in Xen), but I chose to put Cobbler and it's associated repositories on a virtual machine, which functions like a VA factory. That primordial system can be installed on a Xen host machine and then used to create as many copies and configurations of the VA as we need. The factory contains the Linux distro, the repositories we need, the kickstart file, and other configuration data. I haven't decided whether to put the puppet recipes there or not, but I'm leaning that way.

For the record, here's what I did in Cobbler. After installing and making sure the installation was good (cobbler check), I imported a distro, created a repository, created a profile for the particular kind of system I wanted to build, and added system definitions for the specific VA's I was going to create. (N.B. the double \\\\ indicates that the command ought to be on one line, but I've warpped it for presentation here.

cobbler import --mirror ~/fc8 --name fc8
cobbler repo add --mirror=http://archive.fedoraproject.org/...\\\\ 
  --name=fc8-newkey-repo --keep-updated=Y \\\\
  --rpm-list="puppet facter ruby-libs..."
cobbler profile add --name=base-fc8 --distro=fc8-xen-i386 \\\\
  --kickstart=/root/base-fc8.ks  --repos=fc8-newkey-repo
cobbler system add --name=kib0 --mac=00:16:3E:4B:4B:40 \\\\
  --ip=192.168.122.160 --profile=base-fc8 --hostname=kib0
cobbler system add --name=kib1 --mac=00:16:3E:4B:4B:41 \\\\
  --ip=192.168.122.161 --profile=base-fc8 --hostname=kib1
cobbler system add --name=kib2 --mac=00:16:3E:4B:4B:42 \\\\
  --ip=192.168.122.162 --profile=base-fc8 --hostname=kib2
cobbler system add --name=kib3 --mac=00:16:3E:4B:4B:43 \\\\
 --ip=192.168.122.163 --profile=base-fc8 --hostname=kib3

I customized things the most by editing the default Kickstart file that comes with Cobbler to customize the build to what I needed. After that was all set up, installing a virtual machine was as simple as typing the following command on Dom0:

koan --server=cobbler.kobj.net --virt --nogfx \\\\
   --system=kib0 --virt-bridge=virbr0

Koan stands for "kickstart over a network" and is the command that uses cobbler to install systems from a Cobbler server. I've still got some small kinks to work out, but I'm well on my way to having this all completely automated.

All this automation may seem like overkill, but it's essential in a small shop. Without taking the time to create these systems, a few customers would quickly eat all the time we have and then we've got to hire more people. By creating automated system, we're largely free to move onto creating new features and servicing new customers because existing customer maintenance has been automated to the greatest degree we can. Some might call that survivial--I just call it getting to sleep through the night.