1 Unpacking of prepared system images

It is possible to use ganeti-instance-image to unpack an existing image instead of installing from CD-ROM. The time taken to set this up is quickly repaid if you are installing many similar instances.

The best way to understand this is to read the documentation and also read the create script which ganeti calls when creating a new instance.

Here is some information to help you understand the process.

A ganeti OS definition consists of a collection of scripts: create, rename, export and import.

For ganeti-instance-image you will find these under /usr/share/ganeti/os/image/. The create script can unpack three types of image:

tarballs and dump files are the most efficient as they don't contain any unused disk blocks, and they also allow the guest image to be resized to fit the requested instance disk space. However they only work with ext3/ext4 filesystems (i.e. Linux instances).

For tarballs and dump files, the instance creation script will partition the instance disk into two or three partitions:

Therefore you need to provide two tarballs or dumps, one to be expanded into /boot and one into /. In the case of "dump" image type, for an OS called "ubuntu-12.04" which is 64-bit, the files you would need to provide would be:


You would put these into the same directory, e.g. /var/cache/ganeti-instance-image/ubuntu-12.04

Then you can create a new image definition such as /etc/ganeti/instance-image/variants/ubuntu-12.04.conf containing:

IMAGE_TYPE="dump"   # this is the default
FILESYSTEM="ext4"   # default is ext3
SWAP="yes"          # this is the default
IMAGE_DEBUG="yes"   # while testing

# See below

The full set of options can be seen in /etc/default/ganeti-instance-image

Remember also to add "ubuntu-12.04" to /etc/ganeti/instance-image/variants.list and to replicate these files to all nodes in the cluster, e.g. using gnt-cluster copyfile ...

A new instance can then be installed using:

# gnt-instance add -o image+ubuntu-12.04 .... <instance-name>

which will create the image then call the create script with the correct parameters to unpack the prepared ubuntu-12.04 image.

NOTE: there is a helper script in /usr/share/ganeti/os/image/tools/make-dump which can help you turn an existing instance into dump files. However this only works if the OS has been installed with the correct partitioning and without LVM. So if you want to use this to make (say) a CentOS master image, you should install your gold CentOS image from ISO with manual partitioning:

1.1 Customization

After the image is unpacked, you may want to automate some customization such as installing grub or installing custom ssh keys or passwords. This can be done by setting CUSTOMIZE_DIR to point to a directory containing scripts to be run. See the example hooks for some scripts you can use as a starting point - they are also under /usr/share/doc/ganeti-instance-image/example/hooks

1.2 Reinstallation

Once you have automatic installation working, this allows you to reinstall an instance using gnt-instance reinstall which will completely wipe the OS and return it to the state of a fresh install.

2 Exporting and importing

The "gnt-backup" tool allows VM images to be exported and re-imported, for example for off-site backup or transferring to a different cluster.

The syntax is:

# gnt-backup export -n hostX.ws.nsrc.org <instance-name>

which shuts down the instance if it is running, then exports the disk(s) to a set of files under /var/lib/ganeti/export/<instance-name>/' on the node chosen with-n` (which need not be the instance's primary or secondary node)

The format of the export depends on the OS type definition (-o) from when the VM was created. For our cd.conf definition which has NOMOUNT=yes you will get a qcow2 format image (.img). This means it must be imported to an instance with exactly the same disk size.

When you import it's similar to creating a new instance, and you can use most of the flags you use when creating a new instance. This means you can use this as an alternative way of cloning machines. However by default ganeti will try to re-create the new machine with the same MAC address as the old one, which is rejected:

Failure: prerequisites not met for this operation:
error type: resource_not_unique, error details:
MAC address aa:00:00:41:1f:57 already in use in cluster

Therefore you have to tell it to assign a new MAC address when importing if the exported instance is still present.

# gnt-backup import \
  --src-node hostX.ws.nsrc.org --src-dir /var/lib/ganeti/export/centosX \
  -t plain -s 3G -n hostY.ws.nsrc.org --no-name-check --no-ip-check \
  --net 0:mac=generate centos2

If you import this way then a rename script is called (except if you have NOMOUNT=yes which forces qcow2 images), but any other changes are your responsibility - beware especially statically configured IP addresses.

2.1 Bugs

The export and import scripts create temporary files in the /tmp directory which may cause the host root filesystem to fill. There is a fix in ganeti-instance-image 0.6 which allows you to specify EXPORT_DIR for export, but not for import although you can easily fix this yourself.