1 Objectives

NOTE: all the commands in this exercise are run as the root user. If you are logged in as a regular user, use sudo -s to get a root shell.

$ sudo -s

2 Installing instances (creating VMs)

Before we create and install VMs, we need to change default settings in the cluster (this should have been done before, but we're doing it to be safe).

Do this on the MASTER node:

# gnt-cluster modify -H kvm:kernel_path=,initrd_path=,vnc_bind_address=

2.1 Install an instance (VM) with debootstrap

We are going to install our VMs using debootstrap - we don't feel like spending time installing an OS using an ISO image: it takes too long, and that's not what we would be doing in production if we had many customers or users asking for VMs all the time.

First, we need to adjust a few things.

2.1.1 Replace the debootstrap installer with a newer version

There is an old version of the package "ganeti-instance-debootstrap" in Debian wheezy, but a newer one in wheezy-backports (version 0.14 at the time of writing). So check that you have the newer version:

# apt-get install ganeti-instance-debootstrap/wheezy-backports

You will need to do this on all nodes in the cluster, because instances can be created on any node and you want them to be consistent.

Test that the "instance OS" type is recognized: on the master node,

# gnt-os list

This should show: debootstrap+default. If not, try gnt-os diagnose for more information.

2.1.2 Configure parameters for our VM

We're almost ready to build our VM. Before, we need to set a few parameters so it builds the version and architecture we want.

Have a look in the file /etc/default/ganeti-instance-debootstrap, which contains the default debootstrap settings for Ganeti.

You can see various options that we covered before in the scripted installs:

ARCH, SUITE and EXTRA_PKGS in particular, control which version of the OS and architecture, as well as which package, get installed.

We need to override (change) two of these parameters: ARCH and EXTRA_PKGS, but we're not going to modify this file.

Instead, we'll edit the file /etc/ganeti/instance-debootstrap/variants/default.conf, which allows us to make the change without touching the installed version. Like this, when Ganeti is upgraded, your settings file isn't overwritten.


# editor /etc/ganeti/instance-debootstrap/variants/default.conf

And add the following lines:



2.1.3 Create a Debian instance

As with the previous exercise: each pair of people can create a VM called debianX and place it on your node hostX, but you will need to be logged into the MASTER node of your cluster to issue these commands

# gnt-instance add -t plain -o debootstrap+default \
    -s 4G -B minmem=256M,maxmem=512M -n hostX.ws.nsrc.org \
    --no-name-check --no-ip-check debianX

The build is going to take a while the first time - it should fetch most packages from the local cache, which should speed things up.

It needs to fetch packages just like for debootstrap in the previous labs. Be patient...

Sat Jan 18 09:52:14 2014 * disk 0, size 4.0G
Sat Jan 18 09:52:14 2014 * creating instance disks...
Sat Jan 18 09:52:15 2014 adding instance debianX to cluster config
Sat Jan 18 09:52:15 2014  - INFO: Waiting for instance debianX to sync disks
Sat Jan 18 09:52:15 2014  - INFO: Instance debianX's disks are in sync
Sat Jan 18 09:52:15 2014 * running the instance OS create scripts...
Sat Jan 18 10:02:53 2014 * starting instance...

Ok, the build is done! The good thing is, in the future, Ganeti has cached the image of your host (by default, for two weeks), so if you install more VMs with the same command, it will only take a few seconds to create and start the machine 1.

By default, the root password for your VM is the empty string.

Unfortunately, there is currently a problem currently with this way of installing VMs on Ganeti.

Your VM will not come up, and KVM will hang with 99% cpu:

# top

 5981 root      20   0  765m  11m 3144 S  98.8  1.1   6:05.41 kvm

If you attach a VNC console you will see it stuck at "Booting from Hard Disk...". This is because the grub bootloader wasn't installed in the instance 2.

Let's fix it.

Bring down the host (immediate shutdown):

# gnt-instance shutdown --timeout=0 debianX
Waiting for job 211 for debianX ...

We're going to boot the instance using the kernel from the Host, just temporarily, so we can fix the bug:

# gnt-instance start \
    -H kernel_path=/vmlinuz,initrd_path=/initrd.img,kernel_args="ro single" \

Wait a few seconds, then start a serial console:

# gnt-instance console debianX


You should see:

Login incorrect.
Give root password for maintenance
(or type Control-D to continue):

Here, hit Enter. If all goes well, you are in:


Now we can fix the GRUB bootloader on the VM:

Note: you are typing this in the console, NOT ON YOUR HOST!

Update the GRUB config and install the bootloader in the boot sector of our VM disk:

# update-grub

Generating grub.cfg ...
cat: /boot/grub/video.lst: No such file or directory
Found linux image: /boot/vmlinuz-3.2.0-4-amd64
Found initrd image: /boot/initrd.img-3.2.0-4-amd64

# grub-install /dev/vda

Installation finished. No error reported.

Exit the serial console by pressing ^] (control + right square bracket)

Back at the master node prompt, shutdown and restart the guest:

# gnt-instance reboot --shutdown-timeout=30 debianX

Wait until the prompt returns. If all goes well, you should now be able to get a console on your instance, either using gnt-instance console debianX or using VNC 3

3 Network configuration

Once you have a console on your VM, type the following:

# ifconfig eth0

Does your VM have an IP address? If not, edit /etc/network/interfaces so that it looks like this:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

You can now bring up the interface:

# ifup eth0

Once you have an IP address, test that network connectivity works, by pinging the class webserver:

# ping -c5 www.ws.nsrc.org

You should also be able to use SSH to login to your guest from your laptop, with username root and the password you configured.

Congratulations! The remainder of this worksheet is extra information in case you finish early, or which you can review at your leisure later

4 Extra exercises

4.1 Install packages

Let's make sure we are using the in-class package cache:

Edit or create the file /etc/apt/apt.conf and add this line:

Acquire::http::Proxy "http://apt.ws.nsrc.org:3142/";

Now we can install, for example, the apache web server 4

# apt-get install apache2

4.2 Set a default root password for new instances

Ganeti provides some sample "hooks" you can use to customize the VMs you create.

Normally you would do this once on the master node and then use gnt-cluster copyfile to copy the files to the other nodes, but for this exercise, you can each do it on your own host.

Now, having instances with an empty root password is very insecure, so let's say we want to automatically set a root password of our choosing. To do this we need a small shell script, and a data file containing the desired password.

First, we need to copy the script into the 'hooks' directory:

# cd /etc/ganeti/instance-debootstrap/hooks
# cp /usr/share/doc/ganeti-instance-debootstrap/examples/defaultpasswords .
# chmod +x defaultpasswords

(Notice the "." which means "into the current directory")

And now we need to give it a password file:

# cd /etc/ganeti/instance-debootstrap/hooks
# mkdir confdata
# cd confdata
# cp /usr/share/doc/ganeti-instance-debootstrap/examples/confdata/defaultpasswords .
# chmod 600 defaultpasswords

Edit this file "defaultpasswords" and change the default password from s3cr3t to the password we use in class. This file should be just one line:


where XXXXXX is the password.

Now new instances will be created with the root password you have entered.

4.3 Additional hooks

Here's another example of how to automatically customize your OS installation. Let's say we want to install /etc/resolv.conf with particular contents.

Change to the hooks directory:

# cd /etc/ganeti/instance-debootstrap/hooks

Create a new file in this directory called resolv with the following contents:

if [ ! -d "$TARGET/etc" ]; then
  echo "Missing target etc directory"
  exit 1

cat <<EOS >"$TARGET/etc/resolv.conf"
search ws.nsrc.org

Make it executable:

# chmod +x resolv

Now the next time you install a debootstrap image, it should have this customization applied automatically.

Another use of hooks is configuring network interfaces automatically. You can create a script /etc/ganeti/instance-debootstrap/hooks/interfaces like this:


if [ -z "$TARGET" -o ! -d "$TARGET" ]; then
  echo "Missing target directory"
  exit 1

if [ ! -d "$TARGET/etc/network" ]; then
  echo "Missing target network directory"
  exit 1

if [ -z "$NIC_COUNT" ]; then
  echo "Missing NIC COUNT"
  exit 1

if [ -n "$NIC_0_IP" ]; then

  cat > $TARGET/etc/network/interfaces <<EOF
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address         $NIC_0_IP
        bridge_ports    eth0
        bridge_stp      off
        bridge_fd       0
        bridge_maxwait  0

elif [ "$NIC_COUNT" -gt 0 ]; then

  cat > $TARGET/etc/network/interfaces <<EOF
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp


Again it needs to be made executable. Now if you allocate a static IP address to your NIC at VM creation time it will be configured in the interfaces file; if you do not, then it will still be configured for DHCP.

More information on the environment variables passed to this script can be found in man ganeti-os-interface

4.4 Installing Ubuntu

You can also create other Debian-derived OSes this way. Here is how to install Ubuntu 12.04 ("precise")

On the master node, create /etc/ganeti/instance-debootstrap/variants/precise.conf


Update /etc/ganeti/instance-debootstrap/variants.list so it looks like this:


Distribute these files to the other nodes:

# gnt-cluster copyfile /etc/ganeti/instance-debootstrap/variants/precise.conf
# gnt-cluster copyfile /etc/ganeti/instance-debootstrap/variants.list
# gnt-os list

Now you can create an instance of Ubuntu Precise using the same gnt-instance add command as before but with -o debootstrap+precise as the operating system selection.

  1. If you are curious: ls -l /var/cache/ganeti-instance-debootstrap/

  2. There is a hook for doing this, but it's broken. There is a modified version you can download - it only works when creating plain (non-drbd) images, but you can convert them to drbd afterwards. This is likely to be fixed with the OS installation redesign

  3. Reminder: gnt-instance list -o name,pnode,network_port then connect with VNC to the host & port listed ~

  4. Note: for any package which you want present in every VM you build, you can add it to the EXTRA_PKGS section of the Ganeti debootstrap configuration. You can make different OS variants for different uses.