1 Objectives

NOTE: all the commands in this exercise are run as the root user. If you are logged in as a regular user, use sudo -s to get a root shell.

$ sudo -s
# 

2 Installing instances (creating VMs)

Before we create and install VMs, we need to change default settings in the cluster (this should have been done before, but we're doing it to be safe).

Do this on the MASTER node:

# gnt-cluster modify -H kvm:kernel_path=,initrd_path=,vnc_bind_address=0.0.0.0

2.1 Install an instance (VM) with debootstrap

We are going to install our VMs using debootstrap - we don't feel like spending time installing an OS using an ISO image: it takes too long, and that's not what we would be doing in production if we had many customers or users asking for VMs all the time.

First, we need to adjust a few things.

2.1.1 Replace the debootstrap installer with a newer version

There is an old version of the package "ganeti-instance-debootstrap" in Debian wheezy, but a newer one in wheezy-backports (version 0.14 at the time of writing). So check that you have the newer version:

# apt-get install ganeti-instance-debootstrap/wheezy-backports

You will need to do this on all nodes in the cluster, because instances can be created on any node and you want them to be consistent.

Test that the "instance OS" type is recognized: on the master node,

# gnt-os list

This should show: debootstrap+default. If not, try gnt-os diagnose for more information.

2.1.2 Set a default root password for all instances

Ganeti provides some sample "hooks" you can use to customize the VMs you create.

Normally you would do this once on the master node and then use gnt-cluster copyfile to copy the files to the other nodes, but for this exercise, you can each do it on your own host.

For example, let's say we want all our instances to have a root password of our choosing. To do this we need a small shell script, and a data file containing the desired password.

First, we need to copy the script into the 'hooks' directory:

# cd /etc/ganeti/instance-debootstrap/hooks
# cp /usr/share/doc/ganeti-instance-debootstrap/examples/defaultpasswords .
# chmod +x defaultpasswords

(Notice the "." which means "into the current directory")

And now we need to give it a password file:

# mkdir confdata
# cd confdata
# cp /usr/share/doc/ganeti-instance-debootstrap/examples/confdata/defaultpasswords .
# chmod 600 defaultpasswords

Edit this file "defaultpasswords" and change the default password from s3cr3t to the password we use in class. This file should be just one line:

root:XXXXXX

where XXXXXX is the password.

2.1.3 Configure parameters for our VM

We're almost ready to build our VM. Before, we need to set a few parameters so it builds the version and architecture we want.

Have a look in the file /etc/default/ganeti-instance-debootstrap, which contains the default debootstrap settings for Ganeti.

You can see various options that we covered before in the scripted installs:

ARCH, SUITE and EXTRA_PKGS in particular, control which version of the OS and architecture, as well as which package, get installed.

We need to override (change) two of these parameters: ARCH and EXTRA_PKGS, but we're not going to modify this file.

Instead, we'll edit the file /etc/ganeti/instance-debootstrap/variants/default.conf, which allows us to make the change without touching the installed version. Like this, when Ganeti is upgraded, your settings file isn't overwritten.

Therefore:

# editor /etc/ganeti/instance-debootstrap/variants/default.conf

And add the following lines:

ARCH="amd64"
EXTRA_PKGS="acpi-support-base,console-tools,udev,linux-image-amd64,sudo,vim,grub-pc,openssh-server"
PROXY="http://apt.ws.nsrc.org:3142/"

Summary:

2.1.4 Create a Debian instance

As with the previous exercise: each pair of people can create a VM called debianX and place it on your node hostX, but you will need to be logged into the MASTER node of your cluster to issue these commands

# gnt-instance add -t plain -o debootstrap+default \
    --disk 0:size=4G -B minmem=256M,maxmem=512M \
    -n hostX.ws.nsrc.org \
    --no-name-check --no-ip-check \
    debianX

Note: if you are only going to have one disk, instead of writing --disk 0:size=4G, you could simply write -s 4G.

The build is going to take a while the first time - it should fetch most packages from the local cache, which should speed things up.

It needs to fetch packages just like for debootstrap in the previous labs. Be patient...

Sat Jan 18 09:52:14 2014 * disk 0, size 4.0G
Sat Jan 18 09:52:14 2014 * creating instance disks...
Sat Jan 18 09:52:15 2014 adding instance debianX to cluster config
Sat Jan 18 09:52:15 2014  - INFO: Waiting for instance debianX to sync disks
Sat Jan 18 09:52:15 2014  - INFO: Instance debianX's disks are in sync
Sat Jan 18 09:52:15 2014 * running the instance OS create scripts...
Sat Jan 18 10:02:53 2014 * starting instance...

Ok, the build is done! The good thing is, in the future, Ganeti has cached the image of your host (by default, for two weeks), so if you install more VMs with the same command, it will only take a few seconds to create and start the machine 1.

Unfortunately, there is currently a problem currently with this way of installing VMs on Ganeti.

Your VM will not come up, and KVM will hang with 99% cpu:

# top

...
  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
 5981 root      20   0  765m  11m 3144 S  98.8  1.1   6:05.41 kvm

If you attach a VNC console you will see it stuck at "Booting from Hard Disk...". This is because the grub bootloader wasn't installed in the instance 2.

Let's fix it.

Bring down the host (immediate shutdown):

# gnt-instance shutdown --timeout=0 debianX
Waiting for job 211 for debianX ...

We're going to boot the instance using the kernel from the Host, just temporarily, so we can fix the bug:

# gnt-instance start \
    -H kernel_path=/vmlinuz,initrd_path=/initrd.img,kernel_args="ro single" \
    debianX

Wait a few seconds, then start a serial console:

# gnt-instance console debianX

Press ENTER

You should see:

Login incorrect.
Give root password for maintenance
(or type Control-D to continue):

Here, enter the password we set for the root (class password). If all goes well, you are in:

root@debianX:~#

Now we can fix the GRUB bootloader on the VM:

Note: you are typing this in the console, NOT ON YOUR HOST!

Update the GRUB config and install the bootloader in the boot sector of our VM disk:

# update-grub

Generating grub.cfg ...
cat: /boot/grub/video.lst: No such file or directory
Found linux image: /boot/vmlinuz-3.2.0-4-amd64
Found initrd image: /boot/initrd.img-3.2.0-4-amd64
done

# grub-install /dev/vda

Installation finished. No error reported.

Exit the serial console by pressing ^] (control + right square bracket)

Back at the master node prompt, shutdown and restart the guest:

# gnt-instance reboot --shutdown-timeout=30 debianX

Wait until the prompt returns. If all goes well, you should now be able to get a console on your instance, either using gnt-instance console debianX or using VNC 3

3 Network configuration

Once you have a console on your VM, type the following:

# ifconfig eth0

Does your VM have an IP address? If not, edit /etc/network/interfaces so that it looks like this:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

You can now bring up the interface:

# ifup eth0

Once you have an IP address, test that network connectivity works, by pinging the class webserver:

# ping -c5 www.ws.nsrc.org

You should also be able to use SSH to login to your guest from your laptop, with username root and the password you configured.

Congratulations! The remainder of this worksheet is extra information in case you finish early, or which you can review at your leisure later

4 Extra exercises

4.1 Install packages

Let's make sure we are using the in-class package cache:

Edit or create the file /etc/apt/apt.conf and add this line:

Acquire::http::Proxy "http://apt.ws.nsrc.org:3142/";

Now we can install, for example, the apache web server 4

# apt-get install apache2

4.2 Additional hooks

Here's an example of how to automatically customize your OS installation. Let's say we want to install /etc/resolv.conf with particular contents.

Change to the hooks directory:

# cd /etc/ganeti/instance-debootstrap/hooks

Create a new file in this directory called resolv with the following contents:

#!/bin/bash
if [ ! -d "$TARGET/etc" ]; then
  echo "Missing target etc directory"
  exit 1
fi

cat <<EOS >"$TARGET/etc/resolv.conf"
nameserver 10.10.0.241
nameserver 10.10.0.242
search ws.nsrc.org
EOS

Make it executable:

# chmod +x resolv

Now the next time you install a debootstrap image, it should have this customization applied automatically.

Another use of hooks is configuring network interfaces automatically. You can create a script /etc/ganeti/instance-debootstrap/hooks/interfaces like this:

#!/bin/bash

if [ -z "$TARGET" -o ! -d "$TARGET" ]; then
  echo "Missing target directory"
  exit 1
fi

if [ ! -d "$TARGET/etc/network" ]; then
  echo "Missing target network directory"
  exit 1
fi

if [ -z "$NIC_COUNT" ]; then
  echo "Missing NIC COUNT"
  exit 1
fi

if [ -n "$NIC_0_IP" ]; then

  cat > $TARGET/etc/network/interfaces <<EOF
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address         $NIC_0_IP
        netmask         255.255.255.0
        gateway         10.10.255.254
        dns-nameservers 10.10.0.241
        bridge_ports    eth0
        bridge_stp      off
        bridge_fd       0
        bridge_maxwait  0
EOF

elif [ "$NIC_COUNT" -gt 0 ]; then

  cat > $TARGET/etc/network/interfaces <<EOF
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
EOF

fi

Again it needs to be made executable. Now if you allocate a static IP address to your NIC at VM creation time it will be configured in the interfaces file; if you do not, then it will still be configured for DHCP.

More information on the environment variables passed to this script can be found in man ganeti-os-interface

4.3 Installing Ubuntu

You can also create other Debian-derived OSes this way. Here is how to install Ubuntu 12.04 ("precise")

On the master node, create /etc/ganeti/instance-debootstrap/variants/precise.conf

MIRROR="http://archive.ubuntu.com/ubuntu/"
PROXY="http://apt.ws.nsrc.org:3142/"
SUITE="precise"
EXTRA_PKGS="acpid,console-tools,udev,linux-image-virtual,openssh-server"
COMPONENTS="main,restricted,universe,multiverse"
ARCH="amd64"

Update /etc/ganeti/instance-debootstrap/variants.list so it looks like this:

default
precise

Distribute these files to the other nodes:

# gnt-cluster copyfile /etc/ganeti/instance-debootstrap/variants/precise.conf
# gnt-cluster copyfile /etc/ganeti/instance-debootstrap/variants.list
# gnt-os list

Now you can create an instance of Ubuntu Precise using the same gnt-instance add command as before but with -o debootstrap+precise as the operating system selection.


  1. If you are curious: ls -l /var/cache/ganeti-instance-debootstrap/

  2. There is a hook for doing this, but it's broken. There is a modified version you can download - it only works when creating plain (non-drbd) images, but you can convert them to drbd afterwards. This is likely to be fixed with the OS installation redesign

  3. Reminder: gnt-instance list -o name,pnode,network_port then connect with VNC to the host & port listed ~

  4. Note: for any package which you want present in every VM you build, you can add it to the EXTRA_PKGS section of the Ganeti debootstrap configuration. You can make different OS variants for different uses.