NOTE: all the commands in this exercise are run as the root user. If you are logged in as a regular user, use sudo -s to get a root shell.
$ sudo -s
# Before we create and install VMs, we need to change default settings in the cluster (this should have been done before, but we're doing it to be safe).
Do this on the MASTER node:
# gnt-cluster modify -H kvm:kernel_path=,initrd_path=,vnc_bind_address=0.0.0.0We are going to install our VMs using debootstrap - we don't feel like spending time installing an OS using an ISO image: it takes too long, and that's not what we would be doing in production if we had many customers or users asking for VMs all the time.
First, we need to adjust a few things.
There is an old version of the package "ganeti-instance-debootstrap" in Debian wheezy, but a newer one in wheezy-backports (version 0.14 at the time of writing). So check that you have the newer version:
# apt-get install ganeti-instance-debootstrap/wheezy-backportsYou will need to do this on all nodes in the cluster, because instances can be created on any node and you want them to be consistent.
Test that the "instance OS" type is recognized: on the master node,
# gnt-os listThis should show: debootstrap+default. If not, try gnt-os diagnose for more information.
Ganeti provides some sample "hooks" you can use to customize the VMs you create.
Normally you would do this once on the master node and then use gnt-cluster copyfile to copy the files to the other nodes, but for this exercise, you can each do it on your own host.
For example, let's say we want all our instances to have a root password of our choosing. To do this we need a small shell script, and a data file containing the desired password.
First, we need to copy the script into the 'hooks' directory:
# cd /etc/ganeti/instance-debootstrap/hooks
# cp /usr/share/doc/ganeti-instance-debootstrap/examples/defaultpasswords .
# chmod +x defaultpasswords(Notice the "." which means "into the current directory")
And now we need to give it a password file:
# mkdir confdata
# cd confdata
# cp /usr/share/doc/ganeti-instance-debootstrap/examples/confdata/defaultpasswords .
# chmod 600 defaultpasswordsEdit this file "defaultpasswords" and change the default password from s3cr3t to the password we use in class. This file should be just one line:
root:XXXXXXwhere XXXXXX is the password.
We're almost ready to build our VM. Before, we need to set a few parameters so it builds the version and architecture we want.
Have a look in the file /etc/default/ganeti-instance-debootstrap, which contains the default debootstrap settings for Ganeti.
You can see various options that we covered before in the scripted installs:
ARCH, SUITE and EXTRA_PKGS in particular, control which version of the OS and architecture, as well as which package, get installed.
We need to override (change) two of these parameters: ARCH and EXTRA_PKGS, but we're not going to modify this file.
Instead, we'll edit the file /etc/ganeti/instance-debootstrap/variants/default.conf, which allows us to make the change without touching the installed version. Like this, when Ganeti is upgraded, your settings file isn't overwritten.
Therefore:
# editor /etc/ganeti/instance-debootstrap/variants/default.confAnd add the following lines:
ARCH="amd64"
EXTRA_PKGS="acpi-support-base,console-tools,udev,linux-image-amd64,sudo,vim,grub-pc,openssh-server"
PROXY="http://apt.ws.nsrc.org:3142/"Summary:
As with the previous exercise: each pair of people can create a VM called
debianXand place it on your nodehostX, but you will need to be logged into the MASTER node of your cluster to issue these commands
# gnt-instance add -t plain -o debootstrap+default \
--disk 0:size=4G -B minmem=256M,maxmem=512M \
-n hostX.ws.nsrc.org \
--no-name-check --no-ip-check \
debianXNote: if you are only going to have one disk, instead of writing --disk 0:size=4G, you could simply write -s 4G.
The build is going to take a while the first time - it should fetch most packages from the local cache, which should speed things up.
It needs to fetch packages just like for debootstrap in the previous labs. Be patient...
Sat Jan 18 09:52:14 2014 * disk 0, size 4.0G
Sat Jan 18 09:52:14 2014 * creating instance disks...
Sat Jan 18 09:52:15 2014 adding instance debianX to cluster config
Sat Jan 18 09:52:15 2014 - INFO: Waiting for instance debianX to sync disks
Sat Jan 18 09:52:15 2014 - INFO: Instance debianX's disks are in sync
Sat Jan 18 09:52:15 2014 * running the instance OS create scripts...
Sat Jan 18 10:02:53 2014 * starting instance...Ok, the build is done! The good thing is, in the future, Ganeti has cached the image of your host (by default, for two weeks), so if you install more VMs with the same command, it will only take a few seconds to create and start the machine 1.
Unfortunately, there is currently a problem currently with this way of installing VMs on Ganeti.
Your VM will not come up, and KVM will hang with 99% cpu:
# top
...
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5981 root 20 0 765m 11m 3144 S 98.8 1.1 6:05.41 kvmIf you attach a VNC console you will see it stuck at "Booting from Hard Disk...". This is because the grub bootloader wasn't installed in the instance 2.
Let's fix it.
Bring down the host (immediate shutdown):
# gnt-instance shutdown --timeout=0 debianX
Waiting for job 211 for debianX ...We're going to boot the instance using the kernel from the Host, just temporarily, so we can fix the bug:
# gnt-instance start \
-H kernel_path=/vmlinuz,initrd_path=/initrd.img,kernel_args="ro single" \
debianXWait a few seconds, then start a serial console:
# gnt-instance console debianXPress ENTER
You should see:
Login incorrect.
Give root password for maintenance
(or type Control-D to continue):Here, enter the password we set for the root (class password). If all goes well, you are in:
root@debianX:~#Now we can fix the GRUB bootloader on the VM:
Note: you are typing this in the console, NOT ON YOUR HOST!
Update the GRUB config and install the bootloader in the boot sector of our VM disk:
# update-grub
Generating grub.cfg ...
cat: /boot/grub/video.lst: No such file or directory
Found linux image: /boot/vmlinuz-3.2.0-4-amd64
Found initrd image: /boot/initrd.img-3.2.0-4-amd64
done
# grub-install /dev/vda
Installation finished. No error reported.Exit the serial console by pressing ^] (control + right square bracket)
Back at the master node prompt, shutdown and restart the guest:
# gnt-instance reboot --shutdown-timeout=30 debianXWait until the prompt returns. If all goes well, you should now be able to get a console on your instance, either using gnt-instance console debianX or using VNC 3
Once you have a console on your VM, type the following:
# ifconfig eth0Does your VM have an IP address? If not, edit /etc/network/interfaces so that it looks like this:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcpYou can now bring up the interface:
# ifup eth0Once you have an IP address, test that network connectivity works, by pinging the class webserver:
# ping -c5 www.ws.nsrc.orgYou should also be able to use SSH to login to your guest from your laptop, with username root and the password you configured.
Congratulations! The remainder of this worksheet is extra information in case you finish early, or which you can review at your leisure later
Let's make sure we are using the in-class package cache:
Edit or create the file /etc/apt/apt.conf and add this line:
Acquire::http::Proxy "http://apt.ws.nsrc.org:3142/";Now we can install, for example, the apache web server 4
# apt-get install apache2Here's an example of how to automatically customize your OS installation. Let's say we want to install /etc/resolv.conf with particular contents.
Change to the hooks directory:
# cd /etc/ganeti/instance-debootstrap/hooksCreate a new file in this directory called resolv with the following contents:
#!/bin/bash
if [ ! -d "$TARGET/etc" ]; then
echo "Missing target etc directory"
exit 1
fi
cat <<EOS >"$TARGET/etc/resolv.conf"
nameserver 10.10.0.241
nameserver 10.10.0.242
search ws.nsrc.org
EOSMake it executable:
# chmod +x resolvNow the next time you install a debootstrap image, it should have this customization applied automatically.
Another use of hooks is configuring network interfaces automatically. You can create a script /etc/ganeti/instance-debootstrap/hooks/interfaces like this:
#!/bin/bash
if [ -z "$TARGET" -o ! -d "$TARGET" ]; then
echo "Missing target directory"
exit 1
fi
if [ ! -d "$TARGET/etc/network" ]; then
echo "Missing target network directory"
exit 1
fi
if [ -z "$NIC_COUNT" ]; then
echo "Missing NIC COUNT"
exit 1
fi
if [ -n "$NIC_0_IP" ]; then
cat > $TARGET/etc/network/interfaces <<EOF
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address $NIC_0_IP
netmask 255.255.255.0
gateway 10.10.255.254
dns-nameservers 10.10.0.241
bridge_ports eth0
bridge_stp off
bridge_fd 0
bridge_maxwait 0
EOF
elif [ "$NIC_COUNT" -gt 0 ]; then
cat > $TARGET/etc/network/interfaces <<EOF
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp
EOF
fiAgain it needs to be made executable. Now if you allocate a static IP address to your NIC at VM creation time it will be configured in the interfaces file; if you do not, then it will still be configured for DHCP.
More information on the environment variables passed to this script can be found in man ganeti-os-interface
You can also create other Debian-derived OSes this way. Here is how to install Ubuntu 12.04 ("precise")
On the master node, create /etc/ganeti/instance-debootstrap/variants/precise.conf
MIRROR="http://archive.ubuntu.com/ubuntu/"
PROXY="http://apt.ws.nsrc.org:3142/"
SUITE="precise"
EXTRA_PKGS="acpid,console-tools,udev,linux-image-virtual,openssh-server"
COMPONENTS="main,restricted,universe,multiverse"
ARCH="amd64"Update /etc/ganeti/instance-debootstrap/variants.list so it looks like this:
default
preciseDistribute these files to the other nodes:
# gnt-cluster copyfile /etc/ganeti/instance-debootstrap/variants/precise.conf
# gnt-cluster copyfile /etc/ganeti/instance-debootstrap/variants.list
# gnt-os listNow you can create an instance of Ubuntu Precise using the same gnt-instance add command as before but with -o debootstrap+precise as the operating system selection.
If you are curious: ls -l /var/cache/ganeti-instance-debootstrap/↩
There is a hook for doing this, but it's broken. There is a modified version you can download - it only works when creating plain (non-drbd) images, but you can convert them to drbd afterwards. This is likely to be fixed with the OS installation redesign↩
Reminder: gnt-instance list -o name,pnode,network_port then connect with VNC to the host & port listed ~↩
Note: for any package which you want present in every VM you build, you can add it to the EXTRA_PKGS section of the Ganeti debootstrap configuration. You can make different OS variants for different uses.↩