**Ganeti networking**

# Objectives

In this exercise, you will:

* Configure a separate replication network for DRBD traffic
* Configure a separate service network for VMs to communicate to the
  outside world

whilst keeping the existing 10.10.10.X network for management traffic
(i.e. administering the ganeti nodes)

By the end, your host will have three interfaces:

**************************************************************
*                   management 10.10.10.X                    *
*                -----------+--------------                  *
*                           |                                *
*                         br-lan                             *
*                           |                                *
*                 +---------+---------+                      *
*                 |        eth0       | node X               *
*                 |                   |                      *
*                 | eth2        eth1  |                      *
*                 +--+-----------+----+                      *
*                    |           |                           *
*                  br-svc        |                           *
*                    |           |                           *
*        VMs --------+           +------> to other nodes     *
*                                         10.10.100.X        *
**************************************************************

Ideally these would be three physical interfaces (e.g.  eth0, eth1 and
eth2), but they could also be VLANs tagged out of the same physical
interface (e.g.  eth0.100 for replication and and eth0.255 for service
traffic).  Your instructor will tell you which setup you are using.

# Become root

All of the actions in this exercise are done as "root", so if you are not
root already, type:

~~~
$ sudo -s
#
~~~

# VLAN tools

Ensure you have the vlan management tools installed:

~~~~~~~~~~~~~
# apt-get install vlan
~~~~~~~~~~~~~

# Replication network

Let's start with the Replication network. To do this, add the following
to the end of `/etc/network/interfaces`, below the `br-lan` section.

~~~~~~~~~~~~~~~~~~~
# Replication network
auto eth1
iface eth1 inet static
        address 10.10.100.X
        netmask 255.255.255.0
~~~~~~~~~~~~~~~~~~~

Remember to replace X with the number of your node.

> In a VLAN environment, replace `eth1` with `eth0.100`. This naming convention
> using dot is quite common, and it means that packets on this sub-interface
> enter and leave physical port eth0 with VLAN tag 100 added.

Now activate the interface:

~~~
# ifup eth1
~~~

> In a VLAN environment: `ifup eth0.100`
> 

Check that you can ping all the other nodes in your cluster on their
10.10.100.X addresses.

Note that we have not created a bridge.  This network will just be used for
the nodes to send DRBD traffic to each other.  The individual VMs do not
need to have access to this network.

## Configuring Ganeti: MASTER NODE ONLY

Now we have to change the cluster configuration so that Ganeti uses the
secondary network for DRBD traffic.

Like all configuration changes, this is done on the MASTER node. Only one
person should do this section.

Don't do this until all the nodes in your cluster can ping each other
on their 10.10.100.X networks.

First, shutdown all running DRBD instances.

~~~
# gnt-instance list -o name,pnode,snodes,status
~~~

For any instances which have a secondary node and have status
'running', issue the command to shut them down:

~~~
# gnt-instance shutdown <NAME>
~~~

Now add a secondary IP address (-s) to the master node itself:

~~~
# gnt-node modify -s 10.10.100.X --force nodeX.virt.nsrc.org
~~~

and repeat for the other nodes in the cluster:

~~~
# gnt-node modify -s 10.10.100.Y --force nodeY.virt.nsrc.org
~~~

Check that all the nodes in the cluster have a secondary IP:

~~~
# gnt-node list -o name,sip
~~~

Once this has been done, you can restart your DRBD instances.

# Service network

Now add the Service network to every node in your cluster. Same as before,
go to the end of `/etc/network/interfaces`, and add the following lines:

~~~~~~~~~~~~~~
# Service network
auto eth2
iface eth2 inet manual

auto br-svc
iface br-svc inet manual
        bridge_ports    eth2
        bridge_stp      off
        bridge_fd       0
        bridge_maxwait  0
~~~~~~~~~~~~~~

> In a VLAN environment, replace `eth2` with `eth0.255`
> 

This time we have made a bridge so that the VMs can attach to it, but notice
that we have NOT configured an IP address for the node on br-svc.  The VMs
will send their ethernet frames through the bridge, but there is no need for
the physical host itself to have an address on this network.  Ensuring the
host OS is not visible to the guests improves security.

Activate the interface:

~~~
# ifup br-svc
~~~

Since the node doesn't have an address on this network, we can't test using
ping.  However you can check that the bridge exists and has the right
interface in it:

~~~
# brctl show
bridge name     bridge id               STP enabled     interfaces
br-svc          8000.d4ae52c12e7e       no              eth2
~~~

## Attaching VMs to the service network

Each of you can take one of your VMs and move its network interface to
the service network, by logging into the MASTER node and using the
`gnt-instance modify` command:

~~~
# gnt-instance modify --net 0:modify,link=br-svc wordpressX
~~~

You'll need to reboot the instance after you made the change:

~~~
# gnt-instance reboot wordpressX
~~~

When it restarts, the virtual eth0 interface on the instance should be
connected to br-svc instead of br-lan.

Go into the VM's VNC console to find out what IP address it has picked up on
the new network.  You should find that it now has a 10.10.255.X address.

Repeat for other VMs.

You can use check the status using `gnt-instance list`:

~~~
# gnt-instance list -o name,nic.bridge/0
~~~

## Changing the default interface: MASTER NODE ONLY

You can now configure the cluster so that all subsequent VMs you create
will be connected to br-svc instead of br-lan.

~~~
# gnt-cluster modify -N link=br-svc
~~~

# Review

After these changes, your `/etc/network/interfaces` should look something
like this (IPs should be the ones for your PC, of course):

~~~~~~~~~~~~~~~~~~~~~
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet manual

auto br-lan
iface br-lan inet static
        address 10.10.10.X
        netmask 255.255.255.0
        gateway 10.10.10.254
        dns-nameservers 10.10.10.241
        bridge_ports    eth0
        bridge_stp      off
        bridge_fd       0
        bridge_maxwait  0

# Replication network
auto eth1
iface eth1 inet static
        address 10.10.100.X
        netmask 255.255.255.0

# Service network
auto eth2
iface eth2 inet manual

auto br-svc
iface br-svc inet manual
        bridge_ports    eth2
        bridge_stp      off
        bridge_fd       0
        bridge_maxwait  0
~~~~~~~~~~~~~~~~~~~~~

(Note: on a production machine it's a good idea to reboot after making major
changes to `/etc/network/interfaces` to check they are picked up correctly
at system startup)

# Optional extra exercise

Create an instance which has two virtual NICs: eth0 connected to br-lan and
eth1 connected to br-svc.  Use the information in the Ganeti cheat sheet,
the presentations or `man gnt-instance` to work out how.

<!-- Markdeep: --><style class="fallback">body{visibility:hidden;white-space:pre;font-family:monospace}</style><script src="markdeep.min.js"></script><script src="https://casual-effects.com/markdeep/latest/markdeep.min.js"></script><script>window.alreadyProcessedMarkdeep||(document.body.style.visibility="visible")</script>