In this exercise, you will:
whilst keeping the existing 100.64.0.X network for management traffic (i.e. administering the host machines)
By the end, your host will have three interfaces:
management 100.64.0.X
-----------+--------------
|
br-lan
|
+---------+---------+
| ens3 | host X
| |
|ens3.255 ens3.100|
+--+-----------+----+
| |
br-svc |
| |
VMs --------+ +------> to other hosts
100.64.100.XThese could be three physical interfaces (e.g. ens3, ens4 and ens5), but in this exercise we will use VLANs. You can think of ens3, ens3.100 and ens3.255 as 3 different interfaces.
All of the actions in this exercise are done as "root", so if you are not root already, type:
$ sudo -s
#Ensure you have the vlan management tools installed:
# apt-get install vlanLet's start with the Replication network. To do this, edit the network configuration file /etc/netplan/01-netcfg.yaml, and add this at the end of it:
vlans:
# Replication vlan
ens3.100:
link: ens3
id: 100
addresses: [100.64.100.X/24]The word vlans: should be in the same column as the word bridges: and ethernets:, e.g. your file should look something like this:
network:
version: 2
renderer: networkd
ethernets:
ens3:
dhcp4: false
bridges:
br-lan:
interfaces: [ens3]
addresses: [100.64.1.X/22]
gateway4: 100.64.0.1
parameters:
stp: false
forward-delay: 0
#
# NEW STUFF BELOW
#
vlans:
# Replication vlan
ens3.100:
link: ens3
id: 100
addresses: [100.64.100.X/24]Remember to replace X with the number of your host.
You can test the network configuration file with the following command:
# netplan --debug generateCheck for errors.
This creates a new "sub-interface", called ens3.100. This naming convention using dot is quite common, and it means that packets on this sub-interface enter and leave physical port ens3 with VLAN tag 100 added.
Now activate the interface:
# netplan applyIf for some reason you get an error on the MASTER node, with a complaint about br-lan:0, then you need to do the following:
# systemctl ganeti stop
# ifconfig br-lan:0 0.0.0.0
# netplan apply
# systemctl ganeti startifconfig br-lan:0 0.0.0.0 means "remove the IP address from br-lan:0".
Check that br-lan:0 does have an IP afteer you restart ganeti
Check that you can ping all the other hosts in your cluster on their 100.64.100.X addresses.
Note that we have not created a bridge. This network will just be used for the hosts to send DRBD traffic to each other. The individual VMs should not have access to this network (and therefore there is no need for a virtual switch/bridge to attach them to).
Now we have to change the cluster configuration so that Ganeti uses the secondary network for DRBD traffic.
Like all configuration changes, this is done on the MASTER node. Only one person should do this section.
Don't do this until all the hosts in your cluster can ping each other on their 100.64.100.X networks.
First, shutdown all running DRBD instances.
# gnt-instance list -o name,pnode,snodes,statusFor any instances which have a secondary node and have status 'running', issue the command to shut them down:
# gnt-instance shutdown <NAME>Now add a secondary IP address (-s) to the master node itself:
# gnt-node modify -s 100.64.100.X --force hostX.ws.nsrc.organd repeat for the other nodes in the cluster:
# gnt-node modify -s 100.64.100.Y --force hostY.ws.nsrc.orgCheck that all the nodes in the cluster have a secondary IP:
# gnt-node list -o name,sipOnce this has been done, you can restart your DRBD instances.
Now add the Service network to every host in your cluster. Same as before, go to the end of /etc/netplan/01-config.yaml, and update the vlans: bit, adding a new vlan ens3.255 below the section for ens3.100:, like so:
vlans:
# Replication vlan
ens3.100:
link: ens3
id: 100
addresses: [100.64.100.X/24]
# Service vlan - NEW
ens3.255:
link: ens3
id: 255
~~~
Then further up, edit the bridges section and add a `br-svc` bridge, below
the `br-lan` bridge, like so:
~~~
bridges:
br-lan:
interfaces: [ens3]
addresses: [100.64.1.X/22]
gateway4: 100.64.0.1
parameters:
stp: false
forward-delay: 0
# Service bridge - NEW
br-svc:
interfaces: [ ens3.255 ]
parameters:
stp: false
forward-delay: 0
~~~
This time we have made a bridge so that the VMs can attach to it, but notice
that we have NOT configured an IP address for the host on br-svc. The VMs
will send their ethernet frames through the bridge, but there is no need for
the physical host itself to have an address on this network. Ensuring the
host OS is not visible improves security.
Test the configuration and activate the interface:
~~~
# netplan --debug generate
# netplan apply
~~~
Since the host doesn't have an address on this network, we can't test using
ping. However you can check that the bridge exists and has the right
interface in it:
~~~
# brctl show
bridge name bridge id STP enabled interfaces
br-svc 8000.d4ae52c12e7e no ens3.255
~~~
## Attaching VMs to the service network
Each of you can take one of your VMs and move its network interface to
the service network, by logging into the MASTER node and using the
`gnt-instance modify` command:
~~~
# gnt-instance modify --net 0:modify,link=br-svc wordpressX
~~~
You'll need to reboot the instance after you made the change:
~~~
# gnt-instance reboot wordpressX
~~~
When it restarts, the virtual eth0 interface on the instance should be
connected to br-svc instead of br-lan.
Go into the VM's VNC console to find out what IP address it has picked up on
the new network. You should find that it now has a 100.64.255.X address.
Repeat for other VMs.
You can use check the status using `gnt-instance list`:
~~~
# gnt-instance list -o name,nic.bridge/0
~~~
## Changing the default interface: MASTER NODE ONLY
You can now configure the cluster so that all subsequent VMs you create
will be connected to br-svc instead of br-lan.
~~~
# gnt-cluster modify -N link=br-svc
~~~
# Review
After these changes, your `/etc/netplan/01-config.yaml` should look something
like this (IPs should be the ones for your PC, of course):
network: version: 2 renderer: networkd ethernets: ens3: dhcp4: false bridges: br-lan: interfaces: [ens3] addresses: [100.64.1.X/22] gateway4: 100.64.0.1 parameters: stp: false forward-delay: 0
# Service bridge
br-svc:
interfaces: [ ens3.255 ]
parameters:
stp: false
forward-delay: 0vlans: # Replication vlan ens3.100: link: ens3 id: 100 addresses: [100.64.100.X/24]
# Service vlan
ens3.255:
link: ens3
id: 255~~~~~
(Note: on a production machine it's a good idea to reboot after making major changes to /etc/netplan/01-config.yaml to check they are picked up correctly at system startup)
Create an instance which has two virtual NICs: eth0 connected to br-lan and eth1 connected to br-svc. Use the information in the Ganeti cheat sheet, the presentations or man gnt-instance to work out how.