We should now have 3 LXD hosts set up (one on each physical server). For each group of 3 servers, you will have to pick one server in your group will become LXD cluster master (this is a bit like Ganeti, but we are NOT interacting with Ganeti at all in this lab).
On each LXD Host, run the following commands (this should have been done during lxd init, but we want to be sure they are correct).
$ lxc config set core.https_address IP.OF.YOUR.HOST:8443
$ lxc config set core.trust_password 'class-password'... remember to replace IP.OF.YOUR.VM with the IP of your Host on br-lan, e.g. 100.64.1.X)
Now, on the machine you've picked to be the LXD cluster master (not everyone can be! only 1 machine in each group!), write:
$ lxc cluster enable NAME_OF_MASTER_LXD_NODE... where "NAME_OF_MASTER_LXD_NODE" is the hostname of the host that you picked to be master of thee cluster - if it's host6, then use 'host6` :)
After a few seconds, you should see:
Clustering enabledLook at your cluster:
$ lxc cluster list+-------+-----------------------------+----------+--------------+----------------+-------------+--------+-------------------+
| NAME | URL | ROLES | ARCHITECTURE | FAILURE DOMAIN | DESCRIPTION | STATE | MESSAGE |
+-------+-----------------------------+----------+--------------+----------------+-------------+--------+-------------------+
| hostX | https://100.64.0.XX | database | x86_64 | default | | ONLINE | Fully operational |
+-------+-----------------------------+----------+--------------+----------------+-------------+--------+-------------------+A one-node cluster isn't very useful... Let's add more LXD hosts!
Make note of your IP address on br-lan, as we'll be using that for others to join our cluster (hostX = 100.64.1.X)
Now, we'll get the other hosts to join. Make sure they are running LXD as well :)
Unfortunately, there isn't a way to preserve containers on nodes which join an existing cluster. So first, we need to do a few things:
$ lxc listStop and delete all containers (remember, only do this on the other two Hosts that are NOT LXD cluster master)
$ lxc stop --all
$ lxc delete <..1..>
$ lxc delete <..2..>Then we need to totally wipe the cluster config (ONLY on the other hosts!)
# snap remove --purge lxdNow, reinstall LXD (it actually just unpacks the downloaded LXD snap package from erlier).
# snap install lxdBefore you can re-init the LXD node, you need a "Join Token", obtained from the cluster master -- how do we do this ?
ON CLUSTER MASTER:
You ask LXD to issue a "Join Token" which is a long string containing all information for the new node to join including the master name, IP, and password:
Remember, run this on the master (the one you ran "lxd cluster enable" on earlier):
hostX# lxc cluster add hostY
Member hostY join token:
eyJzZXJ2Z [.....] Q5YTg5ZjY4ODA5MzEzZmE2NDUwNDJlIn0=The Join Token will be several lines long - if you are logged in via SSH, then the easiest is to COPY the text, and in another window, log in to the new node (hostY).
You can also paste the Join Token into a file:
hostX# editor /tmp/jointoken
<paste join token here>
eyJzZXJ2Z [.....] Q5YTg5ZjY4ODA5MzEzZmE2NDUwNDJlIn0=Save the file, then scp that file over to the other node (hostY).
It's time to run lxd init!
$ sudo lxd initHere is how to answer the questions - those prefixed with -> need data from you, the others can be just left to the default.
-> Would you like to use LXD clustering? (yes/no) [default=no]: yes
What IP address or DNS name should be used to reach this node? [default=100.64.1.XX]:
-> Are you joining an existing cluster? (yes/no) [default=no]: yes
-> Do you have a join token? (yes/no/[token]) [default=no]: yes
-> Please provide join token: eyJzZXJ2Z [.....] Q5YTg5ZjY4ODA5MzEzZmE2NDUwNDJlIn0=
-> All existing data is lost when joining a cluster, continue? (yes/no) [default=no] yes
Choose "zfs.pool_name" property for storage pool "default":
-> Choose "size" property for storage pool "default": 20GB
Choose "source" property for storage pool "default":
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: noIf everything goes well, you've joined hostY to the cluster -- congratulations!
Verify this:
$ lxc cluster listYou should see:
+------+-----------------------------+------------------+--------------+----------------+-------------+--------+-------------------+
| NAME | URL | ROLES | ARCHITECTURE | FAILURE DOMAIN | DESCRIPTION | STATE | MESSAGE |
+------+-----------------------------+------------------+--------------+----------------+-------------+--------+-------------------+
| hostX | https://100.64.1.X:8443 | database | x86_64 | default | | ONLINE | Fully operational |
+------+-----------------------------+------------------+--------------+----------------+-------------+--------+-------------------+
| hostY | https://100.64.1.Y:8443 | database-standby | x86_64 | default | | ONLINE | Fully operational |
+------+-----------------------------+------------------+--------------+----------------+-------------+--------+-------------------+Repeat for the third node in the group!
Now, you can launch/init containers from our nsrc-images repository like earlier.
Try creating containers, then try moving them to another node, and starting them up!
Notice, when listing running containers:
+-------+---------+-------------------+------+-----------+-----------+----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | LOCATION |
+-------+---------+-------------------+------+-----------+-----------+----------+
| u2004 | RUNNING | 100.64.0.* (eth0) | | CONTAINER | 0 | host |
+-------+---------+-------------------+------+-----------+-----------+----------+
| u-2 | RUNNING | 100.64.0.* (eth0) | | CONTAINER | 0 | hostX |
+-------+---------+-------------------+------+-----------+-----------+----------+... there is now a LOCATION column to let you know where containers are running.
lxc stop container_name
lxc move container_name --target hostY