# Proxmox Networking Lab

In this lab, you’ll create a new “public” bridge for VMs to sit on, separate from the Proxmox management network.

The public subnets will be different for each cluster:

- cluster1: 100.64.11.0/24
- cluster2: 100.64.12.0/24
- cluster3: 100.64.13.0/24
- cluster4: 100.64.14.0/24
- cluster5: 100.64.15.0/24
- cluster6: 100.64.16.0/24

Except where indicated, you will be working in your groups on your assigned nodes, or your group’s VM (groupXY-web).

# Updating node network configuration

Network interface and bridge configuration changes need to be made on each node separately - they are not replicated. Therefore, each group will need to do this on the node that they manage.

In the Proxmox GUI, navigate to your node nodeXY. Select “Network” (under “System”) in the second column.

Double-click on the currently unused and inactive interface `enp8s0`:

This should take you to an editing sreen:

Select “Autostart” and save. (Do not set any IP address; this interface is only for carrying traffic to and from the outside world, via the bridge you’re just about to create)

Now from the drop-down “Create” menu, select “Linux Bridge”

This will bring up a settings page:

Here, set:

- Name: retain the automatic selection of “vmbr1”
- Autostart: checked
- Bridge ports: `enp8s0` (you have to type it)
- Comment: VM public subnet
- Do NOT set any IP address. (For security, the Proxmox node itself should not be reachable on this subnet; it carries only VM traffic to and from the outside)
- Click Create

At the bottom of the screen should be a list of differences to be applied to `/etc/network/interfaces`, with each line to be removed prefixed by `-` and each line to be added to be prefixed with `+`.

At the top of the screen, click “Apply Configuration”:

The “Active” status of enp8s0 should change to “Yes”.

# Update VM configuration

Find your groupXY-web server. If it’s running, shut it down.

Select Hardware, and double-click on Network Device (net0)

Change the bridge from vmbr0 to vmbr1, and save.

Start the VM, get a console and login. Get its IP address:

    ip address list     # or "ip a l" for short

You should find it now has an address on a different subnet - since it’s connected to a different bridge which has a different DHCP server on it.

See if you can ping this IP address from your laptop, and see if you can access your web server from a browser using this new IP address.

# Proxmox firewall

## Enabling firewall

Only one person in the entire cluster should do this step. Therefore, get together with all the other groups in your cluster to agree who will do it

Before enabling the firewall, create an IPSet called “management” with the IPs we are accessing from. It must be called exactly “management”; Proxmox allows SSH, and HTTPS on port 8006, from this IPSet.

Go to Datacenter, Firewall > IPSet

On the left panel, click Create. Enter name “management”. (It must be called exactly “management” in lower case; Proxmox allows SSH, and HTTPS on port 8006, from an IPSet with this name)

Select the “management” IPSet in the left side, and then next to IP/CIDR on the right side, click Add.

Enter subnet `100.64.0.0/22` and a comment saying this is the class wifi network.

This ensures we won’t lose management access to Proxmox when we enable the firewall.

Next, we need to add some rules to allow traffic between the cluster nodes (linstor, ceph, etc)

First, create a new IPSet called “clusterX-nodes” (changing clusterX as appropriate).

Inside this IPSet, create individual entries with the IP address of each node and the monitoring server:

- 100.64.0.1X0 # comment: monX
- 100.64.0.1X1 # comment: nodeX1
- 100.64.0.1X2 # comment: nodeX2
- 100.64.0.1X3 # comment: nodeX3
- 100.64.0.1X4 # comment: nodeX4
- 100.64.0.1X5 # comment: nodeX5

When finished, it will look this this:

Next click on Firewall in the second column. Here you will add two new rules.

The first rule is to allow traffic between the management IPs of the nodes (this includes linstor API)

- Enable: yes
- Source: +clusterX-nodes (select this from the dropdown)
- Destination: +clusterX-nodes (ditto)
- Comment: Allow inter-node traffic

The second rule is to allow all traffic on the storage network (this includes ceph OSD and linstor DRBD)

- Enable: yes
- Source: 100.64.5.0/24
- Destination: 100.64.5.0/24
- Comment: Allow storage replication

Finally, it’s time to enable the firewall. Go to Firewall > Options, double-click on Enable, and check the box to enable the firewall.

## Check for breakage

Get a shell on `nodeX1` and try `linstor node list` (or `linstor n l`)

If it looks like this…

    root@node01:~# linstor node list
    ╭────────────────────────────────────────────────────────────────────────────────────────────────╮
    ┊ Node   ┊ NodeType   ┊ Addresses                 ┊ State                                        ┊
    ╞════════════════════════════════════════════════════════════════════════════════════════════════╡
    ┊ node01 ┊ CONTROLLER ┊ 100.64.0.101:3370 (PLAIN) ┊ Online                                       ┊
    ┊ node02 ┊ SATELLITE  ┊ 100.64.0.102:3366 (PLAIN) ┊ OFFLINE (Auto-eviction: 2025-08-20 05:39:57) ┊
    ┊ node03 ┊ SATELLITE  ┊ 100.64.0.103:3366 (PLAIN) ┊ OFFLINE (Auto-eviction: 2025-08-20 05:39:57) ┊
    ┊ node04 ┊ SATELLITE  ┊ 100.64.0.104:3366 (PLAIN) ┊ OFFLINE (Auto-eviction: 2025-08-20 05:39:54) ┊
    ┊ node05 ┊ SATELLITE  ┊ 100.64.0.105:3366 (PLAIN) ┊ OFFLINE (Auto-eviction: 2025-08-20 05:39:57) ┊
    ╰────────────────────────────────────────────────────────────────────────────────────────────────╯

…then it’s broken! Check your firewall rules above.

## Configuring firewalling on an instance

Now break back into your groups, to work on your virtual machines

Make sure your groupXY-web instance is running, and find its IP address (e.g. by logging in at the console and running `ip address list`)

Ping this IP address from your laptop. It should respond. (If not, please ask for help).

Select the instance in the left navigation column, and Firewall > Options in the second column. Is the firewall enabled? If not, then double-click on this option and enable it.

Stop the ping on your laptop, and wait 5 seconds for the PVE firewall change to be applied.

Now restart the ping from your laptop. You will probably find that the ping traffic is still being allowed. Why is it not dropped? The answer is because firewalling has to be enabled on the network interface as well.

Select “Hardware” in column 2, then double click on “Network Device (net0)”

This will edit the network setings. Enable “Firewall” and save.

Again, stop the ping, wait 5 seconds, and start it again. It should now stop responding. Also, if you enter this IP address into your web browser, you should be unable to load your web page.

While your VM is still selected in column 1, click on “Firewall” in column 2, and click Add to add a rule.

- Enable: yes
- Protocol: TCP
- Destination port: 80
- Comment: allow inbound web

Once this rule is created, within a few seconds you should be able to view your web page in a browser - but there will still be no response to ping.

Add another rule to allow pings:

- Enable: yes
- Procotol: ICMP
- ICMP type: echo-request (8)
- Comment: allow ping

Within a few seconds, you should be able to ping your instance.

## Show the firewall rule configuration

Get a shell on your Proxmox node (or any Proxmox node in your cluster)

Have a look at where the filewall configuration files are stored:

    ls /etc/pve/firewall/

Pick your group’s virtual machine ID, and look at its text config:

    cat /etc/pve/firewall/NNN.fw    # NNN is your vm id

Do you recognise the rules you created? Having them in this text format makes them easy to backup and to audit.