Overview
The following page describes the steps to manually add a new Ethernet bridge (vmbrX) to an existing stoney cloud installation.
Before:
+----------------+----------------+----------------+----------------+
IP interfaces | 192.168.140.1X | 10.1.110.1X | 10.1.120.1X | 10.1.130.1X |
+----------------+----------------+----------------+----------------+
+----------------+----------------+----------------+----------------+
Bridging interfaces | vmbr0 | vmbr1 | vmbr2 | vmbr3 |
+----------------+----------------+----------------+----------------+
+----------------+----------------+----------------+----------------+
VLAN interfaces | vlan140 | vlan110 | vlan120 | vlan130 |
+----------------+----------------+----------------+----------------+
+-------------------------------------------------------------------+
Bonding interfaces | bond0 (bonding.mode=802.3ad) |
+-------------------------------------------------------------------+
+----------------+----------------+----------------+----------------+
Physical interfaces | eth0 | eth1 | eth2 | ethN |
+----------------+----------------+----------------+----------------+
After adding an additional bridge (vmbr4):
+----------------+----------------+----------------+----------------+----------------+
IP interfaces | 192.168.140.1X | 10.1.110.1X | 10.1.120.1X | 10.1.130.1X | 192.168.141.1X |
+----------------+----------------+----------------+----------------+----------------+
+----------------+----------------+----------------+----------------+----------------+
Bridging interfaces | vmbr0 | (vmbr1) | (vmbr2) | (vmbr3) | vmbr4 |
+----------------+----------------+----------------+----------------+----------------+
+----------------+----------------+----------------+----------------+----------------+
VLAN interfaces | vlan140 | vlan110 | vlan120 | vlan130 | vlan141 |
+----------------+----------------+----------------+----------------+----------------+
+------------------------------------------------------------------------------------+
Bonding interfaces | bond0 (bonding.mode=802.3ad) |
+------------------------------------------------------------------------------------+
+----------------+----------------+----------------+---------------------------------+
Physical interfaces | eth0 | eth1 | eth2 | ethN |
+----------------+----------------+----------------+---------------------------------+
Prerequisites
To be able to add and configure a new bridge, you need to have the following informations present:
- The name of the bridge, such as
vmbr4(vmbr1, vmbr2 and vmbr3 are reserved for existing networks) - VLAN ID, such as
141 - IP prefix, such as
192.0.2.0/24 - Default gateway address (if any), such as
192.0.2.1 - IP address for the DHCP server on the VM node, such as
192.0.2.13 - Broadcast address, such as
192.0.2.255
Make sure, that you have configured the new VLAN on all involved switches and routers beforehand.
Configuration
Add the new VLAN on top of the bonding interface
Login to the first VM node and manually add the new VLAN to the existing bonding interface (bond0). In the following example VLAN 141 is added:
ip link add link bond0 name vlan141 type vlan id 141 ip link set vlan141 up ip link show vlan141
71: vlan141@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff
In order to bring the newly configured VLAN up after a system reboot, the network configuration needs to be extended. Edit /etc/conf.d/net and add the following entry:
$EDITOR /etc/conf.d/net
# [...] #----------------------------------------------------------------------------- # VLAN (802.1q support) vlans_bond0="130 120 110 140 141" # pub VLAN vlan140_name="vlan140" config_vlan140="null" ### Add your new vlan at the end of the VLAN section # pub2 VLAN vlan141_name="vlan141" config_vlan141="null" # [...]
Create a new bridging device
As the new VLAN interface is now up, an additional bridge can be added.
Again the network configuration needs to be edited. Append the configuration of the new bridge, below the existing vmbr0 bridge configuration. In the following example the bridge vmbr4 is added on top of VLAN 141 with a prefix of 192.168.141.13/24:
$EDITOR /etc/conf.d/net
# [...] #----------------------------------------------------------------------------- # Bridging (802.1d) interfaces bridge_vmbr0="vlan140" bridge_add_vlan140="vmbr0" config_vmbr0="192.168.140.13/24 brd 192.168.140.255" # 192.168.140.13 is the ip address of the vm-node. routes_vmbr0="default via 192.168.140.1" ### Add your new bridge at the end of the bridge section bridge_vmbr4="vlan141" bridge_add_vlan141="vmbr4" config_vmbr4="192.168.141.13/24 brd 192.168.141.255" # 192.168.141.13 is the ip address of the vm-node. rc_net_vmbr4_provide="!net" # Otherwise stopping or restarting the bridge, will bring all net-services down.
To bring the bridge up, an init script is required. Simply symlink the net.lo script to the new bridge name's script, here net.vmbr4 is assumed:
ln -s /etc/init.d/net.lo /etc/init.d/net.vmbr4 rc-update add net.vmbr4 default /etc/init.d/net.vmbr4 start
* Caching service dependencies ... [ ok ] * Bringing up interface vmbr4 * Creating bridge vmbr4 ... * Adding ports to vmbr4 * vlan141 ... [ ok ] * 192.168.141.13/24 ... [ ok ]
Check if everything looks fine:
ip addr show vmbr4 && brctl show
82: vmbr4: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 38:60:77:9c:98:40 brd ff:ff:ff:ff:ff:ff
inet 192.168.141.13/24 brd 192.168.141.255 scope global vmbr4
valid_lft forever preferred_lft forever
bridge name bridge id STP enabled interfaces
vmbr0 8000.3860779c9840 no kvm-0217_0
vlan140
vmbr4 8000.3860779c9840 no kvm-0227_0
vlan141
Or if you would like to copy&paste:
vlanId=141 bridgeName=vmbr4 nodeIp=192.168.141.13 prefixLength=24 broadcast=192.168.141.255 cat << EOF >> /etc/conf.d/net bridge_${bridgeName}="vlan${vlanId}" bridge_add_vlan${vlanId}="${bridgeName}" config_${bridgeName}="${nodeIp}/${prefixLength} brd ${broadcast}" rc_net_${bridgeName}_provide="!net" EOF ln -s /etc/init.d/net.lo /etc/init.d/net.${bridgeName} rc-update add net.${bridgeName} default /etc/init.d/net.${bridgeName} start ip addr show ${bridgeName} && brctl show
Delete a bridging device
Tear the bridge down but check if it would also stop all network services:
# Set your bridge name bridgeName=<NAME OF YOUR BRIDGE> # e.g.: vmbr4 # Check if it excludes the net services grep rc_net_${bridgeName}_provide /etc/conf.d/net
If grep produces output you might consider running the initscript with the -D options (otherwise network services, like sshd, might get stopped).
Otherwise just stop the bridge and remove the initscript:
/etc/init.d/net.${bridgeName} stop rc-update del net.${bridgeName} default rm /etc/init.d/net.${bridgeName}
Remove the bridge from the network configuration:
$EDITOR /etc/conf.d/net
# [...] ### Remove the options for the bridge you want to delete #bridge_vmbr4="vlan141" #bridge_add_vlan141="vmbr4" #config_vmbr4="192.168.141.13/24 brd 192.168.141.255" # 192.168.141.13 is the ip address of the vm-node. #rc_net_vmbr4_provide="!net" # Otherwise stopping or restarting the bridge, will bring all net-services down. # [...]
Remove a VLAN on a bonding interface
Delete the VLAN from the interface bond0:
# Set your vlan id vlanId=<VLAN ID> # e.g. 141 # Shut the vlan down ip link set vlan${vlanId} down # Remove it from the interface bond0 ip link delete vlan${vlanId} type vlan
Edit /etc/conf.d/net and remove your VLAN:
$EDITOR /etc/conf.d/net
# [...] ### Remove your vlan options # pub2 VLAN # vlan141_name="vlan141" # config_vlan141="null" # [...]