Latest revision as of 10:03, 21 February 2014

Overview

The following page describes the steps to manually add a new Ethernet bridge (vmbrX) to an existing stoney cloud installation. In this example we presume you want to add the new Ethernet bridge vmbr4.

Before adding a new bridge

                     +----------------+----------------+----------------+----------------+
IP interfaces        | 192.168.140.1X |   10.1.110.1X  |   10.1.120.1X  |   10.1.130.1X  |
                     +----------------+----------------+----------------+----------------+
                     +----------------+----------------+----------------+----------------+
Bridging interfaces  |     vmbr0      |    (vmbr1)     |    (vmbr2)     |    (vmbr3)     |
                     +----------------+----------------+----------------+----------------+
                     +----------------+----------------+----------------+----------------+
VLAN interfaces      |    vlan140     |    vlan110     |    vlan120     |    vlan130     |
                     +----------------+----------------+----------------+----------------+
                     +-------------------------------------------------------------------+
Bonding interfaces   |                        bond0 (bonding.mode=802.3ad)               |
                     +-------------------------------------------------------------------+
                     +----------------+----------------+----------------+----------------+
Physical interfaces  |      eth0      |      eth1      |      eth2      |      ethN      |
                     +----------------+----------------+----------------+----------------+

After adding an additional bridge (vmbr4)

                     +----------------+----------------+----------------+----------------+----------------+
IP interfaces        | 192.168.140.1X |   10.1.110.1X  |   10.1.120.1X  |   10.1.130.1X  | 192.168.141.1X |
                     +----------------+----------------+----------------+----------------+----------------+
                     +----------------+----------------+----------------+----------------+----------------+
Bridging interfaces  |     vmbr0      |    (vmbr1)     |    (vmbr2)     |    (vmbr3)     |     vmbr4      |
                     +----------------+----------------+----------------+----------------+----------------+
                     +----------------+----------------+----------------+----------------+----------------+
VLAN interfaces      |    vlan140     |    vlan110     |    vlan120     |    vlan130     |    vlan141     |
                     +----------------+----------------+----------------+----------------+----------------+
                     +------------------------------------------------------------------------------------+
Bonding interfaces   |                        bond0 (bonding.mode=802.3ad)                                |
                     +------------------------------------------------------------------------------------+
                     +----------------+----------------+----------------+---------------------------------+
Physical interfaces  |      eth0      |      eth1      |      eth2      |      ethN                       |
                     +----------------+----------------+----------------+---------------------------------+

Prerequisites

To be able to add and configure a new bridge, you need to have the following informations present:

• The name of the bridge, such as vmbr4 (vmbr1, vmbr3 and vmbr3 are reserved for existing networks)
• VLAN ID, such as 141
• IP prefix, such as 192.168.141.0/24
• Default gateway address (if any), such as 192.168.141.1
• IP address for the DHCP server on the VM node, such as 192.168.141.13
• Broadcast address, such as 192.168.141.255

Make sure, that you have configured the new VLAN on all involved switches and routers beforehand.

Configuration

Add the new VLAN on top of the bonding interface

Login to the first VM node and manually add the new VLAN to the existing bonding interface (bond0). In the following example VLAN 141 is added:

ip link add link bond0 name vlan141 type vlan id 141
ip link set vlan141 up
ip link show vlan141

71: vlan141@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT 
    link/ether 38:60:77:9c:98:40 brd ff:ff:ff:ff:ff:ff

In order to bring the newly configured VLAN up after a system reboot, the network configuration needs to be extended. Edit /etc/conf.d/net and add the following entries:

$EDITOR /etc/conf.d/net

# [...]
 
#-----------------------------------------------------------------------------
# VLAN (802.1q support)
 
### Add the new vlan to the bond device
vlans_bond0="130 120 110 140 141"
 
# pub VLAN
vlan140_name="vlan140"
config_vlan140="null"
 
### Add your new vlan at the end of the VLAN section
# pub2 VLAN
vlan141_name="vlan141"
config_vlan141="null"
 
# [...]

Create a new bridging device

As the new VLAN interface is now up, an additional bridge can be added.

Again the network configuration needs to be edited. Append the configuration of the new bridge, below the existing vmbr0 bridge configuration. In the following example the bridge vmbr4 is added on top of VLAN 141 with a prefix of 192.168.141.13/24:

$EDITOR /etc/conf.d/net

# [...]
 
#-----------------------------------------------------------------------------
# Bridging (802.1d) interfaces
 
bridge_vmbr0="vlan140"
bridge_add_vlan140="vmbr0"
config_vmbr0="192.168.140.13/24 brd 192.168.140.255"    # 192.168.140.13 is the ip address of the vm-node.
routes_vmbr0="default via 192.168.140.1"
 
### Add your new bridge at the end of the bridge section
bridge_vmbr4="vlan141"
bridge_add_vlan141="vmbr4"
config_vmbr4="192.168.141.13/24 brd 192.168.141.255"    # 192.168.141.13 is the ip address of the vm-node (only required on primary- and secondary master node for the dhcp server, on all others use: null)
rc_net_vmbr4_provide="!net"                             # Otherwise stopping or restarting the bridge, will bring all net-services down.

To bring the bridge up, an init script is required. Simply symlink the net.lo script to the new bridge name's script, here net.vmbr4 is assumed:

ln -s /etc/init.d/net.lo /etc/init.d/net.vmbr4
rc-update add net.vmbr4 default
/etc/init.d/net.vmbr4 start

 * Caching service dependencies ...                                       [ ok ]
 * Bringing up interface vmbr4
 *   Creating bridge vmbr4 ...
 *   Adding ports to vmbr4
 *     vlan141 ...                                                        [ ok ]
 *   192.168.141.13/24 ...                                                [ ok ]

Check if everything looks fine:

ip addr show vmbr4 && brctl show

82: vmbr4: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 38:60:77:9c:98:40 brd ff:ff:ff:ff:ff:ff
    inet 192.168.141.13/24 brd 192.168.141.255 scope global vmbr4
       valid_lft forever preferred_lft forever
bridge name     bridge id               STP enabled     interfaces
vmbr0           8000.3860779c9840       no              kvm-0217_0
                                                        vlan140
vmbr4           8000.3860779c9840       no              kvm-0227_0
                                                        vlan141

Or if you would like to copy&paste:

vlanId=141
bridgeName=vmbr4
nodeIp=192.168.141.13
prefixLength=24
broadcast=192.168.141.255
 
cat << EOF >> /etc/conf.d/net
 
bridge_${bridgeName}="vlan${vlanId}"
bridge_add_vlan${vlanId}="${bridgeName}"
config_${bridgeName}="${nodeIp}/${prefixLength} brd ${broadcast}"
rc_net_${bridgeName}_provide="!net"
EOF
 
ln -s /etc/init.d/net.lo /etc/init.d/net.${bridgeName}
rc-update add net.${bridgeName} default
/etc/init.d/net.${bridgeName} start
ip addr show ${bridgeName} && brctl show

Delete a bridging device

Tear the bridge down but check if it would also stop all network services:

# Set your bridge name
bridgeName=<NAME OF YOUR BRIDGE>   # e.g.: vmbr4
 
# Check if it excludes the net services
grep rc_net_${bridgeName}_provide /etc/conf.d/net

If grep produces output you might consider running the initscript with the -D options (otherwise network services, like sshd, might get stopped).

Otherwise just stop the bridge and remove the initscript:

/etc/init.d/net.${bridgeName} stop
 
rc-update del net.${bridgeName} default
 
rm /etc/init.d/net.${bridgeName}

Remove the bridge from the network configuration:

$EDITOR /etc/conf.d/net

# [...]
 
### Remove the options for the bridge you want to delete
#bridge_vmbr4="vlan141"
#bridge_add_vlan141="vmbr4"
#config_vmbr4="192.168.141.13/24 brd 192.168.141.255"    # 192.168.141.13 is the ip address of the vm-node.
#rc_net_vmbr4_provide="!net"                             # Otherwise stopping or restarting the bridge, will bring all net-services down.
 
# [...]

Remove a VLAN on a bonding interface

Delete the VLAN from the interface bond0:

# Set your vlan id
vlanId=<VLAN ID>   # e.g. 141
 
# Shut the vlan down
ip link set vlan${vlanId} down
 
# Remove it from the interface bond0
ip link delete vlan${vlanId} type vlan

Edit /etc/conf.d/net and remove your VLAN:

$EDITOR /etc/conf.d/net

# [...]
 
### Remove your vlan options
# pub2 VLAN
# vlan141_name="vlan141"
# config_vlan141="null"
 
# [...]

Links

• Gentoo Handbook - VLAN (802.1q support)
• ArchWiki - VLAN
• Linux Foundation - Bridge
• Gentoo Handbook - Bridging (802.1d support)
• Gentoo Wiki - Network Bridge