Difference between revisions of "Manually adding a new bridge"

From stoney cloud
Jump to: navigation, search
[unchecked revision][unchecked revision]
(→‎Create a new bridging device)
(→‎Create a new bridging device)
 
(25 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
= Overview =
 
= Overview =
−
The following page describes the steps to manually add a new Ethernet bridge (<code>vmbrX</code>) to an existing stoney cloud installation.
+
The following page describes the steps to manually add a new Ethernet bridge (<code>vmbrX</code>) to an existing stoney cloud installation. In this example we presume you want to add the new Ethernet bridge <code>vmbr4</code>.
 +
 
 +
== Before adding a new bridge ==
 +
<pre>
 +
                    +----------------+----------------+----------------+----------------+
 +
IP interfaces        | 192.168.140.1X |  10.1.110.1X  |  10.1.120.1X  |  10.1.130.1X  |
 +
                    +----------------+----------------+----------------+----------------+
 +
                    +----------------+----------------+----------------+----------------+
 +
Bridging interfaces  |    vmbr0      |    (vmbr1)    |    (vmbr2)    |    (vmbr3)    |
 +
                    +----------------+----------------+----------------+----------------+
 +
                    +----------------+----------------+----------------+----------------+
 +
VLAN interfaces      |    vlan140    |    vlan110    |    vlan120    |    vlan130    |
 +
                    +----------------+----------------+----------------+----------------+
 +
                    +-------------------------------------------------------------------+
 +
Bonding interfaces  |                        bond0 (bonding.mode=802.3ad)              |
 +
                    +-------------------------------------------------------------------+
 +
                    +----------------+----------------+----------------+----------------+
 +
Physical interfaces  |      eth0      |      eth1      |      eth2      |      ethN      |
 +
                    +----------------+----------------+----------------+----------------+
 +
</pre>
 +
 
 +
== After adding an additional bridge (vmbr4) ==
 +
<pre>
 +
                    +----------------+----------------+----------------+----------------+----------------+
 +
IP interfaces        | 192.168.140.1X |  10.1.110.1X  |  10.1.120.1X  |  10.1.130.1X  | 192.168.141.1X |
 +
                    +----------------+----------------+----------------+----------------+----------------+
 +
                    +----------------+----------------+----------------+----------------+----------------+
 +
Bridging interfaces  |    vmbr0      |    (vmbr1)    |    (vmbr2)    |    (vmbr3)    |    vmbr4      |
 +
                    +----------------+----------------+----------------+----------------+----------------+
 +
                    +----------------+----------------+----------------+----------------+----------------+
 +
VLAN interfaces      |    vlan140    |    vlan110    |    vlan120    |    vlan130    |    vlan141    |
 +
                    +----------------+----------------+----------------+----------------+----------------+
 +
                    +------------------------------------------------------------------------------------+
 +
Bonding interfaces  |                        bond0 (bonding.mode=802.3ad)                                |
 +
                    +------------------------------------------------------------------------------------+
 +
                    +----------------+----------------+----------------+---------------------------------+
 +
Physical interfaces  |      eth0      |      eth1      |      eth2      |      ethN                      |
 +
                    +----------------+----------------+----------------+---------------------------------+
 +
</pre>
  
 
= Prerequisites =
 
= Prerequisites =
 
To be able to add and configure a new bridge, you need to have the following informations present:
 
To be able to add and configure a new bridge, you need to have the following informations present:
 +
* The name of the bridge, such as <code>vmbr4</code> (<code>vmbr1</code>, <code>vmbr3</code> and <code>vmbr3</code> are reserved for existing networks)
 
* VLAN ID, such as <code>141</code>
 
* VLAN ID, such as <code>141</code>
−
* IP prefix, such as <code>192.0.2.0/24</code>
+
* IP prefix, such as <code>192.168.141.0/24</code>
−
* Default gateway address  (if any), such as <code>192.0.2.1</code>
+
* Default gateway address  (if any), such as <code>192.168.141.1</code>
−
* IP address for the DHCP server on the VM node, such as <code>192.0.2.13</code>
+
* IP address for the DHCP server on the VM node, such as <code>192.168.141.13</code>
−
* Broadcast address, such as <code>192.0.2.255</code>
+
* Broadcast address, such as <code>192.168.141.255</code>
  
 
Make sure, that you have configured the new VLAN on all involved switches and routers beforehand.
 
Make sure, that you have configured the new VLAN on all involved switches and routers beforehand.
Line 22: Line 61:
 
<pre>
 
<pre>
 
71: vlan141@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT  
 
71: vlan141@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT  
−
     link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff
+
     link/ether 38:60:77:9c:98:40 brd ff:ff:ff:ff:ff:ff
 
</pre>
 
</pre>
  
−
In order to bring the newly configured VLAN up after a system reboot, the network configuration needs to be extended. Edit <code>/etc/conf.d/net</code> and add the following entry:
+
In order to bring the newly configured VLAN up after a system reboot, the network configuration needs to be extended. Edit <code>/etc/conf.d/net</code> and add the following entries:
−
<source lang="bash">
+
$EDITOR /etc/conf.d/net
−
$EDITOR /etc/conf.d/net
+
−
</source>
+
  
 
<source lang="bash">
 
<source lang="bash">
Line 35: Line 72:
 
#-----------------------------------------------------------------------------
 
#-----------------------------------------------------------------------------
 
# VLAN (802.1q support)
 
# VLAN (802.1q support)
−
+
 
 +
### Add the new vlan to the bond device
 
vlans_bond0="130 120 110 140 141"
 
vlans_bond0="130 120 110 140 141"
  
Line 53: Line 91:
 
As the new VLAN interface is now up, an additional bridge can be added.
 
As the new VLAN interface is now up, an additional bridge can be added.
  
−
Again the network configuration needs to be edited. Append the configuration of the new bridge, below the existing <code>vmbr0</code> bridge configuration. In the following example the bridge <code>vmbr4</code> is added on top of VLAN <code>141</code> with a prefix of <code>192.0.2.13/24</code>:
+
Again the network configuration needs to be edited. Append the configuration of the new bridge, below the existing <code>vmbr0</code> bridge configuration. In the following example the bridge <code>vmbr4</code> is added on top of VLAN <code>141</code> with a prefix of <code>192.168.141.13/24</code>:
−
<source lang="bash">
+
$EDITOR /etc/conf.d/net
−
$EDITOR /etc/conf.d/net
+
−
</source>
+
  
 
<source lang="bash">
 
<source lang="bash">
Line 64: Line 100:
 
# Bridging (802.1d) interfaces
 
# Bridging (802.1d) interfaces
  
−
bridge_vmbr0="vlanXXX"
+
bridge_vmbr0="vlan140"
−
bridge_add_vlanXXX="vmbr0"
+
bridge_add_vlan140="vmbr0"
−
config_vmbr0="XXX.XXX.XXX.XXX/XX brd XXX.XXX.XXX.XXX"
+
config_vmbr0="192.168.140.13/24 brd 192.168.140.255"   # 192.168.140.13 is the ip address of the vm-node.
−
routes_vmbr0="default via XXX.XXX.XXX.XXX"
+
routes_vmbr0="default via 192.168.140.1"
  
 
### Add your new bridge at the end of the bridge section
 
### Add your new bridge at the end of the bridge section
 
bridge_vmbr4="vlan141"
 
bridge_vmbr4="vlan141"
 
bridge_add_vlan141="vmbr4"
 
bridge_add_vlan141="vmbr4"
−
config_vmbr4="192.0.2.13/24 brd 192.0.2.255"
+
config_vmbr4="192.168.141.13/24 brd 192.168.141.255"   # 192.168.141.13 is the ip address of the vm-node (only required on primary- and secondary master node for the dhcp server, on all others use: null)
−
rc_net_vmbr4_provide="!net" # Otherwise stopping or restarting the bridge, will bring all net-services down.
+
rc_net_vmbr4_provide="!net"                             # Otherwise stopping or restarting the bridge, will bring all net-services down.
 
</source>
 
</source>
−
 
  
 
To bring the bridge up, an init script is required. Simply symlink the <code>net.lo</code> script to the new bridge name's script, here <code>net.vmbr4</code> is assumed:
 
To bring the bridge up, an init script is required. Simply symlink the <code>net.lo</code> script to the new bridge name's script, here <code>net.vmbr4</code> is assumed:
Line 89: Line 124:
 
  *  Adding ports to vmbr4
 
  *  Adding ports to vmbr4
 
  *    vlan141 ...                                                        [ ok ]
 
  *    vlan141 ...                                                        [ ok ]
−
  *  192.0.2.13/24 ...                                                   [ ok ]
+
  *  192.168.141.13/24 ...                                               [ ok ]
 
</pre>
 
</pre>
  
Line 97: Line 132:
 
</source>
 
</source>
 
<pre>
 
<pre>
−
72: vmbr4: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP  
+
82: vmbr4: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP  
−
     link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff
+
     link/ether 38:60:77:9c:98:40 brd ff:ff:ff:ff:ff:ff
−
     inet 192.0.2.13/24 brd 192.0.2.255 scope global vmbr4
+
     inet 192.168.141.13/24 brd 192.168.141.255 scope global vmbr4
 
       valid_lft forever preferred_lft forever
 
       valid_lft forever preferred_lft forever
−
 
+
bridge name     bridge id               STP enabled     interfaces
−
bridge name bridge id STP enabled interfaces
+
vmbr0          8000.3860779c9840      no              kvm-0217_0
−
vmbr4 8000.3860779c9840 no vlan141
+
                                                        vlan140
 +
vmbr4           8000.3860779c9840       no             kvm-0227_0
 +
                                                        vlan141
 
</pre>
 
</pre>
  
Line 111: Line 148:
 
vlanId=141
 
vlanId=141
 
bridgeName=vmbr4
 
bridgeName=vmbr4
−
nodeIp=192.0.2.13
+
nodeIp=192.168.141.13
 
prefixLength=24
 
prefixLength=24
−
broadcast=192.0.2.255
+
broadcast=192.168.141.255
  
 
cat << EOF >> /etc/conf.d/net
 
cat << EOF >> /etc/conf.d/net
Line 120: Line 157:
 
bridge_add_vlan${vlanId}="${bridgeName}"
 
bridge_add_vlan${vlanId}="${bridgeName}"
 
config_${bridgeName}="${nodeIp}/${prefixLength} brd ${broadcast}"
 
config_${bridgeName}="${nodeIp}/${prefixLength} brd ${broadcast}"
−
rc_net_${bridgeName}_provide="!net" # Otherwise stopping or restarting the bridge, will bring all net-services down.
+
rc_net_${bridgeName}_provide="!net"
 
EOF
 
EOF
  
Line 127: Line 164:
 
/etc/init.d/net.${bridgeName} start
 
/etc/init.d/net.${bridgeName} start
 
ip addr show ${bridgeName} && brctl show
 
ip addr show ${bridgeName} && brctl show
 +
</source>
 +
 +
== Delete a bridging device ==
 +
Tear the bridge down but check if it would also stop all network services:
 +
<source lang="bash">
 +
# Set your bridge name
 +
bridgeName=<NAME OF YOUR BRIDGE>  # e.g.: vmbr4
 +
 +
# Check if it excludes the net services
 +
grep rc_net_${bridgeName}_provide /etc/conf.d/net
 +
</source>
 +
If grep produces output you might consider running the initscript with the ''-D'' options (otherwise network services, like sshd, might get stopped).
 +
 +
Otherwise just stop the bridge and remove the initscript:
 +
<source lang="bash">
 +
/etc/init.d/net.${bridgeName} stop
 +
 +
rc-update del net.${bridgeName} default
 +
 +
rm /etc/init.d/net.${bridgeName}
 +
</source>
 +
 +
Remove the bridge from the network configuration:
 +
$EDITOR /etc/conf.d/net
 +
 +
<source lang="bash">
 +
# [...]
 +
 +
### Remove the options for the bridge you want to delete
 +
#bridge_vmbr4="vlan141"
 +
#bridge_add_vlan141="vmbr4"
 +
#config_vmbr4="192.168.141.13/24 brd 192.168.141.255"    # 192.168.141.13 is the ip address of the vm-node.
 +
#rc_net_vmbr4_provide="!net"                            # Otherwise stopping or restarting the bridge, will bring all net-services down.
 +
 +
# [...]
 +
</source>
 +
 +
== Remove a VLAN on a bonding interface ==
 +
Delete the VLAN from the interface bond0:
 +
<source lang="bash">
 +
# Set your vlan id
 +
vlanId=<VLAN ID>  # e.g. 141
 +
 +
# Shut the vlan down
 +
ip link set vlan${vlanId} down
 +
 +
# Remove it from the interface bond0
 +
ip link delete vlan${vlanId} type vlan
 +
</source>
 +
 +
Edit <code>/etc/conf.d/net</code> and remove your VLAN:
 +
$EDITOR /etc/conf.d/net
 +
 +
<source lang="bash">
 +
# [...]
 +
 +
### Remove your vlan options
 +
# pub2 VLAN
 +
# vlan141_name="vlan141"
 +
# config_vlan141="null"
 +
 +
# [...]
 
</source>
 
</source>
  
Line 135: Line 234:
 
* [http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=4&chap=3#doc_chap7 Gentoo Handbook - Bridging (802.1d support)]
 
* [http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=4&chap=3#doc_chap7 Gentoo Handbook - Bridging (802.1d support)]
 
* [http://wiki.gentoo.org/wiki/Network_bridge Gentoo Wiki - Network Bridge]
 
* [http://wiki.gentoo.org/wiki/Network_bridge Gentoo Wiki - Network Bridge]
 +
 +
[[Category:Workarounds]]

Latest revision as of 10:03, 21 February 2014

Overview

The following page describes the steps to manually add a new Ethernet bridge (vmbrX) to an existing stoney cloud installation. In this example we presume you want to add the new Ethernet bridge vmbr4.

Before adding a new bridge

                     +----------------+----------------+----------------+----------------+
IP interfaces        | 192.168.140.1X |   10.1.110.1X  |   10.1.120.1X  |   10.1.130.1X  |
                     +----------------+----------------+----------------+----------------+
                     +----------------+----------------+----------------+----------------+
Bridging interfaces  |     vmbr0      |    (vmbr1)     |    (vmbr2)     |    (vmbr3)     |
                     +----------------+----------------+----------------+----------------+
                     +----------------+----------------+----------------+----------------+
VLAN interfaces      |    vlan140     |    vlan110     |    vlan120     |    vlan130     |
                     +----------------+----------------+----------------+----------------+
                     +-------------------------------------------------------------------+
Bonding interfaces   |                        bond0 (bonding.mode=802.3ad)               |
                     +-------------------------------------------------------------------+
                     +----------------+----------------+----------------+----------------+
Physical interfaces  |      eth0      |      eth1      |      eth2      |      ethN      |
                     +----------------+----------------+----------------+----------------+

After adding an additional bridge (vmbr4)

                     +----------------+----------------+----------------+----------------+----------------+
IP interfaces        | 192.168.140.1X |   10.1.110.1X  |   10.1.120.1X  |   10.1.130.1X  | 192.168.141.1X |
                     +----------------+----------------+----------------+----------------+----------------+
                     +----------------+----------------+----------------+----------------+----------------+
Bridging interfaces  |     vmbr0      |    (vmbr1)     |    (vmbr2)     |    (vmbr3)     |     vmbr4      |
                     +----------------+----------------+----------------+----------------+----------------+
                     +----------------+----------------+----------------+----------------+----------------+
VLAN interfaces      |    vlan140     |    vlan110     |    vlan120     |    vlan130     |    vlan141     |
                     +----------------+----------------+----------------+----------------+----------------+
                     +------------------------------------------------------------------------------------+
Bonding interfaces   |                        bond0 (bonding.mode=802.3ad)                                |
                     +------------------------------------------------------------------------------------+
                     +----------------+----------------+----------------+---------------------------------+
Physical interfaces  |      eth0      |      eth1      |      eth2      |      ethN                       |
                     +----------------+----------------+----------------+---------------------------------+

Prerequisites

To be able to add and configure a new bridge, you need to have the following informations present:

  • The name of the bridge, such as vmbr4 (vmbr1, vmbr3 and vmbr3 are reserved for existing networks)
  • VLAN ID, such as 141
  • IP prefix, such as 192.168.141.0/24
  • Default gateway address (if any), such as 192.168.141.1
  • IP address for the DHCP server on the VM node, such as 192.168.141.13
  • Broadcast address, such as 192.168.141.255

Make sure, that you have configured the new VLAN on all involved switches and routers beforehand.

Configuration

Add the new VLAN on top of the bonding interface

Login to the first VM node and manually add the new VLAN to the existing bonding interface (bond0). In the following example VLAN 141 is added: 

ip link add link bond0 name vlan141 type vlan id 141
ip link set vlan141 up
ip link show vlan141
71: vlan141@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT 
    link/ether 38:60:77:9c:98:40 brd ff:ff:ff:ff:ff:ff

In order to bring the newly configured VLAN up after a system reboot, the network configuration needs to be extended. Edit /etc/conf.d/net and add the following entries: 

$EDITOR /etc/conf.d/net

# [...]
 
#-----------------------------------------------------------------------------
# VLAN (802.1q support)
 
### Add the new vlan to the bond device
vlans_bond0="130 120 110 140 141"
 
# pub VLAN
vlan140_name="vlan140"
config_vlan140="null"
 
### Add your new vlan at the end of the VLAN section
# pub2 VLAN
vlan141_name="vlan141"
config_vlan141="null"
 
# [...]

Create a new bridging device

As the new VLAN interface is now up, an additional bridge can be added.

Again the network configuration needs to be edited. Append the configuration of the new bridge, below the existing vmbr0 bridge configuration. In the following example the bridge vmbr4 is added on top of VLAN 141 with a prefix of 192.168.141.13/24: 

$EDITOR /etc/conf.d/net

# [...]
 
#-----------------------------------------------------------------------------
# Bridging (802.1d) interfaces
 
bridge_vmbr0="vlan140"
bridge_add_vlan140="vmbr0"
config_vmbr0="192.168.140.13/24 brd 192.168.140.255"    # 192.168.140.13 is the ip address of the vm-node.
routes_vmbr0="default via 192.168.140.1"
 
### Add your new bridge at the end of the bridge section
bridge_vmbr4="vlan141"
bridge_add_vlan141="vmbr4"
config_vmbr4="192.168.141.13/24 brd 192.168.141.255"    # 192.168.141.13 is the ip address of the vm-node (only required on primary- and secondary master node for the dhcp server, on all others use: null)
rc_net_vmbr4_provide="!net"                             # Otherwise stopping or restarting the bridge, will bring all net-services down.

To bring the bridge up, an init script is required. Simply symlink the net.lo script to the new bridge name's script, here net.vmbr4 is assumed: 

ln -s /etc/init.d/net.lo /etc/init.d/net.vmbr4
rc-update add net.vmbr4 default
/etc/init.d/net.vmbr4 start
 * Caching service dependencies ...                                       [ ok ]
 * Bringing up interface vmbr4
 *   Creating bridge vmbr4 ...
 *   Adding ports to vmbr4
 *     vlan141 ...                                                        [ ok ]
 *   192.168.141.13/24 ...                                                [ ok ]

Check if everything looks fine: 

ip addr show vmbr4 && brctl show
82: vmbr4: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 38:60:77:9c:98:40 brd ff:ff:ff:ff:ff:ff
    inet 192.168.141.13/24 brd 192.168.141.255 scope global vmbr4
       valid_lft forever preferred_lft forever
bridge name     bridge id               STP enabled     interfaces
vmbr0           8000.3860779c9840       no              kvm-0217_0
                                                        vlan140
vmbr4           8000.3860779c9840       no              kvm-0227_0
                                                        vlan141


Or if you would like to copy&paste: 

vlanId=141
bridgeName=vmbr4
nodeIp=192.168.141.13
prefixLength=24
broadcast=192.168.141.255
 
cat << EOF >> /etc/conf.d/net
 
bridge_${bridgeName}="vlan${vlanId}"
bridge_add_vlan${vlanId}="${bridgeName}"
config_${bridgeName}="${nodeIp}/${prefixLength} brd ${broadcast}"
rc_net_${bridgeName}_provide="!net"
EOF
 
ln -s /etc/init.d/net.lo /etc/init.d/net.${bridgeName}
rc-update add net.${bridgeName} default
/etc/init.d/net.${bridgeName} start
ip addr show ${bridgeName} && brctl show

Delete a bridging device

Tear the bridge down but check if it would also stop all network services: 

# Set your bridge name
bridgeName=<NAME OF YOUR BRIDGE>   # e.g.: vmbr4
 
# Check if it excludes the net services
grep rc_net_${bridgeName}_provide /etc/conf.d/net

If grep produces output you might consider running the initscript with the -D options (otherwise network services, like sshd, might get stopped).

Otherwise just stop the bridge and remove the initscript: 

/etc/init.d/net.${bridgeName} stop
 
rc-update del net.${bridgeName} default
 
rm /etc/init.d/net.${bridgeName}

Remove the bridge from the network configuration: 

$EDITOR /etc/conf.d/net

# [...]
 
### Remove the options for the bridge you want to delete
#bridge_vmbr4="vlan141"
#bridge_add_vlan141="vmbr4"
#config_vmbr4="192.168.141.13/24 brd 192.168.141.255"    # 192.168.141.13 is the ip address of the vm-node.
#rc_net_vmbr4_provide="!net"                             # Otherwise stopping or restarting the bridge, will bring all net-services down.
 
# [...]

Remove a VLAN on a bonding interface

Delete the VLAN from the interface bond0: 

# Set your vlan id
vlanId=<VLAN ID>   # e.g. 141
 
# Shut the vlan down
ip link set vlan${vlanId} down
 
# Remove it from the interface bond0
ip link delete vlan${vlanId} type vlan

Edit /etc/conf.d/net and remove your VLAN: 

$EDITOR /etc/conf.d/net

# [...]
 
### Remove your vlan options
# pub2 VLAN
# vlan141_name="vlan141"
# config_vlan141="null"
 
# [...]

Links

Retrieved from "https://wiki.stoney-cloud.org/mediawiki/index.php?title=Manually_adding_a_new_bridge&oldid=3143"