Xen and the Art of Consolidation Tom Eastep 2006-05-08 2006 Thomas M. Eastep Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover, and with no Back-Cover Texts. A copy of the license is included in the section entitled GNU Free Documentation License. This article applies to Shorewall 3.0 and later. If you are running a version of Shorewall earlier than Shorewall 3.0.0 then please see the documentation for that release.
Before Xen Prior to adopting Xen, I had a home office crowded with 5 systems, three monitors a scanner and a printer. The systems were: Firewall Public Server in a DMZ (mail) Private Server (wookie) My personal Linux Desktop (ursa) My work system (docked laptop running Windows XP). The result was a very crowded and noisy room.
After Xen Xen has allowed me to reduce the noise and clutter considerably. I now have three systems with two monitors. I've also replaced the individual printer and scanner with a Multifunction FAX/Scanner/Printer. The systems now include: Combination Firewall/Public Server/Private Server/Wireless Gateway using Xen (created by building out my Linux desktop system). My work system. My Linux desktop (wookie, which is actually the old public server box) All of the Linux systems run SuSE 10.0 or SuSE 10.1 Beta. If you are unfamiliar with Xen networking, I recommend that you read the first section of the companion Xen and Shorewall article. Here is a high-level diagram of our network. As shown in this diagram, the Xen system has three physical network interfaces. These are: eth0 -- conntected to the switch in my office. That switch is cabled to a second switch in my wife's office where my wife has her desktop and networked printer (I sure wish that there had been wireless back when I strung that CAT-5 cable halfway across the house). eth1 -- connected to our DSL "Modem". eth2 -- connected to a Wireless Access Point (WAP) that interfaces to our wireless network. There are four Xen domains. Dom0 (DNS name ursa.shorewall.net) is used as a local file server (NFS and Samba). The first DomU (Dom name firewall, DNS name gateway.shorewall.net) is used as our main firewall. The second DomU (Dom name lists, DNS name lists.shorewall.net) is used as a public Web/FTP/Mail/DNS server. The third DomU (Dom name wireless, DNS name wireless.shorewall.net) is used as a gateway to our wireless network. A seperate wireless gateway is necessary because Xen 3.0 only supports three virtual interfaces per DomU and the firewall DomU already has three interfaces. Shorewall runs in Dom0, in the firewall domain and in the wireless gateway. As the developer of Shorewall, I have enough experience to be very comfortable with Linux networking and Shorewall/iptables. I arrived at this configuration after a lot of trial and error experimentation (see Xen and Shorewall). If you are a Linux networking novice, I recommend that you do not attempt a configuration like this one for your first Shorewall installation. You are very likely to frustrate both yourself and the Shorewall support team. Rather I suggest that you start with something simple like a standalone installation in a domU; once you are comfortable with that then you will be ready to try something more substantial. As Paul Gear says: Shorewall might make iptables easy, but it doesn't make understanding fundamental networking principles, traffic shaping, or multi-ISP routing any easier. The same goes for Xen networking.
Domain Configuration Below are the relevant configuration files for the four domains. The "loopback.nloopbacks=..." entries are used to restrict the number of "vethn" devices that the Xen kernel creates. I use partitions on my hard drives for DomU storage devices.
/boot/grub/menu.lst — here is the entry that boots Xen in Dom0. title XEN root (hd0,1) kernel /boot/xen.gz dom0_mem=458752 sched=bvt module /boot/vmlinuz-xen root=/dev/hda2 vga=0x31a selinux=0 resume=/dev/hda1 splash=silent showopts loopback.nloopbacks=1 module /boot/initrd-xen /etc/xen/auto/01-firewall — configuration file for the firewall domain # -*- mode: python; -*- # configuration name: name = "firewall" # usable ram: memory = 256 # kernel and initrd: kernel = "/boot/vmlinuz-xen" ramdisk = "/boot/initrd-xen" # boot device: root = "/dev/hdb2" # boot to run level: extra = "loopback.nloopbacks=0 3" # network interface: vif = [ 'mac=aa:cc:00:00:00:02, bridge=xenbr0', 'mac=aa:cc:00:00:00:03, bridge=xenbr1' , 'mac=00:a0:cc:63:66:89, bridge=xenbr2' ] hostname = name # storage devices: disk = [ 'phy:hdb2,hdb2,w' ] /etc/xen/auto/02-lists — configuration file for the lists domain # -*- mode: python; -*- # configuration name: name = "lists" # usable ram: memory = 512 # kernel and initrd: kernel = "/boot/vmlinuz-xen" ramdisk = "/boot/initrd-xen" # boot device: root = "/dev/hda3" # boot to run level: extra = "loopback.nloopbacks=0 3" # network interface: vif = [ 'mac=aa:cc:00:00:00:01, bridge=xenbr1' ] hostname = name # storage devices: disk = [ 'phy:hda3,hda3,w' ] /etc/xen/auto/03-wireless — configuration file for the wireless domain. # -*- mode: python; -*- # configuration name: name = "wireless" # usable ram: memory = 256 # kernel and initrd: kernel = "/boot/vmlinuz-xen" ramdisk = "/boot/initrd-xen" # boot device: root = "/dev/hdb4" # boot to run level: extra = "loopback.nloopbacks=0 3" # network interface: vif = [ 'mac=aa:cc:00:00:00:04, bridge=xenbr0', 'mac=00:a0:cc:d1:db:12, bridge=xenbr3' ] dhcp = 'dhcp' hostname = name # storage devices: disk = [ 'phy:hdb4,hdb4,w' ]
With all four Xen domains up and running, the system looks as shown in the following diagram. The zones correspond to the Shorewall zones in the Dom0 configuration. Under some circumstances, UDP and/or TCP communication from a domU won't work for no obvious reason. That happened with the lists domain in my setup. Looking at the IP traffic with tcpdump -nvvi eth1 in the firewall domU showed that UDP packets from the lists domU had incorrect checksums. That problem was corrected by arranging for the following command to be executed in the lists domain when its eth0 device was brought up: ethtool -K eth0 tx off SuSE 10.0 includes Xen 3.0 which does not support PCI delegation PCI delegation was a feature of Xen 2.0 but that capability was dropped in 3.0. It has been restored in Xen 3.0.2 and once I upgrade this system to SuSE 10.1 (which includes Xen 3.0.2), I intend to implement PCI delegation and remove three of the four bridges. I will probably combine the wireless and firewall domains at that time as well. ; I therefore use a bridged configuration with four bridges (one for each network interface). When Shorewall starts during bootup of Dom0, it creates the four bridges using this /etc/shorewall/init extension script:
for bridge in xenbr0 xenbr1 xenbr2 xenbr3; do if [ -z "$(/sbin/brctl show 2> /dev/null | fgrep $bridge)" ]; then /sbin/brctl addbr $bridge case $bridge in xenbr2) mac=`ip link show eth1 | grep 'link\/ether' | sed -e 's/.*ether \(..:..:..:..:..:..\).*/\1/'` [ "$mac" = "fe:ff:ff:ff:ff:ff" ] || /sbin/ip link set dev eth1 addr fe:ff:ff:ff:ff:ff /sbin/ip link set dev eth1 up /sbin/brctl addif xenbr2 eth1 ;; xenbr3) mac=`ip link show eth2 | grep 'link\/ether' | sed -e 's/.*ether \(..:..:..:..:..:..\).*/\1/'` [ "$mac" = "fe:ff:ff:ff:ff:ff" ] || /sbin/ip link set dev eth2 addr fe:ff:ff:ff:ff:ff /sbin/ip link set dev eth2 up /sbin/brctl addif xenbr3 eth2 ;; esac /sbin/ip link set dev $bridge up fi done
Dom0 Configuration The goals for the Shorewall configuration in Dom0 are as follows: Allow traffic to flow unrestricted through the four bridges. This is done by configuring the hosts connected to each bridge as a separate zone and relying on Shorewall's implicit intra-zone ACCEPT policy to permit traffic through the bridge. Ensure that there is no stray traffic between the zones. This is a "belt+suspenders" measure since there should be no routing between the bridges (because they don't have IP addresses). The configuration is a simple one:
/etc/shorewall/zones: #ZONE TYPE OPTIONS IN OUT # OPTIONS OPTIONS fw firewall Wifi ipv4 loc ipv4 dmz ipv4 net ipv4 #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE /etc/shorewall/policy (Note the unusual use of an ACCEPT all->all policy): #SOURCE DEST POLICY LOG LIMIT:BURST # LEVEL Wifi all REJECT info all Wifi REJECT info dmz all REJECT info all dmz REJECT info net all REJECT info all net REJECT info all all ACCEPT #LAST LINE -- DO NOT REMOVE /etc/shorewall/interfaces: #ZONE INTERFACE BROADCAST OPTIONS Wifi xenbr3 - routeback loc xenbr0 192.168.1.255 dhcp,routeback dmz xenbr1 - routeback net xenbr2 - routeback #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/rules: #ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ # PORT PORT(S) DEST LIMIT GROUP #SECTION ESTABLISHED #SECTION RELATED SECTION NEW #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE
Firewall DomU Configuration In the firewall DomU, I run a conventional three-interface firewall with Proxy ARP DMZ -- it is very similar to the firewall described in the Shorewall Setup Guide. The firewall runs a routed OpenVPN server to provide roadwarrior access for our two laptops. Here is the firewall's view of the network: The two laptops can be directly attached to the LAN as shown above or they can be attached wirelessly through the wireless gateway -- their IP addresses are the same in either case; when they are directly attached, the IP address is assigned by the DHCP server running on the firewall and when they are attached wirelessly, the IP address is assigned by OpenVPN. The Shorewall configuration files are shown below. All routing and secondary IP addresses are handled in the SuSE network configuration.
/etc/shorewall/shorewall.conf: STARTUP_ENABLED=Yes VERBOSITY=0 LOGFILE=/var/log/firewall LOGFORMAT="Shorewall:%s:%s:" LOGTAGONLY=No LOGRATE= LOGBURST= LOGALLNEW= BLACKLIST_LOGLEVEL= MACLIST_LOG_LEVEL=$LOG TCP_FLAGS_LOG_LEVEL=$LOG RFC1918_LOG_LEVEL=$LOG SMURF_LOG_LEVEL=$LOG LOG_MARTIANS=No IPTABLES=/usr/sbin/iptables PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin SHOREWALL_SHELL=/bin/bash SUBSYSLOCK= MODULESDIR= CONFIG_PATH=/etc/shorewall:/usr/share/shorewall RESTOREFILE=standard IPSECFILE=zones IP_FORWARDING=On ADD_IP_ALIASES=No ADD_SNAT_ALIASES=No RETAIN_ALIASES=No TC_ENABLED=Internal CLEAR_TC=Yes MARK_IN_FORWARD_CHAIN=Yes CLAMPMSS=Yes ROUTE_FILTER=No DETECT_DNAT_IPADDRS=Yes MUTEX_TIMEOUT=60 ADMINISABSENTMINDED=Yes BLACKLISTNEWONLY=Yes DELAYBLACKLISTLOAD=No MODULE_SUFFIX= DISABLE_IPV6=Yes BRIDGING=No DYNAMIC_ZONES=No PKTTYPE=No RFC1918_STRICT=Yes MACLIST_TTL=60 SAVE_IPSETS=No MAPOLDACTIONS=No FASTACCEPT=Yes BLACKLIST_DISPOSITION=DROP MACLIST_TABLE=mangle MACLIST_DISPOSITION=DROP TCP_FLAGS_DISPOSITION=DROP /etc/shorewall/zones: #ZONE TYPE OPTIONS IN OUT # OPTIONS OPTIONS fw firewall net ipv4 #Internet loc ipv4 #Local wired Zone dmz:loc ipv4 #DMZ -- server running in virtual machine at 206.124.146.177 vpn ipv4 #Open VPN clients #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE /etc/shorewall/policy: #SOURCE DEST POLICY LOG LIMIT:BURST # LEVEL $FW $FW ACCEPT $FW net ACCEPT loc net ACCEPT $FW vpn ACCEPT vpn net ACCEPT vpn loc ACCEPT loc vpn ACCEPT $FW loc ACCEPT loc $FW REJECT $LOG net all DROP $LOG 10/sec:40 all all REJECT $LOG #LAST LINE -- DO NOT REMOVE /etc/shorewall/params (edited): MIRRORS=<comma-separated list of Shorewall mirrors> NTPSERVERS=<comma-separated list of NTP servers I sync with> POPSERVERS=<comma-separated list of server IP addresses> LOG=info INT_IF=eth0 DMZ_IF=eth1 EXT_IF=eth3 OMAK=<IP address at our second home> #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE /etc/shorewall/init: echo 1 > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_be_liberal /etc/shorewall/interfaces: #ZONE INTERFACE BROADCAST OPTIONS net $EXT_IF 206.124.146.255 dhcp,norfc1918,logmartians,blacklist,tcpflags,nosmurfs dmz $DMZ_IF 192.168.0.255 logmartians loc $INT_IF 192.168.1.255 dhcp,routeback,logmartians vpn tun+ - #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/nat: #EXTERNAL INTERFACE INTERNAL ALL LOCAL # INTERFACES 206.124.146.178 $EXT_IF 192.168.1.3 No No 206.124.146.180 $EXT_IF 192.168.1.6 No No #LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE /etc/shorewall/masq (Note the cute trick here and in the proxyarp file that follows that allows me to access the DSL "Modem" using it's default IP address (192.168.1.1)): #INTERFACE SUBNET ADDRESS PROTO PORT(S) IPSEC +$EXT_IF:192.168.1.1 0.0.0.0/0 192.168.1.254 $EXT_IF 192.168.0.0/22 206.124.146.179 #LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE /etc/shorewall/proxyarp: #ADDRESS INTERFACE EXTERNAL HAVEROUTE PERSISTENT 192.168.1.1 $EXT_IF $INT_IF yes 206.124.146.177 $DMZ_IF $EXT_IF yes #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/tunnels: #TYPE ZONE GATEWAY GATEWAY # ZONE openvpnserver:udp net 0.0.0.0/0 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/actions: #ACTION Mirrors # Accept traffic from Shorewall Mirrors #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE /etc/shorewall/action.Mirrors: #TARGET SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE # PORT PORT(S) DEST LIMIT ACCEPT $MIRRORS #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/rules: SECTION NEW ############################################################################################################################################################################### #ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ # PORT PORT(S) DEST LIMIT GROUP ############################################################################################################################################################################### REJECT:$LOG loc net tcp 25 REJECT:$LOG loc net udp 1025:1031 # # Stop NETBIOS crap # REJECT loc net tcp 137,445 REJECT loc net udp 137:139 # # Stop my idiotic work laptop from sending to the net with an HP source/dest IP address # DROP loc:!192.168.0.0/22 net ############################################################################################################################################################################### # Local Network to Firewall # DROP loc:!192.168.0.0/22 fw # Silently drop traffic with an HP source IP from my XP box ACCEPT loc fw tcp 22 ACCEPT loc fw tcp time,631,8080 ACCEPT loc fw udp 161,ntp,631 ACCEPT loc:192.168.1.5 fw udp 111 DROP loc fw tcp 3185 #SuSE Meta pppd Ping/ACCEPT loc fw REDIRECT loc 3128 tcp 80 - !206.124.146.177 ############################################################################################################################################################################### # Road Warriors to Firewall # ACCEPT vpn fw tcp ssh,time,631,8080 ACCEPT vpn fw udp 161,ntp,631 Ping/ACCEPT vpn fw ############################################################################################################################################################################### # Road Warriors to DMZ # ACCEPT vpn dmz udp domain ACCEPT vpn dmz tcp www,smtp,smtps,domain,ssh,imap,https,imaps,ftp,10023,pop3 - Ping/ACCEPT vpn dmz ############################################################################################################################################################################### # Local network to DMZ # ACCEPT loc dmz udp domain ACCEPT loc dmz tcp ssh,smtps,www,ftp,imaps,domain,https - ACCEPT loc dmz tcp smtp Trcrt/ACCEPT loc dmz ############################################################################################################################################################################### # Internet to ALL -- drop NewNotSyn packets # dropNotSyn net fw tcp dropNotSyn net loc tcp dropNotSyn net dmz tcp ############################################################################################################################################################################### # Internet to DMZ # ACCEPT net dmz udp domain LOG:$LOG net:64.126.128.0/18 dmz tcp smtp ACCEPT net dmz tcp smtps,www,ftp,imaps,domain,https - ACCEPT net dmz tcp smtp - 206.124.146.177,206.124.146.178 ACCEPT net dmz udp 33434:33454 Mirrors net dmz tcp rsync Limit:$LOG:SSHA,3,60\ net dmz tcp 22 Trcrt/ACCEPT net dmz ############################################################################################################################################################################## # # Net to Local # # When I'm "on the road", the following two rules allow me VPN access back home using PPTP. # DNAT net loc:192.168.1.4 tcp 1729 DNAT net loc:192.168.1.4 gre # # Roadwarrior access to Wookie # ACCEPT net:$OMAK loc tcp 22 Limit:$LOG:SSHA,3,60\ net loc tcp 22 # # ICQ # ACCEPT net loc:192.168.1.3 tcp 113,4000:4100 # # Bittorrent # ACCEPT net loc:192.168.1.3 tcp 6881:6889,6969 ACCEPT net loc:192.168.1.3 udp 6881:6889,6969 # # Real Audio # ACCEPT net loc:192.168.1.3 udp 6970:7170 # # Skype # ACCEPT net loc:192.168.1.6 tcp 1194 # # Traceroute # Trcrt/ACCEPT net loc:192.168.1.3 # # Silently Handle common probes # REJECT net loc tcp www,ftp,https DROP net loc icmp 8 ############################################################################################################################################################################### # DMZ to Internet # ACCEPT dmz net udp domain,ntp ACCEPT dmz net tcp echo,ftp,ssh,smtp,whois,domain,www,81,https,cvspserver,2702,2703,8080 ACCEPT dmz net:$POPSERVERS tcp pop3 Ping/ACCEPT dmz net # # Some FTP clients seem prone to sending the PORT command split over two packets. This prevents the FTP connection tracking # code from processing the command and setting up the proper expectation. The following rule allows active FTP to work in these cases # but logs the connection so I can keep an eye on this potential security hole. # ACCEPT:$LOG dmz net tcp 1024: 20 ############################################################################################################################################################################### # Local to DMZ # ACCEPT loc dmz udp domain,xdmcp ACCEPT loc dmz tcp www,smtp,smtps,domain,ssh,imap,rsync,https,imaps,ftp,10023,pop3,3128 Trcrt/ACCEPT loc dmz ############################################################################################################################################################################### # DMZ to Local # ACCEPT dmz loc:192.168.1.5 udp 123 ACCEPT dmz loc:192.168.1.5 tcp 21 Ping/ACCEPT dmz loc ############################################################################################################################################################################### # DMZ to Firewall -- ntp & snmp, Silently reject Auth # ACCEPT dmz fw tcp 161,ssh ACCEPT dmz fw udp 161 REJECT dmz fw tcp auth Ping/ACCEPT dmz fw ############################################################################################################################################################################### # Internet to Firewall # REJECT net fw tcp www,ftp,https DROP net fw icmp 8 ACCEPT net fw udp 33434:33454 ACCEPT net:$OMAK fw udp ntp ACCEPT net fw tcp auth ACCEPT net:$OMAK fw tcp 22 Limit:$LOG:SSHA,3,60\ net fw tcp 22 Trcrt/ACCEPT net fw ############################################################################################################################################################################### # Firewall to DMZ # ACCEPT fw dmz tcp domain,www,ftp,ssh,smtp,https,993,465 ACCEPT fw dmz udp domain REJECT fw dmz udp 137:139 Ping/ACCEPT fw dmz ############################################################################################################################################################################## # Avoid logging Freenode.net probes # DROP net:82.96.96.3 all #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/tcdevices #INTERFACE IN-BANDWITH OUT-BANDWIDTH $EXT_IF 1.3mbit 384kbit #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/tcclasses#INTERFACE MARK RATE CEIL PRIORITY OPTIONS $EXT_IF 10 full full 1 tcp-ack,tos-minimize-delay $EXT_IF 20 9*full/10 9*full/10 2 default $EXT_IF 30 6*full/10 6*full/10 3 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/tcrules#MARK SOURCE DEST PROTO PORT(S) CLIENT USER TEST # PORT(S) 1:110 192.168.0.0/22 $EXT_IF #Our internel nets get priority #over the server 1:130 206.124.146.177 $EXT_IF tcp - 873 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE
Wireless Gateway DomU Configuration The Shorewall configuration in the 'wireless' DomU is very simple. It's sole purpose is to protect the local network from the Wireless net by restricting wireless access to clients that have established an OpenVPN Bridged connection. This configuration illustrates that you can use any system on your internal LAN as a wireless gateway -- it doesn't have to be your main firewall (and it doesn't have to run in a Xen domain either and it doesn't even have to run Linux). Our wireless gateway runs a DHCP server that assigns wireless hosts an IP address in 192.168.3.0/24 -- The OpenVPN server running on the gateway assigns its clients an IP address in 192.168.1.0/24 so, thanks to bridging, these clients appear to be physically attached to the LAN). That allows our two laptops to have the same IP address in 192.168.1.0/24 regardless of whether they are connected to the LAN directly or are connected via the wireless network. The 'tap0' device used by OpenVPN is bridged to eth2 using this SuSE-specific startup script (see the Shorewall Bridge Documentation for scripts that work on other distributions):
/etc/init.d/bridge: #!/bin/sh # # The Shoreline Firewall (Shorewall) Packet Filtering Firewall - V3.0 # # This program is under GPL [http://www.gnu.org/copyleft/gpl.htm] # # (c) 1999,2000,2001,2002,2003,2004,2005 - Tom Eastep (teastep@shorewall.net) # # On most distributions, this file should be called /etc/init.d/shorewall. # # Complete documentation is available at http://shorewall.net # # This program is free software; you can redistribute it and/or modify # it under the terms of Version 2 of the GNU General Public License # as published by the Free Software Foundation. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA # # If an error occurs while starting or restarting the firewall, the # firewall is automatically stopped. # # Commands are: # # bridge start Starts the bridge # bridge restart Restarts the bridge # bridge reload Restarts the bridge # bridge stop Stops the bridge # bridge status Displays bridge status # # chkconfig: 2345 4 99 # description: Packet filtering firewall ### BEGIN INIT INFO # Provides: bridge # Required-Start: boot.udev # Required-Stop: # Default-Start: 2 3 5 # Default-Stop: 0 1 6 # Description: starts and stops the bridge ### END INIT INFO ################################################################################ # Interfaces to be bridged -- may be listed by device name or by MAC address # INTERFACES="eth2" # # Tap Devices # TAPS="tap0" ################################################################################ # Give Usage Information # ################################################################################ usage() { echo "Usage: $0 start|stop|reload|restart|status" exit 1 } ################################################################################# # Find the interface with the passed MAC address ################################################################################# find_interface_by_mac() { local mac=$1 first second rest dev /sbin/ip link ls | while read first second rest; do case $first in *:) dev=$second ;; *) if [ "$second" = $mac ]; then echo ${dev%:} return fi esac done } ################################################################################ # Convert MAC addresses to interface names ################################################################################ get_interfaces() { local interfaces= interface for interface in $INTERFACES; do case $interface in *:*:*) interface=$(find_interface_by_mac $interface) [ -n "$interface" ] || echo "WARNING: Can't find an interface with MAC address $mac" ;; esac interfaces="$interfaces $interface" done INTERFACES="$interfaces" } ################################################################################ # Configure the Bridge -- IP configuration is left to the SuSE network scripts ################################################################################ do_start() { local interface get_interfaces for interface in $TAPS; do /usr/sbin/openvpn --mktun --dev $interface done /sbin/brctl addbr br0 for interface in $INTERFACES $TAPS; do /sbin/ip link set $interface up /sbin/brctl addif br0 $interface done } ################################################################################ # Stop the Bridge ################################################################################ do_stop() { local interface get_interfaces for interface in $INTERFACES $TAPS; do /sbin/brctl delif br0 $interface /sbin/ip link set $interface down done /sbin/ip link set br0 down /sbin/brctl delbr br0 for interface in $TAPS; do /usr/sbin/openvpn --rmtun --dev $interface done } ################################################################################ # E X E C U T I O N B E G I N S H E R E # ################################################################################ command="$1" case "$command" in start) do_start ;; stop) do_stop ;; restart|reload) do_stop do_start ;; status) /sbin/brctl show ;; *) usage ;; esac As mentioned in the comments in the above script, IP configuration of the bridge is done through the normal SuSE network scripts. Here's the config file: BOOTPROTO='static' BROADCAST='' IPADDR='192.168.1.7' MTU='' NAME='Ethernet Network Card' NETMASK='255.255.255.0' NETWORK='' REMOTE_IPADDR='' STARTMODE='auto' USERCONTROL='no' _nm_name='br0' Since there is no requirement to restrict connections through the bridge, I set BRIDGING=No in /etc/shorewall/shorewall.conf. /etc/shorewall/zones: #ZONE TYPE OPTIONS IN OUT # OPTIONS OPTIONS fw firewall Wifi ipv4 loc ipv4 #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE /etc/shorewall/interfaces: #ZONE INTERFACE BROADCAST OPTIONS Wifi eth4 192.168.3.255 dhcp,maclist loc br0 192.168.1.255 dhcp,routeback #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/policy (again, note the use of an ACCEPT all->all policy): #SOURCE DEST POLICY LOG LIMIT:BURST # LEVEL Wifi all REJECT info all all ACCEPT #LAST LINE -- DO NOT REMOVE /etc/shorewall/tunnels: #TYPE ZONE GATEWAY GATEWAY # ZONE openvpnserver Wifi 192.168.3.0/24 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/maclist (Note that this system runs Shorewall 3.2 so there is an additional 'TARGET' column): #TARGET INTERFACE MAC IP ADDRESSES (Optional) ACCEPT eth4 00:04:5a:0e:85:b9 #WAP11 ACCEPT eth4 00:06:25:45:33:3c #WET11 ACCEPT eth4 00:0b:cd:53:cc:97 192.168.3.8 #TIPPER ACCEPT eth4 00:0f:66:ef:b6:f6 192.168.3.8 #TIPPER1 ACCEPT eth4 00:12:79:3d:fe:2e 192.168.3.6 #Work Laptop ACCEPT eth4 - 192.168.3.254 #Broadcast/Multicast from us DROP:info eth4 - 192.168.3.0/24 DROP:info eth4 - 169.254.0.0/16 #Stop autoconfigured hosts. #LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE The routing table on the wireless gateway is as follows:
192.168.3.0/24 dev eth4 proto kernel scope link src 192.168.3.254 192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.7 169.254.0.0/16 dev eth4 scope link 127.0.0.0/8 dev lo scope link default via 192.168.1.254 dev br0
The route to 169.254.0.0/16 is automatically generated by the SuSE network scripts so I include that network in the /etc/shorewall/maclist file for completeness. /etc/shorewall/rules: #ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ # PORT PORT(S) DEST LIMIT GROUP #SECTION ESTABLISHED #SECTION RELATED SECTION NEW ACCEPT Wifi loc:192.168.1.5 udp 123 #Allow NTP before OpenVPN is up. #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE
Here is the configuration of OpenVPN on this system:
/etc/openvpn/server-bridge (Note that I prefer to push two /1 routes rather than to use the redirect-gateway directive on the client systems; I find that redirect-gateway occasionally leaves a client system with no default gateway): dev tap0 local 192.168.3.254 server-bridge 192.168.1.7 255.255.255.0 192.168.1.64 192.168.1.71 client-to-client dh dh1024.pem ca /etc/certs/cacert.pem crl-verify /etc/certs/crl.pem cert /etc/certs/wireless.pem key /etc/certs/wireless_key.pem port 1194 comp-lzo user nobody group nogroup keepalive 15 45 ping-timer-rem persist-tun persist-key client-config-dir /etc/openvpn/bridge-clients ccd-exclusive verb 3 push "route 0.0.0.0 128.0.0.0 192.168.1.254" push "route 128.0.0.0 128.0.0.0 192.168.1.254" /etc/bridge-clients/tipper.shorewall.net (used to assign a fixed IP address to clients -- there are other similar files in this directory): ifconfig-push 192.168.1.8 255.255.255.0
Finally, here's a copy of the OpenVPN configuration file on tipper, one of the laptops:
dev tap remote 192.168.3.254 tls-remote wireless.shorewall.net client tls-client explicit-exit-notify ca /etc/certs/cacert.pem cert /etc/certs/tipper.pem key /etc/certs/tipper_key.pem port 1194 comp-lzo ping 15 ping-restart 45 ping-timer-rem persist-tun persist-key mute-replay-warnings verb 3