shorewall_code/docs/XenMyWay.xml
2006-03-10 23:15:16 +00:00

679 lines
28 KiB
XML

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<article>
<!--$Id$-->
<articleinfo>
<title>Xen and the Art of Consolidation</title>
<authorgroup>
<author>
<firstname>Tom</firstname>
<surname>Eastep</surname>
</author>
</authorgroup>
<pubdate>2006-03-10</pubdate>
<copyright>
<year>2006</year>
<holder>Thomas M. Eastep</holder>
</copyright>
<legalnotice>
<para>Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version
1.2 or any later version published by the Free Software Foundation; with
no Invariant Sections, with no Front-Cover, and with no Back-Cover
Texts. A copy of the license is included in the section entitled
<quote><ulink url="GnuCopyright.htm">GNU Free Documentation
License</ulink></quote>.</para>
</legalnotice>
</articleinfo>
<section>
<title>Before Xen</title>
<para>Prior to adopting <ulink url="Xen.html">Xen</ulink>, I had a home
office crowded with 5 systems, three monitors a scanner and a printer. The
systems were:</para>
<orderedlist>
<listitem>
<para>Firewall</para>
</listitem>
<listitem>
<para>Public Server in a DMZ (mail)</para>
</listitem>
<listitem>
<para>Private Server (wookie)</para>
</listitem>
<listitem>
<para>My personal Linux Desktop (ursa)</para>
</listitem>
<listitem>
<para>My work system (docked laptop running Windows XP).</para>
</listitem>
</orderedlist>
<para>The result was a very crowded and noisy room.</para>
</section>
<section>
<title>After Xen</title>
<para>Xen has allowed me to reduce the noise and clutter considerably. I
now have three systems with two monitors. I've also replaced the
individual printer and scanner with a Multifunction
FAX/Scanner/Printer.</para>
<para>The systems now include:</para>
<orderedlist>
<listitem>
<para>Combination Firewall/Public Server/Private Server using Xen
(created by building out my Linux desktop system).</para>
</listitem>
<listitem>
<para>My work system.</para>
</listitem>
<listitem>
<para>My Linux desktop (wookie, which is actually the old public
server box)</para>
</listitem>
</orderedlist>
<para>All of the Linux systems run SuSE 10.0.</para>
<para>If you are unfamiliar with Xen networking, I recommend that you read
the first section of the companion <ulink url="Xen.html">Xen and
Shorewall</ulink> article.</para>
<para>Here is a high-level diagram of our network.</para>
<graphic align="center" fileref="images/Xen5.png" />
<para>As shown in this diagram, the Xen system has three physical network
interfaces. These are:</para>
<itemizedlist>
<listitem>
<para><filename class="devicefile">eth0</filename> -- conntected to
the switch in my office. That switch is cabled to a second switch in
my wife's office where there is my wife's desktop and her networked
printer.</para>
</listitem>
<listitem>
<para><filename class="devicefile">eth1</filename> -- connected to our
DSL "Modem".</para>
</listitem>
<listitem>
<para><filename class="devicefile">eth2</filename> -- connected to a
Wireless Access Point (WAP) that interfaces to our wireless
network.</para>
</listitem>
</itemizedlist>
<para>There are three Xen domains. Dom0 (ursa) is used as a file server.
One DomU (which is usually Domain 1) is used as a firewall and the other
(lists, normally Domain 2) is used as a public Web/FTP/Mail/DNS server.
Because Xen only supports three virtual interfaces per DomU, I also use
ursa as a gateway for our wireless network rather than placing that
function in the firewall DomU (that domain already has three interfaces).
Shorewall runs in both Dom0 and in the firewall domain.</para>
<para>The system has 1.5GB of RAM so I allocate 512MB to each server and
448MB to the firewall (the remaining 64MB is used by Xen).</para>
<para>Below are the relevant configuration files for the three domains.
The "loopback.nloopbacks=..." entries are used to restrict the number of
"veth<emphasis>n</emphasis>" devices that the Xen kernel creates. I use
partitions on my hard drives for DomU storage devices.</para>
<blockquote>
<para><filename>/boot/grub/menu.lst</filename> — here is the entry that
boots Xen in Dom0.</para>
<programlisting>title XEN
root (hd0,1)
kernel /boot/xen.gz dom0_mem=524288 sched=bvt
module /boot/vmlinuz-xen root=/dev/hda2 vga=0x31a selinux=0 resume=/dev/hda1 splash=silent showopts loopback.nloopbacks=1
module /boot/initrd-xen</programlisting>
<para><filename>/etc/xen/auto/01-gateway</filename> — configuration file
for the firewall domain</para>
<programlisting># -*- mode: python; -*-
# configuration name:
name = "gateway"
# usable ram:
memory = 448
# kernel and initrd:
kernel = "/boot/vmlinuz-xen"
ramdisk = "/boot/initrd-xen"
# boot device:
root = "/dev/hdb2"
# boot to run level:
extra = "loopback.nloopbacks=0"
# network interface:
vif = [ 'mac=aa:cc:00:00:00:02, bridge=xenbr0', 'mac=aa:cc:00:00:00:03, bridge=xenbr1' , 'mac=00:a0:cc:63:66:89, bridge=xenbr2' ]
hostname = name
# storage devices:
disk = [ 'phy:hdb2,hdb2,w' ]</programlisting>
<para><filename>/etc/xen/auto/02-server</filename> — configuration file
for the lists domain</para>
<programlisting># -*- mode: python; -*-
# configuration name:
name = "server"
# usable ram:
memory = 512
# kernel and initrd:
kernel = "/boot/vmlinuz-xen"
ramdisk = "/boot/initrd-xen"
# boot device:
root = "/dev/hda3"
# boot to run level:
extra = "loopback.nloopbacks=0"
# network interface:
vif = [ 'mac=aa:cc:00:00:00:01, bridge=xenbr1' ]
hostname = name
# storage devices:
disk = [ 'phy:hda3,hda3,w' ]</programlisting>
</blockquote>
<para>With all three Xen domains up and running, the system looks as shown
in the following diagram.</para>
<graphic align="center" fileref="images/Xen4.png" />
<para>The zones correspond to the Shorewall zones in the Dom0
configuration.</para>
<para>SuSE 10.0 includes Xen 3.0 which does not support PCI delegation; I
therefore use a bridged configuration with three briges (one for each
network interface). When Shorewall starts during boot, it creates the
three bridges and the tap device <filename
class="devicefile">tap0</filename> and adds tap0 to <filename
class="devicefile">xenbr0</filename>. tap0 is used by <ulink
url="OPENVPN.html">OpenVPN</ulink> to secure the Wifi zone.</para>
<para>Here is <filename>/etc/shorewall/init in Dom0</filename>:</para>
<blockquote>
<programlisting>for bridge in xenbr0 xenbr1 xenbr2; do
if [ -z "$(/sbin/brctl show 2&gt; /dev/null | fgrep $bridge)" ]; then
/sbin/brctl addbr $bridge
/sbin/ip link set dev $bridge up
case $bridge in
xenbr2)
mac=`ip link show eth1 | grep 'link\/ether' | sed -e 's/.*ether \(..:..:..:..:..:..\).*/\1/'`
[ "$mac" = "fe:ff:ff:ff:ff:ff" ] || /sbin/ip link set dev eth1 addr fe:ff:ff:ff:ff:ff
/sbin/ip link set dev eth1 up
/sbin/brctl addif xenbr2 eth1
;;
xenbr0)
if ! qt /sbin/ip link ls dev tap0; then
/usr/sbin/openvpn --mktun --dev tap0
/sbin/ip link set dev tap0 up
/sbin/brctl addif xenbr0 tap0
fi
;;
esac
fi
done</programlisting>
</blockquote>
<para>The goals for the Shorewall configuration in Dom0 are as
follows:</para>
<itemizedlist>
<listitem>
<para>Isolate the Wireless Network so that only VPN access to the
local lan is allowed.</para>
</listitem>
<listitem>
<para>Allow traffic to flow unrestricted through the three
switches.</para>
</listitem>
<listitem>
<para>Ensure that there is no stray traffic between the zones.</para>
</listitem>
</itemizedlist>
<para>The configuration is a simple one:</para>
<blockquote>
<para><filename>/etc/shorewall/zones</filename>:</para>
<programlisting>#ZONE TYPE OPTIONS IN OUT
# OPTIONS OPTIONS
fw firewall
Wifi ipv4
loc ipv4
dmz ipv4
net ipv4
#LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE
</programlisting>
<para><filename>/etc/shorewall/policy</filename>:</para>
<programlisting>#SOURCE DEST POLICY LOG LIMIT:BURST
# LEVEL
Wifi all REJECT info
dmz all REJECT info
all dmz REJECT info
net all REJECT info
all net REJECT info
all all ACCEPT
#LAST LINE -- DO NOT REMOVE</programlisting>
<para><filename>/etc/shorewall/interfaces</filename>:</para>
<programlisting>#ZONE INTERFACE BROADCAST OPTIONS
Wifi eth2 192.168.3.255 dhcp,maclist
loc xenbr0 192.168.1.255 dhcp,routeback
dmz xenbr1 - routeback
net xenbr2 - routeback
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
<para><filename>/etc/shorewall/rules</filename>:</para>
<programlisting>#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/
# PORT PORT(S) DEST LIMIT GROUP
#SECTION ESTABLISHED
#SECTION RELATED
SECTION NEW
#############################################################################################################
#
# BS Address rules
#
DROP Wifi net:15.0.0.0/8
DROP Wifi net:16.0.0.0/8
#
# Insecure Wireless to local network
#
ACCEPT Wifi loc udp 500
ACCEPT Wifi loc udp 53
ACCEPT Wifi loc udp 4500
Ping/ACCEPT Wifi loc
#
# Insecure Wireless to firewall
#
ACCEPT Wifi fw udp 123
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
</blockquote>
<para>In the firewall DomU, I run a conventional three-interface firewall
with Proxy ARP DMZ -- it is very similar to the firewall described in the
<ulink url="shorewall_setup_guide.htm">Shorewall Setup Guide</ulink>. The
firewall runs a routed <ulink url="OPENVPN.html">OpenVPN server</ulink> to
provide roadwarrior access for our two laptops. Here is the firewall's
view of the network:</para>
<graphic align="center" fileref="images/network4.png" />
<para>The Shorewall configuration files are shown below. All routing and
secondary IP addresses are handled in the SuSE network
configuration.</para>
<blockquote>
<para>/etc/shorewall/shorewall.conf:</para>
<programlisting>TARTUP_ENABLED=Yes
VERBOSITY=0
LOGFILE=/var/log/firewall
LOGFORMAT="Shorewall:%s:%s:"
LOGTAGONLY=No
LOGRATE=
LOGBURST=
LOGALLNEW=
BLACKLIST_LOGLEVEL=
MACLIST_LOG_LEVEL=$LOG
TCP_FLAGS_LOG_LEVEL=$LOG
RFC1918_LOG_LEVEL=$LOG
SMURF_LOG_LEVEL=$LOG
LOG_MARTIANS=No
IPTABLES=/usr/sbin/iptables
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
SHOREWALL_SHELL=/bin/bash
SUBSYSLOCK=
MODULESDIR=
CONFIG_PATH=/etc/shorewall:/usr/share/shorewall
RESTOREFILE=standard
IPSECFILE=zones
IP_FORWARDING=On
ADD_IP_ALIASES=No
ADD_SNAT_ALIASES=No
RETAIN_ALIASES=No
TC_ENABLED=Internal
CLEAR_TC=Yes
MARK_IN_FORWARD_CHAIN=Yes
CLAMPMSS=Yes
ROUTE_FILTER=No
DETECT_DNAT_IPADDRS=Yes
MUTEX_TIMEOUT=60
ADMINISABSENTMINDED=Yes
BLACKLISTNEWONLY=Yes
DELAYBLACKLISTLOAD=No
MODULE_SUFFIX=
DISABLE_IPV6=Yes
BRIDGING=No
DYNAMIC_ZONES=No
PKTTYPE=No
RFC1918_STRICT=Yes
MACLIST_TTL=60
SAVE_IPSETS=No
MAPOLDACTIONS=No
FASTACCEPT=Yes
BLACKLIST_DISPOSITION=DROP
MACLIST_TABLE=mangle
MACLIST_DISPOSITION=DROP
TCP_FLAGS_DISPOSITION=DROP</programlisting>
<para><filename>/etc/shorewall/zones</filename>:</para>
<programlisting>#ZONE TYPE OPTIONS IN OUT
# OPTIONS OPTIONS
fw firewall
net ipv4 #Internet
loc ipv4 #Local wired Zone
dmz:loc ipv4 #DMZ -- server running in virtual machine at 206.124.146.177
vpn ipv4 #Open VPN clients
#LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE
</programlisting>
<para><filename>/etc/shorewall/policy</filename>:</para>
<programlisting>#SOURCE DEST POLICY LOG LIMIT:BURST
# LEVEL
$FW $FW ACCEPT
$FW net ACCEPT
loc net ACCEPT
$FW vpn ACCEPT
vpn net ACCEPT
vpn loc ACCEPT
loc vpn ACCEPT
$FW loc ACCEPT
loc $FW REJECT $LOG
net all DROP $LOG 10/sec:40
all all REJECT $LOG
#LAST LINE -- DO NOT REMOVE</programlisting>
<para><filename>/etc/shorewall/params (edited)</filename>:</para>
<programlisting>MIRRORS=&lt;comma-separated list of Shorewall mirrors&gt;
NTPSERVERS=&lt;comma-separated list of NTP servers I sync with&gt;
POPSERVERS=&lt;comma-separated list of server IP addresses&gt;
LOG=info
INT_IF=eth0
DMZ_IF=eth1
EXT_IF=eth3
OMAK=&lt;IP address at our second home&gt;
#LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE
</programlisting>
<para><filename>/etc/shorewall/init</filename>:</para>
<programlisting>echo 1 &gt; /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_be_liberal
</programlisting>
<para><filename>/</filename></para>
<para><filename>/etc/shorewall/interfaces</filename>:</para>
<programlisting>#ZONE INTERFACE BROADCAST OPTIONS
net $EXT_IF 206.124.146.255 dhcp,norfc1918,logmartians,blacklist,tcpflags,nosmurfs
dmz $DMZ_IF 192.168.0.255 logmartians
loc $INT_IF 192.168.1.255 dhcp,routeback,logmartians
vpn tun+ -
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
<para><filename>/etc/shorewall/nat</filename>:</para>
<programlisting>#EXTERNAL INTERFACE INTERNAL ALL LOCAL
# INTERFACES
206.124.146.178 $EXT_IF 192.168.1.5 No No
206.124.146.180 $EXT_IF 192.168.1.6 No No
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE</programlisting>
<para><filename>/etc/shorewall/masq</filename>:</para>
<programlisting>#INTERFACE SUBNET ADDRESS PROTO PORT(S) IPSEC
+$EXT_IF:192.168.1.1 0.0.0.0/0 192.168.1.254
$EXT_IF 192.168.0.0/22 206.124.146.179
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE</programlisting>
<para><filename>/etc/shorewall/proxyarp</filename>:</para>
<programlisting>#ADDRESS INTERFACE EXTERNAL HAVEROUTE PERSISTENT
192.168.1.1 $EXT_IF $INT_IF yes
206.124.146.177 $DMZ_IF $EXT_IF yes
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
<para><filename>/etc/shorewall/tunnels</filename>:</para>
<programlisting>#TYPE ZONE GATEWAY GATEWAY
# ZONE
openvpnserver:udp net 0.0.0.0/0
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
<para><filename>/etc/shorewall/actions</filename>:</para>
<programlisting>#ACTION
Mirrors # Accept traffic from Shorewall Mirrors
#LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE</programlisting>
<para><filename>/etc/shorewall/action.Mirrors</filename>:</para>
<programlisting>#TARGET SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE
# PORT PORT(S) DEST LIMIT
ACCEPT $MIRRORS
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
<para><filename>/etc/shorewall/rules</filename>:</para>
<programlisting>SECTION NEW
###############################################################################################################################################################################
#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/
# PORT PORT(S) DEST LIMIT GROUP
###############################################################################################################################################################################
REJECT:$LOG loc net tcp 25
REJECT:$LOG loc net udp 1025:1031
#
# Stop NETBIOS crap
#
REJECT loc net tcp 137,445
REJECT loc net udp 137:139
#
# Stop my idiotic work laptop from sending to the net with an HP source/dest IP address
#
DROP loc:!192.168.0.0/22 net
###############################################################################################################################################################################
# Local Network to Firewall
#
DROP loc:!192.168.0.0/22 fw # Silently drop traffic with an HP source IP from my XP box
ACCEPT loc fw tcp 22
ACCEPT loc fw tcp time,631,8080
ACCEPT loc fw udp 161,ntp,631
ACCEPT loc:192.168.1.5 fw udp 111
DROP loc fw tcp 3185 #SuSE Meta pppd
Ping/ACCEPT loc fw
REDIRECT loc 3128 tcp 80 - !206.124.146.177
###############################################################################################################################################################################
# Secure wireless to Firewall
#
REDIRECT vpn 3128 tcp 80
###############################################################################################################################################################################
# Road Warriors to Firewall
#
ACCEPT vpn fw tcp ssh,time,631,8080
ACCEPT vpn fw udp 161,ntp,631
Ping/ACCEPT vpn fw
###############################################################################################################################################################################
# Road Warriors to DMZ
#
ACCEPT vpn dmz udp domain
ACCEPT vpn dmz tcp www,smtp,smtps,domain,ssh,imap,https,imaps,ftp,10023,pop3 -
Ping/ACCEPT vpn dmz
###############################################################################################################################################################################
# Local network to DMZ
#
ACCEPT loc dmz udp domain
LOG:$LOG loc:64.126.128.0/18 dmz tcp smtp
ACCEPT loc dmz tcp ssh,smtps,www,ftp,imaps,domain,https -
ACCEPT loc dmz tcp smtp
Trcrt/ACCEPT loc dmz
###############################################################################################################################################################################
# Internet to ALL -- drop NewNotSyn packets
#
dropNotSyn net fw tcp
dropNotSyn net loc tcp
dropNotSyn net dmz tcp
###############################################################################################################################################################################
# Internet to DMZ
#
ACCEPT net dmz udp domain
LOG:$LOG net:64.126.128.0/18 dmz tcp smtp
ACCEPT net dmz tcp smtps,www,ftp,imaps,domain,https -
ACCEPT net dmz tcp smtp - 206.124.146.177,206.124.146.178
ACCEPT net dmz udp 33434:33454
Mirrors net dmz tcp rsync
Limit:$LOG:SSHA,3,60\
net dmz tcp 22
Trcrt/ACCEPT net dmz
##############################################################################################################################################################################
#
# Net to Local
#
# When I'm "on the road", the following two rules allow me VPN access back home using PPTP.
#
DNAT net loc:192.168.1.4 tcp 1729
DNAT net loc:192.168.1.4 gre
#
# Roadwarrior access to Ursa
#
ACCEPT net:$OMAK loc tcp 22
Limit:$LOG:SSHA,3,60\
net loc tcp 22
#
# ICQ
#
ACCEPT net loc:192.168.1.3 tcp 113,4000:4100
#
# Bittorrent
#
ACCEPT net loc:192.168.1.3 tcp 6881:6889,6969
ACCEPT net loc:192.168.1.3 udp 6881:6889,6969
#
# Real Audio
#
ACCEPT net loc:192.168.1.3 udp 6970:7170
# Skype
#
ACCEPT net loc:192.168.1.6 tcp 1194
#
# Traceroute
#
Trcrt/ACCEPT net loc:192.168.1.3
#
# Silently Handle common probes
#
REJECT net loc tcp www,ftp,https
DROP net loc icmp 8
###############################################################################################################################################################################
# DMZ to Internet
#
ACCEPT dmz net udp domain,ntp
ACCEPT dmz net tcp echo,ftp,ssh,smtp,whois,domain,www,81,https,cvspserver,2702,2703,8080
ACCEPT dmz net:$POPSERVERS tcp pop3
Ping/ACCEPT dmz net
#
# Some FTP clients seem prone to sending the PORT command split over two packets. This prevents the FTP connection tracking
# code from processing the command and setting up the proper expectation. The following rule allows active FTP to work in these cases
# but logs the connection so I can keep an eye on this potential security hole.
#
ACCEPT:$LOG dmz net tcp 1024: 20
###############################################################################################################################################################################
# Local to DMZ
#
ACCEPT loc dmz udp domain,xdmcp
ACCEPT loc dmz tcp www,smtp,smtps,domain,ssh,imap,rsync,https,imaps,ftp,10023,pop3,3128
Trcrt/ACCEPT loc dmz
###############################################################################################################################################################################
# DMZ to Local
#
ACCEPT dmz net:192.168.1.254 udp 123
ACCEPT dmz loc:192.168.1.5 tcp 21
Ping/ACCEPT dmz loc
###############################################################################################################################################################################
# DMZ to Firewall -- ntp &amp; snmp, Silently reject Auth
#
ACCEPT dmz fw tcp 161,ssh
ACCEPT dmz fw udp 161
REJECT dmz fw tcp auth
Ping/ACCEPT dmz fw
###############################################################################################################################################################################
# Internet to Firewall
#
REJECT net fw tcp www,ftp,https
DROP net fw icmp 8
ACCEPT net fw udp 33434:33454
ACCEPT net:$OMAK fw udp ntp
ACCEPT net fw tcp auth
ACCEPT net:$OMAK fw tcp 22
Limit:$LOG:SSHA,3,60\
net fw tcp 22
Trcrt/ACCEPT net fw
###############################################################################################################################################################################
# Firewall to DMZ
#
ACCEPT fw dmz tcp domain,www,ftp,ssh,smtp,https,993,465
ACCEPT fw dmz udp domain
REJECT fw dmz udp 137:139
Ping/ACCEPT fw dmz
##############################################################################################################################################################################
# Avoid logging Freenode.net probes
#
DROP net:82.96.96.3 all
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
</blockquote>
</section>
</article>