diff --git a/docs/MultiISP.xml b/docs/MultiISP.xml index 6ed514c4d..507523192 100644 --- a/docs/MultiISP.xml +++ b/docs/MultiISP.xml @@ -776,7 +776,12 @@ DROP:info net:192.168.1.0/24 all
- Example + Legacy Example + + This section describes the legacy method of configuring multiple + uplinks. It is deprecated in favor of the USE_DEFAULT_RT=Yes + configuration described below. The configuration in the figure at the top of this section would be specified in /etc/shorewall/providers as @@ -1276,6 +1281,16 @@ lillycat: # + The configuration in the figure at the top of this section would + be specified in /etc/shorewall/providers as + follows. + + #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY +ISP1 1 1 - eth0 206.124.146.254 track - +ISP2 2 2 - eth1 130.252.99.254 track - + + The remainder of the example is the same. + Although 'balance' is automatically assumed when USE_DEFAULT_RT=Yes, you can easily cause all traffic to use one provider except when you explicitly direct it to use the other provider via @@ -2317,7 +2332,7 @@ wlan0 192.168.0.0/24
A Complete Working Example - This section describes the network at shorewall.net early in 2009. + This section describes the network at shorewall.net in late 2012. The configuration is as follows: @@ -2326,196 +2341,237 @@ wlan0 192.168.0.0/24 - Avvanta -- A slow (1.5mb/384kb) DSL service with 5 static IP - addresses. + ComcastC -- A consumer-grade Comcast cable line with a + dynamic IP address. - Comcast -- A fast (20mb/10mb) Cable circuit with a single - dynamic address. + ComcastB -- A Comcast Business-class line with 5 static IP + addresses. A local network consisting of wired and wireless client systems. - A Linksys WRT300N wireless router is used as an access point for the - wireless hosts. + A wireless-N router is used as an access point for the wireless + hosts. - A DMZ hosting a single server (lists.shorewall.net aka - www1.shorewall.net, ftp1.shorewall.net,etc.) + A DMZ hosting a two servers (one has two public IP addresses - + one for receiving email and one for sending) and a system dedicaed to + running irssi (usually via IPv6) The network is pictured in the following diagram: - + - Because of the speed of the cable provider, all traffic uses that - provider unless there is a specific need for the traffic to use the DSL - line. + The Business Gateway manages a gigabit local network with address + 10.1.10.1/24. So The firewall is given address 10.1.10.11/24 and the + gateway is configured to route the public IP block via that address. The + gateway's firewall is only enabled for the 10.1.10/0/24 network. - - - Responses to connections from the Internet to one of the DSL IP - addresses -- the track option takes - care of that. - + Because the business network is faster and more reliable, the + configuration favors sending local network traffic via that uplink rather + than the consumer line. - - Connections initiated by the server and connections requested by - clients on the firewall that have bound their local socket to one of - the DSL IP addresses. Two entries in - /etc/shorewall/rtrules take care of that - traffic. - - + Here are the key entries in + /etc/shorewall/params: - As a consequence, I have disabled all route filtering on the - firewall and only use the balance option - in /etc/shorewall/providers on the Comcast provider - whose default route in the main table is established by DHCP. By - specifying the fallback option on - Avvanta, I ensure that there is still a default route if Comcast is down. - lsm is used to monitor the links. + LOG=NFLOG - /etc/sysctl.conf: +INT_IF=eth2 +TUN_IF=tun+ +COMB_IF=eth1 +COMC_IF=eth0 - net.ipv4.conf.all.rp_filter = 0 +STATISTICAL= +PROXY= +FALLBACK= +PROXYDMZ= +SQUID2= - /etc/shorewall/shorewall.conf: + The last three variables are used to configure the firewall + differently to exercise various Shorewall features. - ROUTE_FILTER=No -RESTORE_DEFAULT_ROUTE=No + Here are the key entries in + /etc/shorewall/shorewall.conf: - RESTORE_DEFAULT_ROUTE=No causes the default route in the main table - to be deleted when the Comcast link is unavailable. That way, the default - route in the default table will be used until Comcast is available - again. + ############################################################################### +# F I R E W A L L O P T I O N S +############################################################################### - /etc/shorewall/providers: +... - #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY -Avvanta 1 0x100 main eth0 206.124.146.254 track,loose,fallback eth2,eth4,tun* -Comcast 2 0x200 main eth3 detect track,balance eth2,eth4,tun* -#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE +ACCOUNTING_TABLE=mangle - The loose option on Avvanta results - in fewer routing rules. The first two routing rules below insure that all - traffic from Avvanta-assigned IP addresses is sent via the Avvanta - provider. The 'tun*' included in the COPY column is there because I run a - routed OpenVPN server on the firewall. +... - /etc/shorewall/rtrules: +AUTOMAKE=Yes - #SOURCE DEST PROVIDER PRIORITY -- 172.20.0.0/24 main 1000 # Addresses assigned by routed OpenVPN server -206.124.146.176/30 - Avvanta 26000 -206.124.146.180 - Avvanta 26000 -- 216.168.3.44 Avvanta 26000 # Avvanta NNTP Server -- verifies source IP address -#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE +BLACKLISTNEWONLY=Yes - The /etc/shorewall/rtrules entries provide all - of the provider selection necessary so my - /etc/shorewall/tcrules file is used exclusively for - traffic shaping of the Avvanta line. Note that I still need to provide - values in the MARK colum of /etc/shorewall/providers - because I specify track on both - providers. +... - Here is the output of shorewall show - routing: +EXPAND_POLICIES=No - Routing Rules +EXPORTMODULES=Yes -0: from all lookup local -1000: from all to 172.20.0.0/24 lookup main -10000: from all fwmark 0x100 lookup Avvanta -10001: from all fwmark 0x200 lookup Comcast -20256: from 71.227.156.229 lookup Comcast -26000: from 206.124.146.176/30 lookup Avvanta -26000: from 206.124.146.180 lookup Avvanta -26000: from all to 216.168.3.44 lookup Avvanta -32766: from all lookup main -32767: from all lookup default +FASTACCEPT=No -Table Avvanta: +.. -206.124.146.254 dev eth0 scope link src 206.124.146.176 -206.124.146.177 dev eth4 scope link -172.20.1.0/24 dev eth2 proto kernel scope link src 172.20.1.254 -206.124.146.0/24 dev eth0 proto kernel scope link src 206.124.146.176 -169.254.0.0/16 dev eth0 scope link -default via 206.124.146.254 dev eth0 src 206.124.146.176 +KEEP_RT_TABLES=Yes -Table Comcast: +LEGACY_FASTSTART=Yes -206.124.146.177 dev eth4 scope link -71.227.156.1 dev eth3 scope link src 71.227.156.229 -172.20.1.0/24 dev eth2 proto kernel scope link src 172.20.1.254 -71.227.156.0/23 dev eth3 proto kernel scope link src 71.227.156.229 -default via 71.227.156.1 dev eth3 src 71.227.156.229 +LOAD_HELPERS_ONLY=Yes -Table default: +... -default via 206.124.146.254 dev eth0 metric 1 +MARK_IN_FORWARD_CHAIN=No -Table local: +MODULE_SUFFIX=ko -broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1 -broadcast 172.20.1.0 dev eth2 proto kernel scope link src 172.20.1.254 -broadcast 206.124.146.255 dev eth0 proto kernel scope link src 206.124.146.176 -local 206.124.146.179 dev eth0 proto kernel scope host src 206.124.146.176 -local 206.124.146.178 dev eth0 proto kernel scope host src 206.124.146.176 -local 206.124.146.176 dev eth0 proto kernel scope host src 206.124.146.176 -local 206.124.146.176 dev eth4 proto kernel scope host src 206.124.146.176 -broadcast 71.227.157.255 dev eth3 proto kernel scope link src 71.227.156.229 -broadcast 71.227.156.0 dev eth3 proto kernel scope link src 71.227.156.229 -local 172.20.1.254 dev eth2 proto kernel scope host src 172.20.1.254 -local 127.0.0.2 dev lo proto kernel scope host src 127.0.0.1 -broadcast 172.20.1.255 dev eth2 proto kernel scope link src 172.20.1.254 -local 71.227.156.229 dev eth3 proto kernel scope host src 71.227.156.229 -broadcast 206.124.146.0 dev eth0 proto kernel scope link src 206.124.146.176 -broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1 -local 206.124.146.180 dev eth0 proto kernel scope host src 206.124.146.176 -local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1 -local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1 +MULTICAST=No -Table main: +MUTEX_TIMEOUT=60 -206.124.146.177 dev eth4 scope link -172.20.1.0/24 dev eth2 proto kernel scope link src 172.20.1.254 -206.124.146.0/24 dev eth0 proto kernel scope link src 206.124.146.176 -71.227.156.0/23 dev eth3 proto kernel scope link src 71.227.156.229 -169.254.0.0/16 dev eth0 scope link -127.0.0.0/8 dev lo scope link -default via 71.227.156.1 dev eth3 +NULL_ROUTE_RFC1918=Yes + +OPTIMIZE=31 + +OPTIMIZE_ACCOUNTING=No + +REQUIRE_INTERFACE=No + +RESTORE_DEFAULT_ROUTE=No + +RETAIN_ALIASES=No + +ROUTE_FILTER=No + +SAVE_IPSETS= + +TC_ENABLED=No + +TC_EXPERT=No + +TC_PRIOMAP="2 3 3 3 2 3 1 1 2 2 2 2 2 2 2 2" + +TRACK_PROVIDERS=Yes + +USE_DEFAULT_RT=Yes + +USE_PHYSICAL_NAMES=Yes + +ZONE2ZONE=- + +################################################################################ +# P A C K E T M A R K L A Y O U T +################################################################################ + +TC_BITS=8 + +PROVIDER_BITS=2 + +PROVIDER_OFFSET=16 + +MASK_BITS=8 + +ZONE_BITS=0 + + I use USE_DEFAULT_RT=Yes and since there are only two providers, two + provider bits are all that are required. + + Here is /etc/shorewall/zones: + + fw firewall +loc ip #Local Zone +net ip #Internet +smc:net ip #10.0.1.0/24 +vpn ip #OpenVPN clients +dmz ip #LXC Containers /etc/shorewall/interfaces: - #ZONE INTERFACE BROADCAST OPTIONS -loc eth2 detect dhcp,routeback -dmz eth4 detect -net eth0 detect dhcp,blacklist,tcpflags,optional -net eth3 detect dhcp,blacklist,tcpflags,optional -#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE + #ZONE INTERFACE OPTIONS +loc INT_IF dhcp,physical=$INT_IF,required,wait=5,routefilter,nets=172.20.1.0/24 +net COMB_IF optional,sourceroute=0,routefilter=0,arp_ignore=1,proxyarp=0,physical=$COMB_IF,upnp,nosmurfs,tcpflags +net COMC_IF optional,sourceroute=0,routefilter=0,arp_ignore=1,proxyarp=0,physical=$COMC_IF,upnp,nosmurfs,tcpflags,dhcp +vpn TUN_IF+ physical=tun+,ignore=1 +dmz br0 routeback,proxyarp=1 +- lo ignore - /etc/shorewall/masq: + /etc/shorewall/providers: - #INTERFACE SOURCE ADDRESS PROTO PORT(S) IPSEC + #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY +?if $FALLBACK +ComcastB 1 0x10000 - COMB_IF 70.90.191.126 loose,fallback +ComcastC 2 0x20000 - COMC_IF detect loose,fallback +?elsif $STATISTICAL +ComcastB 1 0x10000 - COMB_IF 70.90.191.126 loose,load=0.66666667 +ComcastC 2 0x20000 - COMC_IF detect loose,load=0.33333333 +?else +ComcastB 1 0x10000 - COMB_IF 70.90.191.126 loose,balance=2 +ComcastC 2 0x20000 - COMC_IF detect loose,balance +?endif +?if $PROXY && ! $SQUID +Squid 3 - - lo - tproxy +?endif + -COMMENT Masquerade Local Network -eth3 0.0.0.0/0 -eth0 !206.124.146.0/24 206.124.146.179 + Notice that in the current balance mode, as in the STAISTICAL mode, + the business line is favored 2:1 over the consumer line. -#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE + Here is /etc/shorewall/rtrules: - All traffic leaving eth3 must use the dynamic IP address assigned to - that interface as the SOURCE address. All traffic leaving eth0 that does - not have a SOURCE address falling within the Avvanta subnet - (206.124.146.0/24) must have its SOURCE address changed to - 206.124.146.179. + #SOURCE DEST PROVIDER PRIORITY +70.90.191.121 - ComcastB 1000 +70.90.191.123 - ComcastB 1000 +&COMC_IF - ComcastC 1000 +172.20.1.145 - ComcastC 1000 +172.20.1.146 - ComcastC 1000 +br0 - ComcastB 11000 + + For reference, this configuration generates these routing + rules: + + root@gateway:~# ip rule ls +0: from all lookup local +999: from all lookup main +1000: from 70.90.191.121 lookup Primary +1000: from 70.90.191.123 lookup Primary +1000: from 67.170.121.6 lookup Backup +1000: from 172.20.1.145 lookup Backup +1000: from 172.20.1.146 lookup Backup +10000: from all fwmark 0x10000/0x30000 lookup Primary +10001: from all fwmark 0x20000/0x30000 lookup Backup +11000: from all iif br0 lookup Primary +32765: from all lookup balance +32767: from all lookup default +root@gateway:~# + + /etc/shorewall/tcrules is not used to support + Multi-ISP: + + #MARK SOURCE DEST PROTO DEST SOURCE +# PORT(S) PORT(S) +FORMAT 2 +TTL(+1):P INT_IF - +SAME:P INT_IF - tcp 80,443 +?if $PROXY && ! $SQUID2 + DIVERT COMB_IF - tcp - 80 + DIVERT COMC_IF - tcp - 80 + DIVERT br0 172.20.1.0/24 tcp - 80 + TPROXY(3129,172.20.1.254) INT_IF - tcp 80 + ?if $PROXYDMZ + TPROXY(3129,172.20.1.254) br0 - tcp 80 + ?endif +?endof +
diff --git a/docs/images/Network2012a.dia b/docs/images/Network2012a.dia new file mode 100644 index 000000000..2eec02867 Binary files /dev/null and b/docs/images/Network2012a.dia differ diff --git a/docs/images/Network2012a.png b/docs/images/Network2012a.png new file mode 100644 index 000000000..e7a51ca25 Binary files /dev/null and b/docs/images/Network2012a.png differ