mirror of
https://gitlab.com/shorewall/code.git
synced 2024-11-22 15:43:30 +01:00
2215 lines
92 KiB
XML
2215 lines
92 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
||
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
|
||
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
|
||
<article>
|
||
<!--ble$Id$-->
|
||
|
||
<articleinfo>
|
||
<title>Shorewall and Multiple Internet Connections</title>
|
||
|
||
<authorgroup>
|
||
<author>
|
||
<firstname>Tom</firstname>
|
||
|
||
<surname>Eastep</surname>
|
||
</author>
|
||
</authorgroup>
|
||
|
||
<pubdate><?dbtimestamp format="Y/m/d"?></pubdate>
|
||
|
||
<copyright>
|
||
<year>2005</year>
|
||
|
||
<year>2006</year>
|
||
|
||
<year>2007</year>
|
||
|
||
<year>2008</year>
|
||
|
||
<year>2009</year>
|
||
|
||
<holder>Thomas M. Eastep</holder>
|
||
</copyright>
|
||
|
||
<legalnotice>
|
||
<para>Permission is granted to copy, distribute and/or modify this
|
||
document under the terms of the GNU Free Documentation License, Version
|
||
1.2 or any later version published by the Free Software Foundation; with
|
||
no Invariant Sections, with no Front-Cover, and with no Back-Cover
|
||
Texts. A copy of the license is included in the section entitled
|
||
<quote><ulink url="GnuCopyright.htm">GNU Free Documentation
|
||
License</ulink></quote>.</para>
|
||
</legalnotice>
|
||
</articleinfo>
|
||
|
||
<warning>
|
||
<para>This document describes the Multi-ISP facility in Shorewall 4.3.5
|
||
and later. If you are running an earlier release, please see the
|
||
documentation for that release.</para>
|
||
</warning>
|
||
|
||
<warning>
|
||
<para>Reading just Shorewall documentation is probably not going to give
|
||
you enough background to use this material. Shorewall may make iptables
|
||
easy but the Shorewall team doesn't have the resources to be able to
|
||
spoon-feed Linux policy routing to you (please remember that the user's
|
||
manual for a tractor doesn't teach you to grow corn either). You will
|
||
likely need to refer to the following additional information:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>The LARTC HOWTO: <ulink
|
||
url="http://www.lartc.org">http://www.lartc.org</ulink></para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Output of <command>man ip</command></para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Output of <command>ip route help</command> and <command>ip rule
|
||
help</command></para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</warning>
|
||
|
||
<section id="Support">
|
||
<title>Multiple Internet Connection Support</title>
|
||
|
||
<para>Shorewall includes limited support for multiple Internet
|
||
connections. Limitations of this support are as follows:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>It utilizes static routing configuration. If there is a change
|
||
in the routing topopogy, Shorewall must be restarted.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>The routing changes are made and the route cache is purged when
|
||
Shorewall is started <emphasis role="bold">and when Shorewall is
|
||
restarted</emphasis> (unless you specify the "-n" option to
|
||
<command>shorewall restart</command>). Ideally, restarting the packet
|
||
filter should have no effect on routing.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>For most routing applications, <ulink
|
||
url="http://www.quagga.net/">Quagga</ulink> is a better solution
|
||
although it requires that your ISPs offer routing protocol
|
||
support.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
|
||
<section id="Overview">
|
||
<title>Overview</title>
|
||
|
||
<para>Let's assume that a firewall is connected via two separate
|
||
Ethernet interfaces to two different ISPs.<footnote>
|
||
<para>While we describe a setup using different ISPs in this
|
||
article, the facility also works with two uplinks from the same
|
||
ISP.</para>
|
||
</footnote> as in the following diagram.</para>
|
||
|
||
<graphic align="center" fileref="images/TwoISPs.png" valign="middle" />
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>eth0 connects to ISP1. The IP address of eth0 is
|
||
206.124.146.176 and the ISP's gateway router has IP address
|
||
206.124.146.254.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>eth1 connects to ISP 2. The IP address of eth1 is
|
||
130.252.99.27 and the ISP's gateway router has IP address
|
||
130.252.99.254.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>eth2 connects to the local LAN. Its IP configuration is not
|
||
relevant to this discussion.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
|
||
<para>Each of these <firstterm>providers</firstterm> is described in an
|
||
entry in the file <filename>/etc/shorewall/providers</filename>.</para>
|
||
|
||
<para>Entries in <filename>/etc/shorewall/providers</filename> can
|
||
specify that outgoing connections are to be load-balanced between the
|
||
two ISPs. Entries in <filename>/etc/shorewall/tcrules</filename> and
|
||
<filename>/etc/shorewall/route_rules</filename> can be used to direct
|
||
particular outgoing connections to one ISP or the other. Use of
|
||
<filename>/etc/shorewall/tcrules</filename> is not required for
|
||
<filename>/etc/shorewall/providers</filename> to work, but in most
|
||
cases, you must select a unique MARK value for each provider so
|
||
Shorewall can set up the correct marking rules for you.</para>
|
||
|
||
<para>When you use the <emphasis role="bold">track</emphasis> option in
|
||
<filename>/etc/shorewall/providers</filename>, connections from the
|
||
Internet are automatically routed back out of the correct interface and
|
||
through the correct ISP gateway. This works whether the connection is
|
||
handled by the firewall itself or if it is routed or port-forwarded to a
|
||
system behind the firewall.</para>
|
||
|
||
<para>Shorewall will set up the routing and will update the
|
||
<filename>/etc/iproute2/rt_tables</filename> to include the table names
|
||
and numbers of the tables that it adds.</para>
|
||
|
||
<caution>
|
||
<para>This feature uses <ulink url="traffic_shaping.htm">packet
|
||
marking</ulink> to control the routing. As a consequence, there are
|
||
some restrictions concerning entries in
|
||
<filename>/etc/shorewall/tcrules</filename>:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Packet marking for traffic control purposes may not be done
|
||
in the PREROUTING table for connections involving providers with
|
||
'track' specified (see below).</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>You may not use the SAVE or RESTORE options unless you also
|
||
set HIGH_ROUTE_MARKS=Yes in
|
||
<filename>/etc/shorewall/shorewall.conf</filename>.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>You may not use connection marking unless you also set
|
||
HIGH_ROUTE_MARKS=Yes in
|
||
<filename>/etc/shorewall/shorewall.conf</filename>.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</caution>
|
||
|
||
<para>The <filename>/etc/shorewall/providers</filename> file can also be
|
||
used in other routing scenarios. See the <ulink
|
||
url="Shorewall_Squid_Usage.html">Squid documentation</ulink> for an
|
||
example.</para>
|
||
</section>
|
||
|
||
<section id="providers">
|
||
<title>/etc/shorewall/providers File</title>
|
||
|
||
<para>Entries in this file have the following columns. As in all
|
||
Shorewall configuration files, enter "-" in a column if you don't want
|
||
to enter any value.</para>
|
||
|
||
<variablelist>
|
||
<varlistentry>
|
||
<term>NAME</term>
|
||
|
||
<listitem>
|
||
<para>The provider name. Must begin with a letter and consist of
|
||
letters and digits. The provider name becomes the name of the
|
||
generated routing table for this provider.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>NUMBER</term>
|
||
|
||
<listitem>
|
||
<para>A number between 1 and 252. This becomes the routing table
|
||
number for the generated table for this provider.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>MARK</term>
|
||
|
||
<listitem>
|
||
<para>A mark value used in your /etc/shorewall/tcrules file to
|
||
direct packets to this provider. Shorewall will also mark
|
||
connections that have seen input from this provider with this
|
||
value and will restore the packet mark in the PREROUTING CHAIN.
|
||
Mark values must be in the range 1-255.</para>
|
||
|
||
<para>Alternatively, you may set HIGH_ROUTE_MARKS=Yes in
|
||
<filename>/etc/shorewall/shorewall.conf</filename>. This allows
|
||
you to:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Use connection marks for traffic shaping, provided that
|
||
you assign those marks in the FORWARD chain.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Use mark values > 255 for provider marks in this
|
||
column.</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>These mark values must be a multiple of 256 in the
|
||
range 256-65280 (hex equivalent 0x100 - 0xFF00 with the
|
||
low-order 8 bits being zero); or</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Set WIDE_TC_MARKS=Yes in <ulink
|
||
url="manpages/shorewall.conf.html">shorewall.conf
|
||
</ulink>(5) and use mark values in the range 0x10000 -
|
||
0xFF0000 with the low-order 16 bits being zero.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</listitem>
|
||
</itemizedlist>
|
||
|
||
<para>This column may be omitted if you don´t use packet marking
|
||
to direct connections to a particular provider and you don´t
|
||
specify <emphasis role="bold">track</emphasis> in the OPTIONS
|
||
column.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>DUPLICATE</term>
|
||
|
||
<listitem>
|
||
<para>Gives the name or number of a routing table to duplicate.
|
||
May be 'main' or the name or number of a previously declared
|
||
provider. For most applications, you want to specify 'main' here.
|
||
This field should be be specified as '-' when USE_DEFAULT_RT=Yes
|
||
in <filename>shorewall.conf</filename></para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>INTERFACE</term>
|
||
|
||
<listitem>
|
||
<para>The name of the interface to the provider. Where multiple
|
||
providers share the same interface, you must follow the name of
|
||
the interface by a colon (":") and the IP address assigned by this
|
||
provider (e.g., eth0:206.124.146.176). See <link
|
||
linkend="Shared">below</link> for additional
|
||
considerations.</para>
|
||
|
||
<para>The interface must have been previously defined in <ulink
|
||
url="manpages/shorewall-interfaces.html">shorewall-interfaces</ulink>
|
||
(5). In general, that interface should not have the
|
||
<option>proxyarp</option> option specified unless
|
||
<option>loose</option> is given in the OPTIONS column of this
|
||
entry.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>GATEWAY</term>
|
||
|
||
<listitem>
|
||
<para>The IP address of the provider's Gateway router.</para>
|
||
|
||
<para>You can enter <emphasis role="bold">detect</emphasis> here
|
||
and Shorewall will attempt to automatically determine the gateway
|
||
IP address.</para>
|
||
|
||
<para><emphasis role="bold">Hint:</emphasis> <emphasis
|
||
role="bold">"detect"</emphasis> is appropriate for use in cases
|
||
where the interface named in the INTERFACE column is dynamically
|
||
configured via DHCP etc. Be sure, however, that you don't have
|
||
stale dhcp client state files in <filename
|
||
class="directory">/var/lib/dhcpcd </filename>or
|
||
<filename>/var/lib/dhclient-*.lease</filename> because Shorewall
|
||
may try to use those stale files to determine the gateway
|
||
address.</para>
|
||
|
||
<para>The GATEWAY may be omitted (enter '-') for point-to-point
|
||
links.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>OPTIONS</term>
|
||
|
||
<listitem>
|
||
<para>A comma-separated list from the following:</para>
|
||
|
||
<variablelist>
|
||
<varlistentry>
|
||
<term>track</term>
|
||
|
||
<listitem>
|
||
<para><important>
|
||
<para>Beginning with Shorwall 4.3.3, <emphasis
|
||
role="bold">track</emphasis> defaults to the setting of
|
||
the <option>TRACK_PROVIDERS</option> option in <ulink
|
||
url="manpages/shorewall.conf">shorewall.conf
|
||
</ulink>(5). To disable this option when you have
|
||
specified TRACK_PROVIDERS=Yes, you must specify
|
||
<emphasis role="bold">notrack</emphasis> (see
|
||
below).</para>
|
||
</important>If specified, connections FROM this interface
|
||
are to be tracked so that responses may be routed back out
|
||
this same interface.</para>
|
||
|
||
<para>You want to specify 'track' if Internet hosts will be
|
||
connecting to local servers through this provider. Any time
|
||
that you specify 'track', you will normally want to also
|
||
specify 'balance' (see below). 'track' will also ensure that
|
||
outgoing connections remain stay anchored to a single
|
||
provider and don't try to switch providers when route cache
|
||
entries expire.</para>
|
||
|
||
<para>Use of this feature requires that your kernel and
|
||
iptables include CONNMARK target and connmark match support
|
||
(<emphasis role="bold">Warning</emphasis>: Until recently,
|
||
standard <trademark>Debian</trademark> and
|
||
<trademark>Ubuntu</trademark> kernels lacked that support.
|
||
<emphasis>Both Lenny and Jaunty do have the proper
|
||
support</emphasis>).</para>
|
||
|
||
<important>
|
||
<para>If you are running a version of Shorewall earlier
|
||
than 4.4.3 and are using
|
||
<filename>/etc/shorewall/providers</filename> because you
|
||
have multiple Internet connections, we recommend that you
|
||
specify <emphasis role="bold">track</emphasis> even if you
|
||
don't need it. It helps maintain long-term connections in
|
||
which there are significant periods with no
|
||
traffic.</para>
|
||
</important>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>balance</term>
|
||
|
||
<listitem>
|
||
<para>The providers that have <emphasis
|
||
role="bold">balance</emphasis> specified will get outbound
|
||
traffic load-balanced among them. Balancing will not be
|
||
perfect, as it is route based, and routes are cached. This
|
||
means that routes to often-used sites will always be over
|
||
the same provider.</para>
|
||
|
||
<para>By default, each provider is given the same weight (1)
|
||
. You can change the weight of a given provider by following
|
||
<emphasis role="bold">balance</emphasis> with "=" and the
|
||
desired weight (e.g., balance=2). The weights reflect the
|
||
relative bandwidth of the providers connections and should
|
||
be small numbers since the kernel actually creates
|
||
additional default routes for each weight increment.</para>
|
||
|
||
<important>
|
||
<para>If you are using
|
||
<filename>/etc/shorewall/providers</filename> because you
|
||
have multiple Internet connections, we recommend that you
|
||
specify <emphasis role="bold">balance</emphasis> even if
|
||
you don't need it. You can still use entries in
|
||
<filename>/etc/shorewall/tcrules</filename> and
|
||
<filename>/etc/shorewall/route_rules</filename> to force
|
||
all traffic to one provider or another.<note>
|
||
<para>If you don't heed this advice then please read
|
||
and follow the advice in <ulink
|
||
url="FAQ.htm#faq57">FAQ 57</ulink> and <ulink
|
||
url="FAQ.htm#faq58">FAQ 58</ulink>.</para>
|
||
</note></para>
|
||
</important>
|
||
|
||
<important>
|
||
<para>If you specify <emphasis
|
||
role="bold">balance</emphasis> and still find that all
|
||
traffic is going out through only one provider, you may
|
||
need to install a kernel built with
|
||
CONFIG_IP_ROUTE_MULTIPATH_CACHED=n. Several users have
|
||
reported that this change has corrected similar
|
||
problems.</para>
|
||
|
||
<para>The SuSE 10.0 kernel is subject to this problem, and
|
||
<ulink
|
||
url="https://bugzilla.novell.com/show_bug.cgi?id=190908">
|
||
a kernel oops may result in this circumstance.</ulink>
|
||
SUSE 10.1 and SLES 10 have
|
||
CONFIG_IP_ROUTE_MULTIPATH_CACHED=n set by default. The
|
||
source of the problem seems to be <ulink
|
||
url="http://news.gmane.org/find-root.php?message_id=%3c00da01c5b35a%24b12b9860%241b00a8c0%40cruncher%3e">an
|
||
incompatibility between the LARTC patches and
|
||
CONFIG_IP_ROUTE_MULTIPATH_CACHED.</ulink></para>
|
||
</important>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>loose</term>
|
||
|
||
<listitem>
|
||
<para>Do not generate routing rules that force traffic whose
|
||
source IP is an address of the INTERFACE to be routed to
|
||
this provider. Useful for defining providers that are to be
|
||
used only when the appropriate packet mark is
|
||
applied.</para>
|
||
|
||
<para>Shorewall makes no attempt to consolidate the routing
|
||
rules added when <emphasis role="bold">loose</emphasis> is
|
||
not specified. So, if you have multiple IP addresses on a
|
||
provider interface, you may be able to replace the rules
|
||
that Shorewall generates with one or two rules in
|
||
<filename>/etc/shorewall/route_rules</filename>. In that
|
||
case, you can specify <emphasis role="bold">loose</emphasis>
|
||
to suppress Shorewall's rule generation. See the <link
|
||
linkend="Complete">example</link> below.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>notrack</term>
|
||
|
||
<listitem>
|
||
<para>Added in Shorewall 4.4.3. This option turns off the
|
||
<emphasis role="bold">track</emphasis> option.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>optional</term>
|
||
|
||
<listitem>
|
||
<note>
|
||
<para>This option is deprecated in favor of the
|
||
<option>optional</option> <ulink
|
||
url="manpages/shorewall-interfaces.html">interface
|
||
option</ulink>. That option performs the same
|
||
function.</para>
|
||
</note>
|
||
|
||
<para>Shorewall will determine if this interface is up and
|
||
has a configured IP address. If it is not, a warning is
|
||
issued and this provider is not configured.</para>
|
||
|
||
<note>
|
||
<para><emphasis role="bold">optional</emphasis> is
|
||
designed to detect interface states that will cause
|
||
<command>shorewall start</command> or <command>shorewall
|
||
restart</command> to fail; just because an interface is in
|
||
a state that Shorewall can [re]start without error doesn't
|
||
mean that traffic can actually be sent through the
|
||
interface.</para>
|
||
|
||
<para>You can supply an 'isusable' <ulink
|
||
url="shorewall_extension_scripts.htm">extension
|
||
script</ulink> to extend Shorewall's interface state
|
||
detection. See also the <link
|
||
linkend="LinkMonitor">Gateway Monitoring and
|
||
Failover</link> section below.</para>
|
||
</note>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>src=<replaceable>source-address</replaceable></term>
|
||
|
||
<listitem>
|
||
<para>Specifies the source address to use when routing to
|
||
this provider and none is known (the local client has bound
|
||
to the 0 address). May not be specified when an
|
||
<replaceable>address</replaceable> is given in the INTERFACE
|
||
column. If this option is not used, Shorewall substitutes
|
||
the primary IP address on the interface named in the
|
||
INTERFACE column.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>mtu=<replaceable>number</replaceable></term>
|
||
|
||
<listitem>
|
||
<para>Specifies the MTU when forwarding through this
|
||
provider. If not given, the MTU of the interface named in
|
||
the INTERFACE column is assumed.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>fallback[=<replaceable>weight</replaceable>]</term>
|
||
|
||
<listitem>
|
||
<para>Indicates that a default route through the provider
|
||
should be added to the <firstterm>default</firstterm>
|
||
routing table (table 253). If a
|
||
<replaceable>weight</replaceable> is given, a balanced route
|
||
is added with the weight of this provider equal to the
|
||
specified <replaceable>weight</replaceable>. If the option
|
||
is given without a <replaceable>weight</replaceable>, a
|
||
separate default route is added through the provider's
|
||
gateway; the route has a metric equal to the provider's
|
||
NUMBER. The option is ignored with a warning message if
|
||
USE_DEFAULT_RT=Yes in
|
||
<filename>shorewall.conf</filename>.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
</variablelist>
|
||
|
||
<para>For those of you who are confused between<emphasis
|
||
role="bold"> track</emphasis> and <emphasis
|
||
role="bold">balance</emphasis>:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para><emphasis role="bold">track</emphasis> governs incoming
|
||
connections (but is also useful for binding long-running
|
||
connections to the same interface).</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para><emphasis role="bold">balance</emphasis> governs
|
||
outgoing connections.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>COPY</term>
|
||
|
||
<listitem>
|
||
<para>A comma-separated list of interface names. Wildcards
|
||
specified using an asterisk ("*") are permitted (e.g., tun*
|
||
).</para>
|
||
|
||
<para>When you specify an existing table in the DUPLICATE column,
|
||
Shorewall copies all routes through the interface specified in the
|
||
INTERFACE column plus the interfaces listed in this column.
|
||
Normally, you will list all interfaces on your firewall in this
|
||
column except those Internet interfaces specified in the INTERFACE
|
||
column of entries in this file.</para>
|
||
|
||
<note>
|
||
<para>Beginning with Shorewall 4.4.15, provider routing tables
|
||
can be augmeted with additional routes through use of the <link
|
||
linkend="routes">/etc/shorewall/routes</link> file.</para>
|
||
</note>
|
||
</listitem>
|
||
</varlistentry>
|
||
</variablelist>
|
||
</section>
|
||
|
||
<section id="Providers">
|
||
<title>What an entry in the Providers File Does</title>
|
||
|
||
<para>Adding another entry in the providers file simply creates an
|
||
alternate routing table for you (see the<ulink
|
||
url="http://www.lartc.org"> LARTC Howto</ulink>). The table will usually
|
||
contain two routes:</para>
|
||
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>A host route to the specified GATEWAY through the specified
|
||
INTERFACE.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>A default route through the GATEWAY.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
|
||
<para>Note that the first route is omitted if "-" is specified as the
|
||
GATEWAY; in that case, the default route does not specify a gateway
|
||
(point-to-point link).</para>
|
||
|
||
<para>If the DUPLICATE column is non-empty, then routes from the table
|
||
named in that column are copied into the new table. By default, all
|
||
routes (except default routes) are copied. The set of routes copied can
|
||
be restricted using the COPY column which lists the interfaces whose
|
||
routes you want copied. You will generally want to include all local
|
||
interfaces in this list. You should exclude the loopback interface (lo)
|
||
and any interfaces that do not have an IP configuration. You should also
|
||
omit interfaces like <emphasis role="bold">tun</emphasis> interfaces
|
||
that are created dynamically. Traffic to networks handled by those
|
||
interfaces should be routed through the main table using entries in
|
||
<filename>/etc/shorewall/route_rules</filename> (see Example 2 <link
|
||
linkend="Examples">below</link>) or by using <link
|
||
linkend="USE_DEFAULT_RT">USE_DEFAULT_RT=Yes</link>.</para>
|
||
|
||
<para>In addition:</para>
|
||
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>Unless <emphasis role="bold">loose</emphasis> is specified, an
|
||
ip rule is generated for each IP address on the INTERFACE that
|
||
routes traffic from that address through the associated routing
|
||
table.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>If you specify <emphasis role="bold">track</emphasis>, then
|
||
connections which have had at least one packet arrive on the
|
||
interface listed in the INTERFACE column have their connection mark
|
||
set to the value in the MARK column. In the PREROUTING chain,
|
||
packets with a connection mark have their packet mark set to the
|
||
value of the associated connection mark; packets marked in this way
|
||
bypass any prerouting rules that you create in
|
||
<filename>/etc/shorewall/tcrules</filename>. This ensures that
|
||
packets associated with connections from outside are always routed
|
||
out of the correct interface.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>If you specify <emphasis role="bold">balance</emphasis>, then
|
||
Shorewall will replace the 'default' route with weight 100 in the
|
||
'main' routing table with a load-balancing route among those
|
||
gateways where <emphasis role="bold">balance</emphasis> was
|
||
specified. So if you configure default routes, be sure that their
|
||
weight is less than 100 or the route added by Shorewall will not be
|
||
used.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
|
||
<para>That's <emphasis role="bold">all</emphasis> that these entries do.
|
||
You still have to follow the principle stated in the <ulink
|
||
url="Shorewall_and_Routing.html">Shorewall Routing
|
||
documentation</ulink>:</para>
|
||
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>Routing determines where packets are to be sent.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Once routing determines where the packet is to go, the
|
||
firewall (Shorewall) determines if the packet is allowed to go there
|
||
and controls rewriting of the SOURCE IP address
|
||
(SNAT/MASQUERADE).</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
|
||
<para>The bottom line is that if you want traffic to go out through a
|
||
particular provider then you <emphasis>must </emphasis>mark that traffic
|
||
with the provider's MARK value in
|
||
<filename>/etc/shorewall/tcrules</filename> and you must do that marking
|
||
in the PREROUTING chain; or, you must provide the appropriate rules in
|
||
<filename>/etc/shorewall/route_rules</filename>.</para>
|
||
</section>
|
||
|
||
<section>
|
||
<title>/etc/shorewall/masq and Multi-ISP</title>
|
||
|
||
<para>If you masquerade a local network, you will need to add masquerade
|
||
rules for both external interfaces. Referring to the diagram above, if
|
||
each of the interfaces has only a single IP address and you have no
|
||
systems with public IP addresses behind your firewall, then I suggest
|
||
the following simple entries:</para>
|
||
|
||
<programlisting>#INTERFACE SOURCE ADDRESS
|
||
eth0 0.0.0.0/0 206.124.146.176
|
||
eth1 0.0.0.0/0 130.252.99.27</programlisting>
|
||
|
||
<para>If you have a public subnet (for example 206.124.146.176/30)
|
||
behind your firewall, then use exclusion:</para>
|
||
|
||
<programlisting>#INTERFACE SOURCE ADDRESS
|
||
eth0 !206.124.146.176/29 206.124.146.176
|
||
eth1 0.0.0.0/0 130.252.99.27</programlisting>
|
||
|
||
<para>Note that exclusion is only used on the interface corresponding to
|
||
internal subnetwork.</para>
|
||
|
||
<para>If you have multiple IP addresses on one of your interfaces, you
|
||
can use a similar technique -- simplY exclude the smallest network that
|
||
contains all of those addresses from being masqueraded.</para>
|
||
|
||
<warning>
|
||
<para>Entries in <filename>/etc/shorewall/masq</filename> have no
|
||
effect on which ISP a particular connection will be sent through. That
|
||
is rather the purpose of entries in
|
||
<filename>/etc/shorewall/tcrules</filename> and
|
||
<filename>/etc/shorewall/route_rules</filename>.</para>
|
||
</warning>
|
||
</section>
|
||
|
||
<section id="Martians">
|
||
<title>Martians</title>
|
||
|
||
<para>One problem that often arises with Multi-ISP configuration is
|
||
'Martians'. If your Internet interfaces are configured with the
|
||
<emphasis role="bold">routefilter</emphasis> option in
|
||
<filename>/etc/shorewall/interfaces</filename> (remember that if you set
|
||
that option, you should also select <emphasis
|
||
role="bold">logmartians</emphasis>), then things may not work correctly
|
||
and you will see messages like this:</para>
|
||
|
||
<programlisting>Feb 9 17:23:45 gw.ilinx kernel: martian source 206.124.146.176 from 64.86.88.116, on dev eth1
|
||
Feb 9 17:23:45 gw.ilinx kernel: ll header: 00:a0:24:2a:1f:72:00:13:5f:07:97:05:08:00</programlisting>
|
||
|
||
<para>The above message is somewhat awkwardly phrased. The source IP in
|
||
this incoming packet was 64.86.88.116 and the destination IP address was
|
||
206.124.146.176. Another gotcha is that the incoming packet has already
|
||
had the destination IP address changed for DNAT or because the original
|
||
outgoing connection was altered by an entry in
|
||
<filename>/etc/shorewall/masq</filename> (SNAT or Masquerade). So the
|
||
destination IP address (206.124.146.176) may not have been the
|
||
destination IP address in the packet as it was initially
|
||
received.</para>
|
||
|
||
<para>There a couple of common causes for these problems:</para>
|
||
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>You have connected both of your external interfaces to the
|
||
same hub/switch. Connecting multiple firewall interfaces to a common
|
||
hub or switch is always a bad idea that will result in
|
||
hard-to-diagnose problems.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>You are specifying both the <emphasis
|
||
role="bold">loose</emphasis> and <emphasis
|
||
role="bold">balance</emphasis> options on your provider(s). This can
|
||
cause individual connections to ping-pong back and forth between the
|
||
interfaces which is almost guaranteed to cause problems.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>You are redirecting traffic from the firewall system out of
|
||
one interface or the other using packet marking in your
|
||
<filename>/etc/shorewall/tcrules</filename> file. A better approach
|
||
is to configure the application to use the appropriate local IP
|
||
address (the IP address of the interface that you want the
|
||
application to use). See <link linkend="Local">below</link>.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
|
||
<para>If all else fails, remove the <emphasis
|
||
role="bold">routefilter</emphasis> option from your external interfaces.
|
||
If you do this, you may wish to add rules to log and drop packets from
|
||
the Internet that have source addresses in your local networks. For
|
||
example, if the local LAN in the above diagram is 192.168.1.0/24, then
|
||
you would add this rule:</para>
|
||
|
||
<programlisting>#ACTION SOURCE DEST
|
||
DROP:info net:192.168.1.0/24 all</programlisting>
|
||
|
||
<para>Be sure the above rule is added before any other rules with
|
||
<emphasis>net</emphasis> in the SOURCE column.</para>
|
||
</section>
|
||
|
||
<section id="Example1">
|
||
<title id="Example">Example</title>
|
||
|
||
<para>The configuration in the figure at the top of this section would
|
||
be specified in <filename>/etc/shorewall/providers</filename> as
|
||
follows.</para>
|
||
|
||
<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
||
ISP1 1 1 main eth0 206.124.146.254 track,balance eth2
|
||
ISP2 2 2 main eth1 130.252.99.254 track,balance eth2</programlisting>
|
||
|
||
<para>Other configuration files go something like this:</para>
|
||
|
||
<para><filename>/etc/shorewall/interfaces</filename>:</para>
|
||
|
||
<programlisting>#ZONE INTERFACE BROADCAST OPTIONS
|
||
net eth0 detect …
|
||
net eth1 detect …</programlisting>
|
||
|
||
<para><filename>/etc/shorewall/policy</filename>:</para>
|
||
|
||
<programlisting>#SOURCE DESTINATION POLICY LIMIT:BURST
|
||
net net DROP</programlisting>
|
||
|
||
<para><filename>/etc/shorewall/masq</filename>:</para>
|
||
|
||
<programlisting>#INTERFACE SOURCE ADDRESS
|
||
eth0 0.0.0.0/0 206.124.146.176
|
||
eth1 0.0.0.0/0 130.252.99.27</programlisting>
|
||
</section>
|
||
|
||
<section id="Applications">
|
||
<title>Routing a Particular Application Through a Specific
|
||
Interface</title>
|
||
|
||
<para>This continues the example in the preceding section.</para>
|
||
|
||
<para>Now suppose that you want to route all outgoing SMTP traffic from
|
||
your local network through ISP 2. You would make this entry in <ulink
|
||
url="traffic_shaping.htm">/etc/shorewall/tcrules</ulink> (and if you are
|
||
running a version of Shorewall earlier than 3.0.0, you would set
|
||
TC_ENABLED=Yes in <ulink
|
||
url="???">/etc/shorewall/shorewall.conf</ulink>).</para>
|
||
|
||
<programlisting>#MARK SOURCE DEST PROTO PORT(S) CLIENT USER TEST
|
||
# PORT(S)
|
||
2:P <local network> 0.0.0.0/0 tcp 25</programlisting>
|
||
|
||
<para>Note that traffic from the firewall itself must be handled in a
|
||
different rule:</para>
|
||
|
||
<programlisting>#MARK SOURCE DEST PROTO PORT(S) CLIENT USER TEST
|
||
# PORT(S)
|
||
2 $FW 0.0.0.0/0 tcp 25</programlisting>
|
||
</section>
|
||
|
||
<section id="PortForwarding">
|
||
<title>Port Forwarding</title>
|
||
|
||
<para>Shorewall provides considerable flexibility for port forwarding in
|
||
a multi-ISP environment.</para>
|
||
|
||
<para>Normal port forwarding rules such as the following will forward
|
||
from both providers.</para>
|
||
|
||
<para><filename>/etc/shorewall/rules</filename>:</para>
|
||
|
||
<programlisting>#ACTION SOURCE DEST PROTO DEST PORT(S) SOURCE ORIGINAL
|
||
# PORTS(S) DEST
|
||
DNAT net loc:192.168.1.3 tcp 25</programlisting>
|
||
|
||
<para>Continuing the above example, to forward only connection requests
|
||
from ISP 1, you can either:</para>
|
||
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>Qualify the SOURCE by ISP 1's interface:</para>
|
||
|
||
<programlisting>#ACTION SOURCE DEST PROTO DEST PORT(S) SOURCE ORIGINAL
|
||
# PORTS(S) DEST
|
||
DNAT net<emphasis role="bold">:eth0</emphasis> loc:192.168.1.3 tcp 25</programlisting>
|
||
|
||
<para>or</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Specify the IP address of ISP 1 in the ORIGINAL DEST
|
||
column:</para>
|
||
|
||
<programlisting>#ACTION SOURCE DEST PROTO DEST PORT(S) SOURCE ORIGINAL
|
||
# PORTS(S) DEST
|
||
DNAT net loc:192.168.1.3 tcp 25 <emphasis
|
||
role="bold">- 206.124.146.176</emphasis></programlisting>
|
||
</listitem>
|
||
</orderedlist>
|
||
</section>
|
||
|
||
<section id="morethan2">
|
||
<title>More than 2 Providers</title>
|
||
|
||
<para>When there are more than two providers, you need to extend the
|
||
two-provider case in the expected way:</para>
|
||
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>For each external address, you need an entry in
|
||
<filename>/etc/shorewall/masq</filename> to handle the case where a
|
||
connection using that address as the SOURCE is sent out of the
|
||
interfaces other than the one that the address is configured
|
||
on.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>For each external interface, you need to add an entry to
|
||
<filename>/etc/shorewall/masq</filename>.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
|
||
<para>If we extend the above example to add eth3 with IP address
|
||
16.105.78.4 with gateway 16.105.78.254, then:</para>
|
||
|
||
<para><filename>/etc/shorewall/providers</filename>:<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
||
ISP1 1 1 main eth0 206.124.146.254 track,balance eth2
|
||
ISP2 2 2 main eth1 130.252.99.254 track,balance eth2
|
||
ISP3 3 3 main eth3 16.105.78.254 track,balance eth2</programlisting></para>
|
||
|
||
<para><filename>/etc/shorewall/masq</filename>:<programlisting>#INTERFACE SUBNET ADDRESS
|
||
eth0 0.0.0.0/0 206.124.146.176
|
||
eth1 0.0.0.0/0 130.252.99.27
|
||
eth3 0.0.0.0/0 16.105.78.4</programlisting></para>
|
||
</section>
|
||
|
||
<section id="Local">
|
||
<title>Applications running on the Firewall -making them use a
|
||
particular provider</title>
|
||
|
||
<para>As <link linkend="Applications">noted above</link>, separate
|
||
entries in <filename>/etc/shorewall/tcrules</filename> are required for
|
||
traffic originating from the firewall.</para>
|
||
|
||
<para>Experience has shown that in some cases, problems occur with
|
||
applications running on the firewall itself. This is especially true
|
||
when you have specified <emphasis role="bold">routefilter</emphasis> on
|
||
your external interfaces in /etc/shorewall/interfaces (see <link
|
||
linkend="Martians">above</link>). When this happens, it is suggested
|
||
that you have the application use specific local IP addresses rather
|
||
than 0.</para>
|
||
|
||
<para>Examples:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Squid: In <filename>squid.conf</filename>, set <emphasis
|
||
role="bold">tcp_outgoing_address</emphasis> to the IP address of the
|
||
interface that you want Squid to use.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>In OpenVPN, set <emphasis role="bold">local
|
||
</emphasis>(<emphasis role="bold">--local</emphasis> on the command
|
||
line) to the IP address that you want the server to receive
|
||
connections on.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
|
||
<para>Note that some traffic originating on the firewall doesn't have a
|
||
SOURCE IP address before routing. At least one Shorewall user reports
|
||
that an entry in <filename>/etc/shorewall/route_rules</filename> with
|
||
'lo' in the SOURCE column seems to be the most reliable way to direct
|
||
such traffic to a particular ISP.</para>
|
||
|
||
<para>Example:</para>
|
||
|
||
<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
||
lo - shorewall 1000</programlisting>
|
||
</section>
|
||
|
||
<section id="route_rules">
|
||
<title>/etc/shorewall/route_rules</title>
|
||
|
||
<para>The <filename>route_rules</filename> file allows assigning certain
|
||
traffic to a particular provider just as entries in the
|
||
<filename>tcrules</filename> file. The difference between the two files
|
||
is that entries in <filename>route_rules</filename> are independent of
|
||
Netfilter.</para>
|
||
|
||
<section id="Routing_rules">
|
||
<title>Routing Rules</title>
|
||
|
||
<para>Routing rules are maintained by the Linux kernel and can be
|
||
displayed using the <command>ip rule ls</command> command. When
|
||
routing a packet, the rules are processed in turn until the packet is
|
||
successfully routed.</para>
|
||
|
||
<programlisting>gateway:~ # <command>ip rule ls</command>
|
||
0: from all lookup local <=== Local (to the firewall) IP addresses
|
||
10001: from all fwmark 0x1 lookup Blarg <=== This and the next rule are generated by the
|
||
10002: from all fwmark 0x2 lookup Comcast 'MARK' values in /etc/shorewall/providers.
|
||
20000: from 206.124.146.176 lookup Blarg <=== This and the next rule are generated unless
|
||
20256: from 24.12.22.33 lookup Comcast 'loose' is specified; based in the output of 'ip addr ls'
|
||
32766: from all lookup main <=== This is the routing table shown by 'iproute -n'
|
||
32767: from all lookup default <=== This table is usually empty
|
||
gateway:~ #</programlisting>
|
||
|
||
<para>In the above example, there are two providers: Blarg and Comcast
|
||
with MARK 1 going to Blarg and mark 2 going to Comcast.</para>
|
||
</section>
|
||
|
||
<section id="route_rules_columns">
|
||
<title>Columns in the route_rules file</title>
|
||
|
||
<para>Columns in the file are:</para>
|
||
|
||
<variablelist>
|
||
<varlistentry>
|
||
<term>SOURCE (Optional)</term>
|
||
|
||
<listitem>
|
||
<para>An ip address (network or host) that matches the source IP
|
||
address in a packet. May also be specified as an interface name
|
||
optionally followed by ":" and an address. If the device 'lo' is
|
||
specified, the packet must originate from the firewall
|
||
itself.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>DEST (Optional)</term>
|
||
|
||
<listitem>
|
||
<para>An ip address (network or host) that matches the
|
||
destination IP address in a packet.</para>
|
||
|
||
<para>If you choose to omit either SOURCE or DEST, place "-" in
|
||
that column. Note that you may not omit both SOURCE and
|
||
DEST.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>PROVIDER</term>
|
||
|
||
<listitem>
|
||
<para>The provider to route the traffic through. May be
|
||
expressed either as the provider name or the provider
|
||
number.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>PRIORITY</term>
|
||
|
||
<listitem>
|
||
<para>The rule's priority which determines the order in which
|
||
the rules are processed.</para>
|
||
|
||
<para>1000-1999 Before Shorewall-generated 'MARK' rules</para>
|
||
|
||
<para>11000- 11999 After 'MARK' rules but before
|
||
Shorewall-generated rules for ISP interfaces.</para>
|
||
|
||
<para>26000-26999 After ISP interface rules but before 'default'
|
||
rule.</para>
|
||
|
||
<para>Rules with equal priority are applied in the order in
|
||
which they appear in the file.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
</variablelist>
|
||
</section>
|
||
|
||
<section>
|
||
<title>Multi-ISP and VPN</title>
|
||
|
||
<para>For those VPN types that use routing to direct traffic to remote
|
||
VPN clients (including but not limited to OpenVPN in routed mode and
|
||
PPTP), the VPN software adds a host route to the <emphasis
|
||
role="bold">main</emphasis> table for each VPN client. The best
|
||
approach is to use USE_DEFAULT_RT=Yes as described <link
|
||
linkend="USE_DEFAULT_RT">below</link>. If that isn't possible, you
|
||
must add a routing rule in the 1000-1999 range to specify the
|
||
<emphasis role="bold">main</emphasis> table for traffic addressed to
|
||
those clients. See<link linkend="Openvpn"> Example 2</link>
|
||
below.</para>
|
||
|
||
<para>If you have an IPSEC gateway on your firewall, be sure to
|
||
arrange for ESP packets to be routed out of the same interface that
|
||
you have configured your keying daemon to use.</para>
|
||
</section>
|
||
|
||
<section id="Examples">
|
||
<title>Examples</title>
|
||
|
||
<para><emphasis role="bold">Example 1:</emphasis> You want all traffic
|
||
entering the firewall on eth1 to be routed through Comcast.</para>
|
||
|
||
<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
||
eth1 - Comcast 1000</programlisting>
|
||
|
||
<para>With this entry, the output of <command>ip rule ls</command>
|
||
would be as follows.</para>
|
||
|
||
<para><programlisting>gateway:~ # <command>ip rule ls</command>
|
||
0: from all lookup local
|
||
<emphasis role="bold">1000: from all iif eth1 lookup Comcast</emphasis>
|
||
10001: from all fwmark 0x1 lookup Blarg
|
||
10002: from all fwmark 0x2 lookup Comcast
|
||
20000: from 206.124.146.176 lookup Blarg
|
||
20256: from 24.12.22.33 lookup Comcast
|
||
32766: from all lookup main
|
||
32767: from all lookup default
|
||
gateway:~ #</programlisting>Note that because we used a priority of 1000, the
|
||
test for <filename class="devicefile">eth1</filename> is inserted
|
||
before the fwmark tests.</para>
|
||
|
||
<para id="Openvpn"><emphasis role="bold">Example 2:</emphasis> You use
|
||
OpenVPN (routed setup w/tunX) in combination with multiple providers.
|
||
In this case you have to set up a rule to ensure that the OpenVPN
|
||
traffic is routed back through the tunX interface(s) rather than
|
||
through any of the providers. 10.8.0.0/24 is the subnet chosen in your
|
||
OpenVPN configuration (server 10.8.0.0 255.255.255.0).</para>
|
||
|
||
<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
||
- 10.8.0.0/24 main 1000</programlisting>
|
||
</section>
|
||
</section>
|
||
|
||
<section id="routes">
|
||
<title>/etc/shorewall/routes File</title>
|
||
|
||
<para>Beginning with Shorewall 4.4.15, additional routes can be added to
|
||
the provider routing tables using the /etc/shorewall/routes file.</para>
|
||
|
||
<para>The columns in the file are as follows.</para>
|
||
|
||
<variablelist>
|
||
<varlistentry>
|
||
<term><emphasis role="bold">PROVIDER</emphasis></term>
|
||
|
||
<listitem>
|
||
<para>The name or number of a provider defined in <ulink
|
||
url="shorewall-providers.html">shorewall-providers</ulink>
|
||
(5).</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term><emphasis role="bold">DEST</emphasis></term>
|
||
|
||
<listitem>
|
||
<para>Destination host address or network address.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term><emphasis role="bold">GATEWAY</emphasis> (Optional)</term>
|
||
|
||
<listitem>
|
||
<para>If specified, gives the IP address of the gateway to the
|
||
DEST.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term><emphasis role="bold">DEVICE</emphasis> (Optional)</term>
|
||
|
||
<listitem>
|
||
<para>Specifies the device route. If neither DEVICE nor GATEWAY is
|
||
given, then the INTERFACE specified for the PROVIDER in <ulink
|
||
url="manpages/shorewall-providers.html">shorewall-providers</ulink>
|
||
(5).</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
</variablelist>
|
||
|
||
<para>Assume the following entry in
|
||
<filename>/etc/shorewall/providers</filename>:</para>
|
||
|
||
<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
||
Comcast 1 - xxx eth2 .... </programlisting>
|
||
|
||
<para>The following table gives some example entries in the file and the
|
||
<command>ip route</command> command which results.</para>
|
||
|
||
<programlisting><emphasis role="bold">#PROVIDER DEST GATEWAY DEVICE</emphasis> | <emphasis
|
||
role="bold"> Generated Command</emphasis>
|
||
Comcast 172.20.1.0/24 - eth0 | ip -4 route add 172.20.1.0/24 dev eth0 table 1
|
||
Comcast 192.168.4.0/24 172.20.1.1 | ip -4 route add 192.168.1.0/24 via 172.20.1.1 table 1
|
||
Comcast 192.168.4.0/24 | ip -4 route add 192.168.4.0/24 dev eth2 table 1 </programlisting>
|
||
</section>
|
||
|
||
<section>
|
||
<title>Looking at the routing tables</title>
|
||
|
||
<para>To look at the various routing tables, you must use the <emphasis
|
||
role="bold">ip</emphasis> utility. To see the entire routing
|
||
configuration (including rules), the command is <command>shorewall show
|
||
routing</command>. To look at an individual provider's table use
|
||
<command>ip route ls table <replaceable>provider</replaceable></command>
|
||
where <replaceable>provider</replaceable> can be either the provider
|
||
name or number.</para>
|
||
|
||
<para>Example:</para>
|
||
|
||
<programlisting>lillycat:- #<command>ip route ls</command>
|
||
144.77.167.142 dev ppp0 proto kernel scope link src 144.177.121.199
|
||
71.190.227.208 dev ppp1 proto kernel scope link src 71.24.88.151
|
||
192.168.7.254 dev eth1 scope link src 192.168.7.1
|
||
192.168.7.253 dev eth1 scope link src 192.168.7.1
|
||
192.168.7.0/24 dev eth1 proto kernel scope link src 192.168.7.1
|
||
192.168.5.0/24 via 192.168.4.2 dev eth0
|
||
192.168.4.0/24 dev eth0 proto kernel scope link src 192.168.4.223
|
||
192.168.1.0/24 via 192.168.4.222 dev eth0
|
||
default
|
||
nexthop dev ppp1 weight 2
|
||
nexthop dev ppp0 weight 1
|
||
lillycat: #ip <command>route ls table 1</command>
|
||
144.77.167.142 dev ppp0 proto kernel scope link src 144.177.121.199
|
||
192.168.5.0/24 via 192.168.4.2 dev eth0
|
||
192.168.4.0/24 dev eth0 proto kernel scope link src 192.168.4.223
|
||
192.168.1.0/24 via 192.168.4.222 dev eth0
|
||
default dev ppp0 scope link
|
||
lillycat: #</programlisting>
|
||
</section>
|
||
|
||
<section id="USE_DEFAULT_RT">
|
||
<title>USE_DEFAULT_RT</title>
|
||
|
||
<para>USE_DEFAULT_RT is an option in <ulink
|
||
url="manpages/shorewall.conf.html">shorewall.conf</ulink> (5).</para>
|
||
|
||
<para>One of the drawbacks of the Multi-ISP support as described in the
|
||
preceding sections is that changes to the main table made by
|
||
applications are not added to the individual provider tables. This makes
|
||
route rules such as described in <link linkend="Openvpn">one of the
|
||
examples above</link> necessary.</para>
|
||
|
||
<para>USE_DEFAULT_RT=Yes works around that problem by passing packets
|
||
through the main table first rather than last. This has a number of
|
||
implications:</para>
|
||
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>Both the DUPLICATE and the COPY columns in the providers file
|
||
must remain empty or contain "-". The individual provider routing
|
||
tables generated when USE_DEFAULT_RT=Yes contain only a host route
|
||
to the gateway and a default route via the gateway.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>The <emphasis role="bold">balance</emphasis> option is assumed
|
||
for all interfaces that do not have the <emphasis
|
||
role="bold">loose</emphasis> option. When you want both <emphasis
|
||
role="bold">balance</emphasis> and <emphasis
|
||
role="bold">loose</emphasis>, both must be specified.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>The default route generated by Shorewall is added to the
|
||
<emphasis>default</emphasis> routing table (253) rather than to the
|
||
main routing table (254).</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Packets are sent through the main routing table by a routing
|
||
rule with priority 999. The priority range 1-998 may be used for
|
||
inserting rules that bypass the main table.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>You should disable all default route management outside of
|
||
Shorewall. If a default route is inadvertently added to the main
|
||
table while Shorewall is started, then all policy routing will stop
|
||
working except for those routing rules in the priority range
|
||
1-998.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>For ppp interfaces, the GATEWAY may remain unspecified ("-").
|
||
For those interfaces managed by dhcpcd or dhclient, you may specify
|
||
'detect' in the GATEWAY column; Shorewall will use the dhcp client's
|
||
database to determine the gateway IP address. All other interfaces
|
||
must have a GATEWAY specified explicitly.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
|
||
<para>Although 'balance' is automatically assumed when
|
||
USE_DEFAULT_RT=Yes, you can easily cause all traffic to use one provider
|
||
except when you explicitly direct it to use the other provider via
|
||
<ulink
|
||
url="manpages/shorewall-route_rules.html">shorewall-route_rules</ulink>
|
||
(5) or <ulink
|
||
url="manpages/shorewall-tcrules.html">shorewall-tcrules</ulink>
|
||
(5).</para>
|
||
|
||
<para>Example (send all traffic through the 'shorewall' provider unless
|
||
otherwise directed).</para>
|
||
|
||
<para>/etc/shorewall/providers:<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS
|
||
linksys 1 1 - wlan0 172.20.1.1 track,balance=1,optional
|
||
shorewall 2 2 - eth0 192.168.1.254 track,balance=2,optional</programlisting>/etc/shorewall/route_rules:<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
||
- - shorewall 11999</programlisting></para>
|
||
|
||
<para>Tuomo Soini describes the following issue when using
|
||
USE_DEFAULT_RT=Yes.</para>
|
||
|
||
<para>He has a /27 network (let.s call it 70.90.191.0/27) from his
|
||
primary ISP and his secondary ISP supplies him with a dynamic IP address
|
||
on the 91.156.0.0/19 network. From the output of <command>shorewall show
|
||
routing</command>:</para>
|
||
|
||
<programlisting>999: from all lookup main
|
||
10000: from all fwmark 0x100 lookup ISP1
|
||
10001: from all fwmark 0x200 lookup ISP2</programlisting>
|
||
|
||
<para>Note that the main routing table is consulted prior to the marks
|
||
for his two provlders. When clients in the large /19 network connected
|
||
to his /27 (through ISP1), the responses were routed out of the ISP2
|
||
interface because the main routing table included a route to the
|
||
/19.</para>
|
||
|
||
<para>The solution was to add an additional entry to route_rules:</para>
|
||
|
||
<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
||
70.90.191.0/27 91.156.0.0/19 ISP1 900</programlisting>
|
||
|
||
<para>With this additional entry, the routing rules are as below and
|
||
traffic from the /27 is returned via ISP1.</para>
|
||
|
||
<programlisting>900: from 70.90.191.0/27 to 91.156.0.0/19 lookup ISP1
|
||
999: from all lookup main
|
||
10000: from all fwmark 0x100 lookup ISP1
|
||
10001: from all fwmark 0x200 lookup ISP2</programlisting>
|
||
</section>
|
||
|
||
<section id="LinkMonitor">
|
||
<title>Gateway Monitoring and Failover</title>
|
||
|
||
<para>There are a couple of options available for monitoring the status
|
||
of provider links and taking action when a failure occurs. Both of these
|
||
options assume that each provider has a unique nexthop gateway; if two
|
||
or more providers use the same gateway router then neither option is
|
||
suitable.</para>
|
||
|
||
<para>You specify the <option>optional</option> option in
|
||
<filename>/etc/shorewall/interfaces</filename>:</para>
|
||
|
||
<programlisting>#ZONE INTERFACE BROADCAST OPTIONS
|
||
net eth0 detect <emphasis role="bold">optional</emphasis>
|
||
net eth1 detect <emphasis role="bold">optional</emphasis></programlisting>
|
||
|
||
<section id="swping">
|
||
<title>SWPING</title>
|
||
|
||
<para>Shorewall includes a sample monitoring script
|
||
<filename>swping</filename>. The <filename>swping</filename> file is
|
||
available in the main directory contained in the Shorewall-common
|
||
tarball and is included in the Shorewall-common documentation
|
||
directory in the Shorewall-common RPM. The script is inspired by
|
||
Angsuman Chakraborty's <ulink
|
||
url="http://blog.taragana.com/index.php/archive/how-to-load-balancing-failover-with-dual-multi-wan-adsl-cable-connections-on-linux/">gwping</ulink>
|
||
script.</para>
|
||
|
||
<important>
|
||
<para>These samples are offered <emphasis>as is</emphasis> — they
|
||
work for me but I don't make any claim that they will work for
|
||
anyone else. But if you have a need for automated link monitoring,
|
||
they offer you a place to start.</para>
|
||
</important>
|
||
|
||
<important>
|
||
<para>If you have installed Shorewall-init, you should disable its
|
||
ifup/ifdown/NetworkManager integration (set IFUPDOWN=0 in the <ulink
|
||
url="Manpages/shorewall-init.html">Shorewall-init configuration
|
||
file</ulink>).</para>
|
||
</important>
|
||
|
||
<para>The script should be copied to a directory on root's PATH such
|
||
as <filename>/usr/local/sbin/</filename>.</para>
|
||
|
||
<para>The script works by sending pings to <emphasis>target</emphasis>
|
||
IP addresses through each external interface. These targets must not
|
||
depend on any routes other than those that are present in the main
|
||
routing table. That ensures that a route is available to the target
|
||
even when the target's interface is not working and Shorewall has
|
||
omitted it from the routing configuration. An interface is assumed to
|
||
be <firstterm>up</firstterm> when a specified number (UP_COUNT) of
|
||
consecutive ping operations succeed. Similarly, an interface is
|
||
assumed to be <firstterm>down</firstterm> when a specified number
|
||
(DOWN_COUNT) of consecutive ping operations fail. You can specify the
|
||
interval between pings (PING_INTERVAL).</para>
|
||
|
||
<para>The script monitors two interfaces but it is a trivial exercise
|
||
to extend it to more than two. At the top are a number of variables to
|
||
set:</para>
|
||
|
||
<programlisting>#
|
||
# IP family -- 4 or 6
|
||
#
|
||
FAMILY=4
|
||
#
|
||
# The commands to run when the status of a line changes. Multiple commands may be specified
|
||
# when separated by semicolons (";")
|
||
#
|
||
COMMAND=
|
||
|
||
...
|
||
#
|
||
# Interfaces to monitor -- you may use shell variables from your params file
|
||
#
|
||
IF1=eth0
|
||
IF2=eth1
|
||
#
|
||
# Sites to Ping. Must depend only on routes in the 'main' routing table. If not specified,
|
||
# the interface is assumed to be managed by dhcpcd and the script uses the gateway address
|
||
# from /var/lib/dhcpcd/dhcpcd-${IFx}.info
|
||
#
|
||
TARGET1=
|
||
TARGET2=
|
||
#
|
||
# How often to ping
|
||
#
|
||
PING_INTERVAL=5
|
||
#
|
||
# Value for ping's -W option
|
||
#
|
||
PING_TIMEOUT=2
|
||
#
|
||
# This many successive pings must succeed for the interface to be marked up when it is down
|
||
#
|
||
UP_COUNT=5
|
||
#
|
||
# This many successive pings must fail for the interface to be marked down when it is up
|
||
#
|
||
DOWN_COUNT=2</programlisting>
|
||
|
||
<para>If you leave COMMAND empty, the script sets its value
|
||
automatically depending on whether Shorewall-lite is installed.</para>
|
||
|
||
<para>When the status of an interface changes:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>For each interface, a file is placed in ${VARDIR} (normally
|
||
/var/lib/shorewall) to record the status of the interface: either
|
||
0 (UP) or 1 (DOWN). The name of the file is
|
||
<filename><replaceable>interface</replaceable>.status</filename>
|
||
where <replaceable>interface</replaceable> is the interface (e.g.,
|
||
<filename>eth0.status</filename>).</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>A <command>shorewall -f restart</command> command is
|
||
executed (<command>shorewall-lite restart</command>, if
|
||
Shorewall-lite is installed).</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>The contents of the main routing table are displayed.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
|
||
<para>The .status files are intended to be used with the following
|
||
<filename>/etc/shorewall/isusable</filename> script.<programlisting>local status=0
|
||
|
||
[ -f ${VARDIR}/${1}.status ] && status=$(cat ${VARDIR}/${1}.status)
|
||
|
||
return $status</programlisting></para>
|
||
|
||
<para>The above script is installed in <filename
|
||
class="directory">/etc/shorewall</filename>, beginning with Shorewall
|
||
4.3.11.</para>
|
||
|
||
<para>Also included is a sample init script
|
||
(<filename>swping.init</filename>) to start the monitoring daemon.
|
||
Copy it to<filename> /etc/init.d/swping</filename> and use your
|
||
distribution's SysV init tools to cause it to be run at boot. It works
|
||
on <trademark>OpenSuSE</trademark> 11.0 -- YMMV. Modify the PROG and
|
||
STATEDIR variables as needed.</para>
|
||
|
||
<para>As an alternative to using the init script, you can add the
|
||
following to <filename>/etc/shorewall/started</filename>:</para>
|
||
|
||
<programlisting>if [ "$COMMAND" = start ]; then
|
||
killall -9 swping 2> /dev/null #be sure that there are none left running
|
||
/usr/local/sbin/swping &
|
||
fi</programlisting>
|
||
|
||
<para>and add this to
|
||
<filename>/etc/shorewall/stopped</filename>.</para>
|
||
|
||
<para><programlisting>if [ "$COMMAND" = stop -o "$COMMAND" = clear ]; then
|
||
killall -9 swping 2> /dev/null
|
||
fi</programlisting></para>
|
||
|
||
<para>This simple script has a number of limitations:</para>
|
||
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>It only works on IPv4 or IPv6 but not both at once. So if
|
||
you want to monitor both IPv4 and IPv6, you need to clone the
|
||
script are run two copies; one for IPv4 and one for IPv6.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>It can only detect the gateway for interfaces managed by
|
||
dhcpcd.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>It's method of determining whether an interface is up or
|
||
down is crude. You will normally specify the default gateway for
|
||
each provider as the sites to ping and being able to ping the
|
||
default gateway is not a surefire indication that the provider is
|
||
usable. The method of determining whether a site is up or down is
|
||
also crude.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Because of the crudeness of the algorithm, hysteresis may
|
||
occur.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>It is tricky to configure a system such that the system
|
||
works correctly when one of its providers is down unless you
|
||
largely don't care which interface is used.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
</section>
|
||
|
||
<section id="lsm">
|
||
<title>Link Status Monitor (LSM)</title>
|
||
|
||
<para><ulink url="http://lsm.foobar.fi/">Link Status Monitor</ulink>
|
||
was written by Mika Ilmaranta <ilmis at nullnet.fi> and performs
|
||
more sophisticated monitoring than the simple swping script described
|
||
in the preceding section.</para>
|
||
|
||
<important>
|
||
<para>If you have installed Shorewall-init, you should disable its
|
||
ifup/ifdown/NetworkManager integration (set IFUPDOWN=0 in the <ulink
|
||
url="Manpages/shorewall-init.html">Shorewall-init configuration
|
||
file</ulink>) before installing LSM.</para>
|
||
</important>
|
||
|
||
<para>Like many Open Source products, LSM is poorly documented. It's
|
||
main configuration file is normally kept in
|
||
<filename>/etc/lsm/lsm.conf</filename>, but the file's name is passed
|
||
as an argument to the lsm program so you can name it anything you
|
||
want.</para>
|
||
|
||
<para>The sample <filename>lsm.conf</filename> included with the
|
||
product shows some of the possibilities for configuration. One feature
|
||
that is not mentioned in the sample is that an "include" directive is
|
||
supported. This allows additional files to be sourced in from the main
|
||
configuration file.</para>
|
||
|
||
<para>LSM monitors the status of the links defined in its
|
||
configuration file and runs a user-provided script when the status of
|
||
a link changes. The script name is specified in the
|
||
<firstterm>eventscript</firstterm> option in the configuration file.
|
||
Key arguments to the script are as follows:</para>
|
||
|
||
<variablelist>
|
||
<varlistentry>
|
||
<term>$1</term>
|
||
|
||
<listitem>
|
||
<para>The state of the link ('up' or 'down')</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>$2</term>
|
||
|
||
<listitem>
|
||
<para>The name of the connection as specified in the
|
||
configuration file.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>$4</term>
|
||
|
||
<listitem>
|
||
<para>The name of the network interface associated with the
|
||
connection.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
|
||
<varlistentry>
|
||
<term>$5</term>
|
||
|
||
<listitem>
|
||
<para>The email address of the person specified to receive
|
||
notifications. Specified in the
|
||
<firstterm>warn_email</firstterm> option in the configuration
|
||
file.</para>
|
||
</listitem>
|
||
</varlistentry>
|
||
</variablelist>
|
||
|
||
<para>It is the responsibility of the script to perform any action
|
||
needed in reaction to the connection state change. The default script
|
||
supplied with LSM composes an email and sends it to $5.</para>
|
||
|
||
<para>I personally use LSM here at shorewall.net (configuration is
|
||
described <link linkend="Complete">below</link>). I have set things up
|
||
so that:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Shorewall [re]starts lsm during processing of the
|
||
<command>start</command> and <command>restore</command> commands.
|
||
I don't have Shorewall restart lsm during Shorewall
|
||
<command>restart</command> because I restart Shorewall much more
|
||
often than the average user is likely to do.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Shorewall starts lsm because I have a dynamic IP address
|
||
from one of my providers (Comcast); Shorewall detects the default
|
||
gateway to that provider and creates a secondary configuration
|
||
file (<filename>/etc/lsm/shorewall.conf</filename>) that contains
|
||
the link configurations. That file is included by
|
||
<filename>/etc/lsm/lsm.conf</filename>.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>The script run by LSM during state change
|
||
(<filename>/etc/lsm/script) </filename>writes a<filename>
|
||
${VARDIR}/xxx.status</filename> file when the status of an
|
||
interface changes. Those files are read by the
|
||
<filename>isusable</filename> extension script (see below).</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
|
||
<para>Below are my relevant configuration files.</para>
|
||
|
||
<warning>
|
||
<para>These files only work with Shorewall-perl 4.4 Beta 2 and
|
||
later.</para>
|
||
</warning>
|
||
|
||
<para><filename>/etc/shorewall/isusable</filename>:</para>
|
||
|
||
<programlisting>local status=0
|
||
#
|
||
# Read the status file (if any) created by /etc/lsm/script
|
||
#
|
||
[ -f ${VARDIR}/${1}.status ] && status=$(cat ${VARDIR}/${1}.status)
|
||
|
||
return $status</programlisting>
|
||
|
||
<para><filename>/etc/shorewall/lib.private</filename>:</para>
|
||
|
||
<programlisting>###############################################################################
|
||
# Create /etc/lsm/shorewall.conf
|
||
# Remove the current interface status files
|
||
# Start lsm
|
||
###############################################################################
|
||
start_lsm() {
|
||
#
|
||
# Kill any existing lsm process(es)
|
||
#
|
||
killall lsm 2> /dev/null
|
||
#
|
||
# Create the Shorewall-specific part of the LSM configuration. This file is
|
||
# included by /etc/lsm/lsm.conf
|
||
#
|
||
# Avvanta has a static gateway while Comcast's is dynamic
|
||
#
|
||
cat <<EOF > /etc/lsm/shorewall.conf
|
||
connection {
|
||
name=Avvanta
|
||
checkip=206.124.146.254
|
||
device=$EXT_IF
|
||
ttl=2
|
||
}
|
||
|
||
connection {
|
||
name=Comcast
|
||
checkip=${SW_ETH0_GATEWAY:-71.231.152.1}
|
||
device=$COM_IF
|
||
ttl=1
|
||
}
|
||
EOF
|
||
#
|
||
# Since LSM assumes that interfaces start in the 'up' state, remove any
|
||
# existing status files that might have an interface in the down state
|
||
#
|
||
rm -f /var/lib/shorewall/*.status
|
||
#
|
||
# Run LSM -- by default, it forks into the background
|
||
#
|
||
/usr/sbin/lsm /etc/lsm/lsm.conf >> /var/log/lsm
|
||
}</programlisting>
|
||
|
||
<para>eth0 has a dynamic IP address so I need to use the
|
||
Shorewall-detected gateway address ($SW_ETH1_GATEWAY). I supply a
|
||
default value to be used in the event that detection fails.</para>
|
||
|
||
<note>
|
||
<para>In Shorewall 4.4.7 and earlier, the variable name is
|
||
ETH1_GATEWAY.</para>
|
||
</note>
|
||
|
||
<para><filename>/etc/shorewall/started</filename>:</para>
|
||
|
||
<programlisting>##################################################################################
|
||
# [re]start lsm if this is a 'start' command or if lsm isn't running
|
||
##################################################################################
|
||
if [ "$COMMAND" = start -o -z "$(ps ax | grep 'lsm ' | grep -v 'grep ' )" ]; then
|
||
start_lsm
|
||
fi</programlisting>
|
||
|
||
<para><filename>/etc/shorewall/restored</filename>:</para>
|
||
|
||
<programlisting>##################################################################################
|
||
# Start lsm if it isn't running
|
||
##################################################################################
|
||
if [ -z "$(ps ax | grep 'lsm ' | grep -v 'grep ' )" ]; then
|
||
start_lsm
|
||
fi</programlisting>
|
||
|
||
<para><filename>/etc/lsm/lsm.conf</filename>:</para>
|
||
|
||
<programlisting>#
|
||
# Defaults for the connection entries
|
||
#
|
||
defaults {
|
||
name=defaults
|
||
checkip=127.0.0.1
|
||
eventscript=/etc/lsm/script
|
||
max_packet_loss=20
|
||
max_successive_pkts_lost=7
|
||
min_packet_loss=5
|
||
min_successive_pkts_rcvd=10
|
||
interval_ms=2000
|
||
timeout_ms=2000
|
||
warn_email=teastep@shorewall.net
|
||
check_arp=0
|
||
sourceip=
|
||
ttl=0
|
||
}
|
||
|
||
include /etc/lsm/shorewall.conf</programlisting>
|
||
|
||
<para><filename>/etc/lsm/script</filename><programlisting>#!/bin/sh
|
||
#
|
||
# (C) 2009 Mika Ilmaranta <ilmis@nullnet.fi>
|
||
# (C) 2009 Tom Eastep <teastep@shorewall.net>
|
||
#
|
||
# License: GPLv2
|
||
#
|
||
|
||
STATE=${1}
|
||
NAME=${2}
|
||
CHECKIP=${3}
|
||
DEVICE=${4}
|
||
WARN_EMAIL=${5}
|
||
REPLIED=${6}
|
||
WAITING=${7}
|
||
TIMEOUT=${8}
|
||
REPLY_LATE=${9}
|
||
CONS_RCVD=${10}
|
||
CONS_WAIT=${11}
|
||
CONS_MISS=${12}
|
||
AVG_RTT=${13}
|
||
|
||
if [ -f /usr/share/shorewall-lite/lib.base ]; then
|
||
VARDIR=/var/lib/shorewall-lite
|
||
STATEDIR=/etc/shorewall-lite
|
||
else
|
||
VARDIR=/var/lib/shorewall
|
||
STATEDIR=/etc/shorewall
|
||
fi
|
||
|
||
[ -f ${STATEDIR}/vardir ] && . ${STATEDIR}/vardir
|
||
|
||
cat <<EOM | mail -s "${NAME} ${STATE}, DEV ${DEVICE}" ${WARN_EMAIL}
|
||
|
||
Hi,
|
||
|
||
Connection ${NAME} is now ${STATE}.
|
||
|
||
Following parameters were passed:
|
||
newstate = ${STATE}
|
||
name = ${NAME}
|
||
checkip = ${CHECKIP}
|
||
device = ${DEVICE}
|
||
warn_email = ${WARN_EMAIL}
|
||
|
||
Packet counters:
|
||
replied = ${REPLIED} packets replied
|
||
waiting = ${WAITING} packets waiting for reply
|
||
timeout = ${TIMEOUT} packets that have timed out (= packet loss)
|
||
reply_late = ${REPLY_LATE} packets that received a reply after timeout
|
||
cons_rcvd = ${CONS_RCVD} consecutively received replies in sequence
|
||
cons_wait = ${CONS_WAIT} consecutive packets waiting for reply
|
||
cons_miss = ${CONS_MISS} consecutive packets that have timed out
|
||
avg_rtt = ${AVG_RTT} average rtt, notice that waiting and timed out packets have rtt = 0 when calculating this
|
||
|
||
Your LSM Daemon
|
||
|
||
EOM
|
||
|
||
[ ${STATE} = up ] && state=0 || state=1
|
||
|
||
echo $state > ${VARDIR}/${DEVICE}.status
|
||
|
||
/sbin/shorewall restart -f >> /var/log/lsm 2>&1
|
||
|
||
/sbin/shorewall show routing >> /var/log/lsm
|
||
|
||
exit 0
|
||
|
||
#EOF</programlisting>:</para>
|
||
</section>
|
||
</section>
|
||
|
||
<section id="Shared">
|
||
<title>Two Providers Sharing an Interface</title>
|
||
|
||
<para>Shared interface support has the following characteristics:</para>
|
||
|
||
<orderedlist numeration="loweralpha">
|
||
<listitem>
|
||
<para>Only Ethernet (or Ethernet-like) interfaces can be used. For
|
||
inbound traffic, the MAC addresses of the gateway routers are used
|
||
to determine which provider a packet was received through. Note that
|
||
only routed traffic can be categorized using this technique.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>You must specify the address on the interface that corresponds
|
||
to a particular provider in the INTERFACE column by following the
|
||
interface name with a colon (":") and the address.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Entries in <filename>/etc/shorewall/masq</filename> must be
|
||
qualified by the provider name (or number).</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>This feature requires Realm Match support in your kernel and
|
||
iptables.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>You must add route_rules entries for networks that are
|
||
accessed through a particular provider.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>If you have additional IP addresses through either provider,
|
||
you must add <filename>route_rules</filename> to direct traffic FROM
|
||
each of those addresses through the appropriate provider.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>You must manually add MARK rules for traffic known to come
|
||
from each provider.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>You must specify a gateway IP address in the GATEWAY column
|
||
of<filename> /etc/shorewall/providers</filename>; <emphasis
|
||
role="bold">detect</emphasis> is not permitted.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
|
||
<para>Taken together, b. and h. effectively preclude using this
|
||
technique with dynamic IP addresses.</para>
|
||
|
||
<para>Example:</para>
|
||
|
||
<para>This is our home network circa fall 2008. We have two Internet
|
||
providers:</para>
|
||
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>Comcast -- Cable modem with one dynamic IP address.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Avvanta -- ADSL with 5 static IP addresses.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
|
||
<para>Because the old <trademark>Compaq</trademark>
|
||
<trademark>Presario</trademark> that I use for a firewall only has three
|
||
PCI slots and no onboard Ethernet, it doesn't have enough Ethernet
|
||
controllers to support both providers. So I use a Linksys WRT300n pre-N
|
||
router as a gateway to Comcast. Note that because the Comcast IP address
|
||
is dynamic, I could not share a single firewall interface between the
|
||
two providers directly.</para>
|
||
|
||
<para>On my personal laptop (ursa), I have 9 virtual machines running
|
||
various Linux distributions. <emphasis>It is the Shorewall configuration
|
||
on ursa that I will describe here</emphasis>.</para>
|
||
|
||
<para>Below is a diagram of our network:<graphic align="center"
|
||
fileref="images/Network2008a.png" /></para>
|
||
|
||
<para>The local wired network in my office is connected to both gateways
|
||
and uses the private (RFC 1918) network 172.20.1.0/24. The Comcast
|
||
gateway has local IP address 172.20.1.1 while the Avvanta gateway has
|
||
local IP address 172.20.1.254. Ursa's eth0 interface has a single IP
|
||
address (172.20.1.130).</para>
|
||
|
||
<para>This configuration uses USE_DEFAULT_RT=Yes in
|
||
<filename>shorewall.conf </filename>(see <link
|
||
linkend="USE_DEFAULT_RT">above</link>).</para>
|
||
|
||
<para>Here is the <filename>providers</filename> file:</para>
|
||
|
||
<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
||
comcast 1 1 - eth0:172.20.1.130 172.20.1.1 track,loose,balance,optional
|
||
avvanta 2 2 - eth0:172.20.1.130 172.20.1.254 track,optional,loose
|
||
wireless 3 3 - wlan0 172.20.1.1 track,optional</programlisting>
|
||
|
||
<para>Several things to note:</para>
|
||
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>172.20.1.130 is specified as the <filename
|
||
class="devicefile">eth0</filename> IP address for both
|
||
providers.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Both wired providers have the <emphasis
|
||
role="bold">loose</emphasis> option. This prevents Shorewall from
|
||
automatically generating routing rules based on the source IP
|
||
address.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Only <emphasis role="bold">comcast</emphasis> has the
|
||
<emphasis role="bold">balance</emphasis> option. With
|
||
USE_DEFAULT_RT=yes, that means that <emphasis
|
||
role="bold">comcast</emphasis> will be the default provider. While
|
||
<emphasis role="bold">balance</emphasis> is the default, with
|
||
USE_DEFAULT_RT=Yes, it must be specified explicitly when <emphasis
|
||
role="bold">loose</emphasis> is also specified.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>I always disable the <emphasis role="bold">wireless</emphasis>
|
||
interface when the laptop is connected to the wired network.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>I use a different Shorewall configuration when I take the
|
||
laptop on the road.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
|
||
<para>Here is the route_rules file:<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
||
- 206.124.146.176/31 avvanta 1000
|
||
- 206.124.146.178/31 avvanta 1000
|
||
- 206.124.146.180/32 avvanta 1000</programlisting></para>
|
||
|
||
<para>Those rules direct traffic to the five static Avvanta IP addresses
|
||
(only two are currently used) through the <emphasis
|
||
role="bold">avvanta</emphasis> provider.</para>
|
||
|
||
<para>Here is the tcrules file (MARK_IN_FORWARD_CHAIN=No in
|
||
<filename>shorewall.conf</filename>):<programlisting>#MARK SOURCE DEST PROTO PORT(S) CLIENT USER TEST LENGTH TOS CONNBYTES HELPER
|
||
# PORT(S)
|
||
2 $FW 0.0.0.0/0 tcp 21
|
||
2 $FW 0.0.0.0/0 tcp - - - - - - - ftp
|
||
2 $FW 0.0.0.0/0 tcp 119</programlisting></para>
|
||
|
||
<para>These rules:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Use <emphasis role="bold">avvanta</emphasis> for FTP.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Use <emphasis role="bold">avvanta</emphasis> for NTTP</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
|
||
<para>The remaining files are for a rather standard two-interface config
|
||
with a bridge as the local interface.</para>
|
||
|
||
<para><filename>zones</filename>:<programlisting>#ZONE IPSEC OPTIONS IN OUT
|
||
# ONLY OPTIONS OPTIONS
|
||
fw firewall
|
||
net ipv4
|
||
kvm ipv4</programlisting><filename>policy</filename>:<programlisting>net net NONE
|
||
fw net ACCEPT
|
||
fw kvm ACCEPT
|
||
kvm all ACCEPT
|
||
net all DROP info
|
||
all all REJECT info</programlisting></para>
|
||
|
||
<para>interfaces:<programlisting>#ZONE INTERFACE BROADCAST OPTIONS GATEWAY
|
||
#
|
||
net eth0 detect dhcp,tcpflags,routefilter,blacklist,logmartians,optional,arp_ignore
|
||
net wlan0 detect dhcp,tcpflags,routefilter,blacklist,logmartians,optional
|
||
kvm br0 detect routeback #Virtual Machines</programlisting><note>
|
||
<para><filename class="devicefile">wlan0</filename> is the wireless
|
||
adapter in the notebook. Used when the laptop is in our home but not
|
||
connected to the wired network.</para>
|
||
</note></para>
|
||
|
||
<para>masq:<programlisting>#INTERFACE SUBNET ADDRESS PROTO PORT(S) IPSEC
|
||
eth0 192.168.0.0/24
|
||
wlan0 192.168.0.0/24</programlisting><note>
|
||
<para>Because the firewall has only a single external IP address, I
|
||
don't need to specify the providers in the masq rules.</para>
|
||
</note></para>
|
||
</section>
|
||
</section>
|
||
|
||
<section id="Complete">
|
||
<title>A Complete Working Example</title>
|
||
|
||
<para>This section describes the network at shorewall.net early in 2009.
|
||
The configuration is as follows:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Two providers:</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Avvanta -- A slow (1.5mb/384kb) DSL service with 5 static IP
|
||
addresses.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Comcast -- A fast (20mb/10mb) Cable circuit with a single
|
||
<emphasis>dynamic</emphasis> address.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>A local network consisting of wired and wireless client systems.
|
||
A Linksys WRT300N wireless router is used as an access point for the
|
||
wireless hosts.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>A DMZ hosting a single server (lists.shorewall.net aka
|
||
www1.shorewall.net, ftp1.shorewall.net,etc.)</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
|
||
<para>The network is pictured in the following diagram:</para>
|
||
|
||
<graphic align="center" fileref="images/Network2009.png" />
|
||
|
||
<para>Because of the speed of the cable provider, all traffic uses that
|
||
provider unless there is a specific need for the traffic to use the DSL
|
||
line.</para>
|
||
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Responses to connections from the Internet to one of the DSL IP
|
||
addresses -- the <emphasis role="bold">track</emphasis> option takes
|
||
care of that.</para>
|
||
</listitem>
|
||
|
||
<listitem>
|
||
<para>Connections initiated by the server and connections requested by
|
||
clients on the firewall that have bound their local socket to one of
|
||
the DSL IP addresses. Two entries in
|
||
<filename>/etc/shorewall/route_rules</filename> take care of that
|
||
traffic.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
|
||
<para>As a consequence, I have disabled all route filtering on the
|
||
firewall and only use the <emphasis role="bold">balance</emphasis> option
|
||
in <filename>/etc/shorewall/providers</filename> on the Comcast provider
|
||
whose default route in the main table is established by DHCP. By
|
||
specifying the <emphasis role="bold">fallback</emphasis> option on
|
||
Avvanta, I ensure that there is still a default route if Comcast is down.
|
||
<link linkend="lsm">lsm</link> is used to monitor the links.</para>
|
||
|
||
<para><filename>/etc/sysctl.conf</filename>:</para>
|
||
|
||
<programlisting>net.ipv4.conf.all.rp_filter = 0</programlisting>
|
||
|
||
<para><filename>/etc/shorewall/shorewall.conf</filename>:</para>
|
||
|
||
<programlisting>ROUTE_FILTER=No
|
||
RESTORE_DEFAULT_ROUTE=No</programlisting>
|
||
|
||
<para>RESTORE_DEFAULT_ROUTE=No causes the default route in the main table
|
||
to be deleted when the Comcast link is unavailable. That way, the default
|
||
route in the default table will be used until Comcast is available
|
||
again.</para>
|
||
|
||
<para><filename>/etc/shorewall/providers</filename>:</para>
|
||
|
||
<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
||
Avvanta 1 0x100 main eth0 206.124.146.254 track,loose,fallback eth2,eth4,tun*
|
||
Comcast 2 0x200 main eth3 detect track,balance eth2,eth4,tun*
|
||
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE</programlisting>
|
||
|
||
<para>The <emphasis role="bold">loose</emphasis> option on Avvanta results
|
||
in fewer routing rules. The first two routing rules below insure that all
|
||
traffic from Avvanta-assigned IP addresses is sent via the Avvanta
|
||
provider. The 'tun*' included in the COPY column is there because I run a
|
||
routed OpenVPN server on the firewall.</para>
|
||
|
||
<para><filename>/etc/shorewall/route_rules</filename>:</para>
|
||
|
||
<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
||
- 172.20.0.0/24 main 1000 # Addresses assigned by routed OpenVPN server
|
||
206.124.146.176/30 - Avvanta 26000
|
||
206.124.146.180 - Avvanta 26000
|
||
- 216.168.3.44 Avvanta 26000 # Avvanta NNTP Server -- verifies source IP address
|
||
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
|
||
|
||
<para>The <filename>/etc/shorewall/route_rules </filename>entries provide
|
||
all of the provider selection necessary so my
|
||
<filename>/etc/shorewall/tcrules</filename> file is used exclusively for
|
||
traffic shaping of the Avvanta line. Note that I still need to provide
|
||
values in the MARK colum of <filename>/etc/shorewall/providers</filename>
|
||
because I specify <emphasis role="bold">track</emphasis> on both
|
||
providers.</para>
|
||
|
||
<para>Here is the output of <command>shorewall show
|
||
routing</command>:</para>
|
||
|
||
<programlisting>Routing Rules
|
||
|
||
0: from all lookup local
|
||
1000: from all to 172.20.0.0/24 lookup main
|
||
10000: from all fwmark 0x100 lookup Avvanta
|
||
10001: from all fwmark 0x200 lookup Comcast
|
||
20256: from 71.227.156.229 lookup Comcast
|
||
26000: from 206.124.146.176/30 lookup Avvanta
|
||
26000: from 206.124.146.180 lookup Avvanta
|
||
26000: from all to 216.168.3.44 lookup Avvanta
|
||
32766: from all lookup main
|
||
32767: from all lookup default
|
||
|
||
Table Avvanta:
|
||
|
||
206.124.146.254 dev eth0 scope link src 206.124.146.176
|
||
206.124.146.177 dev eth4 scope link
|
||
172.20.1.0/24 dev eth2 proto kernel scope link src 172.20.1.254
|
||
206.124.146.0/24 dev eth0 proto kernel scope link src 206.124.146.176
|
||
169.254.0.0/16 dev eth0 scope link
|
||
default via 206.124.146.254 dev eth0 src 206.124.146.176
|
||
|
||
Table Comcast:
|
||
|
||
206.124.146.177 dev eth4 scope link
|
||
71.227.156.1 dev eth3 scope link src 71.227.156.229
|
||
172.20.1.0/24 dev eth2 proto kernel scope link src 172.20.1.254
|
||
71.227.156.0/23 dev eth3 proto kernel scope link src 71.227.156.229
|
||
default via 71.227.156.1 dev eth3 src 71.227.156.229
|
||
|
||
Table default:
|
||
|
||
default via 206.124.146.254 dev eth0 metric 1
|
||
|
||
Table local:
|
||
|
||
broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1
|
||
broadcast 172.20.1.0 dev eth2 proto kernel scope link src 172.20.1.254
|
||
broadcast 206.124.146.255 dev eth0 proto kernel scope link src 206.124.146.176
|
||
local 206.124.146.179 dev eth0 proto kernel scope host src 206.124.146.176
|
||
local 206.124.146.178 dev eth0 proto kernel scope host src 206.124.146.176
|
||
local 206.124.146.176 dev eth0 proto kernel scope host src 206.124.146.176
|
||
local 206.124.146.176 dev eth4 proto kernel scope host src 206.124.146.176
|
||
broadcast 71.227.157.255 dev eth3 proto kernel scope link src 71.227.156.229
|
||
broadcast 71.227.156.0 dev eth3 proto kernel scope link src 71.227.156.229
|
||
local 172.20.1.254 dev eth2 proto kernel scope host src 172.20.1.254
|
||
local 127.0.0.2 dev lo proto kernel scope host src 127.0.0.1
|
||
broadcast 172.20.1.255 dev eth2 proto kernel scope link src 172.20.1.254
|
||
local 71.227.156.229 dev eth3 proto kernel scope host src 71.227.156.229
|
||
broadcast 206.124.146.0 dev eth0 proto kernel scope link src 206.124.146.176
|
||
broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1
|
||
local 206.124.146.180 dev eth0 proto kernel scope host src 206.124.146.176
|
||
local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1
|
||
local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1
|
||
|
||
Table main:
|
||
|
||
206.124.146.177 dev eth4 scope link
|
||
172.20.1.0/24 dev eth2 proto kernel scope link src 172.20.1.254
|
||
206.124.146.0/24 dev eth0 proto kernel scope link src 206.124.146.176
|
||
71.227.156.0/23 dev eth3 proto kernel scope link src 71.227.156.229
|
||
169.254.0.0/16 dev eth0 scope link
|
||
127.0.0.0/8 dev lo scope link
|
||
default via 71.227.156.1 dev eth3 </programlisting>
|
||
|
||
<para><filename>/etc/shorewall/interfaces</filename>:</para>
|
||
|
||
<programlisting>#ZONE INTERFACE BROADCAST OPTIONS
|
||
loc eth2 detect dhcp,routeback
|
||
dmz eth4 detect
|
||
net eth0 detect dhcp,blacklist,tcpflags,optional
|
||
net eth3 detect dhcp,blacklist,tcpflags,optional
|
||
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
|
||
|
||
<para><filename>/etc/shorewall/masq</filename>:</para>
|
||
|
||
<programlisting>#INTERFACE SOURCE ADDRESS PROTO PORT(S) IPSEC
|
||
|
||
COMMENT Masquerade Local Network
|
||
eth3 0.0.0.0/0
|
||
eth0 !206.124.146.0/24 206.124.146.179
|
||
|
||
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE</programlisting>
|
||
|
||
<para>All traffic leaving eth3 must use the dynamic IP address assigned to
|
||
that interface as the SOURCE address. All traffic leaving eth0 that does
|
||
not have a SOURCE address falling within the Avvanta subnet
|
||
(206.124.146.0/24) must have its SOURCE address changed to
|
||
206.124.146.179.</para>
|
||
</section>
|
||
</article>
|