mirror of
https://gitlab.com/shorewall/code.git
synced 2024-11-08 08:44:05 +01:00
7fcba909df
git-svn-id: https://shorewall.svn.sourceforge.net/svnroot/shorewall/trunk@9355 fbd18981-670d-0410-9b5c-8dc0c1a9a2bb
1614 lines
69 KiB
XML
1614 lines
69 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
|
|
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
|
|
<article>
|
|
<!--ble$Id$-->
|
|
|
|
<articleinfo>
|
|
<title>Shorewall and Multiple Internet Connections</title>
|
|
|
|
<authorgroup>
|
|
<author>
|
|
<firstname>Tom</firstname>
|
|
|
|
<surname>Eastep</surname>
|
|
</author>
|
|
</authorgroup>
|
|
|
|
<pubdate><?dbtimestamp format="Y/m/d"?></pubdate>
|
|
|
|
<copyright>
|
|
<year>2005</year>
|
|
|
|
<year>2006</year>
|
|
|
|
<year>2007</year>
|
|
|
|
<year>2008</year>
|
|
|
|
<year>2009</year>
|
|
|
|
<holder>Thomas M. Eastep</holder>
|
|
</copyright>
|
|
|
|
<legalnotice>
|
|
<para>Permission is granted to copy, distribute and/or modify this
|
|
document under the terms of the GNU Free Documentation License, Version
|
|
1.2 or any later version published by the Free Software Foundation; with
|
|
no Invariant Sections, with no Front-Cover, and with no Back-Cover
|
|
Texts. A copy of the license is included in the section entitled
|
|
<quote><ulink url="GnuCopyright.htm">GNU Free Documentation
|
|
License</ulink></quote>.</para>
|
|
</legalnotice>
|
|
</articleinfo>
|
|
|
|
<warning>
|
|
<para>This document describes the Multi-ISP facility in Shorewall 4.0 and
|
|
later. If you are running an earlier release, please see the documentation
|
|
for that release.</para>
|
|
</warning>
|
|
|
|
<warning>
|
|
<para>Reading just Shorewall documentation is probably not going to give
|
|
you enough background to use this material. Shorewall may make iptables
|
|
easy but the Shorewall team doesn't have the resources to be able to
|
|
spoon-feed Linux policy routing to you (please remember that the user's
|
|
manual for a tractor doesn't teach you to grow corn either). You will
|
|
likely need to refer to the following additional information:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>The LARTC HOWTO: <ulink
|
|
url="http://www.lartc.org">http://www.lartc.org</ulink></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Output of <command>man ip</command></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Output of <command>ip route help</command> and <command>ip rule
|
|
help</command></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</warning>
|
|
|
|
<section id="Support">
|
|
<title>Multiple Internet Connection Support</title>
|
|
|
|
<para>Beginning with Shorewall 2.3.2, limited support is included for
|
|
multiple Internet connections. Limitations of this support are as
|
|
follows:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>It utilizes static routing configuration. As such, there is no
|
|
provision for reacting to the failure of any of the uplinks.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>The routing changes are made and the route cache is purged when
|
|
Shorewall is started <emphasis role="bold">and when Shorewall is
|
|
restarted</emphasis> (unless you specify the "-n" option to
|
|
<command>shorewall restart</command>). Ideally, restarting the packet
|
|
filter should have no effect on routing.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Prior to Shorewall 3.4.0, the routes and route rules added by
|
|
this support were not completely removed during <command>shorewall
|
|
stop</command>, <command>shorewall clear</command> or
|
|
<command>shorewall restart</command>.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>For most routing applications, <ulink
|
|
url="http://www.quagga.net/">Quagga</ulink> is a better solution
|
|
although it requires that your ISPs offer routing protocol
|
|
support.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<section id="Overview">
|
|
<title>Overview</title>
|
|
|
|
<para>Let's assume that a firewall is connected via two separate
|
|
Ethernet interfaces to two different ISPs as in the following
|
|
diagram.</para>
|
|
|
|
<graphic align="center" fileref="images/TwoISPs.png" valign="middle" />
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>eth0 connects to ISP1. The IP address of eth0 is
|
|
206.124.146.176 and the ISP's gateway router has IP address
|
|
206.124.146.254.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>eth1 connects to ISP 2. The IP address of eth1 is
|
|
130.252.99.27 and the ISP's gateway router has IP address
|
|
130.252.99.254.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>eth2 connects to the local LAN. Its IP configuration is not
|
|
relevant to this discussion.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Each of these <firstterm>providers</firstterm> is described in an
|
|
entry in the file <filename>/etc/shorewall/providers</filename>.</para>
|
|
|
|
<para>Entries in <filename>/etc/shorewall/providers</filename> can
|
|
specify that outgoing connections are to be load-balanced between the
|
|
two ISPs. Entries in <filename>/etc/shorewall/tcrules</filename> can be
|
|
used to direct particular outgoing connections to one ISP or the other.
|
|
Use of <filename>/etc/shorewall/tcrules</filename> is not required for
|
|
<filename>/etc/shorewall/providers</filename> to work, but you must
|
|
select a unique MARK value for each provider so Shorewall can set up the
|
|
correct marking rules for you.</para>
|
|
|
|
<para>When you use the <emphasis role="bold">track</emphasis> option in
|
|
<filename>/etc/shorewall/providers</filename>, connections from the
|
|
Internet are automatically routed back out of the correct interface and
|
|
through the correct ISP gateway. This works whether the connection is
|
|
handled by the firewall itself or if it is routed or port-forwarded to a
|
|
system behind the firewall.</para>
|
|
|
|
<para>Shorewall will set up the routing and will update the
|
|
<filename>/etc/iproute2/rt_tables</filename> to include the table names
|
|
and numbers of the tables that it adds.</para>
|
|
|
|
<caution>
|
|
<para>This feature uses <ulink url="traffic_shaping.htm">packet
|
|
marking</ulink> to control the routing. As a consequence, there are
|
|
some restrictions concerning entries in
|
|
<filename>/etc/shorewall/tcrules</filename>:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Packet marking for traffic control purposes may not be done
|
|
in the PREROUTING table for connections involving providers with
|
|
'track' specified (see below).</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You may not use the SAVE or RESTORE options.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You may not use connection marking.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</caution>
|
|
|
|
<para>The <filename>/etc/shorewall/providers</filename> file can also be
|
|
used in other routing scenarios. See the <ulink
|
|
url="Shorewall_Squid_Usage.html">Squid documentation</ulink> for an
|
|
example.</para>
|
|
</section>
|
|
|
|
<section id="providers">
|
|
<title>/etc/shorewall/providers File</title>
|
|
|
|
<para>Entries in this file have the following columns. As in all
|
|
Shorewall configuration files, enter "-" in a column if you don't want
|
|
to enter any value.</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term>NAME</term>
|
|
|
|
<listitem>
|
|
<para>The provider name. Must begin with a letter and consist of
|
|
letters and digits. The provider name becomes the name of the
|
|
generated routing table for this provider.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>NUMBER</term>
|
|
|
|
<listitem>
|
|
<para>A number between 1 and 252. This becomes the routing table
|
|
number for the generated table for this provider.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>MARK</term>
|
|
|
|
<listitem>
|
|
<para>A mark value used in your /etc/shorewall/tcrules file to
|
|
direct packets to this provider. Shorewall will also mark
|
|
connections that have seen input from this provider with this
|
|
value and will restore the packet mark in the PREROUTING CHAIN.
|
|
Mark values must be in the range 1-255.</para>
|
|
|
|
<para>Beginning with Shorewall version 3.2.0 Beta 6, you may use
|
|
may set HIGH_ROUTE_MARKS=Yes in
|
|
<filename>/etc/shorewall/shorewall.conf</filename>. This allows
|
|
you to:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Use connection marks for traffic shaping, provided that
|
|
you assign those marks in the FORWARD chain.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Use mark values > 255 for provider marks in this
|
|
column. These mark values must be a multiple of 256 in the
|
|
range 256-65280 (hex equivalent 0x100 - 0xFF00 with the
|
|
low-order 8 bits being zero).</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>DUPLICATE</term>
|
|
|
|
<listitem>
|
|
<para>Gives the name or number of a routing table to duplicate.
|
|
May be 'main' or the name or number of a previously declared
|
|
provider. For most applications, you want to specify 'main' here.
|
|
This field should be be specified as '-' when USE_DEFAULT_RT=Yes
|
|
in <filename>shorewall.conf</filename></para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>INTERFACE</term>
|
|
|
|
<listitem>
|
|
<para>The name of the interface to the provider. Where multiple
|
|
providers share the same interface (which is not recommended), you
|
|
must follow the name of the interface by a colon (":") and the IP
|
|
address assigned by this provider (e.g., eth0:206.124.146.176).
|
|
See <link linkend="Shared">below</link> for additional
|
|
considerations.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>GATEWAY</term>
|
|
|
|
<listitem>
|
|
<para>The IP address of the provider's Gateway router.</para>
|
|
|
|
<para>You can enter <emphasis role="bold">detect</emphasis> here
|
|
and Shorewall will attempt to automatically determine the gateway
|
|
IP address.</para>
|
|
|
|
<para><emphasis role="bold">Hint:</emphasis> <emphasis
|
|
role="bold">"detect"</emphasis> is appropriate for use in cases
|
|
where the interface named in the INTERFACE column is dynamically
|
|
configured via DHCP etc.</para>
|
|
|
|
<para>The GATEWAY may be omitted (enter '-') for point-to-point
|
|
links.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>OPTIONS</term>
|
|
|
|
<listitem>
|
|
<para>A comma-separated list from the following:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term>track</term>
|
|
|
|
<listitem>
|
|
<para>If specified, connections FROM this interface are to
|
|
be tracked so that responses may be routed back out this
|
|
same interface.</para>
|
|
|
|
<para>You want to specify 'track' if Internet hosts will be
|
|
connecting to local servers through this provider. Any time
|
|
that you specify 'track', you will normally want to also
|
|
specify 'balance' (see below).</para>
|
|
|
|
<para>Use of this feature requires that your kernel and
|
|
iptables include CONNMARK target and connmark match support
|
|
(<emphasis role="bold">Warning</emphasis>: Standard
|
|
<trademark>Debian</trademark> and
|
|
<trademark>Ubuntu</trademark> kernels are lacking that
|
|
support!).</para>
|
|
|
|
<warning>
|
|
<para>A <emphasis role="bold">bug</emphasis> in Shorewall
|
|
versions 3.2.0-3.2.10, 3.4.0-3.4.6 and 4.0.0-4.0.2
|
|
prevents proper handling of PREROUTING marks when
|
|
HIGH_ROUTE_MARKS=No and the <emphasis
|
|
role="bold">track</emphasis> option is specified. Patches
|
|
are available to correct this problem:</para>
|
|
|
|
<para>Shorewall version 3.2.0-3.4.3: <ulink
|
|
url="http://www1.shorewall.net/pub/shorewall/3.2/shorewall-3.2.10/errata/patches/Shorewall/patch-3.2.10-2.diff">http://www1.shorewall.net/pub/shorewall/3.2/shorewall-3.2.10/errata/patches/Shorewall/patch-3.2.10-2.diff
|
|
</ulink></para>
|
|
|
|
<para>Shorewall version 3.4.4-3.4.6: <ulink
|
|
url="http://www1.shorewall.net/pub/shorewall/3.4/shorewall-3.4.6/errata/patches/Shorewall/patch-3.4.6-1.diff">http://www1.shorewall.net/pub/shorewall/3.4/shorewall-3.4.6/errata/patches/Shorewall/patch-3.4.6-1.diff</ulink></para>
|
|
|
|
<para>Shorewall-shell version 4.0.0-4.0.2: <ulink
|
|
url="http://www1.shorewall.net/pub/shorewall/4.0/shorewall-4.0.2/errata/patches/Shorewall-shell/patch-shell-4.0.2-2.diff">http://www1.shorewall.net/pub/shorewall/4.0/shorewall-4.0.2/errata/patches/Shorewall-shell/patch-shell-4.0.2-2.diff</ulink></para>
|
|
</warning>
|
|
|
|
<important>
|
|
<para>If you are using
|
|
<filename>/etc/shorewall/providers</filename> because you
|
|
have multiple Internet connections, we recommend that you
|
|
specify 'track' even if you don't need it. It helps
|
|
maintain long-term connections in which there are
|
|
significant periods with no traffic.</para>
|
|
</important>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>balance</term>
|
|
|
|
<listitem>
|
|
<para>The providers that have 'balance' specified will get
|
|
outbound traffic load-balanced among them. Balancing will
|
|
not be perfect, as it is route based, and routes are cached.
|
|
This means that routes to often-used sites will always be
|
|
over the same provider.</para>
|
|
|
|
<para>By default, each provider is given the same weight (1)
|
|
. You can change the weight of a given provider by following
|
|
<emphasis>balance</emphasis> with "=" and the desired weight
|
|
(e.g., balance=2). The weights reflect the relative
|
|
bandwidth of the providers connections and should be small
|
|
numbers since the kernel actually creates additional default
|
|
routes for each weight increment.</para>
|
|
|
|
<important>
|
|
<para>If you are using
|
|
<filename>/etc/shorewall/providers</filename> because you
|
|
have multiple Internet connections, we recommend that you
|
|
specify 'balance' even if you don't need it. You can still
|
|
use entries in <filename>/etc/shorewall/tcrules</filename>
|
|
to force all traffic to one provider or another.<note>
|
|
<para>If you don't heed this advice then please read
|
|
and follow the advice in <ulink
|
|
url="FAQ.htm#faq57">FAQ 57</ulink> and <ulink
|
|
url="FAQ.htm#faq58">FAQ 58</ulink>.</para>
|
|
</note></para>
|
|
</important>
|
|
|
|
<important>
|
|
<para>If you specify 'balance' and still find that all
|
|
traffic is going out through only one provider, you may
|
|
need to install a kernel built with
|
|
CONFIG_IP_ROUTE_MULTIPATH_CACHED=n. Several users have
|
|
reported that this change has corrected similar
|
|
problems.</para>
|
|
|
|
<para>The SuSE 10.0 kernel is subject to this problem, and
|
|
<ulink
|
|
url="https://bugzilla.novell.com/show_bug.cgi?id=190908">
|
|
a kernel oops may result in this circumstance.</ulink>
|
|
SUSE 10.1 and SLES 10 have
|
|
CONFIG_IP_ROUTE_MULTIPATH_CACHED=n set by default. The
|
|
source of the problem seems to be <ulink
|
|
url="http://news.gmane.org/find-root.php?message_id=%3c00da01c5b35a%24b12b9860%241b00a8c0%40cruncher%3e">an
|
|
incompatibility between the LARTC patches and
|
|
CONFIG_IP_ROUTE_MULTIPATH_CACHED.</ulink></para>
|
|
</important>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>loose</term>
|
|
|
|
<listitem>
|
|
<para>Do not include routing rules that force traffic whose
|
|
source IP is an address of the INTERFACE to be routed to
|
|
this provider. Useful for defining providers that are to be
|
|
used only when the appropriate packet mark is applied.
|
|
Should not be specified together with <emphasis
|
|
role="bold">balance</emphasis>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>optional (added in Shorewall 3.2.2)</term>
|
|
|
|
<listitem>
|
|
<para>Shorewall will determine of this interface is up and
|
|
has a configured IPv4 address. If it is not, a warning is
|
|
issued and this provider is not configured.</para>
|
|
|
|
<note>
|
|
<para>'optional' is designed to detect interface states
|
|
that will cause <command>shorewall start</command> or
|
|
<command>shorewall restart</command> to fail; just because
|
|
an interface is in a state that Shorewall can [re]start
|
|
without error doesn't mean that traffic can actually be
|
|
sent through the interface.</para>
|
|
|
|
<para>Beginning with Shorewall-perl 4.0.3, you can supply
|
|
an 'isusable' <ulink
|
|
url="shorewall_extension_scripts.htm">extension
|
|
script</ulink> to extend Shorewall's interface state
|
|
detection.</para>
|
|
</note>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>src=<replaceable>source-address</replaceable> (Added in
|
|
Shorewall-perl 4.1.5)</term>
|
|
|
|
<listitem>
|
|
<para>Specifies the source address to use when routing to
|
|
this provider and none is known (the local client has bound
|
|
to the 0 address). May not be specified when an
|
|
<replaceable>address</replaceable> is given in the INTERFACE
|
|
column. If this option is not used, Shorewall substitutes
|
|
the primary IP address on the interface named in the
|
|
INTERFACE column.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>mtu=<replaceable>number</replaceable> (Added in
|
|
Shorewall-perl 4.1.5)</term>
|
|
|
|
<listitem>
|
|
<para>Specifies the MTU when forwarding through this
|
|
provider. If not given, the MTU of the interface named in
|
|
the INTERFACE column is assumed.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><emphasis
|
|
role="bold">fallback[=<replaceable>weight</replaceable>]
|
|
(Added in Shorewall-perl 4.2.5)</emphasis></term>
|
|
|
|
<listitem>
|
|
<para>Indicates that a default route through the provider
|
|
should be added to the default routing table (table 253). If
|
|
a <replaceable>weight</replaceable> is given, a balanced
|
|
route is added with the weight of this provider equal to the
|
|
specified <replaceable>weight</replaceable>. If the option
|
|
is given without a <replaceable>weight</replaceable>, an
|
|
separate default route is added through the provider's
|
|
gateway; the route has a metric equal to the provider's
|
|
NUMBER. The option is ignored with a warning message if
|
|
USE_DEFAULT_RT=Yes in
|
|
<filename>shorewall.conf</filename>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
|
|
<para>For those of you who are terminally confused
|
|
between<emphasis role="bold"> track</emphasis> and <emphasis
|
|
role="bold">balance</emphasis>:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><emphasis role="bold">track</emphasis> governs incoming
|
|
connections.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis role="bold">balance</emphasis> governs
|
|
outgoing connections.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>COPY</term>
|
|
|
|
<listitem>
|
|
<para>A comma-separated list if interface names. Wildcards
|
|
specified using an asterisk ("*") are permitted (e.g., tun*
|
|
).</para>
|
|
|
|
<para>When you specify an existing table in the DUPLICATE column,
|
|
Shorewall copies all routes through the interface specified in the
|
|
INTERFACE column plus the interfaces listed in this column.
|
|
Normally, you will list all interfaces on your firewall in this
|
|
column except those Internet interfaces specified in the INTERFACE
|
|
column of entries in this file.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</section>
|
|
|
|
<section id="Providers">
|
|
<title>What an entry in the Providers File Does</title>
|
|
|
|
<para>Adding another entry in the providers file simply creates an
|
|
alternate routing table for you. The table will usually contain two
|
|
routes:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>A host route to the specified GATEWAY through the specified
|
|
INTERFACE.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>A default route through the GATEWAY.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>Note that the first route is omitted if "-" is specified as the
|
|
GATEWAY; in that case, the default route does not specify a gateway
|
|
(point-to-point link).</para>
|
|
|
|
<para>If the DUPLICATE column is non-empty, then routes from the table
|
|
named in that column are copied into the new table. By default, all
|
|
routes (except default routes) are copied. The set of routes copied can
|
|
be restricted using the COPY column which lists the interfaces whose
|
|
routes you want copied. You will generally want to include all local
|
|
interfaces in this list. You should exclude the loopback interface (lo)
|
|
and any interfaces that do not have an IPv4 configuration. You should
|
|
also omit interfaces like <emphasis role="bold">tun</emphasis>
|
|
interfaces that are created dynamically. Traffic to networks handled by
|
|
those interfaces should be routed through the main table using entries
|
|
in <filename>/etc/shorewall/route_rules</filename> (see Example 2 <link
|
|
linkend="Examples">below</link>).</para>
|
|
|
|
<para>In addition:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>Unless <emphasis role="bold">loose</emphasis> is specified, an
|
|
ip rule is generated for each IP address on the INTERFACE that
|
|
routes traffic from that address through the associated routing
|
|
table.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>If you specify <emphasis role="bold">track</emphasis>, then
|
|
connections which have had at least one packet arrive on the
|
|
interface listed in the INTERFACE column have their connection mark
|
|
set to the value in the MARK column. In the PREROUTING chain,
|
|
packets with a connection mark have their packet mark set to the
|
|
value of the associated connection mark; packets marked in this way
|
|
bypass any prerouting rules that you create in
|
|
<filename>/etc/shorewall/tcrules</filename>. This ensures that
|
|
packets associated with connections from outside are always routed
|
|
out of the correct interface.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>If you specify <emphasis role="bold">balance</emphasis>, then
|
|
Shorewall will replace the 'default' route with weight 100 in the
|
|
'main' routing table with a load-balancing route among those
|
|
gateways where <emphasis role="bold">balance</emphasis> was
|
|
specified. So if you configure default routes, be sure that their
|
|
weight is less than 100 or the route added by Shorewall will not be
|
|
used.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>That's <emphasis role="bold">all</emphasis> that these entries do.
|
|
You still have to follow the principle stated in the <ulink
|
|
url="Shorewall_and_Routing.html">Shorewall Routing
|
|
documentation</ulink>:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>Routing determines where packets are to be sent.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Once routing determines where the packet is to go, the
|
|
firewall (Shorewall) determines if the packet is allowed to go
|
|
there.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>The bottom line is that if you want traffic to go out through a
|
|
particular provider then you <emphasis>must </emphasis>mark that traffic
|
|
with the provider's MARK value in
|
|
<filename>/etc/shorewall/tcrules</filename> and you must do that marking
|
|
in the PREROUTING chain; or, you must provide the appropriate rules in
|
|
<filename>/etc/shorewall/route_rules</filename>.</para>
|
|
</section>
|
|
|
|
<section id="swping">
|
|
<title>Gateway Monitoring and Failover</title>
|
|
|
|
<para>Beginning with Shorewall 4.2.6, Shorewall includes a sample
|
|
monitoring script <filename>swping</filename>. The
|
|
<filename>swping</filename> file is available in the main directory
|
|
contained in the Shorewall-common tarball and is included in the
|
|
Shorewall-common documentation directory on the Shorewall-common
|
|
RPM.</para>
|
|
|
|
<para>For those not on 4.2.6 yet, the script may be downloaded from
|
|
<ulink
|
|
url="http://www.shorewall.net/pub/shorewall/contrib/MultiISP-failover/">http://www.shorewall.net/pub/shorewall/contrib/MultiISP-failover/</ulink>.</para>
|
|
|
|
<important>
|
|
<para>These samples are offered <emphasis>as is</emphasis> — they work
|
|
for me but I don't make any claim that they will work for anyone else.
|
|
But if you have a need for automated link monitoring, they offer you a
|
|
place to start.</para>
|
|
</important>
|
|
|
|
<para>The script monitors two interfaces but it is a trivial exercise to
|
|
extend it to more than two. At the top are a number of variables to
|
|
set:</para>
|
|
|
|
<programlisting># The commands to run when the status of a line changes. Both commands will be executed.
|
|
#
|
|
COMMAND1=
|
|
COMMAND2="ip -4 route ls"
|
|
...
|
|
#
|
|
# Interfaces to monitor -- you may use shell variables from your params file
|
|
#
|
|
IF1=eth0
|
|
IF2=eth1
|
|
#
|
|
# Sites to Ping. Must not depend on the associated interface having a default route through it.
|
|
#
|
|
TARGET1=
|
|
TARGET2=
|
|
#
|
|
# How often to ping
|
|
#
|
|
PING_INTERVAL=5
|
|
#
|
|
# Value for ping's -W option
|
|
#
|
|
PING_TIMEOUT=2
|
|
#
|
|
# This many successive pings must succeed for the interface to be marked up when it is down
|
|
#
|
|
UP_COUNT=5
|
|
#
|
|
# This many successive pings must fail for the interface to be marked down when it is up
|
|
#
|
|
DOWN_COUNT=2</programlisting>
|
|
|
|
<para>If you leave COMMAND1 empty, the script sets its value
|
|
automatically depending on whether Shorewall-lite is installed.</para>
|
|
|
|
<para>When the status of an interface changes:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>For each interface, a file is placed in /etc/shorewall to
|
|
record the status of the interface: either 0 (UP) or 1 (DOWN). The
|
|
name of the file is<emphasis> interface</emphasis>.status where
|
|
<emphasis>interface</emphasis> is the interface (e.g.,
|
|
<filename>eth0.status</filename>).</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>A <command>shorewall -f restart</command> command is executed
|
|
(<command>shorewall-lite restart</command>, if Shorewall-lite is
|
|
installed).</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>The contents of the main routing table are displayed.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>The .status files are intended to be used with the following
|
|
<filename>/etc/shorewall/isusable</filename> script.<programlisting>local status=0
|
|
|
|
case $1 in
|
|
<emphasis role="bold">eth0|eth1</emphasis>)
|
|
[ -f /etc/shorewall/${1}.status ] && status=$(cat /etc/shorewall/${1}.status)
|
|
;;
|
|
esac
|
|
|
|
return $status</programlisting></para>
|
|
|
|
<para>Be sure that you modify the interface names to match your
|
|
configuration.</para>
|
|
|
|
<para>Also included is a sample init script
|
|
(<filename>swping.init</filename>) to start the monitoring daemon. Copy
|
|
it to<filename> /etc/init.d/swping</filename> and use your
|
|
distribution's SysV init tools to cause it to be run at boot. It works
|
|
on <trademark>OpenSuSE</trademark> 11.0 -- YMMV. Modify the PROG and
|
|
STATEDIR variables as needed.</para>
|
|
|
|
<para>This simple script has a number of limitations:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>It only works on IPv4 (but is easy to modify for IPv6)</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>It's method of determining whether an interface is up or down
|
|
is crude. You will normally specify the default gateway for each
|
|
provider as the sites to ping and being able to ping the default
|
|
gateway is not a surefire indication that the provider is usable.
|
|
The method of determining whether a site is up or down is also
|
|
crude.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Because of the crudeness of the algorithm, hysteresis may
|
|
occur.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>It is tricky to configure a system such that a system works
|
|
correctly when one of its providers is down unless you largely don't
|
|
care which interface is used.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
</section>
|
|
|
|
<section id="Martians">
|
|
<title>Martians</title>
|
|
|
|
<para>One problem that often arises with Multi-ISP configuration is
|
|
'Martians'. If your Internet interfaces are configured with the
|
|
<emphasis role="bold">routefilter</emphasis> option in
|
|
<filename>/etc/shorewall/interfaces</filename> (remember that if you set
|
|
that option, you should also select <emphasis
|
|
role="bold">logmartians</emphasis>), then things may not work correctly
|
|
and you will see messages like this:</para>
|
|
|
|
<programlisting>Feb 9 17:23:45 gw.ilinx kernel: martian source 206.124.146.176 from 64.86.88.116, on dev eth1
|
|
Feb 9 17:23:45 gw.ilinx kernel: ll header: 00:a0:24:2a:1f:72:00:13:5f:07:97:05:08:00</programlisting>
|
|
|
|
<para>The above message is somewhat awkwardly phrased. The source IP in
|
|
this incoming packet was 64.86.88.116 and the destination IP address is
|
|
206.124.146.176. Another gotcha is that the incoming packet has already
|
|
had the destination IP address changed for DNAT or because the original
|
|
outgoing connection was altered by an entry in
|
|
<filename>/etc/shorewall/masq</filename> (SNAT or Masquerade). So the
|
|
destination IP address (206.124.146.176) may not have been the
|
|
destination IP address in the packet as it was initially
|
|
received.</para>
|
|
|
|
<para>There a couple of common causes for these problems:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>You have connected both of your external interfaces to the
|
|
same hub/switch. Connecting multiple firewall interfaces to a common
|
|
hub or switch is always a bad idea that will result in
|
|
hard-to-diagnose problems.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You are specifying both the <emphasis
|
|
role="bold">loose</emphasis> and <emphasis
|
|
role="bold">balance</emphasis> options on your provider(s). This
|
|
causes individual connections to ping-pong back and forth between
|
|
the interfaces which is guaranteed to cause problems.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You are redirecting traffic from the local system out of one
|
|
interface or the other using packet marking in your
|
|
<filename>/etc/shorewall/tcrules</filename> file. A better approach
|
|
is to configure the application to use the appropriate local IP
|
|
address (the IP address of the interface that you want the
|
|
application to use). See <link linkend="Local">below</link>.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>If all else fails, remove the <emphasis
|
|
role="bold">routefilter</emphasis> option from your external interfaces.
|
|
If you do this, you may wish to add rules to log and drop packets from
|
|
the Internet that have source addresses in your local networks. For
|
|
example, if the local LAN in the above diagram is 192.168.1.0/24, then
|
|
you would add this rule:</para>
|
|
|
|
<programlisting>#ACTION SOURCE DEST
|
|
DROP:info net:192.168.1.0/24 all</programlisting>
|
|
|
|
<para>Be sure the above rule is added before any other rules with
|
|
<emphasis>net</emphasis> in the SOURCE column.</para>
|
|
</section>
|
|
|
|
<section id="Example1">
|
|
<title id="Example">Example</title>
|
|
|
|
<para>The configuration in the figure at the top of this section would
|
|
be specified in <filename>/etc/shorewall/providers</filename> as
|
|
follows.</para>
|
|
|
|
<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
|
ISP1 1 1 main eth0 206.124.146.254 track,balance eth2
|
|
ISP2 2 2 main eth1 130.252.99.254 track,balance eth2</programlisting>
|
|
|
|
<para>Other configuration files go something like this:</para>
|
|
|
|
<para><filename>/etc/shorewall/interfaces</filename>:</para>
|
|
|
|
<programlisting>#ZONE INTERFACE BROADCAST OPTIONS
|
|
net eth0 detect …
|
|
net eth1 detect …</programlisting>
|
|
|
|
<para><filename>/etc/shorewall/policy</filename>:</para>
|
|
|
|
<programlisting>#SOURCE DESTINATION POLICY LIMIT:BURST
|
|
net net DROP</programlisting>
|
|
|
|
<para>Regardless of whether you have masqueraded hosts or not, the
|
|
following entries are required in
|
|
<filename>/etc/shorewall/masq</filename> if you plan to redirect
|
|
connections from the firewall using entries in
|
|
<filename>/etc/shorewall/tcrules</filename> or if you specify <emphasis
|
|
role="bold">balance</emphasis> on your providers.</para>
|
|
|
|
<programlisting>#INTERFACE SUBNET ADDRESS
|
|
eth0 130.252.99.27 206.124.146.176
|
|
eth1 206.124.146.176 130.252.99.27</programlisting>
|
|
|
|
<para>Those entries ensure that traffic originating on the firewall and
|
|
redirected via packet marks always has the source IP address
|
|
corresponding to the interface that it is routed out of.</para>
|
|
|
|
<note>
|
|
<para>If you have a Dynamic IP address on either of the interfaces,
|
|
you can use shell variables to construct the above rules. For example,
|
|
if <filename class="devicefile">eth0</filename> had a dynamic IP
|
|
address, then:</para>
|
|
|
|
<para><filename>/etc/shorewall/params</filename>:</para>
|
|
|
|
<programlisting>ETH0_IP=$(find_first_interface_address eth0)</programlisting>
|
|
|
|
<para>For optional interfaces, use
|
|
<emphasis>find_first_interface_address_if_any</emphasis> in place of
|
|
<emphasis>find_first_interface_address</emphasis>.</para>
|
|
|
|
<para>/etc/shorewall/masq:</para>
|
|
|
|
<programlisting>#INTERFACE SUBNET ADDRESS
|
|
eth0 130.252.99.27 $ETH0_IP
|
|
eth1 $ETH0_IP 130.252.99.27</programlisting>
|
|
</note>
|
|
|
|
<para>If you have masqueraded hosts, be sure to update
|
|
<filename>/etc/shorewall/masq</filename> to masquerade to both ISPs. For
|
|
example, if you masquerade all hosts connected to <filename
|
|
class="devicefile">eth2</filename> then:</para>
|
|
|
|
<programlisting>#INTERFACE SUBNET ADDRESS
|
|
eth0 eth2 206.124.146.176
|
|
eth1 eth2 130.252.99.27</programlisting>
|
|
|
|
<warning>
|
|
<para>Entries in <filename>/etc/shorewall/masq</filename> have no
|
|
effect on which ISP a particular connection will be sent through. That
|
|
is rather the purpose of entries in
|
|
<filename>/etc/shorewall/tcrules</filename> or
|
|
<filename>/etc/shorewall/route_rules</filename>.</para>
|
|
</warning>
|
|
|
|
<para>Now suppose that you want to route all outgoing SMTP traffic from
|
|
your local network through ISP 2. You would make this entry in <ulink
|
|
url="traffic_shaping.htm">/etc/shorewall/tcrules</ulink> (and if you are
|
|
running a version of Shorewall earlier than 3.0.0, you would set
|
|
TC_ENABLED=Yes in <ulink
|
|
url="???">/etc/shorewall/shorewall.conf</ulink>).</para>
|
|
|
|
<programlisting>#MARK SOURCE DEST PROTO PORT(S) CLIENT USER TEST
|
|
# PORT(S)
|
|
2:P <local network> 0.0.0.0/0 tcp 25</programlisting>
|
|
</section>
|
|
|
|
<section id="morethan2">
|
|
<title>More than 2 Providers</title>
|
|
|
|
<para>When there are more than two providers, you need to extend the
|
|
two-provider case in the expected way:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>For each external address, you need an entry in
|
|
<filename>/etc/shorewall/masq</filename> to handle the case where a
|
|
connection using that address as the SOURCE is sent out of the
|
|
interfaces other than the one that the address is configured
|
|
on.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>For each external interface, you need to add an entry to
|
|
<filename>/etc/shorewall/masq</filename> for each internal network
|
|
that needs to be masqueraded (or use SNAT) through that
|
|
interface.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>If we extend the above example to add eth3 with IP address
|
|
16.105.78.4 with gateway 16.105.78.254, then:</para>
|
|
|
|
<para><filename>/etc/shorewall/providers</filename>:<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
|
ISP1 1 1 main eth0 206.124.146.254 track,balance eth2
|
|
ISP2 2 2 main eth1 130.252.99.254 track,balance eth2
|
|
ISP3 3 3 main eth3 16.105.78.254 track,balance eth2</programlisting></para>
|
|
|
|
<para><filename>/etc/shorewall/masq</filename>:<programlisting>#INTERFACE SUBNET ADDRESS
|
|
eth0 130.252.99.27 206.124.146.176
|
|
eth3 130.252.99.27 16.105.78.4
|
|
eth1 206.124.146.176 130.252.99.27
|
|
eth3 206.124.146.176 16.105.78.4
|
|
eth0 16.106.78.4 206.124.146.176
|
|
eth1 16.106.78.4 130.252.99.27
|
|
eth0 eth2 206.124.146.176
|
|
eth1 eth2 130.252.99.27
|
|
eth3 eth2 16.105.78.4</programlisting></para>
|
|
</section>
|
|
|
|
<section id="Local">
|
|
<title>Applications running on the Firewall</title>
|
|
|
|
<para>Experience has shown that in some cases, problems occur with
|
|
applications running on the firewall itself. This is especially true
|
|
when you have specified <emphasis role="bold">routefilter</emphasis> on
|
|
your external interfaces in /etc/shorewall/interfaces (see <link
|
|
linkend="Martians">above</link>). When this happens, it is suggested
|
|
that you have the application use specific local IP addresses rather
|
|
than 0.</para>
|
|
|
|
<para>Examples:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Squid: In <filename>squid.conf</filename>, set <emphasis
|
|
role="bold">tcp_outgoing_address</emphasis> to the IP address of the
|
|
interface that you want Squid to use.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>In OpenVPN, set <emphasis role="bold">local
|
|
</emphasis>(<emphasis role="bold">--local</emphasis> on the command
|
|
line) to the IP address that you want the server to receive
|
|
connections on.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Note that some traffic originating on the firewall doesn't have a
|
|
SOURCE IP address before routing. At least one Shorewall user reports
|
|
that an entry in <filename>/etc/shorewall/route_rules</filename> with
|
|
'lo' in the SOURCE column seems to be the most reliable way to direct
|
|
such traffic to a particular ISP.</para>
|
|
</section>
|
|
|
|
<section id="IPSEC">
|
|
<title>IPSEC</title>
|
|
|
|
<para>If you have an IPSEC gateway on your firewall, be sure to arrange
|
|
for ESP packets to be routed out of the same interface that you have
|
|
configured your keying daemon to use.</para>
|
|
</section>
|
|
|
|
<section id="route_rules">
|
|
<title>/etc/shorewall/route_rules</title>
|
|
|
|
<para>The <filename>/etc/shorewall/route_rules</filename> file was added
|
|
in Shorewall version 3.2.0. The <filename>route_rules</filename> file
|
|
allows assigning certain traffic to a particular provider just as
|
|
entries in the <filename>tcrules</filename> file. The difference between
|
|
the two files is that entries in <filename>route_rules</filename> are
|
|
independent of Netfilter.</para>
|
|
|
|
<section id="Routing_rules">
|
|
<title>Routing Rules</title>
|
|
|
|
<para>Routing rules are maintained by the Linux kernel and can be
|
|
displayed using the <command>ip rule ls</command> command. When
|
|
routing a packet, the rules are processed in turn until the packet is
|
|
successfully routed.</para>
|
|
|
|
<programlisting>gateway:~ # <command>ip rule ls</command>
|
|
0: from all lookup local <=== Local (to the firewall) IP addresses
|
|
10001: from all fwmark 0x1 lookup Blarg <=== This and the next rule are generated by the
|
|
10002: from all fwmark 0x2 lookup Comcast 'MARK' values in /etc/shorewall/providers.
|
|
20000: from 206.124.146.176 lookup Blarg <=== This and the next rule are generated unless
|
|
20256: from 24.12.22.33 lookup Comcast 'loose' is specified; based in the output of 'ip addr ls'
|
|
32766: from all lookup main <=== This is the routing table shown by 'iproute -n'
|
|
32767: from all lookup default <=== This table is usually empty
|
|
gateway:~ #</programlisting>
|
|
|
|
<para>In the above example, there are two providers: Blarg and Comcast
|
|
with MARK 1 going to Blarg and mark 2 going to Comcast.</para>
|
|
</section>
|
|
|
|
<section id="route_rules_columns">
|
|
<title>Columns in the route_rules file</title>
|
|
|
|
<para>Columns in the file are:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term>SOURCE (Optional)</term>
|
|
|
|
<listitem>
|
|
<para>An ip address (network or host) that matches the source IP
|
|
address in a packet. May also be specified as an interface name
|
|
optionally followed by ":" and an address. If the device 'lo' is
|
|
specified, the packet must originate from the firewall
|
|
itself.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>DEST (Optional)</term>
|
|
|
|
<listitem>
|
|
<para>An ip address (network or host) that matches the
|
|
destination IP address in a packet.</para>
|
|
|
|
<para>If you choose to omit either SOURCE or DEST, place "-" in
|
|
that column. Note that you may not omit both SOURCE and
|
|
DEST.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>PROVIDER</term>
|
|
|
|
<listitem>
|
|
<para>The provider to route the traffic through. May be
|
|
expressed either as the provider name or the provider
|
|
number.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>PRIORITY</term>
|
|
|
|
<listitem>
|
|
<para>The rule's priority which determines the order in which
|
|
the rules are processed.</para>
|
|
|
|
<para>1000-1999 Before Shorewall-generated 'MARK' rules</para>
|
|
|
|
<para>11000- 11999 After 'MARK' rules but before
|
|
Shorewall-generated rules for ISP interfaces.</para>
|
|
|
|
<para>26000-26999 After ISP interface rules but before 'default'
|
|
rule.</para>
|
|
|
|
<para>Rules with equal priority are applied in the order in
|
|
which they appear in the file.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</section>
|
|
|
|
<section id="Examples">
|
|
<title>Examples</title>
|
|
|
|
<para><emphasis role="bold">Example 1:</emphasis> You want all traffic
|
|
entering the firewall on eth1 to be routed through Comcast.</para>
|
|
|
|
<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
|
eth1 - Comcast 1000</programlisting>
|
|
|
|
<para>With this entry, the output of <command>ip rule ls</command>
|
|
would be as follows.</para>
|
|
|
|
<para><programlisting>gateway:~ # <command>ip rule ls</command>
|
|
0: from all lookup local
|
|
1000: from all iif eth1 lookup Comcast
|
|
10001: from all fwmark 0x1 lookup Blarg
|
|
10002: from all fwmark 0x2 lookup Comcast
|
|
20000: from 206.124.146.176 lookup Blarg
|
|
20256: from 24.12.22.33 lookup Comcast
|
|
32766: from all lookup main
|
|
32767: from all lookup default
|
|
gateway:~ #</programlisting>Note that because we used a priority of 1000, the
|
|
test for <filename class="devicefile">eth1</filename> is inserted
|
|
before the fwmark tests.</para>
|
|
|
|
<para id="Openvpn"><emphasis role="bold">Example 2:</emphasis> You use
|
|
OpenVPN (routed setup w/tunX) in combination with multiple providers.
|
|
In this case you have to set up a rule to ensure that the OpenVPN
|
|
traffic is routed back through the tunX interface(s) rather than
|
|
through any of the providers. 10.8.0.0/24 is the subnet chosen in your
|
|
OpenVPN configuration (server 10.8.0.0 255.255.255.0).</para>
|
|
|
|
<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
|
- 10.8.0.0/24 main 1000</programlisting>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="USE_DEFAULT_RT">
|
|
<title>USE_DEFAULT_RT</title>
|
|
|
|
<para>Beginning with Shorewall 4.2.0 Beta3, Shorewall-perl has supported
|
|
a USE_DEFAULT_RT option in <ulink
|
|
url="manpages/shorewall.conf.html">shorewall.conf</ulink> (5).</para>
|
|
|
|
<para>One of the drawbacks of the Multi-ISP support as described in the
|
|
preceding sections is that changes to the main table made by
|
|
applications are not added to the individual provider tables. This makes
|
|
route rules such as described in <link linkend="Openvpn">one of the
|
|
examples above</link> necessary.</para>
|
|
|
|
<para>USE_DEFAULT_RT=Yes works around that problem by passing packets
|
|
through the main table first rather than last. This has a number of
|
|
implications:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>Both the DUPLICATE and the COPY columns in the providers file
|
|
must remain empty or contain "-". The individual provider routing
|
|
tables generated when USE_DEFAULT_RT=Yes contain only a host route
|
|
to the gateway and a default route via the gateway.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>The <emphasis role="bold">balance</emphasis> option is assumed
|
|
for all interfaces that do not have the <emphasis
|
|
role="bold">loose</emphasis> option. When you want both <emphasis
|
|
role="bold">balance</emphasis> and <emphasis
|
|
role="bold">loose</emphasis>, both must be specified.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>The default route generated by Shorewall is added to the
|
|
<emphasis>default</emphasis> routing table (253) rather than to the
|
|
main routing table (254).</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Packets are sent through the main routing table by a routing
|
|
rule with priority 999. In ), the priority range 1-998 may be used
|
|
for inserting rules that bypass the main table.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>All provider gateways must be specified explicitly in the
|
|
GATEWAY column. 'detect' may not be specified. Note that for ppp
|
|
interfaces, the GATEWAY may remain unspecified ("-").</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You should disable all default route management outside of
|
|
Shorewall. If a default route is inadvertently added to the main
|
|
table while Shorewall is started, then all policy routing will stop
|
|
working except for those routing rules in the priority range
|
|
1-998.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>Although 'balance' is automatically assumed when
|
|
USE_DEFAULT_RT=Yes, you can easily cause all traffic to use one provider
|
|
except when you explicitly direct it to use the other provider via
|
|
<ulink
|
|
url="manpages/shorewall-route_rules.html">shorewall-route_rules</ulink>
|
|
(5) or <ulink
|
|
url="manpages/shorewall-tcrules.html">shorewall-tcrules</ulink>
|
|
(5).</para>
|
|
|
|
<para>Example (send all traffic through the 'shorewall' provider unless
|
|
otherwise directed).</para>
|
|
|
|
<para>/etc/shorewall/providers:<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS
|
|
linksys 1 1 - wlan0 172.20.1.1 track,balance=1,optional
|
|
shorewall 2 2 - eth0 192.168.1.254 track,balance=2,optional</programlisting>/etc/shorewall/route_rules:<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
|
- - shorewall 11999</programlisting></para>
|
|
</section>
|
|
|
|
<section id="Shared">
|
|
<title>Two Providers Sharing an Interface</title>
|
|
|
|
<para>Shared interface support is available only in Shorewall-perl 4.2.0
|
|
and later.</para>
|
|
|
|
<orderedlist numeration="loweralpha">
|
|
<listitem>
|
|
<para>Only Ethernet (or Ethernet-like) interfaces can be used. For
|
|
inbound traffic, the MAC addresses of the gateway routers are used
|
|
to determine which provider a packet was received through. Note that
|
|
only routed traffic can be categorized using this technique.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You must specify the address on the interface that corresponds
|
|
to a particular provider in the INTERFACE column by following the
|
|
interface name with a colon (":") and the address.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Entries in <filename>/etc/shorewall/masq</filename> must be
|
|
qualified by the provider name (or number).</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>This feature requires Realm Match support in your kernel and
|
|
iptables.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You must add route_rules entries for networks that are
|
|
accessed through a particular provider.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>If you have additional IP addresses through either provider,
|
|
you must add <filename>route_rules</filename> to direct traffic FROM
|
|
each of those addresses through the appropriate provider.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You must manually add MARK rules for traffic known to come
|
|
from each provider.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You must specify a gateway IP address in the GATEWAY column
|
|
of<filename> /etc/shorewall/providers</filename>; <emphasis
|
|
role="bold">detect</emphasis> is not permitted.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>Taken together, b. and h. effectively preclude using this
|
|
technique with dynamic IP addresses.</para>
|
|
|
|
<para>Example:</para>
|
|
|
|
<para>This is our home network circa fall 2008. We have two Internet
|
|
providers:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>Comcast -- Cable modem with one dynamic IP address.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Avvanta -- ADSL with 5 static IP addresses.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>Because the old <trademark>Compaq</trademark>
|
|
<trademark>Presario</trademark> that I use for a firewall only has three
|
|
PCI slots and no onboard Ethernet, it doesn't have enough Ethernet
|
|
controllers to support both providers. So I use a Linksys WRT300n pre-N
|
|
router as a gateway to Comcast. Note that because the Comcast IP address
|
|
is dynamic, I could not share a single firewall interface between the
|
|
two providers directly.</para>
|
|
|
|
<para>On my personal laptop (ursa), I have 9 virtual machines running
|
|
various Linux distributions. <emphasis>It is the Shorewall configuration
|
|
on ursa that I will describe here</emphasis>.</para>
|
|
|
|
<para>Below is a diagram of our network:<graphic align="center"
|
|
fileref="images/Network2008a.png" /></para>
|
|
|
|
<para>The local wired network in my office is connected to both gateways
|
|
and uses the private (RFC 1918) network 172.20.1.0/24. The Comcast
|
|
gateway has local IP address 172.20.1.1 while the Avvanta gateway has
|
|
local IP address 172.20.1.254. Ursa's eth0 interface has a single IP
|
|
address (172.20.1.130).</para>
|
|
|
|
<para>This configuration uses USE_DEFAULT_RT=Yes in
|
|
<filename>shorewall.conf </filename>(see <link
|
|
linkend="USE_DEFAULT_RT">above</link>).</para>
|
|
|
|
<para>Here is the <filename>providers</filename> file:</para>
|
|
|
|
<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
|
comcast 1 1 - eth0:172.20.1.130 172.20.1.1 track,loose,balance,optional
|
|
avvanta 2 2 - eth0:172.20.1.130 172.20.1.254 track,optional,loose
|
|
wireless 3 3 - wlan0 172.20.1.1 track,optional</programlisting>
|
|
|
|
<para>Several things to note:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>172.20.1.130 is specified as the <filename
|
|
class="devicefile">eth0</filename> IP address for both
|
|
providers.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Both wired providers have the <emphasis
|
|
role="bold">loose</emphasis> option. This prevents Shorewall from
|
|
automatically generating routing rules based on the source IP
|
|
address.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Only <emphasis role="bold">comcast</emphasis> has the
|
|
<emphasis role="bold">balance</emphasis> option. With
|
|
USE_DEFAULT_RT=yes, that means that <emphasis
|
|
role="bold">comcast</emphasis> will be the default provider. While
|
|
<emphasis role="bold">balance</emphasis> is the default, with
|
|
USE_DEFAULT_RT=Yes, it must be specified explicitly when <emphasis
|
|
role="bold">loose</emphasis> is also specified.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>I always disable the <emphasis role="bold">wireless</emphasis>
|
|
interface when the laptop is connected to the wired network.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>I use a different Shorewall configuration when I take the
|
|
laptop on the road.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>Here is the route_rules file:<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
|
- 206.124.146.176/31 avvanta 1000
|
|
- 206.124.146.178/31 avvanta 1000
|
|
- 206.124.146.180/32 avvanta 1000</programlisting></para>
|
|
|
|
<para>Those rules direct traffic to the five static Avvanta IP addresses
|
|
(only two are currently used) through the <emphasis
|
|
role="bold">avvanta</emphasis> provider.</para>
|
|
|
|
<para>Here is the tcrules file (MARK_IN_FORWARD_CHAIN=No in
|
|
<filename>shorewall.conf</filename>):<programlisting>#MARK SOURCE DEST PROTO PORT(S) CLIENT USER TEST LENGTH TOS CONNBYTES HELPER
|
|
# PORT(S)
|
|
2 $FW 0.0.0.0/0 tcp 21
|
|
2 $FW 0.0.0.0/0 tcp - - - - - - - ftp
|
|
2 $FW 0.0.0.0/0 tcp 119</programlisting></para>
|
|
|
|
<para>These rules:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Use <emphasis role="bold">avvanta</emphasis> for FTP.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Use <emphasis role="bold">avvanta</emphasis> for NTTP</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>The remaining files are for a rather standard two-interface config
|
|
with a bridge as the local interface.</para>
|
|
|
|
<para><filename>zones</filename>:<programlisting>#ZONE IPSEC OPTIONS IN OUT
|
|
# ONLY OPTIONS OPTIONS
|
|
fw firewall
|
|
net ipv4
|
|
kvm ipv4</programlisting><filename>policy</filename>:<programlisting>net net NONE
|
|
fw net ACCEPT
|
|
fw kvm ACCEPT
|
|
kvm all ACCEPT
|
|
net all DROP info
|
|
all all REJECT info</programlisting></para>
|
|
|
|
<para>interfaces:<programlisting>#ZONE INTERFACE BROADCAST OPTIONS GATEWAY
|
|
#
|
|
net eth0 detect dhcp,tcpflags,routefilter,blacklist,logmartians,optional,arp_ignore
|
|
net wlan0 detect dhcp,tcpflags,routefilter,blacklist,logmartians,optional
|
|
kvm br0 detect routeback #Virtual Machines</programlisting><note>
|
|
<para><filename class="devicefile">wlan0</filename> is the wireless
|
|
adapter in the notebook. Used when the laptop is in our home but not
|
|
connected to the wired network.</para>
|
|
</note></para>
|
|
|
|
<para>masq:<programlisting>#INTERFACE SUBNET ADDRESS PROTO PORT(S) IPSEC
|
|
eth0 192.168.0.0/24
|
|
wlan0 192.168.0.0/24</programlisting><note>
|
|
<para>Because the firewall has only a single external IP address, I
|
|
don't need to specify the providers in the masq rules.</para>
|
|
</note></para>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="Complete">
|
|
<title>A Complete Working Example</title>
|
|
|
|
<para>This section describes the network at shorewall.net early in 2009.
|
|
The configuration is as follows:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Two providers:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Avvanta -- A slow (1.5mb/384kb) DSL service with 5 static IP
|
|
addresses.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Comcast -- A fast (20mb/10mb) Cable circuit with a single
|
|
<emphasis>dynamic</emphasis> address.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>A local network consisting of wired and wireless client systems.
|
|
A Linksys WRT300N wireless router is used as an access point for the
|
|
wireless hosts.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>A DMZ hosting a single server (lists.shorewall.net aka
|
|
www1.shorewall.net, ftp1.shorewall.net,etc.)</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>The network is pictured in the following diagram:</para>
|
|
|
|
<graphic align="center" fileref="images/Network2009.png" />
|
|
|
|
<para>Because of the speed of the cable provider, all traffic uses that
|
|
provider unless there is a specific need for the traffic to use the DSL
|
|
line.</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Responses to connections from the Internet to one of the DSL IP
|
|
addresses -- the <emphasis role="bold">track</emphasis> option takes
|
|
care of that.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Connections initiated by the server and connections requested by
|
|
clients on the firewall that have bound their local socket to one of
|
|
the DSL IP addresses. Two entries in
|
|
<filename>/etc/shorewall/route_rules</filename> take care of that
|
|
traffic.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>As a consequence, I have disabled all route filtering on the
|
|
firewall and only use the <emphasis role="bold">balance</emphasis> option
|
|
in <filename>/etc/shorewall/providers</filename> on the Comcast provider
|
|
whose default route in the main table is established by DHCP. By
|
|
specifying the <emphasis role="bold">fallback</emphasis> option on
|
|
Avvanta, I ensure that there is still a default route if Comcast is down.
|
|
<link linkend="swping">swping</link> is used to monitor the links.</para>
|
|
|
|
<para><filename>/etc/sysctl.conf</filename>:</para>
|
|
|
|
<programlisting>net.ipv4.conf.all.rp_filter = 0</programlisting>
|
|
|
|
<para><filename>/etc/shorewall/shorewall.conf</filename>:</para>
|
|
|
|
<programlisting>ROUTE_FILTER=No
|
|
RESTORE_DEFAULT_ROUTE=No</programlisting>
|
|
|
|
<para>The RESTORE_DEFAULT_ROUTE option was added in Shorewall-perl 4.2.6
|
|
and causes the default route in the main table to be deleted when the
|
|
Comcast link is unavailable. That way, the default route in the default
|
|
table will be used until Comcast is available again.</para>
|
|
|
|
<para><filename>/etc/shorewall/providers</filename>:</para>
|
|
|
|
<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
|
Avvanta 1 0x100 main eth0 206.124.146.254 track,loose,fallback eth2,eth4,tun*
|
|
Comcast 2 0x200 main eth3 detect track,balance eth2,eth4,tun*
|
|
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE</programlisting>
|
|
|
|
<para>The <emphasis role="bold">loose</emphasis> option on Avvanta results
|
|
in fewer routing rules. The first two routing rules below insure that all
|
|
traffic from Avvanta-assigned IP addresses is sent via the Avvanta
|
|
provider. Note that because the Comcast line has a dynamic IP address, I
|
|
am not able to use USE_DEFAULT_RT=Yes in
|
|
<filename>/etc/shorewall/shorewall.conf</filename>. The 'tun*' included in
|
|
the COPY column is there because I run a routed OpenVPN server on the
|
|
firewall.</para>
|
|
|
|
<para><filename>/etc/shorewall/route_rules</filename>:</para>
|
|
|
|
<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
|
- 172.20.0.0/24 main 1000 # Addresses assigned by routed OpenVPN server
|
|
206.124.146.176/30 - Avvanta 26000
|
|
206.124.146.180 - Avvanta 26000
|
|
- 216.168.3.44 Avvanta 26000 # Avvanta NNTP Server -- verifies source IP address
|
|
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
|
|
|
|
<para>The <filename>/etc/shorewall/route_rules </filename>entries provide
|
|
all of the provider selection necessary so my
|
|
<filename>/etc/shorewall/tcrules</filename> file is used exclusively for
|
|
traffic shaping of the Avvanta line. Note that I still need to provide
|
|
values in the MARK colum of <filename>/etc/shorewall/providers</filename>
|
|
because I specify <emphasis role="bold">track</emphasis> on both
|
|
providers.</para>
|
|
|
|
<para>Here is the output of <command>shorewall show
|
|
routing</command>:</para>
|
|
|
|
<programlisting>Routing Rules
|
|
|
|
0: from all lookup local
|
|
1000: from all to 172.20.0.0/24 lookup main
|
|
10000: from all fwmark 0x100 lookup Avvanta
|
|
10001: from all fwmark 0x200 lookup Comcast
|
|
20256: from 71.227.156.229 lookup Comcast
|
|
26000: from 206.124.146.176/30 lookup Avvanta
|
|
26000: from 206.124.146.180 lookup Avvanta
|
|
26000: from all to 216.168.3.44 lookup Avvanta
|
|
32766: from all lookup main
|
|
32767: from all lookup default
|
|
|
|
Table Avvanta:
|
|
|
|
206.124.146.254 dev eth0 scope link src 206.124.146.176
|
|
206.124.146.177 dev eth4 scope link
|
|
172.20.1.0/24 dev eth2 proto kernel scope link src 172.20.1.254
|
|
206.124.146.0/24 dev eth0 proto kernel scope link src 206.124.146.176
|
|
169.254.0.0/16 dev eth0 scope link
|
|
default via 206.124.146.254 dev eth0 src 206.124.146.176
|
|
|
|
Table Comcast:
|
|
|
|
206.124.146.177 dev eth4 scope link
|
|
71.227.156.1 dev eth3 scope link src 71.227.156.229
|
|
172.20.1.0/24 dev eth2 proto kernel scope link src 172.20.1.254
|
|
71.227.156.0/23 dev eth3 proto kernel scope link src 71.227.156.229
|
|
default via 71.227.156.1 dev eth3 src 71.227.156.229
|
|
|
|
Table default:
|
|
|
|
default via 206.124.146.254 dev eth0 metric 1
|
|
|
|
Table local:
|
|
|
|
broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1
|
|
broadcast 172.20.1.0 dev eth2 proto kernel scope link src 172.20.1.254
|
|
broadcast 206.124.146.255 dev eth0 proto kernel scope link src 206.124.146.176
|
|
local 206.124.146.179 dev eth0 proto kernel scope host src 206.124.146.176
|
|
local 206.124.146.178 dev eth0 proto kernel scope host src 206.124.146.176
|
|
local 206.124.146.176 dev eth0 proto kernel scope host src 206.124.146.176
|
|
local 206.124.146.176 dev eth4 proto kernel scope host src 206.124.146.176
|
|
broadcast 71.227.157.255 dev eth3 proto kernel scope link src 71.227.156.229
|
|
broadcast 71.227.156.0 dev eth3 proto kernel scope link src 71.227.156.229
|
|
local 172.20.1.254 dev eth2 proto kernel scope host src 172.20.1.254
|
|
local 127.0.0.2 dev lo proto kernel scope host src 127.0.0.1
|
|
broadcast 172.20.1.255 dev eth2 proto kernel scope link src 172.20.1.254
|
|
local 71.227.156.229 dev eth3 proto kernel scope host src 71.227.156.229
|
|
broadcast 206.124.146.0 dev eth0 proto kernel scope link src 206.124.146.176
|
|
broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1
|
|
local 206.124.146.180 dev eth0 proto kernel scope host src 206.124.146.176
|
|
local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1
|
|
local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1
|
|
|
|
Table main:
|
|
|
|
206.124.146.177 dev eth4 scope link
|
|
172.20.1.0/24 dev eth2 proto kernel scope link src 172.20.1.254
|
|
206.124.146.0/24 dev eth0 proto kernel scope link src 206.124.146.176
|
|
71.227.156.0/23 dev eth3 proto kernel scope link src 71.227.156.229
|
|
169.254.0.0/16 dev eth0 scope link
|
|
127.0.0.0/8 dev lo scope link
|
|
default via 71.227.156.1 dev eth3 </programlisting>
|
|
|
|
<para><filename>/etc/shorewall/interfaces</filename>:</para>
|
|
|
|
<programlisting>#ZONE INTERFACE BROADCAST OPTIONS
|
|
loc eth2 detect dhcp,routeback
|
|
dmz eth4 detect
|
|
net eth0 detect dhcp,blacklist,tcpflags,optional
|
|
net eth3 detect dhcp,blacklist,tcpflags,optional
|
|
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>
|
|
|
|
<para><filename>/etc/shorewall/masq</filename>:</para>
|
|
|
|
<programlisting>#INTERFACE SOURCE ADDRESS PROTO PORT(S) IPSEC
|
|
|
|
COMMENT Masquerade Local Network
|
|
eth3 0.0.0.0/0
|
|
eth0 !206.124.146.0/24 206.124.146.179
|
|
|
|
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE</programlisting>
|
|
|
|
<para>All traffic leaving eth3 must use the dynamic IP address assigned to
|
|
that interface as the SOURCE address. All traffic leaving eth0 that does
|
|
not have a SOURCE address falling within the Avvanta subnet
|
|
(206.124.146.0/24) must have its SOURCE address changed to
|
|
206.124.146.179.</para>
|
|
</section>
|
|
</article>
|