forked from extern/shorewall_code
ec9a9c790e
git-svn-id: https://shorewall.svn.sourceforge.net/svnroot/shorewall/trunk@7368 fbd18981-670d-0410-9b5c-8dc0c1a9a2bb
1003 lines
43 KiB
XML
1003 lines
43 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
|
|
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
|
|
<article>
|
|
<!--$Id$-->
|
|
|
|
<articleinfo>
|
|
<title>Shorewall and Multiple Internet Connections</title>
|
|
|
|
<authorgroup>
|
|
<author>
|
|
<firstname>Tom</firstname>
|
|
|
|
<surname>Eastep</surname>
|
|
</author>
|
|
</authorgroup>
|
|
|
|
<pubdate><?dbtimestamp format="Y/m/d"?></pubdate>
|
|
|
|
<copyright>
|
|
<year>2005</year>
|
|
|
|
<year>2006</year>
|
|
|
|
<year>2007</year>
|
|
|
|
<holder>Thomas M. Eastep</holder>
|
|
</copyright>
|
|
|
|
<legalnotice>
|
|
<para>Permission is granted to copy, distribute and/or modify this
|
|
document under the terms of the GNU Free Documentation License, Version
|
|
1.2 or any later version published by the Free Software Foundation; with
|
|
no Invariant Sections, with no Front-Cover, and with no Back-Cover
|
|
Texts. A copy of the license is included in the section entitled
|
|
<quote><ulink url="GnuCopyright.htm">GNU Free Documentation
|
|
License</ulink></quote>.</para>
|
|
</legalnotice>
|
|
</articleinfo>
|
|
|
|
<warning>
|
|
<para>Make sure you are<emphasis role="bold"> running a current,
|
|
vendor-supported distribution</emphasis>, before attempting to perform
|
|
this setup, older distributions do not meet the minimum requirements,and
|
|
you will need to recompile iptables, kernel and other software on your
|
|
system. If you don't follow this advice,<emphasis role="bold">we will
|
|
<emphasis role="bold">not</emphasis> be able to help
|
|
you</emphasis>.</para>
|
|
</warning>
|
|
|
|
<warning>
|
|
<para>Reading just Shorewall documentation is probably not going to give
|
|
you enough background to use this material. Shorewall may make iptables
|
|
easy but the Shorewall team doesn't have the resources to be able to
|
|
spoon-feed Linux policy routing to you (please remember that the user's
|
|
manual for a tractor doesn't teach you to grow corn either). You will
|
|
likely need to refer to the following additional information:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>The LARTC HOWTO: <ulink
|
|
url="http://www.lartc.org">http://www.lartc.org</ulink></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Output of <command>man ip</command></para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Output of <command>ip route help</command> and <command>ip rule
|
|
help</command></para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</warning>
|
|
|
|
<section id="Support">
|
|
<title>Multiple Internet Connection Support</title>
|
|
|
|
<para>Beginning with Shorewall 2.3.2, limited support is included for
|
|
multiple internet connections. Limitations of this support are as
|
|
follows:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>It utilizes static routing configuration. As such, there is no
|
|
provision for reacting to the failure of either of the uplinks.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>The routing changes are made and the route cache is purged when
|
|
Shorewall is started <emphasis role="bold">and when Shorewall is
|
|
restarted</emphasis> (unless you specify the "-n" option to
|
|
<command>shorewall restart</command>). Ideally, restarting the packet
|
|
filter should have no effect on routing.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Prior to Shorewall 3.4.0, the routes and route rules added by
|
|
this support were not completely removed during <command>shorewall
|
|
stop</command>, <command>shorewall clear</command> or
|
|
<command>shorewall restart</command>.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<section id="Overview">
|
|
<title>Overview</title>
|
|
|
|
<para>Let's assume that a firewall is connected via two separate
|
|
ethernet interfaces to two different ISPs as in the following
|
|
diagram.</para>
|
|
|
|
<graphic align="center" fileref="images/TwoISPs.png" valign="middle" />
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>eth0 connects to ISP1. The IP address of eth0 is
|
|
206.124.146.176 and the ISP's gateway router has IP address
|
|
206.124.146.254.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>eth1 connects to ISP 2. The IP address of eth1 is
|
|
130.252.99.27 and the ISP's gateway router has IP address
|
|
130.252.99.254.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>eth2 connects to the local LAN. Its IP configuration is not
|
|
relevant to this discussion.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Each of these <firstterm>providers</firstterm> is described in an
|
|
entry in the file <filename>/etc/shorewall/providers</filename>.</para>
|
|
|
|
<para>Entries in <filename>/etc/shorewall/providers</filename> can
|
|
specify that outgoing connections are to be load-balanced between the
|
|
two ISPs. Entries in <filename>/etc/shorewall/tcrules</filename> can be
|
|
used to direct particular outgoing connections to one ISP or the other.
|
|
Use of <filename>/etc/shorewall/tcrules</filename> is not required for
|
|
<filename>/etc/shorewall/providers</filename> to work, but you must
|
|
select a unique MARK value for each provider so Shorewall can set up the
|
|
correct marking rules for you.</para>
|
|
|
|
<para>When you use the <emphasis role="bold">track</emphasis> option in
|
|
<filename>/etc/shorewall/providers</filename>, connections from the
|
|
internet are automatically routed back out of the correct interface and
|
|
through the correct ISP gateway. This works whether the connection is
|
|
handled by the firewall itself or if it is routed or port-forwarded to a
|
|
system behind the firewall.</para>
|
|
|
|
<para>Shorewall will set up the routing and will update the
|
|
<filename>/etc/iproute2/rt_tables</filename> to include the table names
|
|
and numbers of the tables that it adds.</para>
|
|
|
|
<caution>
|
|
<para>This feature uses <ulink url="traffic_shaping.htm">packet
|
|
marking</ulink> to control the routing. As a consequence, there are
|
|
some restrictions concerning entries in
|
|
<filename>/etc/shorewall/tcrules</filename>:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Packet marking for traffic control purposes may not be done
|
|
in the PREROUTING table for connections involving providers with
|
|
'track' specified (see below).</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You may not use the SAVE or RESTORE options.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You may not use connection marking.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</caution>
|
|
|
|
<para>The <filename>/etc/shorewall/providers</filename> file can also be
|
|
used in other routing scenarios. See the <ulink
|
|
url="Shorewall_Squid_Usage.html">Squid documentation</ulink> for an
|
|
example.</para>
|
|
</section>
|
|
|
|
<section id="providers">
|
|
<title>/etc/shorewall/providers File</title>
|
|
|
|
<para>Entries in this file have the following columns. As in all
|
|
Shorewall configuration files, enter "-" in a column if you don't want
|
|
to enter any value.</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term>NAME</term>
|
|
|
|
<listitem>
|
|
<para>The provider name. Must begin with a letter and consist of
|
|
letters and digits. The provider name becomes the name of the
|
|
generated routing table for this provider.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>NUMBER</term>
|
|
|
|
<listitem>
|
|
<para>A number between 1 and 252. This becomes the routing table
|
|
number for the generated table for this provider.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>MARK</term>
|
|
|
|
<listitem>
|
|
<para>A mark value used in your /etc/shorewall/tcrules file to
|
|
direct packets to this provider. Shorewall will also mark
|
|
connections that have seen input from this provider with this
|
|
value and will restore the packet mark in the PREROUTING CHAIN.
|
|
Mark values must be in the range 1-255.</para>
|
|
|
|
<para>Beginning with Shorewall version 3.2.0 Beta 6, you may use
|
|
may set HIGH_ROUTE_MARKS=Yes in
|
|
<filename>/etc/shorewall/shorewall.conf</filename>. This allows
|
|
you to:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Use connection marks for traffic shaping, provided that
|
|
you assign those marks in the FORWARD chain.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Use mark values > 255 for provider marks in this
|
|
column. These mark values must be a multiple of 256 in the
|
|
range 256-65280 (hex equivalent 0x100 - 0xFF00 with the
|
|
low-order 8 bits being zero).</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>DUPLICATE</term>
|
|
|
|
<listitem>
|
|
<para>Gives the name or number of a routing table to duplicate.
|
|
May be 'main' or the name or number of a previously declared
|
|
provider. For most applications, you want to specify 'main'
|
|
here.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>INTERFACE</term>
|
|
|
|
<listitem>
|
|
<para>The name of the interface to the provider.</para>
|
|
|
|
<caution>
|
|
<para>The Shorewall implementation of Multi-ISP support assumes
|
|
that each provider has its own interface.</para>
|
|
</caution>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>GATEWAY</term>
|
|
|
|
<listitem>
|
|
<para>The IP address of the provider's Gateway router.</para>
|
|
|
|
<para>You can enter <emphasis role="bold">detect</emphasis> here
|
|
and Shorewall will attempt to automatically determine the gateway
|
|
IP address.</para>
|
|
|
|
<para><emphasis role="bold">Hint:</emphasis> <emphasis
|
|
role="bold">"detect"</emphasis> is appropriate for use in cases
|
|
where the interface named in the INTERFACE column is dynamically
|
|
configured via DHCP etc.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>OPTIONS</term>
|
|
|
|
<listitem>
|
|
<para>A comma-separated list from the following:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term>track</term>
|
|
|
|
<listitem>
|
|
<para>If specified, connections FROM this interface are to
|
|
be tracked so that responses may be routed back out this
|
|
same interface.</para>
|
|
|
|
<para>You want to specify 'track' if internet hosts will be
|
|
connecting to local servers through this provider. Any time
|
|
that you specify 'track', you will also want to specify
|
|
'balance' (see below).</para>
|
|
|
|
<para>Use of this feature requires that your kernel and
|
|
iptables include CONNMARK target and connmark match support
|
|
(<emphasis role="bold">Warning</emphasis>: Standard
|
|
<trademark>Debian</trademark> and
|
|
<trademark>Ubuntu</trademark> kernels are lacking that
|
|
support!).</para>
|
|
|
|
<warning>
|
|
<para>A <emphasis role="bold">bug</emphasis> in Shorewall
|
|
versions 3.2.0-3.2.10, 3.4.0-3.4.6 and 4.0.0-4.0.2
|
|
prevents proper handling of PREROUTING marks when
|
|
HIGH_ROUTE_MARKS=No and the <emphasis
|
|
role="bold">track</emphasis> option is specified. Patches
|
|
are available to correct this problem:</para>
|
|
|
|
<para>Shorewall version 3.2.0-3.4.3: <ulink
|
|
url="http://www1.shorewall.net/pub/shorewall/3.2/shorewall-3.2.10/errata/patches/Shorewall/patch-3.2.10-2.diff">http://www1.shorewall.net/pub/shorewall/3.2/shorewall-3.2.10/errata/patches/Shorewall/patch-3.2.10-2.diff
|
|
</ulink></para>
|
|
|
|
<para>Shorewall version 3.4.4-3.4.6: <ulink
|
|
url="http://www1.shorewall.net/pub/shorewall/3.4/shorewall-3.4.6/errata/patches/Shorewall/patch-3.4.6-1.diff">http://www1.shorewall.net/pub/shorewall/3.4/shorewall-3.4.6/errata/patches/Shorewall/patch-3.4.6-1.diff</ulink></para>
|
|
|
|
<para>Shorewall-shell version 4.0.0-4.0.2: <ulink
|
|
url="http://www1.shorewall.net/pub/shorewall/4.0/shorewall-4.0.2/errata/patches/Shorewall-shell/patch-shell-4.0.2-2.diff">http://www1.shorewall.net/pub/shorewall/4.0/shorewall-4.0.2/errata/patches/Shorewall-shell/patch-shell-4.0.2-2.diff</ulink></para>
|
|
</warning>
|
|
|
|
<important>
|
|
<para>If you are using
|
|
<filename>/etc/shorewall/providers</filename> because you
|
|
have multiple internet connections, we recommend that you
|
|
specify 'track' even if you don't need it. It helps
|
|
maintain long-term connections in which there are
|
|
significant periods with no traffic.</para>
|
|
</important>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>balance</term>
|
|
|
|
<listitem>
|
|
<para>The providers that have 'balance' specified will get
|
|
outbound traffic load-balanced among them. Balancing will
|
|
not be perfect, as it is route based, and routes are cached.
|
|
This means that routes to often-used sites will always be
|
|
over the same provider.</para>
|
|
|
|
<para>By default, each provider is given the same weight (1)
|
|
. You can change the weight of a given provider by following
|
|
<emphasis>balance</emphasis> with "=" and the desired weight
|
|
(e.g., balance=2). The weights reflect the relative
|
|
bandwidth of the providers connections and should be small
|
|
numbers since the kernel actually creates additional default
|
|
routes for each weight increment.</para>
|
|
|
|
<important>
|
|
<para>If you are using
|
|
<filename>/etc/shorewall/providers</filename> because you
|
|
have multiple internet connections, we recommend that you
|
|
specify 'balance' even if you don't need it. You can still
|
|
use entries in <filename>/etc/shorewall/tcrules</filename>
|
|
to force all traffic to one provider or another.<note>
|
|
<para>If you don't heed this advice then be prepared
|
|
to read <ulink url="FAQ.htm#faq57">FAQ 57</ulink> and
|
|
<ulink url="FAQ.htm#faq58">FAQ 58</ulink>.</para>
|
|
</note></para>
|
|
</important>
|
|
|
|
<important>
|
|
<para>If you specify 'balance' and still find that all
|
|
traffic is going out through only one provider, you may
|
|
need to install a kernel built with
|
|
CONFIG_IP_ROUTE_MULTIPATH_CACHED=n. Several users have
|
|
reported that this change has corrected similar
|
|
problems.</para>
|
|
|
|
<para>The SuSE 10.0 kernel is subject to this problem, and
|
|
<ulink
|
|
url="https://bugzilla.novell.com/show_bug.cgi?id=190908">
|
|
a kernel oops may result in this circumstance.</ulink>
|
|
SUSE 10.1 and SLES 10 have
|
|
CONFIG_IP_ROUTE_MULTIPATH_CACHED=n set by default. The
|
|
source of the problem seems to be <ulink
|
|
url="http://news.gmane.org/find-root.php?message_id=%3c00da01c5b35a%24b12b9860%241b00a8c0%40cruncher%3e">an
|
|
incompatibility between the LARTC patches and
|
|
CONFIG_IP_ROUTE_MULTIPATH_CACHED.</ulink></para>
|
|
</important>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>loose</term>
|
|
|
|
<listitem>
|
|
<para>Do not include routing rules that force traffic whose
|
|
source IP is an address of the INTERFACE to be routed to
|
|
this provider. Useful for defining providers that are to be
|
|
used only when the appropriate packet mark is applied.
|
|
Should not be specified together with <emphasis
|
|
role="bold">balance</emphasis>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>optional (added in Shorewall 3.2.2)</term>
|
|
|
|
<listitem>
|
|
<para>Shorewall will determine of this interface is up and
|
|
has a configured IPv4 address. If it is not, a warning is
|
|
issued and this provider is not configured.</para>
|
|
|
|
<note>
|
|
<para>'optional' is designed to detect interface states
|
|
that will cause <command>shorewall start</command> or
|
|
<command>shorewall restart</command> to fail; just because
|
|
an interface is in a state that Shorewall can [re]start
|
|
without error doesn't mean that traffic can actually be
|
|
sent through the interface.</para>
|
|
|
|
<para>Beginning with Shorewall-perl 4.0.3, you can supply
|
|
an 'isusable' <ulink
|
|
url="shorewall_extension_scripts.htm">extension
|
|
script</ulink> to extend Shorewall's interface state
|
|
detection.</para>
|
|
</note>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
|
|
<para>For those of you who are termnally confused between<emphasis
|
|
role="bold"> track</emphasis> and <emphasis
|
|
role="bold">balance</emphasis>:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><emphasis role="bold">track</emphasis> governs incoming
|
|
connections.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><emphasis role="bold">balance</emphasis> governs
|
|
outgoing connections.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>COPY</term>
|
|
|
|
<listitem>
|
|
<para>A comma-separated list if interface names. Wildcards
|
|
specified using an asterisk ("*") are permitted (e.g., tun*
|
|
).</para>
|
|
|
|
<para>When you specify an existing table in the DUPLICATE column,
|
|
Shorewall copies all routes through the interface specified in the
|
|
INTERFACE column plus the interfaces listed in this column.
|
|
Normally, you will list all interfaces on your firewall in this
|
|
column except those internet interfaces specified in the INTERFACE
|
|
column of entries in this file.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</section>
|
|
|
|
<section id="Providers">
|
|
<title>What an entry in the Providers File Does</title>
|
|
|
|
<para>Adding another entry in the providers file simply creates an
|
|
alternate routing table for you. In addition:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>Unless <emphasis role="bold">loose</emphasis> is specified, an
|
|
ip rule is generated for each IP address on the INTERFACE that
|
|
routes traffic from that address through the associated routing
|
|
table.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>If you specify <emphasis role="bold">track</emphasis>, then
|
|
connections which have had at least one packet arrive on the
|
|
interface listed in the INTERFACE column have their connection mark
|
|
set to the value in the MARK column. In the PREROUTING chain,
|
|
packets with a connection mark have their packet mark set to the
|
|
value of the associated connection mark; packets marked in this way
|
|
bypass any prerouting rules that you create in
|
|
<filename>/etc/shorewall/tcrules</filename>. This ensures that
|
|
packets associated with connections from outside are always routed
|
|
out of the correct interface.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>If you specify <emphasis role="bold">balance</emphasis>, then
|
|
Shorewall will replace the 'default' route with weight 100 in the
|
|
'main' routing table with a load-balancing route among those
|
|
gateways where <emphasis role="bold">balance</emphasis> was
|
|
specified. So if you configure default routes, be sure that their
|
|
weight is less than 100 or the route added by Shorewall will not be
|
|
used.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>That's <emphasis role="bold">all</emphasis> that these entries do.
|
|
You still have to follow the principle stated in the <ulink
|
|
url="Shorewall_and_Routing.html">Shorewall Routing
|
|
documentation</ulink>:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>Routing determines where packets are to be sent.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Once routing determines where the packet is to go, the
|
|
firewall (Shorewall) determines if the packet is allowed to go
|
|
there.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>The bottom line is that if you want traffic to go out through a
|
|
particular provider then you <emphasis>must </emphasis>mark that traffic
|
|
with the provider's MARK value in
|
|
<filename>/etc/shorewall/tcrules</filename> and you must do that marking
|
|
in the PREROUTING chain; or, you must provide the appropriate rules in
|
|
<filename>/etc/shorewall/route_rules</filename>.</para>
|
|
|
|
<warning id="Undo">
|
|
<para>If you are running a Shorewall version prior to 3.4.0, entries
|
|
in <filename>/etc/shorewall/providers</filename> permanently alter
|
|
your firewall/gateway's routing; that is, the effect of these changes
|
|
is not reversed by <command>shorewall stop</command> or
|
|
<command>shorewall clear</command>. To restore routing to its original
|
|
state, you may have to restart your network. This can usually be done
|
|
by <command>/etc/init.d/network restart</command> or
|
|
<command>/etc/init.d/networking restart</command>. Check your
|
|
distribution's networking documentation.</para>
|
|
|
|
<para>Here are some additional things to consider:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>You can mitigate the effect of the Shorewall-generated
|
|
changes to your routing table by specifying a
|
|
<emphasis>metric</emphasis> for each default route that you
|
|
configure. Shorewall will generate a load-balancing default route
|
|
(assuming that <emphasis role="bold">balance</emphasis> has been
|
|
specified for some of the providers) that does not include a
|
|
metric and that will therefore not replace any existing route that
|
|
has a non-zero metric.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>The <command>-n</command> option to <command>shorewall
|
|
restart</command> and <command>shorewall restore</command> can be
|
|
used to prevent the command from changing your routing.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>The <filename>/etc/shorewall/stopped</filename> file can
|
|
also be used to restore routing when you stop Shorewall. With your
|
|
firewall in its normal (single-table) routing configuration, you
|
|
can capture the contents as follows:</para>
|
|
|
|
<programlisting>ip route ls > routes</programlisting>
|
|
|
|
<para>Here's what the <filename>routes</filename> file looked like
|
|
after I did that on my firewall:</para>
|
|
|
|
<programlisting>192.168.1.1 dev eth3 scope link
|
|
206.124.146.177 dev eth1 scope link
|
|
192.168.2.2 dev tun0 proto kernel scope link src 192.168.2.1
|
|
192.168.2.0/24 via 192.168.2.2 dev tun0
|
|
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.254
|
|
206.124.146.0/24 dev eth3 proto kernel scope link src 206.124.146.176
|
|
169.254.0.0/16 dev eth0 scope link
|
|
127.0.0.0/8 dev lo scope link
|
|
default via 206.124.146.254 dev eth3</programlisting>
|
|
|
|
<para>Now edit the file as shown below:</para>
|
|
|
|
<programlisting><command>ip route flush table main
|
|
ip route add</command> 192.168.1.1 dev eth3 scope link
|
|
<command>ip route add </command>206.124.146.177 dev eth1 scope link
|
|
<command>ip route add </command>192.168.2.2 dev tun0 proto kernel scope link src 192.168.2.1
|
|
<command>ip route add </command>192.168.2.0/24 via 192.168.2.2 dev tun0
|
|
<command>ip route add </command>192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.254
|
|
<command>ip route add </command>206.124.146.0/24 dev eth3 proto kernel scope link src 206.124.146.176
|
|
<command>ip route add </command>169.254.0.0/16 dev eth0 scope link
|
|
<command>ip route add </command>127.0.0.0/8 dev lo scope link
|
|
<command>ip route add </command>default via 206.124.146.254 dev eth3
|
|
<command>ip route flush cache</command></programlisting>
|
|
|
|
<para>Now paste the contents of that file into
|
|
<filename>/etc/shorewall/stopped</filename>.</para>
|
|
|
|
<para>You might also want to consider adding the following to the
|
|
file:</para>
|
|
|
|
<programlisting><command>ip rule ls</command> | while read priority rule; do
|
|
case ${priority%:} in
|
|
0|3276[67])
|
|
;;
|
|
*)
|
|
ip rule del $rule
|
|
;;
|
|
esac
|
|
done</programlisting>
|
|
|
|
<para>That code will delete all but the default routing
|
|
rules.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</warning>
|
|
</section>
|
|
|
|
<section id="Provider_Doesnt">
|
|
<title>What an entry in the Providers File Does NOT Do</title>
|
|
|
|
<para>Given that Shorewall is simply a tool to configure Netfilter and
|
|
does not run continuously in your system, entries in the providers file
|
|
<emphasis role="bold">do not provide any automatic failover in the event
|
|
of failure of one of your Internet connections</emphasis>.</para>
|
|
</section>
|
|
|
|
<section id="Martians">
|
|
<title>Martians</title>
|
|
|
|
<para>One problem that often arises with Multi-ISP configuration is
|
|
'Martians'. If your internet interfaces are configured with the
|
|
<emphasis role="bold">routefilter</emphasis> option in
|
|
<filename>/etc/shorewall/interfaces</filename> (remember that if you set
|
|
that option, you should also select <emphasis
|
|
role="bold">logmartians</emphasis>), then things may not work correctly
|
|
and you will see messages like this:</para>
|
|
|
|
<programlisting>Feb 9 17:23:45 gw.ilinx kernel: martian source 206.124.146.176 from 64.86.88.116, on dev eth1
|
|
Feb 9 17:23:45 gw.ilinx kernel: ll header: 00:a0:24:2a:1f:72:00:13:5f:07:97:05:08:00</programlisting>
|
|
|
|
<para>The above message is somewhat awkwardly phrased. The source IP in
|
|
this incoming packet was 64.86.88.116 and the destination IP address is
|
|
206.124.146.176. Another gotcha is that the incoming packet has already
|
|
had the destination IP address changed for DNAT or because the original
|
|
outgoing connection was altered by an entry in
|
|
<filename>/etc/shorewall/masq</filename> (SNAT or Masquerade). So the
|
|
destination IP address (206.124.146.176) may not have been the
|
|
destination IP address in the packet as it was initially
|
|
received.</para>
|
|
|
|
<para>There a couple of common causes for these problems:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>You have connected both of your external interfaces to the
|
|
same hub/switch. Connecting multiple firewall interfaces to a common
|
|
hub or switch is always a bad idea that will result in
|
|
hard-to-diagnose problems.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You are specifying both the <emphasis
|
|
role="bold">loose</emphasis> and <emphasis
|
|
role="bold">balance</emphasis> options on your provider(s). This
|
|
causes individual connections to ping-pong back and forth between
|
|
the interfaces which is guaranteed to cause problems.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>You are redirecting traffic from the local system out of one
|
|
interface or the other using packet marking in your
|
|
<filename>/etc/shorewall/tcrules</filename> file. A better approach
|
|
is to configure the application to use the appropriate local IP
|
|
address (the IP address of the interface that you want the
|
|
application to use). See <link linkend="Local">below</link>.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>If all else fails, remove the <emphasis
|
|
role="bold">routefilter</emphasis> option from your external interfaces.
|
|
If you do this, you may wish to add rules to log and drop packets from
|
|
the Internet that have source addresses in your local networks. For
|
|
example, if the local LAN in the above diagram is 192.168.1.0/24, then
|
|
you would add this rule:</para>
|
|
|
|
<programlisting>#ACTION SOURCE DEST
|
|
DROP:info net:192.168.1.0/24 all</programlisting>
|
|
|
|
<para>Be sure the above rule is added before any other rules with
|
|
<emphasis>net</emphasis> in the SOURCE column.</para>
|
|
</section>
|
|
|
|
<section id="Example1">
|
|
<title id="Example">Example</title>
|
|
|
|
<para>The configuration in the figure at the top of this section would
|
|
be specified in <filename>/etc/shorewall/providers</filename> as
|
|
follows.</para>
|
|
|
|
<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
|
ISP1 1 1 main eth0 206.124.146.254 track,balance eth2
|
|
ISP2 2 2 main eth1 130.252.99.254 track,balance eth2</programlisting>
|
|
|
|
<para>Other configuration files go something like this:</para>
|
|
|
|
<para><filename>/etc/shorewall/interfaces</filename>:</para>
|
|
|
|
<programlisting>#ZONE INTERFACE BROADCAST OPTIONS
|
|
net eth0 detect …
|
|
net eth1 detect …</programlisting>
|
|
|
|
<para><filename>/etc/shorewall/policy</filename>:</para>
|
|
|
|
<programlisting>#SOURCE DESTINATION POLICY LIMIT:BURST
|
|
net net DROP</programlisting>
|
|
|
|
<para>Regardless of whether you have masqueraded hosts or not, the
|
|
following entries are required in
|
|
<filename>/etc/shorewall/masq</filename> if you plan to redirect
|
|
connections from the firewall using entries in
|
|
<filename>/etc/shorewall/tcrules</filename> or if you specify <emphasis
|
|
role="bold">balance</emphasis> on your providers.</para>
|
|
|
|
<programlisting>#INTERFACE SUBNET ADDRESS
|
|
eth0 130.252.99.27 206.124.146.176
|
|
eth1 206.124.146.176 130.252.99.27</programlisting>
|
|
|
|
<para>Those entries ensure that traffic originating on the firewall and
|
|
redirected via packet marks always has the source IP address
|
|
corresponding to the interface that it is routed out of.</para>
|
|
|
|
<note>
|
|
<para>If you have a Dynamic IP address on either of the interfaces,
|
|
you can use shell variables to construct the above rules. For example,
|
|
if <filename class="devicefile">eth0</filename> had a dynamic IP
|
|
address, then:</para>
|
|
|
|
<para><filename>/etc/shorewall/params</filename>:</para>
|
|
|
|
<programlisting>ETH0_IP=$(find_first_interface_address eth0)</programlisting>
|
|
|
|
<para>/etc/shorewall/masq:</para>
|
|
|
|
<programlisting>#INTERFACE SUBNET ADDRESS
|
|
eth0 130.252.99.27 $ETH0_IP
|
|
eth1 $ETH0_IP 130.252.99.27</programlisting>
|
|
</note>
|
|
|
|
<para>If you have masqueraded hosts, be sure to update
|
|
<filename>/etc/shorewall/masq</filename> to masquerade to both ISPs. For
|
|
example, if you masquerade all hosts connected to <filename
|
|
class="devicefile">eth2</filename> then:</para>
|
|
|
|
<programlisting>#INTERFACE SUBNET ADDRESS
|
|
eth0 eth2 206.124.146.176
|
|
eth1 eth2 130.252.99.27</programlisting>
|
|
|
|
<warning>
|
|
<para>Entries in <filename>/etc/shorewall/masq</filename> have no
|
|
effect on which ISP a particular connection will be sent through. That
|
|
is rather the purpose of entries in
|
|
<filename>/etc/shorewall/tcrules</filename> or
|
|
<filename>/etc/shorewall/route_rules</filename>.</para>
|
|
</warning>
|
|
|
|
<para>Now suppose that you want to route all outgoing SMTP traffic from
|
|
your local network through ISP 2. You would make this entry in <ulink
|
|
url="traffic_shaping.htm">/etc/shorewall/tcrules</ulink> (and if you are
|
|
running a version of Shorewall earlier than 3.0.0, you would set
|
|
TC_ENABLED=Yes in <ulink
|
|
url="???">/etc/shorewall/shorewall.conf</ulink>).</para>
|
|
|
|
<programlisting>#MARK SOURCE DEST PROTO PORT(S) CLIENT USER TEST
|
|
# PORT(S)
|
|
2:P <local network> 0.0.0.0/0 tcp 25</programlisting>
|
|
</section>
|
|
|
|
<section id="morethan2">
|
|
<title>More than 2 Providers</title>
|
|
|
|
<para>When there are more than two providers, you need to extend the
|
|
two-provider case in the expected way:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>For each external address, you need an entry in
|
|
<filename>/etc/shorewall/masq</filename> to handle the case where a
|
|
connection using that address as the SOURCE is sent out of the
|
|
interfaces other than the one that the address is configured
|
|
on.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>For each external interface, you need to add an entry to
|
|
<filename>/etc/shorewall/masq</filename> for each internal network
|
|
that needs to be masqueraded (or use SNAT) through that
|
|
interface.</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<para>If we extend the above example to add eth3 with IP address
|
|
16.105.78.4 with gateway 16.105.78.254, then:</para>
|
|
|
|
<para><filename>/etc/shorewall/providers</filename>:<programlisting>#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
|
|
ISP1 1 1 main eth0 206.124.146.254 track,balance eth2
|
|
ISP2 2 2 main eth1 130.252.99.254 track,balance eth2
|
|
ISP3 3 3 main eth3 16.105.78.254 track,balance eth2</programlisting></para>
|
|
|
|
<para><filename>/etc/shorewall/masq</filename>:<programlisting>#INTERFACE SUBNET ADDRESS
|
|
eth0 130.252.99.27 206.124.146.176
|
|
eth3 130.252.99.27 16.105.78.4
|
|
eth1 206.124.146.176 130.252.99.27
|
|
eth3 206.124.146.176 16.105.78.4
|
|
eth0 16.106.78.4 206.124.146.176
|
|
eth1 16.106.78.4 130.252.99.27
|
|
eth0 eth2 206.124.146.176
|
|
eth1 eth2 130.252.99.27
|
|
eth3 eth2 16.105.78.4</programlisting></para>
|
|
</section>
|
|
|
|
<section id="Local">
|
|
<title>Applications running on the Firewall</title>
|
|
|
|
<para>Experience has shown that in some cases, problems occur with
|
|
applications running on the firewall itself. This is especially true
|
|
when you have specified <emphasis role="bold">routefilter</emphasis> on
|
|
your external interfaces in /etc/shorewall/interfaces (see <link
|
|
linkend="Martians">above</link>). When this happens, it is suggested
|
|
that you have the application use specific local IP addresses rather
|
|
than 0.</para>
|
|
|
|
<para>Examples:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Squid: In <filename>squid.conf</filename>, set <emphasis
|
|
role="bold">tcp_outgoing_address</emphasis> to the IP address of the
|
|
interface that you want Squid to use.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>In OpenVPN, set <emphasis role="bold">local
|
|
</emphasis>(<emphasis role="bold">--local</emphasis> on the command
|
|
line) to the IP address that you want the server to receive
|
|
connections on.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Note that some traffic originating on the firewall doesn't have a
|
|
SOURCE IP address before routing. At least one Shorewall user reports
|
|
that an entry in <filename>/etc/shorewall/route_rules</filename> with
|
|
'lo' in the SOURCE column seems to be the most reliable way to direct
|
|
such traffic to a particular ISP.</para>
|
|
</section>
|
|
|
|
<section id="route_rules">
|
|
<title>/etc/shorewall/route_rules</title>
|
|
|
|
<para>The <filename>/etc/shorewall/route_rules</filename> file was added
|
|
in Shorewall version 3.2.0. The <filename>route_rules</filename> file
|
|
allows assigning certain traffic to a particular provider just as
|
|
entries in the <filename>tcrules</filename> file. The difference between
|
|
the two files is that entries in <filename>route_rules</filename> are
|
|
independent of Netfilter.</para>
|
|
|
|
<warning>
|
|
<para>Prior to Shorewall version 3.4.0, entries in
|
|
<filename>/etc/shorewall/route_rules</filename> permanently alter your
|
|
firewall/gateway's routing; that is, the effect of these changes is
|
|
not reversed by <command>shorewall stop</command> or
|
|
<command>shorewall clear</command>. To restore routing to its original
|
|
state, you may have to restart your network. This can usually be done
|
|
by <command>/etc/init.d/network restart</command> or
|
|
<command>/etc/init.d/networking restart</command>. Check your
|
|
distribution's networking documentation.</para>
|
|
|
|
<para>Please also see the <link linkend="Undo">warning in the section
|
|
above entitled <emphasis>What an Entry in the Providers File
|
|
Does</emphasis></link>.</para>
|
|
</warning>
|
|
|
|
<section id="Routing_rules">
|
|
<title>Routing Rules</title>
|
|
|
|
<para>Routing rules are maintained by the Linux kernel and can be
|
|
displayed using the <command>ip rule ls</command> command. When
|
|
routing a packet, the rules are processed in turn until the packet is
|
|
successfully routed.</para>
|
|
|
|
<programlisting>gateway:~ # <command>ip rule ls</command>
|
|
0: from all lookup local <=== Local (to the firewall) IP addresses
|
|
10001: from all fwmark 0x1 lookup Blarg <=== This and the next rule are generated by the
|
|
10002: from all fwmark 0x2 lookup Comcast 'MARK' values in /etc/shorewall/providers.
|
|
20000: from 206.124.146.176 lookup Blarg <=== This and the next rule are generated unless
|
|
20256: from 24.12.22.33 lookup Comcast 'loose' is specified; based in the output of 'ip addr ls'
|
|
32766: from all lookup main <=== This is the routing table shown by 'iproute -n'
|
|
32767: from all lookup default <=== This table is usually empty
|
|
gateway:~ #</programlisting>
|
|
|
|
<para>In the above example, there are two providers: Blarg and Comcast
|
|
with MARK 1 going to Blarg and mark 2 going to Comcast.</para>
|
|
</section>
|
|
|
|
<section id="route_rules_columns">
|
|
<title>Columns in the route_rules file</title>
|
|
|
|
<para>Columns in the file are:</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term>SOURCE (Optional)</term>
|
|
|
|
<listitem>
|
|
<para>An ip address (network or host) that matches the source IP
|
|
address in a packet. May also be specified as an interface name
|
|
optionally followed by ":" and an address. If the device 'lo' is
|
|
specified, the packet must originate from the firewall
|
|
itself.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>DEST (Optional)</term>
|
|
|
|
<listitem>
|
|
<para>An ip address (network or host) that matches the
|
|
destination IP address in a packet.</para>
|
|
|
|
<para>If you choose to omit either SOURCE or DEST, place "-" in
|
|
that column. Note that you may not omit both SOURCE and
|
|
DEST.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>PROVIDER</term>
|
|
|
|
<listitem>
|
|
<para>The provider to route the traffic through. May be
|
|
expressed either as the provider name or the provider
|
|
number.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>PRIORITY</term>
|
|
|
|
<listitem>
|
|
<para>The rule's priority which determines the order in which
|
|
the rules are processed.</para>
|
|
|
|
<para>1000-1999 Before Shorewall-generated 'MARK' rules</para>
|
|
|
|
<para>11000- 11999 After 'MARK' rules but before
|
|
Shorewall-generated rules for ISP interfaces.</para>
|
|
|
|
<para>26000-26999 After ISP interface rules but before 'default'
|
|
rule.</para>
|
|
|
|
<para>Rules with equal priority are applied in the order in
|
|
which they appear in the file.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
|
|
<para>Example 1: You want all traffic entering the firewall on eth1 to
|
|
be routed through Comcast.</para>
|
|
|
|
<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
|
eth1 - Comcast 1000</programlisting>
|
|
|
|
<para>With this entry, the output of <command>ip rule ls</command>
|
|
would be as follows.</para>
|
|
|
|
<para><programlisting>gateway:~ # <command>ip rule ls</command>
|
|
0: from all lookup local
|
|
1000: from all iif eth1 lookup Comcast
|
|
10001: from all fwmark 0x1 lookup Blarg
|
|
10002: from all fwmark 0x2 lookup Comcast
|
|
20000: from 206.124.146.176 lookup Blarg
|
|
20256: from 24.12.22.33 lookup Comcast
|
|
32766: from all lookup main
|
|
32767: from all lookup default
|
|
gateway:~ #</programlisting>Note that because we used a priority of 1000, the
|
|
test for <filename class="devicefile">eth1</filename> is inserted
|
|
before the fwmark tests.</para>
|
|
|
|
<para>Example 2: You use OpenVPN (routed setup w/tunX) in combination
|
|
with multiple providers. In this case you have to set up a rule to
|
|
ensure that the OpenVPN traffic is routed back through the tunX
|
|
interface(s) rather than through any of the providers. 10.8.0.0/24 is
|
|
the subnet choosen in your OpenVPN configuration (server 10.8.0.0
|
|
255.255.255.0).</para>
|
|
|
|
<programlisting>#SOURCE DEST PROVIDER PRIORITY
|
|
- 10.8.0.0/24 main 1000</programlisting>
|
|
</section>
|
|
</section>
|
|
</section>
|
|
</article> |