shorewall_code/Shorewall-docs2/Shorewall_and_Routing.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<article>
  <!--$Id$-->

  <articleinfo>
    <title>Shorewall and Routing</title>

    <authorgroup>
      <author>
        <firstname>Tom</firstname>

        <surname>Eastep</surname>
      </author>
    </authorgroup>

    <pubdate>2005-11-16</pubdate>

    <copyright>
      <year>2005</year>

      <holder>Thomas M. Eastep</holder>
    </copyright>

    <legalnotice>
      <para>Permission is granted to copy, distribute and/or modify this
      document under the terms of the GNU Free Documentation License, Version
      1.2 or any later version published by the Free Software Foundation; with
      no Invariant Sections, with no Front-Cover, and with no Back-Cover
      Texts. A copy of the license is included in the section entitled
      <quote><ulink url="GnuCopyright.htm">GNU Free Documentation
      License</ulink></quote>.</para>
    </legalnotice>
  </articleinfo>

  <section>
    <title>Routing vs. Firewalling.</title>

    <para>One of the most misunderstood aspects of Shorewall is its
    relationship with routing. This article attempts to clear some of the fog
    that surrounds this issue.</para>

    <para>As a general principle:</para>

    <orderedlist>
      <listitem>
        <para>Routing determines where packets are to be sent.</para>
      </listitem>

      <listitem>
        <para>Once routing determines where the packet is to go, the firewall
        (Shorewall) determines if the packet is allowed to go there.</para>
      </listitem>
    </orderedlist>

    <para>There are ways that Shorewall can affect routing which are described
    in the following sections.</para>
  </section>

  <section>
    <title>Routing and Netfilter</title>

    <para>The following diagram shows the relationship between routing
    decisions and Netfilter.</para>

    <graphic fileref="images/Netfilter.png" />

    <para>The light blue boxes indicate where routing decisions are made. Upon
    exit from one of these boxes, if the packet is being sent to another
    system then the interface and the next hop have been uniquely
    determined.</para>

    <para>The green boxes show where Netfilter processing takes place (as
    directed by Shorewall). You will notice that there are two different paths
    through this maze, depending on where the packet originates. We will look
    at each of these separately.</para>

    <section>
      <title>Packets Entering the Firewall from Outside</title>

      <para>When a packet arrives from outside, it first undergoes Netfilter
      PREROUTING processing. In Shorewall terms:</para>

      <orderedlist>
        <listitem>
          <para>Packets may be marked using entries in the <ulink
          url="???">/etc/shorewall/tcrules</ulink> file. Entries in that file
          containing ":P" in the mark column are applied here as are rules
          that default to the MARK_IN_FORWARD_CHAIN=No setting in
          <filename>/etc/shorewall/shorewall.conf</filename>. These marks may
          be used to specify that the packet should be routed using an
          <firstterm>alternate routing table</firstterm>; see the <ulink
          url="Shorewall_Squid_Usage.html">Shorewall Squid
          documentation</ulink> for examples.</para>

          <caution>
            <para>Marking packets then using the <emphasis>fwmark</emphasis>
            selector in your "<emphasis role="bold">ip rule add</emphasis>"
            commands should NOT be your first choice. In most cases, you can
            use the <emphasis>from</emphasis> or <emphasis>dev</emphasis>
            selector instead.</para>
          </caution>
        </listitem>

        <listitem>
          <para>The destination IP address may be rewritten as a consequence
          of:</para>

          <itemizedlist>
            <listitem>
              <para>DNAT[-] rules.</para>
            </listitem>

            <listitem>
              <para>REDIRECT[-] rules.</para>
            </listitem>

            <listitem>
              <para>Entries in <filename>/etc/shorewall/nat</filename>.</para>
            </listitem>
          </itemizedlist>
        </listitem>
      </orderedlist>

      <para>So the only influence that Shorewall has over where these packets
      go is via NAT or by marking them so that they may be routed using an
      alternate routing table.</para>
    </section>

    <section>
      <title>Packets Originating on the Firewall</title>

      <para>Processing of packets that originate on the firewall itself are
      initially routed using the default routing table then passed through the
      OUTPUT chains. Shorewall can influence what happens here:</para>

      <orderedlist>
        <listitem>
          <para>Packets may be marked using entries in the <ulink
          url="???">/etc/shorewall/tcrules</ulink> file (rules with "$FW" in
          the SOURCE column). These marks may be used to specify that the
          packet should be re-routed using an alternate routing table.</para>
        </listitem>

        <listitem>
          <para>The destination IP address may be rewritten as a consequence
          of:</para>

          <itemizedlist>
            <listitem>
              <para>DNAT[-] rules that specify $FW as the SOURCE.</para>
            </listitem>

            <listitem>
              <para>Entries in <filename>/etc/shorewall/nat</filename> that
              have "Yes" in LOCAL column.</para>
            </listitem>
          </itemizedlist>
        </listitem>
      </orderedlist>

      <para>So again in this case, the only influence that Shorewall has over
      the packet destination is NAT or marking.</para>
    </section>
  </section>

  <section>
    <title>Alternate Routing Table Configuration</title>

    <para>The <ulink url="Shorewall_Squid_Usage.html">Shorewall Squid
    documentation</ulink> shows how alternate routing tables can be created
    and used. That documentation shows how you can use logic in
    <filename>/etc/shorewall/init</filename> to create and populate an
    alternate table and to add a routing rule for its use. It is fine to use
    that technique so long as you understand that you are basically just using
    the Shorewall init script (<filename>/etc/init.d/shorewall</filename>) to
    configure your alternate routing table at boot time and that <emphasis
    role="bold">other than as described in the previous section, there is no
    connection between Shorewall and routing when using Shorewall versions
    prior to 2.3.2.</emphasis></para>
  </section>

  <section>
    <title>Routing and Proxy ARP</title>

    <para>There is one instance where Shorewall creates main routing table
    entries. When an entry in <filename>/etc/shorewall/proxyarp</filename>
    contains "No" in the HAVEROUTE column then Shorewall will create a host
    route to the IP address listed in the ADDRESS column through the interface
    named in the INTERFACE column. <emphasis role="bold">This is the only case
    where Shorewall directly manipulates the main routing
    table</emphasis>.</para>

    <para>Example:</para>

    <para><filename>/etc/shorewall/proxyarp</filename>:</para>

    <programlisting>#ADDRESS        INTERFACE       EXTERNAL        HAVEROUTE       PERSISTENT
206.124.146.177 eth1            eth0            No
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>

    <para>The above entry will cause Shorewall to execute the following
    command:</para>

    <programlisting><emphasis role="bold">ip route add 206.124.146.177 dev eth1</emphasis></programlisting>
  </section>

  <section>
    <title>Multiple Internet Connection Support in Shorewall 2.4.2 and
    Later</title>

    <para>Beginning with Shorewall 2.3.2, support is included for multiple
    internet connections. If you wish to use this feature, we recommend
    strongly that you upgrade to version 2.4.2 or later. This section assumes
    that you have so upgraded.</para>

    <section>
      <title>Overview</title>

      <para>Let's assume that a firewall is connected via two separate
      ethernet interfaces to two different ISPs as in the following
      diagram.</para>

      <graphic fileref="images/TwoISPs.png" />

      <itemizedlist>
        <listitem>
          <para>eth0 connects to ISP1. The IP address of eth0 is
          206.124.146.176 and the ISP's gateway router has IP address
          206.124.146.254.</para>
        </listitem>

        <listitem>
          <para>eth1 connects to ISP 2. The IP address of eth1 is
          130.252.99.27 and the ISP's gateway router has IP address
          130.252.99.254.</para>
        </listitem>
      </itemizedlist>

      <para>Each of these <firstterm>providers</firstterm> is described in an
      entry in the file <filename>/etc/shorewall/providers</filename>.</para>

      <para>Entries in <filename>/etc/shorewall/providers</filename> can
      specify that outgoing connections are to be load-balanced between the
      two ISPs. Entries in <filename>/etc/shorewall/tcrules</filename> can be
      used to direct particular outgoing connections to one ISP or the other.
      Use of <filename>/etc/shorewall/tcrules</filename> is not required for
      <filename>/etc/shorewall/providers</filename> to work, but you must
      select a unique MARK value for each provider so Shorewall can set up the
      correct marking rules for you.</para>

      <para>When using <filename>/etc/shorewall/providers</filename>,
      connections from the internet are automatically routed back out of the
      correct interface and through the correct ISP gateway. This works
      whether the connection is handled by the firewall itself or if it is
      routed or port-forwarded to a system behind the firewall.</para>

      <para>Shorewall will set up the routing and will update the
      <filename>/etc/iproute2/rt_tables</filename> to include the table names
      and number of the tables that it adds.</para>

      <caution>
        <para>This feature uses <ulink url="traffic_shaping.htm">packet
        marking</ulink> to control the routing. As a consequence, there are
        some restrictions concerning entries in
        <filename>/etc/shorewall/tcrules</filename>:</para>

        <itemizedlist>
          <listitem>
            <para>Packet marking for traffic control purposes may not be done
            in the PREROUTING table for connections involving providers with
            'track' specified (see below).</para>
          </listitem>

          <listitem>
            <para>You may not use the SAVE or RESTORE options.</para>
          </listitem>

          <listitem>
            <para>You may not use connection marking.</para>
          </listitem>
        </itemizedlist>
      </caution>

      <para>Use of this feature requires that your kernel and iptables support
      CONNMARK target and conntrack match support. It does NOT require the
      ROUTE target extension.</para>

      <warning>
        <para>The current version of iptables (1.3.1) is broken with respect
        to CONNMARK and iptables-save/iptables-restore. This means that if you
        configure multiple ISPs, <command>shorewall restore</command> may
        fail. If it does, you may patch your iptables using the patch at
        <ulink
        url="http://shorewall.net/pub/shorewall/contrib/iptables/CONNMARK.diff">http://shorewall.net/pub/shorewall/contrib/iptables/CONNMARK.diff</ulink>.</para>
      </warning>

      <para>The <filename>/etc/shorewall/providers</filename> file can also be
      used in other routing scenarios. See the <ulink
      url="Shorewall_Squid_Usage.html">Squid documentation</ulink> for an
      example.</para>
    </section>

    <section>
      <title>/etc/shorewall/providers File</title>

      <para>Entries in this file have the following columns. As in all
      Shorewall configuration files, enter "-" in a column if you don't want
      to enter any value.</para>

      <variablelist>
        <varlistentry>
          <term>NAME</term>

          <listitem>
            <para>The provider name. Must begin with a letter and consist of
            letters and digits. The provider name becomes the name of the
            generated routing table for this provider.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>NUMBER</term>

          <listitem>
            <para>A number between 1 and 252. This becomes the routing table
            number for the generated table for this provider.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>MARK</term>

          <listitem>
            <para>A mark value used in your /etc/shorewall/tcrules file to
            direct packets to this provider. Shorewall will also mark
            connections that have seen input from this provider with this
            value and will restore the packet mark in the PREROUTING
            CHAIN.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>DUPLICATE</term>

          <listitem>
            <para>Gives the name or number of a routing table to duplicate.
            May be 'main' or the name or number of a previously declared
            provider. For most applications, you want to specify 'main'
            here.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>INTERFACE</term>

          <listitem>
            <para>The name of the interface to the provider.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>GATEWAY</term>

          <listitem>
            <para>The IP address of the provider's Gateway router.</para>

            <para>You can enter <emphasis role="bold">detect</emphasis> here
            and Shorewall will attempt to automatically determine the gateway
            IP address.</para>

            <para><emphasis role="bold">Hint:</emphasis> <emphasis
            role="bold">"detect"</emphasis> is appropriate for use in cases
            where the interface named in the INTERFACE column is dynamically
            configured via DHCP etc.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>OPTIONS</term>

          <listitem>
            <para>A comma-separated list from the following:</para>

            <variablelist>
              <varlistentry>
                <term>track</term>

                <listitem>
                  <para>If specified, connections FROM this interface are to
                  be tracked so that responses may be routed back out this
                  same interface.</para>

                  <para>You want specify 'track' if internet hosts will be
                  connecting to local servers through this provider. Any time
                  that you specify 'track', you will also want to specify
                  'balance' (see below).</para>
                </listitem>
              </varlistentry>

              <varlistentry>
                <term>balance</term>

                <listitem>
                  <para>The providers that have 'balance' specified will get
                  outbound traffic load-balanced among them. Balancing will
                  not be perfect, as it is route based, and routes are cached.
                  This means that routes to often-used sites will always be
                  over the same provider.</para>

                  <para>By default, each provider is given the same weight (1)
                  . Beginning with 2.4.0-RC3, you can change the weight of a
                  given provider by following <emphasis>balance</emphasis>
                  with "=" and the desired weight (e.g., balance=2). The
                  weights reflect the relative bandwidth of the providers
                  connections and should be small numbers since the kernel
                  actually creates additional default routes for each weight
                  increment.</para>
                </listitem>
              </varlistentry>

              <varlistentry>
                <term>loose</term>

                <listitem>
                  <para>Do not include routing rules that force traffic whose
                  source IP is an address of the INTERFACE to be routed to
                  this provider. Useful for defining providers that are to be
                  used only when the appropriate packet mark is
                  applied.</para>
                </listitem>
              </varlistentry>
            </variablelist>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>COPY</term>

          <listitem>
            <para>When you specify an existing table in the DUPLICATE column,
            Shorewall copies all routes through the interface specified in the
            INTERFACE column plus the interfaces listed in this column. At a
            minumum, you should list all interfaces on your firewall in this
            column except those internet interfaces specified in the INTERFACE
            column of entries in this file.</para>
          </listitem>
        </varlistentry>
      </variablelist>
    </section>

    <section>
      <title>What an entry in the Providers File Does</title>

      <para>Adding another entry in the providers file simply creates an
      alternate routing table for you. In addition:</para>

      <orderedlist>
        <listitem>
          <para>Unless <emphasis role="bold">loose</emphasis> is specified, an
          ip rule is generated for each IP address on the INTERFACE that
          routes traffic from that address through the associated routing
          table.</para>
        </listitem>

        <listitem>
          <para>If you specify <emphasis role="bold">track</emphasis>, then
          connections which have had at least one packet arrive on the
          interface listed in the INTERFACE column have their connection mark
          set to the value in the MARK column. In the PREROUTING chain,
          packets with that connmark have their packet mark set to that value;
          packets so marked then bypass any prerouting rules that you create
          in <filename>/etc/shorewall/tcrules</filename>. This ensures that
          packets associated with connections from outside are always routed
          out of the correct interface.</para>
        </listitem>

        <listitem>
          <para>If you specify <emphasis role="bold">balance</emphasis>, then
          Shorewall will replace the 'default' route with weight 100 in the
          'main' routing table with a load-balancing route among those
          gateways where <emphasis role="bold">balance</emphasis> was
          specified. So if you configure default routes, be sure that their
          weight is less than 100 or the route added by Shorewall will not be
          used.</para>
        </listitem>
      </orderedlist>

      <para>That's <emphasis role="bold">all</emphasis> that these entries do.
      You still have to follow the principle stated at the top of this
      article:</para>

      <orderedlist>
        <listitem>
          <para>Routing determines where packets are to be sent.</para>
        </listitem>

        <listitem>
          <para>Once routing determines where the packet is to go, the
          firewall (Shorewall) determines if the packet is allowed to go
          there.</para>
        </listitem>
      </orderedlist>

      <para>The bottom line is that if you want traffic to go out through a
      particular provider then you <emphasis>must </emphasis>mark that traffic
      with the provider's MARK value in
      <filename>/etc/shorewall/tcrules</filename> and you must do that marking
      in the PREROUTING chain.</para>

      <warning>
        <para>Entries in <filename>/etc/shorewall/providers</filename>
        permanently alter your firewall/gateway's routing; that is, the effect
        of these changes is not reversed by <command>shorewall stop</command>
        or <command>shorewall clear</command>. To restore routing to its
        original state, you will have to restart your network. This can
        usually be done by <command>/etc/init.d/network restart</command> or
        <command>/etc/init.d/networking restart</command>. Check your
        distribution's networking documentation.</para>

        <para>You can mitigate the effect of the Shorewall-generated changes
        to your routing table by specifying a <emphasis>metric</emphasis> for
        each default route that you configure. Shorewall will generate a
        load-balancing default route (assuming that <emphasis
        role="bold">balance</emphasis> has been specified for some of the
        providers) that does not include a metric and that will therefore not
        replace any existing route that has a non-zero metric.</para>
      </warning>
    </section>

    <section>
      <title>Example</title>

      <para>The configuration in the figure at the top of this section would
      be specified in <filename>/etc/shorewall/providers</filename> as
      follows. Assume tht there is a single internal interface, <filename
      class="devicefile">eth2</filename>.</para>

      <programlisting>#NAME   NUMBER  MARK    DUPLICATE       INTERFACE       GATEWAY         OPTIONS          COPY
ISP1    1       1       main            eth0            206.124.146.254 track,balance    eth2
ISP2    2       2       main            eth1            130.252.99.254  track,balance    eth2</programlisting>

      <para>Other configuration files go something like this:</para>

      <para><filename>/etc/shorewall/interfaces</filename>:</para>

      <programlisting>#ZONE    INTERFACE    BROADCAST       OPTIONS
net      eth0         detect          …          
net      eth1         detect          …</programlisting>

      <para><filename>/etc/shorewall/policy</filename>:</para>

      <programlisting>#SOURCE    DESTINATION    POLICY     LIMIT:BURST
net        net            DROP</programlisting>

      <para>If you have masqueraded hosts, be sure to update
      <filename>/etc/shorewall/masq</filename> to masquerade to both ISPs. For
      example, if you masquerade all hosts connected to <filename
      class="devicefile">eth2</filename> then:</para>

      <programlisting>#INTERFACE       SUBNET            ADDRESS
eth0             eth2              206.124.146.176
eth1             eth2              130.252.99.27</programlisting>

      <warning>
        <para>Entries in <filename>/etc/shorewall/masq</filename> have no
        effect on which ISP a particular connection will be sent through. That
        is rather the purpuse of entries in
        <filename>/etc/shorewall/tcrules</filename>.</para>
      </warning>

      <para>Now suppose that you want to route all outgoing SMTP traffic from
      your local network through ISP 2. You would make this entry in <ulink
      url="traffic_shaping.htm">/etc/shorewall/tcrules</ulink> (and you would
      set TC_ENABLED=Yes in <ulink
      url="???">/etc/shorewall/shorewall.conf</ulink>).</para>

      <programlisting>#MARK           SOURCE          DEST            PROTO   PORT(S) CLIENT  USER    TEST
#                                                               PORT(S)
2:P             &lt;local network&gt; 0.0.0.0/0       tcp     25</programlisting>
    </section>
  </section>
</article>