shorewall_code/Shorewall-docs2/Shorewall_and_Routing.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<article>
  <!--$Id$-->

  <articleinfo>
    <title>Shorewall and Routing</title>

    <authorgroup>
      <author>
        <firstname>Tom</firstname>

        <surname>Eastep</surname>
      </author>
    </authorgroup>

    <pubdate>2005-06-02</pubdate>

    <copyright>
      <year>2005</year>

      <holder>Thomas M. Eastep</holder>
    </copyright>

    <legalnotice>
      <para>Permission is granted to copy, distribute and/or modify this
      document under the terms of the GNU Free Documentation License, Version
      1.2 or any later version published by the Free Software Foundation; with
      no Invariant Sections, with no Front-Cover, and with no Back-Cover
      Texts. A copy of the license is included in the section entitled
      <quote><ulink url="GnuCopyright.htm">GNU Free Documentation
      License</ulink></quote>.</para>
    </legalnotice>
  </articleinfo>

  <section>
    <title>Routing vs. Firewalling.</title>

    <para>One of the most misunderstood aspects of Shorewall is its
    releationship with routing. This article attempts to clear some of the fog
    that surrounds this issue.</para>

    <para>As a general principle:</para>

    <orderedlist>
      <listitem>
        <para>Routing determines where packets are to be sent.</para>
      </listitem>

      <listitem>
        <para>Once routing determines where the packet is to go, the firewall
        (Shorewall) determines if the packet is allowed to go there.</para>
      </listitem>
    </orderedlist>

    <para>There are ways that Shorewall can affect routing which are described
    in the following sections.</para>
  </section>

  <section>
    <title>Routing and Netfilter</title>

    <para>The following diagram shows the relationship between routing
    decisions and Netfilter.</para>

    <graphic fileref="images/Netfilter.png" />

    <para>The light blue boxes indicate where routing decisions are made. Upon
    exit from one of these boxes, if the packet is being sent to another
    system then the interface and the next hop have been uniquely
    determined.</para>

    <para>The green boxes show where Netfilter processing takes place (as
    directed by Shorewall). You will notice that there are two different paths
    through this maze, depending on where the packet originates. We will look
    at each of these separately.</para>

    <section>
      <title>Packets Entering the Firewall from Outside</title>

      <para>When a packet arrives from outside, it first undergoes Netfilter
      PREROUTING processing. In Shorewall terms:</para>

      <orderedlist>
        <listitem>
          <para>Packets may be marked using entries in the <ulink
          url="???">/etc/shorewall/tcrules</ulink> file. Entries in that file
          containing ":P" in the mark column are applied here as are rules
          that default to the MARK_IN_FORWARD_CHAIN=No setting in
          <filename>/etc/shorewall/shorewall.conf</filename>. These marks may
          be used to specify that the packet should be routed using an
          <firstterm>alternate routing table</firstterm>; see the <ulink
          url="Shorewall_Squid_Usage.html">Shorewall Squid
          documentation</ulink> for examples.</para>

          <caution>
            <para>Marking packets then using the <emphasis>fwmark</emphasis>
            selector in your "<emphasis role="bold">ip rule add</emphasis>"
            commands should NOT be your first choice. In most cases, you can
            use the <emphasis>from</emphasis> or <emphasis>dev</emphasis>
            selector instead.</para>
          </caution>
        </listitem>

        <listitem>
          <para>The destination IP address may be rewritten as a consequence
          of:</para>

          <itemizedlist>
            <listitem>
              <para>DNAT[-] rules.</para>
            </listitem>

            <listitem>
              <para>REDIRECT[-] rules.</para>
            </listitem>

            <listitem>
              <para>Entries in <filename>/etc/shorewall/nat</filename>.</para>
            </listitem>
          </itemizedlist>
        </listitem>
      </orderedlist>

      <para>So the only influence that Shorewall has over where these packets
      go is via NAT or by marking them so that they may be routed using an
      alternate routing table.</para>
    </section>

    <section>
      <title>Packets Originating on the Firewall</title>

      <para>Processing of packets that originate on the firewall itself are
      initially routed using the default routing table then passed through the
      OUTPUT chains. Shorewall can influence what happens here:</para>

      <orderedlist>
        <listitem>
          <para>Packets may be marked using entries in the <ulink
          url="???">/etc/shorewall/tcrules</ulink> file (rules with "$FW" in
          the SOURCE column). These marks may be used to specify that the
          packet should be re-routed using an alternate routing table.</para>
        </listitem>

        <listitem>
          <para>The destination IP address may be rewritten as a consequence
          of:</para>

          <itemizedlist>
            <listitem>
              <para>DNAT[-] rules that specify $FW as the SOURCE.</para>
            </listitem>

            <listitem>
              <para>Entries in <filename>/etc/shorewall/nat</filename> that
              have "Yes" in LOCAL column.</para>
            </listitem>
          </itemizedlist>
        </listitem>
      </orderedlist>

      <para>So again in this case, the only influence that Shorewall has over
      the packet destination is NAT or marking.</para>
    </section>
  </section>

  <section>
    <title>Alternate Routing Table Configuration</title>

    <para>The <ulink url="Shorewall_Squid_Usage.html">Shorewall Squid
    documentation</ulink> shows how alternate routing tables can be created
    and used. That documentation shows how you can use logic in
    <filename>/etc/shorewall/init</filename> to create and populate an
    alternate table and to add a routing rule for its use. It is fine to use
    that technique so long as you understand that you are basically just using
    the Shorewall init script (<filename>/etc/init.d/shorewall</filename>) to
    configure your alternate routing table at boot time and that <emphasis
    role="bold">other than as described in the previous section, there is no
    connection between Shorewall and routing when using Shorewall versions
    prior to 2.3.2.</emphasis></para>
  </section>

  <section>
    <title>Routing and Proxy ARP</title>

    <para>There is one instance where Shorewall creates main routing table
    entries. When an entry in <filename>/etc/shorewall/proxyarp</filename>
    contains "No" in the HAVEROUTE column then Shorewall will create a host
    route to the IP address listed in the ADDRESS column through the interface
    named in the INTERFACE column. <emphasis role="bold">This is the only case
    where Shorewall directly manipulates the main routing
    table</emphasis>.</para>

    <para>Example:</para>

    <para><filename>/etc/shorewall/proxyarp</filename>:</para>

    <programlisting>#ADDRESS        INTERFACE       EXTERNAL        HAVEROUTE       PERSISTENT
206.124.146.177 eth1            eth0            No
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE</programlisting>

    <para>The above entry will cause Shorewall to execute the following
    command:</para>

    <programlisting><emphasis role="bold">ip route add 206.124.146.177 dev eth1</emphasis></programlisting>
  </section>

  <section>
    <title>Multiple Internet Connection Support in Shorewall 2.3.2 and
    Later</title>

    <para>Beginning with Shorewall 2.3.2, support is included for multiple
    internet connections.</para>

    <section>
      <title>Overview</title>

      <para>Let's assume that a firewall is connected via two separate
      ethernet interfaces to two different ISPs as in the following
      diagram.</para>

      <graphic fileref="images/TwoISPs.png" />

      <itemizedlist>
        <listitem>
          <para>eth0 connects to ISP1. The IP address of eth0 is
          206.124.146.176 and the ISP's gateway router has IP address
          206.124.146.254.</para>
        </listitem>

        <listitem>
          <para>eth1 connects to ISP 2. The IP address of eth1 is
          130.252.99.27 and the ISP's gateway router has IP address
          130.252.99.254.</para>
        </listitem>
      </itemizedlist>

      <para>Each of these <firstterm>providers</firstterm> is described in an
      entry in the file <filename>/etc/shorewall/providers</filename>.</para>

      <para>Entries in <filename>/etc/shorewall/providers</filename> can
      specify that outgoing connections are to be load-balanced between the
      two ISPs. Entries in <filename>/etc/shorewall/tcrules</filename> can be
      used to direct particular outgoing connections to one ISP or the
      other.</para>

      <para>Connections from the internet are automatically routed back out of
      the correct interface and through the correct ISP gateway. This works
      whether the connection is handled by the firewall itself or if it is
      routed or port-forwarded to a system behind the firewall.</para>

      <para>Shorewall will set up the routing and will update the
      <filename>/etc/iproute2/rt_tables</filename> to include the table names
      and number of the tables that it adds.</para>

      <caution>
        <para>This feature uses <ulink url="traffic_shaping.htm">packet
        marking</ulink> to control the routing. As a consequence, there are
        some restrictions concerning entries in
        <filename>/etc/shorewall/tcrules</filename>:</para>

        <itemizedlist>
          <listitem>
            <para>Packet marking for traffic control purposes may not be done
            in the PREROUTING table for connections involving providers with
            'track' specified (see below).</para>
          </listitem>

          <listitem>
            <para>You may not use the SAVE or RESTORE options.</para>
          </listitem>

          <listitem>
            <para>You may not use connection marking.</para>
          </listitem>
        </itemizedlist>
      </caution>

      <para>Use of this feature requires that your kernel and iptables support
      CONNMARK target and conntrack match support. It does NOT require the
      ROUTE target extension.</para>

      <warning>
        <para>The current version of iptables (1.3.1) is broken with respect
        to CONNMARK and iptables-save/iptables-restore. This means that if you
        configure multiple ISPs, <command>shorewall restore</command> may
        fail. If it does, you may patch your iptables using the patch at
        <ulink
        url="http://shorewall.net/pub/shorewall/contrib/iptables/CONNMARK.diff">http://shorewall.net/pub/shorewall/contrib/iptables/CONNMARK.diff</ulink>.</para>
      </warning>

      <para>The <filename>/etc/shorewall/providers</filename> file can also be
      used in other routing scenarios. See the <ulink
      url="Shorewall_Squid_Usage.html">Squid documentation</ulink> for an
      example.</para>
    </section>

    <section>
      <title>/etc/shorewall/providers File</title>

      <para>Entries in this file have the following columns. As in all
      Shorewall configuration files, enter "-" in a column if you don't want
      to enter any value.</para>

      <glossary>
        <glossdiv>
          <title>/etc/shorewall/providers:</title>

          <glossentry>
            <glossterm>NAME</glossterm>

            <glossdef>
              <para>The provider name. Must begin with a letter and consist of
              letters and digits. The provider name becomes the name of the
              generated routing table for this provider.</para>
            </glossdef>
          </glossentry>

          <glossentry>
            <glossterm>NUMBER</glossterm>

            <glossdef>
              <para>A number between 1 and 252. This becomes the routing table
              number for the generated table for this provider.</para>
            </glossdef>
          </glossentry>

          <glossentry>
            <glossterm>MARK</glossterm>

            <glossdef>
              <para>A mark value used in your /etc/shorewall/tcrules file to
              direct packets to this provider. Shorewall will also mark
              connections that have seen input from this provider with this
              value and will restore the packet mark in the PREROUTING
              CHAIN.</para>
            </glossdef>
          </glossentry>

          <glossentry>
            <glossterm>DUPLICATE</glossterm>

            <glossdef>
              <para>Gives the name or number of a routing table to duplicate.
              May be 'main' or the name or number of a previously declared
              provider. For most applications, you want to specify 'main'
              here.</para>
            </glossdef>
          </glossentry>

          <glossentry>
            <glossterm>INTERFACE</glossterm>

            <glossdef>
              <para>The name of the interface to the provider.</para>
            </glossdef>
          </glossentry>

          <glossentry>
            <glossterm>GATEWAY</glossterm>

            <glossdef>
              <para>The IP address of the provider's Gateway router.</para>

              <para>You can enter <emphasis role="bold">detect</emphasis> here
              and Shorewall will attempt to automatically determine the
              gateway IP address.</para>
            </glossdef>
          </glossentry>

          <glossentry>
            <glossterm>OPTIONS</glossterm>

            <glossdef>
              <para>A comma-separated list from the following:</para>

              <glosslist>
                <glossentry>
                  <glossterm>track</glossterm>

                  <glossdef>
                    <para>If specified, connections FROM this interface are to
                    be tracked so that responses may be routed back out this
                    same interface.</para>

                    <para>You want specify 'track' if internet hosts will be
                    connecting to local servers through this provider.</para>
                  </glossdef>
                </glossentry>

                <glossentry>
                  <glossterm>balance</glossterm>

                  <glossdef>
                    <para>The providers that have 'balance' specified will get
                    outbound traffic load-balanced among them. Balancing will
                    not be perfect, as it is route based, and routes are
                    cached. This means that routes to often-used sites will
                    always be over the same provider.</para>

                    <para>By default, each provider is given the same weight
                    (1) . Beginning with 2.4.0-RC3, you can change the weight
                    of a given provider by following
                    <emphasis>balance</emphasis> with "=" and the desired
                    weight (e.g., balance=2). The weights reflect the relative
                    bandwidth of the providers connections and should be small
                    numbers since the kernel actually creates additional
                    default routes for each weight increment.</para>
                  </glossdef>
                </glossentry>
              </glosslist>
            </glossdef>
          </glossentry>
        </glossdiv>
      </glossary>
    </section>

    <section>
      <title>What an entry in the Providers File Does</title>

      <para>Adding another entry in the providers file simply creates an
      alternate routing table for you. In addition:</para>

      <orderedlist>
        <listitem>
          <para>An ip rule is generated for each IP address on the INTERFACE
          that routes traffic from that address through the associated routing
          table.</para>
        </listitem>

        <listitem>
          <para>If you specify <emphasis role="bold">track</emphasis>, then
          connections which have had at least one packet arrive on the
          interface listed in the INTERFACE column have their connection mark
          set to the value in the MARK column. In the PREROUTING chain,
          packets with that connmark have their packet mark set to that value;
          packets so marked then bypass any prerouting rules that you create
          in <filename>/etc/shorewall/tcrules</filename>. This ensures that
          packets associated with connections from outside are always routed
          out of the correct interface.</para>
        </listitem>

        <listitem>
          <para>If you specify <emphasis role="bold">balance</emphasis>, then
          Shorewall will replace the 'default' route with weight 100 in the
          'main' routing table with a load-balancing route among those
          gateways where <emphasis role="bold">balance</emphasis> was
          specified. So if you configure default routes, be sure that their
          weight is less than 100 or the route added by Shorewall will not be
          used.</para>
        </listitem>
      </orderedlist>

      <para>That's <emphasis role="bold">all</emphasis> that these entries do.
      You still have to follow the principle stated at the top of this
      article:</para>

      <orderedlist>
        <listitem>
          <para>Routing determines where packets are to be sent.</para>
        </listitem>

        <listitem>
          <para>Once routing determines where the packet is to go, the
          firewall (Shorewall) determines if the packet is allowed to go
          there.</para>
        </listitem>
      </orderedlist>

      <para>The bottom line is that if you want traffic to go out through a
      particular provider then you <emphasis>must </emphasis>mark that traffic
      with the provider's MARK value in
      <filename>/etc/shorewall/tcrules</filename> and you must do that marking
      in the PREROUTING chain.</para>

      <warning>
        <para>Entries in <filename>/etc/shorewall/providers</filename>
        permanently alter your firewall/gateway's routing; that is, the effect
        of these changes is not reversed by <command>shorewall stop</command>
        or <command>shorewall clear</command>. To restore routing to its
        original state, you will have to restart your network. This can
        usually be done by <command>/etc/init.d/network restart</command> or
        <command>/etc/init.d/networking restart</command>. Check your
        distribution's networking documentation.</para>

        <para>You can mitigate the effect of the Shorewall-generated changes
        to your routing table by specifying a <emphasis>metric</emphasis> for
        each default route that you configure. Shorewall will generate a
        load-balancing default route (assuming that <emphasis
        role="bold">balance</emphasis> has been specified for some of the
        providers) that does not include a metric and that will therefore not
        replace any existing route that has a non-zero metric.</para>
      </warning>
    </section>

    <section>
      <title>Example</title>

      <para>The configuration in the figure at the top of this section would
      be specified in <filename>/etc/shorewall/providers</filename> as
      follows:</para>

      <programlisting>#NAME   NUMBER  MARK    DUPLICATE       INTERFACE       GATEWAY         OPTIONS
ISP1    1       1       main            eth0            206.124.146.254 track,balance
ISP2    2       2       main            eth1            130.252.99.254  track,balance</programlisting>

      <para>Other configuration files go something like this:</para>

      <para><filename>/etc/shorewall/interfaces</filename>:</para>

      <programlisting>#ZONE    INTERFACE    BROADCAST       OPTIONS
net      eth0         detect          …          
net      eth1         detect          …</programlisting>

      <para><filename>/etc/shorewall/policy</filename>:</para>

      <programlisting>#SOURCE    DESTINATION    POLICY     LIMIT:BURST
net        net            DROP</programlisting>

      <para>If you have masqueraded hosts, be sure to update
      <filename>/etc/shorewall/masq</filename> to masquerade to both ISPs. For
      example, if you masquerade all hosts connected to <filename
      class="devicefile">eth2</filename> then:</para>

      <programlisting>#INTERFACE       SUBNET            ADDRESS
eth0             eth2              206.124.146.176
eth1             eth2              130.252.99.27</programlisting>

      <para>Now suppose that you want to route all outgoing SMTP traffic from
      your local network through ISP 2. You would make this entry in <ulink
      url="traffic_shaping.htm">/etc/shorewall/tcrules</ulink> (and you would
      set TC_ENABLED=Yes in <ulink
      url="???">/etc/shorewall/shorewall.conf</ulink>).</para>

      <programlisting>#MARK           SOURCE          DEST            PROTO   PORT(S) CLIENT  USER    TEST
#                                                               PORT(S)
2:P             &lt;local network&gt; 0.0.0.0/0       tcp     25</programlisting>
    </section>
  </section>

  <section id="RouteTarget">
    <title>Experimental Routing with Shorewall 2.3.2 and Later</title>

    <para>Beginning with Shorewall 2.3.2, Shorewall is integrated with the
    ROUTE target extension available from Netfilter Patch-O-Matic-NG (<ulink
    url="http://www.netfilter.org">http://www.netfilter.org</ulink>).</para>

    <warning>
      <para>As of this writing, I know of no distribution that is shipping a
      kernel or iptables with the ROUTE target patch included. This means that
      you must patch and build your own kernel and iptables in order to be
      able to use the feature described in this section. <emphasis
      role="bold">This code remains experimental</emphasis> since there is no
      intent by the Netfilter team to ever submit the ROUTE target patch for
      inclusion in the official kernels from kernel.org.</para>
    </warning>

    <para>See <ulink url="FAQ.htm#faq42">Shorewall FAQ 42</ulink> for
    information about determining if your kernel and iptables have this
    support enabled. You must be running Shorewall 2.3.2 or later to make this
    determination.</para>

    <para>Routing with Shorewall is specified through entries in
    <filename>/etc/shorewall/routes</filename>. Note that entries in the
    <filename>/etc/shorewall/routes</filename> file override the routing
    specified in your routing tables. These rules generate Netfilter rules in
    the mangle tables FORWARD chain or OUTPUT chain depending whether the
    packets are being routed through the firewall or originate on the firewall
    itself (see the flow diagram at the top of this article).</para>

    <para>Columns in this file are as follows:</para>

    <glosslist>
      <glossentry>
        <glossterm>SOURCE</glossterm>

        <glossdef>
          <para>Source of the packet. May be any of the following:</para>

          <itemizedlist>
            <listitem>
              <para>A host or network address</para>
            </listitem>

            <listitem>
              <para>A network interface name.</para>
            </listitem>

            <listitem>
              <para>The name of an ipset prefaced with "+"</para>
            </listitem>

            <listitem>
              <para>$FW (for packets originating on the firewall)</para>
            </listitem>

            <listitem>
              <para>A MAC address in Shorewall format</para>
            </listitem>

            <listitem>
              <para>A range of IP addresses (assuming that your kernel and
              iptables support range match)</para>
            </listitem>

            <listitem>
              <para>A network interface name followed by ":" and an address or
              address range.</para>
            </listitem>
          </itemizedlist>
        </glossdef>
      </glossentry>

      <glossentry>
        <glossterm>DEST</glossterm>

        <glossdef>
          <para>Destination of the packet. May be any of the following:</para>

          <itemizedlist>
            <listitem>
              <para>A host or network address</para>
            </listitem>

            <listitem>
              <para>A network interface name (determined from routing
              table(s))</para>
            </listitem>

            <listitem>
              <para>The name of an ipset prefaced with "+"</para>
            </listitem>

            <listitem>
              <para>A network interface name followed by ":" and an address or
              address range.</para>
            </listitem>
          </itemizedlist>
        </glossdef>
      </glossentry>

      <glossentry>
        <glossterm>PROTO</glossterm>

        <glossdef>
          <para>Protocol - Must be a protocol listed in /etc/protocols, a
          number or "ipp2p", a number, or "all". "ipp2p" require ipp2p match
          support in your kernel and iptables.</para>
        </glossdef>
      </glossentry>

      <glossentry>
        <glossterm>PORT(S)</glossterm>

        <glossdef>
          <para>Destination Ports. A comma-separated list of Port names (from
          /etc/services), port numbers or port ranges; if the protocol is
          "icmp", this column is interpreted as the destination
          icmp-type(s).</para>

          <para>If the protocol is ipp2p, this column is interpreted as an
          ipp2p option without the leading "--" (example "bit" for
          bit-torrent). If no PORT is given, "ipp2p" is assumed.</para>

          <para>This column is ignored if PROTOCOL = all but must be entered
          if any of the following field is supplied. In that case, it is
          suggested that this field contain "-"</para>
        </glossdef>
      </glossentry>

      <glossentry>
        <glossterm>SOURCE PORT(S)</glossterm>

        <glossdef>
          <para>Optional) Source port(s). If omitted, any source port is
          acceptable. Specified as a comma-separated list of port names, port
          numbers or port ranges.</para>
        </glossdef>
      </glossentry>

      <glossentry>
        <glossterm>TEST</glossterm>

        <glossdef>
          <para>Defines a test on the existing packet or connection mark. The
          rule will match only if the test returns true. Tests have the
          format</para>

          <blockquote>
            <para>[!]&lt;value&gt;[/&lt;mask&gt;][:C]</para>
          </blockquote>

          <para>where:</para>

          <glosslist>
            <glossentry>
              <glossterm>!</glossterm>

              <glossdef>
                <para>Inverts the test (not equal)</para>
              </glossdef>
            </glossentry>

            <glossentry>
              <glossterm>&lt;value&gt;</glossterm>

              <glossdef>
                <para>Value of the packet or connection mark.</para>
              </glossdef>
            </glossentry>

            <glossentry>
              <glossterm>&lt;mask&gt;</glossterm>

              <glossdef>
                <para>A mask to be applied to the mark before testing</para>
              </glossdef>
            </glossentry>

            <glossentry>
              <glossterm>:C</glossterm>

              <glossdef>
                <para>Designates a connection mark. If omitted, the packet
                mark's value is tested</para>
              </glossdef>
            </glossentry>
          </glosslist>
        </glossdef>
      </glossentry>

      <glossentry>
        <glossterm>INTERFACE</glossterm>

        <glossdef>
          <para>The interface that the packet is to be routed out of. If you
          do not specify this field then you must place "-" in this column and
          enter an IP address in the GATEWAY column.</para>
        </glossdef>
      </glossentry>

      <glossentry>
        <glossterm>GATEWAY</glossterm>

        <glossdef>
          <para>The gateway that the packet is to be forwarded through.</para>
        </glossdef>
      </glossentry>
    </glosslist>

    <para>The idea here is that traffic that matches the SOURCE, DEST, PROTO,
    PORT(S), SOURCE PORT(S) and TEST columns is routed out of the INTERFACE
    through the optional GATEWAY.</para>

    <blockquote>
      <para>Example:</para>

      <para>Your local interface is eth1 and your DMZ interface is eth2. You
      want to run Squid as a transparent proxy for HTTP on 192.168.3.22 in
      your DMZ. You would use the following entry in
      /etc/shorewall/routes:</para>

      <programlisting>#SOURCE                 DEST            PROTO   PORT(S) SOURCE  TEST    INTERFACE       GATEWAY
#                                                       PORT(S)
eth1                    0.0.0.0/0       tcp     80      -       -       eth1            192.168.3.22</programlisting>

      <para>This entry specifies that "traffic coming in through eth1 to TCP
      port 80 is to be routed out of eth1 to gateway 192.168.3.22".</para>
    </blockquote>
  </section>
</article>