shorewall_code/docs/FTP.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<article id="FTP">
  <!--$Id$-->

  <articleinfo>
    <title>Shorewall and FTP</title>

    <authorgroup>
      <author>
        <firstname>Tom</firstname>

        <surname>Eastep</surname>
      </author>
    </authorgroup>

    <pubdate><?dbtimestamp format="Y/m/d"?></pubdate>

    <copyright>
      <year>2003</year>

      <year>2004</year>

      <year>2005</year>

      <year>2006</year>

      <holder>Thomas M. Eastep</holder>
    </copyright>

    <legalnotice>
      <para>Permission is granted to copy, distribute and/or modify this
      document under the terms of the GNU Free Documentation License, Version
      1.2 or any later version published by the Free Software Foundation; with
      no Invariant Sections, with no Front-Cover, and with no Back-Cover
      Texts. A copy of the license is included in the section entitled
      <quote><ulink url="GnuCopyright.htm">GNU Free Documentation
      License</ulink></quote>.</para>
    </legalnotice>
  </articleinfo>

  <caution>
    <para><emphasis role="bold">This article applies to Shorewall 4.0 and
    later. If you are running a version of Shorewall earlier than Shorewall
    4.0.0 then please see the documentation for that
    release.</emphasis></para>
  </caution>

  <section id="Protocol">
    <title>FTP Protocol</title>

    <para>FTP transfers involve two TCP connections. The first <emphasis
    role="bold">control</emphasis> connection goes from the FTP client to port
    21 on the FTP server. This connection is used for logon and to send
    commands and responses between the endpoints. Data transfers (including
    the output of <quote>ls</quote> and <quote>dir</quote> commands) requires
    a second data connection. The <emphasis role="bold">data</emphasis>
    connection is dependent on the <emphasis role="bold">mode</emphasis> that
    the client is operating in:</para>

    <variablelist>
      <varlistentry>
        <term>Passive Mode</term>

        <listitem>
          <para>(often the default for web browsers) -- The client issues a
          PASV command. Upon receipt of this command, the server listens on a
          dynamically-allocated port then sends a PASV reply to the client.
          The PASV reply gives the IP address and port number that the server
          is listening on. The client then opens a second connection to that
          IP address and port number.</para>
        </listitem>
      </varlistentry>

      <varlistentry>
        <term>Active Mode</term>

        <listitem>
          <para>(often the default for line-mode clients) -- The client
          listens on a dynamically-allocated port then sends a PORT command to
          the server. The PORT command gives the IP address and port number
          that the client is listening on. The server then opens a connection
          to that IP address and port number; the <emphasis role="bold">source
          port</emphasis> for this connection is 20 (ftp-data in
          /etc/services).</para>
        </listitem>
      </varlistentry>
    </variablelist>

    <para>You can see these commands in action using your linux ftp
    command-line client in debugging mode. Note that my ftp client defaults to
    passive mode and that I can toggle between passive and active mode by
    issuing a <quote>passive</quote> command:</para>

    <programlisting>[teastep@wookie Shorewall]$ <emphasis role="bold">ftp ftp1.shorewall.net</emphasis>
Connected to lists.shorewall.net.
220-=(&lt;*&gt;)=-.:. (( Welcome to PureFTPd 1.0.12 )) .:.-=(&lt;*&gt;)=-
220-You are user number 1 of 50 allowed.
220-Local time is now 10:21 and the load is 0.14. Server port: 21.
220 You will be disconnected after 15 minutes of inactivity.
500 Security extensions not implemented
500 Security extensions not implemented
KERBEROS_V4 rejected as an authentication type
Name (ftp1.shorewall.net:teastep): <command>ftp</command>
331-Welcome to ftp.shorewall.net
331-
331 Any password will work
Password:
230 Any password will work
Remote system type is UNIX.
Using binary mode to transfer files.
ftp&gt; <emphasis role="bold">debug</emphasis>
Debugging on (debug=1).
ftp&gt; <emphasis role="bold">ls</emphasis>
---&gt; <emphasis>PASV</emphasis>
<emphasis>227 Entering Passive Mode (192,168,1,193,195,210)</emphasis>
---&gt; LIST
150 Accepted data connection
drwxr-xr-x    5 0        0            4096 Nov  9  2002 archives
drwxr-xr-x    2 0        0            4096 Feb 12  2002 etc
drwxr-sr-x    6 0        50           4096 Feb 19 15:24 pub
226-Options: -l
226 3 matches total
ftp&gt; <emphasis role="bold">passive</emphasis>
Passive mode off.
ftp&gt; <emphasis role="bold">ls</emphasis>
<emphasis>---&gt; PORT 192,168,1,3,142,58</emphasis>
200 PORT command successful
---&gt; LIST
150 Connecting to port 36410
drwxr-xr-x    5 0        0            4096 Nov  9  2002 archives
drwxr-xr-x    2 0        0            4096 Feb 12  2002 etc
drwxr-sr-x    6 0        50           4096 Feb 19 15:24 pub
226-Options: -l
226 3 matches total
ftp&gt;</programlisting>

    <para>Things to notice:</para>

    <orderedlist>
      <listitem>
        <para>The commands that I issued are <emphasis role="bold">strongly
        emphasized</emphasis>.</para>
      </listitem>

      <listitem>
        <para>Commands sent by the client to the server are preceded by
        ---&gt;</para>
      </listitem>

      <listitem>
        <para>Command responses from the server over the control connection
        are numbered.</para>
      </listitem>

      <listitem>
        <para>FTP uses a comma as a separator between the bytes of the IP
        address.</para>
      </listitem>

      <listitem>
        <para>When sending a port number, FTP sends the MSB then the LSB and
        separates the two bytes by a comma. As shown in the PORT command, port
        142,58 translates to 142*256+58 = 36410.</para>
      </listitem>
    </orderedlist>
  </section>

  <section id="Conntrack">
    <title>Linux FTP connection-tracking</title>

    <para>Given the normal loc-&gt;net policy of ACCEPT, passive mode access
    from local clients to remote servers will always work but active mode
    requires the firewall to dynamically open a <quote>hole</quote> for the
    server's connection back to the client. Similarly, if you are running an
    FTP server in your local zone then active mode should always work but
    passive mode requires the firewall to dynamically open a
    <quote>hole</quote> for the client's second connection to the server. This
    is the role of FTP connection-tracking support in the Linux kernel.</para>

    <para>Where any form of NAT (SNAT, DNAT, Masquerading) on your firewall is
    involved, the PORT commands and PASV responses may also need to be
    modified by the firewall. This is the job of the FTP nat support kernel
    function.</para>

    <para>Including FTP connection-tracking and NAT support normally means
    that the modules <quote>nf_conntrack_ftp</quote> and
    <quote>nf_nat_ftp</quote> need to be loaded. Shorewall automatically loads
    these <quote>helper</quote> modules from
    /lib/modules/&lt;<emphasis>kernel-version</emphasis>&gt;/kernel/net/netfilter/
    and you can determine if they are loaded using the <quote>lsmod</quote>
    command. The &lt;<emphasis>kernel-version</emphasis>&gt; may be obtained
    by typing</para>

    <programlisting><command>uname -r</command></programlisting>

    <important>
      <para>Note: If you are running kernel 2.6.19 or earlier, then the module
      names are <emphasis role="bold">ip_nat_ftp</emphasis> and <emphasis
      role="bold">ip_conntrack_ftp</emphasis> and they are normally loaded
      from
      /lib/modules/&lt;<emphasis>kernel-version</emphasis>&gt;/kernel/net/ipv4/netfilter/.</para>
    </important>

    <important>
      <para>Because the ftp helper modules must read and modify commands being
      sent over the command channel, they won't work when the command channel
      is encrypted through use of TLS/SSL.</para>
    </important>

    <example id="Example1">
      <title>Example (Kernel 3.2.20)</title>

      <programlisting>[root@lists etc]# lsmod
Module                  Size  Used by    Not tainted
iptable_filter          3072  1 
iptable_mangle          2816  0 
iptable_nat             7684  0 
iptable_raw             2048  0 
ip_tables              12232  4 iptable_raw,iptable_mangle,iptable_nat,iptable_filter
ipt_addrtype            1920  0 
ipt_ah                  2048  0 
ipt_CLUSTERIP           8708  0 
ipt_ecn                 2304  0 
ipt_ECN                 3072  0 
ipt_iprange             1920  0 
ipt_LOG                 6528  0 
ipt_MASQUERADE          3456  0 
ipt_NETMAP              2048  0 
ipt_owner               2048  0 
ipt_recent              9496  0 
ipt_REDIRECT            2048  0 
ipt_REJECT              4608  0 
ipt_SAME                2432  0 
ipt_TCPMSS              4096  0 
ipt_tos                 1664  0 
ipt_TOS                 2304  0 
ipt_ttl                 1920  0 
ipt_TTL                 2432  0 
ipt_ULOG                8068  0 
nf_conntrack           59864  28 ipt_MASQUERADE,ipt_CLUSTERIP,nf_nat_tftp,nf_nat_snmp_basic,nf_nat_sip,nf_nat_pptp,nf_nat_irc,nf_nat_h323,nf_nat_ftp,nf_nat_amanda,nf_conntrack_ama
nda,nf_conntrack_tftp,nf_conntrack_sip,nf_conntrack_proto_sctp,nf_conntrack_pptp,nf_conntrack_proto_gre,nf_conntrack_netlink,nf_conntrack_netbios_ns,nf_conntrack_irc,nf_conntrack_
h323,nf_conntrack_ftp,xt_helper,xt_state,xt_connmark,xt_conntrack,iptable_nat,nf_nat,nf_conntrack_ipv4
nf_conntrack_amanda     5248  1 nf_nat_amanda
<emphasis role="bold">nf_conntrack_ftp</emphasis>        9728  1 nf_nat_ftp
nf_conntrack_h323      50396  1 nf_nat_h323
nf_conntrack_ipv4      17932  2 iptable_nat
nf_conntrack_irc        7064  1 nf_nat_irc
nf_conntrack_netbios_ns     3072  0 
nf_conntrack_netlink    26240  0 
nf_conntrack_pptp       6912  1 nf_nat_pptp
nf_conntrack_proto_gre     5632  1 nf_conntrack_pptp
nf_conntrack_proto_sctp     8328  0 
nf_conntrack_sip        9748  1 nf_nat_sip
nf_conntrack_tftp       5780  1 nf_nat_tftp
nf_nat                 17964  14 ipt_SAME,ipt_REDIRECT,ipt_NETMAP,ipt_MASQUERADE,nf_nat_tftp,nf_nat_sip,nf_nat_pptp,nf_nat_proto_gre,nf_nat_irc,nf_nat_h323,nf_nat_ftp,nf_nat_amand
a,nf_conntrack_netlink,iptable_nat
nf_nat_amanda           2432  0 
<emphasis role="bold">nf_nat_ftp</emphasis>              3584  0 
nf_nat_h323             7808  0 
nf_nat_irc              2816  0 
nf_nat_pptp             3840  0 
nf_nat_proto_gre        3204  1 nf_nat_pptp
nf_nat_sip              4608  0 
nf_nat_snmp_basic      10372  0 
nf_nat_tftp             1920  0 
xt_CLASSIFY             1920  0 
xt_comment              1920  0 
xt_connmark             2432  0 
xt_conntrack            2944  0 
xt_dccp                 3588  0 
xt_hashlimit           10252  0 
xt_helper               2688  0 
xt_length               1920  0 
xt_limit                2688  0 
xt_mac                  1920  0 
xt_mark                 1920  0 
xt_MARK                 2304  0 
xt_multiport            3328  1 
xt_NFLOG                2176  0 
xt_NFQUEUE              2048  0 
xt_physdev              2704  2 
xt_pkttype              1920  0 
xt_policy               3840  0 
xt_state                2560  0 
xt_tcpmss               2304  0 
xt_tcpudp               3328  0 
[root@lists etc]#</programlisting>
    </example>

    <para>If you want Shorewall to load these modules from an alternate
    directory, you need to set the MODULESDIR variable in
    /etc/shorewall/shorewall.conf to point to that directory.</para>
  </section>

  <section>
    <title>FTP with Kernel 3.5 and Later</title>

    <para>Because of the potential for attackers to subvert Netfilter helpers
    like the one for FTP, the Netfilter team are in the process of eliminating
    the automatic association of helpers to connections. In the 3.5 kernel, it
    is possible to disable this automatic association, and the team have
    announced that automatic association will eventually be eliminated. While
    it is certainly more secure to add explicit rules that create these
    associations, for Shorewall to require users to add those rules would
    present a gross inconvenience during a Shorewall upgrade. To make
    Shorewall and kernel upgrades as smooth as possible, several new features
    were added to the Shorewall 4.5.7:</para>

    <itemizedlist>
      <listitem>
        <para>Shorewall automatically disables the kernel's automatic
        association of helpers to connections on kernel 3.5 and later.</para>
      </listitem>

      <listitem>
        <para>An automatic association of helpers with connections that
        performs the same function as in the pre-3.5 kernels has been added.
        This automatic association is controlled by the AUTOHELPERS
        shorewall.conf option which is set to 'Yes' by default.</para>
      </listitem>

      <listitem>
        <para>A HELPERS column has been added to the /etc/shorewall/rules In
        the NEW section: When the ACTION is ACCEPT, DNAT or REDIRECT, the
        specified helper is automatically associated with the
        connection.</para>
      </listitem>

      <listitem>
        <para>HELPERS may be specified in action files, macros and in the
        rules file itself. In the RELATED section: The rule will only match
        related connections that have the named helper attached. - The
        standard Macros for applications requiring a helper (FTP, IRC, etc)
        have been modified to automatically specify the correct helper in the
        HELPER column.</para>
      </listitem>

      <listitem>
        <para>HELPER is now a valid action in /etc/shorewall/rules. This
        action requires that a helper be present in the HELPER column and
        causes the specified helper to be associated with connections matching
        the rule. No destination zone should be specified in HELPER rules.
        HELPER rules allow specification of a helper for connections that are
        ACCEPTed by the applicable policy.</para>

        <para>Example (loc-&gt;net policy is ACCEPT) - In
        /etc/shorewall/rules:</para>

        <programlisting>#ACTION     SOURCE       DEST
FTP(HELPER) loc          - </programlisting>

        <para>or equivalently</para>

        <programlisting>#ACTION     SOURCE       DEST    PROTO  DPORT
HELPER      loc          -       tcp    21   { helper=ftp }</programlisting>
      </listitem>

      <listitem>
        <para>The set of enabled helpers (either by AUTOHELPERS=Yes or by the
        HELPERS column) can be taylored using the new HELPERS option in
        shorewall.conf.</para>
      </listitem>
    </itemizedlist>

    <para>By making AUTOHELPERS=Yes the default, users can upgrade their
    systems to a 3.5+ kernel without disrupting the operation of their
    firewalls. Beyond such upgrades, we suggest setting AUTOHELPERS=No and
    follow one of two strategies:</para>

    <itemizedlist>
      <listitem>
        <para>Use the HELPERS column in the rules file to enable helpers as
        needed (preferred); or</para>
      </listitem>

      <listitem>
        <para>Taylor the conntrack file to enable helpers on only those
        connections that are required.</para>
      </listitem>
    </itemizedlist>

    <para>With either of these approaches, the list if available helpers can
    be trimmed using the HELPERS option and rules can be added to the RELATED
    section of the rules file to further restrict the effect of helpers. The
    implementation of these new function places conditional rules in the
    /etc/shorewall[6]/conntrack file. These rules are included conditionally
    based in the setting of AUTOHELPERS.</para>

    <para>Example:</para>

    <programlisting>#ACTION                 SOURCE          DESTINATION     PROTO   DPORT           SPORT   USER            SWITCH
?if $AUTOHELPERS &amp;&amp; __CT_TARGET
?if __FTP_HELPER
CT:helper:ftp           all             -               tcp     21
?endif
...
?endif</programlisting>

    <para>__FTP_HELPER evaluates to false if the HELPERS setting is non-empty
    and 'ftp' is not listed in that setting. For example, if you only need FTP
    access from your 'loc' zone, then add this rule outside of the outer-most
    ?if....?endif shown above.</para>

    <programlisting>#ACTION                 SOURCE          DESTINATION     PROTO   DPORT           SPORT   USER            SWITCH
...
CT:helper:ftp           loc             -               tcp     21</programlisting>

    <para>For an overview of Netfilter Helpers and Shorewall's support for
    dealing with them, see <ulink
    url="Helpers.html">http://www.shorewall.net/Helpers.html</ulink>.</para>

    <para>See <ulink
    url="https://home.regit.org/netfilter-en/secure-use-of-helpers/">https://home.regit.org/netfilter-en/secure-use-of-helpers/</ulink>
    for additional information.</para>
  </section>

  <section id="Ports">
    <title>FTP on Non-standard Ports</title>

    <para>If you are running kernel 3.5 or later and Shorewall 4.5.7 or later,
    then please read the preceding section. You can add appropriate entries
    into <ulink url="manpages/shorewall-rules.html">shorewall-rules(5)</ulink>
    or <ulink
    url="manpages/shorewall-conntrack.html">shorewall-conntrack(5)</ulink> to
    associate the FTP helpers with a nonstandard port.</para>

    <para>Examples using port 12345:</para>

    <para><filename>/etc/shorewall/rules:</filename></para>

    <programlisting>#ACTION         SOURCE         DEST                 PROTO     DPORT
DNAT            net            loc:192.168.1.2:21   tcp       12345  { helper=ftp }the</programlisting>

    <para>That entry will accept ftp connections on port 12345 from the net
    and forward them to host 192.168.1..2 and port 21 in the loc zone.</para>

    <para><filename>/etc/shorewall/conntrack:</filename></para>

    <programlisting>#ACTION                 SOURCE          DESTINATION     PROTO   DPORT           SPORT   USER            SWITCH
...
CT:helper:ftp           loc             -               tcp     12345</programlisting>

    <para>That rule automatically associates the ftp helper with TCP port
    12345 from the 'loc' zone.</para>

    <para>Otherwise, read on.</para>

    <note>
      <para>If you are running <emphasis role="bold">kernel 2.6.19 or
      earlier</emphasis>, replace <emphasis
      role="bold">nf_conntrack_ftp</emphasis> with <emphasis
      role="bold">ip_conntrack_ftp</emphasis> in the following instructions.
      Similarly, replace <emphasis role="bold">nf_nat_ftp</emphasis> with
      <emphasis role="bold">ip_nat_ftp</emphasis>.</para>
    </note>

    <para>The above discussion about commands and responses makes it clear
    that the FTP connection-tracking and NAT helpers must scan the traffic on
    the control connection looking for PASV and PORT commands as well as PASV
    responses. If you run an FTP server on a nonstandard port or you need to
    access such a server, you must therefore let the helpers know by
    specifying the port in <filename>/etc/shorewall/modules</filename> entries
    for the helpers. You should create<filename>
    /etc/shorewall/modules</filename> by copying
    <filename>/usr/share/shorewall/modules</filename>.<caution>
        <para>You must have modularized FTP connection tracking support in
        order to use FTP on a non-standard port.</para>
      </caution></para>

    <example id="Example2">
      <title>if you run an FTP server that listens on port 49 or you need to
      access a server on the Internet that listens on that port then you would
      have:</title>

      <programlisting>loadmodule nf_conntrack_ftp ports=21,49
loadmodule nf_nat_ftp                   # NOTE: With kernels prior to 2.6.11, you must specify the ports on this line also</programlisting>

      <para><note>
          <para>you MUST include port 21 in the ports list or you may have
          problems accessing regular FTP servers.</para>
        </note></para>

      <para>If there is a possibility that these modules might be loaded
      before Shorewall starts, then you should include the port list in
      /etc/modules.conf:</para>

      <programlisting>options nf_conntrack_ftp ports=21,49
options nf_nat_ftp</programlisting>

      <para><important>
          <para>Once you have made these changes to /etc/shorewall/modules
          and/or /etc/modules.conf, you must either:</para>

          <orderedlist>
            <listitem>
              <para>Unload the modules and restart shorewall:</para>

              <programlisting><command>rmmod nf_nat_ftp; rmmod nf_conntrack_ftp; shorewall restart</command></programlisting>
            </listitem>

            <listitem>
              <para>Reboot</para>
            </listitem>
          </orderedlist>
        </important></para>
    </example>
  </section>

  <section id="Rules">
    <title>Rules</title>

    <warning>
      <para>If you run an FTP server behind your firewall and your server
      offers a method of specifying the external IP address of your firewall,
      DON'T USE THAT FEATURE OF YOUR SERVER. Using that option will defeat the
      purpose of the ftp helper modules and can result in a server that
      doesn't work.</para>
    </warning>

    <para>If the policy from the source zone to the destination zone is ACCEPT
    and you don't need DNAT (see <ulink url="FAQ.htm#faq30">FAQ 30</ulink>)
    then <emphasis role="bold">you need no rule</emphasis>.</para>

    <para>Otherwise, for FTP you need exactly <emphasis
    role="bold">one</emphasis> rule:</para>

    <programlisting>#ACTION      SOURCE     DESTINATION    PROTO     DPORT      SPORT       ORIGDEST
ACCEPT or    &lt;<emphasis>source</emphasis>&gt;   &lt;<emphasis>destination</emphasis>&gt;  tcp       21         -           &lt;external IP addr&gt; if
DNAT                                                                    ACTION = DNAT</programlisting>

    <para>You need an entry in the ORIGDEST column only if the ACTION is DNAT,
    you have multiple external IP addresses and you want a specific IP address
    to be forwarded to your server.</para>

    <para>Note that you do <emphasis role="bold">NOT </emphasis>need a rule
    with 20 (ftp-data) in the DPORT column. If you post your rules on the
    mailing list and they show 20 in the DPORT column, we will know that you
    haven't read this article and will either ignore your post or tell you to
    RTFM.</para>

    <para>Shorewall includes an FTP macro that simplifies creation of FTP
    rules. The macro source is in
    <filename>/usr/share/shorewall/macro.FTP</filename>. Using the macro is
    the preferred way to generate the rules described above. Here are a couple
    of examples.</para>

    <para><example id="Example3">
        <title>Server running behind a Masquerading Gateway</title>

        <para>Suppose that you run an FTP server on 192.168.1.5 in your local
        zone using the standard port (21). You need this rule:</para>

        <programlisting>#ACTION      SOURCE     DESTINATION     PROTO     DPORT      SPORT       ORIGDEST
FTP(DNAT)    net       loc:192.168.1.5</programlisting>
      </example><example id="Example4">
        <title>Allow your DMZ FTP access to the Internet</title>

        <programlisting>#ACTION      SOURCE     DESTINATION     PROTO     DPORT      SPORT       ORIGDEST
FTP(ACCEPT)  dmz        net</programlisting>
      </example></para>

    <para>Note that the FTP connection tracking in the kernel cannot handle
    cases where a PORT command (or PASV reply) is broken across two packets or
    is missing the ending &lt;cr&gt;/&lt;lf&gt;. When such cases occur, you
    will see a console message similar to this one:</para>

    <programlisting>Apr 28 23:55:09 gateway kernel: conntrack_ftp: partial PORT 715014972+1</programlisting>

    <para>or this one:</para>

    <programlisting>21:37:40 insert-master kernel: [832161.057782] <emphasis
        role="bold">nf_ct_ftp: dropping 
packet</emphasis> IN=eth4 OUT= MAC=00:0a:cd:1a:d1:95:00:22:6b:be:3c:41:08:00 
SRC=66.199.187.46 DST=192.168.41.1 LEN=102 TOS=0x00 PREC=0x00 TTL=45 
ID=30239 DF PROTO=TCP SPT=21 DPT=50892 SEQ=698644583 ACK=3438176321 
WINDOW=46 RES=0x00 ACK PSH URGP=0 OPT (0101080A932DFE0231935CF7) MARK=0x1</programlisting>

    <para>I see this problem occasionally with the FTP server in my DMZ. My
    solution is to add the following rule:</para>

    <programlisting>#ACTION      SOURCE     DESTINATION     PROTO     DPORT      SPORT       ORIGDEST
ACCEPT:info  dmz        net             tcp       -          20</programlisting>

    <para>The above rule accepts and logs all active mode connections from my
    DMZ to the net.</para>
  </section>
</article>