Shorewall and FTP

FTP transfers involve two TCP connections. The first control connection goes from the FTP client to port 21 on the FTP server. This connection is used for logon and to send commands and responses between the endpoints. Data transfers (including the output of "ls" and "dir" commands) requires a second data connection. The data connection is dependent on the mode that the client is operating in:

Passive Mode (default for web browsers) -- The client issues a PASV command. Upon receipt of this command, the server listens on a dynamically-allocated port then sends a PASV reply to the client. The PASV reply gives the IP address and port number that the server is listening on. The client then opens a second connection to that IP address and port number.
Active Mode (often the default for line-mode clients) -- The client listens on a dynamically-allocated port then sends a PORT command to the server. The PORT command gives the IP address and port number that the client is listening on. The server then opens a connection to that IP address and port number; the source port for this connection is 20 (ftp-data in /etc/services).

You can see these commands in action using your linux ftp command-line client in debugging mode. Note that my ftp client defaults to passive mode and that I can toggle between passive and active mode by issuing a "passive" command:

[teastep@wookie Shorewall]$ ftp ftp1.shorewall.net
Connected to lists.shorewall.net.
220-=(<*>)=-.:. (( Welcome to PureFTPd 1.0.12 )) .:.-=(<*>)=-
220-You are user number 1 of 50 allowed.
220-Local time is now 10:21 and the load is 0.14. Server port: 21.
220 You will be disconnected after 15 minutes of inactivity.
500 Security extensions not implemented
500 Security extensions not implemented
KERBEROS_V4 rejected as an authentication type
Name (ftp1.shorewall.net:teastep): ftp
331-Welcome to ftp.shorewall.net
331-
331 Any password will work
Password:
230 Any password will work
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> debug
Debugging on (debug=1).
ftp> ls
---> PASV
227 Entering Passive Mode (192,168,1,193,195,210)
---> LIST
150 Accepted data connection
drwxr-xr-x    5 0        0            4096 Nov  9  2002 archives
drwxr-xr-x    2 0        0            4096 Feb 12  2002 etc
drwxr-sr-x    6 0        50           4096 Feb 19 15:24 pub
226-Options: -l
226 3 matches total
ftp> passive
Passive mode off.
ftp> ls
---> PORT 192,168,1,3,142,58
200 PORT command successful
---> LIST
150 Connecting to port 36410
drwxr-xr-x    5 0        0            4096 Nov  9  2002 archives
drwxr-xr-x    2 0        0            4096 Feb 12  2002 etc
drwxr-sr-x    6 0        50           4096 Feb 19 15:24 pub
226-Options: -l
226 3 matches total
ftp>

Things to notice:

The commands that I issued are in green.
Commands sent by the client to the server are preceded by --->
Command responses from the server over the control connection are numbered.
FTP uses a comma as a separator between the bytes of the IP address; and
When sending a port number, FTP sends the MSB then the LSB and separates the two bytes by a comma. As shown in the PORT command, port 142,58 translates to 142*256+58 = 36410.

Given the normal loc->net policy of ACCEPT, passive mode access from local clients to remote servers will always work but active mode requires the firewall to dynamically open a "hole" for the server's connection back to the client. Similarly, if you are running an FTP server in your local zone then active mode should always work but passive mode requires the firewall to dynamically open a "hole" for the client's second connection to the server. This is the role of FTP connection-tracking support in the Linux kernel.

Where any form of NAT (SNAT, DNAT, Masquerading) on your firewall is involved, the PORT commands and PASV responses may also need to be modified by the firewall. This is the job of the FTP nat support kernel function.

Including FTP connection-tracking and NAT support normally means that the modules "ip_conntrack_ftp" and "ip_nat_ftp" need to be loaded. Shorewall automatically loads these "helper" modules from /lib/modules/<kernel-version>/kernel/net/ipv4/netfilter/ and you can determine if they are loaded using the 'lsmod' command:

Example:

[root@lists etc]# lsmod
Module                  Size  Used by    Not tainted
autofs                 12148   0  (autoclean) (unused)
ipt_TOS                 1560  12  (autoclean)
ipt_LOG                 4120   5  (autoclean)
ipt_REDIRECT            1304   1  (autoclean)
ipt_REJECT              3736   4  (autoclean)
ipt_state               1048  13  (autoclean)
ip_nat_irc              3152   0  (unused)
ip_nat_ftp              3888   0  (unused)
ip_conntrack_irc        3984   1
ip_conntrack_ftp        5008   1
ipt_multiport           1144   2  (autoclean)
ipt_conntrack           1592   0  (autoclean)
iptable_filter          2316   1  (autoclean)
iptable_mangle          2680   1  (autoclean)
iptable_nat            20568   3  (autoclean) [ipt_REDIRECT ip_nat_irc ip_nat_ftp]
ip_conntrack           26088   5  (autoclean) [ipt_REDIRECT ipt_state ip_nat_irc ip_nat_ftp ip_conntrack_irc ip_conntrack_ftp ipt_conntrack iptable_nat]
ip_tables              14488  12  [ipt_TOS ipt_LOG ipt_REDIRECT ipt_REJECT ipt_state ipt_multiport ipt_conntrack iptable_filter iptable_mangle iptable_nat]
tulip                  42464   0  (unused)
e100                   50596   1
keybdev                 2752   0  (unused)
mousedev                5236   0  (unused)
hid                    20868   0  (unused)
input                   5632   0  [keybdev mousedev hid]
usb-uhci               24684   0  (unused)
usbcore                73280   1  [hid usb-uhci]
ext3                   64704   2
jbd                    47860   2  [ext3]
[root@lists etc]#

If you want Shorewall to load these modules from an alternate directory, you need to set the MODULESDIR variable in /etc/shorewall/shorewall.conf to point to that directory.

Server configuration is covered in the /etc/shorewall/rules documentation,

For a client, you must open outbound TCP port 21.

The above discussion about commands and responses makes it clear that the FTP connection-tracking and NAT helpers must scan the traffic on the control connection looking for PASV and PORT commands as well as PASV responses. If you run an FTP server on a nonstandard port or you need to access such a server, you must therefore let the helpers know by specifying the port in /etc/shorewall/modules entries for the helpers. For example, if you run an FTP server that listens on port 49 or you need to access a server on the internet that listens on that port then you would have:

loadmodule ip_conntrack_ftp ports=21,49
loadmodule ip_nat_ftp ports=21,49

Note that you MUST include port 21 in the ports list or you may have problems accessing regular FTP servers.

If there is a possibility that these modules might be loaded before Shorewall starts, then you should include the port list in /etc/modules.conf:

options ip_conntrack_ftp ports=21,49
options ip_nat_ftp ports=21,49

IMPORTANT: Once you have made these changes to /etc/shorewall/modules and/or /etc/modules.conf, you must either:

Unload the modules and restart shorewall: (rmmod ip_nat_ftp; rmmod ip_conntrack_ftp; shorewall restart); or
Reboot

One problem that I see occasionally involves active mode and the FTP server in my DMZ. I see the active data connection to certain client IP addresses being continuously rejected by my firewall. It is my conjecture that there is some broken client out there that is sending a PORT command that is being either missed or mis-interpreted by the FTP connection tracking helper yet it is being accepted by my FTP server. My solution is to add the following rule:

ACTION
SOURCE
DESTINATION
PROTOCOL
PORT(S)
SOURCE
PORT(S)
ORIGINAL
DESTINATION

ACCEPT:info
dmz
net
tcp
-
20

The above rule accepts and logs all active mode connections from my DMZ to the net.

Last updated 9/17/2003 - Tom Eastep

ACTION	SOURCE	DESTINATION	PROTOCOL	PORT(S)	SOURCE PORT(S)	ORIGINAL DESTINATION
ACCEPT:info	dmz	net	tcp	-	20