Upgrade Issues

Upgrade Issues Tom Eastep 2002 2003 2004 2005 2006 Thomas M. Eastep Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover, and with no Back-Cover Texts. A copy of the license is included in the section entitled GNU Free Documentation License.

Important It is important that you read all of the sections on this page where the version number mentioned in the section title is later than what you are currently running. In the descriptions that follows, the term group refers to a particular network or subnetwork (which may be 0.0.0.0/0 or it may be a host address) accessed through a particular interface. Examples: eth0:0.0.0.0/0 eth2:192.168.1.0/24 eth3:192.0.2.123 You can use the shorewall check command to see the groups associated with each of your zones.

Version >= 3.2.0 If you are upgrading from version 2.4 or earlier, please read the 3.0.0 upgrade considerations below. A number of macros have been split into two. The macros affected are: IMAP LDAP NNTP POP3 SMTP Each of these macros now handles only traffic on the native (plaintext) port. There is a corresponding macro with S added to the end of the name for the SSL version of the same protocol. Thus each macro results in the insertion of only one port per invocation. The Web macro has not been split, but two new macros, HTTP and HTTPS have been created. The Web macro is deprecated in favour of these new macros, and may be removed from future Shorewall releases. These changes have been made to ensure no unexpected ports are opened due to the use of macros. In previous Shorewall releases, DNAT and REDIRECT rules supported a special syntax for exclusion of a subnet from the effect of the rule. Example:

Z2 is a subzone of Z1: DNAT Z1!Z2 loc:192.168.1.4 ...

That feature has never worked correctly when Z2 is a dynamic zone. Furthermore, now that Shorewall supports exclusion lists, the capability is redundant since the above rule can now be written in the form: DNAT Z1:!<list of exclusions> loc:192.168.1.4 ... Beginning with Shorewall 3.2.0, the special exclusion syntax will no longer be supported. Important if you use the QUEUE target. In the /etc/shorewall/rules file and in actions, you may now specify 'tcp:syn' in the PROTO column. 'tcp:syn' is equivalent to 'tcp' but also requires that the SYN flag is set and the RST, FIN and ACK flags be off ("--syn" is added to the iptables rule). As part of this change, Shorewall no longer adds the "--syn" option to TCP rules that specify QUEUE as their target. Extension Scripts may require change In previous releases, extension scripts were executed during [re]start by using the Bourne Shell "." operator. In addition to executing commands during [re]start, these scripts had to "save" the commands to be executed during shorewall restore. This clumsiness has been eliminated in Shorewall 3.2. In Shorewall 3.2, extension scripts are copied in-line into the compiled program and are executed in-line during start, restart and restore. This applies to all extension scripts except those associated with a chain or action -- those extension scripts continue to be processed at compile time. This new approach has two implications for existing scripts. It is no longer necessary to save the commands; so functions like 'save_command', 'run_and_save_command' and 'ensure_and_save_command' need no longer be called. The generated program will contain functions with these names: save_command() - does nothing run_and_save_command() - runs the passed command ensure_and_save_command() - runs the passed command and stops the firewall if the command fails. These functions should provide for transparent migration of scripts that use them until you can get around to eliminating their use completely. When the extension script is copied into the compiled program, it is indented to line up with the surrounding code. If you have 'awk' installed on your system, the Shorewall compiler will correctly handle line continuation (last character on the line = "\"). If you do not have awk, it will not be possible to use line-continuation in your extension scripts. In no case is it possible to continue a quoted string over multiple lines without having additional whitespace inserted into the string. Beginning with this release, the way in which packet marking in the PREROUTING chain interracts with the 'track' option in /etc/shorewall/providers has changed in two ways: Packets arriving on a tracked interface are now passed to the PREROUTING marking chain so that they may be marked with a mark other than the 'track' mark (the connection still retains the 'track' mark). When HIGH_ROUTE_MARKS=Yes, you can still clear the mark on packets in the PREROUTING chain (i.e., you can specify a mark value of zero). Kernel version 2.6.16 introduces 'xtables', a new common packet filtering and connection tracking facility that supports both IPv4 and IPv6. Because a different set of kernel modules must be loaded for xtables, Shorewall now includes two 'modules' files: /usr/share/shorewall/modules -- the former /etc/shorewall/modules /usr/share/shorewall/xmodules -- a new file that support xtables. If you wish to use the new file, then simply execute this command: cp -f /usr/share/shorewall/xmodules /etc/shorewall/modules (Versions >= 3.2.3) Previously, CLASSIFY tcrules were always processed out of the POSTROUTING chain. Beginning with this release, they are processed out of the POSTROUTING chain *except* when the SOURCE is $FW[:<address>] in which case the rule is processed out of the OUTPUT chain. With correctly-coded rulesets, this change should have no effect. Users having incorrectly-coded tcrules may need to change them. Example:

#MARK/ SOURCE DEST PROTO DEST SOURCE #CLASSIFY PORTS(S) PORT(S) 1:110 $FW eth3 tcp - 22

While the user may have expected this rule to only affect traffic from the firewall itself, the rule was really equivalent to this one:

#MARK/ SOURCE DEST PROTO DEST SOURCE #CLASSIFY PORTS(S) PORT(S) 1:110 0.0.0.0/0 eth3 tcp - 22

So after this change, the second rule will be required rather than the first if that is what was really wanted.

Version >= 3.0.0 The "monitor" command has been eliminated. The "DISPLAY" and "COMMENTS" columns in the /etc/shorewall/zones file have been removed and have been replaced by the former columns of the /etc/shorewall/ipsec file. The latter file has been removed. Additionally the FW option in shorewall.conf has been deprecated and is no longer set to 'fw' by default. New users are expected to define the firewall zone in /etc/shorewall/zones. Adhering to the principle of least astonishment, the old /etc/shorewall/ipsec file will continue to be supported. A new IPSECFILE variable in /etc/shorewall/shorewall.conf determines the name of the file that Shorewall looks in for IPSEC information. If that variable is not set or is set to the empty value then IPSECFILE=ipsec is assumed. So if you simply upgrade and don't do something idiotic like replace your current shorewall.conf file with the new one, your old configuration will continue to work. A dummy 'ipsec' file is included in the release so that your package manager (e.g., rpm) won't remove your existing file. The shorewall.conf file included in this release sets IPSECFILE=zones so that new users are expected to use the new zone file format. The DROPINVALID option has been removed from shorewall.conf. The behavior will be as if DROPINVALID=No had been specified. If you wish to drop invalid state packets, use the dropInvalid built-in action. The 'nobogons' interface and hosts option as well as the BOGON_LOG_LEVEL option have been eliminated. Most of the standard actions have been replaced by parameterized macros (see below). So for example, the action.AllowSMTP and action.DropSMTP have been removed an a parameterized macro macro.SMTP has been added to replace them. In order that current users don't have to immediately update their rules and user-defined actions, Shorewall can substitute an invocation of the a new macro for an existing invocation of one of the old actions. So if your rules file calls AllowSMTP, Shorewall will replace that call with SMTP/ACCEPT. Because this substitution is expensive, it is conditional based on the setting of MAPOLDACTIONS in shorewall.conf. If this option is set to YES or if it is not set (such as if you are using your old shorewall.conf file) then Shorewall will perform the substitution. Once you have converted to use the new macros, you can set MAPOLDACTIONS=No and invocations of those actions will go much quicker during 'shorewall [re]start'. The STATEDIR variable in /etc/shorewall/shorewall.conf has been removed. STATEDIR is now fixed at /var/lib/shorewall. If you have previously set STATEDIR to another directory, please copy the files from that directory to /var/lib/shorewall/ before [re]starting Shorewall after the upgrade to this version. The "shorewall status" command now just gives the status of Shorewall (started or not-started). The previous status command has been renamed "dump". The command also shows the state relative to the state diagram at http://shorewall.net/starting_and_stopping_shorewall.htm. In addition to the state, the time and date at which that state was entered is shown. Note that at least one "shorewall [re]start" must be issued after upgrading to this release before "shorewall status" will show anything but "Unknown" for the state. The "shorewall forget" command now removes the dynamic blacklist save file (/var/lib/shorewall/save). In previous versions of Shorewall, the rules generated by entries in /etc/shorewall/tunnels preceded those rules generated by entries in /etc/shorewall/rules. Beginning with this release, the rules generated by entries in the tunnels file will appear *AFTER* the rules generated by the rules file. This may cause you problems if you have REJECT, DENY or CONTINUE rules in your rules file that would cause the tunnel transport packets to not reach the rules that ACCEPT them. See http://www.shorewall.net/VPNBasics.html for information on the rules generated by entries in the tunnels file. The NEWNOTSYN and LOGNEWNOTSYN options in shorewall.conf have been removed as have the 'newnotsyn' options in /etc/shorewall/interfaces and /etc/shorewall/hosts. TCP new-not-syn packets may be blocked using the 'dropNotSyn' or 'rejNotSyn' built-in actions. Example: Reject all new-not-syn packets from the net and log them at the 'info' level. #ACTION SOURCE DEST PROTO SECTION NEW rejNotSyn:info net all tcp Note that the rule is added at the front of the NEW section of the rules file. A new TC_SCRIPT option replaces TC_ENABLED in shorewall.conf. If the option is not set then the internal shaper (tc4shorewall by Arne Bernin) is used. Otherwise, the script named in the variable is used. Users who currently use an /etc/shorewall/tcstart file and wish to continue to do so should set TC_SCRIPT=/etc/shorewall/tcstart in shorewall.conf.

Version >= 2.4.0 Shorewall now enforces the restriction that mark values used in /etc/shorewall/tcrules are less than 256. If you are using mark values >= 256, you must change your configuration before you upgrade. The value "ipp2p" is no longer accepted in the PROTO column of the /etc/shorewall/rules file. This support has never worked as intended and cannot be made to work in a consistent way. A "Howto" article on filtering P2P with Shorewall and ipp2p will be forthcoming. LEAF/Bering packages for 2.4.0 and later releases are not available from shorewall.net. See the LEAF site for those packages.

Version >= 2.2.0 Shorewall configuration files except shorewall.conf are now empty (they contain only comments). If you wish to retain the defaults in any of the following files, you should copy these files before upgrading them then restore them after the upgrade: /etc/shorewall/zones /etc/shorewall/policy /etc/shorewall/tos The following builtin actions have been removed and have been replaced by the new action logging implementation described in the new features below. logNotSyn rLogNotSyn dLogNotSyn If shorewall.conf is upgraded to the latest version, it needs to be modified to set STARTUP_ENABLED=Yes. The Leaf/Bering version of Shorewall was previously named: shorwall-<version>.lrp Beginning with 2.1, that file will now be named: shorewall-lrp-<version>.tgz Simply rename that file to 'shorwall.lrp' when installing it on your LEAF/Bering system. The ORIGINAL DEST column of the /etc/shorewall/rules file may no longer contain a second (SNAT) address. You must use an entry in /etc/shorewall/masq instead. Example from Shorewall FAQ #1:

Prior to Shorewall 2.1: /etc/shorewall/interfaces #ZONE INTERFACE BROADCAST OPTIONS loc eth1 detect routeback /etc/shorewall/rules #ACTION SOURCE DEST PROTO DEST PORT SOURCE ORIGINAL # PORT DEST DNAT loc loc:192.168.1.12 tcp 80 - 130.252.100.69:192.168.1.254 Shorewall 2.1 and Later: /etc/shorewall/interfaces #ZONE INTERFACE BROADCAST OPTIONS loc eth1 detect routeback /etc/shorewall/masq: #INTERFACE SUBNETS ADDRESS PROTO PORT(S) eth1 eth1 192.168.1.254 tcp 80 /etc/shorewall/rules: #ACTION SOURCE DEST PROTO DEST PORT SOURCE ORIGINAL # PORT DEST DNAT loc loc:192.168.1.12 tcp 80 - 130.252.100.69

The 'logunclean' and 'dropunclean' options that were deprecated in Shorewall 2.0 have now been removed completely. The default port for 'openvpn' tunnels (/etc/shorewall/tunnels) has been changed to 1194 to match a similar change in the OpenVPN product. The IANA has registered port 1194 for use by OpenVPN. A new IPTABLES variable has been added to shorewall.conf. This variable names the iptables executable that Shorewall will use. The variable is set to "/sbin/iptables". If you use the new shorewall.conf, you may need to change this setting to maintain compatibility with your current setup (if you use your existing shorewall.conf that does not set IPTABLES then you should experience no change in behavior).

Version >= 2.0.2 RC1 If you are upgrading from Shorewall 1.4.x and you have commands in your /etc/shorewall/common file that are not directly related to the common chain then you will want to move those commands to /etc/shorewall/initdone.

Version >= 2.0.2 Beta 1 Extension Scripts - In order for extension scripts to work properly with the new iptables-save/restore integration introduced in Shorewall 2.0.2 Beta 1, some change may be required to your extension scripts. If your extension scripts are executing commands other than iptables then those commands must also be written to the restore file (a temporary file in /var/lib/shorewall that is renamed /var/lib/shorewall/restore-base at the completion of the /sbin/shorewall command). The following functions should be of help: save_command() -- saves the passed command to the restore file. Example: save_command echo Operation Complete That command would simply write "echo Operation Complete" to the restore file. run_and_save_command() -- saves the passed command to the restore file then executes it. The return value is the exit status of the command. Example: run_and_save_command "echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_all" Note that as in this example, when the command involves file redirection then the entire command must be enclosed in quotes. This applies to all of the functions described here. ensure_and_save_command() -- runs the passed command. If the command fails, the firewall is restored to it's prior saved state and the operation is terminated. If the command succeeds, the command is written to the restore file Dynamic Zone support. - If you don't need to use the shorewall add and shorewall delete commands, you should set DYNAMIC_ZONES=No in /etc/shorewall/shorewall.conf.

Version >= 2.0.1 The function of 'norfc1918' is now split between that option and a new 'nobogons' option. The rfc1918 file released with Shorewall now contains entries for only those three address ranges reserved by RFC 1918. A 'nobogons' interface option has been added which handles bogon source addresses (those which are reserved by the IANA, those reserved for DHCP auto-configuration and the class C test-net reserved for testing and documentation examples). This will allow users to perform RFC 1918 filtering without having to deal with out of date data from IANA. Those who are willing to update their /usr/share/shorewall/bogons file regularly can specify the 'nobogons' option in addition to 'norfc1918'. The level at which bogon packets are logged is specified in the new BOGON_LOG_LEVEL variable in shorewall.conf. If that option is not specified or is specified as empty (e.g, BOGON_LOG_LEVEL="") then bogon packets whose TARGET is 'logdrop' in /usr/share/shorewall/bogons are logged at the 'info' level. If you have a file named rfc1918 in your /etc/shorewall directory, you must either remove that file or you must copy /usr/share/shorewall/rfc1918 to /etc/shorewall and modify that file appropriately. Do not leave the old rfc1918 file in /etc/shorewall.

VERSION >= 2.0.0-Beta1 The 'dropunclean' and 'logunclean' interface options are no longer supported. If either option is specified in /etc/shorewall/interfaces, a threatening message will be generated. The NAT_BEFORE_RULES option has been removed from shorewall.conf. The behavior of Shorewall 2.0 is as if NAT_BEFORE_RULES=No had been specified. In other words, DNAT rules now always take precedence over one-to-one NAT specifications. The default value for the ALL INTERFACES column in /etc/shorewall/nat has changed. In Shorewall 1.*, if the column was left empty, a value of "Yes" was assumed. This has been changed so that a value of "No" is now assumed. The following files don't exist in Shorewall 2.0: /etc/shorewall/common.def /etc/shorewall/common /etc/shorewall/icmpdef /etc/shorewall/action.template (moved to /usr/share/shorewall/action.template) The /etc/shorewall/action file now allows an action to be designated as the "common" action for a particular policy type by following the action name with ":" and the policy (DROP, REJECT or ACCEPT). The file /usr/share/shorewall/actions.std has been added to define those actions that are released as part of Shorewall 2.0 In that file are two actions as follows: Drop:DROP Reject:REJECT The Drop action is the common action for DROP policies while the Reject action is the default action for REJECT policies. These actions will be performed on packets prior to applying the DROP or REJECT policy respectively. In the first release, the difference between "Reject" and "Drop" is that "Reject" REJECTs SMB traffic while "Drop" silently drops such traffic. As described above, Shorewall allows a common action for ACCEPT policies but does not specify such an action in the default configuration. For more information see the User-defined Action Page. The /etc/shorewall directory no longer contains users file or a usersets file. Similar functionality is now available using user-defined actions. Now, action files created by copying /usr/share/shorewall/action.template may now specify a USER and or GROUP name/id in the final column just like in the rules file (see below). It is thus possible to create actions that control traffic from a list of users and/or groups. The last column in /etc/shorewall/rules is now labeled USER/GROUP and may contain: [!]<user number>[:] [!]<user name>[:] [!]:<group number> [!]:<group name> [!]<user number>:<group number> [!]<user name>:<group number> [!]<user inumber>:<group name> [!]<user name>:<group name> If your kernel has IPV6 support (recent SUSE for example), and you don't use IPV6 then you will probably want to set DISABLE_IPV6=Yes in /etc/shorewall/shorewall.conf. You must have ipv6tables installed.

Version >= 1.4.9 The default value of NEWNOTSYN set in /etc/shorewall/shorewall.conf has been changed from 'No' to 'Yes'. I find that NEWNOTSYN=No tends to result in lots of "stuck" connections because any network timeout during TCP session tear down results in retries being dropped (Netfilter has removed the connection from the conntrack table but the end-points haven't completed shutting down the connection). I therefore have chosen NEWNOTSYN=Yes as the default value and I advise caution in using NEWNOTSYN=Yes.

Version >= 1.4.8 The meaning of ROUTE_FILTER=Yes has changed. Previously this setting was documented as causing route filtering to occur on all network interfaces; this didn't work. Beginning with this release, ROUTE_FILTER=Yes causes route filtering to occur on all interfaces brought up while Shorewall is running. This means that it may be appropriate to set ROUTE_FILTER=Yes and use the routefilter option in /etc/shorewall/interfaces entries.

Version >= 1.4.6 The NAT_ENABLED, MANGLE_ENABLED and MULTIPORT options have been removed from shorewall.conf. These capabilities are now automatically detected by Shorewall. An undocumented feature previously allowed entries in the host file as follows: zone eth1:192.168.1.0/24,eth2:192.168.2.0/24 This capability was never documented and has been removed in 1.4.6 to allow entries of the following format: zone eth1:192.168.1.0/24,192.168.2.0/24

Version >= 1.4.4 If you are upgrading from 1.4.3 and have set the LOGMARKER variable in /etc/shorewall/shorewall.conf, then you must set the new LOGFORMAT variable appropriately and remove your setting of LOGMARKER.

Version 1.4.4 If you have zone names that are 5 characters long, you may experience problems starting Shorewall because the in a logging rule is too long. Upgrade to Version 1.4.4a to fix this problem.

Version >= 1.4.2 There are some cases where you may want to handle traffic from a particular group to itself. While I personally think that such a setups are ridiculous, there are two cases covered in this documentation where it can occur: In FAQ #2 When running Squid as a transparent proxy in your local zone. If you have either of these cases, you will want to review the current documentation and change your configuration accordingly.

Version >= 1.4.1 Beginning with Version 1.4.1, traffic between groups in the same zone is accepted by default. Previously, traffic from a zone to itself was treated just like any other traffic; any matching rules were applied followed by enforcement of the appropriate policy. With 1.4.1 and later versions, unless you have explicit rules for traffic from Z to Z or you have an explicit Z to Z policy (where "Z" is some zone) then traffic between the groups in zone Z will be accepted. If you do have one or more explicit rules for Z to Z or if you have an explicit Z to Z policy then the behavior is as it was in prior versions. If you have a Z Z ACCEPT policy for a zone to allow traffic between two interfaces to the same zone, that policy can be removed and traffic between the interfaces will traverse fewer rules than previously. If you have a Z Z DROP or Z Z REJECT policy or you have Z->Z rules then your configuration should not require any change. If you are currently relying on a implicit policy (one that has "all" in either the SOURCE or DESTINATION column) to prevent traffic between two interfaces to a zone Z and you have no rules for Z->Z then you should add an explicit DROP or REJECT policy for Z to Z. Sometimes, you want two separate zones on one interface but you don't want Shorewall to set up any infrastructure to handle traffic between them. The <filename>zones</filename>, <filename>interfaces</filename> and, <filename>hosts</filename> file contents /etc/shorewall/zones z1 Zone1 The first Zone z2 Zone2 The second Zone /etc/shorewall/interfaces z2 eth1 192.168.1.255 /etc/shorewall/hosts z1 eth1:192.168.1.3 Here, zone z1 is nested in zone z2 and the firewall is not going to be involved in any traffic between these two zones. Beginning with Shorewall 1.4.1, you can prevent Shorewall from setting up any infrastructure to handle traffic between z1 and z2 by using the new NONE policy: The contents of <filename>policy</filename> /etc/shorewall/policy z1 z2 NONE z2 z1 NONE Note that NONE policies are generally used in pairs unless there is asymmetric routing where only the traffic on one direction flows through the firewall and you are using a NONE policy in the other direction.

Version 1.4.1 In Version 1.4.1, Shorewall will never create rules to deal with traffic from a given group back to itself. The multi interface option is no longer available so if you want to route traffic between two subnetworks on the same interface then I recommend that you upgrade to Version 1.4.2 and use the routeback interface or host option.

Version >= 1.4.0 Shorewall >=1.4.0 requires the iproute package ('ip' utility). Unfortunately, some distributions call this package iproute2 which will cause the upgrade of Shorewall to fail with the diagnostic: error: failed dependencies:iproute is needed by shorewall-1.4.0-1This may be worked around by using the option of rpm (rpm -Uvh --nodeps your_shorewall_rpm.rpm). If you are upgrading from a version < 1.4.0, then: The noping and forwardping interface options are no longer supported nor is the FORWARDPING option in shorewall.conf. ICMP echo-request (ping) packets are treated just like any other connection request and are subject to rules and policies. Interface names of the form <device>:<integer> in /etc/shorewall/interfaces now generate a Shorewall error at startup (they always have produced warnings in iptables). The MERGE_HOSTS variable has been removed from shorewall.conf. Shorewall 1.4 behaves like 1.3 did when MERGE_HOSTS=Yes; that is zone contents are determined by BOTH the interfaces and hosts files when there are entries for the zone in both files. The routestopped option in the interfaces and hosts file has been eliminated; use entries in the routestopped file instead. The Shorewall 1.2 syntax for DNAT and REDIRECT rules is no longer accepted; you must convert to using the new syntax. The ALLOWRELATED variable in shorewall.conf is no longer supported. Shorewall 1.4 behavior is the same as 1.3 with ALLOWRELATED=Yes. Late-arriving DNS replies are now dropped by default; there is no need for your own /etc/shorewall/common file simply to avoid logging these packets. The firewall, functions and version files have been moved to /usr/share/shorewall. The icmp.def file has been removed. If you include it from /etc/shorewall/icmpdef, you will need to modify that file. If you followed the advice in FAQ #2 and call find_interface_address in /etc/shorewall/params, that code should be moved to /etc/shorewall/init.

Version 1.4.0 The multi interface option is no longer supported. Shorewall will generate rules for sending packets back out the same interface that they arrived on in two cases: There is an explicit policy for the source zone to or from the destination zone. An explicit policy names both zones and does not use the all reserved word. There are one or more rules for traffic for the source zone to or from the destination zone including rules that use the all reserved word. Exception: if the source zone and destination zone are the same then the rule must be explicit - it must name the zone in both the SOURCE and DESTINATION columns.