Shorewall 4.0.1 ---------------------------------------------------------------------------- R E L E A S E H I G H L I G H T S ---------------------------------------------------------------------------- 1) This is the first Shorewall release that fully integrates the new Shorewall-perl compiler. See the "New Features" section below. 2) You are now offered a choice as to which compiler(s) you install. In 4.0.0, there are the following packages: - Shorewall-common ( common files ) - Shorewall-shell ( the shell-based compiler ) - Shorewall-perl (the Perl-based compiler ) You must install at least one of the compiler packages (you may install them both) along with Shorewall-common. YOU DO NOT NEED TO UNINSTALL ANY OF YOUR CURRENT PACKAGES. See the Migration Considerations below for further information. 3) The facilities for supporting bridge/firewalls under earlier releases are deprecated and their documentation is omitted from the 4.0 distribution. New bridge support is implemented in the Shorewall-perl compiler. This support utilizes the reduced-function physdev match support available in Linux kernel 2.6.20 and later. Problems corrected in 4.0.1. 1) The Shorewall Lite installer was producing an empty shorewall-lite manpage. Since the installer runs as part of creating the RPM, the RPM also suffered from this problem. The 4.0.0 Shorewall-lite packages were re-uploaded with this problem corrected. 2) The Shorewall Lite uninstaller incorrectly removed /sbin/shorewall rather than /sbin/shorewall-lite. 3) Both the Shorewall and Shorewall Lite uninstallers did a "shorewall clear" if Shorewall [Lite] was running. Now, the Shorewall Lite uninstaller correctly does "shorewall-lite clear" and both uninstallers only perform the 'clear' operation if the other product is not installed. This prevents the removal of one of the two products from clearing the firewall configuration established by the other one. 4) The 'ipsec' OPTION in /etc/shorewall/hosts was mis-handled by Shorewall-perl. If the zone type was changed to 'ipsec' or 'ipsec4' and the 'ipsec' option removed from the hosts file entry, the configuration worked properly. 5) If a CLASSID was specified in a tcrule and TC_ENABLED=No, then Shorewall-perl produced the following: Compiling... Use of uninitialized value in string ne at /usr/share/shorewall-perl/Shorewall/Tc.pm line 285, <$currentfile> line 18. ERROR: Class Id n:m is not associated with device eth0 : /etc/shorewall/tcrules (line 18) 6) If IPTABLES was not specified in shorewall.conf, Shorewall-perl was locating the binary using the PATH environmental variable rather than the PATH setting in shorewall.conf. If no PATH was available when Shorewall-perl was run and IPTABLES was not set in shorewall.conf, the following messages were issued: Use of uninitialized value in split at /usr/share/shorewall-perl/Shorewall/Config.pm line 1054. ERROR: Can't find iptables executable ERROR: Shorewall restart failed 7) If the "Mangle FORWARD Chain" capability was supported, entries in the /etc/shorewall/ecn file would cause invalid iptables commands to be generated. This problem occurred with both compilers. 8) Shorewall now starts at reboot after an upgrade from shorewall < 4.0.0. Previously, Shorewall was not started automatically at reboot after an upgrade using the RPM. 9) Shorewall-perl was generating invalid iptables-restore input when a log level was specified with the dropBcast and allowBcast builtin actions and when a log level followed by '!' was used with any builtin actions. 10) Shorewall-perl was incorrectly rejecting 'min' as a valid unit of time in rate-limiting specifications. 11) Certain errors occurring during start/restart/safe-start/safe-restart/try processing could cause the lockfile to be left behind. This resulted in a 60-second delay the next time one of these commands was run. Other changes in Shorewall 4.0.1. 1) A new EXPAND_POLICIES option is added to shorewall.conf. The option is recognized by Shorewall-perl and is ignored by Shorewall-shell. Normally, when the SOURCE or DEST columns in shorewall-policy(5) contains 'all', a single policy chain is created and the policy is enforced in that chain. For example, if the policy entry is #SOURCE DEST POLICY LOG # LEVEL net all DROP info then the chain name is 'net2all' which is also the chain named in Shorewall log messages generated as a result of the policy. If EXPAND_POLICIES=Yes, then Shorewall-perl will create a separate chain for each pair of zones covered by the policy. This makes the resulting log messages easier to interpret since the chain in the messages will have a name of the form 'a2b' where 'a' is the SOURCE zone and 'b' is the DEST zone. See http://linuxman.wikispaces.com/PPPPPPS for more information. 2) The Shorewall-perl dependency on the "Address Type Match" capability has been relaxed. This allows Shorewall 4.0.1 to be used on releases like RHEL4 that don't support that capability. 3) Shorewall-perl now detects dead policy file entries that result when an entry is masked by an earlier entry. Example: all all REJECT info loc net ACCEPT 4) Recent kernels are apparently hard to configure and we have been seeing a lot of problem reports where the root cause is the lack of state match support in the kernel. This problem is difficult to diagnose when using Shorewall-perl so the generated shell program now checks specifically for this problem and terminates with an error if the capability doesn't exist. Migration Considerations: 1) Beginning with Shorewall 4.0.0, there is no single 'shorewall' package. Rather there are two compiler packages (shorewall-shell and shorewall-perl) and a set of base files (shorewall-common) which are required by either compiler package. Although the names of the packages are changing, you can upgrade without having to uninstall/reinstall. To repeat: YOU DO NOT NEED TO UNINSTALL ANY EXISTING PACKAGE. If you attempt to upgrade using the shorewall-common RPM, you get this result: gateway:~ # rpm -Uvh shorewall-common-4.0.0.noarch.rpm error: Failed dependencies: shorewall_compiler is needed by shorewall-common-4.0.0-1.noarch gateway:~ # You must either: rpm -Uvh shorewall-shell-4.0.0.noarch.rpm \ shorewall-common-4.0.0.noarch.rpm or rpm -Uvh shorewall-shell-4.0.0.noarch.rpm \ shorewall-perl-4.0.0.noarch.rpm \ shorewall-common-4.0.0.noarch.rpm If you don't want to use shorewall-perl exclusively then use the second command above then rpm -e shorewall-shell If you are upgrading using the tarball, you must install shorewall-shell and/or shorewall-perl before you upgrade using shorewall-common. Otherwise, the install.sh script fails with: ERROR: No Shorewall compiler is installed The shorewall-shell and shorewall-perl packages are installed from the tarball in the expected way; untar the package, and run the install.sh script. Example 1: You have 'shorewall' installed and you want to continue to use the shorewall-shell compiler. tar -jxf shorewall-common-4.0.0.tar.bz2 tar -jxf shorewall-shell-4.0.0.tar.bz2 cd shorewall-shell-4.0.0 ./install.sh cd ../shorewall-common-4.0.0 ./install.sh shorewall check shorewall restart Example 2: You have shorewall 3.4.4 and shorewall-perl 4.0.0-Beta7 installed and you want to upgrade to 4.0. You do not need the shell-based compiler. tar -jxf shorewall-common-4.0.0.tar.bz2 tar -jxf shorewall-perl-4.0.0.tar.bz2 cd shorewall-perl-4.0.0 ./install.sh cd ../shorewall-common-4.0.0 ./install.sh shorewall check shorewall restart Be sure to modify shorewall.conf if it still has SHOREWALL_COMPILER=shell. 2) The ROUTE_FILTER and LOG_MARTIANS options in shorewall.conf work slightly differently in Shorewall 4.0.0. In prior releases, leaving these options empty was equivalent to setting them to 'No' which caused the corresponding flag in /proc to be reset for all interfaces. Beginning in Shorewall 4.0.0, leaving these options empty causes Shorewall to leave the flags in /proc as they are. You must set the option to 'No' in order to obtain the old behavior. 3) The -f option is no longer the default when Shorewall is started at boot time (usually via /etc/init.d/shorewall). With Shorewall-perl, "shorewall start" is nearly as fast as "shorewall restore" and "shorewall start" uses the current configuration which avoids confusion. If you plan on continuing to use Shorewall-shell and you want to use the "-f" option at boot time, then you must add the following to /etc/sysconfig/shorewall or /etc/default/shorewall: OPTIONS="-f" If you currently have neither of those files, you will need to create one of them. 4) This issue will only affect you if you use Shorewall Lite and have modified /usr/share/configpath to specify a different LITEDIR. The implementation of LITEDIR has always been unsatisfactory. Furthermore, there have been other cases where people have asked to be able to designate the state directory (default /var/lib/shorewall[-lite]). To meet these objectives: a) The LITEDIR variable has been eliminated in /usr/share/shorewall[-lite]/configpath. b) A new file /etc/shorewall[-lite]/vardir has been added. This file is not created by default but may be added as needed. It is expected to contain a single variable assignment: VARDIR= Example: VARDIR=/root/shorewall To change VARDIR, copy the old directory to the new one before you restart Shorewall[-lite]. To use this feature with Shorewall-lite, all packages involved (compiler, shorewall-common and shorewall-lite) must be version 4.0.0-RC2 or later. ---------------------------------------------------------------------------- N E W F E A T U R E S ---------------------------------------------------------------------------- 1) Shorewall-perl This Shorewall package includes a complete rewrite of the compiler in Perl. I decided to make Shorewall-perl a separate package for several reasons: a) Embedded applications are unlikely to adopt Shorewall-perl; even Mini-Perl has a substantial disk and RAM footprint. b) Because of the gross incompatibilities between the new compiler and the old (see below), migration to the new compiler must be voluntary. ------------------------------------------------------------------------ T H E G O O D N E W S: ------------------------------------------------------------------------ a) The compiler has a small disk footprint. b) The compiler is very fast. c) The compiler generates a firewall script that uses iptables-restore; so the script is very fast. d) The new compiler does a much better job of validating the configuration and catches many errors that resulted in run-time failures with the old compiler. e) Use of the Shorewall-perl is optional! The old slow clunky Bourne-shell compiler is still available. ------------------------------------------------------------------------ T H E B A D N E W S: ------------------------------------------------------------------------ There are a number of incompatibilities between the Perl-based compiler and the Bourne-shell one. a) The Perl-based compiler requires the following capability in your kernel and iptables. - multiport match This capability is in current distributions. b) Now that Netfilter has features to deal reasonably with port lists, I see no reason to duplicate those features in Shorewall. The Bourne-shell compiler goes to great pain (in some cases) to break very long port lists ( > 15 where port ranges in lists count as two ports) into individual rules. In the new compiler, I'm avoiding the ugliness required to do that. The new compiler just generates an error if your list is too long. It will also produce an error if you insert a port range into a port list and you don't have extended multiport support. c) The old BRIDGING=Yes support has been replaced by new bridge support that uses the reduced 'physdev match' capabilities found in kernel 2.6.20 and later. This new implementation may be used where it is desired to control traffic through a bridge. The new implementation includes the following features: a) A new "Bridge Port" zone type is defined. Specify 'bport' or 'bport4' in the TYPE column of /etc/shorewall/zones. Bridge Port zones should be a sub-zone of a regular ipv4 zone that represents all hosts attached to the bridge. b) A new 'bridge' option is defined for entries in /etc/shorewall/interfaces. Bridges should have this option specified, even if you don't want to filter traffic going through the bridge. c) Bridge ports must now be defined in /etc/shorewall/interfaces. The INTERFACE column contains both the bridge name and the port name separated by a colon (e.g., "br0:eth1"). No OPTIONS are allowed for bridge ports. The bridge must be defined before its ports and must have the 'bridge' option. Bridge Port (BP) zones have a number of limitations: a) Each BP zone may only be associated with ports on a single bridge. b) BP zones may not be associated with interfaces that are not bridge ports. c) You may not have policies or rules where the DEST is a BP zone but the source is not a BP zone. If you need such rules, you must use the BP zone's parent zone as the DEST zone. Example (Bridge br0 with ports eth1 and tap0): /etc/shorewall/zones: fw firewall net ipv4 loc ipv4 lan:loc bport vpn:loc bport /etc/shorewall/interfaces: net eth0 - ... loc br0 - ... lan eth1 vpn tap0 When using the /etc/shorewall/hosts file to define a bport4 zone, you specify only the port name: Example: /etc/shorewall/zones: fw firewall net ipv4 loc ipv4 lan:loc bport vpn:loc bport /etc/shorewall/hosts lan eth1:192.168.2.0/24 ... The structure of the accounting rules changes slightly when there are bridges defined in the Shorewall configuration. Because of the restrictions imposed by Netfilter in kernel 2.6.21 and later, output accounting rules must be segregated from forwarding and input rules. To accomplish this separation, Shorewall-perl creates two accounting chains: - accounting - for input and forwarded traffic. - accountout - for output traffic. If the CHAIN column contains '-', then: - If the SOURCE column in a rule includes the name of the firewall zone (e.g., $FW), then the rule is add only to the accountout chain. - Otherwise, if the DEST in the rule is any or all or 0.0.0.0/0, then the rule is added to both accounting and accountout. - Otherwise, the rule is added to accounting only. See http://www.shorewall.net/4.0/bridge-Shorewall-perl.html for additional information about the new bridge support. d) The BROADCAST column in the interfaces file is essentially unused; if you enter anything in this column but '-' or 'detect', you will receive a warning. e) Because the compiler is written in Perl, some of your extension scripts from earlier versions will no longer work because Shorewall-perl runs those extension scripts at compile-time rather than at run-time. Compile-time scripts are: initdone maclog All per-chain scripts including those associated with actions. Compile-time extension scripts are executed using the Perl 'eval `cat `' mechanism. Be sure that each script returns a 'true' value; otherwise, the compiler will assume that the script failed and will abort the compilation. All scripts will need to begin with the following line: use Shorewall::Chains; For more complex scripts, you may need to 'use' other Shorewall Perl modules -- browse /usr/share/shorewall-perl/Shorewall/ to see what's available. When a script is invoked, the $chainref scalar variable will hold a reference to a chain table entry. $chainref->{name} contains the name of the chain $chainref->{table} holds the table name To add a rule to the chain: add_rule( $chainref, ); Where is a scalar argument holding the rule text. Do not include "-A " Example: add_rule( $chainref, '-j ACCEPT' ); To insert a rule into the chain: insert_rule( $chainref, , ); The log_rule_limit function works like it does in the shell compiler with two exceptions: - You pass the chain reference rather than the name of the chain. - The commands are 'add' and 'insert' rather than '-A' and '-I'. - There is only a single "pass as-is to iptables" argument (so you must quote that part). Example: log_rule_limit( 'info' , $chainref , $chainref->{name}, 'DROP' , '', #Limit '' , #Log tag 'add', #Command '-p tcp' #Pass as-is ); Note that in the 'initdone' script, there is no default chain ($chainref). You can objtain a reference to a standard chain by: my $chainref = $chain_table{}{}; Example: my $chainref = $chain_table{'filter'}{'INPUT'}; The continue script is eliminated. That script was designed to allow you to add special rules during [re]start. Shorewall-perl doesn't need such rules. See http://www.shorewall.net/4.0/shorewall_extension_scripts.htm for further information about extension scripts under Shorewall-perl. f) The 'refresh' command now works like 'restart' with the following exceptions: - The refresh command is rejected if Shorewall is not running. - The refresh command only rebuilds the 'blacklst' chain. - A directory name may not be specified in the refresh command. g) The /etc/shorewall/tos file now has zone-independent SOURCE and DEST columns as do all other files except the rules and policy files. The SOURCE column may be one of the following: [all:]
[,...] [all:][:
[,...]] $FW[:
[,...]] The DEST column may be one of the following: [all:]
[,...] [all:][:
[,...]] This is a permanent change. The old zone-based rules have never worked right and this is a good time to replace them. I've tried to make the new syntax cover the most common cases without requiring change to existing files. In particular, it will handle the tos file released with Shorewall 1.4 and earlier. h) Shorewall-perl insists that ipset names begin with a letter and be composed of alphanumeric characters and underscores (_). When used in a Shorewall configuration file, the name must be preceded by a plus sign (+) as with the shell-based compiler. Shorewall-perl is now out of the ipset load/reload business. With scripts generated by the Perl-based Compiler, the Netfilter ruleset is never cleared. That means that there is no opportunity for Shorewall to load/reload your ipsets since that cannot be done while there are any current rules using ipsets. So: i) Your ipsets must be loaded before Shorewall starts. You are free to try to do that with the following code in /etc/shorewall/start: if [ "$COMMAND" = start ]; then ipset -U :all: :all: ipset -F ipset -X ipset -R < /my/ipset/contents fi The file '/my/ipset/contents' (not its real name of course) will normally be produced using the ipset -S command. The above will work most of the time but will fail in a 'shorewall stop' - 'shorewall start' sequence if you use ipsets in your routestopped file (see below). ii) Your ipsets may not be reloaded until Shorewall is stopped or cleared. iii) If you specify ipsets in your routestopped file then Shorewall must be cleared in order to reload your ipsets. As a consequence, scripts generated by the Perl-based compiler will ignore /etc/shorewall/ipsets and will issue a warning if you set SAVE_IPSETS=Yes in shorewall.conf. i) Because the configuration files (with the exception of /etc/shorewall/params) are now processed by the Perl-based compiler rather than by the shell, only the basic forms of Shell expansion ($variable and ${variable}) are supported. The more exotic forms such as ${variable:=default} are not supported. Both variables defined in /etc/shorewall/params and environmental variables (exported by the shell) can be used in configuration files. j) USE_ACTIONS=No is not supported. That option is intended to minimize Shorewall's footprint in embedded applications. As a consequence, Default Macros are not supported. k) DELAYBLACKLISTLOAD=Yes is not supported. The entire ruleset is atomically loaded with one execution of iptables-restore. l) MAPOLDACTIONS=Yes is not supported. People should have converted to using macros by now. m) The pre Shorewall-3.0 format of the zones file is not supported; neither is the /etc/shorewall/ipsec file. n) BLACKLISTNEWONLY=No is not permitted with FASTACCEPT=Yes. This combination doesn't work in previous versions of Shorewall so the Perl-based compiler simply rejects it. o) Shorewall-perl has a single rule generator that is used for all rule-oriented files. So it is important that the syntax is consistent between files. With shorewall-shell, there is a special syntax in the SOURCE column of /etc/shorewall/masq to designate "all traffic entering the firewall on this interface except...". Example: #INTERFACE SOURCE ADDRESSES eth0 eth1!192.168.4.9 ... Shorewall-perl uses syntax that is consistent with the rest of Shorewall: #INTERFACE SOURCE ADDRESSES eth0 eth1:!192.168.4.9 ... p) The 'allowoutUPnP' built-in action is no longer supported. The Netfilter team have removed support for '-m owner --owner-cmd' which that action depended on. q) The treatment of the following interface options has changed under Shorewall-perl. - arp_filter - routefilter - logmartians - proxy_arp - sourceroute With the Shorewall-shell compiler, Shorewall resets these options on all interfaces then sets the option on those interfaces for which the option is defined in /etc/shorewall/interfaces. Under Shorewall-perl, these options can be specified with the value 0 or 1 (e.g., proxy_arp=0). If no value is specified, the value 1 is assumed. Shorewall will modify only the setting of those interfaces for which the option is specified and will set the option to the given value. A fatal compilation error is also generated if you specify one of these options with a wildcard interface (one ending with '+'). r) The LOG_MARTIANS and ROUTE_FILTER options are now tri-valued in Shorewall-perl. Yes - Same as before No - Same as before except that it applies regardless of whether any interfaces have the logmartians/routefilter option Keep - Shorewall ignores the option entirely (which is the default). s) Shorewall-perl support nn 'optional' option has been added to /etc/shorewall/interfaces. This option is recognized by Shorewall-perl but not by Shorewall-shell. When 'optional' is specified for an interface, Shorewall will be silent when: - a /proc/sys/net/ipv4/conf/ entry for the interface cannot be modified (including for proxy ARP). - The first address of the interface cannot be obtained. I specify 'optional' on interfaces to Xen virtual machines that may or may not be running when Shorewall is [re]started. CAUTION: Use 'optional' at your own risk. If you [re]start Shorewall when an 'optional' interface is not available and then do a 'shorewall save', subsequent 'shorewall restore' and 'shorewall -f start' operations will instantiate a ruleset that does not support that interface, even if it is available at the time of the restore/start. t) Shorewall-perl validates all IP addresses and addresses ranges in rules. DNS names are resolved and an error is issued for any name that cannot be resolved. u) Shorewall-perl checks configuration files for the presense of characters that can cause problems if they are allowed into the generated firewall script: - Double Quotes. These are prohibited except in the shorewall.conf and params files. - Single Quotes. These are prohibited except in the shorewall.conf and params files and in COMMENT lines. - Single back quotes. These are prohibited except in the shorewall.conf and params files. - Backslash. Probibited except as the last character on a line to denote line continuation. v) Under Shorewall-perl, macros may invoke other macros with the restriction that such macros may not be invoked within an action body. When marcros are invoked recursively, the parameter passed to an invocation are automatically propagated to lower level macros. Macro invocations may be nested to a maximum level of 5. w) The PKTTYPE option is ignored by Shorewall-perl. Shorewall-perl will use Address type match if it is available; otherwise, it will behave as if PKTTYPE=No had been specified. x) Shorewall-perl detects dead policy file entries that result when an entry is masked by an earlier more general entry. Example: all all REJECT info loc net ACCEPT ------------------------------------------------------------------------ P R E R E Q U I S I T E S ------------------------------------------------------------------------ - Perl (I use Perl 5.8.8 but other versions should work fine) - Perl Cwd Module - Perl File::Basename Module - Perl File::Temp Module - Perl Getopt::Long Module ------------------------------------------------------------------------ U S I N G T H E N E W C O M P I L E R ------------------------------------------------------------------------ If you only install one compiler, then that compiler will be used. If you install both compilers, then the compiler actually used depends on the SHOREWALL_COMPILER setting in shorewall.conf. The value of this new option can be either 'perl' or 'shell'. If you add 'SHOREWALL_COMPILER=perl' to /etc/shorewall/shorewall.conf then by default, the new compiler will be used on the system. If you add it to shorewall.conf in a separate directory (such as a Shorewall-lite export directory) then the new compiler will only be used when you compile from that directory. If you only install one compiler, it is suggested that you do not set SHOREWALL_COMPILER. You can also select the compiler to use on the command line using the 'C option: '-C shell' means use the shell compiler '-C perl' means use the perl compiler The -C option overrides the setting in shorewall.conf. Example: shorewall restart -C perl 2) Thanks to Paul Gear, an IPPServer macro has been added. Be sure to read the comments in the macro file before trying to use this macro. 3) Eariler generations of Shorewall Lite required that remote root login via ssh be enabled in order to use the 'load' and 'reload' commands. Beginning with this release, you may define an alternative means for accessing the remote firewall system. Two new options have been added to shorewall.conf: RSH_COMMAND RCP_COMMAND The default values for these are as follows: RSH_COMMAND: ssh ${root}@${system} ${command} RCP_COMMAND: scp ${files} ${root}@${system}:${destination} Shell variables that will be set when the commands are envoked are as follows: root - root user. Normally 'root' but may be overridden using the '-r' option. system - The name/IP address of the remote firewall system. command - For RSH_COMMAND, the command to be executed on the firewall system. files - For RCP_COMMAND, a space-separated list of files to be copied to the remote firewall system. destination - The directory on the remote system that the files are to be copied into. 4) The accounting, masq, rules and tos files now have a 'MARK' column similar to the column of the same name in the tcrules file. This column allows filtering by MARK and CONNMARK value (CONNMARK is only accepted under Shorewall Perl). 5) SOURCE and DEST are now reserved zone names to avoid problems with bi-directional macro definitions which use these as names as key words. 6) The "shorewall show zones" command now flags zone members that have been added using "shorewall add" by preceding them with a plus sign ("+"). Example: Shorewall 3.9.4 Zones at gateway - Mon May 14 07:48:16 PDT 2007 fw (firewall) net (ipv4) eth0:0.0.0.0/0 loc (ipv4) br0:0.0.0.0/0 eth4:0.0.0.0/0 eth5:0.0.0.0/0 +eth1:0.0.0.0/0 dmz (ipv4) eth3:0.0.0.0/0 vpn (ipv4) tun+:0.0.0.0/0 In the above output, "eth1:0.0.0.0/0" was dynamically added to the 'loc' zone. As part of this change, "shorewall delete" will only delete entries that have been added dynamically. In earlier versions, any entry could be deleted although the ruleset was only changed by deleting entries that had been added dynamically. 7) The 'shorewall version' command now lists the version of the installed compiler(s) if the -a option is used: gateway:/bulk/backup # shorewall version -a 4.0.0-Beta1 Shorewall-shell 4.0.0-Beta1 Shorewall-perl 4.0.0-Beta1 gateway:/bulk/backup # 8) The Perl compiler is externalized. Both the compiler.pl program and the Perl Module interface are documented. The compiler program is /usr/share/shorewall-perl/compiler.pl: compiler.pl [