Shorewall 3.1.5

Note to users upgrading from Shorewall 2.x or 3.0

   Most problems associated with upgrades come from two causes:

   - The user didn't read and follow the migration considerations in these
     release notes.

   - The user mis-handled the /etc/shorewall/shorewall.conf file during
     upgrade. Shorewall is designed to allow the default behavior of
     the product to evolve over time. To make this possible, the design
     assumes that you will not replace your current shorewall.conf file
     during upgrades. If you feel absolutely compelled to have the latest
     comments and options in your shorewall.conf then you must proceed
     carefully.

   While you are at it, if you have a file named /etc/shorewall/rfc1918 then
   please check that file. If it has addresses listed that are NOT in one of
   these three ranges, then please rename the file to
   /etc/shorewall/rfc1918.old.

	10.0.0.0 - 10.255.255.255
	172.16.0.0 - 172.31.255.255
	192.168.0.0 - 192.168.255.255

   Please see the "Migration Considerations" below for additional upgrade
   information.

Problems Corrected in 3.1.5

1)  Compilation has been speeded up by 10-15%.

2)  Specifying a GATEWAY IP address in /etc/shorewall/providers no longer
    causes "shorewall [re]start to fail".

Other changes in 3.1.5

1)  Synonyms have been removed from the /sbin/shorewall command set. The
    'generate' and 'reload' command have been removed. The 'reload' command
    has also been removed from the program generated by 'compile'.

2)  Scripts compiled without the -e option are now more forgiving when run
    on systems with a different Shorewall version installed. If the
    installed version meets minimum version requirements, the script will
    run -- it no longer requires an exact match.

    That having been said, the minimum version required for scripts
    compiled with 3.1.5 is 3.1.5. I plan to add new features to the library
    (/usr/share/shorewall/functions) in advance of using them in compiled
    scripts so that scripts compiled on one version of Shorewall should be
    able to run on the next several newer minor releases.

Migration Considerations:

1)  A number of macros have been split into two.  The macros affected are:
	IMAP LDAP NNTP POP3 SMTP
    Each of these macros now handles only traffic on the native (plaintext)
    port.  There is a corresponding macro with S added to the end of the
    name for the SSL version of the same protocol.  Thus each macro results
    in the insertion of only one port per invocation.

    The Web macro has not been split, but two new macros, HTTP and HTTPS have
    been created.  The Web macro is deprecated in favour of these new macros,
    and may be removed from future Shorewall releases.

    These changes have been made to ensure no unexpected ports are opened due
    to the use of macros.

2)  In previous Shorewall releases, DNAT and REDIRECT rules supported a
    special syntax for exclusion of a subnet from the effect of the rule.

    Example:

	Z2 is a subzone of Z1:

	DNAT     Z1!Z2        loc:192.168.1.4        ...

    That syntax has never worked correctly when Z2 is a dynamic zone.
    Furthermore, now that Shorewall supports exclusion lists, the capability
    is redundant since the above rule can now be written in the form:

	DNAT     Z1:!<list of exclusions>   loc:192.168.1.4   ...

    Beginning with Shorewall 3.2.0, the special exclusion syntax will no
    longer be supported.

New Features:

1)  A new 'shorewall compile' command has been added.

     shorewall compile [ -e ] [ -d <distro> ] [ <config directory> ] <script
file>

    where:

	-e                    Generates an error if the configuration uses
	                      an option that would prevent the generated
	                      script from running on a system other than
	                      where the 'compile' command is running (see
	                      additional consideration a) below).
	                      Also allows the generated script to run
	                      on a system without Shorewall installed.
	-d <distribution>     Compile the script for execution on the
	                      distribution specified by <distro>. Currently,
	                      'suse' is the only valid <distro>.

			      Note that specifying a distribution should
			      only be required if you intend to install
			      the compiled script in /etc/init.d on the
			      target system.

	                      Example:

				shorewall compile -d suse foo

			      Additional distributions are expected to be
			      supported shortly.

	<config directory>    Is an optional directory to be searched for
	                      configuration files prior to those listed
	                      in CONFIG_DIR in
			      /etc/shorewall/shorewall.conf.
	<script file>         Is the name of the output file.

    The 'compile' command processes the configuration and generates a
    script file which may then be executed (either directly or using the
    'shorewall restore' command) to configure the firewall.

    The generated script contains error checking and will terminate if an
    important command fails. Before terminating:

    a) The script will check for the existence of the restore script
       specified by the RESTOREFILE variable in shorewall.conf. If that
       restore script exists, it is executed.

    b) If the restore script doesn't exist but Shorewall appears to be
       installed on the system, an "/sbin/shorewall stop" command is executed.

    Some additional considerations:

    a) It is possible to run 'compile' on one system and then run the
       generated script on another system but there are certain
       limitations.

       1) The same version of Shorewall must be running on the remote system
          unless you use the "-e" option when you compile the script.
       2) The 'detectnets' interface option is not allowed.
       3) You must supply the file /etc/shorewall/capabilities to provide
          the compiler with knowledge of the capabilities of the system
          where the script is to be run. The /etc/shorewall/capabilities
          file included in this release includes instructions for its
          use.

    b) If you have extension scripts, they may need modification. Some of
       the scripts will be run at compile time, rather than when the
       generated script is executed. The standard functions like
       'run_iptables' and 'log_rule_limit' will write the iptables command
       to the script file rather than executing the command. As always, you
       can check $COMMAND to determine which shorewall command is being
       executed.

       Extension Scripts that are run at compile time rather than at
       run-time are:

	    - params
	    - init
	    - continue
	    - initdone
	    - start
	    - started
	    - All scripts associated with a given chain such as Action
              chains

    If you need to interject run-time code into the generated script then
    you need to write it to file descriptor 3. Here is an example of creating
    tap device tap0 and adding it to bridge xenbr0; the text will be indented
    to line up with the surrounding text:

    cat >&3 << __EOF__
    ${INDENT}if ! qt /sbin/ip link ls dev tap0; then
    ${INDENT}    /usr/sbin/openvpn --mktun --dev tap0
    ${INDENT}    /sbin/ip link set dev tap0 up
    ${INDENT}    /sbin/brctl addif xenbr0 tap0
    ${INDENT}fi

    __EOF__

    This results in the following code in the script:

    if ! qt /sbin/ip link ls dev tap0; then
        /usr/sbin/openvpn --mktun --dev tap0
        /sbin/ip link set dev tap0 up
        /sbin/brctl addif xenbr0 tap0
    fi


    (Yes -- there is an extra blank line at the end)

    If you need to expand variables in the generated text, be sure to escape
    the '$' symbol.

    Example:

    cat >&3 << __EOF__

    ${INDENT}addr=\$(ip -f inet addr show $interface 2> /dev/null | grep inet | head -n1)
    ${INDENT}if [ -n "\$addr" ]; then
    ${INDENT}    addr=\$(echo \$addr | sed 's/inet //;s/\/.*//;s/ peer.*//')
    ${INDENT}    for network in 10.0.0.0/8 176.16.0.0/12 192.168.0.0/16; do
    ${INDENT}        if in_network \$addr \$network; then
    ${INDENT}            startup_error "The 'norfc1918' option has been specified on an interface with an RFC 1918 address. Interface:$interface"
    ${INDENT}        fi
    ${INDENT}    done
    ${INDENT}fi

    __EOF__

    Compilation generates a complete program. This program is suitable for
    installation into /etc/init.d and, when generated with the "-e" option,
    can serve as your firewall on a system that doesn't even have Shorewall
    installed.

    The generated program supports the following commands:

	<program> [ -q ] [ -v ] [ -n ] start
	<program> [ -q ] [ -v ] [ -n ] stop
	<program> [ -q ] [ -v ] [ -n ] clear
	<program> [ -q ] [ -v ] [ -n ] restart
	<program> [ -q ] [ -v ] [ -n ] status
	<program> [ -q ] [ -v ] [ -n ] version

    The "shorewall start" and "shorewall restart" commands have been
    rewritten to use compilation. They both compile a temporary program
    then run it. This results in a slightly longer elapsed time than the
    similar commands required under earlier versions of Shorewall but new
    connections are blocked for a much smaller percentage of that time.

    Under Shorewall 3.1.5, "shorewall restart" takes roughly 16.5 seconds
    on my firewall:

	real    0m16.599s
	user    0m6.292s
	sys     0m9.885s

    Of the elapsed 16.5 seconds, new connections are disabled less than
    3.5 seconds. Here are some numbers for comparison:

    A) shorewall restart (Shorewall 3.0.4)

	real    0m17.540s
	user    0m5.956s
	sys     0m10.737s

    B) ./foo restart # foo created using "shorewall compile"

	real    0m3.297s
	user    0m1.444s
	sys     0m1.728s

    C) shorewall restore (Shorewall 3.0.4) # Restores from file generated by
                                           # "shorewall save"

	real    0m1.164s
	user    0m0.556s
	sys     0m0.608s

    D) shorewall restore (shorewall 3.1.5)

	real    0m1.637s
	user    0m0.728s
	sys     0m0.584s

    The time difference between B and C reflects the difference between
    "iptables-restore" and multiple executions of "iptables". The time
    difference between C and D results from the fact that the "restore"
    command in Shorewall 3.1 runs the compiled program in a way that
    turns all iptables commands into no-ops then invokes
    iptables-restore. The system is a 1.4Ghz Celeron with 512MB RAM.

    As a final part of this change, the "check" command now compiles the
    current configuration then discards the generated script. So "check"
    performs all of the same checks that compile does. Note that there is
    still no guarantee that the generated script won't encounter run-time
    errors.

2)  The /etc/shorewall/maclist file has a new column layout. The first column
    is now DISPOSITION. This column determines what to do with matching
    packets and can have the value ACCEPT or DROP (if MACLIST_TABLE=filter, it
    can also contain REJECT). This change is upward compatible so your existing
    maclist file can still be used.

    ACCEPT, DROP and REJECT may be optionally followed by a log level to
    cause the packet to be logged.

3)  Shorewall has always been very noisy (lots of messages). No more.

    You set the default level of verbosity using the VERBOSITY option in
    shorewall.conf. If you don't set it (as would be the case of you use your
    old shorewall.conf file) then VERBOSITY defaults to a value of 2 which is
    the old default. A value of 1 suppresses some of the output (like the old
    -q option did) while a value of 0 makes Shorewall almost silent. A value
    of -1 suppresses all output except warning and error messages.

    The value specified in the 3.2 shorewall.conf is 1. So you can make
    Shorewall as verbose as previously using a single -v and you can make it
    silent by using a single -q.

    If the default is set at 2, you can still make a command silent by using
    two "q"s (e.g., shorewall -qq restart).

    In summary, each "q" subtracts one from VERBOSITY while each "v" adds one
    to VERBOSITY.

    The "shorewall show log", "shorewall logwatch" and "shorewall dump"
    commands require VERBOSITY to be greater than or equal to 3 to display MAC
    addresses.This is consistent with the previous implementation which
    required a single -v to enable MAC display but means that if you set
    VERBOSITY=0 in shorewall.conf, then you will need to include -vvv in
    commands that display log records in order to have MACs displayed.