Shorewall 3.1.4

Note to users upgrading from Shorewall 2.x or 3.0

   Most problems associated with upgrades come from two causes:

   - The user didn't read and follow the migration considerations in these
     release notes.

   - The user mis-handled the /etc/shorewall/shorewall.conf file during
     upgrade. Shorewall is designed to allow the default behavior of
     the product to evolve over time. To make this possible, the design
     assumes that you will not replace your current shorewall.conf file
     during upgrades. If you feel absolutely compelled to have the latest
     comments and options in your shorewall.conf then you must proceed
     carefully.

   While you are at it, if you have a file named /etc/shorewall/rfc1918 then
   please check that file. If it has addresses listed that are NOT in one of
   these three ranges, then please rename the file to
   /etc/shorewall/rfc1918.old.

	10.0.0.0 - 10.255.255.255
	172.16.0.0 - 172.31.255.255
	192.168.0.0 - 192.168.255.255

   Please see the "Migration Considerations" below for additional upgrade
   information.

Problems Corrected in 3.1.4

1) "shorewall check" generated an error if there were entries in
   /etc/shorewall/massq.

2) Bridging now works.

3) The handling of the QUEUE target in the ESTABLISHED section has been
   corrected. Previously, the "--syn" option was being added unconditionally
   to rules with the QUEUE target with the result that no TCP packets in the
   ESTABLISHED state would be sent to QUEUE.

New Features added in 3.1.4

1) The /etc/shorewall/maclist file has a new column layout. The first column
   is now DISPOSITION. This column determines what to do with matching
   packets and can have the value ACCEPT or DROP (if MACLIST_TABLE=filter, it
   can also contain REJECT). This change is upward compatible so your existing
   maclist file can still be used.

   ACCEPT, DROP and REJECT may be optionally followed by a log level to
   cause the packet to be logged.

2) Shorewall has always been very noisy (lots of messages). No more.

   You set the default level of verbosity using the VERBOSITY option in
   shorewall.conf. If you don't set it (as would be the case of you use your
   old shorewall.conf file) then VERBOSITY defaults to a value of 2 which is
   the old default. A value of 1 suppresses some of the output (like the old
   -q option did) while a value of 0 makes Shorewall almost silent.

   The value specified in the 3.2 shorewall.conf is 1. So you can make
   Shorewall as verbose as previously using a single -v and you can make it
   silent by using a single -q.

   If the default is set at 2, you can still make a command silent by using
   two "q"s (e.g., shorewall -qq restart).

   In summary, each "q" subtracts one from VERBOSITY while each "v" adds one
   to VERBOSITY.

   The "shorewall show log", "shorewall logwatch" and "shorewall dump"
   commands require VERBOSE to be greater than or equal to 3 to display MAC
   addresses.This is consistent with the previous implementation which
   required a single -v to enable MAC display but means that if you set
   VERBOSITY=0 in shorewall.conf, then you will need to include -vvv in
   commands that display log records in order to have MACs displayed.

3) Shorewall now implements 'start' and 'restart' using a "compile and
   go" approach. See the details under the first new feature below.

4) The "-p" option to the 'compile' command is gone. Generation of a
   complete program is now the default.

Migration Considerations:

1)  A number of macros have been split into two.  The macros affected are:
	IMAP LDAP NNTP POP3 SMTP
    Each of these macros now handles only traffic on the native (plaintext)
    port.  There is a corresponding macro with S added to the end of the
    name for the SSL version of the same protocol.  Thus each macro results
    in the insertion of only one port per invocation.

    The Web macro has not been split, but two new macros, HTTP and HTTPS have
    been created.  The Web macro is deprecated in favour of these new macros,
    and may be removed from future Shorewall releases.

    These changes have been made to ensure no unexpected ports are opened due
    to the use of macros.

New Features:

1)  A new 'shorewall generate' command has been added.

     shorewall generate [ -v ] [ -q ] [ -e ] [ <config directory> ] <script
file>

    where:

	-v and -q             are described elsewhere in this document.
	-e                    Generates an error if the configuration uses
	                      an option that would prevent the generated
	                      script from running on a system other than
	                      where the 'generate' command is running (see
	                      additional consideration a) below).
	                      Also allows the generated script to run
	                      on a system without Shorewall installed.
	<config directory>    Is an optional directory to be searched for
	                      configuration files prior to those listed
	                      in CONFIG_DIR in
			      /etc/shorewall/shorewall.conf.
	<script file>         Is the name of the output file.

    The 'generate' command processes the configuration and writes a script
    file which may then be executed (either directly or using the
    'shorewall restore' command) to configure the firewall.

    'compile' is a synonym for 'generate':

      shorewall compile [ -v ] [ -q ] [ -e ] [ <config directory> ] <script file>

    The generated script contains error checking and will terminate if an
    important command fails. Before terminating:

    a) The script will check for the existence of the restore script
       specified by the RESTOREFILE variable in shorewall.conf. If that
       restore script exists, it is executed.

    b) If the restore script doesn't exist but Shorewall appears to be
       installed on the system, an "/sbin/shorewall stop" command is executed.

    Some additional considerations:

    a) It is possible to run 'generate' ('compile') on one system and then
       run the generated script on another system but there are certain
       limitations.

       1) The same version of Shorewall must be running on the remote system
          unless you use the "-e" option when you compile the script.
       2) The 'detectnets' interface option is not allowed.

    b) If you have extension scripts, they may need modification. Some of
       the scripts will be run at compile time, rather than when the
       generated script is executed. The standard functions like
       'run_iptables' and 'log_rule_limit' will write the iptables command
       to the script file rather than executing the command. As always, you
       can check $COMMAND to determine which shorewall command is being
       executed.

       Extension Scripts that are run at compile time rather than at
       run-time are:

	    - params
	    - init
	    - continue
	    - initdone
	    - start
	    - started
	    - All scripts associated with a given chain such as Action
              chains

    If you need to interject run-time code into the generated script then
    you need to write it to file descriptor 3. Here is an example of creating
    tap device tap0 and adding it to bridge xenbr0; the text will be indented
    to line up with the surrounding text:

    cat >&3 << __EOF__
    ${INDENT}if ! qt /sbin/ip link ls dev tap0; then
    ${INDENT}    /usr/sbin/openvpn --mktun --dev tap0
    ${INDENT}    /sbin/ip link set dev tap0 up
    ${INDENT}    /sbin/brctl addif xenbr0 tap0
    ${INDENT}fi

    __EOF__

    This results in the following code in the script:

    if ! qt /sbin/ip link ls dev tap0; then
        /usr/sbin/openvpn --mktun --dev tap0
        /sbin/ip link set dev tap0 up
        /sbin/brctl addif xenbr0 tap0
    fi


    (Yes -- there is an extra blank line at the end)

    If you need to expand variables in the generated text, be sure to escape
    the '$' symbol.

    Example:

    cat >&3 << __EOF__

    ${INDENT}addr=\$(ip -f inet addr show $interface 2> /dev/null | grep inet | head -n1)
    ${INDENT}if [ -n "\$addr" ]; then
    ${INDENT}    addr=\$(echo \$addr | sed 's/inet //;s/\/.*//;s/ peer.*//')
    ${INDENT}    for network in 10.0.0.0/8 176.16.0.0/12 192.168.0.0/16; do
    ${INDENT}        if in_network \$addr \$network; then
    ${INDENT}            startup_error "The 'norfc1918' option has been specified on an interface with an RFC 1918 address. Interface:$interface"
    ${INDENT}        fi
    ${INDENT}    done
    ${INDENT}fi

    __EOF__

    In addition to 'generate', a 'shorewall reload' command has been added.

	shorewall reload [ -v ] [ -q ] [ <config directory> ]

    where -v, -q and <config directory> are as above.

    The 'reload' command creates a script using 'generate' and if there are
    no errors, it then restores that script. It is equivalent to:

    if shorewall generate /var/lib/shorewall/.reload; then
        restore .reload;
    fi

    The advantage of using reload over restart is that reload results in new
    connections being dropped for a much shorter time. Here are the results
    of tests that I conducted on my own firewall:

    A) shorewall restart (Shorewall 3.0.4)

	real    0m17.540s
	user    0m5.956s
	sys     0m10.737s

    B) ./foo restart # foo created using "shorewall compile"

	real    0m3.505s
	user    0m1.332s
	sys     0m2.164s


    C) shorewall restore (Shorewall 3.0.4) # Restores from file generated by
                                           # "shorewall save"

	real    0m1.164s
	user    0m0.556s
	sys     0m0.608s

    The time difference between B and C reflects the difference between
    "iptables-restore" and multiple executions of "iptables". The system is
    a 1.4Ghz Celeron with 512MB RAM.

    Compilation generates a complete program. This program is suitable for
    installation into /etc/init.d and, when generated with the "-e" option,
    can serve as your firewall on a system that doesn't even have Shorewall
    installed.

    The program supports the following commands:

	<program> [ -q ] [ -v ] [ -n ] start
	<program> [ -q ] [ -v ] [ -n ] stop
	<program> [ -q ] [ -v ] [ -n ] clear
	<program> [ -q ] [ -v ] [ -n ] {restart|reload}
	<program> [ -q ] [ -v ] [ -n ] status
	<program> [ -q ] [ -v ] [ -n ] version

    The "shorewall start" and "shorewall restart" commands have been
    rewritten to use compilation. They both compile a temporary program
    then run it. This results in a slightly longer elapsed time than the
    similar commands required under earlier versions of Shorewall but new
    connections are blocked for a much smaller percentage of that time as
    shown by the numbers above. Under Shorewall 3.1.4, "shorewall restart"
    takes roughly 18.5 seconds on my firewall:

	real    0m18.529s
	user    0m6.240s
	sys     0m10.165s

    As a final part of this change, the "check" command now compiles the
    current configuration then discards the generated script. So "check"
    performs all of the same checks that compile does. Note that there is
    still no guarantee that the generated script won't encounter run-time
    errors.