shorewall_code/Shorewall-common/releasenotes.txt

Shorewall 3.9.3
----------------------------------------------------------------------------
                   R E L E A S E  H I G H L I G H T S
----------------------------------------------------------------------------
1) This is the first Shorewall release that fully integrates the new
   Shorewall-perl compiler. See the "New Features" section below.

2) You are now offered a choice as to which compiler(s) you install. In
   3.9.2, there are the following packages:

   - Shorewall ( common files )
   - Shorewall-shell ( the shell-based compiler )
   - Shorewall-perl (the Perl-based compiler )

You must install Shorewall and at least one of the compiler packages
(you may install them both).

Problems corrected in 3.9.4.

1)  The value zero was being ignored in SOURCE PORT(S) and DEST PORT(S)
    columns.

2)  logmartions=0 was being treated the same as logmartians=1.

3)  The 'arp_ignore' interface option caused an internal error in
    validate_interfaces_file() and other errors.

4)  The 'detectnets' option was accepted but did nothing.

5)  Run-time errors were reported if interface or zone options were
    missing a value. 

6)  The routeback option was accepted on multi-zone interfaces; the
    documentation indicates that this option is not appropriate on such
    interfaces.

7)  'shorewall check -d' resulted in a run-time error.

8)  Intra-zone policies other than ACCEPT didn't work.

9)  When MACLIST_TABLE=filter, MAC validation produced invalid
    iptables-restore input.

Other changes in Shorewall 3.9.4

1)  The LOG_MARTIANS and ROUTE_FILTER options are not tri-valued.

	Yes -  Same as before
	No  -  Same as before except that it applies regardless of
	       whether any interfaces have the logmartians/routefilter
	       option
	Keep - Shorewall ignores the option entirely.

Migration Considerations:

1)  You cannot simply upgrade your existing Shorewall package. You must
    upgrade Shorewall *and* install one or both of the compilers.

    If you attempt to upgrade using the RPM, you get this result:

    gateway:~ # rpm -Uvh shorewall-3.9.2-1.noarch.rpm 
    error: Failed dependencies:
        shorewall_compiler is needed by shorewall-3.9.2-1.noarch
    gateway:~ #

    You must either:

	rpm -U shorewall-3.9.2.noarch.rpm shorewall-shell-3.9.2.noarch.rpm

    or

	rpm -U shorewall-3.9.2.noarch.rpm shorewall-perl-3.9.2.noarch.rpm
	
    or
	rpm -i shorewall-shell-3.9.2.noarch.rpm
	rpm -U shorewall-3.9.2.noarch.rpm

    or
	rpm -i shorewall-perl-3.9.2.noarch.rpm
	rpm -U shorewall-3.9.2.noarch.rpm

    If you are upgrading using the tarball, you must install either
    shorewall-shell or shorewall-perl before you upgrade
    Shorewall. Otherwise, the install.sh script fails with:

         ERROR: No Shorewall compiler is installed

    The shorewall-shell and shorewall-perl packages are installed from
    the tarball in the expected way; untar the package, and run the
    install.sh script.

----------------------------------------------------------------------------
                         N E W  F E A T U R E S
----------------------------------------------------------------------------
1)  Shorewall-perl

    This companion product to Shorewall 3.4.2 and later includes a complete
    rewrite of the compiler in Perl.

    I decided to make Shorewall-perl a separate product for several reasons:

    a) Embedded applications are unlikely to adopt Shorewall-perl; even
       Mini-Perl has a substantial disk and Ram footprint.

    b) Because of the gross incompatibilities between the new compiler and the
       old (see below), migration to the new compiler must be voluntary.

    c) By allowing Shorewall-perl to co-exist with the current
       Shorewall stable release (3.4), I'm hoping that the new compiler
       will get more testing and validation than it would if I were to
       package it with a new development version of Shorewall itself.

    d) Along the same vein, I think that users will be more likely to
       experiment with the new compiler if they can easily fall back to
       the old one if things get sticky.
    ------------------------------------------------------------------------
                      T H E   G O O D   N E W S:
    ------------------------------------------------------------------------
    a) The compiler has a small disk footprint.
    b) The compiler is very fast.
    c) The compiler generates a firewall script that uses iptables-restore;
       so the script is very fast.
    d) Use of the perl compiler is optional! The old slow clunky
       Bourne-shell compiler is still available.
    ------------------------------------------------------------------------
                      T H E   B A D   N E W S:
    ------------------------------------------------------------------------
    There are a number of incompatibilities between the Perl-based compiler
    and the Bourne-shell one. Some of these will probably go away by first
    official release but most will not.

    a) The Perl-based compiler requires the following capabilities in your
       kernel and iptables.

       - addrtype match (may be relaxed later)
       - multiport match (will not be relaxed)

       These capabilities are in current distributions.

    b) Now that Netfilter has features to deal reasonably with port lists,
       I see no reason to duplicate those features in Shorewall. The
       Bourne-shell compiler goes to great pain (in some cases) to
       break very long port lists ( > 15 where port ranges in lists count
       as two ports) into individual rules. In the new compiler, I'm
       avoiding the ugliness required to do that. The new compiler just
       generates an error if your list is too long. It will also produce
       an error if you insert a port range into a port list and you don't
       have extended multiport support.

    c) BRIDGING=Yes is not supported. The kernel code necessary to
       support this option was removed in Linux kernel 2.6.20.

    d) The BROADCAST column in the interfaces file is essentially unused;
       if you enter anything in this column but '-' or 'detect', you will
       receive a warning. This will be relaxed if and when the addrtype
       match requirement is relaxed.

    e) Because the compiler is now written in Perl, your compile-time
       extension scripts from earlier versions will no longer work.
       Compile-time extension scripts are executed using the Perl
       'eval `cat <file>`' mechanism. Be sure that each script returns a
       'true' value; otherwise, the compiler will assume that the script
       failed and will abort the compilation.

       When a script is invoked, the $chainref scalar variable will hold a
       reference to a chain table entry. 

           $chainref->{name}  contains the name of the chain
           $chainref->{table} holds the table name
       
           To add a rule to the chain:

	      add_rule $chainref, <the rule>

           Where

              <the rule> is a scalar argument holding the rule text. Do
              not include "-A <chain name>"

           Example:

              add_rule $chainref, '-j ACCEPT';

           To insert a rule into the chain:

              insert_rule $chainref, <rulenum>, <the rule>

           The log_rule_limit function works like it does in the shell
           compiler with two exceptions:

              - You pass the chain reference rather than the name of
                the chain.
	      - The commands are 'add' and 'insert' rather than '-A'
                and '-I'.
	      - There is only a single "pass as-is to iptables"
                argument (so you must quote that part).

           Example:

	      log_rule_limit
	          'info' , 
	          $chainref , 
	          $chainref->{name},
	          'DROP' , 
	          '',    #Limit
	          '' ,   #Log tag
	          'add';	      

    f) The 'refresh' command is now synonymous with 'restart'.

    g) Some run-time scripts will need to be changed to write their
       iptables commands to file descriptor 3 in iptables-restore
       format rather than running those commands.

	    maclog

       Details to follow.

       Some run-time scripts are simply eliminated because they no
       longer make any sense under Shorewall-perl:

	initdone  - The these two scripts assumed a model where the
	continue    chains were built in parallel. In the
		    iptables-restore model, chains are built serially
		    within tables and tables are build serially.  

	refresh   - The 'refresh' command is the same as 'restart'
	refreshed

    h) The /etc/shorewall/tos file now has zone-independent SOURCE and
       DEST columns as do all other files except the rules and policy
       files.

       The SOURCE column may be one of the following:

           [all:]<address>[,...]
           [all:]<interface>[:<address>[,...]]
           $FW[:<address>[,...]]

       The DEST column may be one of the following:
       
           [all:]<address>[,...]
           [all:]<interface>[:<address>[,...]]

       This is a permanent change. The old zone-based rules have never
       worked right and this is a good time to replace them. I've tried
       to make the new syntax cover the most common cases without
       requiring change to existing files. In particular, it will
       handle the tos file released with Shorewall 1.4 and earlier.

    i) Currently, support for ipsets is untested. That will change with
       future pre-releases but one thing is certain -- Shorewall is now
       out of the ipset load/reload business. With scripts generated by
       the Perl-based Compiler, the Netfilter ruleset is never
       cleared. That means that there is no opportunity for Shorewall
       to load/reload your ipsets since that cannot be done while there
       are any current rules using ipsets.

       So:

	i)   Your ipsets must be loaded before Shorewall starts. You
	     are free to try to do that with the following code in
	     /etc/shorewall/start:

	     if [ "$COMMAND" = start ]; then
		ipset -U :all: :all:
		ipset -F
		ipset -X
		ipset -R < /my/ipset/contents
	     fi

	     The file '/my/ipset/contents' (not its real name of
	     course) will normally be produced using the ipset -S
	     command.

	     The above will work most of the time but will fail in a
	     'shorewall stop' - 'shorewall start' sequence if you
	     use ipsets in your routestopped file (see below).

	ii)  Your ipsets may not be reloaded until Shorewall is stopped
	     or cleared.

	iii) If you specify ipsets in your routestopped file then
	     Shorewall must be cleared in order to reload your ipsets.

       As a consequence, scripts generated by the Perl-based compiler
       will ignore /etc/shorewall/ipsets and will issue a warning if
       you set SAVE_IPSETS=Yes in shorewall.conf.

    j) Because the configuration files (with the exception of
       /etc/shorewall/params) are now processed by the Perl-based
       compiler rather than by the shell, only the basic forms of Shell
       expansion ($variable and ${variable}) are supported. The more
       exotic forms such as ${variable:=default} are not
       supported. Both variables defined in /etc/shorewall/params and
       environmental variables (exported by the shell) can be used in
       configuration files.

    h) USE_ACTIONS=No is not supported. That option is intended to
    minimize Shorewall's footprint in embedded applications. As a
    consequence, Default Macros are not supported.

    i) DELAYBLACKLISTLOAD=Yes is not supported. The entire ruleset is
       atomically loaded with one execution of iptables-restore.

    j) MAPOLDACTIONS=Yes is not supported. People should have converted
    to using macros by now.

    k) The pre Shorewall-3.0 format of the zones file is not supported;
       neither is the /etc/shorewall/ipsec file.

    l) BLACKLISTNEWONLY=No is not permitted with FASTACCEPT=Yes. This
       combination doesn't work in previous versions of Shorewall so
       the Perl-based compiler simply rejects it.

2)  An 'optional' option has been added to
    /etc/shorewall/interfaces. When 'optional' is specified for an
    interface, Shorewall will be silent when:

    - a /proc/sys/net/ipv4/conf/ entry for the interface cannot be
      modified (including for proxy ARP).

    - The first address of the interface cannot be obtained.

    I specify 'optional' on interfaces to Xen virtual machines that may
    or may not be running when Shorewall is [re]started.

3)  The treatment of the following interface options has changed under
    Shorewall-perl.

    -	arp_filter
    -	routefilter
    -   logmartians
    -	proxy_arp
    -	sourceroute

    With the Shorewall-shell compiler, Shorewall resets these options
    on all interfaces then sets the option on those interfaces
    for which the option is defined in /etc/shorewall/interfaces.

    Under Shorewall-perl, these options can be specified with the value
    0 or 1 (e.g., proxy_arp=0). If no value is specified, the value 1
    is assumed. Shorewall will modify only the setting of those
    interfaces for which the option is specified and will set the
    option to the given value.

    A fatal compilation error is also generated if you specify one of
    these options with a wildcard interface (one ending with '+').

4)  Thanks to Paul Gear, an IPPServer macro has been added. Be sure to
    read the comments in the macro file before trying to use this
    macro.

----------------------------------------------------------------------------
                      P R E R E Q U I S I T E S
----------------------------------------------------------------------------
- Perl (I use Perl 5.8.8 but other versions should work fine)
- Perl Cwd Module
- Perl File::Basename Module
- Perl File::Temp Module
----------------------------------------------------------------------------
               U S I N G   T H E   N E W   C O M P I L E R
----------------------------------------------------------------------------
If you only install one compiler, then that compiler will be used.

If you install both compilers, then the compiler actually used depends
on the SHOREWALL_COMPILER setting in shorewall.conf.

The value of this new option can be either 'perl' or 'shell'.

If you add 'SHOREWALL_COMPILER=perl' to /etc/shorewall/shorewall.conf
then by default, the new compiler will be used on the system. If you
add it to shorewall.conf in a separate directory (such as a
Shorewall-lite export directory) then the new compiler will only be
used when you compile from that directory.

If you only install one compiler, it is suggested that you do not set
SHOREWALL_COMPILER.

If you install Shorewall-perl under Shorewall 3.9.2 or later, you can
select the compiler to use on the command line using the 'C option:

       '-C shell' means use the shell compiler
       '-C perl' means use the perl compiler

The -C option overrides the setting in shorewall.conf. 

Example:

	shorewall restart -C perl

Regardless of the setting of SHOREWALL_COMPILER, there is one change in
Shorewall operation that is triggered simply by installing
shorewall-perl. Your params file will be processed during compilation 
with the shell's '-a' option  which causes any variables that you set
or create in that file to be automatically exported. Since the params
file is processed before shorewall.conf, using -a insures that the
settings of your params variables are available to the new compiler
should it's use be specified in shorewall.conf.
----------------------------------------------------------------------------
                      C H A N G E  H I S T O R Y
----------------------------------------------------------------------------
Problems corrected in Shorewall 3.9.3

1)  If a rule specified a source or destination port of 0 for TCP or UDP it was 
    ignored.

    The test for the presence of a source or destination port if the protocol is 
    not specified also ignored port 0.

    Patch courtesy of Steven Springl.

2)  An entry in the USER/GROUP column no longer generates a corrupted rule.

3)  The value zero (0) is no longer ignored in the USER/GROUP column.

4)  A number of problems associated with detected addresses and routed
    networks were corrected. These problems surfaced only when the same
    interface required more than one of the following to be detected:

   - First address
   - All addresses
   - Routed networks

5)  The 'dropInvalid' built-in action now correctly generates a DROP
    rule rather than a REJECT rule.

6)  The Shorewall-perl compiler was not treating 'none' in the SOURCE
    or DEST column of the rules file correctly.

7)  The Shorewall-perl compiler did not accept 'tcp:syn' in the PROTO
    column.

8)  The Shorewall-perl compiler generated an invalid rule when 
    $FW was the SOURCE of a SAME rule (iptables/netfilter do not
    support SAME in the OUTPUT chain).

9)  When 'all' appeared in the SOURCE column and there were any NONE
    policies, then a compilation error occurred.

        ERROR: Rules may not override a NONE policy

10) The reserved zone names 'all' and 'none' were not being flagged
    when used as the name of a zone.

11) The Shorewall-perl compiler now raises an error if there is no
    firewall zone declared.

12) If 'all' appeared in the SOURCE column and an undefined zone was
    specified in the DEST column of /etc/shorewall/rules, then a Perl
    run-time diagnostic was produced.