Shorewall-perlTomEastep200720092012Thomas M. EastepPermission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version
1.2 or any later version published by the Free Software Foundation; with
no Invariant Sections, with no Front-Cover, and with no Back-Cover
Texts. A copy of the license is included in the section entitled
GNU Free Documentation
License.Shorewall-perl - What is it?Shorewall-perl was released as a companion product to Shorewall in
Shorewall 4.0.0.Shorewall-perl contained a re-implementation of the Shorewall
compiler written in Perl. The advantages of using Shorewall-perl over
Shorewall-shell (the shell-based compiler included in earlier Shorewall
3.x releases) were:The Shorewall-perl compiler was much faster.The script generated by the compiler used
iptables-restore to instantiate the Netfilter
configuration. So it ran much faster than the script generated by the
Shorewall-shell compiler and did not stop new connections during
shorewall restart.The Shorewall-perl compiler did more thorough checking of the
configuration than the Shorewall-shell compiler did.The error messages produced by the compiler were better, more
consistent and always included the file name and line number where the
error was detected.Going forward, the Shorewall-perl compiler got all enhancements;
the Shorewall-shell compilerl only got those enhancements that were
easy to retrofit.Beginning with Shorewall 4.3.5, Shorewall-perl is an integral part
of Shorewall and support for Shorewall-shell has been discontinued.Shorewall-perl - The down sideWhile there are significant advantages to using Shorewall-perl,
there are also disadvantages.IncompatibilitiesThere are a number of incompatibilities between the Shorewall-perl
compiler and the earlier one.The Perl-based compiler requires multiport match in your
kernel and iptables.BRIDGING=Yes is not supported. The kernel code necessary to
support this option was removed in Linux kernel 2.6.20. Alternative bridge support
is provided by Shorewall-perl.DYNAMIC_ZONES=Yes is not supported in Shorewall-perl 4.2.
Use an ipset to define your
dytnamic zones. In Shorewall 4.4, dynamic zone support based on
ipsets was added to Shorewall.The BROADCAST column in the interfaces file is essentially
unused if your kernel/iptables has Address Type Match support. If
that support is present and you enter anything in this column but
'-' or 'detect', you will receive a warning.The 'refresh' command is now similar to restart with the
exceptions that:The command fails if Shorewall is not running.A directory name cannot be specified in the
command.The refresh command does not alter the Netfilter
configuration except for the static blacklist (it also refreshes
the mangle table, beginning with Shorewall 4.2.0).With the shell-based compiler, extension scripts were copied
into the compiled script and executed at run-time. In many cases,
this approach doesn't work with Shorewall Perl because (almost) the
entire rule set is built by the compiler. As a result,
Shorewall-perl runs some extension scripts at compile-time rather
than at run-time. Because the compiler is written in Perl, your
extension scripts from earlier versions will no longer work.The following table summarizes when the various extension
scripts are run:Compile-time (Must be written
in Perl)Run-timeEliminatedinitdoneclearcontinuemacloginitPer-chain (including those associated with
actions)startstartedstopstoppedtcclearCompile-time extension scripts are executed using the Perl
'eval `cat <file>`' mechanism. Be sure that each script
returns a 'true' value; otherwise, the Shorewall-perl compiler will
assume that the script failed and will abort the compilation.When a script is invoked, the $chainref scalar variable will usually hold a
reference to a chain table entry.$chainref->{name}
contains the name of the chain$chainref->{table}
holds the table nameTo add a rule to the chain:add_rule $chainref,
the-ruleWherethe rule is a scalar argument
holding the rule text. Do not include "-A
chain-name"Example:add_rule $chainref, '-j ACCEPT';To insert a rule into the chain:insert_rule $chainref, rulenum,
the-ruleThe log_rule_limit function works like it does in the shell
compiler with three exceptions:You pass the chain reference rather than the name of the
chain.The commands are 'add' and 'insert' rather than '-A' and
'-I'.There is only a single "pass as-is to iptables" argument
(so you must quote that part)Example: log_rule_limit
'info' ,
$chainref ,
$chainref->{name},
'DROP' ,
'', #Limit
'' , #Log tag
'add'
'-p tcp '; Here is an example of an actual initdone script used with
Shorewall 3.4:run_iptables -t mangle -I PREROUTING -p esp -j MARK --set-mark 0x50
run_iptables -t filter -I INPUT -p udp --dport 1701 -m mark --mark 0x50 -j ACCEPT
run_iptables -t filter -I OUTPUT -p udp --sport 1701 -j ACCEPT
Here is the corresponding script used with
Shorewall-perl:use Shorewall::Chains;
insert_rule $mangle_table->{PREROUTING}, 1, "-p esp -j MARK --set-mark 0x50";
insert_rule $filter_table->{INPUT}, 1, "-p udp --dport 1701 -m mark --mark 0x50 -j ACCEPT";
insert_rule $filter_table->{OUTPUT}, 1, "-p udp --sport 1701 -j ACCEPT";
1;The initdone script is unique because the $chainref variable
is not set before the script is called. The above script illustrates
how the $mangle_table, $filter_table, and $nat_table references can
be used to add or insert rules in arbitrary chains.The /etc/shorewall/tos file now has
zone-independent SOURCE and DEST columns as do all other files
except the rules and policy files.The SOURCE column may be one of the following:[all:]<address>[,...][all:]<interface>[:<address>[,...]]$FW[:<address>[,...]]The DEST column may be one of the following:[all:]<address>[,...][all:]<interface>[:<address>[,...]]This is a permanent change. The old zone-based rules have
never worked right and this is a good time to replace them. I've
tried to make the new syntax cover the most common cases without
requiring change to existing files. In particular, it will handle
the tos file released with Shorewall 1.4 and earlier.Shorewall-perl insists that ipset names begin with a letter
and be composed of alphanumeric characters and underscores (_). When
used in a Shorewall configuration file, the name must be preceded by
a plus sign (+) as with the shell-based compiler.From Shorewall-perl 4.0.0 - Shorewall 4.4.5, Shorewall was out
of the ipset load/reload business with the exception of ipsets used
for dynamic zones:
With scripts generated by the Perl-based Compiler, the
Netfilter rule set is never cleared. That means that there is no
opportunity for Shorewall to load/reload your ipsets since that
cannot be done while there are any current rules using
ipsets.So:Your ipsets must be loaded before Shorewall starts. You
are free to try to do that with the following code in
/etc/shorewall/init (it works for me; your mileage
may vary):if [ "$COMMAND" = start ]; then
ipset -U :all: :all:
ipset -U :all: :default:
ipset -F
ipset -X
ipset -R < /etc/shorewall/ipsets
fiThe file /etc/shorewall/ipsets will
normally be produced using the ipset -S
command. I have this in my
/etc/shorewall/stop file:if ipset -S > /etc/shorewall/ipsets.tmp; then
mv -f /etc/shorewall/ipsets /etc/shorewall/ipsets.bak
mv /etc/shorewall/ipsets.tmp /etc/shorewall/ipsets
fiThe above extension scripts will work most of the time
but will fail in a shorewall stop -
shorewall start sequence if you use ipsets
in your routestopped file (see below).Your ipsets may not be reloaded until Shorewall is
stopped or cleared.If you specify ipsets in your routestopped file then
Shorewall must be cleared in order to reload your
ipsets.As a consequence, scripts generated by the Perl-based
compiler will ignore /etc/shorewall/ipsets
and will issue a warning if you set SAVE_IPSETS=Yes in
shorewall.conf.
Beginning with Shorewall 4.4.6 (and 4.5.3), SAVE_IPSETS=Yes is
once again supported. See shorewall.conf(5).Because the configuration files (with the exception of
/etc/shorewall/params) are now processed by the
Shorewall-perl compiler rather than by the shell, only the basic
forms of Shell expansion ($variable and ${variable}) are supported.
The more exotic forms such as ${variable:=default} are not
supported. Both variables defined in /etc/shorewall/params and
environmental variables (exported by the shell) can be used in
configuration files.USE_ACTIONS=No is not supported. That option is intended to
minimize Shorewall's footprint in embedded applications. As a
consequence, Default Macros are not supported by
Shorewall-perl.DELAYBLACKLISTLOAD=Yes is not supported. The entire rule set
is atomically loaded with one execution of
iptables-restore.MAPOLDACTIONS=Yes is not supported. People should have
converted to using macros by now.The pre Shorewall-3.0 format of the zones file is not
supported (IPSECFILE=ipsec); neither is the
/etc/shorewall/ipsec file.BLACKLISTNEWONLY=No is not permitted with FASTACCEPT=Yes. This
combination doesn't work in previous versions of Shorewall so the
Perl-based compiler simply rejects it.Shorewall-perl has a single rule generator that is used for
all rule-oriented files. This implementation enforces consistency of
syntax between files.With shorewall-shell, there is a special syntax in the SOURCE
column of /etc/shorewall/masq to designate "all traffic entering the
firewall on this interface except...".Example:#INTERFACE SOURCE ADDRESSES
eth0 eth1!192.168.4.9 ...Shorewall-perl
uses syntax that is consistent with the rest of
Shorewall:#INTERFACE SOURCE ADDRESSES
eth0 eth1:!192.168.4.9 ...The 'allowoutUPnP' built-in action is no longer supported. In
kernel 2.6.14, the Netfilter team have removed support for '-m owner
--owner-cmd' which that action depended on.The PKTTYPE option is ignored by Shorewall-perl.
Shorewall-perl will use Address type match if it is available;
otherwise, it will behave as if PKTTYPE=No had been
specified.Shorewall-perl detects dead policy file entries that result
when an entry is masked by an earlier more general entry.Example:#SOURCE DEST POLICY LOG LEVEL
all all REJECT info
loc net ACCEPTShorewall-shell silently accepts the above even though the
loc->net policy is useless. Shorewall-perl generates a fatal
compilation error.In the SOURCE column of the rules file, when an interface name
is followed by a list of IP addresses, the behavior of
Shorewall-perl differs from that of Shorewall-shell.Example:#ACTION SOURCE DEST PROTO DEST
# PORT(S)
ACCEPT loc:eth0:192.168.1.3,192.168.1.5 $FW tcp 22With
Shorewall-shell, this rule accepts SSH connection to the firewall
from 192.168.1.3 through eth0 or from 192.168.1.5 through any
interface. With Shorewall-perl, the rule accepts SSH connections
through eth0 from 192.168.1.3 and through eth0 from 192.168.1.5.
Shorewall-shell supports this syntax that gives the same result as
Shorewall-perl.#ACTION SOURCE DEST PROTO DEST
# PORT(S)
ACCEPT loc:eth0:192.168.1.3,eth0:192.168.1.5 $fw tcp 22
Shorewall-perl does not support this alternative syntax.Shorewall-perl gives a warning if a zone name is entered in
the DEST column of a nonat rule. Nonat rules
include:DNAT-REDIRECT-NONATSo rather than this:#ACTION SOURCE DEST PROTO DEST PORT(S)
DNAT- net loc:192.168.1.3 tcp 21you instead want:#ACTION SOURCE DEST PROTO DEST PORT(S)
DNAT- net 192.168.1.3 tcp 21Supplying an interface name in the SOURCE column of
/etc/shorewall/masq is deprecated as of Shorewall 4.4. Entering the
name of an interface there will result in a compile-time
warning:WARNING: Using an interface as the masq SOURCE requires the
interface to be up and configured when Shorewall
starts/restartsTo avoid this warning, replace interface names by the
corresponding network() in CIDR format (e.g.,
192.168.144.0/24).Dependence on PerlShorewall-perl is dependent on Perl which has a large disk
footprint. This makes Shorewall-perl less desirable in an embedded
environment. The best way to work around this limitation is to install
Shorewall-perl on an administrative system and employ Shorewall-lite on
your embedded systems. Shorewall-perl will run on Windows under Cygwin and on an Apple MacIntosh running OS X.
Install from a shell prompt using the
install.sh script.Installing Shorewall Version 4.0 or 4.2Shorewall 4.2 contains six packages, four of which are also included
in Shorewall 4.0:Shorewall-shell - the old
shell-based compiler and related components.Shorewall-perl - the new
Perl-based compiler.Shorewall-common - the part of
Shorewall common to both compilers.Shorewall-lite- same as the 3.4
version of Shorewall Lite. Can run scripts generated by either
Shorewall-perl or Shorewall-shell.Shorewall6 - The utilities for
creating and operating an Ipv6 firewall. Requires Shorewall-perl and
Shorewall-common. Introduced in Shorewall 4.2.4.Shorewall6-lite - Ipv6
equivalent of Shorewall Lite. Can run scripts generated by
Shoreall-perl 4.2.4 and later.If you upgrade to Shorewall Version 4.0 or 4.2, you must install
Shorewall-shell and/or Shorewall-perl; in fact, if you are using the
tarball for your installation, you must install Shorewall-shell and/or
Shorewall-perl before you upgrade
Shorewall. See the upgrade issues
for details.Compiler Selection (Shorewall 4.0-4.2)If you only install one compiler, then that compiler will be
used.If you install both compilers, then the compiler actually used for
IPv4 depends on the SHOREWALL_COMPILER setting in
shorewall.conf.The value of this new option can be either 'perl' or 'shell'.If you add 'SHOREWALL_COMPILER=perl' to
/etc/shorewall/shorewall.conf then by default, the
new compiler will be used on the system. If you add it to
shorewall.conf in a separate directory (such as a
Shorewall-lite export directory) then the new compiler will only be used
when you compile from that directory.If you only install one compiler, it is suggested that you do not
set SHOREWALL_COMPILER.If both compilers are installed, you can select the compiler to use
on the command line using the 'C option:'-C shell' means use the shell compiler'-C perl' means use the perl compilerThe -C option overrides the setting in
shorewall.conf.Example:shorewall restart -C perlWhen the Shorewall-perl compiler has been selected, the
params file is processed using the shell
option which causes all variables set within the file
to be exported automatically by the shell. The Shorewall-perl compiler
uses the current environmental variables to perform variable expansion
within the other Shorewall configuration files.The Shorewall Perl ModulesIn Shorewall 4.4 and later, Shorewall's Perl modules are installed
by default in /usr/share/shorewall/Shorewall and the names of the packages
are of the form Shorewall::name. So by using this
directiveuse lib '/usr/share/shorewall';You can then load the modules via normal Perl use statements./usr/share/shorewall/compiler.plWhile the compiler is normally run indirectly using
/sbin/shorewall, it can be run directly as well.compiler.pl [ option ... ] [ filename ]If a filename is given, then the
configuration will be compiled and the output placed in the named file.
If filename is not given, then the configuration
will simply be syntax checked.Options are:-v<verbosity>--verbosity=<verbosity>The <verbosity> is a number
between 0 and 2 and corresponds to the VERBOSITY setting in
shorewall.conf. This setting controls the verbosity
of the compiler itself.The VERBOSITY setting in the
shorewall.conf file read by the compiler will
determine the default verbosity for the compiled program.-e--exportIf given, the configuration will be compiled for export
to another system.-d
<directory>--directory=<directory>If this option is omitted, the configuration in
/etc/shorewall is compiled/checked. Otherwise, the configuration in the
named directory will be compiled/checked.-t--timestampIf given, each progress message issued by the compiler
and by the compiled program will be timestamped.--debugIf given, when a warning or error message is issued, it
is supplemented with a stack trace. Requires the Carp Perl
module.--refresh=<chainlist>If given, the compiled script's 'refresh' command will
refresh the chains in the comma-separated
<chainlist> rather than
'blacklst'.--log=<logfile>If given, compiler will log to this file provider that
--log_verbosity is > -1.--log_verbosity=-1|0|1|2If given, controls the verbosity of logging to the log specified
by the --log parameter.-f=4|6--family=4|6Specifies whether an IPv4 or an IPv6 firewall is to be
created.--previewAdded in Shorewall 4.4.6. If no filename is given, this option
causes the generated input to iptables-input to be displayed on standard
output.-c--confessAdded in Shorewall 4.4.20. Causes error and warning messages to
include a Perl stack trace. Useful for finding the place in the code
where the message is generated.-u--updateAdded in Shorewall 4.4.21. Causes the .conf file to be updated
(shorewall update command).-a--annotateAdded in Shorewall 4.4.21. Causes the updated .conf file to be
annotated with documentation. Ignored unless --update is also specified.--convertAdded in Shorewall 4.4.26. Causes the compiler to convert an
existing blacklist file into an
equivalent blrules
file.--config_path=path[:path]...Added in Shorewall 4.4.26. Search path for configuration
files.--shorewallrc=pathAdded in Shorewall 4.5.1. Specifies the location of the
shorewallrc file.Example (compiles the configuration in the current directory
generating a script named 'firewall' and using VERBOSITY
2)./usr/share/shorewall/compiler.pl -v 2 -d . firewallPrior to Shorewall 4.4.19, the Perl-based compiler did not
process /etc/shorewall/params. To include
definitions in that file, you would need to do something like the
following:. /usr/share/shorewall/lib.base # In case /etc/shorewall/params does INCLUDE
set -a # Export all variables set in /etc/shorewall/params
. /etc/shorewall/params
set +a
/usr/share/compiler.pl ...Shorewall::CompilerTo avoid a proliferation of parameters to
Shorewall::Compiler::compile(), that function uses named parameters.
Parameter names are:script ('object' is also accepted but deprecated)Output script file. If omitted or '', the configuration is
syntax checked.directoryDirectory. If omitted or '', configuration files are located
using CONFIG_PATH. Otherwise, the directory named by this
parameter is searched first.verbosityVerbosity; range -1 to 2timestamp0|1 -- timestamp messages.debug0|1 -- include stack trace in warning/error messages.export0|1 -- compile for export.chainsList of chains to be reloaded by 'refresh'logFile to log compiler messages to.log_verbosityLog Verbosity; range -1 to 2.familyAddress family: 4 or 6preview0 or 1. Added in Shorewall 4.4.6. A value of 1 causes the
generated ruleset to be printed to standard output.confess0 or 1. Added in Shorewall 4.4.20. A value of 1 causes error
and warning messages to include a Perl stack trace. Useful for
finding the place in the code where the message is
generated.update0 or 1. Added in Shorewall 4.4.21. A value of 1 causes the
.conf file to be updated (shorewall update
command).config_pathList of pathnames separated by ':' or ''. Added in Shorewall
4.4.26. The list of directories where the compiler is to search
for input files.shorewallrcPathname of the shorewallrc file. Added in Shorewall
4.5.1.Those parameters that are supplied must have defined values.
Defaults are: script: '' ('check' command)directory: ''verbosity: 1timestamp: 0debug: 0export: 0chains: ''log: ''log_verbosity: -1family: 4preview: 0confess: 0update: 0config_path: Contents of
/var/lib/shorewall/configpath or
/var/lib/shorewall6/configpath depending on the
setting of family.shorewallrc: ''.Example: use lib '/usr/share/shorewall/';
use Shorewall::Compiler;
compiler( script => '/root/firewall', log => '/root/compile.log', log_verbosity => 2 ); Shorewall::Chainsuse lib '/usr/share/shorewall';
use Shorewall::Chains;
my $chainref1 = chain_new $table, $name1;
add_rule $chainref1, $rule;
insert_rule $chainref1, $ordinal, $rule;
my $chainref2 = new_manual_chain $name3;
my $chainref3 = ensure_manual_chain $name;
log_rule_limit $level, $chainref3, $name, $disposition, $limit, $tag, $command, $predicates;
my $chainref4 = $chain_table{$table}{$name};
my $chainref5 = $nat_table{$name};
my $chainref6 = $mangle_table{$name};
my $chainref7 = $filter_table{$name};Shorewall::Chains is
Shorewall-perl's interface to iptables/netfilter. It creates a
chain table (%chain_table) which is populated as
the various tables are processed. The table (actually a hash) is
two-dimensional with the first dimension being the Netfilter table name
(raw, mangle, nat and filter) and the second dimension being the chain
name. Each table is a hash reference -- the hash defines the attributes
of the chain. See the large comment at the beginning of the module
(/usr/share/shorewall-perl/Shorewall/Chains.pm).The module export the chain table along with three hash references
into the table:$nat_tableReference to the 'nat' portion of the table
($chain_table{nat}). This is a hash whose key is the chain name.
This variable is not set when an IPv6 firewall is being
created.$mangle_tableReference to the 'mangle' portion of the table
($chain_table{mangle}). This is a hash whose key is the chain
name.$filterReference to the 'filter' portion of the table
($chain_table{filter}). This is a hash whose key is the chain
name.You can create a new chain in any of the tables using new_chain(). Arguments to the function
are:$table'nat', 'mangle', or 'filter'.$nameName of the chain to create.The function creates a hash at $chain_table{$table}{$name} and
populates the hash with default values. A reference to the hash is
returned.Each chain table entry includes a list of rules to be added to the
chain. These rules are written to the iptables-restore input file when
the resulting script is executed. To append a rule to that list, call
add_rule(). Arguments are:$chainrefA reference to the chain table entry.$ruleThe rule to add. Do not include the leading '-A ' in this
argument -- it will be supplied by the function.To insert a rule into that list, call insert_rule(). Arguments are:$chainrefA reference to the chain table entry.$ordinalThe position of the inserted rule in the list. A value of 1
inserts the rule at the head of the list, a value of 2 places the
rule second in the list, and so on.$ruleThe rule to add. Do not include the leading '-I' in this
argument -- it will be supplied by the function.To create a manual chain,
use the new_manual_chain() function. The function accepts a single
argument which is the name of the chain. The function returns a
reference to the resulting chain table entry.A companion function, ensure_manual_chain(), can be called when a
manual chain of the desired name may have already been created. If a
manual chain table entry with the passed name already exists, a
reference to the chain table entry is returned. Otherwise, the function
calls new_manual_chain() and returns
the result.To create a logging rule, call log_rule_limit(). Arguments are:$levelThe log level. May be specified as a name or as a
number.$chainrefChain table reference for the chain to which the rule is to
be added.$nameThe chain name to be reported in the log message (see
LOGFORMAT in shorewall.conf(5)).$dispositionThe disposition to be reported in the log message (see
LOGFORMAT in shorewall.conf(5)).$limitRate limiting match. If an empty string is passed, the
LOGRATE/LOGBURST (shorewall.conf(5)) is
used.$tagLog tag.$commandIf 'add', append the log rule to the chain. If 'insert',
then insert the rule at the beginning of the chain.$predicatesAny additional matches that are to be applied to the
rule.Shorewall::Configuse lib '/usr/share/shorewall';
use Shorewall::Config;
warning message "This entry is bogus";
fatal_error "You have made an error";
progress_message "This will only be seen if VERBOSITY >= 2";
progress_message2 "This will only be seen if VERBOSITY >= 1";
progress_message3 "This will be seen unless VERBOSITY < 0";
The shorewall() function may
be optionally included:use lib '/usr/share/shorewall';
use Shorewall::Config qw/shorewall/;
shorewall $config_file_entry;The Shorewall::Config module
provides basic services to Shorewall-perl. By default, it exports the
functions that produce progress messages and warning/error
messages.To issue a warning message, call warning_message(). The single argument describes
the warning.To raise a fatal error, call fatal_error(). Again, the single argument
described the error.In both cases, the function will augment the warning/error with
the current configuration file and line number, if any. fatal_error() raised an exception via either
confess() or die(), depending on whether the debugging stack
trace is enabled or not..The three 'progress message' functions conditionally produce
output depending on the current verbosity setting.The shorewall() function is used
by embedded Perl
scripts to generate entries to be included in the current
configuration file.