forked from extern/shorewall_code
479 lines
18 KiB
XML
479 lines
18 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
|
|
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
|
|
<article>
|
|
<!--$Id$-->
|
|
|
|
<articleinfo>
|
|
<title>Shorewall Internals</title>
|
|
|
|
<authorgroup>
|
|
<author>
|
|
<firstname>Tom</firstname>
|
|
|
|
<surname>Eastep</surname>
|
|
</author>
|
|
</authorgroup>
|
|
|
|
<pubdate><?dbtimestamp format="Y/m/d"?></pubdate>
|
|
|
|
<copyright>
|
|
<year>2012</year>
|
|
|
|
<holder>Thomas M. Eastep</holder>
|
|
</copyright>
|
|
|
|
<legalnotice>
|
|
<para>Permission is granted to copy, distribute and/or modify this
|
|
document under the terms of the GNU Free Documentation License, Version
|
|
1.2 or any later version published by the Free Software Foundation; with
|
|
no Invariant Sections, with no Front-Cover, and with no Back-Cover
|
|
Texts. A copy of the license is included in the section entitled
|
|
<quote><ulink url="GnuCopyright.htm">GNU Free Documentation
|
|
License</ulink></quote>.</para>
|
|
</legalnotice>
|
|
</articleinfo>
|
|
|
|
<section>
|
|
<title>Introduction</title>
|
|
|
|
<para>This document provides an overview of Shorewall internals. It is
|
|
intended to ease the task of approaching the Shorewall code base by
|
|
providing a roadmap of what you will find there.</para>
|
|
|
|
<section>
|
|
<title>History</title>
|
|
|
|
<para>Shorewall was originally written entirely in Bourne Shell. The
|
|
chief advantage of this approach was that virtually any platform
|
|
supports the shell, including small embedded environments. The initial
|
|
release was in early 2001. This version ran iptables, ip, etc.
|
|
immediately after processing the corresponding configuration entry. If
|
|
an error was encountered, the firewall was stopped. For this reason, the
|
|
<filename>routestopped</filename> file had to be very simple and
|
|
foolproof.</para>
|
|
|
|
<para>In Shorewall 3.2.0 (July 2006), the implementation was changed to
|
|
use the current compile-then-execute architecture. This was
|
|
accompilished by modifying the existing code rather than writing a
|
|
compiler/generator from scratch. The resulting code was fragile and hard
|
|
to maintain. 3.2.0 also marked the introduction of
|
|
Shorewall-lite.</para>
|
|
|
|
<para>By 2007, the compiler had become unmaintainable and needed to be
|
|
rewritten. I made the decision to write the compiler in Perl and
|
|
released it as a separate Shorewall-perl packets in Shorewall 4.0.0
|
|
(July 2007). The shell-based compiler was packaged in a Shorewall-shell
|
|
package. An option (SHOREWALL_COMPILER) in shorewall.conf specified
|
|
which compiler to use. The Perl-based compiler was siginificantly
|
|
faster, and the compiled script also ran much faster thanks to its use
|
|
of iptables-restore.</para>
|
|
|
|
<para>Shorewall6 was introduced in Shorewall 4.2.4 (December
|
|
2008).</para>
|
|
|
|
<para>Support for the old Shell-based compiler was eliminated in
|
|
Shorewall 4.4.0 (July 2009).</para>
|
|
|
|
<para>Shorewall 4.5.0 (February 2012) marked the introduction of the
|
|
current architecture and packaging.</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Architecture</title>
|
|
|
|
<para>The components of the Shorewall product suite fall into five broad
|
|
categories:</para>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>Build/Install subsystem</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<orderedlist>
|
|
<listitem>
|
|
<para>Command Line Interface (CLI)</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Run-time Libraries</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Compiler</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Configuration files (including actions and macros)</para>
|
|
</listitem>
|
|
</orderedlist>
|
|
|
|
<section>
|
|
<title>Build/Install Subsystem</title>
|
|
|
|
<para>The Shorewall Build/Install subsystem packages the products for
|
|
release and installs them on an end-user's or a packager's system. It
|
|
is diagrammed in the following graphic.</para>
|
|
|
|
<graphic fileref="images/BuildInstall.png" />
|
|
|
|
<para>The build environment components are not released and are
|
|
discussed in the <ulink url="Build.html">Shorewall Build
|
|
Article</ulink>.</para>
|
|
|
|
<para>The end-user/packager environment consists of the
|
|
<filename>configure</filename> and <filename>configure.pl</filename>
|
|
programs in Shorewall-core and an <filename>install.sh</filename>
|
|
program in each product.</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>CLI</title>
|
|
|
|
<para>The CLI is written entirely in Bourne Shell so as to allow it to
|
|
run on small embedded systems within the -lite products. The CLI
|
|
programs themselves are very small; then set global variables then
|
|
call into the CLI libraries. Here's an example
|
|
(/sbin/shorewall):</para>
|
|
|
|
<programlisting>PRODUCT=shorewall
|
|
|
|
#
|
|
# This is modified by the installer when ${SHAREDIR} != /usr/share
|
|
#
|
|
. /usr/share/shorewall/shorewallrc
|
|
|
|
g_program=$PRODUCT
|
|
g_libexec="$LIBEXECDIR"
|
|
g_sharedir="$SHAREDIR"/shorewall
|
|
g_sbindir="$SBINDIR"
|
|
g_perllib="$PERLLIBDIR"
|
|
g_confdir="$CONFDIR"/shorewall
|
|
g_readrc=1
|
|
|
|
. $g_sharedir/lib.cli
|
|
|
|
shorewall_cli $@</programlisting>
|
|
|
|
<para>As you can see, it sets the PRODUCT variable, loads the
|
|
shorewallrc file, sets the global variables (all of which have names
|
|
beginning with "g_", loads <filename>lib.cli</filename>, and calls
|
|
shorewall_cli passing its own arguments.</para>
|
|
|
|
<para>There are two CLI libraries: <filename>lib.cli</filename> in
|
|
Shorewall Core and <filename>lib.cli-std </filename>in Shorewall. The
|
|
<filename>lib.cli</filename> library is always loaded by the CLI
|
|
programs; <filename>lib-cli-std</filename> is also loaded when the
|
|
product is 'shorewall' or 'shorewall6'.
|
|
<filename>lib.cli-std</filename> overloads some functions in
|
|
<filename>lib.cli</filename> and also provides logic for the
|
|
additional commands supported by the full products.</para>
|
|
|
|
<para>The CLI libraries load two additional Shell libraries from
|
|
Shorewall.core: <filename>lib.base</filename> and
|
|
<filename>lib.common</filename> (actually,
|
|
<filename>lib.base</filename> loads <filename>lib.common</filename>).
|
|
These libraries are separete from <filename>lib.cli</filename> for
|
|
both historical and practicle reasons. <filename>lib.base</filename>
|
|
(aka functions) can be loaded by application programs, although this
|
|
was more common in the early years of Shorewall. In addition to being
|
|
loaded by the CLIs, <filename>lib.common</filename> is also copied
|
|
into the generated script by the compilers.</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Run-time Libraries</title>
|
|
|
|
<para>Thare are two libraries that are copied into the generated
|
|
script by the compiler: <filename>lib.common</filename> from
|
|
Shorewall-core and <filename>lib.core</filename> from Shorewall. The
|
|
"outer block" of the generated script comes from the Shorewall file
|
|
<filename>prog.footer</filename>.</para>
|
|
</section>
|
|
|
|
<section id="Compiler">
|
|
<title>Compiler</title>
|
|
|
|
<para>With the exception of the <filename>getparams</filename> Shell
|
|
program, the compiler is written in Perl. The compiler main program is
|
|
compiler.pl from Shorewall.conf; it's run-line arguments are described
|
|
in the <ulink url="Shorewall-perl.html%23compiler.pl">Shorewall Perl
|
|
Article</ulink>. It is invoked by the <emphasis>compiler</emphasis>
|
|
function in <filename>lib.cli-std</filename>.</para>
|
|
|
|
<para>The compiler is modularized as follows:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><filename>Accounting.pm</filename> (Shorewall::Accounting).
|
|
Processes the <filename>accounting</filename> file.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Chains.pm</filename> (Shorewall::Chains). This is
|
|
the module that provides an interface to iptables/Netfilter for
|
|
the other modules. The optimizer is included in this
|
|
module.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Config.pm</filename> (Shorewall::Config). This is
|
|
a multi-purpose module that supplies several related
|
|
services:</para>
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>Error and Progress message production.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Pre-processor. Supplies all configuration file handling
|
|
including variable expansion, ?IF...?ELSE...?ENDIF processing,
|
|
INCLUDE directives and embedded Shell and Perl.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Output script file creation with functions to write into
|
|
the script. The latter functions are no-ops when the
|
|
<command>check</command> command is being executed.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Capability Detection</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Compiler.pm</filename> (Shorewall::Compiler). The
|
|
compiler() function in this module contains the top-leve of the
|
|
compiler.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>IPAddrs.pm</filename> (Shorewall::IPAddrs) - IP
|
|
Address validation and manipulation (both IPv4 and IPv6). Also
|
|
interfaces to NSS for protocol/service name resolution.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Misc.pm</filename> (Shorewall::Misc) - Provides
|
|
services that don't fit well into the other modules.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Nat.pm</filename> (Shorewall::Nat) - Handles all
|
|
nat table rules. Processes the <filename>masq</filename>,
|
|
<filename>nat</filename> and <filename>netmap</filename>
|
|
files.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Proc.pm</filename> (Shorewall::Proc) - Handles
|
|
manipulation of <filename>/proc/sys/</filename>.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Providers.pm</filename> (Shorewall::Providers) -
|
|
Handles policy routing; processes the
|
|
<filename>providers</filename> file.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Proxyarp.pm</filename> (Shorewall::Proxyarp) -
|
|
Processes the <filename>proxyarp</filename> file.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Raw.pm</filename> (Shorewall::Raw) - Handles the
|
|
raw table; processes the <filename>conntrack</filename> (formerly
|
|
<filename>notrack</filename>) file.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Rules.pm</filename> (Shorewall::Rules) - Contains
|
|
the logic for process the <filename>policy</filename> and
|
|
<filename>rules</filename> files, including
|
|
<filename>macros</filename> and
|
|
<filename>actions</filename>.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Tc.pm</filename> (Shorewall::Tc) - Handles traffic
|
|
shaping.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Tunnels.pm</filename> (Shorewall::Tunnels) -
|
|
Processes the <filename>tunnels</filename> file.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><filename>Zones.pm</filename> (Shorewall::Zones) - Processes
|
|
the <filename>zones</filename>, <filename>interfaces</filename>
|
|
and <filename>hosts</filename> files. Provides the interface to
|
|
zones and interfaces to the other modules.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
<para>Because the params file can contain arbitrary shell code, it
|
|
must be processed by a shell. The body of
|
|
<filename>getparams</filename> is as follows:</para>
|
|
|
|
<programlisting># Parameters:
|
|
#
|
|
# $1 = Path name of params file
|
|
# $2 = $CONFIG_PATH
|
|
# $3 = Address family (4 or 6)
|
|
#
|
|
if [ "$3" = 6 ]; then
|
|
PRODUCT=shorewall6
|
|
else
|
|
PRODUCT=shorewall
|
|
fi
|
|
|
|
#
|
|
# This is modified by the installer when ${SHAREDIR} != /usr/share
|
|
#
|
|
. /usr/share/shorewall/shorewallrc
|
|
|
|
g_program="$PRODUCT"
|
|
g_libexec="$LIBEXECDIR"
|
|
g_sharedir="$SHAREDIR"/shorewall
|
|
g_sbindir="$SBINDIR"
|
|
g_perllib="$PERLLIBDIR"
|
|
g_confdir="$CONFDIR/$PRODUCT"
|
|
g_readrc=1
|
|
|
|
. $g_sharedir/lib.cli
|
|
|
|
CONFIG_PATH="$2"
|
|
|
|
set -a
|
|
|
|
. $1 >&2 # Avoid spurious output on STDOUT
|
|
|
|
set +a
|
|
|
|
export -p</programlisting>
|
|
|
|
<para>The program establishes the environment of the Shorewall or
|
|
Shoreall6 CLI program since that is the environment in which the
|
|
<filename>params</filename> file has been traditionally processed. It
|
|
then sets the -<option>a</option> option so that all newly-created
|
|
variables will be exported and invokes the
|
|
<filename><filename>params</filename></filename> file. Because the
|
|
STDOUT file is a pipe back to the compiler, no spurious output must be
|
|
sent to that file; so <filename>getparams</filename> redirect
|
|
<filename>params</filename> output to STDOUT. After the script has
|
|
executed, an <command>export -p</command> command is executed to send
|
|
the contents of the environ array back to the compiler.</para>
|
|
|
|
<para>Regrettably, the various shells (and even different versions of
|
|
the same shell) produce quite different output from <command>export
|
|
-p</command>. The Perl function Shorewall::Config::getparams() detects
|
|
which species of shell was being used and stores the variable settings
|
|
into the %params hash. Variables that are also in %ENV are only stored
|
|
in %params if there value in the output from the
|
|
<filename>getparams</filename> script is different from that in
|
|
%ENV.</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Configuration Files</title>
|
|
|
|
<para>The configuration files are all well-documented. About the only
|
|
thing worth noting is that some macros and actions are duplicated in
|
|
the Shorewall and Shorewall6 packages. Because the Shorewall6 default
|
|
CONFIG_PATH looks in ${SHAREDIR}/shorewall6 before looking in
|
|
${SHARDIR_/shorewall, this allows Shorewall6 to implement
|
|
IPv6-specific handling where required.</para>
|
|
</section>
|
|
</section>
|
|
|
|
<section>
|
|
<title>The Generated Script</title>
|
|
|
|
<para>The generated script is completely self-contained so as to avoid
|
|
version dependencies between the Shorewall version used to create the
|
|
script and the version of Shorewall-common installed on the remote
|
|
firewall.</para>
|
|
|
|
<para>The operation of the generated script is illustrated in this
|
|
diagram.</para>
|
|
|
|
<graphic fileref="images/RunningScript.png" />
|
|
|
|
<para>The Netfilter ruleset is sometimes dependent on the environment
|
|
when the script runs. Dynamic IP addresses and gateways, for example,
|
|
must be detected when the script runs. As a consequence, it is the
|
|
generated script and not the compiler that creates the input for
|
|
iptables-restore. While that input could be passed to iptables-restore
|
|
in a pipe, it is written to
|
|
<filename>${VARDIR}/.iptables_restore-input</filename> so that it is
|
|
available for post-mortem analysis in the event that iptables-restore
|
|
fails. For the other utilities (ip, tc, ipset, etc), the script runs
|
|
them passing their input on the run-line.</para>
|
|
</section>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Compiler Internals</title>
|
|
|
|
<para>Because the compiler is the most complex part of the Shorewall
|
|
product suite, I've chosen to document it first. Before diving into the
|
|
details of the individual modules, lets take a look at a few general
|
|
things.</para>
|
|
|
|
<section>
|
|
<title>Modularization</title>
|
|
|
|
<para>While the compiler is modularized and uses encapsulation, it is
|
|
not object-oriented. This is due to the fact that much of the compiler
|
|
was written by manually translating the earlier Shell code.</para>
|
|
|
|
<para>Module data is not completely encapsulated. Heavily used tables,
|
|
most notably the Chain Table (%chain_table) in Shorewall::Chains is
|
|
exported for read access. Updates to module data is always
|
|
encapsulated.</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Module Initialization</title>
|
|
|
|
<para>While currently unused and untested, the Compiler modules are
|
|
designed to be able to be loaded into a parent Perl program and the
|
|
compiler executed repeatedly without unloading the modules. To
|
|
accomodate that usage scenario, variable data is not initialized at
|
|
declaration time or in an INIT block, but is rather initialized in an
|
|
<firstterm>initialize</firstterm> function. Because off of these
|
|
functions have the same name ("initialize"), they are not exported but
|
|
are rather called using a fully-qualified name (e.g.,
|
|
"Shorewall::Config::initialize").</para>
|
|
|
|
<para>Most of the the initialization functions accept arguements. Those
|
|
most common argument is the address family (4 or 6), depending on
|
|
whether an IPv4 or IPv6 firewall is being compiled. Each of the modules
|
|
that are address-family dependent have their own $family private (my)
|
|
variable.</para>
|
|
</section>
|
|
|
|
<section>
|
|
<title>Module Dependence</title>
|
|
|
|
<para>Here is the module dependency tree. To simplify the diagram,
|
|
direct dependencies are not shown where there is also a transitive
|
|
dependency.</para>
|
|
|
|
<graphic fileref="images/ModuleDepencency.png" />
|
|
</section>
|
|
|
|
<section>
|
|
<title>Config Module</title>
|
|
|
|
<para></para>
|
|
</section>
|
|
</section>
|
|
</article>
|