shorewall_code/docs/Shorewall-perl.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<article>
  <!--$Id: template.xml 4194 2006-07-07 01:04:16Z judas_iscariote $-->

  <articleinfo>
    <title>Shorewall-perl</title>

    <authorgroup>
      <author>
        <firstname>Tom</firstname>

        <surname>Eastep</surname>
      </author>
    </authorgroup>

    <pubdate><?dbtimestamp format="Y/m/d"?></pubdate>

    <copyright>
      <year>2007</year>

      <holder>Thomas M. Eastep</holder>
    </copyright>

    <legalnotice>
      <para>Permission is granted to copy, distribute and/or modify this
      document under the terms of the GNU Free Documentation License, Version
      1.2 or any later version published by the Free Software Foundation; with
      no Invariant Sections, with no Front-Cover, and with no Back-Cover
      Texts. A copy of the license is included in the section entitled
      <quote><ulink url="GnuCopyright.htm">GNU Free Documentation
      License</ulink></quote>.</para>
    </legalnotice>
  </articleinfo>

  <section id="What">
    <title>Shorewall-perl - What is it?</title>

    <para>Shorewall-perl is a companion product to Shorewall. It requires
    Shorewall 3.4.2 or later (Shorewall 3.4.4 or later recommended).</para>

    <para>Shorewall-perl contains a re-implementation of the Shorewall
    compiler written in Perl. The advantages of using Shorewall-perl over
    Shorewall-shell (the shell-based compiler included in earlier Shorewall
    3.x releases) are:</para>

    <itemizedlist>
      <listitem>
        <para>The Shorewall-perl compiler is much faster.</para>
      </listitem>

      <listitem>
        <para>The script generated by the compiler uses
        <command>iptables-restore</command> to instantiate the Netfilter
        configuration. So it runs much faster than the script generated by the
        Shorewall-shell compiler.</para>
      </listitem>

      <listitem>
        <para>The Shorewall-perl compiler does more thorough checking of the
        configuration than the Shorewall-shell compiler does.</para>
      </listitem>

      <listitem>
        <para>The error messages produced by the compiler are better, more
        consistent and always include the file name and line number where the
        error was detected.</para>
      </listitem>

      <listitem>
        <para>Going forward, the Shorewall-perl compiler will get all
        enhancements; the Shorewall-shell compiler will only get those
        enhancements that are easy to retrofit.</para>
      </listitem>
    </itemizedlist>
  </section>

  <section id="DownSide">
    <title>Shorewall-perl - The down side</title>

    <para>While there are advantages to using Shorewall-perl, there are also
    disadvantages:</para>

    <itemizedlist>
      <listitem>
        <para>There are a number of incompatibilities between the
        Shorewall-perl compiler and the earlier one.</para>

        <orderedlist>
          <listitem>
            <para>The Perl-based compiler requires the following capabilities
            in your kernel and iptables.</para>

            <itemizedlist>
              <listitem>
                <para>addrtype match (may be relaxed later)</para>
              </listitem>

              <listitem>
                <para>multiport match (will not be relaxed)</para>
              </listitem>
            </itemizedlist>

            <para>These capabilities are in current distributions.</para>
          </listitem>

          <listitem>
            <para>Now that Netfilter has features to deal reasonably with port
            lists, I see no reason to duplicate those features in Shorewall.
            The Shorewall-shell compiler goes to great pain (in some cases) to
            break very long port lists ( &gt; 15 where port ranges in lists
            count as two ports) into individual rules. In the new compiler,
            I'm avoiding the ugliness required to do that. The new compiler
            just generates an error if your list is too long. It will also
            produce an error if you insert a port range into a port list and
            you don't have extended multiport support.</para>
          </listitem>

          <listitem>
            <para>BRIDGING=Yes is not supported. The kernel code necessary to
            support this option was removed in Linux kernel 2.6.20. <ulink
            url="bridge-Shorewall-perl.html">Alternative bridge
            support</ulink> is provided by Shorewall-perl.</para>
          </listitem>

          <listitem>
            <para>The BROADCAST column in the interfaces file is essentially
            unused; if you enter anything in this column but '-' or 'detect',
            you will receive a warning. This will be relaxed if and when the
            addrtype match requirement is relaxed.</para>
          </listitem>

          <listitem>
            <para>The 'refresh' command is now similar to restart with the
            exceptios that:</para>

            <itemizedlist>
              <listitem>
                <para>The command fails if Shorewall is not running.</para>
              </listitem>

              <listitem>
                <para>A directory name cannot be specified in the
                command.</para>
              </listitem>

              <listitem>
                <para>The refresh command does not alter the Netfilter
                configuration except for the static blacklist.</para>
              </listitem>
            </itemizedlist>
          </listitem>

          <listitem>
            <para>With the shell-based compiler, extension scripts were copied
            into the compiled script and executed at run-time. In many cases,
            this approach doesn't work with Shorewall Perl because (almost)
            the entire ruleset is built by the compiler. As a result,
            Shorewall-perl runs many extension scripts at compile-time rather
            than at run-time. Because the compiler is written in Perl, your
            extension scripts from earlier versions will no longer
            work.</para>

            <para>The following table summarizes when the various extension
            scripts are run:<informaltable frame="all">
                <tgroup cols="3">
                  <tbody>
                    <row>
                      <entry><emphasis
                      role="bold">Compile-time</emphasis></entry>

                      <entry><emphasis role="bold">Run-time</emphasis></entry>

                      <entry><emphasis
                      role="bold">Eliminated</emphasis></entry>
                    </row>

                    <row>
                      <entry>initdone</entry>

                      <entry>clear</entry>

                      <entry>continue</entry>
                    </row>

                    <row>
                      <entry>maclog</entry>

                      <entry>initdone</entry>

                      <entry></entry>
                    </row>

                    <row>
                      <entry>Per-chain (including those associated with
                      actions)</entry>

                      <entry>start</entry>

                      <entry></entry>
                    </row>

                    <row>
                      <entry></entry>

                      <entry>started</entry>

                      <entry></entry>
                    </row>

                    <row>
                      <entry></entry>

                      <entry>stop</entry>

                      <entry></entry>
                    </row>

                    <row>
                      <entry></entry>

                      <entry>stopped</entry>

                      <entry></entry>
                    </row>

                    <row>
                      <entry></entry>

                      <entry>tcclear</entry>

                      <entry></entry>
                    </row>
                  </tbody>
                </tgroup>
              </informaltable></para>

            <para>Compile-time extension scripts are executed using the Perl
            'eval `cat &lt;file&gt;`' mechanism. Be sure that each script
            returns a 'true' value; otherwise, the compiler will assume that
            the script failed and will abort the compilation.</para>

            <para>When a script is invoked, the <emphasis
            role="bold">$chainref</emphasis> scalar variable will hold a
            reference to a chain table entry.</para>

            <simplelist>
              <member><emphasis role="bold">$chainref-&gt;{name}</emphasis>
              contains the name of the chain</member>

              <member><emphasis role="bold">$chainref-&gt;{table}</emphasis>
              holds the table name</member>
            </simplelist>

            <para>To add a rule to the chain:</para>

            <simplelist>
              <member>add_rule $chainref, &lt;<replaceable>the
              rule</replaceable>&gt;</member>
            </simplelist>

            <para>Where</para>

            <simplelist>
              <member>&lt;<replaceable>the rule</replaceable>&gt; is a scalar
              argument holding the rule text. Do not include "-A
              &lt;<replaceable>chain name</replaceable>&gt;"</member>
            </simplelist>

            <para>Example:</para>

            <simplelist>
              <member>add_rule $chainref, '-j ACCEPT';</member>
            </simplelist>

            <para>To insert a rule into the chain:</para>

            <simplelist>
              <member>insert_rule $chainref,
              &lt;<replaceable>rulenum</replaceable>&gt;, &lt;<replaceable>the
              rule</replaceable>&gt;</member>
            </simplelist>

            <para>The log_rule_limit function works like it does in the shell
            compiler with two exceptions:</para>

            <itemizedlist>
              <listitem>
                <para>You pass the chain reference rather than the name of the
                chain.</para>
              </listitem>

              <listitem>
                <para>The commands are 'add' and 'insert' rather than '-A' and
                '-I'.</para>
              </listitem>

              <listitem>
                <para>There is only a single "pass as-is to iptables" argument
                (so you must quote that part</para>
              </listitem>
            </itemizedlist>

            <para>Example:</para>

            <programlisting>    log_rule_limit
              'info' , 
              $chainref , 
              $chainref-&gt;{name},
              'DROP' , 
              '',    #Limit
              '' ,   #Log tag
              'add'
              '-p tcp ';         </programlisting>

            <para>Here is an example of an actual initdone script used with
            Shorewall 3.4:<programlisting>run_iptables -t mangle -I PREROUTING -p esp -j MARK --set-mark 0x50
run_iptables -t filter -I INPUT -p udp --dport 1701 -m mark --mark 0x50 -j ACCEPT
run_iptables -t filter -I OUTPUT -p udp --sport 1701 -j ACCEPT
</programlisting></para>

            <para>Here is the corresponding script used with
            Shorewall-perl:<programlisting>use Shorewall::Chains;

insert_rule $mangle_table-&gt;{PREROUTING}, 1, "-p esp -j MARK --set-mark 0x50";
insert_rule $filter_table-&gt;{INPUT},      1, "-p udp --dport 1701 -m mark --mark 0x50 -j ACCEPT";
insert_rule $filter_table-&gt;{OUTPUT},     1, "-p udp --sport 1701 -j ACCEPT";

1;</programlisting></para>

            <para>The initdone script is unique because the $chainref variable
            is not set before the script is called. The above script
            illustrates how the $mangle_table, $filter_table, and $nat_table
            references can be used to add or insert rules in arbitrary
            chains.</para>
          </listitem>

          <listitem>
            <para>The <filename>/etc/shorewall/tos</filename> file now has
            zone-independent SOURCE and DEST columns as do all other files
            except the rules and policy files.</para>

            <para>The SOURCE column may be one of the following:</para>

            <simplelist>
              <member>[<command>all</command>:]&lt;<replaceable>address</replaceable>&gt;[,...]</member>

              <member>[<command>all</command>:]&lt;<replaceable>interface</replaceable>&gt;[:&lt;<replaceable>address</replaceable>&gt;[,...]]</member>

              <member><command>$FW</command>[:&lt;<replaceable>address</replaceable>&gt;[,...]]</member>
            </simplelist>

            <para>The DEST column may be one of the following:</para>

            <simplelist>
              <member>[<command>all</command>:]&lt;<replaceable>address</replaceable>&gt;[,...]</member>

              <member>[<command>all</command>:]&lt;<replaceable>interface</replaceable>&gt;[:&lt;<replaceable>address</replaceable>&gt;[,...]]</member>
            </simplelist>

            <para>This is a permanent change. The old zone-based rules have
            never worked right and this is a good time to replace them. I've
            tried to make the new syntax cover the most common cases without
            requiring change to existing files. In particular, it will handle
            the tos file released with Shorewall 1.4 and earlier.</para>
          </listitem>

          <listitem>
            <para>Currently, support for ipsets is only lightly tested (any
            volunteers?). That will change with future pre-releases but one
            thing is certain -- Shorewall is now out of the ipset load/reload
            business. With scripts generated by the Perl-based Compiler, the
            Netfilter ruleset is never cleared. That means that there is no
            opportunity for Shorewall to load/reload your ipsets since that
            cannot be done while there are any current rules using
            ipsets.</para>

            <para>So:</para>

            <orderedlist numeration="upperroman">
              <listitem>
                <para>Your ipsets must be loaded before Shorewall starts. You
                are free to try to do that with the following code in
                <filename>/etc/shorewall/start</filename>:</para>

                <programlisting>if [ "$COMMAND" = start ]; then
    ipset -U :all: :all:
    ipset -F
    ipset -X
    ipset -R &lt; /etc/shorewall/ipsets
fi</programlisting>

                <para>The file <filename>/etc/shorewall/ipsets</filename> will
                normally be produced using the <command>ipset -S</command>
                command.</para>

                <para>The above will work most of the time but will fail in a
                <command>shorewall stop</command> - <command>shorewall
                start</command> sequence if you use ipsets in your
                routestopped file (see below).</para>
              </listitem>

              <listitem>
                <para>Your ipsets may not be reloaded until Shorewall is
                stopped or cleared.</para>
              </listitem>

              <listitem>
                <para>If you specify ipsets in your routestopped file then
                Shorewall must be cleared in order to reload your
                ipsets.</para>
              </listitem>
            </orderedlist>

            <para>As a consequence, scripts generated by the Perl-based
            compiler will ignore <filename>/etc/shorewall/ipsets</filename>
            and will issue a warning if you set SAVE_IPSETS=Yes in
            <filename>shorewall.conf</filename>.</para>
          </listitem>

          <listitem>
            <para>Because the configuration files (with the exception of
            <filename>/etc/shorewall/params</filename>) are now processed by
            the Shorewall-perl compiler rather than by the shell, only the
            basic forms of Shell expansion ($variable and ${variable}) are
            supported. The more exotic forms such as ${variable:=default} are
            not supported. Both variables defined in /etc/shorewall/params and
            environmental variables (exported by the shell) can be used in
            configuration files.</para>
          </listitem>

          <listitem>
            <para>USE_ACTIONS=No is not supported. That option is intended to
            minimize Shorewall's footprint in embedded applications. As a
            consequence, Default Macros are not supported.</para>
          </listitem>

          <listitem>
            <para>DELAYBLACKLISTLOAD=Yes is not supported. The entire ruleset
            is atomically loaded with one execution of
            <command>iptables-restore</command>.</para>
          </listitem>

          <listitem>
            <para>MAPOLDACTIONS=Yes is not supported. People should have
            converted to using macros by now.</para>
          </listitem>

          <listitem>
            <para>The pre Shorewall-3.0 format of the zones file is not
            supported; neither is the
            <filename>/etc/shorewall/ipsec</filename> file.</para>
          </listitem>

          <listitem>
            <para>BLACKLISTNEWONLY=No is not permitted with FASTACCEPT=Yes.
            This combination doesn't work in previous versions of Shorewall so
            the Perl-based compiler simply rejects it.</para>
          </listitem>

          <listitem>
            <para>Shorewall-perl has a single rule generator that is used for
            all rule-oriented files. So it is important that the syntax is
            consistent between files.</para>

            <para>With shorewall-shell, there is a special syntax in the
            SOURCE column of /etc/shorewall/masq to designate "all traffic
            entering the firewall on this interface except...".</para>

            <para>Example:<programlisting>#INTERFACE       SOURCE                  ADDRESSES
eth0             eth1!192.168.4.9        ...</programlisting>Shorewall-perl
            uses syntax that is consistent with the rest of
            Shorewall:<programlisting>#INTERFACE       SOURCE                  ADDRESSES
eth0             eth1:!192.168.4.9       ...</programlisting></para>
          </listitem>

          <listitem>
            <para>The 'allowoutUPnP' built-in action is no longer supported.
            In kernel 2.6.14, the Netfilter team have removed support for '-m
            owner --owner-cmd' which that action depended on.</para>
          </listitem>
        </orderedlist>
      </listitem>

      <listitem>
        <para>Shorewall-perl is dependent on Perl (see the next section) which
        has a large disk footprint. This makes Shorewall-perl less desirable
        in an embedded environment.</para>
      </listitem>
    </itemizedlist>
  </section>

  <section id="Prerequisites">
    <title>Shorewall-perl - Prerequisites</title>

    <para>In addition to Shorewall-3.4.2 or later, you need:</para>

    <itemizedlist>
      <listitem>
        <para>Perl (I use Perl 5.8.8 but other versions should work
        fine)</para>
      </listitem>

      <listitem>
        <para>Perl Cwd Module</para>
      </listitem>

      <listitem>
        <para>Perl File::Basename Module</para>
      </listitem>

      <listitem>
        <para>Perl File::Temp Module</para>
      </listitem>

      <listitem>
        <para>Perl Getopts::Long Module</para>
      </listitem>

      <listitem>
        <para>Perl Carp Module</para>
      </listitem>
    </itemizedlist>
  </section>

  <section id="Install">
    <title>Shorewall-perl - Installation</title>

    <caution>
      <para>Shorewall-perl is still part of the <ulink
      url="ReleaseModel.html">current development release</ulink>. Use it at
      your own risk.</para>
    </caution>

    <para>Either</para>

    <programlisting><command>tar -jxf shorewall-perl-4.0.0-Betax.tar.bz2</command>
<command>cd shorewall-perl-4.0.0-Betax</command>
<command>./install.sh</command></programlisting>

    <para>or</para>

    <programlisting><command>rpm -ivh shorewall-perl-4.0.0-0Betax.noarch.rpm</command></programlisting>

    <para>Note that you can also install the current Shorewall 4.0.0 Beta
    version of Shorewall. If you do that and still want access to the legacy
    shell-based compiler, you must also install the Shorewall-shell
    package.</para>
  </section>

  <section id="Using">
    <title>Using Shorewall-perl</title>

    <section id="V3.4.3">
      <title>Using Shorewall-perl under Shorewall 3.4.2 and Shorewall
      3.4.3</title>

      <para>By default, the Shorewall-shell compiler will be used.</para>

      <para>To use the Shorewall-perl compiler, add this to
      <filename>shorewall.conf</filename>:</para>

      <para>SHOREWALL_COMPILER=perl</para>

      <para>If you add this setting to
      <filename>/etc/shorewall/shorewall.conf</filename> then by default, the
      new compiler will be used on the system.</para>

      <para>Regardless of the setting of SHOREWALL_COMPILER, there is one
      change in Shorewall operation that is triggered simply by installing
      Shorewall-perl. Your params file will be processed with the shell's '-a'
      option which causes any variables that you set or create in that file to
      be automatically exported. Since the params file is processed before
      <filename>shorewall.conf</filename>, using -a insures that the settings
      of your params variables are available to the new compiler should it's
      use be specified in <filename>shorewall.conf</filename>.</para>
    </section>

    <section id="V4.0.0">
      <title>Using Shorewall-perl under Shorewall 3.4.4/4.0.0 Beta and
      later.</title>

      <para>If you only install one compiler, then that compiler will be
      used.</para>

      <para>If you install both compilers, then the compiler actually used
      depends on the SHOREWALL_COMPILER setting in
      <filename>shorewall.conf</filename>.</para>

      <para>The value of this new option can be either 'perl' or
      'shell'.</para>

      <para>If you add 'SHOREWALL_COMPILER=perl' to
      <filename>/etc/shorewall/shorewall.conf</filename> then by default, the
      new compiler will be used on the system. If you add it to
      <filename>shorewall.conf</filename> in a separate directory (such as a
      Shorewall-lite export directory) then the new compiler will only be used
      when you compile from that directory (4.0.0 Beta6 and later
      only).</para>

      <para>If you only install one compiler, it is suggested that you do not
      set SHOREWALL_COMPILER.</para>

      <para>If you install Shorewall-perl under Shorewall 3.4.4 or later, you
      can select the compiler to use on the command line using the 'C
      option:<simplelist>
          <member>'-C shell' means use the shell compiler</member>

          <member>'-C perl' means use the perl compiler</member>
        </simplelist>The -C option overrides the setting in
      shorewall.conf.</para>

      <para>Example:<programlisting><command>shorewall restart -C perl</command></programlisting></para>

      <para>When the Shorewall-perl compiler has been selected, the
      <filename>params</filename> file is processed using the
      <option>-a</option> option which causes all variables set within the
      file to be exported automatically by the shell. The Shorewall-perl
      compiler uses the current environmental variables to perform variable
      expansion within the other Shorewall configuration files.</para>
    </section>
  </section>
</article>