shorewall_code/docs/shorewall_logging.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<article id="IPIP">
  <!--$Id$-->

  <articleinfo>
    <title>Shorewall Logging</title>

    <authorgroup>
      <author>
        <firstname>Tom</firstname>

        <surname>Eastep</surname>
      </author>

      <author>
        <surname>Bill Shirley</surname>
      </author>
    </authorgroup>

    <pubdate><?dbtimestamp format="Y/m/d"?></pubdate>

    <copyright>
      <year>2001 - 2016</year>

      <holder>Thomas M. Eastep</holder>

      <holder>Bill Shirley</holder>
    </copyright>

    <legalnotice>
      <para>Permission is granted to copy, distribute and/or modify this
      document under the terms of the GNU Free Documentation License, Version
      1.2 or any later version published by the Free Software Foundation; with
      no Invariant Sections, no Front-Cover Texts, and no Back-Cover
      Texts. A copy of the license is included in the section entitled
      <quote><ulink url="GnuCopyright.htm">GNU Free Documentation
      License</ulink></quote>.</para>
    </legalnotice>
  </articleinfo>

  <caution>
    <para><emphasis role="bold">This article applies to Shorewall 4.3 and
    later. If you are running a version of Shorewall earlier than Shorewall
    4.3.5 then please see the documentation for that
    release.</emphasis></para>
  </caution>

  <section id="Log">
    <title>How to Log Traffic Through a Shorewall Firewall</title>

    <para>The disposition of packets entering a Shorewall firewall is
    determined by one of a number of Shorewall facilities. Only some of these
    facilities permit logging.</para>

    <orderedlist>
      <listitem>
        <para>The packet is part of an established connection. While the
        packet can be logged using LOG rules in the ESTABLISHED section of
        <ulink
        url="manpages/shorewall-rules.html">/etc/shorewall/rules</ulink>, that
        is not recommended because of the large amount of information that may
        be logged.</para>
      </listitem>

      <listitem>
        <para>The packet represents a connection request that is related to an
        established connection (such as a <ulink url="FTP.html">data
        connection associated with an FTP control connection</ulink>). These
        packets may be logged using LOG rules in the RELATED section of <ulink
        url="manpages/shorewall-rules.html">/etc/shorewall/rules</ulink>.</para>
      </listitem>

      <listitem>
        <para>The packet is rejected because of an option in <ulink
        url="manpages/shorewall.conf.html">/etc/shorewall/shorewall.conf</ulink>
        or <ulink
        url="manpages/shorewall-interfaces.html">/etc/shorewall/interfaces</ulink>.
        These packets can be logged by setting the appropriate logging-related
        option in <ulink
        url="manpages/shorewall.conf.html">/etc/shorewall/shorewall.conf</ulink>.</para>
      </listitem>

      <listitem>
        <para>The packet matches a rule in <ulink
        url="manpages/shorewall-rules.html">/etc/shorewall/rules</ulink>. By
        including a syslog level (see below) in the ACTION column of a rule
        (e.g., <quote>ACCEPT<emphasis role="bold">:info</emphasis> net $FW tcp
        22</quote>), the connection attempt will be logged at that
        level.</para>
      </listitem>

      <listitem>
        <para>The packet doesn't match a rule so it is handled by a policy
        defined in <ulink
        url="manpages/shorewall-policy.html">/etc/shorewall/policy</ulink>.
        These may be logged by specifying a syslog level in the LOG LEVEL
        column of the policy's entry (e.g., <quote>loc net ACCEPT <emphasis
        role="bold">info</emphasis></quote>).</para>
      </listitem>
    </orderedlist>
  </section>

  <section id="Where">
    <title>Where the Traffic is Logged and How to Change the
    Destination</title>

    <para>By default, Shorewall directs Netfilter to log using syslog (8).
    Syslog classifies log messages by a <emphasis>facility</emphasis> and a
    <emphasis>priority</emphasis> (using the notation
    <emphasis>facility.priority</emphasis>).</para>

    <para>The facilities defined by syslog are <emphasis>auth, authpriv, cron,
    daemon, kern, lpr, mail, mark, news, syslog, user, uucp</emphasis> and
    <emphasis>local0</emphasis> through <emphasis>local7.</emphasis></para>

    <para>Throughout the Shorewall documentation, I will use the term
    <emphasis>level</emphasis> rather than <emphasis>priority </emphasis>since
    <emphasis>level</emphasis> is the term used by Netfilter. The syslog
    documentation uses the term <emphasis>priority</emphasis>.</para>

    <section id="Levels">
      <title>Syslog Levels</title>

      <para>Syslog levels are a method of describing to syslog (8) the
      importance of a message. A number of Shorewall parameters have a syslog
      level as their value.</para>

      <para>Valid levels are:</para>

      <simplelist>
        <member>7 - <emphasis role="bold">debug</emphasis> (Debug-level
        messages)</member>

        <member>6 - <emphasis role="bold">info</emphasis>
        (Informational)</member>

        <member>5 - <emphasis role="bold">notice</emphasis> (Normal but
        significant Condition)</member>

        <member>4 - <emphasis role="bold">warning</emphasis> (Warning
        Condition)</member>

        <member>3 - <emphasis role="bold">err</emphasis> (Error
        Condition)</member>

        <member>2 - <emphasis role="bold">crit</emphasis> (Critical
        Conditions)</member>

        <member>1 - <emphasis role="bold">alert</emphasis> (must be handled
        immediately)</member>

        <member>0 - <emphasis role="bold">emerg</emphasis> (System is
        unusable)</member>
      </simplelist>

      <para>For most Shorewall logging, a level of 6 (info) is appropriate.
      Shorewall log messages are generated by Netfilter and are logged using
      the <emphasis>kern</emphasis> facility and the level that you specify.
      If you are unsure of the level to choose, 6 (info) is a safe bet. You
      may specify levels by name or by number.</para>

      <para>Beginning with Shorewall 4.5.5, the
      <replaceable>level</replaceable> name or number may be optionally
      followed by a comma-separated list of one or more<replaceable> log
      options</replaceable>. The list is enclosed in parentheses. Log options
      cause additional information to be included in each log message.</para>

      <para>Valid log options are:</para>

      <variablelist>
        <varlistentry>
          <term><emphasis role="bold">ip_options</emphasis></term>

          <listitem>
            <para>Log messages will include the option settings from the IP
            header.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><emphasis role="bold">macdecode</emphasis></term>

          <listitem>
            <para>Decode the MAC address and protocol.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><emphasis role="bold">tcp_sequence</emphasis></term>

          <listitem>
            <para>Include TCP sequence numbers.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><emphasis role="bold">tcp_options</emphasis></term>

          <listitem>
            <para>Include options from the TCP header.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><emphasis role="bold">uid</emphasis></term>

          <listitem>
            <para>Include the UID of the sending program; only valid for
            packets originating on the firewall itself.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <para>Example: <emphasis
      role="bold">info(tcp_options,tcp_sequence)</emphasis></para>

      <para>Syslogd writes log messages to files (typically in <filename
      class="directory">/var/log/</filename>*) based on their facility and
      level. The mapping of these facility/level pairs to log files is done in
      /etc/syslog.conf (5). If you make changes to this file, you must restart
      syslogd before the changes can take effect.</para>

      <para>Syslog may also write to your system console. See <ulink
      url="FAQ.htm#faq16">Shorewall FAQ 16</ulink> for ways to avoid having
      Shorewall messages written to the console.</para>
    </section>

    <section id="ULOG">
      <title>Configuring a Separate Log for Shorewall Messages (ulogd)</title>

      <para>There are a couple of limitations to syslogd-based logging:</para>

      <orderedlist>
        <listitem>
          <para>If you give, for example, kern.info its own log destination
          then that destination will also receive all kernel messages of
          levels 5 (notice) through 0 (emerg).</para>
        </listitem>

        <listitem>
          <para>All kernel.info messages will go to that destination and not
          just those from Netfilter.</para>
        </listitem>
      </orderedlist>

      <para>If your kernel has NFLOG target support (and most vendor-supplied
      kernels do), you may also specify a log level of NFLOG (must be all
      caps). When NFLOG is used, Shorewall will direct Netfilter to log the
      related messages via the NFLOG target which will send them to a process
      called <quote>ulogd</quote>. The ulogd program is included in most
      distributions and is also available from <ulink
      url="http://www.netfilter.org/projects/ulogd/index.html">http://www.netfilter.org/projects/ulogd/index.html</ulink>.
      Ulogd can be configured to log all Shorewall messages to their own log
      file.</para>

      <note>
        <para>The NFLOG logging mechanism is <emphasis
        role="underline">completely separate</emphasis> from syslog. Once you
        switch to NFLOG, the settings in <filename>/etc/syslog.conf</filename>
        have absolutely no effect on your Shorewall logging (except for
        Shorewall status messages which still go to syslog).</para>
      </note>

      <para>You will need to change all instances of log levels (usually
      <quote>info</quote>) in your Shorewall configuration files to
      <quote>NFLOG</quote> - this includes entries in the policy, rules and
      shorewall.conf files. If you initially installed using Shorewall 5.1.2
      or later, you can simply change the setting of LOG_LEVEL in
      shorewall.conf.</para>

      <para>Otherwise, you must search for all instances of log levels in your
      configuration and change them accordingly. If you currently run
      Shorewall 5.1.2 or later, then change them to $LOG_LEVEL and set
      LOG_LEVEL accordingly. If you are running an earlier release, using a
      shell variable simplifies future changes. Here's what I had at one
      time:</para>

      <programlisting>gateway:/etc/shorewall# grep -v ^\# * | egrep '\$LOG|ULOG|LOGFILE'
params:LOG=NFLOG
policy:loc              $FW             REJECT          $LOG
policy:net              all             DROP            $LOG            10/sec:40
policy:all              all             REJECT          $LOG
rules:REJECT:$LOG       loc                             net                     tcp     25
rules:REJECT:$LOG       loc                             net                     udp     1025:1031
rules:REJECT:$LOG       dmz                             net                     udp     1025:1031
rules:ACCEPT:$LOG       dmz                             net                     tcp     1024:                                   20
rules:REJECT:$LOG       $FW                             net                     udp     1025:1031
shorewall.conf:LOGFILE=/var/log/shorewall
shorewall.conf:LOGUNCLEAN=$LOG
shorewall.conf:MACLIST_LOG_LEVEL=$LOG
shorewall.conf:TCP_FLAGS_LOG_LEVEL=$LOG
shorewall.conf:RFC1918_LOG_LEVEL=$LOG
gateway:/etc/shorewall#                                               </programlisting>

      <para>Finally edit <filename>/etc/shorewall/shorewall.conf</filename>
      and set LOGFILE=&lt;<emphasis>file that you wish to log
      to</emphasis>&gt;. This tells the <filename>/sbin/shorewall</filename>
      program where to look for the log when processing its
      <quote><command>show log</command></quote>,
      <quote><command>logwatch</command></quote> and
      <quote><command>dump</command></quote> commands.</para>

      <para>The NFLOG target is a successor to ULOG. Both ULOG and NFLOG may
      be followed by a list of up to three numbers in parentheses.</para>

      <itemizedlist>
        <listitem>
          <para>The first number specifies the netlink group (0-65535). If
          omitted (e.g., NFLOG(,0,10)) then a value of 0 is assumed.</para>
        </listitem>

        <listitem>
          <para>The second number specifies the maximum number of bytes to
          copy. If omitted, 0 (no limit) is assumed.</para>
        </listitem>

        <listitem>
          <para>The third number specifies the number of log messages that
          should be buffered in the kernel before they are sent to user space.
          The default is 1.</para>
        </listitem>
      </itemizedlist>

      <para>Examples:</para>

      <para><filename>/etc/shorewall/shorewall.conf</filename>:
      <programlisting>MACLIST_LOG_LEVEL=NFLOG(1,0,1)</programlisting></para>

      <para><filename>/etc/shorewall/rules</filename>:<programlisting>#ACTION             SOURCE     DEST     PROTO     DPORT
ACCEPT:NFLOG(1,0,1) vpn        fw       tcp       ssh,time,631,8080 </programlisting><important>
          <para>Shorewall considers <emphasis role="bold">ULOG(...)</emphasis>
          and <emphasis role="bold">NFLOG(...)</emphasis> to be <emphasis
          role="bold">log levels</emphasis>, just like info, debug, etc. even
          though they are not defined by syslog.</para>
        </important></para>

      <para>Here is a copy of a ulogd.conf file that logs to
      /var/log/firewall. It was contributed by a Shorewall user on IRC:</para>

      <programlisting>[global]
user="ulogd"
logfile="/var/log/ulogd/ulogd.log"
loglevel=7

plugin="/usr/lib64/ulogd/ulogd_inppkt_NFLOG.so"
plugin="/usr/lib64/ulogd/ulogd_filter_IFINDEX.so"
plugin="/usr/lib64/ulogd/ulogd_filter_IP2STR.so"
plugin="/usr/lib64/ulogd/ulogd_filter_PRINTPKT.so"
plugin="/usr/lib64/ulogd/ulogd_output_LOGEMU.so"
plugin="/usr/lib64/ulogd/ulogd_raw2packet_BASE.so"

stack=log:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,firewall:LOGEMU

[firewall]
file="/var/log/firewall"
sync=1</programlisting>

      <note>
        <para>This sample config file assumes that NFLOG is being used in
        logging rules and policies.</para>
      </note>
    </section>
  </section>

  <section>
    <title id="Backends">Log Backends</title>

    <para>Netfilter logging allows configuration of multiple backends. Logging
    backends provide the The low-level forward of log messages. There are
    currently three backends:</para>

    <variablelist>
      <varlistentry>
        <term>LOG (ipt_LOG and ip6t_LOG).</term>

        <listitem>
          <para>Normal kernel-based logging to a syslog daemon.</para>
        </listitem>
      </varlistentry>

      <varlistentry>
        <term>ULOG (ipt_ULOG)</term>

        <listitem>
          <para>ULOG logging as described ablve. Only available for
          IPv4.</para>
        </listitem>
      </varlistentry>

      <varlistentry>
        <term>netlink (nfnetlink_log)</term>

        <listitem>
          <para>The logging backend behind NFLOG, defined above.</para>
        </listitem>
      </varlistentry>
    </variablelist>

    <para>The currently-available and currently-selected IPv4 and IPv6
    backends are shown in /proc/sys/net/netfilter/nf_log:</para>

    <programlisting>cat /proc/net/netfilter/nf_log
 0 NONE (nfnetlink_log)
 1 NONE (nfnetlink_log)
 2 ipt_ULOG (ipt_ULOG,ipt_LOG,nfnetlink_log)
 3 NONE (nfnetlink_log)
 4 NONE (nfnetlink_log)
 5 NONE (nfnetlink_log)
 6 NONE (nfnetlink_log)
 7 NONE (nfnetlink_log)
 8 NONE (nfnetlink_log)
 9 NONE (nfnetlink_log)
10 ip6t_LOG (ip6t_LOG,nfnetlink_log)
11 NONE (nfnetlink_log)
12 NONE (nfnetlink_log)</programlisting>

    <para>The magic numbers (0-12) are Linux address family numbers (AF_INET
    is 2 and AF_INET6 is 10).</para>

    <para>The name immediately following the number is the currently-selected
    backend, and the ones in parantheses are the ones that are available. You
    can change the currently selected backend by echoing it's name into
    /proc/net/netfilter/nf_log.<replaceable>number</replaceable>.</para>

    <para>Example - change the IPv4 backend to LOG:</para>

    <programlisting>sysctl net.netfilter.nf_log.2=ipt_LOG</programlisting>

    <para>Beginning with Shorewall 4.6.4, you can configure the backend using
    the LOG_BACKEND option in <ulink
    url="manpages/shorewall.conf.html">shorewall.conf(5)</ulink> and <ulink
    url="manpages/shorewall.conf.html">shorewall6.conf(5)</ulink>.</para>
  </section>

  <section id="Syslog-ng">
    <title>Syslog-ng</title>

    <para><ulink
    url="http://marc.info/?l=gentoo-security&amp;amp;m=106040714910563&amp;amp;w=2">Here</ulink>
    is a post describing configuring syslog-ng to work with Shorewall. Recent
    <trademark>SUSE</trademark> releases come preconfigured with syslog-ng
    with Netfilter messages (including Shorewall's) are written to
    <filename>/var/log/firewall</filename>.</para>
  </section>

  <section id="Contents">
    <title>Understanding the Contents of Shorewall Log Messages</title>

    <para>For general information on the contents of Netfilter log messages,
    see <ulink
    url="http://logi.cc/en/2010/07/netfilter-log-format/">http://logi.cc/en/2010/07/netfilter-log-format/</ulink>.</para>

    <para>For Shorewall-specific information, see <ulink
    url="FAQ.htm#faq17">FAQ #17</ulink>.</para>
  </section>

  <section>
    <title>Customizing the Content of Shorewall Log Messages</title>

    <para>In addition to the <link linkend="Levels">options</link> mentioned
    above, a certain amount of customization of the Netfilter-generated
    messages is allowed.</para>

    <section id="LogTags">
      <title>Log Tags</title>

      <para>In a Shorewall logging rule, the log level can be followed by
      a<firstterm> log tag</firstterm> as in "DROP:NFLOG:junk". The generated
      log message will include "<emphasis>chain-name</emphasis> junk
      DROP".</para>
    </section>

    <section>
      <title>LOGTAGONLY</title>

      <para>By setting the LOGTAGONLY option to Yes in <ulink
      url="manpages/shorewall.conf.html">shorewall.conf(5)</ulink> or <ulink
      url="manpages/shorewall.conf.html">shorewall6.conf(5)</ulink>, the
      disposition ('DROP' in the above example) will be omitted. Consider the
      following rule:</para>

      <programlisting>#ACTION                                    SOURCE          DEST           PROTO
REJECT(icmp-proto-unreachable):notice:IPv6 loc             net            41      # who's using IPv6 tunneling</programlisting>

      <para>This rule generates the following warning at compile time:</para>

      <simplelist>
        <member>WARNING: Log Prefix shortened to "Shorewall:IPv6:REJECT(icmp-p
        " /etc/shorewall/rules (line 212)</member>
      </simplelist>

      <para>and produces the rather ugly prefix "Shorewall:IPv6:REJECT(icmp-p
      ".</para>

      <para>Now consider this similar rule:</para>

      <programlisting>#ACTION                                              SOURCE          DEST           PROTO
REJECT(icmp-proto-unreachable):notice:IPv6,tunneling loc             net            41      # who's using IPv6 tunneling</programlisting>

      <para>With LOGTAGONLY=Yes, no warning is generated and the prefix
      becomes "Shorewall:IPv6:tunneling:"</para>

      <para>See the shorewall[6].conf man page for further information about
      how LOGTAGONLY=Yes can be used.</para>
    </section>

    <section>
      <title>Log Levels in shorewall[6].conf</title>

      <para><ulink
      url="manpages/shorewall.conf.html">shorewall.conf(5)</ulink> and <ulink
      url="manpages/shorewall.conf.html">shorewall6.conf(5)</ulink> have a
      number of options whose values are log levels. Beginning with Shorewall
      5.0.0, these specifcations may include a log tag as described <link
      linkend="LogTags">above</link>.</para>
    </section>
  </section>

  <section>
    <title>Some Additional Thoughts on Logging (by Bill Shirley)</title>

    <para>As a side note to the LOGTAGONLY example above, i recommend blocking
    all tunneling because it bypasses the firewall rules:</para>

    <programlisting>#ACTION                                              SOURCE          DEST           PROTO      DPORT
?COMMENT tunneling
REJECT(icmp-proto-unreachable):notice:IPv6,tunneling loc             net            41                   # who's using IPv6 tunneling
REJECT(icmp-port-unreachable)                        loc             net            tcp,udp    teredo
REJECT(icmp-port-unreachable)                        loc             net            tcp,udp    isakmp,ipsec-nat-t</programlisting>

    <para>Here is an example of logging traffic only once:</para>

    <para><filename>/etc/shorewall/init:</filename></para>

    <programlisting>ipset -exist create IPv4 hash:ip timeout 86400
ipset -exist create IPv4-port hash:ip,port timeout 14400</programlisting>

    <para><filename>/etc/shorewall/rules</filename> (at the top):</para>

    <programlisting>#ACTION             SOURCE                         DEST           PROTO
?SECTION NEW
# ------------------
?COMMENT drop previously flagged
DROP                net:+IPv4[src]                 fw
DROP                net:+IPv4-port[src,dst]        fw</programlisting>

    <para>After all the rules have been checked, at the bottom of
    <filename>/etc/shorewall/rules</filename>:</para>

    <programlisting># =============================================================================
# =============================== H@ck0rz =====================================
# =============================================================================
?COMMENT dont whack myself
REJECT:notice           inet:$ME_NET           fw

?COMMENT not public
ADD(+IPv4-port:src,dst) net                     fw      tcp,udp  domain
ADD(+IPv4-port:src,dst) net                     fw      tcp      ldap,ldaps
ADD(+IPv4-port:src,dst) net                     fw      tcp,udp  ipp

?COMMENT H@ck0rz
ADD(+IPv4:src)          net                     fw      tcp      ssh
ADD(+IPv4:src)          net                     fw      tcp      ftp,ftps,sftp,telnet,telnets,exec,login,shell,sunrpc
ADD(+IPv4:src)          net                     fw      tcp,udp  ms-sql-s,ms-sql-m

?COMMENT drop if added
DROP:info:BAN,IPv4      net:+IPv4[src]          fw
DROP:info:BAN,IPv4-port net:+IPv4-port[src,dst] fw</programlisting>

    <para>One final note: I wanted less firewall messages in /var/log/messages
    so I added to rsyslog.conf:</para>

    <programlisting>#### RULES #### &lt;-- find this
if $msg contains 'Shorewall' then {
  action(type="omfile" file="/var/log/shorewall.log")
#  if ($syslogfacility == 0 and $syslogseverity &gt;= 4) then stop    # warning
#  if ($syslogfacility == 0 and $syslogseverity &gt;= 5) then stop    # notice
  if ($syslogfacility == 0 and $syslogseverity &gt;= 6) then stop  # info
}</programlisting>

    <para>I log at 'notice' log level if I want the message in
    <filename>/var/log/messages</filename> and everything goes to
    <filename>/var/log/shorewall.log</filename>. Don't forget to add
    /var/log/shorewall.log to logrotate.</para>
  </section>
</article>