shorewall_code/docs/survey-200603.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<article>
  <!--$Id: template.xml 3517 2006-02-22 22:54:59Z judas_iscariote $-->

  <articleinfo>
    <title></title>

    <authorgroup>
      <author>
        <firstname>Paul</firstname>

        <surname>Gear</surname>
      </author>
    </authorgroup>

    <pubdate>2006-03-18</pubdate>

    <copyright>
      <year>2006</year>

      <holder>Paul D. Gear</holder>
    </copyright>

    <legalnotice>
      <para>Permission is granted to copy, distribute and/or modify this
      document under the terms of the GNU Free Documentation License, Version
      1.2 or any later version published by the Free Software Foundation; with
      no Invariant Sections, with no Front-Cover, and with no Back-Cover
      Texts. A copy of the license is included in the section entitled
      <quote><ulink url="GnuCopyright.htm">GNU Free Documentation
      License</ulink></quote>.</para>
    </legalnotice>
  </articleinfo>

  <section>
    <title>The Shorewall Environment Survey 2006</title>

    <para>In early March 2006, i @@@ embarked on the jorney of surveying
    Shorewall users. Initially this sprang from my own curiosity: it seemed to
    me that some of the systems at work on which i was using Shorewall were
    bigger and more complex than a lot of the ones others were using, and i
    wanted to find out if there were people out there who used it like i did.
    As started thinking about the questions i would ask, i realised that i
    could ask a few more questions that might help us as a project to
    understand a bit more about all of our users.</para>

    <para>I used <ulink url="http://www.zoomerang.com">Zoomerang</ulink> to
    create the survey. It has a number of tools that make it really easy to
    create useful surveys. To get the most benefit out of Zoomerang, you have
    to subscribe to their professional version. In the long term, it would be
    great to see a practical free software alternative that we could
    self-host. A number of free content management systems such as <ulink
    url="http://drupal.org">Drupal</ulink> have a survey module, but when i
    last looked at them, they were much more limited and harder to use than
    Zoomerang.</para>

    <section>
      <title>Take the survey</title>

      <para>The survey is still open as of this writing, and can be accessed
      at <ulink url="http://www.zoomerang.com/survey.zgi?p=WEB2253NNBCN44">the
      Zoomerang survey page</ulink>. Further participation is encouraged. The
      figures quoted in this document reflect the latest results at the time
      of writing.</para>
    </section>

    <section>
      <title>Survey results</title>

      <para>The <ulink
      url="http://www.zoomerang.com/reports/public_report.zgi?ID=L22KHC6BPGLS">public
      results</ulink> of the survey are also available. If you complete the
      survey, a link to the results is provided on the thank you page.</para>
    </section>
  </section>

  <section>
    <title>Detailed results analysis</title>

    <para>An important note about this survey is that it has a very small
    sample size (103 complete responses at the time of writing), so any
    conclusions drawn should be considered tentative. Additionally, since the
    survey was open to multiple responses, it could be that some people
    answered the questions about themselves more than once, despite
    instructions to the contrary in the introduction page.</para>

    <para>If you notice any errors in this analysis, or have any suggestions
    about how to improve it, please contact the author at <ulink
    url="mailto:pgear@shorewall.net">pgear@shorewall.net</ulink>.</para>

    <section>
      <title>Organisations</title>

      <para>Small organisations dominate the spectrum of Shorewall users. The
      largest group (44%) was 1-10 users - mostly SOHO LANs based on the
      comments in that section. Ninety percent (90%) of Shorewall
      installations are in organisations with less than 500 users. The results
      for the questions about organisational size and the number of users
      serviced by Shorewall match fairly closely, which seems to indicate that
      the majority of Shorewall systems are servicing the entire organisation
      in question.</para>

      <para>The vast majority (84%) of Shorewall systems are administered by
      only one person. One question that needs to be asked is, "Why?" Possible
      reasons for this might be:</para>

      <itemizedlist>
        <listitem>
          <para>Most of the organisations in which it is used are small, thus
          most of them will only have one person skilled in the area of packet
          filtering firewalls. This seems a likely scenario, but a cross
          correlation of the results of questions 1 and 2 with question 3
          indicates that the number of administrators is fairly uniform across
          all sizes of organisation and user base.</para>
        </listitem>

        <listitem>
          <para>Shorewall works so well that people don't have to touch it
          much. Obviously, this is the preferred interpretation of the
          Shorewall project team. :-)</para>
        </listitem>

        <listitem>
          <para>Shorewall is too hard for new users to comprehend, so one
          skilled person in an organisation tends to get the job maintaining
          it. Equally obviously, this is a non-preferred interpretation. :-)
          However, being a firewall generator, Shorewall is not likely to
          attract the same sort of users as a web browser or music
          player.</para>
        </listitem>

        <listitem>
          <para>Shorewall administrators are a closed bunch and don't like
          sharing their job around. Given the nature of firewalls and packet
          filtering, this doesn't seem far-fetched.</para>
        </listitem>
      </itemizedlist>

      <para>There doesn't seem to be an easy answer to thus question. In
      retrospect, since there were no responses indicating 10 or more
      administrators, i could have made the granularity of this question
      better. A question about a person's role in the organisation may also
      have been helpful. Possibly we could follow up with a smaller survey,
      specifically about the people and organisations who use
      Shorewall.</para>
    </section>

    <section>
      <title>Users</title>

      <para>Unsurprisingly, 97% of survey respondents were male. Or to put it
      another way: suprisingly, there are actually 3 female Shorewall users.
      Being male seems to be an occupational hazard of life in the IT
      industry, and even more so in the more "nerdy" specialisations like
      Linux and security. :-)</para>

      <para>The largest age group of users is 25-34 years (42% of all
      respondents). There were no retirees (65 and over) or minors (under 18)
      in the responses. The distribution of all remaining age groups was
      fairly even.</para>

      <para>The largest group of users in terms of education was those with a
      Bachelor's degree, followed by those with a high school education.
      Fifty-seven percent (57%) of Shorewall users have a Bachelor's degree or
      better. Many users' highest qualifications are not in an IT-related
      discipline (42%). This remains fairly constant across the spectrum when
      correlated with the highest level of qualifications.</para>

      <para>Those users who do not claim IT as their highest discipline hold a
      wide variety of other qualifications, including agriculture, art,
      business, chemistry, education, various forms of engineering, law,
      mathematics, physics and theology.</para>

      <para>Almost two-thirds of users (62%) use Shorewall as part of their
      paid employment. Of these, 12% (7 of 58) do not use Shorewall as part of
      their official duties. Cross correlation with level of education
      revealed no major variances in this trend depending on level of
      education.</para>

      <para>The majority of users (73%) began using the Internet in the 1990s.
      A smaller majority (61%) have been using the Internet for more than 12
      years (1994 or earlier). (The single response indicating use of the
      Internet (then ARPANET) since the 1960s seems to be an error.)</para>

      <para>The majority of users (70%) began using Linux after it reached a
      certain stage of maturity - around or after the release of kernel 2.0
      (1996). However, nearly all respondents (97%) have been using Linux for
      5 years or more, with almost half (47%) having 10 or more years
      experience with it. It seems fair to say that as a rule, Shorewall
      attracts people with plenty of experience.</para>

      <para>Around one third of users (30%) have been using Shorewall for more
      than 5 years, with two-thirds (66%) having used it since the 1.x series
      (2003 or earlier). It seems fair to say that Shorewall users seem to
      stick with the product once they are familiar with it. On the other
      hand, it seems that Shorewall is not attracting large numbers of new
      users, which is a concern for the future of the project.</para>
    </section>

    <section>
      <title>Hardware</title>

      <para>Ninety-three percent (93%) of users run Shorewall on i386 family
      hardware, with a further 6% running it on x86-64/EM64T platforms. One
      response was received indicating use of Shorewall on MIPS (Linksys WRT
      platform). No responses were received for any other hardware platform.
      While this is not surprising given Intel's</para>

      <para>A good spread of CPU power is shown in the survey responses. The
      largest group was 400-999 MHz (30%), with only 16% of responses
      indicating less than 400 MHz, with the same number greater than 2500
      MHz. A number of responses in the field for additional information
      suggested that the machines used were either recycled desktops, or
      systems that were specifically built to do the job, and had been running
      in that role for a number of years.</para>

      <para>RAM configuration seemed to mostly mirror CPU power, with the
      majority (52%) of systems having between 256 and 1023 MB. A bias towards
      higher RAM figures (only 11% of systems have less than 128 MB; 28% have
      1024 MB or more) reflects the more server-oriented workload that many
      Shorewall systems run (see section @@@ below). (Note that there is an
      error in the released version of the survey for this question: it was a
      multiple choice question rather than single choice, and thus there were
      more results than expected. However, the number of errors doesn't seem
      to be significant.)</para>

      <para>Shorewall systems on the whole tend toward smaller OS hard disks,
      with 42% having disks 39 GB or smaller. The largest group by a small
      margin was 80-159 GB at 23%, with 10-39 GB and 0-9 GB coming in a close
      second and third at 22% and 20% respectively.</para>
    </section>

    <section>
      <title>Network</title>

      <para>The majority of Shorewall systems (82%) use between two and four
      network interfaces. The number of devices connected to systems closely
      mirrors the size of the organisations in which they are used, with 95%
      of systems connecting less than 500 devices, and the largest group (41%)
      connecting 2-10 other devices.</para>

      <para>Ninety percent (90%) of Shorewall systems are connected to 100
      Mbps or faster local networks. Most systems have a broadband Internet
      connection or better, with only 7% having 512 Kbps or less, and 51%
      having 10 Mbps or better. DSL is the most common form of Internet
      connection, with over half the responses (51%).</para>
    </section>

    <section>
      <title>Software</title>

      <para>The most popular Linux distribution on which users run Shorewall
      is Debian (26% of respondents), followed by a group consisting of Fedora
      Core (16%), Red Hat 9 and earlier (13%) and Red Hat Enterprise and
      derivatives (12%). The next group consists of SUSE (9%), Slackware (8%),
      Gentoo (6%), and LEAF/Bering (5%).</para>

      <para>The message about maintaining an up-to-date Shorewall system seems
      to have gotten through, with 61% of respondents running the latest
      version (3.0), and an additional 22% running the previous stable version
      (2.4). Only 14% of users are running unsupported versions (2.2 and
      older).</para>

      <para>The most common roles played by Shorewall systems are:</para>

      <itemizedlist>
        <listitem>
          <para>External firewall/router (78%)</para>
        </listitem>

        <listitem>
          <para>DNS name server (61%)</para>
        </listitem>

        <listitem>
          <para>DHCP server (59%)</para>
        </listitem>

        <listitem>
          <para>Internal firewall/router (56%)</para>
        </listitem>

        <listitem>
          <para>Time server (55%)</para>
        </listitem>
      </itemizedlist>
    </section>

    <section>
      <title>Comments from users</title>

      <para>Here's a sample of the comments we received about the survey
      (carefully sanitised to make us look good ;-).</para>

      <itemizedlist>
        <listitem>
          <para>More power to Shorewall!</para>
        </listitem>

        <listitem>
          <para>Shorewall Rocks! I'm amazed how easy it is every time I need
          to do something, even if it's been 6+ months since the last change!
          :)</para>
        </listitem>

        <listitem>
          <para>Good job and a great product</para>
        </listitem>

        <listitem>
          <para>Shorewall is good, I have recommended it to several people,
          mostly working in the University &amp; academic areas.</para>
        </listitem>

        <listitem>
          <para>Thanks to everyone who contributes to Shorewall. That's a
          *great* piece of software!</para>
        </listitem>

        <listitem>
          <para>Shorewall has been incredible. Tom has given so much of
          himself to this project, I can only say thank you from one person, I
          look up to people like him. I have used Shorewall for many systems,
          I am a contractor that "set up shop" all over the world. Depending
          on the available ISP services, this project has been flexible in
          every situation to date. Also, depending on my needs, it has done
          the same. "IP Tables made easy" is really an accurate
          description.</para>
        </listitem>

        <listitem>
          <para>I'm quite intersted in seeing what the 'cross section' of
          Shorewall users are like. It's made my life a lot easier over the
          years, Thank you.</para>
        </listitem>
      </itemizedlist>
    </section>
  </section>

  <section>
    <title>Conclusions</title>

    <para></para>
  </section>

  <section>
    <title>Possible implications for the Shorewall project</title>

    <para></para>

    <para></para>
  </section>

  <section>
    <title></title>

    <para></para>
  </section>

  <section>
    <title>Possible implications for other free software projects</title>

    <para></para>
  </section>

  <section>
    <title>Lessons learned about surveys</title>

    <para></para>

    <section>
      <title>Things i did right</title>

      <para></para>

      <itemizedlist>
        <listitem>
          <para>Treat it like releasing free software:</para>

          <itemizedlist>
            <listitem>
              <para>release early and often</para>
            </listitem>

            <listitem>
              <para>make branches when you release alpha and beta versions,
              and bring the lessons you learned in those versions into the
              main trunk</para>
            </listitem>
          </itemizedlist>
        </listitem>
      </itemizedlist>
    </section>

    <section>
      <title>Things i did wrong</title>

      <para></para>

      <itemizedlist>
        <listitem>
          <para>Start small and work towards what you want to know. I tried to
          do everything in one survey, and ended up confusing some
          people.</para>
        </listitem>

        <listitem>
          <para></para>
        </listitem>

        <listitem>
          <para>Be prepared beforehand</para>
        </listitem>
      </itemizedlist>

      <para></para>
    </section>
  </section>

  <section>
    <title></title>

    <para></para>
  </section>

  <section>
    <title></title>

    <para></para>
  </section>
</article>