Merge pull request #38 from rfjakob/next

Add benchmark.pl

DESIGN.md

@@ -1,4 +1,3 @@
-
 For notes about internationalization, see README-NLS.
 
 EncFS is a program which provides an encrypted virtual filesystem for Linux
@@ -7,7 +6,8 @@ download the latest version of FUSE ).  FUSE provides a loadable kernel module
 which exports a filesystem interface to user-mode.  EncFS runs entirely in
 user-mode and acts as a transparent encrypted filesystem.
 
-Usage:
+Usage
+-----
 
  - To see command line options, see the man page for encfs and encfsctl, or for
    brief usage message, run the programs without an argument (or '-h'):
@@ -33,7 +33,8 @@ Usage:
    created.
 
 
-Technology:
+Technology
+----------
 
  - Encfs uses algorithms from third-party libraries (OpenSSL is the default) to
    encrypt data and filenames.
@@ -92,7 +93,8 @@ Technology:
    file has a unique initialization vector.  This makes it infeasible to copy a
    whole block from one file to another. 
 
-Backward Compatibility:
+Backward Compatibility
+----------------------
 
    At the top level of the raw (encrypted) storage for an EncFS filesystem is a
    configuration file, created automatically by EncFS when a new filesystem is
@@ -109,7 +111,8 @@ Backward Compatibility:
    newer versions use algorithms and/or new options which were not previously
    available.
 
-Utility:
+Utility
+-------
 
    In addition to the "encfs" main program, a utility "encfsctl" has been
    provided which can perform some operations on encfs filesystems.  Encfsctl
@@ -117,7 +120,8 @@ Utility:
    algorithm used, key length, block size), and more importantly it can also
    change the user-supplied password used to encrypt the volume key.
 
-Dependencies:
+Dependencies
+------------
 
    Encfs uses the OpenSSL toolkit (http://www.openssl.org) by default.
    OpenSSL is not covered by the GPL, and some people are concerned about the

Makefile.am

@@ -17,8 +17,8 @@ ACLOCAL_AMFLAGS = -I m4
 
 .PHONY: test
 test:
-	perl -MTest::Harness -e '$$Test::Harness::verbose=0; runtests @ARGV;' tests/*.pl
+	perl -MTest::Harness -e '$$Test::Harness::verbose=0; runtests @ARGV;' tests/*.t.pl
 
 .PHONY: test-verbose
 test-verbose:
-	perl -MTest::Harness -e '$$Test::Harness::verbose=1; runtests @ARGV;' tests/*.pl
+	perl -MTest::Harness -e '$$Test::Harness::verbose=1; runtests @ARGV;' tests/*.t.pl

PERFORMANCE.md

@@ -0,0 +1,103 @@
+EncFS Performance
+=================
+
+EncFS runs in user-space while eCryptfs runs in the kernel.
+This is why it is often assumed that eCryptfs is faster than EncFS.
+To compare the actual performance of EncFS and eCryptfs on top of
+different backing disks, the EncFS test suite contains an automated
+performance test - [benchmark.pl](tests/benchmark.pl).
+
+performance.pl takes care of setting up EncFS and eCryptfs mounts,
+clearing caches and syncing disks between the tests, and also to unmount
+and clean up everything in the end.
+
+It performance the following tests:
+
+* stream_write: Write 100MB of zeros in 4KB blocks
+* extract: Extract the [linux-3.0.tar.gz archive](https://www.kernel.org/pub/linux/kernel/v3.x/)
+* du: Disk space used after extraction, in megabytes
+* rsync: Do an "rsync -an" of the extracted files.
+  This simulates an rsync to a destination that is
+  (almost) up-to-date. The workload consists mostly
+  of stat() calls.
+* delete: Recursively delete the extracted files
+
+For EncFS, the [default options](tests/benchmark.pl#L47) are used.
+This means:
+
+* AES with 192 bit key
+* Filename encryption
+
+For eCryptfs, the [options used](tests/mount-ecryptfs.expect) are
+
+* AES with 128 bit key
+* Filename encryption
+
+For all the details, take a look at [benchmark.pl](tests/benchmark.pl) .
+
+Results
+-------
+The performance of an overlay filesystem depends a lot on the performance
+of the backing disk. This is why I have tested three different kinds of
+disk:
+
+* Classic HDD: Seagate Barracuda 7200.9, model ST3250824AS
+* Modern SSD: Samsung SSD 840 EVO 250GB
+* Ramdisk: tmpfs mounted on /tmp
+
+All tests are performed on kernel 3.16.3, 64 bit, on an Intel Pentium
+G630 (Sandy Bridge, 2 x 2.7GHz).
+
+If you want to replicate the test, just run
+
+    sudo tests/benchmark.pl /path/to/test/directory
+
+(the test must be run as root as normal users cannot mount ecryptfs or
+clear the caches)
+
+* HDD: Seagate Barracuda 7200.9
+
+Test            | EncFS        | eCryptfs     | EncFS advantage
+----------------|-------------:|-------------:|---------------:
+stream_write    |     32 MiB/s |     38 MiB/s | 0.84
+extract         |  28744 ms    |  30027 ms    | 1.04
+du              |    495 MB    |    784 MB    | 1.58
+rsync           |   3319 ms    |  62486 ms    | 18.83
+delete          |   6462 ms    |  74652 ms    | 11.55
+
+
+* SSD: Samsung SSD 840 EVO 250GB
+
+Test            | EncFS        | eCryptfs     | EncFS advantage
+----------------|-------------:|-------------:|---------------:
+stream_write    |     53 MiB/s |     75 MiB/s | 0.71
+extract         |  26129 ms    |   9692 ms    | 0.37
+du              |    495 MB    |    784 MB    | 1.58
+rsync           |   2725 ms    |   8210 ms    | 3.01
+delete          |   5444 ms    |   9130 ms    | 1.68
+
+* Ramdisk: tmpfs
+
+Test            | EncFS        | eCryptfs     | EncFS advantage
+----------------|-------------:|-------------:|---------------:
+stream_write    |     82 MiB/s |    111 MiB/s | 0.74
+extract         |  22393 ms    |   8117 ms    | 0.36
+du              |    485 MB    |    773 MB    | 1.59
+rsync           |   1931 ms    |    740 ms    | 0.38
+delete          |   4346 ms    |    907 ms    | 0.21
+
+Interpretation
+--------------
+eCryptfs uses a large per-file header (8 KB) which is a big disadvantage
+on classic HDDs. For stat()-heavy operations on HDDs, EncFS is 18x faster.
+
+EncFS stores small files much more efficiently, which is why it consitently
+uses less space than eCryptfs: zero-size files take no space at all,
+other files get a 8-byte header. Because the filesystem allocates space
+in 4KB blocks, the actually used disk space must be rounded up to 4096.
+
+plaintext size | EncFS raw | EncFS du | eCryptfs raw | eCryptfs du
+--------------:|----------:|---------:|-------------:|------------:
+   0           |    0      |    0     |  8192        |  8192
+   1           |    9      | 4096     | 12288        | 12288
+1024           | 1032      | 4096     | 12288        | 12288

README.md

@@ -17,6 +17,8 @@ EncFS encrypts individual files, by translating all requests for the virtual
 EncFS filesystem into the equivalent encrypted operations on the raw
 filesystem.
 
+For more technical details and a usage overview, see [DESIGN.md](DESIGN.md).
+
 ## Status
 
 Over the last 10 years, a number of good alternatives have grown up.  Computing
@@ -31,19 +33,30 @@ depends upon community interest.  In order to make it easier for anyone to
 contribute, it is moving a new home on Github.  So if you're interested in
 EncFS, please dive in!
 
-EncFS still has a few unique features that may be interesing to you:
+## Unique Features
 
-*   `--reverse` mode: Provides an encrypted view of an unencrypted folder.
+EncFS has a few features still not found anywhere else (as of Dec 2014)
-    This enables encrypted remote backups using standard tools like
+that may be interesing to you:
-    rsync.
+
-*   EncFS is typically faster than ecryptfs for stat()-heavy workloads
+### Reverse mode
-    when the backing device is a classical hard disk.
+
-    This is because ecryptfs has to to read each file header to determine
+`encfs --reverse` provides an encrypted view of an unencrypted folder.
-    the file size - EncFS does not. This is one additional seek for each
+This enables encrypted remote backups using standard tools like rsync.
-    stat. On SSDs that have virtually no seek time, that difference may
+
-    disappear.
+### Fast on classical HDDs
-*   EncFS works on network file systems (NFS, CIFS...), while ecryptfs
+
-    is known to still have [problems][1].
+EncFS is typically *much* faster than ecryptfs for stat()-heavy workloads
+when the backing device is a classical hard disk.
+This is because ecryptfs has to to read each file header to determine
+the file size - EncFS does not. This is one additional seek for each
+stat.
+See [PERFORMANCE.md](PERFORMANCE.md) for detailed benchmarks on
+HDD, SSD and ramdisk.
+
+### Works on top of network filesystems
+
+EncFS works on network file systems (NFS, CIFS...), while ecryptfs
+is known to still have [problems][1].
 
 ## GitHub Transition
 

tests/benchmark.pl

@@ -0,0 +1,194 @@
+#!/usr/bin/perl
+
+# Benchmark EncFS against eCryptfs
+
+use Time::HiRes qw( time );
+use File::Temp;
+use warnings;
+
+require("tests/common.pl");
+
+# Download linux-3.0.tar.gz unless it already exists ("-c" flag)
+sub dl {
+    our $linuxgz = "/tmp/linux-3.0.tar.gz";
+    print "# downloading linux-3.0.tar.gz... ";
+    system("wget -nv -c https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.0.tar.gz -O $linuxgz");
+    print "done\n";
+}
+
+# Create a new empty working directory
+sub newWorkingDir {
+    my $prefix     = shift;
+    my $workingDir = mkdtemp("$prefix/encfs-performance-XXXX")
+      || die("Could not create temporary directory");
+
+    return $workingDir;
+}
+
+sub cleanup {
+    print "cleaning up...";
+    my $workingDir = shift;
+    system("umount $workingDir/ecryptfs_plaintext");
+    system("fusermount -u $workingDir/encfs_plaintext");
+    system("rm -Rf $workingDir");
+    print "done\n";
+}
+
+sub mount_encfs {
+    my $workingDir = shift;
+
+    my $c = "$workingDir/encfs_ciphertext";
+    my $p = "$workingDir/encfs_plaintext";
+
+    mkdir($c);
+    mkdir($p);
+
+    delete $ENV{"ENCFS6_CONFIG"};
+    system("./encfs/encfs --extpass=\"echo test\" --standard $c $p > /dev/null");
+    waitForFile("$c/.encfs6.xml") or die("Control file not created");
+
+    print "# encfs mounted on $p\n";
+
+    return $p;
+}
+
+sub mount_ecryptfs {
+    my $workingDir = shift;
+
+    my $c = "$workingDir/ecryptfs_ciphertext";
+    my $p = "$workingDir/ecryptfs_plaintext";
+
+    mkdir($c);
+    mkdir($p);
+
+    system("expect -c \"spawn mount -t ecryptfs $c $p\" ./tests/mount-ecryptfs.expect > /dev/null") == 0
+      or die("ecryptfs mount failed - are you root?");
+
+    print "# ecryptfs mounted on $p\n";
+
+    return $p;
+}
+
+# Returns integer $milliseconds from float $seconds
+sub ms {
+    my $seconds      = shift;
+    my $milliseconds = int( $seconds * 1000 );
+    return $milliseconds;
+}
+
+sub benchmark {
+    my $dir = shift;
+    my $start;
+    our $linuxgz;
+    my $delta;
+    my $line;
+
+    my @results = ();
+
+    system("sync");
+    print("# stream_write... ");
+    $start = time();
+    writeZeroes( "$dir/zero", 1024 * 1024 * 100 );
+    system("sync");
+    $delta = time() - $start;
+    push( @results, [ 'stream_write', int( 100 / $delta ), "MiB/s" ] );
+    printf("done\n");
+    unlink("$dir/zero");
+
+    system("sync");
+    system("cat $linuxgz > /dev/null");
+    print("# extract... ");
+    $start = time();
+    system("tar xzf $linuxgz -C $dir");
+    system("sync");
+    $delta = ms( time() - $start );
+    push( @results, [ 'extract', $delta, "ms" ] );
+    print("done\n");
+
+    $du = qx(du -sm $dir | cut -f1);
+    push( @results, [ 'du', $du, 'MB' ] );
+    printf( "# disk space used: %d MB\n", $du );
+
+    system("echo 3 > /proc/sys/vm/drop_caches");
+    $start = time();
+    system("rsync -an $dir /tmp");
+    $delta = time() - $start;
+    push( @results, [ 'rsync', ms($delta), 'ms' ] );
+    printf( "# rsync took %d ms\n", ms($delta) );
+
+    system("echo 3 > /proc/sys/vm/drop_caches");
+    system("sync");
+    $start = time();
+    system("rm -Rf $dir/*");
+    system("sync");
+    $delta = time() - $start;
+    push( @results, [ 'delete', ms($delta), 'ms' ] );
+    printf( "# delete took %d ms\n", ms($delta) );
+
+    return \@results;
+}
+
+sub tabulate {
+    my $r;
+
+    $r = shift;
+    my @encfs = @{$r};
+    $r = shift;
+    my @ecryptfs = @{$r};
+
+    print " Test           | EncFS        | eCryptfs     | EncFS advantage\n";
+    print ":---------------|-------------:|-------------:|---------------:\n";
+
+    for ( my $i = 0 ; $i <= $#encfs ; $i++ ) {
+        my $test = $encfs[$i][0];
+        my $unit = $encfs[$i][2];
+
+        my $en = $encfs[$i][1];
+        my $ec = $ecryptfs[$i][1];
+
+        my $ratio = $ec / $en;
+        if ( $unit =~ m!/s! ) {
+            $ratio = $en / $ec;
+        }
+        printf( "%-15s | %6d %-5s | %6d %-5s | %2.2f\n",
+            $test, $en, $unit, $ec, $unit, $ratio );
+    }
+}
+
+sub main {
+    if ( $#ARGV < 0 ) {
+        print "Usage: test/benchmark.pl MOUNTPOINT [MOUNTPOINT] [...]\n";
+        exit(1);
+    }
+
+    if ( $> != 0 ) {
+        print("This test must be run as root!\n");
+        exit(2);
+    }
+
+    dl();
+    my $workingDir;
+    my $mountpoint;
+    my $prefix;
+
+    while ( $prefix = shift(@ARGV) ) {
+        $workingDir = newWorkingDir($prefix);
+
+        print "# mounting encfs\n";
+        $mountpoint = mount_encfs($workingDir);
+        my $encfs_results = benchmark($mountpoint);
+
+        print "# mounting ecryptfs\n";
+        $mountpoint = mount_ecryptfs($workingDir);
+        my $ecryptfs_results = benchmark($mountpoint);
+
+        cleanup($workingDir);
+
+        print "\nResults for $prefix\n";
+        print "==============================\n\n";
+        tabulate( $encfs_results, $ecryptfs_results );
+        print "\n";
+    }
+}
+
+main();

tests/mount-ecryptfs.expect

@@ -0,0 +1,22 @@
+#!/bin/expect -f
+
+# Passed by controlling process via "-c" 
+#spawn mount -t ecryptfs ecryptfs_ciphertext ecryptfs_plaintext
+
+expect "Selection: "
+send "1\n"
+expect "Passphrase: "
+send "test\n"
+expect "Selection \\\[aes\\\]: "
+send "aes\n"
+expect "Selection \\\[16\\\]: "
+send "16\n"
+expect "Enable plaintext passthrough (y/n) \\\[n\\\]: "
+send "n\n"
+expect "Enable filename encryption (y/n) \\\[n\\\]: "
+send "y\n"
+# Filename Encryption Key (FNEK) Signature [d395309aaad4de06]: 
+expect "\\\]: "
+send "\n"
+expect "Mounted eCryptfs"
+send_user "\n"

tests/normal.t.pl

@@ -8,7 +8,7 @@ use File::Copy;
 use File::Temp;
 use IO::Handle;
 
-require("tests/common.inc");
+require("tests/common.pl");
 
 my $tempDir = $ENV{'TMPDIR'} || "/tmp";
 

tests/reverse.t.pl

@@ -9,7 +9,7 @@ use File::Temp;
 use IO::Handle;
 use Errno qw(EROFS);
 
-require("tests/common.inc");
+require("tests/common.pl");
 
 my $tempDir = $ENV{'TMPDIR'} || "/tmp";