Don't require the remote server to have sshuttle installed.

Instead, grab our source code, send it over the link, and have python eval
it and then start the server.py main() function.

Strangely, there's now *less* horrible stuff in ssh.py, because we no longer
have to munge around with the PATH environment variable.  And this
significantly reduces the setup required to get sshuttle going.

Based on a suggestion from Wayne Scott.

README.md

@@ -0,0 +1,163 @@
+sshuttle: where transparent proxy meets VPN meets ssh
+=====================================================
+
+I just spent an afternoon working on a new kind of VPN.  You can get
+the first release, <a href="http://github.com/apenwarr/sshuttle">sshuttle
+0.10, on github</a>.
+
+As far as I know, sshuttle is the only program that solves the following
+common case:
+
+ - Your client machine (or router) is Linux.
+
+ - You have access to a remote network via ssh.
+
+ - You don't necessarily have admin access on the remote network.
+
+ - The remote network has no VPN, or only stupid/complex VPN
+    protocols (IPsec, PPTP, etc). Or maybe you <i>are</i> the
+    admin and you just got frustrated with the awful state of
+    VPN tools.
+
+ - You don't want to create an ssh port forward for every
+    single host/port on the remote network.
+
+ - You hate openssh's port forwarding because it's randomly
+    slow and/or stupid.
+ 
+ - You can't use openssh's PermitTunnel feature because
+    it's disabled by default on openssh servers; plus it does
+    TCP-over-TCP, which has terrible performance (see below).
+    
+
+Prerequisites
+-------------
+
+ - sudo, su, or logged in as root on your client machine.
+   (The server doesn't need admin access.)
+   
+ - If you use Linux on your client machine:
+   iptables installed on the client, including at
+   least the iptables DNAT, REDIRECT, and ttl modules. 
+   These are installed by default on most Linux distributions. 
+   (The server doesn't need iptables and doesn't need to be
+   Linux.)
+   
+ - If you use MacOS or BSD on your client machine:
+   Your kernel needs to be compiled with IPFIREWALL_FORWARD
+   (MacOS has this by default) and you need to have ipfw
+   available. (The server doesn't need to be MacOS or BSD.)
+
+
+This is how you use it:
+-----------------------
+
+ - <tt>git clone git://github.com/apenwarr/sshuttle</tt>
+    on your client and server machines. The server can be
+    any ssh server with python available; the client must
+    be Linux with iptables, and you'll need root or sudo
+    access.
+
+ - <tt>./sshuttle -r username@sshserver 0.0.0.0/0 -vv</tt>
+
+That's it!  Now your local machine can access the remote network as if you
+were right there!  And if your "client" machine is a router, everyone on
+your local network can make connections to your remote network.
+
+You don't need to install sshuttle on the remote server;
+the remote server just needs to have python available. 
+sshuttle will automatically upload and run its source code
+to the remote python interpreter.
+
+This creates a transparent proxy server on your local machine for all IP
+addresses that match 0.0.0.0/0.  (You can use more specific IP addresses if
+you want; use any number of IP addresses or subnets to change which
+addresses get proxied.  Using 0.0.0.0/0 proxies <i>everything</i>, which is
+interesting if you don't trust the people on your local network.)
+
+Any TCP session you initiate to one of the proxied IP addresses will be
+captured by sshuttle and sent over an ssh session to the remote copy of
+sshuttle, which will then regenerate the connection on that end, and funnel
+the data back and forth through ssh.
+
+Fun, right?  A poor man's instant VPN, and you don't even have to have
+admin access on the server.
+
+
+Theory of Operation
+-------------------
+
+sshuttle is not exactly a VPN, and not exactly port forwarding.  It's kind
+of both, and kind of neither.
+
+It's like a VPN, since it can forward every port on an entire network, not
+just ports you specify.  Conveniently, it lets you use the "real" IP
+addresses of each host rather than faking port numbers on localhost.
+
+On the other hand, the way it *works* is more like ssh port forwarding than
+a VPN.  Normally, a VPN forwards your data one packet at a time, and
+doesn't care about individual connections; ie. it's "stateless" with respect
+to the traffic.  sshuttle is the opposite of stateless; it tracks every
+single connection.
+
+You could compare sshuttle to something like the old <a
+href="http://en.wikipedia.org/wiki/Slirp">Slirp</a> program, which was a
+userspace TCP/IP implementation that did something similar.  But it
+operated on a packet-by-packet basis on the client side, reassembling the
+packets on the server side.  That worked okay back in the "real live serial
+port" days, because serial ports had predictable latency and buffering.
+
+But you can't safely just forward TCP packets over a TCP session (like ssh),
+because TCP's performance depends fundamentally on packet loss; it
+<i>must</i> experience packet loss in order to know when to slow down!  At
+the same time, the outer TCP session (ssh, in this case) is a reliable
+transport, which means that what you forward through the tunnel <i>never</i>
+experiences packet loss.  The ssh session itself experiences packet loss, of
+course, but TCP fixes it up and ssh (and thus you) never know the
+difference.  But neither does your inner TCP session, and extremely screwy
+performance ensues.
+
+sshuttle assembles the TCP stream locally, multiplexes it statefully over
+an ssh session, and disassembles it back into packets at the other end.  So
+it never ends up doing TCP-over-TCP.  It's just data-over-TCP, which is
+safe.
+
+
+Useless Trivia
+--------------
+
+Back in 1998 (12 years ago!  Yikes!), I released the first version of <a
+href="http://alumnit.ca/wiki/?TunnelVisionReadMe">Tunnel Vision</a>, a
+semi-intelligent VPN client for Linux.  Unfortunately, I made two big mistakes: 
+I implemented the key exchange myself (oops), and I ended up doing
+TCP-over-TCP (double oops).  The resulting program worked okay - and people
+used it for years - but the performance was always a bit funny.  And nobody
+ever found any security flaws in my key exchange, either, but that doesn't
+mean anything. :)
+
+The same year, dcoombs and I also released Fast Forward, a proxy server
+supporting transparent proxying.  Among other things, we used it for
+automatically splitting traffic across more than one Internet connection (a
+tool we called "Double Vision").
+
+I was still in university at the time.  A couple years after that, one of my
+professors was working with some graduate students on the technology that
+would eventually become <a href="http://www.slipstream.com/">Slipstream
+Internet Acceleration</a>.  He asked me to do a contract for him to build an
+initial prototype of a transparent proxy server for mobile networks.  The
+idea was similar to sshuttle: if you reassemble and then disassemble the TCP
+packets, you can reduce latency and improve performance vs.  just forwarding
+the packets over a plain VPN or mobile network.  (It's unlikely that any of
+my code has persisted in the Slipstream product today, but the concept is
+still pretty cool.  I'm still horrified that people use plain TCP on
+complex mobile networks with crazily variable latency, for which it was
+never really intended.)
+
+That project I did for Slipstream was what first gave me the idea to merge
+the concepts of Fast Forward, Double Vision, and Tunnel Vision into a single
+program that was the best of all worlds.  And here we are, at last, 10 years
+later.  You're welcome.
+
+--
+Avery Pennarun <apenwarr@gmail.com>
+

assembler.py

@@ -0,0 +1,26 @@
+import sys, zlib
+
+z = zlib.decompressobj()
+mainmod = sys.modules[__name__]
+while 1:
+    name = sys.stdin.readline().strip()
+    if name:
+        nbytes = int(sys.stdin.readline())
+        if verbosity >= 2:
+            sys.stderr.write('remote assembling %r (%d bytes)\n'
+                             % (name, nbytes))
+        content = z.decompress(sys.stdin.read(nbytes))
+        exec compile(content, name, "exec")
+
+        # FIXME: this crushes everything into a single module namespace,
+        # then makes each of the module names point at this one. Gross.
+        assert(name.endswith('.py'))
+        modname = name[:-3]
+        mainmod.__dict__[modname] = mainmod
+    else:
+        break
+
+verbose = verbosity
+sys.stderr.flush()
+sys.stdout.flush()
+main()

client.py

@@ -4,26 +4,83 @@ from ssnet import SockWrapper, Handler, Proxy, Mux, MuxWrapper
 from helpers import *
 
 def original_dst(sock):
-    SO_ORIGINAL_DST = 80
+    try:
-    SOCKADDR_MIN = 16
+        SO_ORIGINAL_DST = 80
-    sockaddr_in = sock.getsockopt(socket.SOL_IP, SO_ORIGINAL_DST, SOCKADDR_MIN)
+        SOCKADDR_MIN = 16
-    (proto, port, a,b,c,d) = struct.unpack('!HHBBBB', sockaddr_in[:8])
+        sockaddr_in = sock.getsockopt(socket.SOL_IP,
-    assert(socket.htons(proto) == socket.AF_INET)
+                                      SO_ORIGINAL_DST, SOCKADDR_MIN)
-    ip = '%d.%d.%d.%d' % (a,b,c,d)
+        (proto, port, a,b,c,d) = struct.unpack('!HHBBBB', sockaddr_in[:8])
-    return (ip,port)
+        assert(socket.htons(proto) == socket.AF_INET)
+        ip = '%d.%d.%d.%d' % (a,b,c,d)
+        return (ip,port)
+    except socket.error, e:
+        if e.args[0] == errno.ENOPROTOOPT:
+            return sock.getsockname()
+        raise
 
 
-def iptables_setup(port, subnets):
+class FirewallClient:
-    subnets_str = ['%s/%d' % (ip,width) for ip,width in subnets]
+    def __init__(self, port, subnets):
-    argv = (['sudo', sys.argv[0]] +
+        self.port = port
-            ['-v'] * (helpers.verbose or 0) +
+        self.subnets = subnets
-            ['--iptables', str(port)] + subnets_str)
+        subnets_str = ['%s/%d' % (ip,width) for ip,width in subnets]
-    rv = subprocess.call(argv)
+        argvbase = ([sys.argv[0]] +
-    if rv != 0:
+                    ['-v'] * (helpers.verbose or 0) +
-        raise Fatal('%r returned %d' % (argv, rv))
+                    ['--firewall', str(port)] + subnets_str)
+        argv_tries = [
+            ['sudo'] + argvbase,
+            ['su', '-c', ' '.join(argvbase)],
+            argvbase
+        ]
+
+        # we can't use stdin/stdout=subprocess.PIPE here, as we normally would,
+        # because stupid Linux 'su' requires that stdin be attached to a tty.
+        # Instead, attach a *bidirectional* socket to its stdout, and use
+        # that for talking in both directions.
+        (s1,s2) = socket.socketpair()
+        def setup():
+            # run in the child process
+            s2.close()
+        e = None
+        for argv in argv_tries:
+            try:
+                self.p = subprocess.Popen(argv, stdout=s1, preexec_fn=setup)
+                e = None
+                break
+            except OSError, e:
+                pass
+        self.argv = argv
+        s1.close()
+        self.pfile = s2.makefile('wb+')
+        if e:
+            log('Spawning firewall manager: %r\n' % self.argv)
+            raise Fatal(e)
+        line = self.pfile.readline()
+        self.check()
+        if line != 'READY\n':
+            raise Fatal('%r expected READY, got %r' % (self.argv, line))
+
+    def check(self):
+        rv = self.p.poll()
+        if rv:
+            raise Fatal('%r returned %d' % (self.argv, rv))
+
+    def start(self):
+        self.pfile.write('GO\n')
+        self.pfile.flush()
+        line = self.pfile.readline()
+        self.check()
+        if line != 'STARTED\n':
+            raise Fatal('%r expected STARTED, got %r' % (self.argv, line))
+
+    def done(self):
+        self.pfile.close()
+        rv = self.p.wait()
+        if rv:
+            raise Fatal('cleanup: %r returned %d' % (self.argv, rv))
 
 
-def _main(listener, listenport, use_server, remotename, subnets):
+def _main(listener, fw, use_server, remotename):
     handlers = []
     if use_server:
         if helpers.verbose >= 1:
@@ -47,14 +104,14 @@ def _main(listener, listenport, use_server, remotename, subnets):
 
     # we definitely want to do this *after* starting ssh, or we might end
     # up intercepting the ssh connection!
-    iptables_setup(listenport, subnets)
+    fw.start()
 
     def onaccept():
         sock,srcip = listener.accept()
         dstip = original_dst(sock)
         debug1('Accept: %r:%r -> %r:%r.\n' % (srcip[0],srcip[1],
                                            dstip[0],dstip[1]))
-        if dstip == sock.getsockname():
+        if dstip == listener.getsockname():
             debug1("-- ignored: that's my address!\n")
             sock.close()
             return
@@ -87,6 +144,9 @@ def _main(listener, listenport, use_server, remotename, subnets):
         for s in handlers:
             if s.socks & ready:
                 s.callback()
+        if use_server:
+            mux.callback()
+            mux.check_fullness()
 
 
 def main(listenip, use_server, remotename, subnets):
@@ -96,7 +156,7 @@ def main(listenip, use_server, remotename, subnets):
     if listenip[1]:
         ports = [listenip[1]]
     else:
-        ports = xrange(12300,65536)
+        ports = xrange(12300,9000,-1)
     last_e = None
     bound = False
     debug2('Binding:')
@@ -116,7 +176,9 @@ def main(listenip, use_server, remotename, subnets):
     listenip = listener.getsockname()
     debug1('Listening on %r.\n' % (listenip,))
 
+    fw = FirewallClient(listenip[1], subnets)
+    
     try:
-        return _main(listener, listenip[1], use_server, remotename, subnets)
+        return _main(listener, fw, use_server, remotename)
     finally:
-        iptables_setup(listenip[1], [])
+        fw.done()

firewall.py

@@ -0,0 +1,202 @@
+import subprocess, re
+import helpers
+from helpers import *
+
+
+def ipt_chain_exists(name):
+    argv = ['iptables', '-t', 'nat', '-nL']
+    p = subprocess.Popen(argv, stdout = subprocess.PIPE)
+    for line in p.stdout:
+        if line.startswith('Chain %s ' % name):
+            return True
+    rv = p.wait()
+    if rv:
+        raise Fatal('%r returned %d' % (argv, rv))
+
+
+def ipt(*args):
+    argv = ['iptables', '-t', 'nat'] + list(args)
+    debug1('>> %s\n' % ' '.join(argv))
+    rv = subprocess.call(argv)
+    if rv:
+        raise Fatal('%r returned %d' % (argv, rv))
+
+
+# We name the chain based on the transproxy port number so that it's possible
+# to run multiple copies of sshuttle at the same time.  Of course, the
+# multiple copies shouldn't have overlapping subnets, or only the most-
+# recently-started one will win (because we use "-I OUTPUT 1" instead of
+# "-A OUTPUT").
+def do_iptables(port, subnets):
+    chain = 'sshuttle-%s' % port
+
+    # basic cleanup/setup of chains
+    if ipt_chain_exists(chain):
+        ipt('-D', 'OUTPUT', '-j', chain)
+        ipt('-D', 'PREROUTING', '-j', chain)
+        ipt('-F', chain)
+        ipt('-X', chain)
+
+    if subnets:
+        ipt('-N', chain)
+        ipt('-F', chain)
+        ipt('-I', 'OUTPUT', '1', '-j', chain)
+        ipt('-I', 'PREROUTING', '1', '-j', chain)
+
+        # create new subnet entries
+        for snet,swidth in subnets:
+            ipt('-A', chain, '-j', 'REDIRECT',
+                '--dest', '%s/%s' % (snet,swidth),
+                '-p', 'tcp',
+                '--to-ports', str(port),
+                '-m', 'ttl', '!', '--ttl', '42'  # to prevent infinite loops
+                )
+
+
+def ipfw_rule_exists(n):
+    argv = ['ipfw', 'list']
+    p = subprocess.Popen(argv, stdout = subprocess.PIPE)
+    for line in p.stdout:
+        if line.startswith('%05d ' % n):
+            if line[5:].find('ipttl 42') < 0:
+                raise Fatal('non-sshuttle ipfw rule #%d already exists!' % n)
+            return True
+    rv = p.wait()
+    if rv:
+        raise Fatal('%r returned %d' % (argv, rv))
+
+
+def sysctl_get(name):
+    argv = ['sysctl', '-n', name]
+    p = subprocess.Popen(argv, stdout = subprocess.PIPE)
+    line = p.stdout.readline()
+    rv = p.wait()
+    if rv:
+        raise Fatal('%r returned %d' % (argv, rv))
+    if not line:
+        raise Fatal('%r returned no data' % (argv,))
+    assert(line[-1] == '\n')
+    return line[:-1]
+
+
+def _sysctl_set(name, val):
+    argv = ['sysctl', '-w', '%s=%s' % (name, val)]
+    debug1('>> %s\n' % ' '.join(argv))
+    rv = subprocess.call(argv, stdout = open('/dev/null', 'w'))
+
+
+_oldctls = []
+def sysctl_set(name, val):
+    oldval = sysctl_get(name)
+    if str(val) != str(oldval):
+        _oldctls.append((name, oldval))
+        return _sysctl_set(name, val)
+    
+
+def ipfw(*args):
+    argv = ['ipfw', '-q'] + list(args)
+    debug1('>> %s\n' % ' '.join(argv))
+    rv = subprocess.call(argv)
+    if rv:
+        raise Fatal('%r returned %d' % (argv, rv))
+
+
+def do_ipfw(port, subnets):
+    sport = str(port)
+
+    # cleanup any existing rules
+    if ipfw_rule_exists(port):
+        ipfw('del', sport)
+
+    while _oldctls:
+        (name,oldval) = _oldctls.pop()
+        _sysctl_set(name, oldval)
+
+    if subnets:
+        sysctl_set('net.inet.ip.fw.enable', 1)
+        sysctl_set('net.inet.ip.forwarding', 1)
+        
+        # create new subnet entries
+        for snet,swidth in subnets:
+            ipfw('add', sport, 'fwd', '127.0.0.1,%d' % port,
+                 'log', 'tcp',
+                 'from', 'any', 'to', '%s/%s' % (snet,swidth),
+                 'not', 'ipttl', '42')
+
+
+def program_exists(name):
+    paths = (os.getenv('PATH') or os.defpath).split(os.pathsep)
+    for p in paths:
+        fn = '%s/%s' % (p, name)
+        if os.path.exists(fn):
+            return not os.path.isdir(fn) and os.access(fn, os.X_OK)
+
+
+# This is some voodoo for setting up the kernel's transparent
+# proxying stuff.  If subnets is empty, we just delete our sshuttle rules;
+# otherwise we delete it, then make them from scratch.
+#
+# This code is supposed to clean up after itself by deleting its rules on
+# exit.  In case that fails, it's not the end of the world; future runs will
+# supercede it in the transproxy list, at least, so the leftover rules
+# are hopefully harmless.
+def main(port, subnets):
+    assert(port > 0)
+    assert(port <= 65535)
+
+    if os.getuid() != 0:
+        raise Fatal('you must be root (or enable su/sudo) to set the firewall')
+
+    if program_exists('ipfw'):
+        do_it = do_ipfw
+    elif program_exists('iptables'):
+        do_it = do_iptables
+    else:
+        raise Fatal("can't find either ipfw or iptables; check your PATH")
+
+    # because of limitations of the 'su' command, the *real* stdin/stdout
+    # are both attached to stdout initially.  Clone stdout into stdin so we
+    # can read from it.
+    os.dup2(1, 0)
+
+    debug1('firewall manager ready.\n')
+    sys.stdout.write('READY\n')
+    sys.stdout.flush()
+
+    # ctrl-c shouldn't be passed along to me.  When the main sshuttle dies,
+    # I'll die automatically.
+    os.setsid()
+
+    # we wait until we get some input before creating the rules.  That way,
+    # sshuttle can launch us as early as possible (and get sudo password
+    # authentication as early in the startup process as possible).
+    line = sys.stdin.readline(128)
+    if not line:
+        return  # parent died; nothing to do
+    if line != 'GO\n':
+        raise Fatal('firewall: expected GO but got %r' % line)
+    try:
+        if line:
+            debug1('firewall manager: starting transproxy.\n')
+            do_it(port, subnets)
+            sys.stdout.write('STARTED\n')
+        
+        try:
+            sys.stdout.flush()
+            
+            # Now we wait until EOF or any other kind of exception.  We need
+            # to stay running so that we don't need a *second* password
+            # authentication at shutdown time - that cleanup is important!
+            while sys.stdin.readline(128):
+                pass
+        except IOError:
+            # the parent process died for some reason; he's surely been loud
+            # enough, so no reason to report another error
+            return
+
+    finally:
+        try:
+            debug1('firewall manager: undoing changes.\n')
+        except:
+            pass
+        do_it(port, [])

iptables.py

@@ -1,66 +0,0 @@
-import subprocess, re
-import helpers
-from helpers import *
-
-
-def chain_exists(name):
-    argv = ['iptables', '-t', 'nat', '-nL']
-    p = subprocess.Popen(argv, stdout = subprocess.PIPE)
-    for line in p.stdout:
-        if line.startswith('Chain %s ' % name):
-            return True
-    rv = p.wait()
-    if rv:
-        raise Exception('%r returned %d' % (argv, rv))
-
-
-def ipt(*args):
-    argv = ['iptables', '-t', 'nat'] + list(args)
-    debug1('>> %s\n' % ' '.join(argv))
-    rv = subprocess.call(argv)
-    if rv:
-        raise Exception('%r returned %d' % (argv, rv))
-
-
-# This is some iptables voodoo for setting up the Linux kernel's transparent
-# proxying stuff.  If subnets is empty, we just delete our sshuttle chain;
-# otherwise we delete it, then make it from scratch.
-#
-# We name the chain based on the transproxy port number so that it's possible
-# to run multiple copies of sshuttle at the same time.  Of course, the
-# multiple copies shouldn't have overlapping subnets, or only the most-
-# recently-started one will win (because we use "-I OUTPUT 1" instead of
-# "-A OUTPUT").
-#
-# sshuttle is supposed to clean up after itself by deleting extra chains on
-# exit.  In case that fails, it's not the end of the world; future runs will
-# supercede it in the transproxy list, at least, so the leftover iptables
-# chains are mostly harmless.
-def main(port, subnets):
-    assert(port > 0)
-    assert(port <= 65535)
-
-    chain = 'sshuttle-%s' % port
-
-    # basic cleanup/setup of chains
-    if chain_exists(chain):
-        ipt('-D', 'OUTPUT', '-j', chain)
-        ipt('-D', 'PREROUTING', '-j', chain)
-        ipt('-F', chain)
-        ipt('-X', chain)
-
-    if subnets:
-        ipt('-N', chain)
-        ipt('-F', chain)
-        ipt('-I', 'OUTPUT', '1', '-j', chain)
-        ipt('-I', 'PREROUTING', '1', '-j', chain)
-
-        # create new subnet entries
-        for snet,swidth in subnets:
-            ipt('-A', chain, '-j', 'REDIRECT',
-                '--dest', '%s/%s' % (snet,swidth),
-                '-p', 'tcp',
-                '--to-ports', str(port),
-                '-m', 'ttl', '!', '--ttl', '42'  # to prevent infinite loops
-                )
-    subnets_str = ['%s/%d' % (ip,width) for ip,width in subnets]

main.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 import sys, os, re
-import helpers, options, client, server, iptables
+import helpers, options, client, server, firewall
 from helpers import *
 
 
@@ -45,8 +45,8 @@ def parse_ipport(s):
 
 
 optspec = """
-sshuttle [-l [ip:]port] [-r [username@]sshserver] <subnets...>
+sshuttle [-l [ip:]port] [-r [username@]sshserver[:port]] <subnets...>
-sshuttle --iptables <port> <subnets...>
+sshuttle --firewall <port> <subnets...>
 sshuttle --server
 --
 l,listen=  transproxy to this ip address and port number [default=0]
@@ -54,7 +54,7 @@ r,remote=  ssh hostname (and optional username) of remote sshuttle server
 v,verbose  increase debug message verbosity
 noserver   don't use a separate server process (mostly for debugging)
 server     [internal use only]
-iptables   [internal use only]
+firewall   [internal use only]
 """
 o = options.Options('sshuttle', optspec)
 (opt, flags, extra) = o.parse(sys.argv[1:])
@@ -64,10 +64,10 @@ helpers.verbose = opt.verbose
 try:
     if opt.server:
         sys.exit(server.main())
-    elif opt.iptables:
+    elif opt.firewall:
         if len(extra) < 1:
             o.fatal('at least one argument expected')
-        sys.exit(iptables.main(int(extra[0]),
+        sys.exit(firewall.main(int(extra[0]),
                                parse_subnets(extra[1:])))
     else:
         if len(extra) < 1:
@@ -83,5 +83,6 @@ except Fatal, e:
     log('fatal: %s\n' % e)
     sys.exit(99)
 except KeyboardInterrupt:
-    log('\nKeyboard interrupt: exiting.\n')
+    log('\n')
+    log('Keyboard interrupt: exiting.\n')
     sys.exit(1)

server.py

@@ -1,18 +1,20 @@
 import struct, socket, select
-import ssnet, helpers
+if not globals().get('skip_imports'):
-from ssnet import SockWrapper, Handler, Proxy, Mux, MuxWrapper
+    import ssnet, helpers
-from helpers import *
+    from ssnet import SockWrapper, Handler, Proxy, Mux, MuxWrapper
+    from helpers import *
 
 
 def main():
-    # synchronization header
-    sys.stdout.write('SSHUTTLE0001')
-    sys.stdout.flush()
-
     if helpers.verbose >= 1:
         helpers.logprefix = ' s: '
     else:
         helpers.logprefix = 'server: '
+        
+    # synchronization header
+    sys.stdout.write('SSHUTTLE0001')
+    sys.stdout.flush()
+
     handlers = []
     mux = Mux(socket.fromfd(sys.stdin.fileno(),
                             socket.AF_INET, socket.SOCK_STREAM),
@@ -35,11 +37,15 @@ def main():
         handlers = filter(lambda s: s.ok, handlers)
         for s in handlers:
             s.pre_select(r,w,x)
-        debug2('Waiting: %d[%d,%d,%d]...\n' 
+        debug2('Waiting: %d[%d,%d,%d] (fullness=%d/%d)...\n' 
-               % (len(handlers), len(r), len(w), len(x)))
+               % (len(handlers), len(r), len(w), len(x),
+                  mux.fullness, mux.too_full))
         (r,w,x) = select.select(r,w,x)
         #log('r=%r w=%r x=%r\n' % (r,w,x))
         ready = set(r) | set(w) | set(x)
         for s in handlers:
+            #debug2('check: %r: %r\n' % (s, s.socks & ready))
             if s.socks & ready:
                 s.callback()
+        mux.check_fullness()
+        mux.callback()

ssh.py

@@ -1,38 +1,63 @@
-import sys, os, re, subprocess, socket
+import sys, os, re, subprocess, socket, zlib
 import helpers
+from helpers import *
 
-def connect(rhost):
+
+def readfile(name):
+    basedir = os.path.dirname(os.path.abspath(sys.argv[0]))
+    fullname = os.path.join(basedir, name)
+    return open(fullname, 'rb').read()
+
+
+def empackage(z, filename):
+    content = z.compress(readfile(filename))
+    content += z.flush(zlib.Z_SYNC_FLUSH)
+    return '%s\n%d\n%s' % (filename,len(content), content)
+
+
+def connect(rhostport):
     main_exe = sys.argv[0]
-    nicedir = os.path.split(os.path.abspath(main_exe))[0]
+    l = (rhostport or '').split(':', 1)
-    nicedir = re.sub(r':', "_", nicedir)
+    rhost = l[0]
+    portl = []
+    if len(l) > 1:
+        portl = ['-p', str(int(l[1]))]
+
     if rhost == '-':
         rhost = None
+
+    z = zlib.compressobj(1)
+    content = readfile('assembler.py')
+    content2 = (empackage(z, 'helpers.py') +
+                empackage(z, 'ssnet.py') +
+                empackage(z, 'server.py') +
+                "\n")
+    
+    pyscript = r"""
+                import sys;
+                skip_imports=1;
+                verbosity=%d;
+                exec compile(sys.stdin.read(%d), "assembler.py", "exec")
+                """ % (helpers.verbose or 0, len(content))
+    pyscript = re.sub(r'\s+', ' ', pyscript.strip())
+
+        
     if not rhost:
-        argv = ['sshuttle', '--server'] + ['-v']*(helpers.verbose or 0)
+        argv = ['python', '-c', pyscript]
     else:
-        # WARNING: shell quoting security holes are possible here, so we
+        argv = ['ssh'] + portl + [rhost, '--', "python -c '%s'" % pyscript]
-        # have to be super careful.  We have to use 'sh -c' because
-        # csh-derived shells can't handle PATH= notation.  We can't
-        # set PATH in advance, because ssh probably replaces it.  We
-        # can't exec *safely* using argv, because *both* ssh and 'sh -c'
-        # allow shellquoting.  So we end up having to double-shellquote
-        # stuff here.
-        escapedir = re.sub(r'([^\w/])', r'\\\\\\\1', nicedir)
-        cmd = r"""
-                   sh -c PATH=%s:'$PATH exec sshuttle --server%s'
-               """ % (escapedir, ' -v' * (helpers.verbose or 0))
-        argv = ['ssh', rhost, '--', cmd.strip()]
     (s1,s2) = socket.socketpair()
     def setup():
         # runs in the child process
         s2.close()
-        if not rhost:
-            os.environ['PATH'] = ':'.join([nicedir,
-                                           os.environ.get('PATH', '')])
         os.setsid()
     s1a,s1b = os.dup(s1.fileno()), os.dup(s1.fileno())
     s1.close()
-    p = subprocess.Popen(argv, stdin=s1a, stdout=s1b, preexec_fn=setup)
+    debug2('executing: %r\n' % argv)
+    p = subprocess.Popen(argv, stdin=s1a, stdout=s1b, preexec_fn=setup,
+                         close_fds=True)
     os.close(s1a)
     os.close(s1b)
+    s2.sendall(content)
+    s2.sendall(content2)
     return p, s2

ssnet.py

@@ -1,5 +1,6 @@
 import struct, socket, errno, select
-from helpers import *
+if not globals().get('skip_imports'):
+    from helpers import *
 
 HDR_LEN = 8
 
@@ -71,16 +72,20 @@ class SockWrapper:
     def try_connect(self):
         if not self.connect_to:
             return  # already connected
-        self.rsock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
         self.rsock.setblocking(False)
         try:
             self.rsock.connect(self.connect_to)
+            # connected successfully (Linux)
             self.connect_to = None
         except socket.error, e:
             if e.args[0] in [errno.EINPROGRESS, errno.EALREADY]:
                 pass  # not connected yet
+            elif e.args[0] == errno.EISCONN:
+                # connected successfully (BSD)
+                self.connect_to = None
             elif e.args[0] in [errno.ECONNREFUSED, errno.ETIMEDOUT]:
                 # a "normal" kind of error
+                self.connect_to = None
                 self.seterr(e)
             else:
                 raise  # error we've never heard of?!  barf completely.
@@ -100,6 +105,9 @@ class SockWrapper:
             except socket.error, e:
                 self.seterr(e)
 
+    def too_full(self):
+        return False  # fullness is determined by the socket's select() state
+
     def uwrite(self, buf):
         if self.connect_to:
             return 0  # still connecting
@@ -180,14 +188,16 @@ class Proxy(Handler):
         if self.wrap1.connect_to:
             w.add(self.wrap1.rsock)
         elif self.wrap1.buf:
-            w.add(self.wrap2.wsock)
+            if not self.wrap2.too_full():
+                w.add(self.wrap2.wsock)
         elif not self.wrap1.shut_read:
             r.add(self.wrap1.rsock)
 
         if self.wrap2.connect_to:
             w.add(self.wrap2.rsock)
         elif self.wrap2.buf:
-            w.add(self.wrap1.wsock)
+            if not self.wrap1.too_full():
+                w.add(self.wrap1.wsock)
         elif not self.wrap2.shut_read:
             r.add(self.wrap2.rsock)
 
@@ -214,6 +224,8 @@ class Mux(Handler):
         self.want = 0
         self.inbuf = ''
         self.outbuf = []
+        self.fullness = 0
+        self.too_full = False
         self.send(0, CMD_PING, 'chicken')
 
     def next_channel(self):
@@ -224,16 +236,29 @@ class Mux(Handler):
                 self.chani = 1
             if not self.channels.get(self.chani):
                 return self.chani
+
+    def amount_queued(self):
+        return sum(len(b) for b in self.outbuf)
             
+    def check_fullness(self):
+        if self.fullness > 32768:
+            if not self.too_full:
+                self.send(0, CMD_PING, 'rttest')
+            self.too_full = True
+        #ob = []
+        #for b in self.outbuf:
+        #    (s1,s2,c) = struct.unpack('!ccH', b[:4])
+        #    ob.append(c)
+        #log('outbuf: %d %r\n' % (self.amount_queued(), ob))
+        
     def send(self, channel, cmd, data):
         data = str(data)
         assert(len(data) <= 65535)
         p = struct.pack('!ccHHH', 'S', 'S', channel, cmd, len(data)) + data
         self.outbuf.append(p)
-        debug2(' > channel=%d cmd=%s len=%d\n' 
+        debug2(' > channel=%d cmd=%s len=%d (fullness=%d)\n'
-               % (channel, cmd_to_name[cmd], len(data)))
+               % (channel, cmd_to_name[cmd], len(data), self.fullness))
-        #log('Mux: send queue is %d/%d\n' 
+        self.fullness += len(data)
-        #    % (len(self.outbuf), sum(len(b) for b in self.outbuf)))
 
     def got_packet(self, channel, cmd, data):
         debug2('<  channel=%d cmd=%s len=%d\n' 
@@ -242,6 +267,8 @@ class Mux(Handler):
             self.send(0, CMD_PONG, data)
         elif cmd == CMD_PONG:
             debug2('received PING response\n')
+            self.too_full = False
+            self.fullness = 0
         elif cmd == CMD_EXIT:
             self.ok = False
         elif cmd == CMD_CONNECT:
@@ -256,6 +283,7 @@ class Mux(Handler):
         self.wsock.setblocking(False)
         if self.outbuf and self.outbuf[0]:
             wrote = _nb_clean(os.write, self.wsock.fileno(), self.outbuf[0])
+            debug2('mux wrote: %d/%d\n' % (wrote, len(self.outbuf[0])))
             if wrote:
                 self.outbuf[0] = self.outbuf[0][wrote:]
         while self.outbuf and not self.outbuf[0]:
@@ -263,7 +291,10 @@ class Mux(Handler):
 
     def fill(self):
         self.rsock.setblocking(False)
-        b = _nb_clean(os.read, self.rsock.fileno(), 32768)
+        try:
+            b = _nb_clean(os.read, self.rsock.fileno(), 32768)
+        except OSError, e:
+            raise Fatal('other end: %r' % e)
         #log('<<< %r\n' % b)
         if b == '': # EOF
             self.ok = False
@@ -308,6 +339,7 @@ class MuxWrapper(SockWrapper):
         self.mux = mux
         self.channel = channel
         self.mux.channels[channel] = self.got_packet
+        self.socks = []
         debug2('new channel: %d\n' % channel)
 
     def __del__(self):
@@ -326,9 +358,14 @@ class MuxWrapper(SockWrapper):
             self.shut_write = True
             self.mux.send(self.channel, CMD_EOF, '')
 
+    def too_full(self):
+        return self.mux.too_full
+
     def uwrite(self, buf):
-        if len(buf) > 65535:
+        if self.mux.too_full:
-            buf = buf[:32768]
+            return 0  # too much already enqueued
+        if len(buf) > 2048:
+            buf = buf[:2048]
         self.mux.send(self.channel, CMD_DATA, buf)
         return len(buf)
 
@@ -354,6 +391,7 @@ class MuxWrapper(SockWrapper):
 def connect_dst(ip, port):
     debug2('Connecting to %s:%d\n' % (ip, port))
     outsock = socket.socket()
+    outsock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
     return SockWrapper(outsock, outsock,
                        connect_to = (ip,port),
                        peername = '%s:%d' % (ip,port))