New release 0.73.

This needs to be redone; will do so at a later stage.

.gitignore

@@ -1,2 +1,6 @@
 *.pyc
 *~
+*.8
+/.do_built
+/.do_built.dir
+/.redo

.travis.yml

@@ -0,0 +1,11 @@
+language: python
+python:
+- 2.7
+- 3.5
+- pypy
+
+install:
+    - travis_retry pip install -q pytest mock
+
+script:
+  - PYTHONPATH=. py.test

MANIFEST.in

@@ -0,0 +1,5 @@
+include *.txt
+include *.rst
+include *.py
+include MANIFEST.in
+recursive-include sshuttle *.py

README.md

@@ -1,163 +0,0 @@
-sshuttle: where transparent proxy meets VPN meets ssh
-=====================================================
-
-I just spent an afternoon working on a new kind of VPN.  You can get
-the first release, <a href="http://github.com/apenwarr/sshuttle">sshuttle
-0.10, on github</a>.
-
-As far as I know, sshuttle is the only program that solves the following
-common case:
-
- - Your client machine (or router) is Linux.
-
- - You have access to a remote network via ssh.
-
- - You don't necessarily have admin access on the remote network.
-
- - The remote network has no VPN, or only stupid/complex VPN
-    protocols (IPsec, PPTP, etc). Or maybe you <i>are</i> the
-    admin and you just got frustrated with the awful state of
-    VPN tools.
-
- - You don't want to create an ssh port forward for every
-    single host/port on the remote network.
-
- - You hate openssh's port forwarding because it's randomly
-    slow and/or stupid.
- 
- - You can't use openssh's PermitTunnel feature because
-    it's disabled by default on openssh servers; plus it does
-    TCP-over-TCP, which has terrible performance (see below).
-    
-
-Prerequisites
--------------
-
- - sudo, su, or logged in as root on your client machine.
-   (The server doesn't need admin access.)
-   
- - If you use Linux on your client machine:
-   iptables installed on the client, including at
-   least the iptables DNAT, REDIRECT, and ttl modules. 
-   These are installed by default on most Linux distributions. 
-   (The server doesn't need iptables and doesn't need to be
-   Linux.)
-   
- - If you use MacOS or BSD on your client machine:
-   Your kernel needs to be compiled with IPFIREWALL_FORWARD
-   (MacOS has this by default) and you need to have ipfw
-   available. (The server doesn't need to be MacOS or BSD.)
-
-
-This is how you use it:
------------------------
-
- - <tt>git clone git://github.com/apenwarr/sshuttle</tt>
-    on your client and server machines. The server can be
-    any ssh server with python available; the client must
-    be Linux with iptables, and you'll need root or sudo
-    access.
-
- - <tt>./sshuttle -r username@sshserver 0.0.0.0/0 -vv</tt>
-
-That's it!  Now your local machine can access the remote network as if you
-were right there!  And if your "client" machine is a router, everyone on
-your local network can make connections to your remote network.
-
-You don't need to install sshuttle on the remote server;
-the remote server just needs to have python available. 
-sshuttle will automatically upload and run its source code
-to the remote python interpreter.
-
-This creates a transparent proxy server on your local machine for all IP
-addresses that match 0.0.0.0/0.  (You can use more specific IP addresses if
-you want; use any number of IP addresses or subnets to change which
-addresses get proxied.  Using 0.0.0.0/0 proxies <i>everything</i>, which is
-interesting if you don't trust the people on your local network.)
-
-Any TCP session you initiate to one of the proxied IP addresses will be
-captured by sshuttle and sent over an ssh session to the remote copy of
-sshuttle, which will then regenerate the connection on that end, and funnel
-the data back and forth through ssh.
-
-Fun, right?  A poor man's instant VPN, and you don't even have to have
-admin access on the server.
-
-
-Theory of Operation
--------------------
-
-sshuttle is not exactly a VPN, and not exactly port forwarding.  It's kind
-of both, and kind of neither.
-
-It's like a VPN, since it can forward every port on an entire network, not
-just ports you specify.  Conveniently, it lets you use the "real" IP
-addresses of each host rather than faking port numbers on localhost.
-
-On the other hand, the way it *works* is more like ssh port forwarding than
-a VPN.  Normally, a VPN forwards your data one packet at a time, and
-doesn't care about individual connections; ie. it's "stateless" with respect
-to the traffic.  sshuttle is the opposite of stateless; it tracks every
-single connection.
-
-You could compare sshuttle to something like the old <a
-href="http://en.wikipedia.org/wiki/Slirp">Slirp</a> program, which was a
-userspace TCP/IP implementation that did something similar.  But it
-operated on a packet-by-packet basis on the client side, reassembling the
-packets on the server side.  That worked okay back in the "real live serial
-port" days, because serial ports had predictable latency and buffering.
-
-But you can't safely just forward TCP packets over a TCP session (like ssh),
-because TCP's performance depends fundamentally on packet loss; it
-<i>must</i> experience packet loss in order to know when to slow down!  At
-the same time, the outer TCP session (ssh, in this case) is a reliable
-transport, which means that what you forward through the tunnel <i>never</i>
-experiences packet loss.  The ssh session itself experiences packet loss, of
-course, but TCP fixes it up and ssh (and thus you) never know the
-difference.  But neither does your inner TCP session, and extremely screwy
-performance ensues.
-
-sshuttle assembles the TCP stream locally, multiplexes it statefully over
-an ssh session, and disassembles it back into packets at the other end.  So
-it never ends up doing TCP-over-TCP.  It's just data-over-TCP, which is
-safe.
-
-
-Useless Trivia
---------------
-
-Back in 1998 (12 years ago!  Yikes!), I released the first version of <a
-href="http://alumnit.ca/wiki/?TunnelVisionReadMe">Tunnel Vision</a>, a
-semi-intelligent VPN client for Linux.  Unfortunately, I made two big mistakes: 
-I implemented the key exchange myself (oops), and I ended up doing
-TCP-over-TCP (double oops).  The resulting program worked okay - and people
-used it for years - but the performance was always a bit funny.  And nobody
-ever found any security flaws in my key exchange, either, but that doesn't
-mean anything. :)
-
-The same year, dcoombs and I also released Fast Forward, a proxy server
-supporting transparent proxying.  Among other things, we used it for
-automatically splitting traffic across more than one Internet connection (a
-tool we called "Double Vision").
-
-I was still in university at the time.  A couple years after that, one of my
-professors was working with some graduate students on the technology that
-would eventually become <a href="http://www.slipstream.com/">Slipstream
-Internet Acceleration</a>.  He asked me to do a contract for him to build an
-initial prototype of a transparent proxy server for mobile networks.  The
-idea was similar to sshuttle: if you reassemble and then disassemble the TCP
-packets, you can reduce latency and improve performance vs.  just forwarding
-the packets over a plain VPN or mobile network.  (It's unlikely that any of
-my code has persisted in the Slipstream product today, but the concept is
-still pretty cool.  I'm still horrified that people use plain TCP on
-complex mobile networks with crazily variable latency, for which it was
-never really intended.)
-
-That project I did for Slipstream was what first gave me the idea to merge
-the concepts of Fast Forward, Double Vision, and Tunnel Vision into a single
-program that was the best of all worlds.  And here we are, at last, 10 years
-later.  You're welcome.
-
---
-Avery Pennarun <apenwarr@gmail.com>
-

README.rst

@@ -0,0 +1,258 @@
+sshuttle: where transparent proxy meets VPN meets ssh
+=====================================================
+
+As far as I know, sshuttle is the only program that solves the following
+common case:
+
+- Your client machine (or router) is Linux, FreeBSD, or MacOS.
+
+- You have access to a remote network via ssh.
+
+- You don't necessarily have admin access on the remote network.
+
+- The remote network has no VPN, or only stupid/complex VPN
+  protocols (IPsec, PPTP, etc). Or maybe you *are* the
+  admin and you just got frustrated with the awful state of
+  VPN tools.
+
+- You don't want to create an ssh port forward for every
+  single host/port on the remote network.
+
+- You hate openssh's port forwarding because it's randomly
+  slow and/or stupid.
+
+- You can't use openssh's PermitTunnel feature because
+  it's disabled by default on openssh servers; plus it does
+  TCP-over-TCP, which has terrible performance (see below).
+
+
+Client side Requirements
+------------------------
+
+- sudo, or logged in as root on your client machine.
+  (The server doesn't need admin access.)
+- Python 2.7 or Python 3.5.
+
++-------+--------+------------+-----------------------------------------------+
+| OS    | Method | Features   | Requirements                                  |
++=======+========+============+===============================================+
+| Linux | NAT    | * IPv4 TCP + iptables DNAT, REDIRECT, and ttl modules.     |
++       +--------+------------+-----------------------------------------------+
+|       | TPROXY | * IPv4 TCP + Linux with TPROXY support.                    |
+|       |        | * IPv4 UDP + Python 3.5 preferred (see below).             |
+|       |        | * IPv6 TCP +                                               |
+|       |        | * IPv6 UDP +                                               |
++-------+--------+------------+-----------------------------------------------+
+| BSD   | IPFW   | * IPv4 TCP | Your kernel needs to be compiled with         |
+|       |        |            | `IPFIREWALL_FORWARD` and you need to have ipfw|
+|       |        |            | available.                                    |
++-------+--------+------------+-----------------------------------------------+
+| MacOS | PF     | * IPv4 TCP + You need to have the pfctl command.           |
++-------+--------+------------+-----------------------------------------------+
+
+The IPFW method is depreciated. It was originally required for MacOS support,
+however is no longer maintained. It is likely to get removed from future
+versions of sshuttle.
+
+
+Server side Requirements
+------------------------
+Python 2.7 or Python 3.5. This should match what is used on the client side.
+
+
+Additional Suggested Software
+-----------------------------
+
+- You may want to use autossh, available in various package management
+  systems
+
+
+Additional information for TPROXY
+---------------------------------
+TPROXY is the only method that supports full support of IPv6 and UDP.
+
+Full UDP or DNS support with the TPROXY method requires the `recvmsg()`
+syscall. This is not available in Python 2.7, however is in Python 3.5 and
+later.
+
+- For Python 2.7, you need PyXAPI, available here:
+  http://www.pps.univ-paris-diderot.fr/~ylg/PyXAPI/
+
+There are some things you need to consider for TPROXY to work:
+
+1. The following commands need to be run first as root. This only needs to be
+   done once after booting up::
+
+       ip route add local default dev lo table 100
+       ip rule add fwmark 1 lookup 100
+       ip -6 route add local default dev lo table 100
+       ip -6 rule add fwmark 1 lookup 100
+
+2. The client needs to be run as root. e.g.::
+
+       sudo SSH_AUTH_SOCK="$SSH_AUTH_SOCK" $HOME/tree/sshuttle.tproxy/sshuttle  --method=tproxy ...
+
+3. You do need the `--method=tproxy` parameter, as above.
+
+4. The routes for the outgoing packets must already exist. For example, if your
+   connection does not have IPv6 support, no IPv6 routes will exist, IPv6
+   packets will not be generated and sshuttle cannot intercept them. Add some
+   dummy routes to external interfaces. Make sure they get removed however
+   after sshuttle exits.
+
+
+Obtaining sshuttle
+------------------
+
+- From PyPI::
+
+      pip install sshuttle
+
+- Clone::
+
+      git clone https://github.com/sshuttle/sshuttle.git
+      ./setup.py install
+
+
+Usage
+-----
+
+- Forward all traffic::
+
+      sshuttle -r username@sshserver 0.0.0.0/0 -vv
+
+- By default sshuttle will automatically choose a method to use. Override with
+  the `--method=` parameter.
+
+- There is a shortcut for 0.0.0.0/0 for those that value
+  their wrists::
+
+      sshuttle -r username@sshserver 0/0 -vv
+
+- If you would also like your DNS queries to be proxied
+  through the DNS server of the server you are connect to::
+
+      sshuttle --dns -vvr username@sshserver 0/0
+
+  The above is probably what you want to use to prevent
+  local network attacks such as Firesheep and friends.
+
+(You may be prompted for one or more passwords; first, the local password to
+become root using sudo, and then the remote ssh password.  Or you might have
+sudo and ssh set up to not require passwords, in which case you won't be
+prompted at all.)
+
+
+Usage Notes
+-----------
+
+That's it!  Now your local machine can access the remote network as if you
+were right there.  And if your "client" machine is a router, everyone on
+your local network can make connections to your remote network.
+
+You don't need to install sshuttle on the remote server;
+the remote server just needs to have python available. 
+sshuttle will automatically upload and run its source code
+to the remote python interpreter.
+
+This creates a transparent proxy server on your local machine for all IP
+addresses that match 0.0.0.0/0.  (You can use more specific IP addresses if
+you want; use any number of IP addresses or subnets to change which
+addresses get proxied.  Using 0.0.0.0/0 proxies *everything*, which is
+interesting if you don't trust the people on your local network.)
+
+Any TCP session you initiate to one of the proxied IP addresses will be
+captured by sshuttle and sent over an ssh session to the remote copy of
+sshuttle, which will then regenerate the connection on that end, and funnel
+the data back and forth through ssh.
+
+Fun, right?  A poor man's instant VPN, and you don't even have to have
+admin access on the server.
+
+
+Support
+-------
+
+Mailing list:
+
+* Subscribe by sending a message to <sshuttle+subscribe@googlegroups.com>
+* List archives are at: http://groups.google.com/group/sshuttle
+
+Issue tracker and pull requests at github:
+
+* https://github.com/sshuttle/sshuttle
+
+
+Theory of Operation
+-------------------
+
+sshuttle is not exactly a VPN, and not exactly port forwarding.  It's kind
+of both, and kind of neither.
+
+It's like a VPN, since it can forward every port on an entire network, not
+just ports you specify.  Conveniently, it lets you use the "real" IP
+addresses of each host rather than faking port numbers on localhost.
+
+On the other hand, the way it *works* is more like ssh port forwarding than
+a VPN.  Normally, a VPN forwards your data one packet at a time, and
+doesn't care about individual connections; ie. it's "stateless" with respect
+to the traffic.  sshuttle is the opposite of stateless; it tracks every
+single connection.
+
+You could compare sshuttle to something like the old `Slirp
+<http://en.wikipedia.org/wiki/Slirp>`_ program, which was a userspace TCP/IP
+implementation that did something similar.  But it operated on a
+packet-by-packet basis on the client side, reassembling the packets on the
+server side.  That worked okay back in the "real live serial port" days,
+because serial ports had predictable latency and buffering.
+
+But you can't safely just forward TCP packets over a TCP session (like ssh),
+because TCP's performance depends fundamentally on packet loss; it
+*must* experience packet loss in order to know when to slow down!  At
+the same time, the outer TCP session (ssh, in this case) is a reliable
+transport, which means that what you forward through the tunnel *never*
+experiences packet loss.  The ssh session itself experiences packet loss, of
+course, but TCP fixes it up and ssh (and thus you) never know the
+difference.  But neither does your inner TCP session, and extremely screwy
+performance ensues.
+
+sshuttle assembles the TCP stream locally, multiplexes it statefully over
+an ssh session, and disassembles it back into packets at the other end.  So
+it never ends up doing TCP-over-TCP.  It's just data-over-TCP, which is
+safe.
+
+
+Useless Trivia
+--------------
+This section written by Avery Pennarun <apenwarr@gmail.com>.
+
+Back in 1998 (12 years ago!  Yikes!), I released the first version of `Tunnel
+Vision <http://alumnit.ca/wiki/?TunnelVisionReadMe>`_, a semi-intelligent VPN
+client for Linux.  Unfortunately, I made two big mistakes: I implemented the
+key exchange myself (oops), and I ended up doing TCP-over-TCP (double oops).
+The resulting program worked okay - and people used it for years - but the
+performance was always a bit funny.  And nobody ever found any security flaws
+in my key exchange, either, but that doesn't mean anything. :)
+
+The same year, dcoombs and I also released Fast Forward, a proxy server
+supporting transparent proxying.  Among other things, we used it for
+automatically splitting traffic across more than one Internet connection (a
+tool we called "Double Vision").
+
+I was still in university at the time.  A couple years after that, one of my
+professors was working with some graduate students on the technology that would
+eventually become `Slipstream Internet Acceleration
+<http://www.slipstream.com/>`_.  He asked me to do a contract for him to build
+an initial prototype of a transparent proxy server for mobile networks.  The
+idea was similar to sshuttle: if you reassemble and then disassemble the TCP
+packets, you can reduce latency and improve performance vs.  just forwarding
+the packets over a plain VPN or mobile network.  (It's unlikely that any of my
+code has persisted in the Slipstream product today, but the concept is still
+pretty cool.  I'm still horrified that people use plain TCP on complex mobile
+networks with crazily variable latency, for which it was never really
+intended.)
+
+That project I did for Slipstream was what first gave me the idea to merge
+the concepts of Fast Forward, Double Vision, and Tunnel Vision into a single
+program that was the best of all worlds.  And here we are, at last, 10 years
+later.  You're welcome.

VERSION.txt

@@ -0,0 +1 @@
+0.73

assembler.py

@@ -1,26 +0,0 @@
-import sys, zlib
-
-z = zlib.decompressobj()
-mainmod = sys.modules[__name__]
-while 1:
-    name = sys.stdin.readline().strip()
-    if name:
-        nbytes = int(sys.stdin.readline())
-        if verbosity >= 2:
-            sys.stderr.write('server: assembling %r (%d bytes)\n'
-                             % (name, nbytes))
-        content = z.decompress(sys.stdin.read(nbytes))
-        exec compile(content, name, "exec")
-
-        # FIXME: this crushes everything into a single module namespace,
-        # then makes each of the module names point at this one. Gross.
-        assert(name.endswith('.py'))
-        modname = name[:-3]
-        mainmod.__dict__[modname] = mainmod
-    else:
-        break
-
-verbose = verbosity
-sys.stderr.flush()
-sys.stdout.flush()
-main()

client.py

@@ -1,219 +0,0 @@
-import struct, socket, select, subprocess, errno, re
-import helpers, ssnet, ssh
-from ssnet import SockWrapper, Handler, Proxy, Mux, MuxWrapper
-from helpers import *
-
-def original_dst(sock):
-    try:
-        SO_ORIGINAL_DST = 80
-        SOCKADDR_MIN = 16
-        sockaddr_in = sock.getsockopt(socket.SOL_IP,
-                                      SO_ORIGINAL_DST, SOCKADDR_MIN)
-        (proto, port, a,b,c,d) = struct.unpack('!HHBBBB', sockaddr_in[:8])
-        assert(socket.htons(proto) == socket.AF_INET)
-        ip = '%d.%d.%d.%d' % (a,b,c,d)
-        return (ip,port)
-    except socket.error, e:
-        if e.args[0] == errno.ENOPROTOOPT:
-            return sock.getsockname()
-        raise
-
-
-class FirewallClient:
-    def __init__(self, port, subnets):
-        self.port = port
-        self.auto_nets = []
-        self.subnets = subnets
-        argvbase = ([sys.argv[0]] +
-                    ['-v'] * (helpers.verbose or 0) +
-                    ['--firewall', str(port)])
-        argv_tries = [
-            ['sudo'] + argvbase,
-            ['su', '-c', ' '.join(argvbase)],
-            argvbase
-        ]
-
-        # we can't use stdin/stdout=subprocess.PIPE here, as we normally would,
-        # because stupid Linux 'su' requires that stdin be attached to a tty.
-        # Instead, attach a *bidirectional* socket to its stdout, and use
-        # that for talking in both directions.
-        (s1,s2) = socket.socketpair()
-        def setup():
-            # run in the child process
-            s2.close()
-        e = None
-        for argv in argv_tries:
-            try:
-                self.p = subprocess.Popen(argv, stdout=s1, preexec_fn=setup)
-                e = None
-                break
-            except OSError, e:
-                pass
-        self.argv = argv
-        s1.close()
-        self.pfile = s2.makefile('wb+')
-        if e:
-            log('Spawning firewall manager: %r\n' % self.argv)
-            raise Fatal(e)
-        line = self.pfile.readline()
-        self.check()
-        if line != 'READY\n':
-            raise Fatal('%r expected READY, got %r' % (self.argv, line))
-
-    def check(self):
-        rv = self.p.poll()
-        if rv:
-            raise Fatal('%r returned %d' % (self.argv, rv))
-
-    def start(self):
-        self.pfile.write('ROUTES\n')
-        for (ip,width) in self.subnets+self.auto_nets:
-            self.pfile.write('%s,%d\n' % (ip, width))
-        self.pfile.write('GO\n')
-        self.pfile.flush()
-        line = self.pfile.readline()
-        self.check()
-        if line != 'STARTED\n':
-            raise Fatal('%r expected STARTED, got %r' % (self.argv, line))
-
-    def sethostip(self, hostname, ip):
-        assert(not re.search(r'[^-\w]', hostname))
-        assert(not re.search(r'[^0-9.]', ip))
-        self.pfile.write('HOST %s,%s\n' % (hostname, ip))
-        self.pfile.flush()
-
-    def done(self):
-        self.pfile.close()
-        rv = self.p.wait()
-        if rv:
-            raise Fatal('cleanup: %r returned %d' % (self.argv, rv))
-
-
-def _main(listener, fw, use_server, remotename, seed_hosts, auto_nets):
-    handlers = []
-    if use_server:
-        if helpers.verbose >= 1:
-            helpers.logprefix = 'c : '
-        else:
-            helpers.logprefix = 'client: '
-        (serverproc, serversock) = ssh.connect(remotename)
-        mux = Mux(serversock, serversock)
-        handlers.append(mux)
-
-        expected = 'SSHUTTLE0001'
-        initstring = serversock.recv(len(expected))
-        
-        rv = serverproc.poll()
-        if rv:
-            raise Fatal('server died with error code %d' % rv)
-            
-        if initstring != expected:
-            raise Fatal('expected server init string %r; got %r'
-                            % (expected, initstring))
-
-    def onroutes(routestr):
-        if auto_nets:
-            for line in routestr.strip().split('\n'):
-                (ip,width) = line.split(',', 1)
-                fw.auto_nets.append((ip,int(width)))
-
-        # we definitely want to do this *after* starting ssh, or we might end
-        # up intercepting the ssh connection!
-        #
-        # Moreover, now that we have the --auto-nets option, we have to wait
-        # for the server to send us that message anyway.  Even if we haven't
-        # set --auto-nets, we might as well wait for the message first, then
-        # ignore its contents.
-        mux.got_routes = None
-        fw.start()
-    mux.got_routes = onroutes
-
-    def onhostlist(hostlist):
-        debug2('got host list: %r\n' % hostlist)
-        for line in hostlist.strip().split():
-            if line:
-                name,ip = line.split(',', 1)
-                fw.sethostip(name, ip)
-    mux.got_host_list = onhostlist
-
-    def onaccept():
-        sock,srcip = listener.accept()
-        dstip = original_dst(sock)
-        debug1('Accept: %r:%r -> %r:%r.\n' % (srcip[0],srcip[1],
-                                           dstip[0],dstip[1]))
-        if dstip == listener.getsockname():
-            debug1("-- ignored: that's my address!\n")
-            sock.close()
-            return
-        if use_server:
-            chan = mux.next_channel()
-            mux.send(chan, ssnet.CMD_CONNECT, '%s,%s' % dstip)
-            outwrap = MuxWrapper(mux, chan)
-        else:
-            outwrap = ssnet.connect_dst(dstip[0], dstip[1])
-        handlers.append(Proxy(SockWrapper(sock, sock), outwrap))
-    handlers.append(Handler([listener], onaccept))
-
-    if seed_hosts != None:
-        debug1('seed_hosts: %r\n' % seed_hosts)
-        mux.send(0, ssnet.CMD_HOST_REQ, '\n'.join(seed_hosts))
-    
-    while 1:
-        if use_server:
-            rv = serverproc.poll()
-            if rv:
-                raise Fatal('server died with error code %d' % rv)
-        
-        r = set()
-        w = set()
-        x = set()
-        handlers = filter(lambda s: s.ok, handlers)
-        for s in handlers:
-            s.pre_select(r,w,x)
-        debug2('Waiting: %d[%d,%d,%d]...\n' 
-            % (len(handlers), len(r), len(w), len(x)))
-        (r,w,x) = select.select(r,w,x)
-        #log('r=%r w=%r x=%r\n' % (r,w,x))
-        ready = set(r) | set(w) | set(x)
-        for s in handlers:
-            if s.socks & ready:
-                s.callback()
-        if use_server:
-            mux.callback()
-            mux.check_fullness()
-
-
-def main(listenip, use_server, remotename, seed_hosts, auto_nets, subnets):
-    debug1('Starting sshuttle proxy.\n')
-    listener = socket.socket()
-    listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-    if listenip[1]:
-        ports = [listenip[1]]
-    else:
-        ports = xrange(12300,9000,-1)
-    last_e = None
-    bound = False
-    debug2('Binding:')
-    for port in ports:
-        debug2(' %d' % port)
-        try:
-            listener.bind((listenip[0], port))
-            bound = True
-            break
-        except socket.error, e:
-            last_e = e
-    debug2('\n')
-    if not bound:
-        assert(last_e)
-        raise last_e
-    listener.listen(10)
-    listenip = listener.getsockname()
-    debug1('Listening on %r.\n' % (listenip,))
-
-    fw = FirewallClient(listenip[1], subnets)
-    
-    try:
-        return _main(listener, fw, use_server, remotename,
-                     seed_hosts, auto_nets)
-    finally:
-        fw.done()

firewall.py

@@ -1,268 +0,0 @@
-import subprocess, re, errno
-import helpers
-from helpers import *
-
-
-def ipt_chain_exists(name):
-    argv = ['iptables', '-t', 'nat', '-nL']
-    p = subprocess.Popen(argv, stdout = subprocess.PIPE)
-    for line in p.stdout:
-        if line.startswith('Chain %s ' % name):
-            return True
-    rv = p.wait()
-    if rv:
-        raise Fatal('%r returned %d' % (argv, rv))
-
-
-def ipt(*args):
-    argv = ['iptables', '-t', 'nat'] + list(args)
-    debug1('>> %s\n' % ' '.join(argv))
-    rv = subprocess.call(argv)
-    if rv:
-        raise Fatal('%r returned %d' % (argv, rv))
-
-
-# We name the chain based on the transproxy port number so that it's possible
-# to run multiple copies of sshuttle at the same time.  Of course, the
-# multiple copies shouldn't have overlapping subnets, or only the most-
-# recently-started one will win (because we use "-I OUTPUT 1" instead of
-# "-A OUTPUT").
-def do_iptables(port, subnets):
-    chain = 'sshuttle-%s' % port
-
-    # basic cleanup/setup of chains
-    if ipt_chain_exists(chain):
-        ipt('-D', 'OUTPUT', '-j', chain)
-        ipt('-D', 'PREROUTING', '-j', chain)
-        ipt('-F', chain)
-        ipt('-X', chain)
-
-    if subnets:
-        ipt('-N', chain)
-        ipt('-F', chain)
-        ipt('-I', 'OUTPUT', '1', '-j', chain)
-        ipt('-I', 'PREROUTING', '1', '-j', chain)
-
-        # create new subnet entries
-        for snet,swidth in subnets:
-            ipt('-A', chain, '-j', 'REDIRECT',
-                '--dest', '%s/%s' % (snet,swidth),
-                '-p', 'tcp',
-                '--to-ports', str(port),
-                '-m', 'ttl', '!', '--ttl', '42'  # to prevent infinite loops
-                )
-
-
-def ipfw_rule_exists(n):
-    argv = ['ipfw', 'list']
-    p = subprocess.Popen(argv, stdout = subprocess.PIPE)
-    for line in p.stdout:
-        if line.startswith('%05d ' % n):
-            if line.find('ipttl 42') < 0 and line.find('established') < 0:
-                raise Fatal('non-sshuttle ipfw rule #%d already exists!' % n)
-            return True
-    rv = p.wait()
-    if rv:
-        raise Fatal('%r returned %d' % (argv, rv))
-
-
-def sysctl_get(name):
-    argv = ['sysctl', '-n', name]
-    p = subprocess.Popen(argv, stdout = subprocess.PIPE)
-    line = p.stdout.readline()
-    rv = p.wait()
-    if rv:
-        raise Fatal('%r returned %d' % (argv, rv))
-    if not line:
-        raise Fatal('%r returned no data' % (argv,))
-    assert(line[-1] == '\n')
-    return line[:-1]
-
-
-def _sysctl_set(name, val):
-    argv = ['sysctl', '-w', '%s=%s' % (name, val)]
-    debug1('>> %s\n' % ' '.join(argv))
-    rv = subprocess.call(argv, stdout = open('/dev/null', 'w'))
-
-
-_oldctls = []
-def sysctl_set(name, val):
-    oldval = sysctl_get(name)
-    if str(val) != str(oldval):
-        _oldctls.append((name, oldval))
-        return _sysctl_set(name, val)
-    
-
-def ipfw(*args):
-    argv = ['ipfw', '-q'] + list(args)
-    debug1('>> %s\n' % ' '.join(argv))
-    rv = subprocess.call(argv)
-    if rv:
-        raise Fatal('%r returned %d' % (argv, rv))
-
-
-def do_ipfw(port, subnets):
-    sport = str(port)
-
-    # cleanup any existing rules
-    if ipfw_rule_exists(port):
-        ipfw('del', sport)
-
-    while _oldctls:
-        (name,oldval) = _oldctls.pop()
-        _sysctl_set(name, oldval)
-
-    if subnets:
-        sysctl_set('net.inet.ip.fw.enable', 1)
-        sysctl_set('net.inet.ip.forwarding', 1)
-
-        ipfw('add', sport, 'accept', 'ip',
-             'from', 'any', 'to', 'any', 'established')
-        
-        # create new subnet entries
-        for snet,swidth in subnets:
-            ipfw('add', sport, 'fwd', '127.0.0.1,%d' % port,
-                 'log', 'tcp',
-                 'from', 'any', 'to', '%s/%s' % (snet,swidth),
-                 'not', 'ipttl', '42')
-
-
-def program_exists(name):
-    paths = (os.getenv('PATH') or os.defpath).split(os.pathsep)
-    for p in paths:
-        fn = '%s/%s' % (p, name)
-        if os.path.exists(fn):
-            return not os.path.isdir(fn) and os.access(fn, os.X_OK)
-
-hostmap = {}
-def rewrite_etc_hosts(port):
-    HOSTSFILE='/etc/hosts'
-    BAKFILE='%s.sbak' % HOSTSFILE
-    APPEND='# sshuttle-firewall-%d AUTOCREATED' % port
-    old_content = ''
-    st = None
-    try:
-        old_content = open(HOSTSFILE).read()
-        st = os.stat(HOSTSFILE)
-    except IOError, e:
-        if e.errno == errno.ENOENT:
-            pass
-        else:
-            raise
-    if old_content.strip() and not os.path.exists(BAKFILE):
-        os.link(HOSTSFILE, BAKFILE)
-    tmpname = "%s.%d.tmp" % (HOSTSFILE, port)
-    f = open(tmpname, 'w')
-    for line in old_content.rstrip().split('\n'):
-        if line.find(APPEND) >= 0:
-            continue
-        f.write('%s\n' % line)
-    for (name,ip) in sorted(hostmap.items()):
-        f.write('%-30s %s\n' % ('%s %s' % (ip,name), APPEND))
-    f.close()
-
-    if st:
-        os.chown(tmpname, st.st_uid, st.st_gid)
-        os.chmod(tmpname, st.st_mode)
-    else:
-        os.chown(tmpname, 0, 0)
-        os.chmod(tmpname, 0644)
-    os.rename(tmpname, HOSTSFILE)
-
-
-def restore_etc_hosts(port):
-    global hostmap
-    hostmap = {}
-    rewrite_etc_hosts(port)
-
-
-# This is some voodoo for setting up the kernel's transparent
-# proxying stuff.  If subnets is empty, we just delete our sshuttle rules;
-# otherwise we delete it, then make them from scratch.
-#
-# This code is supposed to clean up after itself by deleting its rules on
-# exit.  In case that fails, it's not the end of the world; future runs will
-# supercede it in the transproxy list, at least, so the leftover rules
-# are hopefully harmless.
-def main(port):
-    assert(port > 0)
-    assert(port <= 65535)
-
-    if os.getuid() != 0:
-        raise Fatal('you must be root (or enable su/sudo) to set the firewall')
-
-    if program_exists('ipfw'):
-        do_it = do_ipfw
-    elif program_exists('iptables'):
-        do_it = do_iptables
-    else:
-        raise Fatal("can't find either ipfw or iptables; check your PATH")
-
-    # because of limitations of the 'su' command, the *real* stdin/stdout
-    # are both attached to stdout initially.  Clone stdout into stdin so we
-    # can read from it.
-    os.dup2(1, 0)
-
-    debug1('firewall manager ready.\n')
-    sys.stdout.write('READY\n')
-    sys.stdout.flush()
-
-    # ctrl-c shouldn't be passed along to me.  When the main sshuttle dies,
-    # I'll die automatically.
-    os.setsid()
-
-    # we wait until we get some input before creating the rules.  That way,
-    # sshuttle can launch us as early as possible (and get sudo password
-    # authentication as early in the startup process as possible).
-    line = sys.stdin.readline(128)
-    if not line:
-        return  # parent died; nothing to do
-
-    subnets = []
-    if line != 'ROUTES\n':
-        raise Fatal('firewall: expected ROUTES but got %r' % line)
-    while 1:
-        line = sys.stdin.readline(128)
-        if not line:
-            raise Fatal('firewall: expected route but got %r' % line)
-        elif line == 'GO\n':
-            break
-        try:
-            (ip,width) = line.strip().split(',', 1)
-        except:
-            raise Fatal('firewall: expected route or GO but got %r' % line)
-        subnets.append((ip, int(width)))
-    try:
-        if line:
-            debug1('firewall manager: starting transproxy.\n')
-            do_it(port, subnets)
-            sys.stdout.write('STARTED\n')
-        
-        try:
-            sys.stdout.flush()
-        except IOError:
-            # the parent process died for some reason; he's surely been loud
-            # enough, so no reason to report another error
-            return
-
-        # Now we wait until EOF or any other kind of exception.  We need
-        # to stay running so that we don't need a *second* password
-        # authentication at shutdown time - that cleanup is important!
-        while 1:
-            line = sys.stdin.readline(128)
-            if line.startswith('HOST '):
-                (name,ip) = line[5:].strip().split(',', 1)
-                hostmap[name] = ip
-                rewrite_etc_hosts(port)
-            elif line:
-                raise Fatal('expected EOF, got %r' % line)
-            else:
-                break
-
-    finally:
-        try:
-            debug1('firewall manager: undoing changes.\n')
-        except:
-            pass
-        do_it(port, [])
-        restore_etc_hosts(port)

helpers.py

@@ -1,25 +0,0 @@
-import sys, os
-
-logprefix = ''
-verbose = 0
-
-def log(s):
-    sys.stdout.flush()
-    sys.stderr.write(logprefix + s)
-    sys.stderr.flush()
-
-def debug1(s):
-    if verbose >= 1:
-        log(s)
-
-def debug2(s):
-    if verbose >= 2:
-        log(s)
-
-def debug3(s):
-    if verbose >= 3:
-        log(s)
-
-
-class Fatal(Exception):
-    pass

main.py

@@ -1,105 +0,0 @@
-#!/usr/bin/env python
-import sys, os, re
-import helpers, options, client, server, firewall, hostwatch
-from helpers import *
-
-
-# list of:
-# 1.2.3.4/5 or just 1.2.3.4
-def parse_subnets(subnets_str):
-    subnets = []
-    for s in subnets_str:
-        m = re.match(r'(\d+)(?:\.(\d+)\.(\d+)\.(\d+))?(?:/(\d+))?$', s)
-        if not m:
-            raise Fatal('%r is not a valid IP subnet format' % s)
-        (a,b,c,d,width) = m.groups()
-        (a,b,c,d) = (int(a or 0), int(b or 0), int(c or 0), int(d or 0))
-        if width == None:
-            width = 32
-        else:
-            width = int(width)
-        if a > 255 or b > 255 or c > 255 or d > 255:
-            raise Fatal('%d.%d.%d.%d has numbers > 255' % (a,b,c,d))
-        if width > 32:
-            raise Fatal('*/%d is greater than the maximum of 32' % width)
-        subnets.append(('%d.%d.%d.%d' % (a,b,c,d), width))
-    return subnets
-
-
-# 1.2.3.4:567 or just 1.2.3.4 or just 567
-def parse_ipport(s):
-    s = str(s)
-    m = re.match(r'(?:(\d+)\.(\d+)\.(\d+)\.(\d+))?(?::)?(?:(\d+))?$', s)
-    if not m:
-        raise Fatal('%r is not a valid IP:port format' % s)
-    (a,b,c,d,port) = m.groups()
-    (a,b,c,d,port) = (int(a or 0), int(b or 0), int(c or 0), int(d or 0),
-                      int(port or 0))
-    if a > 255 or b > 255 or c > 255 or d > 255:
-        raise Fatal('%d.%d.%d.%d has numbers > 255' % (a,b,c,d))
-    if port > 65535:
-        raise Fatal('*:%d is greater than the maximum of 65535' % port)
-    if a == None:
-        a = b = c = d = 0
-    return ('%d.%d.%d.%d' % (a,b,c,d), port)
-
-
-optspec = """
-sshuttle [-l [ip:]port] [-r [username@]sshserver[:port]] <subnets...>
-sshuttle --firewall <port> <subnets...>
-sshuttle --server
---
-l,listen=  transproxy to this ip address and port number [default=0]
-H,auto-hosts scan for remote hostnames and update local /etc/hosts
-N,auto-nets  automatically determine subnets to route
-r,remote=  ssh hostname (and optional username) of remote sshuttle server
-v,verbose  increase debug message verbosity
-seed-hosts= with -H, use these hostnames for initial scan (comma-separated)
-noserver   don't use a separate server process (mostly for debugging)
-server     [internal use only]
-firewall   [internal use only]
-hostwatch  [internal use only]
-"""
-o = options.Options('sshuttle', optspec)
-(opt, flags, extra) = o.parse(sys.argv[1:])
-
-helpers.verbose = opt.verbose
-
-try:
-    if opt.server:
-        if len(extra) != 0:
-            o.fatal('no arguments expected')
-        sys.exit(server.main())
-    elif opt.firewall:
-        if len(extra) != 1:
-            o.fatal('exactly one argument expected')
-        sys.exit(firewall.main(int(extra[0])))
-    elif opt.hostwatch:
-        sys.exit(hostwatch.hw_main(extra))
-    else:
-        if len(extra) < 1 and not opt.auto_nets:
-            o.fatal('at least one subnet (or -N) expected')
-        remotename = opt.remote
-        if remotename == '' or remotename == '-':
-            remotename = None
-        if opt.seed_hosts and not opt.auto_hosts:
-            o.fatal('--seed-hosts only works if you also use -H')
-        if opt.seed_hosts:
-            sh = re.split(r'[\s,]+', (opt.seed_hosts or "").strip())
-        elif opt.auto_hosts:
-            sh = []
-        else:
-            sh = None
-        sys.exit(client.main(parse_ipport(opt.listen or '0.0.0.0:0'),
-                             not opt.noserver,
-                             remotename,
-                             sh,
-                             opt.auto_nets,
-                             parse_subnets(extra)))
-except Fatal, e:
-    log('fatal: %s\n' % e)
-    sys.exit(99)
-except KeyboardInterrupt:
-    log('\n')
-    log('Keyboard interrupt: exiting.\n')
-    sys.exit(1)

options.py

@@ -1,118 +0,0 @@
-import sys, textwrap, getopt, re
-
-class OptDict:
-    def __init__(self):
-        self._opts = {}
-
-    def __setitem__(self, k, v):
-        self._opts[k] = v
-        
-    def __getitem__(self, k):
-        return self._opts[k]
-
-    def __getattr__(self, k):
-        return self[k]
-
-
-class Options:
-    def __init__(self, exe, optspec, optfunc=getopt.gnu_getopt):
-        self.exe = exe
-        self.optspec = optspec
-        self.optfunc = optfunc
-        self._aliases = {}
-        self._shortopts = 'h?'
-        self._longopts = ['help']
-        self._hasparms = {}
-        self._usagestr = self._gen_usage()
-        
-    def _gen_usage(self):
-        out = []
-        lines = self.optspec.strip().split('\n')
-        lines.reverse()
-        first_syn = True
-        while lines:
-            l = lines.pop()
-            if l == '--': break
-            out.append('%s: %s\n' % (first_syn and 'usage' or '   or', l))
-            first_syn = False
-        out.append('\n')
-        while lines:
-            l = lines.pop()
-            if l.startswith(' '):
-                out.append('\n%s\n' % l.lstrip())
-            elif l:
-                (flags, extra) = l.split(' ', 1)
-                extra = extra.strip()
-                if flags.endswith('='):
-                    flags = flags[:-1]
-                    has_parm = 1
-                else:
-                    has_parm = 0
-                flagl = flags.split(',')
-                flagl_nice = []
-                for f in flagl:
-                    f_nice = re.sub(r'\W', '_', f)
-                    self._aliases[f] = flagl[0]
-                    self._aliases[f_nice] = flagl[0]
-                    self._hasparms[f] = has_parm
-                    if len(f) == 1:
-                        self._shortopts += f + (has_parm and ':' or '')
-                        flagl_nice.append('-' + f)
-                    else:
-                        assert(not f.startswith('no-')) # supported implicitly
-                        self._longopts.append(f + (has_parm and '=' or ''))
-                        self._longopts.append('no-' + f)
-                        flagl_nice.append('--' + f)
-                flags_nice = ', '.join(flagl_nice)
-                if has_parm:
-                    flags_nice += ' ...'
-                prefix = '    %-20s  ' % flags_nice
-                argtext = '\n'.join(textwrap.wrap(extra, width=70,
-                                                initial_indent=prefix,
-                                                subsequent_indent=' '*28))
-                out.append(argtext + '\n')
-            else:
-                out.append('\n')
-        return ''.join(out).rstrip() + '\n'
-    
-    def usage(self):
-        sys.stderr.write(self._usagestr)
-        sys.exit(97)
-
-    def fatal(self, s):
-        sys.stderr.write('error: %s\n' % s)
-        return self.usage()
-        
-    def parse(self, args):
-        try:
-            (flags,extra) = self.optfunc(args, self._shortopts, self._longopts)
-        except getopt.GetoptError, e:
-            self.fatal(e)
-
-        opt = OptDict()
-        for f in self._aliases.values():
-            opt[f] = None
-        for (k,v) in flags:
-            while k.startswith('-'):
-                k = k[1:]
-            if k in ['h', '?', 'help']:
-                self.usage()
-            if k.startswith('no-'):
-                k = self._aliases[k[3:]]
-                opt[k] = None
-            else:
-                k = self._aliases[k]
-                if not self._hasparms[k]:
-                    assert(v == '')
-                    opt[k] = (opt._opts.get(k) or 0) + 1
-                else:
-                    try:
-                        vv = int(v)
-                        if str(vv) == v:
-                            v = vv
-                    except ValueError:
-                        pass
-                    opt[k] = v
-        for (f1,f2) in self._aliases.items():
-            opt[f1] = opt[f2]
-        return (opt,flags,extra)

run

@@ -0,0 +1,6 @@
+#!/bin/sh
+if python2 -V 2>/dev/null; then
+	exec python2 -m "sshuttle" "$@"
+else
+	exec python -m "sshuttle" "$@"
+fi

server.py

@@ -1,174 +0,0 @@
-import re, struct, socket, select, subprocess, traceback
-if not globals().get('skip_imports'):
-    import ssnet, helpers, hostwatch
-    from ssnet import SockWrapper, Handler, Proxy, Mux, MuxWrapper
-    from helpers import *
-
-
-def _ipmatch(ipstr):
-    if ipstr == 'default':
-        ipstr = '0.0.0.0/0'
-    m = re.match(r'^(\d+(\.\d+(\.\d+(\.\d+)?)?)?)(?:/(\d+))?$', ipstr)
-    if m:
-        g = m.groups()
-        ips = g[0]
-        width = int(g[4] or 32)
-        if g[1] == None:
-            ips += '.0.0.0'
-            width = min(width, 8)
-        elif g[2] == None:
-            ips += '.0.0'
-            width = min(width, 16)
-        elif g[3] == None:
-            ips += '.0'
-            width = min(width, 24)
-        return (struct.unpack('!I', socket.inet_aton(ips))[0], width)
-
-
-def _ipstr(ip, width):
-    if width >= 32:
-        return ip
-    else:
-        return "%s/%d" % (ip, width)
-
-
-def _maskbits(netmask):
-    if not netmask:
-        return 32
-    for i in range(32):
-        if netmask[0] & (1<<i):
-            return 32-i
-    return 0
-
-
-def _list_routes():
-    argv = ['netstat', '-rn']
-    p = subprocess.Popen(argv, stdout=subprocess.PIPE)
-    routes = []
-    for line in p.stdout:
-        cols = re.split(r'\s+', line)
-        ipw = _ipmatch(cols[0])
-        if not ipw:
-            continue  # some lines won't be parseable; never mind
-        maskw = _ipmatch(cols[2])  # linux only
-        mask = _maskbits(maskw)   # returns 32 if maskw is null
-        width = min(ipw[1], mask)
-        ip = ipw[0] & (((1<<width)-1) << (32-width))
-        routes.append((socket.inet_ntoa(struct.pack('!I', ip)), width))
-    rv = p.wait()
-    if rv != 0:
-        raise Fatal('%r returned %d' % (argv, rv))
-    return routes
-
-
-def list_routes():
-    for (ip,width) in _list_routes():
-        if not ip.startswith('0.') and not ip.startswith('127.'):
-            yield (ip,width)
-
-
-def _exc_dump():
-    exc_info = sys.exc_info()
-    return ''.join(traceback.format_exception(*exc_info))
-
-
-def start_hostwatch(seed_hosts):
-    s1,s2 = socket.socketpair()
-    pid = os.fork()
-    if not pid:
-        # child
-        rv = 99
-        try:
-            s2.close()
-            os.dup2(s1.fileno(), 1)
-            os.dup2(s1.fileno(), 0)
-            s1.close()
-            rv = hostwatch.hw_main(seed_hosts) or 0
-        except Exception, e:
-            log('%s\n' % _exc_dump())
-            rv = 98
-        finally:
-            os._exit(rv)
-    s1.close()
-    return pid,s2
-
-
-class Hostwatch:
-    def __init__(self):
-        self.pid = 0
-        self.sock = None
-
-
-def main():
-    if helpers.verbose >= 1:
-        helpers.logprefix = ' s: '
-    else:
-        helpers.logprefix = 'server: '
-
-    routes = list(list_routes())
-    debug1('available routes:\n')
-    for r in routes:
-        debug1('  %s/%d\n' % r)
-        
-    # synchronization header
-    sys.stdout.write('SSHUTTLE0001')
-    sys.stdout.flush()
-
-    handlers = []
-    mux = Mux(socket.fromfd(sys.stdin.fileno(),
-                            socket.AF_INET, socket.SOCK_STREAM),
-              socket.fromfd(sys.stdout.fileno(),
-                            socket.AF_INET, socket.SOCK_STREAM))
-    handlers.append(mux)
-    routepkt = ''.join('%s,%d\n' % r
-                       for r in routes)
-    mux.send(0, ssnet.CMD_ROUTES, routepkt)
-
-    hw = Hostwatch()
-
-    def hostwatch_ready():
-        assert(hw.pid)
-        content = hw.sock.recv(4096)
-        if content:
-            mux.send(0, ssnet.CMD_HOST_LIST, content)
-        else:
-            raise Fatal('hostwatch process died')
-
-    def got_host_req(data):
-        if not hw.pid:
-            (hw.pid,hw.sock) = start_hostwatch(data.strip().split())
-            handlers.append(Handler(socks = [hw.sock],
-                                    callback = hostwatch_ready))
-    mux.got_host_req = got_host_req
-
-    def new_channel(channel, data):
-        (dstip,dstport) = data.split(',', 1)
-        dstport = int(dstport)
-        outwrap = ssnet.connect_dst(dstip,dstport)
-        handlers.append(Proxy(MuxWrapper(mux, channel), outwrap))
-    mux.new_channel = new_channel
-
-    while mux.ok:
-        if hw.pid:
-            (rpid, rv) = os.waitpid(hw.pid, os.WNOHANG)
-            if rpid:
-                raise Fatal('hostwatch exited unexpectedly: code 0x%04x\n' % rv)
-        
-        r = set()
-        w = set()
-        x = set()
-        handlers = filter(lambda s: s.ok, handlers)
-        for s in handlers:
-            s.pre_select(r,w,x)
-        debug2('Waiting: %d[%d,%d,%d] (fullness=%d/%d)...\n' 
-               % (len(handlers), len(r), len(w), len(x),
-                  mux.fullness, mux.too_full))
-        (r,w,x) = select.select(r,w,x)
-        #log('r=%r w=%r x=%r\n' % (r,w,x))
-        ready = set(r) | set(w) | set(x)
-        for s in handlers:
-            #debug2('check: %r: %r\n' % (s, s.socks & ready))
-            if s.socks & ready:
-                s.callback()
-        mux.check_fullness()
-        mux.callback()

setup.py

@@ -0,0 +1,52 @@
+#!/usr/bin/env python
+
+# Copyright 2012-2014 Brian May
+#
+# This file is part of python-tldap.
+#
+# python-tldap is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# python-tldap is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with python-tldap  If not, see <http://www.gnu.org/licenses/>.
+
+from setuptools import setup, find_packages
+
+with open('VERSION.txt', 'r') as f:
+    version = f.readline().strip()
+
+setup(
+    name="sshuttle",
+    version=version,
+    url='https://github.com/sshuttle/sshuttle',
+    author='Brian May',
+    author_email='brian@linuxpenguins.xyz',
+    description='Transparent proxy server that works as a poor man\'s VPN.',
+    packages=find_packages(),
+    license="GPL2+",
+    long_description=open('README.rst').read(),
+    classifiers=[
+        "Development Status :: 5 - Production/Stable",
+        "Intended Audience :: Developers",
+        "Intended Audience :: End Users/Desktop",
+        "License :: OSI Approved :: "
+            "GNU General Public License v2 or later (GPLv2+)",
+        "Operating System :: OS Independent",
+        "Programming Language :: Python :: 2.7",
+        "Programming Language :: Python :: 3.5",
+        "Topic :: System :: Networking",
+    ],
+    entry_points={
+        'console_scripts': [
+            'sshuttle = sshuttle.__main__',
+        ],
+    },
+    keywords="ssh vpn",
+)

ssh.py

@@ -1,64 +0,0 @@
-import sys, os, re, subprocess, socket, zlib
-import helpers
-from helpers import *
-
-
-def readfile(name):
-    basedir = os.path.dirname(os.path.abspath(sys.argv[0]))
-    fullname = os.path.join(basedir, name)
-    return open(fullname, 'rb').read()
-
-
-def empackage(z, filename):
-    content = z.compress(readfile(filename))
-    content += z.flush(zlib.Z_SYNC_FLUSH)
-    return '%s\n%d\n%s' % (filename,len(content), content)
-
-
-def connect(rhostport):
-    main_exe = sys.argv[0]
-    l = (rhostport or '').split(':', 1)
-    rhost = l[0]
-    portl = []
-    if len(l) > 1:
-        portl = ['-p', str(int(l[1]))]
-
-    if rhost == '-':
-        rhost = None
-
-    z = zlib.compressobj(1)
-    content = readfile('assembler.py')
-    content2 = (empackage(z, 'helpers.py') +
-                empackage(z, 'ssnet.py') +
-                empackage(z, 'hostwatch.py') +
-                empackage(z, 'server.py') +
-                "\n")
-    
-    pyscript = r"""
-                import sys;
-                skip_imports=1;
-                verbosity=%d;
-                exec compile(sys.stdin.read(%d), "assembler.py", "exec")
-                """ % (helpers.verbose or 0, len(content))
-    pyscript = re.sub(r'\s+', ' ', pyscript.strip())
-
-        
-    if not rhost:
-        argv = ['python', '-c', pyscript]
-    else:
-        argv = ['ssh'] + portl + [rhost, '--', "python -c '%s'" % pyscript]
-    (s1,s2) = socket.socketpair()
-    def setup():
-        # runs in the child process
-        s2.close()
-        os.setsid()
-    s1a,s1b = os.dup(s1.fileno()), os.dup(s1.fileno())
-    s1.close()
-    debug2('executing: %r\n' % argv)
-    p = subprocess.Popen(argv, stdin=s1a, stdout=s1b, preexec_fn=setup,
-                         close_fds=True)
-    os.close(s1a)
-    os.close(s1b)
-    s2.sendall(content)
-    s2.sendall(content2)
-    return p, s2

sshuttle

@@ -1 +0,0 @@
-main.py

sshuttle/__main__.py

@@ -0,0 +1,226 @@
+import sys
+import re
+import socket
+import sshuttle.helpers as helpers
+import sshuttle.options as options
+import sshuttle.client as client
+import sshuttle.firewall as firewall
+import sshuttle.hostwatch as hostwatch
+from sshuttle.helpers import family_ip_tuple, log, Fatal
+
+
+# 1.2.3.4/5 or just 1.2.3.4
+def parse_subnet4(s):
+    m = re.match(r'(\d+)(?:\.(\d+)\.(\d+)\.(\d+))?(?:/(\d+))?$', s)
+    if not m:
+        raise Fatal('%r is not a valid IP subnet format' % s)
+    (a, b, c, d, width) = m.groups()
+    (a, b, c, d) = (int(a or 0), int(b or 0), int(c or 0), int(d or 0))
+    if width is None:
+        width = 32
+    else:
+        width = int(width)
+    if a > 255 or b > 255 or c > 255 or d > 255:
+        raise Fatal('%d.%d.%d.%d has numbers > 255' % (a, b, c, d))
+    if width > 32:
+        raise Fatal('*/%d is greater than the maximum of 32' % width)
+    return(socket.AF_INET, '%d.%d.%d.%d' % (a, b, c, d), width)
+
+
+# 1:2::3/64 or just 1:2::3
+def parse_subnet6(s):
+    m = re.match(r'(?:([a-fA-F\d:]+))?(?:/(\d+))?$', s)
+    if not m:
+        raise Fatal('%r is not a valid IP subnet format' % s)
+    (net, width) = m.groups()
+    if width is None:
+        width = 128
+    else:
+        width = int(width)
+    if width > 128:
+        raise Fatal('*/%d is greater than the maximum of 128' % width)
+    return(socket.AF_INET6, net, width)
+
+
+# Subnet file, supporting empty lines and hash-started comment lines
+def parse_subnet_file(s):
+    try:
+        handle = open(s, 'r')
+    except OSError:
+        raise Fatal('Unable to open subnet file: %s' % s)
+
+    raw_config_lines = handle.readlines()
+    config_lines = []
+    for line_no, line in enumerate(raw_config_lines):
+        line = line.strip()
+        if len(line) == 0:
+            continue
+        if line[0] == '#':
+            continue
+        config_lines.append(line)
+
+    return config_lines
+
+
+# list of:
+# 1.2.3.4/5 or just 1.2.3.4
+# 1:2::3/64 or just 1:2::3
+def parse_subnets(subnets_str):
+    subnets = []
+    for s in subnets_str:
+        if ':' in s:
+            subnet = parse_subnet6(s)
+        else:
+            subnet = parse_subnet4(s)
+        subnets.append(subnet)
+    return subnets
+
+
+# 1.2.3.4:567 or just 1.2.3.4 or just 567
+def parse_ipport4(s):
+    s = str(s)
+    m = re.match(r'(?:(\d+)\.(\d+)\.(\d+)\.(\d+))?(?::)?(?:(\d+))?$', s)
+    if not m:
+        raise Fatal('%r is not a valid IP:port format' % s)
+    (a, b, c, d, port) = m.groups()
+    (a, b, c, d, port) = (int(a or 0), int(b or 0), int(c or 0), int(d or 0),
+                          int(port or 0))
+    if a > 255 or b > 255 or c > 255 or d > 255:
+        raise Fatal('%d.%d.%d.%d has numbers > 255' % (a, b, c, d))
+    if port > 65535:
+        raise Fatal('*:%d is greater than the maximum of 65535' % port)
+    if a is None:
+        a = b = c = d = 0
+    return ('%d.%d.%d.%d' % (a, b, c, d), port)
+
+
+# [1:2::3]:456 or [1:2::3] or 456
+def parse_ipport6(s):
+    s = str(s)
+    m = re.match(r'(?:\[([^]]*)])?(?::)?(?:(\d+))?$', s)
+    if not m:
+        raise Fatal('%s is not a valid IP:port format' % s)
+    (ip, port) = m.groups()
+    (ip, port) = (ip or '::', int(port or 0))
+    return (ip, port)
+
+
+def parse_list(list):
+    return re.split(r'[\s,]+', list.strip()) if list else []
+
+
+optspec = """
+sshuttle [-l [ip:]port] [-r [username@]sshserver[:port]] <subnets...>
+sshuttle --firewall <port> <subnets...>
+sshuttle --hostwatch
+--
+l,listen=  transproxy to this ip address and port number
+H,auto-hosts scan for remote hostnames and update local /etc/hosts
+N,auto-nets  automatically determine subnets to route
+dns        capture local DNS requests and forward to the remote DNS server
+ns-hosts=  capture and forward remote DNS requests to the following servers
+method=    auto, nat, tproxy, pf or ipfw
+python=    path to python interpreter on the remote server
+r,remote=  ssh hostname (and optional username) of remote sshuttle server
+x,exclude= exclude this subnet (can be used more than once)
+X,exclude-from=  exclude the subnets in a file (whitespace separated)
+v,verbose  increase debug message verbosity
+e,ssh-cmd= the command to use to connect to the remote [ssh]
+seed-hosts= with -H, use these hostnames for initial scan (comma-separated)
+no-latency-control  sacrifice latency to improve bandwidth benchmarks
+wrap=      restart counting channel numbers after this number (for testing)
+D,daemon   run in the background as a daemon
+s,subnets= file where the subnets are stored, instead of on the command line
+syslog     send log messages to syslog (default if you use --daemon)
+pidfile=   pidfile name (only if using --daemon) [./sshuttle.pid]
+server     (internal use only)
+firewall   (internal use only)
+hostwatch  (internal use only)
+"""
+o = options.Options(optspec)
+(opt, flags, extra) = o.parse(sys.argv[1:])
+
+if opt.daemon:
+    opt.syslog = 1
+if opt.wrap:
+    import sshuttle.ssnet as ssnet
+    ssnet.MAX_CHANNEL = int(opt.wrap)
+helpers.verbose = opt.verbose
+
+try:
+    if opt.firewall:
+        if len(extra) != 0:
+            o.fatal('exactly zero arguments expected')
+        result = firewall.main(opt.method, opt.syslog)
+        sys.exit(result)
+    elif opt.hostwatch:
+        sys.exit(hostwatch.hw_main(extra))
+    else:
+        if len(extra) < 1 and not opt.auto_nets and not opt.subnets:
+            o.fatal('at least one subnet, subnet file, or -N expected')
+        includes = extra
+        excludes = ['127.0.0.0/8']
+        for k, v in flags:
+            if k in ('-x', '--exclude'):
+                excludes.append(v)
+            if k in ('-X', '--exclude-from'):
+                excludes += open(v).read().split()
+        remotename = opt.remote
+        if remotename == '' or remotename == '-':
+            remotename = None
+        nslist = [family_ip_tuple(ns) for ns in parse_list(opt.ns_hosts)]
+        if opt.seed_hosts and not opt.auto_hosts:
+            o.fatal('--seed-hosts only works if you also use -H')
+        if opt.seed_hosts:
+            sh = re.split(r'[\s,]+', (opt.seed_hosts or "").strip())
+        elif opt.auto_hosts:
+            sh = []
+        else:
+            sh = None
+        if opt.subnets:
+            includes = parse_subnet_file(opt.subnets)
+        if not opt.method:
+            method_name = "auto"
+        elif opt.method in ["auto", "nat", "tproxy", "ipfw", "pf"]:
+            method_name = opt.method
+        else:
+            o.fatal("method_name %s not supported" % opt.method)
+        if not opt.listen:
+            ipport_v6 = "auto"  # parse_ipport6('[::1]:0')
+            ipport_v4 = "auto"  # parse_ipport4('127.0.0.1:0')
+        else:
+            ipport_v6 = None
+            ipport_v4 = None
+            list = opt.listen.split(",")
+            for ip in list:
+                if '[' in ip and ']' in ip:
+                    ipport_v6 = parse_ipport6(ip)
+                else:
+                    ipport_v4 = parse_ipport4(ip)
+        return_code = client.main(ipport_v6, ipport_v4,
+                                  opt.ssh_cmd,
+                                  remotename,
+                                  opt.python,
+                                  opt.latency_control,
+                                  opt.dns,
+                                  nslist,
+                                  method_name,
+                                  sh,
+                                  opt.auto_nets,
+                                  parse_subnets(includes),
+                                  parse_subnets(excludes),
+                                  opt.syslog, opt.daemon, opt.pidfile)
+
+        if return_code == 0:
+            log('Normal exit code, exiting...')
+        else:
+            log('Abnormal exit code detected, failing...' % return_code)
+        sys.exit(return_code)
+
+except Fatal as e:
+    log('fatal: %s\n' % e)
+    sys.exit(99)
+except KeyboardInterrupt:
+    log('\n')
+    log('Keyboard interrupt: exiting.\n')
+    sys.exit(1)

sshuttle/assembler.py

@@ -0,0 +1,34 @@
+import sys
+import zlib
+import imp
+
+z = zlib.decompressobj()
+while 1:
+    name = sys.stdin.readline().strip()
+    if name:
+        nbytes = int(sys.stdin.readline())
+        if verbosity >= 2:
+            sys.stderr.write('server: assembling %r (%d bytes)\n'
+                             % (name, nbytes))
+        content = z.decompress(sys.stdin.read(nbytes))
+
+        module = imp.new_module(name)
+        parent, _, parent_name = name.rpartition(".")
+        if parent != "":
+            setattr(sys.modules[parent], parent_name, module)
+
+        code = compile(content, name, "exec")
+        exec(code, module.__dict__)
+        sys.modules[name] = module
+    else:
+        break
+
+sys.stderr.flush()
+sys.stdout.flush()
+
+import sshuttle.helpers
+sshuttle.helpers.verbose = verbosity
+
+import sshuttle.cmdline_options as options
+from sshuttle.server import main
+main(options.latency_control)

sshuttle/client.py

@@ -0,0 +1,653 @@
+import socket
+import errno
+import re
+import signal
+import time
+import subprocess as ssubprocess
+import sshuttle.helpers as helpers
+import os
+import sshuttle.ssnet as ssnet
+import sshuttle.ssh as ssh
+import sshuttle.ssyslog as ssyslog
+import sys
+from sshuttle.ssnet import SockWrapper, Handler, Proxy, Mux, MuxWrapper
+from sshuttle.helpers import log, debug1, debug2, debug3, Fatal, islocal, \
+    resolvconf_nameservers
+from sshuttle.methods import get_method
+
+_extra_fd = os.open('/dev/null', os.O_RDONLY)
+
+
+def got_signal(signum, frame):
+    log('exiting on signal %d\n' % signum)
+    sys.exit(1)
+
+
+_pidname = None
+
+
+def check_daemon(pidfile):
+    global _pidname
+    _pidname = os.path.abspath(pidfile)
+    try:
+        oldpid = open(_pidname).read(1024)
+    except IOError as e:
+        if e.errno == errno.ENOENT:
+            return  # no pidfile, ok
+        else:
+            raise Fatal("can't read %s: %s" % (_pidname, e))
+    if not oldpid:
+        os.unlink(_pidname)
+        return  # invalid pidfile, ok
+    oldpid = int(oldpid.strip() or 0)
+    if oldpid <= 0:
+        os.unlink(_pidname)
+        return  # invalid pidfile, ok
+    try:
+        os.kill(oldpid, 0)
+    except OSError as e:
+        if e.errno == errno.ESRCH:
+            os.unlink(_pidname)
+            return  # outdated pidfile, ok
+        elif e.errno == errno.EPERM:
+            pass
+        else:
+            raise
+    raise Fatal("%s: sshuttle is already running (pid=%d)"
+                % (_pidname, oldpid))
+
+
+def daemonize():
+    if os.fork():
+        os._exit(0)
+    os.setsid()
+    if os.fork():
+        os._exit(0)
+
+    outfd = os.open(_pidname, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o666)
+    try:
+        os.write(outfd, '%d\n' % os.getpid())
+    finally:
+        os.close(outfd)
+    os.chdir("/")
+
+    # Normal exit when killed, or try/finally won't work and the pidfile won't
+    # be deleted.
+    signal.signal(signal.SIGTERM, got_signal)
+
+    si = open('/dev/null', 'r+')
+    os.dup2(si.fileno(), 0)
+    os.dup2(si.fileno(), 1)
+    si.close()
+
+    ssyslog.stderr_to_syslog()
+
+
+def daemon_cleanup():
+    try:
+        os.unlink(_pidname)
+    except OSError as e:
+        if e.errno == errno.ENOENT:
+            pass
+        else:
+            raise
+
+
+class MultiListener:
+
+    def __init__(self, type=socket.SOCK_STREAM, proto=0):
+        self.v6 = socket.socket(socket.AF_INET6, type, proto)
+        self.v4 = socket.socket(socket.AF_INET, type, proto)
+
+    def setsockopt(self, level, optname, value):
+        if self.v6:
+            self.v6.setsockopt(level, optname, value)
+        if self.v4:
+            self.v4.setsockopt(level, optname, value)
+
+    def add_handler(self, handlers, callback, method, mux):
+        socks = []
+        if self.v6:
+            socks.append(self.v6)
+        if self.v4:
+            socks.append(self.v4)
+
+        handlers.append(
+            Handler(
+                socks,
+                lambda sock: callback(sock, method, mux, handlers)
+            )
+        )
+
+    def listen(self, backlog):
+        if self.v6:
+            self.v6.listen(backlog)
+        if self.v4:
+            try:
+                self.v4.listen(backlog)
+            except socket.error as e:
+                # on some systems v4 bind will fail if the v6 suceeded,
+                # in this case the v6 socket will receive v4 too.
+                if e.errno == errno.EADDRINUSE and self.v6:
+                    self.v4 = None
+                else:
+                    raise e
+
+    def bind(self, address_v6, address_v4):
+        if address_v6 and self.v6:
+            self.v6.bind(address_v6)
+        else:
+            self.v6 = None
+        if address_v4 and self.v4:
+            self.v4.bind(address_v4)
+        else:
+            self.v4 = None
+
+    def print_listening(self, what):
+        if self.v6:
+            listenip = self.v6.getsockname()
+            debug1('%s listening on %r.\n' % (what, listenip))
+            debug2('%s listening with %r.\n' % (what, self.v6))
+        if self.v4:
+            listenip = self.v4.getsockname()
+            debug1('%s listening on %r.\n' % (what, listenip))
+            debug2('%s listening with %r.\n' % (what, self.v4))
+
+
+class FirewallClient:
+
+    def __init__(self, method_name):
+        self.auto_nets = []
+        python_path = os.path.dirname(os.path.dirname(__file__))
+        argvbase = ([sys.executable, sys.argv[0]] +
+                    ['-v'] * (helpers.verbose or 0) +
+                    ['--method', method_name] +
+                    ['--firewall'])
+        if ssyslog._p:
+            argvbase += ['--syslog']
+        argv_tries = [
+            ['sudo', '-p', '[local sudo] Password: ',
+                ('PYTHONPATH=%s' % python_path), '--'] + argvbase,
+            argvbase
+        ]
+
+        # we can't use stdin/stdout=subprocess.PIPE here, as we normally would,
+        # because stupid Linux 'su' requires that stdin be attached to a tty.
+        # Instead, attach a *bidirectional* socket to its stdout, and use
+        # that for talking in both directions.
+        (s1, s2) = socket.socketpair()
+
+        def setup():
+            # run in the child process
+            s2.close()
+        e = None
+        if os.getuid() == 0:
+            argv_tries = argv_tries[-1:]  # last entry only
+        for argv in argv_tries:
+            try:
+                if argv[0] == 'su':
+                    sys.stderr.write('[local su] ')
+                self.p = ssubprocess.Popen(argv, stdout=s1, preexec_fn=setup)
+                e = None
+                break
+            except OSError as e:
+                pass
+        self.argv = argv
+        s1.close()
+        if sys.version_info < (3, 0):
+            # python 2.7
+            self.pfile = s2.makefile('wb+')
+        else:
+            # python 3.5
+            self.pfile = s2.makefile('rwb')
+        if e:
+            log('Spawning firewall manager: %r\n' % self.argv)
+            raise Fatal(e)
+        line = self.pfile.readline()
+        self.check()
+        if line[0:5] != b'READY':
+            raise Fatal('%r expected READY, got %r' % (self.argv, line))
+        method_name = line[6:-1]
+        self.method = get_method(method_name.decode("ASCII"))
+        self.method.set_firewall(self)
+
+    def setup(self, subnets_include, subnets_exclude, nslist,
+              redirectport_v6, redirectport_v4, dnsport_v6, dnsport_v4, udp):
+        self.subnets_include = subnets_include
+        self.subnets_exclude = subnets_exclude
+        self.nslist = nslist
+        self.redirectport_v6 = redirectport_v6
+        self.redirectport_v4 = redirectport_v4
+        self.dnsport_v6 = dnsport_v6
+        self.dnsport_v4 = dnsport_v4
+        self.udp = udp
+
+    def check(self):
+        rv = self.p.poll()
+        if rv:
+            raise Fatal('%r returned %d' % (self.argv, rv))
+
+    def start(self):
+        self.pfile.write(b'ROUTES\n')
+        for (family, ip, width) in self.subnets_include + self.auto_nets:
+            self.pfile.write(b'%d,%d,0,%s\n'
+                             % (family, width, ip.encode("ASCII")))
+        for (family, ip, width) in self.subnets_exclude:
+            self.pfile.write(b'%d,%d,1,%s\n'
+                             % (family, width, ip.encode("ASCII")))
+
+        self.pfile.write(b'NSLIST\n')
+        for (family, ip) in self.nslist:
+            self.pfile.write(b'%d,%s\n'
+                             % (family, ip.encode("ASCII")))
+
+        self.pfile.write(
+            b'PORTS %d,%d,%d,%d\n'
+            % (self.redirectport_v6, self.redirectport_v4,
+               self.dnsport_v6, self.dnsport_v4))
+
+        udp = 0
+        if self.udp:
+            udp = 1
+
+        self.pfile.write(b'GO %d\n' % udp)
+        self.pfile.flush()
+
+        line = self.pfile.readline()
+        self.check()
+        if line != b'STARTED\n':
+            raise Fatal('%r expected STARTED, got %r' % (self.argv, line))
+
+    def sethostip(self, hostname, ip):
+        assert(not re.search(r'[^-\w]', hostname))
+        assert(not re.search(r'[^0-9.]', ip))
+        self.pfile.write(b'HOST %s,%s\n' % (hostname, ip))
+        self.pfile.flush()
+
+    def done(self):
+        self.pfile.close()
+        rv = self.p.wait()
+        if rv:
+            raise Fatal('cleanup: %r returned %d' % (self.argv, rv))
+
+
+dnsreqs = {}
+udp_by_src = {}
+
+
+def expire_connections(now, mux):
+    for chan, timeout in dnsreqs.items():
+        if timeout < now:
+            debug3('expiring dnsreqs channel=%d\n' % chan)
+            del mux.channels[chan]
+            del dnsreqs[chan]
+    debug3('Remaining DNS requests: %d\n' % len(dnsreqs))
+    for peer, (chan, timeout) in udp_by_src.items():
+        if timeout < now:
+            debug3('expiring UDP channel channel=%d peer=%r\n' % (chan, peer))
+            mux.send(chan, ssnet.CMD_UDP_CLOSE, '')
+            del mux.channels[chan]
+            del udp_by_src[peer]
+    debug3('Remaining UDP channels: %d\n' % len(udp_by_src))
+
+
+def onaccept_tcp(listener, method, mux, handlers):
+    global _extra_fd
+    try:
+        sock, srcip = listener.accept()
+    except socket.error as e:
+        if e.args[0] in [errno.EMFILE, errno.ENFILE]:
+            debug1('Rejected incoming connection: too many open files!\n')
+            # free up an fd so we can eat the connection
+            os.close(_extra_fd)
+            try:
+                sock, srcip = listener.accept()
+                sock.close()
+            finally:
+                _extra_fd = os.open('/dev/null', os.O_RDONLY)
+            return
+        else:
+            raise
+
+    dstip = method.get_tcp_dstip(sock)
+    debug1('Accept TCP: %s:%r -> %s:%r.\n' % (srcip[0], srcip[1],
+                                              dstip[0], dstip[1]))
+    if dstip[1] == sock.getsockname()[1] and islocal(dstip[0], sock.family):
+        debug1("-- ignored: that's my address!\n")
+        sock.close()
+        return
+    chan = mux.next_channel()
+    if not chan:
+        log('warning: too many open channels.  Discarded connection.\n')
+        sock.close()
+        return
+    mux.send(chan, ssnet.CMD_TCP_CONNECT, b'%d,%s,%d' %
+             (sock.family, dstip[0].encode("ASCII"), dstip[1]))
+    outwrap = MuxWrapper(mux, chan)
+    handlers.append(Proxy(SockWrapper(sock, sock), outwrap))
+    expire_connections(time.time(), mux)
+
+
+def udp_done(chan, data, method, sock, dstip):
+    (src, srcport, data) = data.split(",", 2)
+    srcip = (src, int(srcport))
+    debug3('doing send from %r to %r\n' % (srcip, dstip,))
+    method.send_udp(sock, srcip, dstip, data)
+
+
+def onaccept_udp(listener, method, mux, handlers):
+    now = time.time()
+    t = method.recv_udp(listener, 4096)
+    if t is None:
+        return
+    srcip, dstip, data = t
+    debug1('Accept UDP: %r -> %r.\n' % (srcip, dstip,))
+    if srcip in udp_by_src:
+        chan, timeout = udp_by_src[srcip]
+    else:
+        chan = mux.next_channel()
+        mux.channels[chan] = lambda cmd, data: udp_done(
+            chan, data, method, listener, dstip=srcip)
+        mux.send(chan, ssnet.CMD_UDP_OPEN, listener.family)
+    udp_by_src[srcip] = chan, now + 30
+
+    hdr = "%s,%r," % (dstip[0], dstip[1])
+    mux.send(chan, ssnet.CMD_UDP_DATA, hdr + data)
+
+    expire_connections(now, mux)
+
+
+def dns_done(chan, data, method, sock, srcip, dstip, mux):
+    debug3('dns_done: channel=%d src=%r dst=%r\n' % (chan, srcip, dstip))
+    del mux.channels[chan]
+    del dnsreqs[chan]
+    method.send_udp(sock, srcip, dstip, data)
+
+
+def ondns(listener, method, mux, handlers):
+    now = time.time()
+    t = method.recv_udp(listener, 4096)
+    if t is None:
+        return
+    srcip, dstip, data = t
+    debug1('DNS request from %r to %r: %d bytes\n' % (srcip, dstip, len(data)))
+    chan = mux.next_channel()
+    dnsreqs[chan] = now + 30
+    mux.send(chan, ssnet.CMD_DNS_REQ, data)
+    mux.channels[chan] = lambda cmd, data: dns_done(
+        chan, data, method, listener, srcip=dstip, dstip=srcip, mux=mux)
+    expire_connections(now, mux)
+
+
+def _main(tcp_listener, udp_listener, fw, ssh_cmd, remotename,
+          python, latency_control,
+          dns_listener, seed_hosts, auto_nets,
+          syslog, daemon):
+
+    method = fw.method
+
+    handlers = []
+    if helpers.verbose >= 1:
+        helpers.logprefix = 'c : '
+    else:
+        helpers.logprefix = 'client: '
+    debug1('connecting to server...\n')
+
+    try:
+        (serverproc, serversock) = ssh.connect(
+            ssh_cmd, remotename, python,
+            stderr=ssyslog._p and ssyslog._p.stdin,
+            options=dict(latency_control=latency_control))
+    except socket.error as e:
+        if e.args[0] == errno.EPIPE:
+            raise Fatal("failed to establish ssh session (1)")
+        else:
+            raise
+    mux = Mux(serversock, serversock)
+    handlers.append(mux)
+
+    expected = b'SSHUTTLE0001'
+
+    try:
+        v = 'x'
+        while v and v != b'\0':
+            v = serversock.recv(1)
+        v = 'x'
+        while v and v != b'\0':
+            v = serversock.recv(1)
+        initstring = serversock.recv(len(expected))
+    except socket.error as e:
+        if e.args[0] == errno.ECONNRESET:
+            raise Fatal("failed to establish ssh session (2)")
+        else:
+            raise
+
+    rv = serverproc.poll()
+    if rv:
+        raise Fatal('server died with error code %d' % rv)
+
+    if initstring != expected:
+        raise Fatal('expected server init string %r; got %r'
+                    % (expected, initstring))
+    debug1('connected.\n')
+    print('Connected.')
+    sys.stdout.flush()
+    if daemon:
+        daemonize()
+        log('daemonizing (%s).\n' % _pidname)
+    elif syslog:
+        debug1('switching to syslog.\n')
+        ssyslog.stderr_to_syslog()
+
+    def onroutes(routestr):
+        if auto_nets:
+            for line in routestr.strip().split('\n'):
+                (family, ip, width) = line.split(',', 2)
+                fw.auto_nets.append((int(family), ip, int(width)))
+
+        # we definitely want to do this *after* starting ssh, or we might end
+        # up intercepting the ssh connection!
+        #
+        # Moreover, now that we have the --auto-nets option, we have to wait
+        # for the server to send us that message anyway.  Even if we haven't
+        # set --auto-nets, we might as well wait for the message first, then
+        # ignore its contents.
+        mux.got_routes = None
+        fw.start()
+    mux.got_routes = onroutes
+
+    def onhostlist(hostlist):
+        debug2('got host list: %r\n' % hostlist)
+        for line in hostlist.strip().split():
+            if line:
+                name, ip = line.split(',', 1)
+                fw.sethostip(name, ip)
+    mux.got_host_list = onhostlist
+
+    tcp_listener.add_handler(handlers, onaccept_tcp, method, mux)
+
+    if udp_listener:
+        udp_listener.add_handler(handlers, onaccept_udp, method, mux)
+
+    if dns_listener:
+        dns_listener.add_handler(handlers, ondns, method, mux)
+
+    if seed_hosts is not None:
+        debug1('seed_hosts: %r\n' % seed_hosts)
+        mux.send(0, ssnet.CMD_HOST_REQ, '\n'.join(seed_hosts))
+
+    while 1:
+        rv = serverproc.poll()
+        if rv:
+            raise Fatal('server died with error code %d' % rv)
+
+        ssnet.runonce(handlers, mux)
+        if latency_control:
+            mux.check_fullness()
+
+
+def main(listenip_v6, listenip_v4,
+         ssh_cmd, remotename, python, latency_control, dns, nslist,
+         method_name, seed_hosts, auto_nets,
+         subnets_include, subnets_exclude, syslog, daemon, pidfile):
+
+    if syslog:
+        ssyslog.start_syslog()
+    if daemon:
+        try:
+            check_daemon(pidfile)
+        except Fatal as e:
+            log("%s\n" % e)
+            return 5
+    debug1('Starting sshuttle proxy.\n')
+
+    fw = FirewallClient(method_name)
+
+    features = fw.method.get_supported_features()
+    if listenip_v6 == "auto":
+        if features.ipv6:
+            listenip_v6 = ('::1', 0)
+        else:
+            listenip_v6 = None
+
+    if listenip_v4 == "auto":
+        listenip_v4 = ('127.0.0.1', 0)
+
+    udp = features.udp
+    debug1("UDP enabled: %r\n" % udp)
+
+    if listenip_v6 and listenip_v6[1] and listenip_v4 and listenip_v4[1]:
+        # if both ports given, no need to search for a spare port
+        ports = [0, ]
+    else:
+        # if at least one port missing, we have to search
+        ports = range(12300, 9000, -1)
+
+    # search for free ports and try to bind
+    last_e = None
+    redirectport_v6 = 0
+    redirectport_v4 = 0
+    bound = False
+    debug2('Binding redirector:')
+    for port in ports:
+        debug2(' %d' % port)
+        tcp_listener = MultiListener()
+        tcp_listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+
+        if udp:
+            udp_listener = MultiListener(socket.SOCK_DGRAM)
+            udp_listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        else:
+            udp_listener = None
+
+        if listenip_v6 and listenip_v6[1]:
+            lv6 = listenip_v6
+            redirectport_v6 = lv6[1]
+        elif listenip_v6:
+            lv6 = (listenip_v6[0], port)
+            redirectport_v6 = port
+        else:
+            lv6 = None
+            redirectport_v6 = 0
+
+        if listenip_v4 and listenip_v4[1]:
+            lv4 = listenip_v4
+            redirectport_v4 = lv4[1]
+        elif listenip_v4:
+            lv4 = (listenip_v4[0], port)
+            redirectport_v4 = port
+        else:
+            lv4 = None
+            redirectport_v4 = 0
+
+        try:
+            tcp_listener.bind(lv6, lv4)
+            if udp_listener:
+                udp_listener.bind(lv6, lv4)
+            bound = True
+            break
+        except socket.error as e:
+            if e.errno == errno.EADDRINUSE:
+                last_e = e
+            else:
+                raise e
+
+    debug2('\n')
+    if not bound:
+        assert(last_e)
+        raise last_e
+    tcp_listener.listen(10)
+    tcp_listener.print_listening("TCP redirector")
+    if udp_listener:
+        udp_listener.print_listening("UDP redirector")
+
+    bound = False
+    if dns or nslist:
+        if dns:
+            nslist += resolvconf_nameservers()
+        dns = True
+        # search for spare port for DNS
+        debug2('Binding DNS:')
+        ports = range(12300, 9000, -1)
+        for port in ports:
+            debug2(' %d' % port)
+            dns_listener = MultiListener(socket.SOCK_DGRAM)
+
+            if listenip_v6:
+                lv6 = (listenip_v6[0], port)
+                dnsport_v6 = port
+            else:
+                lv6 = None
+                dnsport_v6 = 0
+
+            if listenip_v4:
+                lv4 = (listenip_v4[0], port)
+                dnsport_v4 = port
+            else:
+                lv4 = None
+                dnsport_v4 = 0
+
+            try:
+                dns_listener.bind(lv6, lv4)
+                bound = True
+                break
+            except socket.error as e:
+                if e.errno == errno.EADDRINUSE:
+                    last_e = e
+                else:
+                    raise e
+        debug2('\n')
+        dns_listener.print_listening("DNS")
+        if not bound:
+            assert(last_e)
+            raise last_e
+    else:
+        dnsport_v6 = 0
+        dnsport_v4 = 0
+        dns_listener = None
+
+    fw.method.check_settings(udp, dns)
+    fw.method.setup_tcp_listener(tcp_listener)
+    if udp_listener:
+        fw.method.setup_udp_listener(udp_listener)
+    if dns_listener:
+        fw.method.setup_udp_listener(dns_listener)
+
+    fw.setup(subnets_include, subnets_exclude, nslist,
+             redirectport_v6, redirectport_v4, dnsport_v6, dnsport_v4,
+             udp)
+
+    try:
+        return _main(tcp_listener, udp_listener, fw, ssh_cmd, remotename,
+                     python, latency_control, dns_listener,
+                     seed_hosts, auto_nets, syslog,
+                     daemon)
+    finally:
+        try:
+            if daemon:
+                # it's not our child anymore; can't waitpid
+                fw.p.returncode = 0
+            fw.done()
+        finally:
+            if daemon:
+                daemon_cleanup()

sshuttle/firewall.py

@@ -0,0 +1,231 @@
+import errno
+import socket
+import signal
+import sshuttle.ssyslog as ssyslog
+import sys
+import os
+from sshuttle.helpers import debug1, debug2, Fatal
+from sshuttle.methods import get_auto_method, get_method
+
+hostmap = {}
+HOSTSFILE = '/etc/hosts'
+
+
+def rewrite_etc_hosts(port):
+    BAKFILE = '%s.sbak' % HOSTSFILE
+    APPEND = '# sshuttle-firewall-%d AUTOCREATED' % port
+    old_content = ''
+    st = None
+    try:
+        old_content = open(HOSTSFILE).read()
+        st = os.stat(HOSTSFILE)
+    except IOError as e:
+        if e.errno == errno.ENOENT:
+            pass
+        else:
+            raise
+    if old_content.strip() and not os.path.exists(BAKFILE):
+        os.link(HOSTSFILE, BAKFILE)
+    tmpname = "%s.%d.tmp" % (HOSTSFILE, port)
+    f = open(tmpname, 'w')
+    for line in old_content.rstrip().split('\n'):
+        if line.find(APPEND) >= 0:
+            continue
+        f.write('%s\n' % line)
+    for (name, ip) in sorted(hostmap.items()):
+        f.write('%-30s %s\n' % ('%s %s' % (ip, name), APPEND))
+    f.close()
+
+    if st:
+        os.chown(tmpname, st.st_uid, st.st_gid)
+        os.chmod(tmpname, st.st_mode)
+    else:
+        os.chown(tmpname, 0, 0)
+        os.chmod(tmpname, 0o644)
+    os.rename(tmpname, HOSTSFILE)
+
+
+def restore_etc_hosts(port):
+    global hostmap
+    hostmap = {}
+    rewrite_etc_hosts(port)
+
+
+# Isolate function that needs to be replaced for tests
+def setup_daemon():
+    if os.getuid() != 0:
+        raise Fatal('you must be root (or enable su/sudo) to set the firewall')
+
+    # don't disappear if our controlling terminal or stdout/stderr
+    # disappears; we still have to clean up.
+    signal.signal(signal.SIGHUP, signal.SIG_IGN)
+    signal.signal(signal.SIGPIPE, signal.SIG_IGN)
+    signal.signal(signal.SIGTERM, signal.SIG_IGN)
+    signal.signal(signal.SIGINT, signal.SIG_IGN)
+
+    # ctrl-c shouldn't be passed along to me.  When the main sshuttle dies,
+    # I'll die automatically.
+    os.setsid()
+
+    # because of limitations of the 'su' command, the *real* stdin/stdout
+    # are both attached to stdout initially.  Clone stdout into stdin so we
+    # can read from it.
+    os.dup2(1, 0)
+
+    return sys.stdin, sys.stdout
+
+
+# This is some voodoo for setting up the kernel's transparent
+# proxying stuff.  If subnets is empty, we just delete our sshuttle rules;
+# otherwise we delete it, then make them from scratch.
+#
+# This code is supposed to clean up after itself by deleting its rules on
+# exit.  In case that fails, it's not the end of the world; future runs will
+# supercede it in the transproxy list, at least, so the leftover rules
+# are hopefully harmless.
+def main(method_name, syslog):
+    stdin, stdout = setup_daemon()
+
+    if method_name == "auto":
+        method = get_auto_method()
+    else:
+        method = get_method(method_name)
+
+    if syslog:
+        ssyslog.start_syslog()
+        ssyslog.stderr_to_syslog()
+
+    debug1('firewall manager ready method name %s.\n' % method.name)
+    stdout.write('READY %s\n' % method.name)
+    stdout.flush()
+
+    # we wait until we get some input before creating the rules.  That way,
+    # sshuttle can launch us as early as possible (and get sudo password
+    # authentication as early in the startup process as possible).
+    line = stdin.readline(128)
+    if not line:
+        return  # parent died; nothing to do
+
+    subnets = []
+    if line != 'ROUTES\n':
+        raise Fatal('firewall: expected ROUTES but got %r' % line)
+    while 1:
+        line = stdin.readline(128)
+        if not line:
+            raise Fatal('firewall: expected route but got %r' % line)
+        elif line.startswith("NSLIST\n"):
+            break
+        try:
+            (family, width, exclude, ip) = line.strip().split(',', 3)
+        except:
+            raise Fatal('firewall: expected route or NSLIST but got %r' % line)
+        subnets.append((int(family), int(width), bool(int(exclude)), ip))
+    debug2('Got subnets: %r\n' % subnets)
+
+    nslist = []
+    if line != 'NSLIST\n':
+        raise Fatal('firewall: expected NSLIST but got %r' % line)
+    while 1:
+        line = stdin.readline(128)
+        if not line:
+            raise Fatal('firewall: expected nslist but got %r' % line)
+        elif line.startswith("PORTS "):
+            break
+        try:
+            (family, ip) = line.strip().split(',', 1)
+        except:
+            raise Fatal('firewall: expected nslist or PORTS but got %r' % line)
+        nslist.append((int(family), ip))
+        debug2('Got partial nslist: %r\n' % nslist)
+    debug2('Got nslist: %r\n' % nslist)
+
+    if not line.startswith('PORTS '):
+        raise Fatal('firewall: expected PORTS but got %r' % line)
+    _, _, ports = line.partition(" ")
+    ports = ports.split(",")
+    if len(ports) != 4:
+        raise Fatal('firewall: expected 4 ports but got %n' % len(ports))
+    port_v6 = int(ports[0])
+    port_v4 = int(ports[1])
+    dnsport_v6 = int(ports[2])
+    dnsport_v4 = int(ports[3])
+
+    assert(port_v6 >= 0)
+    assert(port_v6 <= 65535)
+    assert(port_v4 >= 0)
+    assert(port_v4 <= 65535)
+    assert(dnsport_v6 >= 0)
+    assert(dnsport_v6 <= 65535)
+    assert(dnsport_v4 >= 0)
+    assert(dnsport_v4 <= 65535)
+
+    debug2('Got ports: %d,%d,%d,%d\n'
+           % (port_v6, port_v4, dnsport_v6, dnsport_v4))
+
+    line = stdin.readline(128)
+    if not line:
+        raise Fatal('firewall: expected GO but got %r' % line)
+    elif not line.startswith("GO "):
+        raise Fatal('firewall: expected GO but got %r' % line)
+
+    _, _, udp = line.partition(" ")
+    udp = bool(int(udp))
+    debug2('Got udp: %r\n' % udp)
+
+    try:
+        do_wait = None
+        debug1('firewall manager: starting transproxy.\n')
+
+        nslist_v6 = [i for i in nslist if i[0] == socket.AF_INET6]
+        subnets_v6 = [i for i in subnets if i[0] == socket.AF_INET6]
+        if port_v6 > 0:
+            do_wait = method.setup_firewall(
+                port_v6, dnsport_v6, nslist_v6,
+                socket.AF_INET6, subnets_v6, udp)
+        elif len(subnets_v6) > 0:
+            debug1("IPv6 subnets defined but IPv6 disabled\n")
+
+        nslist_v4 = [i for i in nslist if i[0] == socket.AF_INET]
+        subnets_v4 = [i for i in subnets if i[0] == socket.AF_INET]
+        if port_v4 > 0:
+            do_wait = method.setup_firewall(
+                port_v4, dnsport_v4, nslist_v4,
+                socket.AF_INET, subnets_v4, udp)
+        elif len(subnets_v4) > 0:
+            debug1('IPv4 subnets defined but IPv4 disabled\n')
+
+        stdout.write('STARTED\n')
+
+        try:
+            stdout.flush()
+        except IOError:
+            # the parent process died for some reason; he's surely been loud
+            # enough, so no reason to report another error
+            return
+
+        # Now we wait until EOF or any other kind of exception.  We need
+        # to stay running so that we don't need a *second* password
+        # authentication at shutdown time - that cleanup is important!
+        while 1:
+            if do_wait is not None:
+                do_wait()
+            line = stdin.readline(128)
+            if line.startswith('HOST '):
+                (name, ip) = line[5:].strip().split(',', 1)
+                hostmap[name] = ip
+                rewrite_etc_hosts(port_v6 or port_v4)
+            elif line:
+                if not method.firewall_command(line):
+                    raise Fatal('expected EOF, got %r' % line)
+            else:
+                break
+    finally:
+        try:
+            debug1('firewall manager: undoing changes.\n')
+        except:
+            pass
+        if port_v6:
+            method.setup_firewall(port_v6, 0, [], socket.AF_INET6, [], udp)
+        if port_v4:
+            method.setup_firewall(port_v4, 0, [], socket.AF_INET, [], udp)
+        restore_etc_hosts(port_v6 or port_v4)

sshuttle/helpers.py

@@ -0,0 +1,88 @@
+import sys
+import socket
+import errno
+
+logprefix = ''
+verbose = 0
+
+
+def log(s):
+    try:
+        sys.stdout.flush()
+        sys.stderr.write(logprefix + s)
+        sys.stderr.flush()
+    except IOError:
+        # this could happen if stderr gets forcibly disconnected, eg. because
+        # our tty closes.  That sucks, but it's no reason to abort the program.
+        pass
+
+
+def debug1(s):
+    if verbose >= 1:
+        log(s)
+
+
+def debug2(s):
+    if verbose >= 2:
+        log(s)
+
+
+def debug3(s):
+    if verbose >= 3:
+        log(s)
+
+
+class Fatal(Exception):
+    pass
+
+
+def resolvconf_nameservers():
+    l = []
+    for line in open('/etc/resolv.conf'):
+        words = line.lower().split()
+        if len(words) >= 2 and words[0] == 'nameserver':
+            l.append(family_ip_tuple(words[1]))
+    return l
+
+
+def resolvconf_random_nameserver():
+    l = resolvconf_nameservers()
+    if l:
+        if len(l) > 1:
+            # don't import this unless we really need it
+            import random
+            random.shuffle(l)
+        return l[0]
+    else:
+        return (socket.AF_INET, '127.0.0.1')
+
+
+def islocal(ip, family):
+    sock = socket.socket(family)
+    try:
+        try:
+            sock.bind((ip, 0))
+        except socket.error as e:
+            if e.args[0] == errno.EADDRNOTAVAIL:
+                return False  # not a local IP
+            else:
+                raise
+    finally:
+        sock.close()
+    return True  # it's a local IP, or there would have been an error
+
+
+def family_ip_tuple(ip):
+    if ':' in ip:
+        return (socket.AF_INET6, ip)
+    else:
+        return (socket.AF_INET, ip)
+
+
+def family_to_string(family):
+    if family == socket.AF_INET6:
+        return "AF_INET6"
+    elif family == socket.AF_INET:
+        return "AF_INET"
+    else:
+        return str(family)

sshuttle/hostwatch.py

@@ -1,18 +1,29 @@
-import subprocess, time, socket, re, select
+import time
-if not globals().get('skip_imports'):
+import socket
-    import helpers
+import re
-    from helpers import *
+import select
+import errno
+import os
+import sys
 
-POLL_TIME = 60*15
+import subprocess as ssubprocess
+import sshuttle.helpers as helpers
+from sshuttle.helpers import log, debug1, debug2, debug3
+
+POLL_TIME = 60 * 15
 NETSTAT_POLL_TIME = 30
-CACHEFILE=os.path.expanduser('~/.sshuttle.hosts')
+CACHEFILE = os.path.expanduser('~/.sshuttle.hosts')
 
 
 _nmb_ok = True
 _smb_ok = True
 hostnames = {}
 queue = {}
-null = open('/dev/null', 'rb+')
+try:
+    null = open('/dev/null', 'wb')
+except IOError as e:
+    log('warning: %s\n' % e)
+    null = os.popen("sh -c 'while read x; do :; done'", 'wb', 4096)
 
 
 def _is_ip(s):
@@ -23,7 +34,7 @@ def write_host_cache():
     tmpname = '%s.%d.tmp' % (CACHEFILE, os.getpid())
     try:
         f = open(tmpname, 'wb')
-        for name,ip in sorted(hostnames.items()):
+        for name, ip in sorted(hostnames.items()):
             f.write('%s,%s\n' % (name, ip))
         f.close()
         os.rename(tmpname, CACHEFILE)
@@ -37,7 +48,7 @@ def write_host_cache():
 def read_host_cache():
     try:
         f = open(CACHEFILE)
-    except IOError, e:
+    except IOError as e:
         if e.errno == errno.ENOENT:
             return
         else:
@@ -45,18 +56,18 @@ def read_host_cache():
     for line in f:
         words = line.strip().split(',')
         if len(words) == 2:
-            (name,ip) = words
+            (name, ip) = words
             name = re.sub(r'[^-\w]', '-', name).strip()
             ip = re.sub(r'[^0-9.]', '', ip).strip()
             if name and ip:
                 found_host(name, ip)
-                
+
 
 def found_host(hostname, ip):
     hostname = re.sub(r'\..*', '', hostname)
     hostname = re.sub(r'[^-\w]', '_', hostname)
-    if (ip.startswith('127.') or ip.startswith('255.') 
+    if (ip.startswith('127.') or ip.startswith('255.')
-        or hostname == 'localhost'):
+            or hostname == 'localhost'):
         return
     oldip = hostnames.get(hostname)
     if oldip != ip:
@@ -89,7 +100,7 @@ def _check_revdns(ip):
         debug3('<    %s\n' % r[0])
         check_host(r[0])
         found_host(r[0], ip)
-    except socket.herror, e:
+    except socket.herror:
         pass
 
 
@@ -100,7 +111,7 @@ def _check_dns(hostname):
         debug3('<    %s\n' % ip)
         check_host(ip)
         found_host(hostname, ip)
-    except socket.gaierror, e:
+    except socket.gaierror:
         pass
 
 
@@ -108,17 +119,17 @@ def _check_netstat():
     debug2(' > netstat\n')
     argv = ['netstat', '-n']
     try:
-        p = subprocess.Popen(argv, stdout=subprocess.PIPE, stderr=null)
+        p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, stderr=null)
         content = p.stdout.read()
         p.wait()
-    except OSError, e:
+    except OSError as e:
         log('%r failed: %r\n' % (argv, e))
         return
 
     for ip in re.findall(r'\d+\.\d+\.\d+\.\d+', content):
         debug3('<    %s\n' % ip)
         check_host(ip)
-        
+
 
 def _check_smb(hostname):
     return
@@ -128,10 +139,10 @@ def _check_smb(hostname):
     argv = ['smbclient', '-U', '%', '-L', hostname]
     debug2(' > smb: %s\n' % hostname)
     try:
-        p = subprocess.Popen(argv, stdout=subprocess.PIPE, stderr=null)
+        p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, stderr=null)
         lines = p.stdout.readlines()
         p.wait()
-    except OSError, e:
+    except OSError as e:
         log('%r failed: %r\n' % (argv, e))
         _smb_ok = False
         return
@@ -182,13 +193,13 @@ def _check_nmb(hostname, is_workgroup, is_master):
     global _nmb_ok
     if not _nmb_ok:
         return
-    argv = ['nmblookup'] + ['-M']*is_master + ['--', hostname]
+    argv = ['nmblookup'] + ['-M'] * is_master + ['--', hostname]
     debug2(' > n%d%d: %s\n' % (is_workgroup, is_master, hostname))
     try:
-        p = subprocess.Popen(argv, stdout=subprocess.PIPE, stderr=null)
+        p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, stderr=null)
         lines = p.stdout.readlines()
         rv = p.wait()
-    except OSError, e:
+    except OSError as e:
         log('%r failed: %r\n' % (argv, e))
         _nmb_ok = False
         return
@@ -223,13 +234,13 @@ def check_workgroup(hostname):
 
 
 def _enqueue(op, *args):
-    t = (op,args)
+    t = (op, args)
-    if queue.get(t) == None:
+    if queue.get(t) is None:
         queue[t] = 0
 
 
 def _stdin_still_ok(timeout):
-    r,w,x = select.select([sys.stdin.fileno()], [], [], timeout)
+    r, w, x = select.select([sys.stdin.fileno()], [], [], timeout)
     if r:
         b = os.read(sys.stdin.fileno(), 4096)
         if not b:
@@ -244,7 +255,7 @@ def hw_main(seed_hosts):
         helpers.logprefix = 'hostwatch: '
 
     read_host_cache()
-        
+
     _enqueue(_check_etc_hosts)
     _enqueue(_check_netstat)
     check_host('localhost')
@@ -256,8 +267,8 @@ def hw_main(seed_hosts):
 
     while 1:
         now = time.time()
-        for t,last_polled in queue.items():
+        for t, last_polled in list(queue.items()):
-            (op,args) = t
+            (op, args) = t
             if not _stdin_still_ok(0):
                 break
             maxtime = POLL_TIME
@@ -270,7 +281,7 @@ def hw_main(seed_hosts):
                 sys.stdout.flush()
             except IOError:
                 break
-                
+
         # FIXME: use a smarter timeout based on oldest last_polled
         if not _stdin_still_ok(1):
             break

sshuttle/linux.py

@@ -0,0 +1,62 @@
+import socket
+import subprocess as ssubprocess
+from sshuttle.helpers import log, debug1, Fatal, family_to_string
+
+
+def nonfatal(func, *args):
+    try:
+        func(*args)
+    except Fatal as e:
+        log('error: %s\n' % e)
+
+
+def ipt_chain_exists(family, table, name):
+    if family == socket.AF_INET6:
+        cmd = 'ip6tables'
+    elif family == socket.AF_INET:
+        cmd = 'iptables'
+    else:
+        raise Exception('Unsupported family "%s"' % family_to_string(family))
+    argv = [cmd, '-t', table, '-nL']
+    p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE)
+    for line in p.stdout:
+        if line.startswith(b'Chain %s ' % name.encode("ASCII")):
+            return True
+    rv = p.wait()
+    if rv:
+        raise Fatal('%r returned %d' % (argv, rv))
+
+
+def ipt(family, table, *args):
+    if family == socket.AF_INET6:
+        argv = ['ip6tables', '-t', table] + list(args)
+    elif family == socket.AF_INET:
+        argv = ['iptables', '-t', table] + list(args)
+    else:
+        raise Exception('Unsupported family "%s"' % family_to_string(family))
+    debug1('>> %s\n' % ' '.join(argv))
+    rv = ssubprocess.call(argv)
+    if rv:
+        raise Fatal('%r returned %d' % (argv, rv))
+
+
+_no_ttl_module = False
+
+
+def ipt_ttl(family, *args):
+    global _no_ttl_module
+    if not _no_ttl_module:
+        # we avoid infinite loops by generating server-side connections
+        # with ttl 42.  This makes the client side not recapture those
+        # connections, in case client == server.
+        try:
+            argsplus = list(args) + ['-m', 'ttl', '!', '--ttl', '42']
+            ipt(family, *argsplus)
+        except Fatal:
+            ipt(family, *args)
+            # we only get here if the non-ttl attempt succeeds
+            log('sshuttle: warning: your iptables is missing '
+                'the ttl module.\n')
+            _no_ttl_module = True
+    else:
+        ipt(family, *args)

sshuttle/methods/__init__.py

@@ -0,0 +1,99 @@
+import os
+import importlib
+import socket
+import struct
+import errno
+from sshuttle.helpers import Fatal, debug3
+
+
+def original_dst(sock):
+    try:
+        SO_ORIGINAL_DST = 80
+        SOCKADDR_MIN = 16
+        sockaddr_in = sock.getsockopt(socket.SOL_IP,
+                                      SO_ORIGINAL_DST, SOCKADDR_MIN)
+        (proto, port, a, b, c, d) = struct.unpack('!HHBBBB', sockaddr_in[:8])
+        # FIXME: decoding is IPv4 only.
+        assert(socket.htons(proto) == socket.AF_INET)
+        ip = '%d.%d.%d.%d' % (a, b, c, d)
+        return (ip, port)
+    except socket.error as e:
+        if e.args[0] == errno.ENOPROTOOPT:
+            return sock.getsockname()
+        raise
+
+
+class Features(object):
+    pass
+
+
+class BaseMethod(object):
+    def __init__(self, name):
+        self.firewall = None
+        self.name = name
+
+    def set_firewall(self, firewall):
+        self.firewall = firewall
+
+    def get_supported_features(self):
+        result = Features()
+        result.ipv6 = False
+        result.udp = False
+        return result
+
+    def get_tcp_dstip(self, sock):
+        return original_dst(sock)
+
+    def recv_udp(self, udp_listener, bufsize):
+        debug3('Accept UDP using recvfrom.\n')
+        data, srcip = udp_listener.recvfrom(bufsize)
+        return (srcip, None, data)
+
+    def send_udp(self, sock, srcip, dstip, data):
+        if srcip is not None:
+            Fatal("Method %s send_udp does not support setting srcip to %r"
+                  % (self.name, srcip))
+        sock.sendto(data, dstip)
+
+    def setup_tcp_listener(self, tcp_listener):
+        pass
+
+    def setup_udp_listener(self, udp_listener):
+        pass
+
+    def check_settings(self, udp, dns):
+        if udp:
+            Fatal("UDP support not supported with method %s.\n" % self.name)
+
+    def setup_firewall(self, port, dnsport, nslist, family, subnets, udp):
+        raise NotImplementedError()
+
+    def firewall_command(self, line):
+        return False
+
+
+def _program_exists(name):
+    paths = (os.getenv('PATH') or os.defpath).split(os.pathsep)
+    for p in paths:
+        fn = '%s/%s' % (p, name)
+        if os.path.exists(fn):
+            return not os.path.isdir(fn) and os.access(fn, os.X_OK)
+
+
+def get_method(method_name):
+    module = importlib.import_module("sshuttle.methods.%s" % method_name)
+    return module.Method(method_name)
+
+
+def get_auto_method():
+    if _program_exists('ipfw'):
+        method_name = "ipfw"
+    elif _program_exists('iptables'):
+        method_name = "nat"
+    elif _program_exists('pfctl'):
+        method_name = "pf"
+    else:
+        raise Fatal(
+            "can't find either ipfw, iptables or pfctl; check your PATH")
+
+    return get_method(method_name)

sshuttle/methods/ipfw.py

@@ -0,0 +1,237 @@
+import sys
+import select
+import socket
+import struct
+import subprocess as ssubprocess
+from sshuttle.helpers import log, debug1, debug3, islocal, \
+    Fatal, family_to_string
+from sshuttle.methods import BaseMethod
+
+
+# python doesn't have a definition for this
+IPPROTO_DIVERT = 254
+
+
+def ipfw_rule_exists(n):
+    argv = ['ipfw', 'list']
+    p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE)
+    found = False
+    for line in p.stdout:
+        if line.startswith('%05d ' % n):
+            if not ('ipttl 42' in line
+                    or ('skipto %d' % (n + 1)) in line
+                    or 'check-state' in line):
+                log('non-sshuttle ipfw rule: %r\n' % line.strip())
+                raise Fatal('non-sshuttle ipfw rule #%d already exists!' % n)
+            found = True
+    rv = p.wait()
+    if rv:
+        raise Fatal('%r returned %d' % (argv, rv))
+    return found
+
+
+_oldctls = {}
+
+
+def _fill_oldctls(prefix):
+    argv = ['sysctl', prefix]
+    p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE)
+    for line in p.stdout:
+        assert(line[-1] == '\n')
+        (k, v) = line[:-1].split(': ', 1)
+        _oldctls[k] = v
+    rv = p.wait()
+    if rv:
+        raise Fatal('%r returned %d' % (argv, rv))
+    if not line:
+        raise Fatal('%r returned no data' % (argv,))
+
+
+def _sysctl_set(name, val):
+    argv = ['sysctl', '-w', '%s=%s' % (name, val)]
+    debug1('>> %s\n' % ' '.join(argv))
+    return ssubprocess.call(argv, stdout=open('/dev/null', 'w'))
+
+
+_changedctls = []
+
+
+def sysctl_set(name, val, permanent=False):
+    PREFIX = 'net.inet.ip'
+    assert(name.startswith(PREFIX + '.'))
+    val = str(val)
+    if not _oldctls:
+        _fill_oldctls(PREFIX)
+    if not (name in _oldctls):
+        debug1('>> No such sysctl: %r\n' % name)
+        return False
+    oldval = _oldctls[name]
+    if val != oldval:
+        rv = _sysctl_set(name, val)
+        if rv == 0 and permanent:
+            debug1('>>   ...saving permanently in /etc/sysctl.conf\n')
+            f = open('/etc/sysctl.conf', 'a')
+            f.write('\n'
+                    '# Added by sshuttle\n'
+                    '%s=%s\n' % (name, val))
+            f.close()
+        else:
+            _changedctls.append(name)
+        return True
+
+
+def _udp_unpack(p):
+    src = (socket.inet_ntoa(p[12:16]), struct.unpack('!H', p[20:22])[0])
+    dst = (socket.inet_ntoa(p[16:20]), struct.unpack('!H', p[22:24])[0])
+    return src, dst
+
+
+def _udp_repack(p, src, dst):
+    addrs = socket.inet_aton(src[0]) + socket.inet_aton(dst[0])
+    ports = struct.pack('!HH', src[1], dst[1])
+    return p[:12] + addrs + ports + p[24:]
+
+
+_real_dns_server = [None]
+
+
+def _handle_diversion(divertsock, dnsport):
+    p, tag = divertsock.recvfrom(4096)
+    src, dst = _udp_unpack(p)
+    debug3('got diverted packet from %r to %r\n' % (src, dst))
+    if dst[1] == 53:
+        # outgoing DNS
+        debug3('...packet is a DNS request.\n')
+        _real_dns_server[0] = dst
+        dst = ('127.0.0.1', dnsport)
+    elif src[1] == dnsport:
+        if islocal(src[0], divertsock.family):
+            debug3('...packet is a DNS response.\n')
+            src = _real_dns_server[0]
+    else:
+        log('weird?! unexpected divert from %r to %r\n' % (src, dst))
+        assert(0)
+    newp = _udp_repack(p, src, dst)
+    divertsock.sendto(newp, tag)
+
+
+def ipfw(*args):
+    argv = ['ipfw', '-q'] + list(args)
+    debug1('>> %s\n' % ' '.join(argv))
+    rv = ssubprocess.call(argv)
+    if rv:
+        raise Fatal('%r returned %d' % (argv, rv))
+
+
+class Method(BaseMethod):
+
+    def setup_firewall(self, port, dnsport, nslist, family, subnets, udp):
+        # IPv6 not supported
+        if family not in [socket.AF_INET, ]:
+            raise Exception(
+                'Address family "%s" unsupported by ipfw method_name'
+                % family_to_string(family))
+        if udp:
+            raise Exception("UDP not supported by ipfw method_name")
+
+        sport = str(port)
+        xsport = str(port + 1)
+
+        # cleanup any existing rules
+        if ipfw_rule_exists(port):
+            ipfw('delete', sport)
+
+        while _changedctls:
+            name = _changedctls.pop()
+            oldval = _oldctls[name]
+            _sysctl_set(name, oldval)
+
+        if subnets or dnsport:
+            sysctl_set('net.inet.ip.fw.enable', 1)
+            changed = sysctl_set('net.inet.ip.scopedroute', 0, permanent=True)
+            if changed:
+                log("\n"
+                    "        WARNING: ONE-TIME NETWORK DISRUPTION:\n"
+                    "        =====================================\n"
+                    "sshuttle has changed a MacOS kernel setting to work around\n"
+                    "a bug in MacOS 10.6.  This will cause your network to drop\n"
+                    "within 5-10 minutes unless you restart your network\n"
+                    "interface (change wireless networks or unplug/plug the\n"
+                    "ethernet port) NOW, then restart sshuttle.  The fix is\n"
+                    "permanent; you only have to do this once.\n\n")
+                sys.exit(1)
+
+            ipfw('add', sport, 'check-state', 'ip',
+                 'from', 'any', 'to', 'any')
+
+        if subnets:
+            # create new subnet entries
+            for f, swidth, sexclude, snet \
+                    in sorted(subnets, key=lambda s: s[1], reverse=True):
+                if sexclude:
+                    ipfw('add', sport, 'skipto', xsport,
+                         'tcp',
+                         'from', 'any', 'to', '%s/%s' % (snet, swidth))
+                else:
+                    ipfw('add', sport, 'fwd', '127.0.0.1,%d' % port,
+                         'tcp',
+                         'from', 'any', 'to', '%s/%s' % (snet, swidth),
+                         'not', 'ipttl', '42', 'keep-state', 'setup')
+
+        # This part is much crazier than it is on Linux, because MacOS (at
+        # least 10.6, and probably other versions, and maybe FreeBSD too)
+        # doesn't correctly fixup the dstip/dstport for UDP packets when it
+        # puts them through a 'fwd' rule.  It also doesn't fixup the
+        # srcip/srcport in the response packet.  In Linux iptables, all that
+        # happens magically for us, so we just redirect the packets and relax.
+        #
+        # On MacOS, we have to fix the ports ourselves.  For that, we use a
+        # 'divert' socket, which receives raw packets and lets us mangle them.
+        #
+        # Here's how it works.  Let's say the local DNS server is 1.1.1.1:53,
+        # and the remote DNS server is 2.2.2.2:53, and the local transproxy
+        # port is 10.0.0.1:12300, and a client machine is making a request from
+        # 10.0.0.5:9999. We see a packet like this:
+        #    10.0.0.5:9999 -> 1.1.1.1:53
+        # Since the destip:port matches one of our local nameservers, it will
+        # match a 'fwd' rule, thus grabbing it on the local machine.  However,
+        # the local kernel will then see a packet addressed to *:53 and not
+        # know what to do with it; there's nobody listening on port 53.  Thus,
+        # we divert it, rewriting it into this:
+        #    10.0.0.5:9999 -> 10.0.0.1:12300
+        # This gets proxied out to the server, which sends it to 2.2.2.2:53,
+        # and the answer comes back, and the proxy sends it back out like this:
+        #    10.0.0.1:12300 -> 10.0.0.5:9999
+        # But that's wrong!  The original machine expected an answer from
+        # 1.1.1.1:53, so we have to divert the *answer* and rewrite it:
+        #    1.1.1.1:53 -> 10.0.0.5:9999
+        #
+        # See?  Easy stuff.
+        if dnsport:
+            divertsock = socket.socket(socket.AF_INET, socket.SOCK_RAW,
+                                       IPPROTO_DIVERT)
+            divertsock.bind(('0.0.0.0', port))  # IP field is ignored
+
+            for f, ip in [i for i in nslist if i[0] == family]:
+                # relabel and then catch outgoing DNS requests
+                ipfw('add', sport, 'divert', sport,
+                     'udp',
+                     'from', 'any', 'to', '%s/32' % ip, '53',
+                     'not', 'ipttl', '42')
+            # relabel DNS responses
+            ipfw('add', sport, 'divert', sport,
+                 'udp',
+                 'from', 'any', str(dnsport), 'to', 'any',
+                 'not', 'ipttl', '42')
+
+            def do_wait():
+                while 1:
+                    r, w, x = select.select([sys.stdin, divertsock], [], [])
+                    if divertsock in r:
+                        _handle_diversion(divertsock, dnsport)
+                    if sys.stdin in r:
+                        return
+        else:
+            do_wait = None
+
+        return do_wait

sshuttle/methods/nat.py

@@ -0,0 +1,70 @@
+import socket
+from sshuttle.helpers import family_to_string
+from sshuttle.linux import ipt, ipt_ttl, ipt_chain_exists, nonfatal
+from sshuttle.methods import BaseMethod
+
+
+class Method(BaseMethod):
+
+    # We name the chain based on the transproxy port number so that it's
+    # possible to run multiple copies of sshuttle at the same time.  Of course,
+    # the multiple copies shouldn't have overlapping subnets, or only the most-
+    # recently-started one will win (because we use "-I OUTPUT 1" instead of
+    # "-A OUTPUT").
+    def setup_firewall(self, port, dnsport, nslist, family, subnets, udp):
+        # only ipv4 supported with NAT
+        if family != socket.AF_INET:
+            raise Exception(
+                'Address family "%s" unsupported by nat method_name'
+                % family_to_string(family))
+        if udp:
+            raise Exception("UDP not supported by nat method_name")
+
+        table = "nat"
+
+        def _ipt(*args):
+            return ipt(family, table, *args)
+
+        def _ipt_ttl(*args):
+            return ipt_ttl(family, table, *args)
+
+        chain = 'sshuttle-%s' % port
+
+        # basic cleanup/setup of chains
+        if ipt_chain_exists(family, table, chain):
+            nonfatal(_ipt, '-D', 'OUTPUT', '-j', chain)
+            nonfatal(_ipt, '-D', 'PREROUTING', '-j', chain)
+            nonfatal(_ipt, '-F', chain)
+            _ipt('-X', chain)
+
+        if subnets or dnsport:
+            _ipt('-N', chain)
+            _ipt('-F', chain)
+            _ipt('-I', 'OUTPUT', '1', '-j', chain)
+            _ipt('-I', 'PREROUTING', '1', '-j', chain)
+
+        if subnets:
+            # create new subnet entries.  Note that we're sorting in a very
+            # particular order: we need to go from most-specific (largest
+            # swidth) to least-specific, and at any given level of specificity,
+            # we want excludes to come first.  That's why the columns are in
+            # such a non- intuitive order.
+            for f, swidth, sexclude, snet \
+                    in sorted(subnets, key=lambda s: s[1], reverse=True):
+                if sexclude:
+                    _ipt('-A', chain, '-j', 'RETURN',
+                         '--dest', '%s/%s' % (snet, swidth),
+                         '-p', 'tcp')
+                else:
+                    _ipt_ttl('-A', chain, '-j', 'REDIRECT',
+                             '--dest', '%s/%s' % (snet, swidth),
+                             '-p', 'tcp',
+                             '--to-ports', str(port))
+
+        if dnsport:
+            for f, ip in [i for i in nslist if i[0] == family]:
+                _ipt_ttl('-A', chain, '-j', 'REDIRECT',
+                         '--dest', '%s/32' % ip,
+                         '-p', 'udp',
+                         '--dport', '53',
+                         '--to-ports', str(dnsport))

sshuttle/methods/pf.py

@@ -0,0 +1,246 @@
+import os
+import sys
+import re
+import socket
+import struct
+import subprocess as ssubprocess
+from fcntl import ioctl
+from ctypes import c_char, c_uint8, c_uint16, c_uint32, Union, Structure, \
+    sizeof, addressof, memmove
+from sshuttle.helpers import debug1, debug2, Fatal, family_to_string
+from sshuttle.methods import BaseMethod
+
+
+def pfctl(args, stdin=None):
+    argv = ['pfctl'] + list(args.split(" "))
+    debug1('>> %s\n' % ' '.join(argv))
+
+    p = ssubprocess.Popen(argv, stdin=ssubprocess.PIPE,
+                          stdout=ssubprocess.PIPE,
+                          stderr=ssubprocess.PIPE)
+    o = p.communicate(stdin)
+    if p.returncode:
+        raise Fatal('%r returned %d' % (argv, p.returncode))
+
+    return o
+
+_pf_context = {'started_by_sshuttle': False, 'Xtoken': ''}
+
+
+# This are some classes and functions used to support pf in yosemite.
+class pf_state_xport(Union):
+    _fields_ = [("port", c_uint16),
+                ("call_id", c_uint16),
+                ("spi", c_uint32)]
+
+
+class pf_addr(Structure):
+
+    class _pfa(Union):
+        _fields_ = [("v4",            c_uint32),      # struct in_addr
+                    ("v6",            c_uint32 * 4),  # struct in6_addr
+                    ("addr8",         c_uint8 * 16),
+                    ("addr16",        c_uint16 * 8),
+                    ("addr32",        c_uint32 * 4)]
+
+    _fields_ = [("pfa",               _pfa)]
+    _anonymous_ = ("pfa",)
+
+
+class pfioc_natlook(Structure):
+    _fields_ = [("saddr", pf_addr),
+                ("daddr", pf_addr),
+                ("rsaddr", pf_addr),
+                ("rdaddr", pf_addr),
+                ("sxport", pf_state_xport),
+                ("dxport", pf_state_xport),
+                ("rsxport", pf_state_xport),
+                ("rdxport", pf_state_xport),
+                ("af", c_uint8),                      # sa_family_t
+                ("proto", c_uint8),
+                ("proto_variant", c_uint8),
+                ("direction", c_uint8)]
+
+pfioc_rule = c_char * 3104  # sizeof(struct pfioc_rule)
+
+pfioc_pooladdr = c_char * 1136  # sizeof(struct pfioc_pooladdr)
+
+MAXPATHLEN = 1024
+
+DIOCNATLOOK = ((0x40000000 | 0x80000000) | (
+    (sizeof(pfioc_natlook) & 0x1fff) << 16) | ((ord('D')) << 8) | (23))
+DIOCCHANGERULE = ((0x40000000 | 0x80000000) | (
+    (sizeof(pfioc_rule) & 0x1fff) << 16) | ((ord('D')) << 8) | (26))
+DIOCBEGINADDRS = ((0x40000000 | 0x80000000) | (
+    (sizeof(pfioc_pooladdr) & 0x1fff) << 16) | ((ord('D')) << 8) | (51))
+
+PF_CHANGE_ADD_TAIL = 2
+PF_CHANGE_GET_TICKET = 6
+
+PF_PASS = 0
+PF_RDR = 8
+
+PF_OUT = 2
+
+_pf_fd = None
+
+
+def pf_get_dev():
+    global _pf_fd
+    if _pf_fd is None:
+        _pf_fd = os.open('/dev/pf', os.O_RDWR)
+
+    return _pf_fd
+
+
+def pf_query_nat(family, proto, src_ip, src_port, dst_ip, dst_port):
+    [proto, family, src_port, dst_port] = [
+        int(v) for v in [proto, family, src_port, dst_port]]
+
+    packed_src_ip = socket.inet_pton(family, src_ip)
+    packed_dst_ip = socket.inet_pton(family, dst_ip)
+
+    assert len(packed_src_ip) == len(packed_dst_ip)
+    length = len(packed_src_ip)
+
+    pnl = pfioc_natlook()
+    pnl.proto = proto
+    pnl.direction = PF_OUT
+    pnl.af = family
+    memmove(addressof(pnl.saddr), packed_src_ip, length)
+    pnl.sxport.port = socket.htons(src_port)
+    memmove(addressof(pnl.daddr), packed_dst_ip, length)
+    pnl.dxport.port = socket.htons(dst_port)
+
+    ioctl(pf_get_dev(), DIOCNATLOOK,
+          (c_char * sizeof(pnl)).from_address(addressof(pnl)))
+
+    ip = socket.inet_ntop(
+        pnl.af, (c_char * length).from_address(addressof(pnl.rdaddr)).raw)
+    port = socket.ntohs(pnl.rdxport.port)
+    return (ip, port)
+
+
+def pf_add_anchor_rule(type, name):
+    ACTION_OFFSET = 0
+    POOL_TICKET_OFFSET = 8
+    ANCHOR_CALL_OFFSET = 1040
+    RULE_ACTION_OFFSET = 3068
+
+    pr = pfioc_rule()
+    ppa = pfioc_pooladdr()
+
+    ioctl(pf_get_dev(), DIOCBEGINADDRS, ppa)
+
+    memmove(addressof(pr) + POOL_TICKET_OFFSET, ppa[4:8], 4)  # pool_ticket
+    memmove(addressof(pr) + ANCHOR_CALL_OFFSET, name,
+            min(MAXPATHLEN, len(name)))  # anchor_call = name
+    memmove(addressof(pr) + RULE_ACTION_OFFSET,
+            struct.pack('I', type), 4)  # rule.action = type
+
+    memmove(addressof(pr) + ACTION_OFFSET, struct.pack(
+        'I', PF_CHANGE_GET_TICKET), 4)  # action = PF_CHANGE_GET_TICKET
+    ioctl(pf_get_dev(), DIOCCHANGERULE, pr)
+
+    memmove(addressof(pr) + ACTION_OFFSET, struct.pack(
+        'I', PF_CHANGE_ADD_TAIL), 4)  # action = PF_CHANGE_ADD_TAIL
+    ioctl(pf_get_dev(), DIOCCHANGERULE, pr)
+
+
+class Method(BaseMethod):
+
+    def get_tcp_dstip(self, sock):
+        pfile = self.firewall.pfile
+
+        peer = sock.getpeername()
+        proxy = sock.getsockname()
+
+        argv = (sock.family, socket.IPPROTO_TCP,
+                peer[0], peer[1], proxy[0], proxy[1])
+        pfile.write("QUERY_PF_NAT %d,%d,%s,%d,%s,%d\n" % argv)
+        pfile.flush()
+        line = pfile.readline()
+        debug2("QUERY_PF_NAT %d,%d,%s,%d,%s,%d" % argv + ' > ' + line)
+        if line.startswith('QUERY_PF_NAT_SUCCESS '):
+            (ip, port) = line[21:].split(',')
+            return (ip, int(port))
+
+        return sock.getsockname()
+
+    def setup_firewall(self, port, dnsport, nslist, family, subnets, udp):
+        global _pf_started_by_sshuttle
+        tables = []
+        translating_rules = []
+        filtering_rules = []
+
+        if family != socket.AF_INET:
+            raise Exception(
+                'Address family "%s" unsupported by pf method_name'
+                % family_to_string(family))
+        if udp:
+            raise Exception("UDP not supported by pf method_name")
+
+        if subnets:
+            includes = []
+            # If a given subnet is both included and excluded, list the
+            # exclusion first; the table will ignore the second, opposite
+            # definition
+            for f, swidth, sexclude, snet in sorted(
+                    subnets, key=lambda s: (s[1], s[2]), reverse=True):
+                includes.append("%s%s/%s" %
+                                ("!" if sexclude else "", snet, swidth))
+
+            tables.append('table <forward_subnets> {%s}' % ','.join(includes))
+            translating_rules.append(
+                'rdr pass on lo0 proto tcp '
+                'to <forward_subnets> -> 127.0.0.1 port %r' % port)
+            filtering_rules.append(
+                'pass out route-to lo0 inet proto tcp '
+                'to <forward_subnets> keep state')
+
+            if dnsport:
+                tables.append('table <dns_servers> {%s}' % ','.join(
+                    [ns[1] for ns in nslist]))
+                translating_rules.append(
+                    'rdr pass on lo0 proto udp to '
+                    '<dns_servers> port 53 -> 127.0.0.1 port %r' % dnsport)
+                filtering_rules.append(
+                    'pass out route-to lo0 inet proto udp to '
+                    '<dns_servers> port 53 keep state')
+
+            rules = '\n'.join(tables + translating_rules + filtering_rules) \
+                    + '\n'
+
+            pf_status = pfctl('-s all')[0]
+            if '\nrdr-anchor "sshuttle" all\n' not in pf_status:
+                pf_add_anchor_rule(PF_RDR, "sshuttle")
+            if '\nanchor "sshuttle" all\n' not in pf_status:
+                pf_add_anchor_rule(PF_PASS, "sshuttle")
+
+            pfctl('-a sshuttle -f /dev/stdin', rules)
+            if sys.platform == "darwin":
+                o = pfctl('-E')
+                _pf_context['Xtoken'] = \
+                    re.search(r'Token : (.+)', o[1]).group(1)
+            elif 'INFO:\nStatus: Disabled' in pf_status:
+                pfctl('-e')
+                _pf_context['started_by_sshuttle'] = True
+        else:
+            pfctl('-a sshuttle -F all')
+            if sys.platform == "darwin":
+                pfctl('-X %s' % _pf_context['Xtoken'])
+            elif _pf_context['started_by_sshuttle']:
+                pfctl('-d')
+
+    def firewall_command(self, line):
+        if line.startswith('QUERY_PF_NAT '):
+            try:
+                dst = pf_query_nat(*(line[13:].split(',')))
+                sys.stdout.write('QUERY_PF_NAT_SUCCESS %s,%r\n' % dst)
+            except IOError as e:
+                sys.stdout.write('QUERY_PF_NAT_FAILURE %s\n' % e)
+
+            sys.stdout.flush()
+            return True
+        else:
+            return False

sshuttle/methods/tproxy.py

@@ -0,0 +1,256 @@
+import struct
+from sshuttle.helpers import family_to_string
+from sshuttle.linux import ipt, ipt_ttl, ipt_chain_exists
+from sshuttle.methods import BaseMethod
+from sshuttle.helpers import debug1, debug3, Fatal
+
+recvmsg = None
+try:
+    # try getting recvmsg from python
+    import socket as pythonsocket
+    getattr(pythonsocket.socket, "recvmsg")
+    socket = pythonsocket
+    recvmsg = "python"
+except AttributeError:
+    # try getting recvmsg from socket_ext library
+    try:
+        import socket_ext
+        getattr(socket_ext.socket, "recvmsg")
+        socket = socket_ext
+        recvmsg = "socket_ext"
+    except ImportError:
+        import socket
+
+
+IP_TRANSPARENT = 19
+IP_ORIGDSTADDR = 20
+IP_RECVORIGDSTADDR = IP_ORIGDSTADDR
+SOL_IPV6 = 41
+IPV6_ORIGDSTADDR = 74
+IPV6_RECVORIGDSTADDR = IPV6_ORIGDSTADDR
+
+if recvmsg == "python":
+    def recv_udp(listener, bufsize):
+        debug3('Accept UDP python using recvmsg.\n')
+        data, ancdata, msg_flags, srcip = listener.recvmsg(
+            4096, socket.CMSG_SPACE(24))
+        dstip = None
+        family = None
+        for cmsg_level, cmsg_type, cmsg_data in ancdata:
+            if cmsg_level == socket.SOL_IP and cmsg_type == IP_ORIGDSTADDR:
+                family, port = struct.unpack('=HH', cmsg_data[0:4])
+                port = socket.htons(port)
+                if family == socket.AF_INET:
+                    start = 4
+                    length = 4
+                else:
+                    raise Fatal("Unsupported socket type '%s'" % family)
+                ip = socket.inet_ntop(family, cmsg_data[start:start + length])
+                dstip = (ip, port)
+                break
+            elif cmsg_level == SOL_IPV6 and cmsg_type == IPV6_ORIGDSTADDR:
+                family, port = struct.unpack('=HH', cmsg_data[0:4])
+                port = socket.htons(port)
+                if family == socket.AF_INET6:
+                    start = 8
+                    length = 16
+                else:
+                    raise Fatal("Unsupported socket type '%s'" % family)
+                ip = socket.inet_ntop(family, cmsg_data[start:start + length])
+                dstip = (ip, port)
+                break
+        return (srcip, dstip, data)
+elif recvmsg == "socket_ext":
+    def recv_udp(listener, bufsize):
+        debug3('Accept UDP using socket_ext recvmsg.\n')
+        srcip, data, adata, flags = listener.recvmsg(
+            (bufsize,), socket.CMSG_SPACE(24))
+        dstip = None
+        family = None
+        for a in adata:
+            if a.cmsg_level == socket.SOL_IP and a.cmsg_type == IP_ORIGDSTADDR:
+                family, port = struct.unpack('=HH', a.cmsg_data[0:4])
+                port = socket.htons(port)
+                if family == socket.AF_INET:
+                    start = 4
+                    length = 4
+                else:
+                    raise Fatal("Unsupported socket type '%s'" % family)
+                ip = socket.inet_ntop(
+                    family, a.cmsg_data[start:start + length])
+                dstip = (ip, port)
+                break
+            elif a.cmsg_level == SOL_IPV6 and a.cmsg_type == IPV6_ORIGDSTADDR:
+                family, port = struct.unpack('=HH', a.cmsg_data[0:4])
+                port = socket.htons(port)
+                if family == socket.AF_INET6:
+                    start = 8
+                    length = 16
+                else:
+                    raise Fatal("Unsupported socket type '%s'" % family)
+                ip = socket.inet_ntop(
+                    family, a.cmsg_data[start:start + length])
+                dstip = (ip, port)
+                break
+        return (srcip, dstip, data[0])
+else:
+    def recv_udp(listener, bufsize):
+        debug3('Accept UDP using recvfrom.\n')
+        data, srcip = listener.recvfrom(bufsize)
+        return (srcip, None, data)
+
+
+class Method(BaseMethod):
+
+    def get_supported_features(self):
+        result = super(Method, self).get_supported_features()
+        result.ipv6 = True
+        result.udp = True
+        return result
+
+    def get_tcp_dstip(self, sock):
+        return sock.getsockname()
+
+    def recv_udp(self, udp_listener, bufsize):
+        srcip, dstip, data = recv_udp(udp_listener, bufsize)
+        if not dstip:
+            debug1(
+                "-- ignored UDP from %r: "
+                "couldn't determine destination IP address\n" % (srcip,))
+            return None
+        return srcip, dstip, data
+
+    def send_udp(self, sock, srcip, dstip, data):
+        if not srcip:
+            debug1(
+                "-- ignored UDP to %r: "
+                "couldn't determine source IP address\n" % (dstip,))
+            return
+        sender = socket.socket(sock.family, socket.SOCK_DGRAM)
+        sender.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        sender.setsockopt(socket.SOL_IP, IP_TRANSPARENT, 1)
+        sender.bind(srcip)
+        sender.sendto(data, dstip)
+        sender.close()
+
+    def setup_tcp_listener(self, tcp_listener):
+        tcp_listener.setsockopt(socket.SOL_IP, IP_TRANSPARENT, 1)
+
+    def setup_udp_listener(self, udp_listener):
+        udp_listener.setsockopt(socket.SOL_IP, IP_TRANSPARENT, 1)
+        if udp_listener.v4 is not None:
+            udp_listener.v4.setsockopt(
+                socket.SOL_IP, IP_RECVORIGDSTADDR, 1)
+        if udp_listener.v6 is not None:
+            udp_listener.v6.setsockopt(SOL_IPV6, IPV6_RECVORIGDSTADDR, 1)
+
+    def setup_firewall(self, port, dnsport, nslist, family, subnets, udp):
+        if family not in [socket.AF_INET, socket.AF_INET6]:
+            raise Exception(
+                'Address family "%s" unsupported by tproxy method'
+                % family_to_string(family))
+
+        table = "mangle"
+
+        def _ipt(*args):
+            return ipt(family, table, *args)
+
+        def _ipt_ttl(*args):
+            return ipt_ttl(family, table, *args)
+
+        mark_chain = 'sshuttle-m-%s' % port
+        tproxy_chain = 'sshuttle-t-%s' % port
+        divert_chain = 'sshuttle-d-%s' % port
+
+        # basic cleanup/setup of chains
+        if ipt_chain_exists(family, table, mark_chain):
+            _ipt('-D', 'OUTPUT', '-j', mark_chain)
+            _ipt('-F', mark_chain)
+            _ipt('-X', mark_chain)
+
+        if ipt_chain_exists(family, table, tproxy_chain):
+            _ipt('-D', 'PREROUTING', '-j', tproxy_chain)
+            _ipt('-F', tproxy_chain)
+            _ipt('-X', tproxy_chain)
+
+        if ipt_chain_exists(family, table, divert_chain):
+            _ipt('-F', divert_chain)
+            _ipt('-X', divert_chain)
+
+        if subnets or dnsport:
+            _ipt('-N', mark_chain)
+            _ipt('-F', mark_chain)
+            _ipt('-N', divert_chain)
+            _ipt('-F', divert_chain)
+            _ipt('-N', tproxy_chain)
+            _ipt('-F', tproxy_chain)
+            _ipt('-I', 'OUTPUT', '1', '-j', mark_chain)
+            _ipt('-I', 'PREROUTING', '1', '-j', tproxy_chain)
+            _ipt('-A', divert_chain, '-j', 'MARK', '--set-mark', '1')
+            _ipt('-A', divert_chain, '-j', 'ACCEPT')
+            _ipt('-A', tproxy_chain, '-m', 'socket', '-j', divert_chain,
+                 '-m', 'tcp', '-p', 'tcp')
+        if subnets and udp:
+            _ipt('-A', tproxy_chain, '-m', 'socket', '-j', divert_chain,
+                 '-m', 'udp', '-p', 'udp')
+
+        if dnsport:
+            for f, ip in [i for i in nslist if i[0] == family]:
+                _ipt('-A', mark_chain, '-j', 'MARK', '--set-mark', '1',
+                     '--dest', '%s/32' % ip,
+                     '-m', 'udp', '-p', 'udp', '--dport', '53')
+                _ipt('-A', tproxy_chain, '-j', 'TPROXY',
+                     '--tproxy-mark', '0x1/0x1',
+                     '--dest', '%s/32' % ip,
+                     '-m', 'udp', '-p', 'udp', '--dport', '53',
+                     '--on-port', str(dnsport))
+
+        if subnets:
+            for f, swidth, sexclude, snet \
+                    in sorted(subnets, key=lambda s: s[1], reverse=True):
+                if sexclude:
+                    _ipt('-A', mark_chain, '-j', 'RETURN',
+                         '--dest', '%s/%s' % (snet, swidth),
+                         '-m', 'tcp', '-p', 'tcp')
+                    _ipt('-A', tproxy_chain, '-j', 'RETURN',
+                         '--dest', '%s/%s' % (snet, swidth),
+                         '-m', 'tcp', '-p', 'tcp')
+                else:
+                    _ipt('-A', mark_chain, '-j', 'MARK', '--set-mark', '1',
+                         '--dest', '%s/%s' % (snet, swidth),
+                         '-m', 'tcp', '-p', 'tcp')
+                    _ipt('-A', tproxy_chain, '-j', 'TPROXY',
+                         '--tproxy-mark', '0x1/0x1',
+                         '--dest', '%s/%s' % (snet, swidth),
+                         '-m', 'tcp', '-p', 'tcp',
+                         '--on-port', str(port))
+
+                if sexclude and udp:
+                    _ipt('-A', mark_chain, '-j', 'RETURN',
+                         '--dest', '%s/%s' % (snet, swidth),
+                         '-m', 'udp', '-p', 'udp')
+                    _ipt('-A', tproxy_chain, '-j', 'RETURN',
+                         '--dest', '%s/%s' % (snet, swidth),
+                         '-m', 'udp', '-p', 'udp')
+                elif udp:
+                    _ipt('-A', mark_chain, '-j', 'MARK', '--set-mark', '1',
+                         '--dest', '%s/%s' % (snet, swidth),
+                         '-m', 'udp', '-p', 'udp')
+                    _ipt('-A', tproxy_chain, '-j', 'TPROXY',
+                         '--tproxy-mark', '0x1/0x1',
+                         '--dest', '%s/%s' % (snet, swidth),
+                         '-m', 'udp', '-p', 'udp',
+                         '--on-port', str(port))
+
+    def check_settings(self, udp, dns):
+        if udp and recvmsg is None:
+            Fatal("tproxy UDP support requires recvmsg function.\n")
+
+        if dns and recvmsg is None:
+            Fatal("tproxy DNS support requires recvmsg function.\n")
+
+        if udp:
+            debug1("tproxy UDP support enabled.\n")
+
+        if dns:
+            debug1("tproxy DNS support enabled.\n")

sshuttle/options.py

@@ -0,0 +1,215 @@
+"""Command-line options parser.
+With the help of an options spec string, easily parse command-line options.
+"""
+import sys
+import os
+import textwrap
+import getopt
+import re
+import struct
+
+
+class OptDict:
+
+    def __init__(self):
+        self._opts = {}
+
+    def __setitem__(self, k, v):
+        if k.startswith('no-') or k.startswith('no_'):
+            k = k[3:]
+            v = not v
+        self._opts[k] = v
+
+    def __getitem__(self, k):
+        if k.startswith('no-') or k.startswith('no_'):
+            return not self._opts[k[3:]]
+        return self._opts[k]
+
+    def __getattr__(self, k):
+        return self[k]
+
+
+def _default_onabort(msg):
+    sys.exit(97)
+
+
+def _intify(v):
+    try:
+        vv = int(v or '')
+        if str(vv) == v:
+            return vv
+    except ValueError:
+        pass
+    return v
+
+
+def _atoi(v):
+    try:
+        return int(v or 0)
+    except ValueError:
+        return 0
+
+
+def _remove_negative_kv(k, v):
+    if k.startswith('no-') or k.startswith('no_'):
+        return k[3:], not v
+    return k, v
+
+
+def _remove_negative_k(k):
+    return _remove_negative_kv(k, None)[0]
+
+
+def _tty_width():
+    if not hasattr(sys.stderr, "fileno"):
+        return _atoi(os.environ.get('WIDTH')) or 70
+    s = struct.pack("HHHH", 0, 0, 0, 0)
+    try:
+        import fcntl
+        import termios
+        s = fcntl.ioctl(sys.stderr.fileno(), termios.TIOCGWINSZ, s)
+    except (IOError, ImportError):
+        return _atoi(os.environ.get('WIDTH')) or 70
+    (ysize, xsize, ypix, xpix) = struct.unpack('HHHH', s)
+    return xsize or 70
+
+
+class Options:
+
+    """Option parser.
+    When constructed, two strings are mandatory. The first one is the command
+    name showed before error messages. The second one is a string called an
+    optspec that specifies the synopsis and option flags and their description.
+    For more information about optspecs, consult the bup-options(1) man page.
+
+    Two optional arguments specify an alternative parsing function and an
+    alternative behaviour on abort (after having output the usage string).
+
+    By default, the parser function is getopt.gnu_getopt, and the abort
+    behaviour is to exit the program.
+    """
+
+    def __init__(self, optspec, optfunc=getopt.gnu_getopt,
+                 onabort=_default_onabort):
+        self.optspec = optspec
+        self._onabort = onabort
+        self.optfunc = optfunc
+        self._aliases = {}
+        self._shortopts = 'h?'
+        self._longopts = ['help']
+        self._hasparms = {}
+        self._defaults = {}
+        self._usagestr = self._gen_usage()
+
+    def _gen_usage(self):
+        out = []
+        lines = self.optspec.strip().split('\n')
+        lines.reverse()
+        first_syn = True
+        while lines:
+            l = lines.pop()
+            if l == '--':
+                break
+            out.append('%s: %s\n' % (first_syn and 'usage' or '   or', l))
+            first_syn = False
+        out.append('\n')
+        last_was_option = False
+        while lines:
+            l = lines.pop()
+            if l.startswith(' '):
+                out.append('%s%s\n' % (last_was_option and '\n' or '',
+                                       l.lstrip()))
+                last_was_option = False
+            elif l:
+                (flags, extra) = l.split(' ', 1)
+                extra = extra.strip()
+                if flags.endswith('='):
+                    flags = flags[:-1]
+                    has_parm = 1
+                else:
+                    has_parm = 0
+                g = re.search(r'\[([^\]]*)\]$', extra)
+                if g:
+                    defval = g.group(1)
+                else:
+                    defval = None
+                flagl = flags.split(',')
+                flagl_nice = []
+                for _f in flagl:
+                    f, dvi = _remove_negative_kv(_f, _intify(defval))
+                    self._aliases[f] = _remove_negative_k(flagl[0])
+                    self._hasparms[f] = has_parm
+                    self._defaults[f] = dvi
+                    if len(f) == 1:
+                        self._shortopts += f + (has_parm and ':' or '')
+                        flagl_nice.append('-' + f)
+                    else:
+                        f_nice = re.sub(r'\W', '_', f)
+                        self._aliases[f_nice] = _remove_negative_k(flagl[0])
+                        self._longopts.append(f + (has_parm and '=' or ''))
+                        self._longopts.append('no-' + f)
+                        flagl_nice.append('--' + _f)
+                flags_nice = ', '.join(flagl_nice)
+                if has_parm:
+                    flags_nice += ' ...'
+                prefix = '    %-20s  ' % flags_nice
+                argtext = '\n'.join(textwrap.wrap(extra, width=_tty_width(),
+                                                  initial_indent=prefix,
+                                                  subsequent_indent=' ' * 28))
+                out.append(argtext + '\n')
+                last_was_option = True
+            else:
+                out.append('\n')
+                last_was_option = False
+        return ''.join(out).rstrip() + '\n'
+
+    def usage(self, msg=""):
+        """Print usage string to stderr and abort."""
+        sys.stderr.write(self._usagestr)
+        e = self._onabort and self._onabort(msg) or None
+        if e:
+            raise e
+
+    def fatal(self, s):
+        """Print an error message to stderr and abort with usage string."""
+        msg = 'error: %s\n' % s
+        sys.stderr.write(msg)
+        return self.usage(msg)
+
+    def parse(self, args):
+        """Parse a list of arguments and return (options, flags, extra).
+
+        In the returned tuple, "options" is an OptDict with known options,
+        "flags" is a list of option flags that were used on the command-line,
+        and "extra" is a list of positional arguments.
+        """
+        try:
+            (flags, extra) = self.optfunc(
+                args, self._shortopts, self._longopts)
+        except getopt.GetoptError as e:
+            self.fatal(e)
+
+        opt = OptDict()
+
+        for k, v in self._defaults.items():
+            k = self._aliases[k]
+            opt[k] = v
+
+        for (k, v) in flags:
+            k = k.lstrip('-')
+            if k in ('h', '?', 'help'):
+                self.usage()
+            if k.startswith('no-'):
+                k = self._aliases[k[3:]]
+                v = 0
+            else:
+                k = self._aliases[k]
+                if not self._hasparms[k]:
+                    assert(v == '')
+                    v = (opt._opts.get(k) or 0) + 1
+                else:
+                    v = _intify(v)
+            opt[k] = v
+        for (f1, f2) in self._aliases.items():
+            opt[f1] = opt._opts.get(f2)
+        return (opt, flags, extra)

sshuttle/server.py

@@ -0,0 +1,337 @@
+import re
+import struct
+import socket
+import traceback
+import time
+import sys
+import os
+
+import sshuttle.ssnet as ssnet
+import sshuttle.helpers as helpers
+import sshuttle.hostwatch as hostwatch
+import subprocess as ssubprocess
+from sshuttle.ssnet import Handler, Proxy, Mux, MuxWrapper
+from sshuttle.helpers import log, debug1, debug2, debug3, Fatal, \
+    resolvconf_random_nameserver
+
+
+def _ipmatch(ipstr):
+    if ipstr == 'default':
+        ipstr = '0.0.0.0/0'
+    m = re.match(r'^(\d+(\.\d+(\.\d+(\.\d+)?)?)?)(?:/(\d+))?$', ipstr)
+    if m:
+        g = m.groups()
+        ips = g[0]
+        width = int(g[4] or 32)
+        if g[1] is None:
+            ips += '.0.0.0'
+            width = min(width, 8)
+        elif g[2] is None:
+            ips += '.0.0'
+            width = min(width, 16)
+        elif g[3] is None:
+            ips += '.0'
+            width = min(width, 24)
+        return (struct.unpack('!I', socket.inet_aton(ips))[0], width)
+
+
+def _ipstr(ip, width):
+    if width >= 32:
+        return ip
+    else:
+        return "%s/%d" % (ip, width)
+
+
+def _maskbits(netmask):
+    if not netmask:
+        return 32
+    for i in range(32):
+        if netmask[0] & _shl(1, i):
+            return 32 - i
+    return 0
+
+
+def _shl(n, bits):
+    return n * int(2 ** bits)
+
+
+def _list_routes():
+    argv = ['netstat', '-rn']
+    p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE)
+    routes = []
+    for line in p.stdout:
+        cols = re.split(r'\s+', line)
+        ipw = _ipmatch(cols[0])
+        if not ipw:
+            continue  # some lines won't be parseable; never mind
+        maskw = _ipmatch(cols[2])  # linux only
+        mask = _maskbits(maskw)   # returns 32 if maskw is null
+        width = min(ipw[1], mask)
+        ip = ipw[0] & _shl(_shl(1, width) - 1, 32 - width)
+        routes.append(
+            (socket.AF_INET, socket.inet_ntoa(struct.pack('!I', ip)), width))
+    rv = p.wait()
+    if rv != 0:
+        log('WARNING: %r returned %d\n' % (argv, rv))
+        log('WARNING: That prevents --auto-nets from working.\n')
+    return routes
+
+
+def list_routes():
+    for (family, ip, width) in _list_routes():
+        if not ip.startswith('0.') and not ip.startswith('127.'):
+            yield (family, ip, width)
+
+
+def _exc_dump():
+    exc_info = sys.exc_info()
+    return ''.join(traceback.format_exception(*exc_info))
+
+
+def start_hostwatch(seed_hosts):
+    s1, s2 = socket.socketpair()
+    pid = os.fork()
+    if not pid:
+        # child
+        rv = 99
+        try:
+            try:
+                s2.close()
+                os.dup2(s1.fileno(), 1)
+                os.dup2(s1.fileno(), 0)
+                s1.close()
+                rv = hostwatch.hw_main(seed_hosts) or 0
+            except Exception:
+                log('%s\n' % _exc_dump())
+                rv = 98
+        finally:
+            os._exit(rv)
+    s1.close()
+    return pid, s2
+
+
+class Hostwatch:
+
+    def __init__(self):
+        self.pid = 0
+        self.sock = None
+
+
+class DnsProxy(Handler):
+
+    def __init__(self, mux, chan, request):
+        Handler.__init__(self, [])
+        self.timeout = time.time() + 30
+        self.mux = mux
+        self.chan = chan
+        self.tries = 0
+        self.request = request
+        self.peers = {}
+        self.try_send()
+
+    def try_send(self):
+        if self.tries >= 3:
+            return
+        self.tries += 1
+
+        family, peer = resolvconf_random_nameserver()
+
+        sock = socket.socket(family, socket.SOCK_DGRAM)
+        sock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
+        sock.connect((peer, 53))
+
+        self.peers[sock] = peer
+
+        debug2('DNS: sending to %r (try %d)\n' % (peer, self.tries))
+        try:
+            sock.send(self.request)
+            self.socks.append(sock)
+        except socket.error as e:
+            if e.args[0] in ssnet.NET_ERRS:
+                # might have been spurious; try again.
+                # Note: these errors sometimes are reported by recv(),
+                # and sometimes by send().  We have to catch both.
+                debug2('DNS send to %r: %s\n' % (peer, e))
+                self.try_send()
+                return
+            else:
+                log('DNS send to %r: %s\n' % (peer, e))
+                return
+
+    def callback(self, sock):
+        peer = self.peers[sock]
+
+        try:
+            data = sock.recv(4096)
+        except socket.error as e:
+            self.socks.remove(sock)
+            del self.peers[sock]
+
+            if e.args[0] in ssnet.NET_ERRS:
+                # might have been spurious; try again.
+                # Note: these errors sometimes are reported by recv(),
+                # and sometimes by send().  We have to catch both.
+                debug2('DNS recv from %r: %s\n' % (peer, e))
+                self.try_send()
+                return
+            else:
+                log('DNS recv from %r: %s\n' % (peer, e))
+                return
+        debug2('DNS response: %d bytes\n' % len(data))
+        self.mux.send(self.chan, ssnet.CMD_DNS_RESPONSE, data)
+        self.ok = False
+
+
+class UdpProxy(Handler):
+
+    def __init__(self, mux, chan, family):
+        sock = socket.socket(family, socket.SOCK_DGRAM)
+        Handler.__init__(self, [sock])
+        self.timeout = time.time() + 30
+        self.mux = mux
+        self.chan = chan
+        self.sock = sock
+        if family == socket.AF_INET:
+            self.sock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
+
+    def send(self, dstip, data):
+        debug2('UDP: sending to %r port %d\n' % dstip)
+        try:
+            self.sock.sendto(data, dstip)
+        except socket.error as e:
+            log('UDP send to %r port %d: %s\n' % (dstip[0], dstip[1], e))
+            return
+
+    def callback(self, sock):
+        try:
+            data, peer = sock.recvfrom(4096)
+        except socket.error as e:
+            log('UDP recv from %r port %d: %s\n' % (peer[0], peer[1], e))
+            return
+        debug2('UDP response: %d bytes\n' % len(data))
+        hdr = "%s,%r," % (peer[0], peer[1])
+        self.mux.send(self.chan, ssnet.CMD_UDP_DATA, hdr + data)
+
+
+def main(latency_control):
+    if helpers.verbose >= 1:
+        helpers.logprefix = ' s: '
+    else:
+        helpers.logprefix = 'server: '
+    debug1('latency control setting = %r\n' % latency_control)
+
+    routes = list(list_routes())
+    debug1('available routes:\n')
+    for r in routes:
+        debug1('  %d/%s/%d\n' % r)
+
+    # synchronization header
+    sys.stdout.write('\0\0SSHUTTLE0001')
+    sys.stdout.flush()
+
+    handlers = []
+    mux = Mux(socket.fromfd(sys.stdin.fileno(),
+                            socket.AF_INET, socket.SOCK_STREAM),
+              socket.fromfd(sys.stdout.fileno(),
+                            socket.AF_INET, socket.SOCK_STREAM))
+    handlers.append(mux)
+    routepkt = ''
+    for r in routes:
+        routepkt += '%d,%s,%d\n' % r
+    mux.send(0, ssnet.CMD_ROUTES, routepkt)
+
+    hw = Hostwatch()
+    hw.leftover = ''
+
+    def hostwatch_ready(sock):
+        assert(hw.pid)
+        content = hw.sock.recv(4096)
+        if content:
+            lines = (hw.leftover + content).split('\n')
+            if lines[-1]:
+                # no terminating newline: entry isn't complete yet!
+                hw.leftover = lines.pop()
+                lines.append('')
+            else:
+                hw.leftover = ''
+            mux.send(0, ssnet.CMD_HOST_LIST, '\n'.join(lines))
+        else:
+            raise Fatal('hostwatch process died')
+
+    def got_host_req(data):
+        if not hw.pid:
+            (hw.pid, hw.sock) = start_hostwatch(data.strip().split())
+            handlers.append(Handler(socks=[hw.sock],
+                                    callback=hostwatch_ready))
+    mux.got_host_req = got_host_req
+
+    def new_channel(channel, data):
+        (family, dstip, dstport) = data.split(',', 2)
+        family = int(family)
+        dstport = int(dstport)
+        outwrap = ssnet.connect_dst(family, dstip, dstport)
+        handlers.append(Proxy(MuxWrapper(mux, channel), outwrap))
+    mux.new_channel = new_channel
+
+    dnshandlers = {}
+
+    def dns_req(channel, data):
+        debug2('Incoming DNS request channel=%d.\n' % channel)
+        h = DnsProxy(mux, channel, data)
+        handlers.append(h)
+        dnshandlers[channel] = h
+    mux.got_dns_req = dns_req
+
+    udphandlers = {}
+
+    def udp_req(channel, cmd, data):
+        debug2('Incoming UDP request channel=%d, cmd=%d\n' % (channel, cmd))
+        if cmd == ssnet.CMD_UDP_DATA:
+            (dstip, dstport, data) = data.split(",", 2)
+            dstport = int(dstport)
+            debug2('is incoming UDP data. %r %d.\n' % (dstip, dstport))
+            h = udphandlers[channel]
+            h.send((dstip, dstport), data)
+        elif cmd == ssnet.CMD_UDP_CLOSE:
+            debug2('is incoming UDP close\n')
+            h = udphandlers[channel]
+            h.ok = False
+            del mux.channels[channel]
+
+    def udp_open(channel, data):
+        debug2('Incoming UDP open.\n')
+        family = int(data)
+        mux.channels[channel] = lambda cmd, data: udp_req(channel, cmd, data)
+        if channel in udphandlers:
+            raise Fatal('UDP connection channel %d already open' % channel)
+        else:
+            h = UdpProxy(mux, channel, family)
+            handlers.append(h)
+            udphandlers[channel] = h
+    mux.got_udp_open = udp_open
+
+    while mux.ok:
+        if hw.pid:
+            assert(hw.pid > 0)
+            (rpid, rv) = os.waitpid(hw.pid, os.WNOHANG)
+            if rpid:
+                raise Fatal(
+                    'hostwatch exited unexpectedly: code 0x%04x\n' % rv)
+
+        ssnet.runonce(handlers, mux)
+        if latency_control:
+            mux.check_fullness()
+
+        if dnshandlers:
+            now = time.time()
+            for channel, h in list(dnshandlers.items()):
+                if h.timeout < now or not h.ok:
+                    debug3('expiring dnsreqs channel=%d\n' % channel)
+                    del dnshandlers[channel]
+                    h.ok = False
+        if udphandlers:
+            for channel, h in list(udphandlers.items()):
+                if not h.ok:
+                    debug3('expiring UDP channel=%d\n' % channel)
+                    del udphandlers[channel]
+                    h.ok = False

sshuttle/ssh.py

@@ -0,0 +1,129 @@
+import sys
+import os
+import re
+import socket
+import zlib
+import imp
+import subprocess as ssubprocess
+import sshuttle.helpers as helpers
+from sshuttle.helpers import debug2
+
+
+def readfile(name):
+    tokens = name.split(".")
+    f = None
+
+    token = tokens[0]
+    token_name = [token]
+    token_str = ".".join(token_name)
+
+    try:
+        f, pathname, description = imp.find_module(token_str)
+
+        for token in tokens[1:]:
+            module = imp.load_module(token_str, f, pathname, description)
+            if f is not None:
+                f.close()
+
+            token_name.append(token)
+            token_str = ".".join(token_name)
+
+            f, pathname, description = imp.find_module(
+                token, module.__path__)
+
+        if f is not None:
+            contents = f.read()
+        else:
+            contents = ""
+
+    finally:
+        if f is not None:
+            f.close()
+
+    return contents.encode("UTF8")
+
+
+def empackage(z, name, data=None):
+    if not data:
+        data = readfile(name)
+    content = z.compress(data)
+    content += z.flush(zlib.Z_SYNC_FLUSH)
+
+    return b'%s\n%d\n%s' % (name.encode("ASCII"), len(content), content)
+
+
+def connect(ssh_cmd, rhostport, python, stderr, options):
+    portl = []
+
+    if (rhostport or '').count(':') > 1:
+        if rhostport.count(']') or rhostport.count('['):
+            result = rhostport.split(']')
+            rhost = result[0].strip('[')
+            if len(result) > 1:
+                result[1] = result[1].strip(':')
+                if result[1] is not '':
+                    portl = ['-p', str(int(result[1]))]
+        # can't disambiguate IPv6 colons and a port number. pass the hostname
+        # through.
+        else:
+            rhost = rhostport
+    else:  # IPv4
+        l = (rhostport or '').split(':', 1)
+        rhost = l[0]
+        if len(l) > 1:
+            portl = ['-p', str(int(l[1]))]
+
+    if rhost == '-':
+        rhost = None
+
+    z = zlib.compressobj(1)
+    content = readfile('sshuttle.assembler')
+    optdata = ''.join("%s=%r\n" % (k, v) for (k, v) in list(options.items()))
+    optdata = optdata.encode("UTF8")
+    content2 = (empackage(z, 'sshuttle') +
+                empackage(z, 'sshuttle.cmdline_options', optdata) +
+                empackage(z, 'sshuttle.helpers') +
+                empackage(z, 'sshuttle.ssnet') +
+                empackage(z, 'sshuttle.hostwatch') +
+                empackage(z, 'sshuttle.server') +
+                b"\n")
+
+    pyscript = r"""
+                import sys;
+                verbosity=%d;
+                exec compile(sys.stdin.read(%d), "assembler.py", "exec")
+                """ % (helpers.verbose or 0, len(content))
+    pyscript = re.sub(r'\s+', ' ', pyscript.strip())
+
+    if not rhost:
+        # ignore the --python argument when running locally; we already know
+        # which python version works.
+        argv = [sys.argv[1], '-c', pyscript]
+    else:
+        if ssh_cmd:
+            sshl = ssh_cmd.split(' ')
+        else:
+            sshl = ['ssh']
+        if python:
+            pycmd = "'%s' -c '%s'" % (python, pyscript)
+        else:
+            pycmd = ("P=python2; $P -V 2>/dev/null || P=python; "
+                     "exec \"$P\" -c '%s'") % pyscript
+        argv = (sshl +
+                portl +
+                [rhost, '--', pycmd])
+    (s1, s2) = socket.socketpair()
+
+    def setup():
+        # runs in the child process
+        s2.close()
+    s1a, s1b = os.dup(s1.fileno()), os.dup(s1.fileno())
+    s1.close()
+    debug2('executing: %r\n' % argv)
+    p = ssubprocess.Popen(argv, stdin=s1a, stdout=s1b, preexec_fn=setup,
+                          close_fds=True, stderr=stderr)
+    os.close(s1a)
+    os.close(s1b)
+    s2.sendall(content)
+    s2.sendall(content2)
+    return p, s2

sshuttle/sshuttle.md

@@ -0,0 +1,279 @@
+% sshuttle(8) Sshuttle 0.46
+% Avery Pennarun <apenwarr@gmail.com>
+% 2011-01-25
+
+# NAME
+
+sshuttle - a transparent proxy-based VPN using ssh
+
+# SYNOPSIS
+
+sshuttle [options...] [-r [username@]sshserver[:port]] \<subnets...\>
+
+
+# DESCRIPTION
+
+sshuttle allows you to create a VPN connection from your
+machine to any remote server that you can connect to via
+ssh, as long as that server has python 2.3 or higher.
+
+To work, you must have root access on the local machine,
+but you can have a normal account on the server.
+
+It's valid to run sshuttle more than once simultaneously on
+a single client machine, connecting to a different server
+every time, so you can be on more than one VPN at once.
+
+If run on a router, sshuttle can forward traffic for your
+entire subnet to the VPN.
+
+
+# OPTIONS
+
+\<subnets...\>
+:   a list of subnets to route over the VPN, in the form
+    `a.b.c.d[/width]`.  Valid examples are 1.2.3.4 (a
+    single IP address), 1.2.3.4/32 (equivalent to 1.2.3.4),
+    1.2.3.0/24 (a 24-bit subnet, ie. with a 255.255.255.0
+    netmask), and 0/0 ('just route everything through the
+    VPN').
+
+-l, --listen=*[ip:]port*
+:   use this ip address and port number as the transparent
+    proxy port.  By default sshuttle finds an available
+    port automatically and listens on IP 127.0.0.1
+    (localhost), so you don't need to override it, and
+    connections are only proxied from the local machine,
+    not from outside machines.  If you want to accept
+    connections from other machines on your network (ie. to
+    run sshuttle on a router) try enabling IP Forwarding in
+    your kernel, then using `--listen 0.0.0.0:0`.
+
+-H, --auto-hosts
+:   scan for remote hostnames and update the local /etc/hosts
+    file with matching entries for as long as the VPN is
+    open.  This is nicer than changing your system's DNS
+    (/etc/resolv.conf) settings, for several reasons.  First,
+    hostnames are added without domain names attached, so
+    you can `ssh thatserver` without worrying if your local
+    domain matches the remote one.  Second, if you sshuttle
+    into more than one VPN at a time, it's impossible to
+    use more than one DNS server at once anyway, but
+    sshuttle correctly merges /etc/hosts entries between
+    all running copies.  Third, if you're only routing a
+    few subnets over the VPN, you probably would prefer to
+    keep using your local DNS server for everything else.
+
+-N, --auto-nets
+:   in addition to the subnets provided on the command
+    line, ask the server which subnets it thinks we should
+    route, and route those automatically.  The suggestions
+    are taken automatically from the server's routing
+    table.
+
+--dns
+:   capture local DNS requests and forward to the remote DNS
+    server.
+
+--python
+:   specify the name/path of the remote python interpreter.
+    The default is just `python`, which means to use the
+    default python interpreter on the remote system's PATH.
+
+-r, --remote=*[username@]sshserver[:port]*
+:   the remote hostname and optional username and ssh
+    port number to use for connecting to the remote server.
+    For example, example.com, testuser@example.com,
+    testuser@example.com:2222, or example.com:2244.
+
+-x, --exclude=*subnet*
+:   explicitly exclude this subnet from forwarding.  The
+    format of this option is the same as the `<subnets>`
+    option.  To exclude more than one subnet, specify the
+    `-x` option more than once.  You can say something like
+    `0/0 -x 1.2.3.0/24` to forward everything except the
+    local subnet over the VPN, for example.
+
+-X, --exclude-from=*file*
+:   exclude the subnets specified in a file, one subnet per
+    line. Useful when you have lots of subnets to exclude.
+
+-v, --verbose
+:   print more information about the session.  This option
+    can be used more than once for increased verbosity.  By
+    default, sshuttle prints only error messages.
+
+-e, --ssh-cmd
+:   the command to use to connect to the remote server. The
+    default is just `ssh`.  Use this if your ssh client is
+    in a non-standard location or you want to provide extra
+    options to the ssh command, for example, `-e 'ssh -v'`.
+
+--seed-hosts
+:   a comma-separated list of hostnames to use to
+    initialize the `--auto-hosts` scan algorithm.
+    `--auto-hosts` does things like poll local SMB servers
+    for lists of local hostnames, but can speed things up
+    if you use this option to give it a few names to start
+    from.
+
+--no-latency-control
+:   sacrifice latency to improve bandwidth benchmarks. ssh
+    uses really big socket buffers, which can overload the
+    connection if you start doing large file transfers,
+    thus making all your other sessions inside the same
+    tunnel go slowly. Normally, sshuttle tries to avoid
+    this problem using a "fullness check" that allows only
+    a certain amount of outstanding data to be buffered at
+    a time.  But on high-bandwidth links, this can leave a
+    lot of your bandwidth underutilized.  It also makes
+    sshuttle seem slow in bandwidth benchmarks (benchmarks
+    rarely test ping latency, which is what sshuttle is
+    trying to control).  This option disables the latency
+    control feature, maximizing bandwidth usage.  Use at
+    your own risk.
+
+-D, --daemon
+:   automatically fork into the background after connecting
+    to the remote server.  Implies `--syslog`.
+
+--syslog
+:   after connecting, send all log messages to the
+    `syslog`(3) service instead of stderr.  This is
+    implicit if you use `--daemon`.
+
+--pidfile=*pidfilename*
+:   when using `--daemon`, save sshuttle's pid to
+    *pidfilename*.  The default is `sshuttle.pid` in the
+    current directory.
+
+--firewall
+:   (internal use only) run the firewall manager.  This is
+    the only part of sshuttle that must run as root.  If
+    you start sshuttle as a non-root user, it will
+    automatically run `sudo` or `su` to start the firewall
+    manager, but the core of sshuttle still runs as a
+    normal user.
+
+--hostwatch
+:   (internal use only) run the hostwatch daemon.  This
+    process runs on the server side and collects hostnames for
+    the `--auto-hosts` option.  Using this option by itself
+    makes it a lot easier to debug and test the `--auto-hosts`
+    feature.
+
+
+# EXAMPLES
+
+Test locally by proxying all local connections, without using ssh:
+
+    $ sshuttle -v 0/0
+
+    Starting sshuttle proxy.
+    Listening on ('0.0.0.0', 12300).
+    [local sudo] Password:
+    firewall manager ready.
+    c : connecting to server...
+     s: available routes:
+     s:   192.168.42.0/24
+    c : connected.
+    firewall manager: starting transproxy.
+    c : Accept: 192.168.42.106:50035 -> 192.168.42.121:139.
+    c : Accept: 192.168.42.121:47523 -> 77.141.99.22:443.
+        ...etc...
+    ^C
+    firewall manager: undoing changes.
+    KeyboardInterrupt
+    c : Keyboard interrupt: exiting.
+    c : SW#8:192.168.42.121:47523: deleting
+    c : SW#6:192.168.42.106:50035: deleting
+
+Test connection to a remote server, with automatic hostname
+and subnet guessing:
+
+    $ sshuttle -vNHr example.org
+
+    Starting sshuttle proxy.
+    Listening on ('0.0.0.0', 12300).
+    firewall manager ready.
+    c : connecting to server...
+     s: available routes:
+     s:   77.141.99.0/24
+    c : connected.
+    c : seed_hosts: []
+    firewall manager: starting transproxy.
+    hostwatch: Found: testbox1: 1.2.3.4
+    hostwatch: Found: mytest2: 5.6.7.8
+    hostwatch: Found: domaincontroller: 99.1.2.3
+    c : Accept: 192.168.42.121:60554 -> 77.141.99.22:22.
+    ^C
+    firewall manager: undoing changes.
+    c : Keyboard interrupt: exiting.
+    c : SW#6:192.168.42.121:60554: deleting
+
+
+# DISCUSSION
+
+When it starts, sshuttle creates an ssh session to the
+server specified by the `-r` option.  If `-r` is omitted,
+it will start both its client and server locally, which is
+sometimes useful for testing.
+
+After connecting to the remote server, sshuttle uploads its
+(python) source code to the remote end and executes it
+there.  Thus, you don't need to install sshuttle on the
+remote server, and there are never sshuttle version
+conflicts between client and server.
+
+Unlike most VPNs, sshuttle forwards sessions, not packets.
+That is, it uses kernel transparent proxying (`iptables
+REDIRECT` rules on Linux, or `ipfw fwd` rules on BSD) to
+capture outgoing TCP sessions, then creates entirely
+separate TCP sessions out to the original destination at
+the other end of the tunnel.
+
+Packet-level forwarding (eg. using the tun/tap devices on
+Linux) seems elegant at first, but it results in
+several problems, notably the 'tcp over tcp' problem.  The
+tcp protocol depends fundamentally on packets being dropped
+in order to implement its congestion control agorithm; if
+you pass tcp packets through a tcp-based tunnel (such as
+ssh), the inner tcp packets will never be dropped, and so
+the inner tcp stream's congestion control will be
+completely broken, and performance will be terrible.  Thus,
+packet-based VPNs (such as IPsec and openvpn) cannot use
+tcp-based encrypted streams like ssh or ssl, and have to
+implement their own encryption from scratch, which is very
+complex and error prone.
+
+sshuttle's simplicity comes from the fact that it can
+safely use the existing ssh encrypted tunnel without
+incurring a performance penalty.  It does this by letting
+the client-side kernel manage the incoming tcp stream, and
+the server-side kernel manage the outgoing tcp stream;
+there is no need for congestion control to be shared
+between the two separate streams, so a tcp-based tunnel is
+fine.
+
+
+# BUGS
+
+On MacOS 10.6 (at least up to 10.6.6), your network will
+stop responding about 10 minutes after the first time you
+start sshuttle, because of a MacOS kernel bug relating to
+arp and the net.inet.ip.scopedroute sysctl.  To fix it,
+just switch your wireless off and on. Sshuttle makes the
+kernel setting it changes permanent, so this won't happen
+again, even after a reboot.
+
+On MacOS, sshuttle will set the kernel boot flag
+net.inet.ip.scopedroute to 0, which interferes with OS X
+Internet Sharing and some VPN clients. To reset this flag,
+you can remove any reference to net.inet.ip.scopedroute from
+/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
+and reboot.
+
+
+# SEE ALSO
+
+`ssh`(1), `python`(1)

sshuttle/ssnet.py

@@ -1,6 +1,17 @@
-import struct, socket, errno, select
+import struct
-if not globals().get('skip_imports'):
+import socket
-    from helpers import *
+import errno
+import select
+import os
+from sshuttle.helpers import log, debug1, debug2, debug3, Fatal
+
+MAX_CHANNEL = 65535
+
+# these don't exist in the socket module in python 2.3!
+SHUT_RD = 0
+SHUT_WR = 1
+SHUT_RDWR = 2
+
 
 HDR_LEN = 8
 
@@ -8,36 +19,67 @@ HDR_LEN = 8
 CMD_EXIT = 0x4200
 CMD_PING = 0x4201
 CMD_PONG = 0x4202
-CMD_CONNECT = 0x4203
+CMD_TCP_CONNECT = 0x4203
-CMD_CLOSE = 0x4204
+CMD_TCP_STOP_SENDING = 0x4204
-CMD_EOF = 0x4205
+CMD_TCP_EOF = 0x4205
-CMD_DATA = 0x4206
+CMD_TCP_DATA = 0x4206
 CMD_ROUTES = 0x4207
 CMD_HOST_REQ = 0x4208
 CMD_HOST_LIST = 0x4209
+CMD_DNS_REQ = 0x420a
+CMD_DNS_RESPONSE = 0x420b
+CMD_UDP_OPEN = 0x420c
+CMD_UDP_DATA = 0x420d
+CMD_UDP_CLOSE = 0x420e
 
 cmd_to_name = {
     CMD_EXIT: 'EXIT',
     CMD_PING: 'PING',
     CMD_PONG: 'PONG',
-    CMD_CONNECT: 'CONNECT',
+    CMD_TCP_CONNECT: 'TCP_CONNECT',
-    CMD_CLOSE: 'CLOSE',
+    CMD_TCP_STOP_SENDING: 'TCP_STOP_SENDING',
-    CMD_EOF: 'EOF',
+    CMD_TCP_EOF: 'TCP_EOF',
-    CMD_DATA: 'DATA',
+    CMD_TCP_DATA: 'TCP_DATA',
     CMD_ROUTES: 'ROUTES',
     CMD_HOST_REQ: 'HOST_REQ',
     CMD_HOST_LIST: 'HOST_LIST',
+    CMD_DNS_REQ: 'DNS_REQ',
+    CMD_DNS_RESPONSE: 'DNS_RESPONSE',
+    CMD_UDP_OPEN: 'UDP_OPEN',
+    CMD_UDP_DATA: 'UDP_DATA',
+    CMD_UDP_CLOSE: 'UDP_CLOSE',
 }
-    
+
+
+NET_ERRS = [errno.ECONNREFUSED, errno.ETIMEDOUT,
+            errno.EHOSTUNREACH, errno.ENETUNREACH,
+            errno.EHOSTDOWN, errno.ENETDOWN]
+
+
+def _add(l, elem):
+    if elem not in l:
+        l.append(elem)
+
+
+def _fds(l):
+    out = []
+    for i in l:
+        try:
+            out.append(i.fileno())
+        except AttributeError:
+            out.append(i)
+    out.sort()
+    return out
 
 
 def _nb_clean(func, *args):
     try:
         return func(*args)
-    except OSError, e:
+    except OSError as e:
         if e.errno not in (errno.EWOULDBLOCK, errno.EAGAIN):
             raise
         else:
+            debug3('%s: err was: %s\n' % (func.__name__, e))
             return None
 
 
@@ -46,14 +88,21 @@ def _try_peername(sock):
         pn = sock.getpeername()
         if pn:
             return '%s:%s' % (pn[0], pn[1])
-    except socket.error, e:
+    except socket.error as e:
         if e.args[0] not in (errno.ENOTCONN, errno.ENOTSOCK):
             raise
     return 'unknown'
 
 
+_swcount = 0
+
+
 class SockWrapper:
+
     def __init__(self, rsock, wsock, connect_to=None, peername=None):
+        global _swcount
+        _swcount += 1
+        debug3('creating new SockWrapper (%d now exist)\n' % _swcount)
         self.exc = None
         self.rsock = rsock
         self.wsock = wsock
@@ -64,34 +113,66 @@ class SockWrapper:
         self.try_connect()
 
     def __del__(self):
-        debug1('%r: deleting\n' % self)
+        global _swcount
+        _swcount -= 1
+        debug1('%r: deleting (%d remain)\n' % (self, _swcount))
         if self.exc:
-            debug1('%r: error was: %r\n' % (self, self.exc))
+            debug1('%r: error was: %s\n' % (self, self.exc))
 
     def __repr__(self):
-        return 'SW:%s' % (self.peername,)
+        if self.rsock == self.wsock:
+            fds = '#%d' % self.rsock.fileno()
+        else:
+            fds = '#%d,%d' % (self.rsock.fileno(), self.wsock.fileno())
+        return 'SW%s:%s' % (fds, self.peername)
 
     def seterr(self, e):
         if not self.exc:
             self.exc = e
+        self.nowrite()
+        self.noread()
 
     def try_connect(self):
+        if self.connect_to and self.shut_write:
+            self.noread()
+            self.connect_to = None
         if not self.connect_to:
             return  # already connected
         self.rsock.setblocking(False)
+        debug3('%r: trying connect to %r\n' % (self, self.connect_to))
         try:
             self.rsock.connect(self.connect_to)
             # connected successfully (Linux)
             self.connect_to = None
-        except socket.error, e:
+        except socket.error as e:
+            debug3('%r: connect result: %s\n' % (self, e))
+            if e.args[0] == errno.EINVAL:
+                # this is what happens when you call connect() on a socket
+                # that is now connected but returned EINPROGRESS last time,
+                # on BSD, on python pre-2.5.1.  We need to use getsockopt()
+                # to get the "real" error.  Later pythons do this
+                # automatically, so this code won't run.
+                realerr = self.rsock.getsockopt(socket.SOL_SOCKET,
+                                                socket.SO_ERROR)
+                e = socket.error(realerr, os.strerror(realerr))
+                debug3('%r: fixed connect result: %s\n' % (self, e))
             if e.args[0] in [errno.EINPROGRESS, errno.EALREADY]:
                 pass  # not connected yet
+            elif e.args[0] == 0:
+                # connected successfully (weird Linux bug?)
+                # Sometimes Linux seems to return EINVAL when it isn't
+                # invalid.  This *may* be caused by a race condition
+                # between connect() and getsockopt(SO_ERROR) (ie. it
+                # finishes connecting in between the two, so there is no
+                # longer an error).  However, I'm not sure of that.
+                #
+                # I did get at least one report that the problem went away
+                # when we added this, however.
+                self.connect_to = None
             elif e.args[0] == errno.EISCONN:
                 # connected successfully (BSD)
                 self.connect_to = None
-            elif e.args[0] in [errno.ECONNREFUSED, errno.ETIMEDOUT,
+            elif e.args[0] in NET_ERRS + [errno.EACCES, errno.EPERM]:
-                               errno.EHOSTUNREACH, errno.ENETUNREACH,
-                               errno.EACCES, errno.EPERM]:
                 # a "normal" kind of error
                 self.connect_to = None
                 self.seterr(e)
@@ -102,16 +183,16 @@ class SockWrapper:
         if not self.shut_read:
             debug2('%r: done reading\n' % self)
             self.shut_read = True
-            #self.rsock.shutdown(socket.SHUT_RD)  # doesn't do anything anyway
+            # self.rsock.shutdown(SHUT_RD)  # doesn't do anything anyway
-        
+
     def nowrite(self):
         if not self.shut_write:
             debug2('%r: done writing\n' % self)
             self.shut_write = True
             try:
-                self.wsock.shutdown(socket.SHUT_WR)
+                self.wsock.shutdown(SHUT_WR)
-            except socket.error, e:
+            except socket.error as e:
-                self.seterr(e)
+                self.seterr('nowrite: %s' % e)
 
     def too_full(self):
         return False  # fullness is determined by the socket's select() state
@@ -122,13 +203,16 @@ class SockWrapper:
         self.wsock.setblocking(False)
         try:
             return _nb_clean(os.write, self.wsock.fileno(), buf)
-        except OSError, e:
+        except OSError as e:
-            # unexpected error... stream is dead
+            if e.errno == errno.EPIPE:
-            self.seterr(e)
+                debug1('%r: uwrite: got EPIPE\n' % self)
-            self.nowrite()
+                self.nowrite()
-            self.noread()
+                return 0
-            return 0
+            else:
-        
+                # unexpected error... stream is dead
+                self.seterr('uwrite: %s' % e)
+                return 0
+
     def write(self, buf):
         assert(buf)
         return self.uwrite(buf)
@@ -141,9 +225,9 @@ class SockWrapper:
         self.rsock.setblocking(False)
         try:
             return _nb_clean(os.read, self.rsock.fileno(), 65536)
-        except OSError, e:
+        except OSError as e:
-            self.seterr(e)
+            self.seterr('uread: %s' % e)
-            return '' # unexpected error... we'll call it EOF
+            return b''  # unexpected error... we'll call it EOF
 
     def fill(self):
         if self.buf:
@@ -151,7 +235,7 @@ class SockWrapper:
         rb = self.uread()
         if rb:
             self.buf.append(rb)
-        if rb == '':  # empty string means EOF; None means temporarily empty
+        if rb == b'':  # empty string means EOF; None means temporarily empty
             self.noread()
 
     def copy_to(self, outwrap):
@@ -159,33 +243,36 @@ class SockWrapper:
             wrote = outwrap.write(self.buf[0])
             self.buf[0] = self.buf[0][wrote:]
         while self.buf and not self.buf[0]:
-            self.buf[0:1] = []
+            self.buf.pop(0)
         if not self.buf and self.shut_read:
             outwrap.nowrite()
 
 
 class Handler:
-    def __init__(self, socks = None, callback = None):
+
+    def __init__(self, socks=None, callback=None):
         self.ok = True
-        self.socks = set(socks or [])
+        self.socks = socks or []
         if callback:
             self.callback = callback
 
     def pre_select(self, r, w, x):
-        r |= self.socks
+        for i in self.socks:
+            _add(r, i)
 
-    def callback(self):
+    def callback(self, sock):
         log('--no callback defined-- %r\n' % self)
-        (r,w,x) = select.select(self.socks, [], [], 0)
+        (r, w, x) = select.select(self.socks, [], [], 0)
         for s in r:
             v = s.recv(4096)
             if not v:
                 log('--closed-- %r\n' % self)
-                self.socks = set()
+                self.socks = []
                 self.ok = False
 
 
 class Proxy(Handler):
+
     def __init__(self, wrap1, wrap2):
         Handler.__init__(self, [wrap1.rsock, wrap1.wsock,
                                 wrap2.rsock, wrap2.wsock])
@@ -193,86 +280,104 @@ class Proxy(Handler):
         self.wrap2 = wrap2
 
     def pre_select(self, r, w, x):
+        if self.wrap1.shut_write:
+            self.wrap2.noread()
+        if self.wrap2.shut_write:
+            self.wrap1.noread()
+
         if self.wrap1.connect_to:
-            w.add(self.wrap1.rsock)
+            _add(w, self.wrap1.rsock)
         elif self.wrap1.buf:
             if not self.wrap2.too_full():
-                w.add(self.wrap2.wsock)
+                _add(w, self.wrap2.wsock)
         elif not self.wrap1.shut_read:
-            r.add(self.wrap1.rsock)
+            _add(r, self.wrap1.rsock)
 
         if self.wrap2.connect_to:
-            w.add(self.wrap2.rsock)
+            _add(w, self.wrap2.rsock)
         elif self.wrap2.buf:
             if not self.wrap1.too_full():
-                w.add(self.wrap1.wsock)
+                _add(w, self.wrap1.wsock)
         elif not self.wrap2.shut_read:
-            r.add(self.wrap2.rsock)
+            _add(r, self.wrap2.rsock)
 
-    def callback(self):
+    def callback(self, sock):
         self.wrap1.try_connect()
         self.wrap2.try_connect()
         self.wrap1.fill()
         self.wrap2.fill()
         self.wrap1.copy_to(self.wrap2)
         self.wrap2.copy_to(self.wrap1)
+        if self.wrap1.buf and self.wrap2.shut_write:
+            self.wrap1.buf = []
+            self.wrap1.noread()
+        if self.wrap2.buf and self.wrap1.shut_write:
+            self.wrap2.buf = []
+            self.wrap2.noread()
         if (self.wrap1.shut_read and self.wrap2.shut_read and
-            not self.wrap1.buf and not self.wrap2.buf):
+                not self.wrap1.buf and not self.wrap2.buf):
             self.ok = False
+            self.wrap1.nowrite()
+            self.wrap2.nowrite()
 
 
 class Mux(Handler):
+
     def __init__(self, rsock, wsock):
         Handler.__init__(self, [rsock, wsock])
         self.rsock = rsock
         self.wsock = wsock
-        self.new_channel = self.got_routes = None
+        self.new_channel = self.got_dns_req = self.got_routes = None
+        self.got_udp_open = self.got_udp_data = self.got_udp_close = None
         self.got_host_req = self.got_host_list = None
         self.channels = {}
         self.chani = 0
         self.want = 0
-        self.inbuf = ''
+        self.inbuf = b''
         self.outbuf = []
         self.fullness = 0
         self.too_full = False
-        self.send(0, CMD_PING, 'chicken')
+        self.send(0, CMD_PING, b'chicken')
 
     def next_channel(self):
         # channel 0 is special, so we never allocate it
-        for timeout in xrange(1024):
+        for timeout in range(1024):
             self.chani += 1
-            if self.chani > 65535:
+            if self.chani > MAX_CHANNEL:
                 self.chani = 1
             if not self.channels.get(self.chani):
                 return self.chani
 
     def amount_queued(self):
-        return sum(len(b) for b in self.outbuf)
+        total = 0
-            
+        for b in self.outbuf:
+            total += len(b)
+        return total
+
     def check_fullness(self):
         if self.fullness > 32768:
             if not self.too_full:
-                self.send(0, CMD_PING, 'rttest')
+                self.send(0, CMD_PING, b'rttest')
             self.too_full = True
-        #ob = []
+        # ob = []
-        #for b in self.outbuf:
+        # for b in self.outbuf:
         #    (s1,s2,c) = struct.unpack('!ccH', b[:4])
         #    ob.append(c)
-        #log('outbuf: %d %r\n' % (self.amount_queued(), ob))
+        # log('outbuf: %d %r\n' % (self.amount_queued(), ob))
-        
+
     def send(self, channel, cmd, data):
-        data = str(data)
+        assert isinstance(data, bytes)
-        assert(len(data) <= 65535)
+        assert len(data) <= 65535
-        p = struct.pack('!ccHHH', 'S', 'S', channel, cmd, len(data)) + data
+        p = struct.pack('!ccHHH', b'S', b'S', channel, cmd, len(data)) + data
         self.outbuf.append(p)
         debug2(' > channel=%d cmd=%s len=%d (fullness=%d)\n'
-               % (channel, cmd_to_name.get(cmd,hex(cmd)),
+               % (channel, cmd_to_name.get(cmd, hex(cmd)),
                   len(data), self.fullness))
         self.fullness += len(data)
 
     def got_packet(self, channel, cmd, data):
-        debug2('<  channel=%d cmd=%s len=%d\n' 
+        debug2('<  channel=%d cmd=%s len=%d\n'
-               % (channel, cmd_to_name.get(cmd,hex(cmd)), len(data)))
+               % (channel, cmd_to_name.get(cmd, hex(cmd)), len(data)))
         if cmd == CMD_PING:
             self.send(0, CMD_PONG, data)
         elif cmd == CMD_PONG:
@@ -281,10 +386,18 @@ class Mux(Handler):
             self.fullness = 0
         elif cmd == CMD_EXIT:
             self.ok = False
-        elif cmd == CMD_CONNECT:
+        elif cmd == CMD_TCP_CONNECT:
             assert(not self.channels.get(channel))
             if self.new_channel:
                 self.new_channel(channel, data)
+        elif cmd == CMD_DNS_REQ:
+            assert(not self.channels.get(channel))
+            if self.got_dns_req:
+                self.got_dns_req(channel, data)
+        elif cmd == CMD_UDP_OPEN:
+            assert(not self.channels.get(channel))
+            if self.got_udp_open:
+                self.got_udp_open(channel, data)
         elif cmd == CMD_ROUTES:
             if self.got_routes:
                 self.got_routes(data)
@@ -301,14 +414,18 @@ class Mux(Handler):
             else:
                 raise Exception('got CMD_HOST_LIST without got_host_list?')
         else:
-            callback = self.channels[channel]
+            callback = self.channels.get(channel)
-            callback(cmd, data)
+            if not callback:
+                log('warning: closed channel %d got cmd=%s len=%d\n'
+                    % (channel, cmd_to_name.get(cmd, hex(cmd)), len(data)))
+            else:
+                callback(cmd, data)
 
     def flush(self):
         self.wsock.setblocking(False)
         if self.outbuf and self.outbuf[0]:
             wrote = _nb_clean(os.write, self.wsock.fileno(), self.outbuf[0])
-            debug2('mux wrote: %d/%d\n' % (wrote, len(self.outbuf[0])))
+            debug2('mux wrote: %r/%d\n' % (wrote, len(self.outbuf[0])))
             if wrote:
                 self.outbuf[0] = self.outbuf[0][wrote:]
         while self.outbuf and not self.outbuf[0]:
@@ -318,24 +435,24 @@ class Mux(Handler):
         self.rsock.setblocking(False)
         try:
             b = _nb_clean(os.read, self.rsock.fileno(), 32768)
-        except OSError, e:
+        except OSError as e:
             raise Fatal('other end: %r' % e)
-        #log('<<< %r\n' % b)
+        # log('<<< %r\n' % b)
-        if b == '': # EOF
+        if b == b'':  # EOF
             self.ok = False
         if b:
             self.inbuf += b
 
     def handle(self):
         self.fill()
-        #log('inbuf is: (%d,%d) %r\n'
+        # log('inbuf is: (%d,%d) %r\n'
         #     % (self.want, len(self.inbuf), self.inbuf))
         while 1:
             if len(self.inbuf) >= (self.want or HDR_LEN):
-                (s1,s2,channel,cmd,datalen) = \
+                (s1, s2, channel, cmd, datalen) = \
                     struct.unpack('!ccHHH', self.inbuf[:HDR_LEN])
-                assert(s1 == 'S')
+                assert(s1 == b'S')
-                assert(s2 == 'S')
+                assert(s2 == b'S')
                 self.want = datalen + HDR_LEN
             if self.want and len(self.inbuf) >= self.want:
                 data = self.inbuf[HDR_LEN:self.want]
@@ -346,12 +463,12 @@ class Mux(Handler):
                 break
 
     def pre_select(self, r, w, x):
-        r.add(self.rsock)
+        _add(r, self.rsock)
         if self.outbuf:
-            w.add(self.wsock)
+            _add(w, self.wsock)
 
-    def callback(self):
+    def callback(self, sock):
-        (r,w,x) = select.select([self.rsock], [self.wsock], [], 0)
+        (r, w, x) = select.select([self.rsock], [self.wsock], [], 0)
         if self.rsock in r:
             self.handle()
         if self.outbuf and self.wsock in w:
@@ -359,6 +476,7 @@ class Mux(Handler):
 
 
 class MuxWrapper(SockWrapper):
+
     def __init__(self, mux, channel):
         SockWrapper.__init__(self, mux.rsock, mux.wsock)
         self.mux = mux
@@ -372,16 +490,28 @@ class MuxWrapper(SockWrapper):
         SockWrapper.__del__(self)
 
     def __repr__(self):
-        return 'SW%r:Mux#%d' % (self.peername,self.channel)
+        return 'SW%r:Mux#%d' % (self.peername, self.channel)
 
     def noread(self):
         if not self.shut_read:
+            debug2('%r: done reading\n' % self)
             self.shut_read = True
+            self.mux.send(self.channel, CMD_TCP_STOP_SENDING, b'')
+            self.maybe_close()
 
     def nowrite(self):
         if not self.shut_write:
+            debug2('%r: done writing\n' % self)
             self.shut_write = True
-            self.mux.send(self.channel, CMD_EOF, '')
+            self.mux.send(self.channel, CMD_TCP_EOF, b'')
+            self.maybe_close()
+
+    def maybe_close(self):
+        if self.shut_read and self.shut_write:
+            debug2('%r: closing connection\n' % self)
+            # remove the mux's reference to us.  The python garbage collector
+            # will then be able to reap our object.
+            self.mux.channels[self.channel] = None
 
     def too_full(self):
         return self.mux.too_full
@@ -391,32 +521,59 @@ class MuxWrapper(SockWrapper):
             return 0  # too much already enqueued
         if len(buf) > 2048:
             buf = buf[:2048]
-        self.mux.send(self.channel, CMD_DATA, buf)
+        self.mux.send(self.channel, CMD_TCP_DATA, buf)
         return len(buf)
 
     def uread(self):
         if self.shut_read:
-            return '' # EOF
+            return b''  # EOF
         else:
             return None  # no data available right now
 
     def got_packet(self, cmd, data):
-        if cmd == CMD_CLOSE:
+        if cmd == CMD_TCP_EOF:
             self.noread()
+        elif cmd == CMD_TCP_STOP_SENDING:
             self.nowrite()
-        elif cmd == CMD_EOF:
+        elif cmd == CMD_TCP_DATA:
-            self.noread()
-        elif cmd == CMD_DATA:
             self.buf.append(data)
         else:
-            raise Exception('unknown command %d (%d bytes)' 
+            raise Exception('unknown command %d (%d bytes)'
                             % (cmd, len(data)))
 
 
-def connect_dst(ip, port):
+def connect_dst(family, ip, port):
     debug2('Connecting to %s:%d\n' % (ip, port))
-    outsock = socket.socket()
+    outsock = socket.socket(family)
     outsock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
     return SockWrapper(outsock, outsock,
-                       connect_to = (ip,port),
+                       connect_to=(ip, port),
-                       peername = '%s:%d' % (ip,port))
+                       peername = '%s:%d' % (ip, port))
+
+
+def runonce(handlers, mux):
+    r = []
+    w = []
+    x = []
+    to_remove = [s for s in handlers if not s.ok]
+    for h in to_remove:
+        handlers.remove(h)
+
+    for s in handlers:
+        s.pre_select(r, w, x)
+    debug2('Waiting: %d r=%r w=%r x=%r (fullness=%d/%d)\n'
+           % (len(handlers), _fds(r), _fds(w), _fds(x),
+               mux.fullness, mux.too_full))
+    (r, w, x) = select.select(r, w, x)
+    debug2('  Ready: %d r=%r w=%r x=%r\n'
+           % (len(handlers), _fds(r), _fds(w), _fds(x)))
+    ready = r + w + x
+    did = {}
+    for h in handlers:
+        for s in h.socks:
+            if s in ready:
+                h.callback(s)
+                did[s] = 1
+    for s in ready:
+        if s not in did:
+            raise Fatal('socket %r was not used by any handler' % s)

sshuttle/ssyslog.py

@@ -0,0 +1,19 @@
+import sys
+import os
+import subprocess as ssubprocess
+
+
+_p = None
+
+
+def start_syslog():
+    global _p
+    _p = ssubprocess.Popen(['logger',
+                            '-p', 'daemon.notice',
+                            '-t', 'sshuttle'], stdin=ssubprocess.PIPE)
+
+
+def stderr_to_syslog():
+    sys.stdout.flush()
+    sys.stderr.flush()
+    os.dup2(_p.stdin.fileno(), 2)

sshuttle/stresstest.py

@@ -0,0 +1,89 @@
+#!/usr/bin/env python
+import socket
+import select
+import struct
+import time
+
+listener = socket.socket()
+listener.bind(('127.0.0.1', 0))
+listener.listen(500)
+
+servers = []
+clients = []
+remain = {}
+
+NUMCLIENTS = 50
+count = 0
+
+
+while 1:
+    if len(clients) < NUMCLIENTS:
+        c = socket.socket()
+        c.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        c.bind(('0.0.0.0', 0))
+        c.connect(listener.getsockname())
+        count += 1
+        if count >= 16384:
+            count = 1
+        print('cli CREATING %d' % count)
+        b = struct.pack('I', count) + 'x' * count
+        remain[c] = count
+        print('cli  >> %r' % len(b))
+        c.send(b)
+        c.shutdown(socket.SHUT_WR)
+        clients.append(c)
+        r = [listener]
+        time.sleep(0.1)
+    else:
+        r = [listener] + servers + clients
+    print('select(%d)' % len(r))
+    r, w, x = select.select(r, [], [], 5)
+    assert(r)
+    for i in r:
+        if i == listener:
+            s, addr = listener.accept()
+            servers.append(s)
+        elif i in servers:
+            b = i.recv(4096)
+            print('srv <<  %r' % len(b))
+            if i not in remain:
+                assert(len(b) >= 4)
+                want = struct.unpack('I', b[:4])[0]
+                b = b[4:]
+                # i.send('y'*want)
+            else:
+                want = remain[i]
+            if want < len(b):
+                print('weird wanted %d bytes, got %d: %r' % (want, len(b), b))
+                assert(want >= len(b))
+            want -= len(b)
+            remain[i] = want
+            if not b:  # EOF
+                if want:
+                    print('weird: eof but wanted %d more' % want)
+                    assert(want == 0)
+                i.close()
+                servers.remove(i)
+                del remain[i]
+            else:
+                print('srv  >> %r' % len(b))
+                i.send('y' * len(b))
+                if not want:
+                    i.shutdown(socket.SHUT_WR)
+        elif i in clients:
+            b = i.recv(4096)
+            print('cli <<  %r' % len(b))
+            want = remain[i]
+            if want < len(b):
+                print('weird wanted %d bytes, got %d: %r' % (want, len(b), b))
+                assert(want >= len(b))
+            want -= len(b)
+            remain[i] = want
+            if not b:  # EOF
+                if want:
+                    print('weird: eof but wanted %d more' % want)
+                    assert(want == 0)
+                i.close()
+                clients.remove(i)
+                del remain[i]
+listener.accept()

sshuttle/tests/test_firewall.py

@@ -0,0 +1,109 @@
+from mock import Mock, patch, call
+import io
+import os
+import os.path
+import shutil
+import filecmp
+
+import sshuttle.firewall
+
+
+def setup_daemon():
+    stdin = io.StringIO(u"""ROUTES
+2,24,0,1.2.3.0
+2,32,1,1.2.3.66
+10,64,0,2404:6800:4004:80c::
+10,128,1,2404:6800:4004:80c::101f
+NSLIST
+2,1.2.3.33
+10,2404:6800:4004:80c::33
+PORTS 1024,1025,1026,1027
+GO 1
+""")
+    stdout = Mock()
+    return stdin, stdout
+
+
+@patch('sshuttle.firewall.HOSTSFILE', new='tmp/hosts')
+@patch('sshuttle.firewall.hostmap', new={
+    'myhost': '1.2.3.4',
+    'myotherhost': '1.2.3.5',
+})
+def test_rewrite_etc_hosts():
+    if not os.path.isdir("tmp"):
+        os.mkdir("tmp")
+
+    with open("tmp/hosts.orig", "w") as f:
+        f.write("1.2.3.3 existing\n")
+
+    shutil.copyfile("tmp/hosts.orig", "tmp/hosts")
+
+    sshuttle.firewall.rewrite_etc_hosts(10)
+    with open("tmp/hosts") as f:
+        line = f.readline()
+        s = line.split()
+        assert s == ['1.2.3.3', 'existing']
+
+        line = f.readline()
+        s = line.split()
+        assert s == ['1.2.3.4', 'myhost',
+                     '#', 'sshuttle-firewall-10', 'AUTOCREATED']
+
+        line = f.readline()
+        s = line.split()
+        assert s == ['1.2.3.5', 'myotherhost',
+                     '#', 'sshuttle-firewall-10', 'AUTOCREATED']
+
+        line = f.readline()
+        assert line == ""
+
+    sshuttle.firewall.restore_etc_hosts(10)
+    assert filecmp.cmp("tmp/hosts.orig", "tmp/hosts", shallow=False) is True
+
+
+@patch('sshuttle.firewall.HOSTSFILE', new='tmp/hosts')
+@patch('sshuttle.firewall.setup_daemon')
+@patch('sshuttle.firewall.get_method')
+def test_main(mock_get_method, mock_setup_daemon):
+    stdin, stdout = setup_daemon()
+    mock_setup_daemon.return_value = stdin, stdout
+
+    if not os.path.isdir("tmp"):
+        os.mkdir("tmp")
+
+    sshuttle.firewall.main("test", False)
+
+    with open("tmp/hosts") as f:
+        line = f.readline()
+        s = line.split()
+        assert s == ['1.2.3.3', 'existing']
+
+        line = f.readline()
+        assert line == ""
+
+    stdout.mock_calls == [
+        call.write('READY test\n'),
+        call.flush(),
+        call.write('STARTED\n'),
+        call.flush()
+    ]
+    mock_setup_daemon.mock_calls == [call()]
+    mock_get_method.mock_calls == [
+        call('test'),
+        call().setup_firewall(
+            1024, 1026,
+            [(10, u'2404:6800:4004:80c::33')],
+            10,
+            [(10, 64, False, u'2404:6800:4004:80c::'),
+                (10, 128, True, u'2404:6800:4004:80c::101f')],
+            True),
+        call().setup_firewall(
+            1025, 1027,
+            [(2, u'1.2.3.33')],
+            2,
+            [(2, 24, False, u'1.2.3.0'), (2, 32, True, u'1.2.3.66')],
+            True),
+        call().setup_firewall()(),
+        call().setup_firewall(1024, 0, [], 10, [], True),
+        call().setup_firewall(1025, 0, [], 2, [], True),
+    ]

sshuttle/tests/test_helpers.py

@@ -0,0 +1,171 @@
+from mock import patch, call
+import sys
+import io
+import socket
+
+import sshuttle.helpers
+
+
+@patch('sshuttle.helpers.logprefix', new='prefix: ')
+@patch('sshuttle.helpers.sys.stdout')
+@patch('sshuttle.helpers.sys.stderr')
+def test_log(mock_stderr, mock_stdout):
+    sshuttle.helpers.log("message")
+    assert mock_stdout.mock_calls == [
+        call.flush(),
+    ]
+    assert mock_stderr.mock_calls == [
+        call.write('prefix: message'),
+        call.flush(),
+    ]
+
+
+@patch('sshuttle.helpers.logprefix', new='prefix: ')
+@patch('sshuttle.helpers.verbose', new=1)
+@patch('sshuttle.helpers.sys.stdout')
+@patch('sshuttle.helpers.sys.stderr')
+def test_debug1(mock_stderr, mock_stdout):
+    sshuttle.helpers.debug1("message")
+    assert mock_stdout.mock_calls == [
+        call.flush(),
+    ]
+    assert mock_stderr.mock_calls == [
+        call.write('prefix: message'),
+        call.flush(),
+    ]
+
+
+@patch('sshuttle.helpers.logprefix', new='prefix: ')
+@patch('sshuttle.helpers.verbose', new=0)
+@patch('sshuttle.helpers.sys.stdout')
+@patch('sshuttle.helpers.sys.stderr')
+def test_debug1_nop(mock_stderr, mock_stdout):
+    sshuttle.helpers.debug1("message")
+    assert mock_stdout.mock_calls == []
+    assert mock_stderr.mock_calls == []
+
+
+@patch('sshuttle.helpers.logprefix', new='prefix: ')
+@patch('sshuttle.helpers.verbose', new=2)
+@patch('sshuttle.helpers.sys.stdout')
+@patch('sshuttle.helpers.sys.stderr')
+def test_debug2(mock_stderr, mock_stdout):
+    sshuttle.helpers.debug2("message")
+    assert mock_stdout.mock_calls == [
+        call.flush(),
+    ]
+    assert mock_stderr.mock_calls == [
+        call.write('prefix: message'),
+        call.flush(),
+    ]
+
+
+@patch('sshuttle.helpers.logprefix', new='prefix: ')
+@patch('sshuttle.helpers.verbose', new=1)
+@patch('sshuttle.helpers.sys.stdout')
+@patch('sshuttle.helpers.sys.stderr')
+def test_debug2_nop(mock_stderr, mock_stdout):
+    sshuttle.helpers.debug2("message")
+    assert mock_stdout.mock_calls == []
+    assert mock_stderr.mock_calls == []
+
+
+@patch('sshuttle.helpers.logprefix', new='prefix: ')
+@patch('sshuttle.helpers.verbose', new=3)
+@patch('sshuttle.helpers.sys.stdout')
+@patch('sshuttle.helpers.sys.stderr')
+def test_debug3(mock_stderr, mock_stdout):
+    sshuttle.helpers.debug3("message")
+    assert mock_stdout.mock_calls == [
+        call.flush(),
+    ]
+    assert mock_stderr.mock_calls == [
+        call.write('prefix: message'),
+        call.flush(),
+    ]
+
+
+@patch('sshuttle.helpers.logprefix', new='prefix: ')
+@patch('sshuttle.helpers.verbose', new=2)
+@patch('sshuttle.helpers.sys.stdout')
+@patch('sshuttle.helpers.sys.stderr')
+def test_debug3_nop(mock_stderr, mock_stdout):
+    sshuttle.helpers.debug3("message")
+    assert mock_stdout.mock_calls == []
+    assert mock_stderr.mock_calls == []
+
+
+@patch('sshuttle.helpers.open', create=True)
+def test_resolvconf_nameservers(mock_open):
+    mock_open.return_value = io.StringIO(u"""
+# Generated by NetworkManager
+search pri
+nameserver 192.168.1.1
+nameserver 192.168.2.1
+nameserver 192.168.3.1
+nameserver 192.168.4.1
+nameserver 2404:6800:4004:80c::1
+nameserver 2404:6800:4004:80c::2
+nameserver 2404:6800:4004:80c::3
+nameserver 2404:6800:4004:80c::4
+""")
+
+    ns = sshuttle.helpers.resolvconf_nameservers()
+    assert ns == [
+        (2, u'192.168.1.1'), (2, u'192.168.2.1'),
+        (2, u'192.168.3.1'), (2, u'192.168.4.1'),
+        (10, u'2404:6800:4004:80c::1'), (10, u'2404:6800:4004:80c::2'),
+        (10, u'2404:6800:4004:80c::3'), (10, u'2404:6800:4004:80c::4')
+    ]
+
+
+@patch('sshuttle.helpers.open', create=True)
+def test_resolvconf_random_nameserver(mock_open):
+    mock_open.return_value = io.StringIO(u"""
+# Generated by NetworkManager
+search pri
+nameserver 192.168.1.1
+nameserver 192.168.2.1
+nameserver 192.168.3.1
+nameserver 192.168.4.1
+nameserver 2404:6800:4004:80c::1
+nameserver 2404:6800:4004:80c::2
+nameserver 2404:6800:4004:80c::3
+nameserver 2404:6800:4004:80c::4
+""")
+    ns = sshuttle.helpers.resolvconf_random_nameserver()
+    assert ns in [
+        (2, u'192.168.1.1'), (2, u'192.168.2.1'),
+        (2, u'192.168.3.1'), (2, u'192.168.4.1'),
+        (10, u'2404:6800:4004:80c::1'), (10, u'2404:6800:4004:80c::2'),
+        (10, u'2404:6800:4004:80c::3'), (10, u'2404:6800:4004:80c::4')
+    ]
+
+
+def test_islocal():
+    assert sshuttle.helpers.islocal("127.0.0.1", socket.AF_INET)
+    assert not sshuttle.helpers.islocal("192.0.2.1", socket.AF_INET)
+    assert sshuttle.helpers.islocal("::1", socket.AF_INET6)
+    assert not sshuttle.helpers.islocal("2001:db8::1", socket.AF_INET6)
+
+
+def test_family_ip_tuple():
+    assert sshuttle.helpers.family_ip_tuple("127.0.0.1") \
+        == (socket.AF_INET, "127.0.0.1")
+    assert sshuttle.helpers.family_ip_tuple("192.168.2.6") \
+        == (socket.AF_INET, "192.168.2.6")
+    assert sshuttle.helpers.family_ip_tuple("::1") \
+        == (socket.AF_INET6, "::1")
+    assert sshuttle.helpers.family_ip_tuple("2404:6800:4004:80c::1") \
+        == (socket.AF_INET6, "2404:6800:4004:80c::1")
+
+
+def test_family_to_string():
+    assert sshuttle.helpers.family_to_string(socket.AF_INET) == "AF_INET"
+    assert sshuttle.helpers.family_to_string(socket.AF_INET6) == "AF_INET6"
+    if sys.version_info < (3, 0):
+        expected = "1"
+        assert sshuttle.helpers.family_to_string(socket.AF_UNIX) == "1"
+    else:
+        expected = 'AddressFamily.AF_UNIX'
+    assert sshuttle.helpers.family_to_string(socket.AF_UNIX) == expected

sshuttle/tests/test_methods_nat.py

@@ -0,0 +1,145 @@
+import pytest
+from mock import Mock, patch, call
+import socket
+import struct
+
+from sshuttle.methods import get_method
+
+
+def test_get_supported_features():
+    method = get_method('nat')
+    features = method.get_supported_features()
+    assert not features.ipv6
+    assert not features.udp
+
+
+def test_get_tcp_dstip():
+    sock = Mock()
+    sock.getsockopt.return_value = struct.pack(
+        '!HHBBBB', socket.ntohs(socket.AF_INET), 1024, 127, 0, 0, 1)
+    method = get_method('nat')
+    assert method.get_tcp_dstip(sock) == ('127.0.0.1', 1024)
+    assert sock.mock_calls == [call.getsockopt(0, 80, 16)]
+
+
+def test_recv_udp():
+    sock = Mock()
+    sock.recvfrom.return_value = "11111", "127.0.0.1"
+    method = get_method('nat')
+    result = method.recv_udp(sock, 1024)
+    assert sock.mock_calls == [call.recvfrom(1024)]
+    assert result == ("127.0.0.1", None, "11111")
+
+
+def test_send_udp():
+    sock = Mock()
+    method = get_method('nat')
+    method.send_udp(sock, None, "127.0.0.1", "22222")
+    assert sock.mock_calls == [call.sendto("22222", "127.0.0.1")]
+
+
+def test_setup_tcp_listener():
+    listener = Mock()
+    method = get_method('nat')
+    method.setup_tcp_listener(listener)
+    assert listener.mock_calls == []
+
+
+def test_setup_udp_listener():
+    listener = Mock()
+    method = get_method('nat')
+    method.setup_udp_listener(listener)
+    assert listener.mock_calls == []
+
+
+def test_check_settings():
+    method = get_method('nat')
+    method.check_settings(True, True)
+    method.check_settings(False, True)
+
+
+def test_firewall_command():
+    method = get_method('nat')
+    assert not method.firewall_command("somthing")
+
+
+@patch('sshuttle.methods.nat.ipt')
+@patch('sshuttle.methods.nat.ipt_ttl')
+@patch('sshuttle.methods.nat.ipt_chain_exists')
+def test_setup_firewall(mock_ipt_chain_exists, mock_ipt_ttl, mock_ipt):
+    mock_ipt_chain_exists.return_value = True
+    method = get_method('nat')
+    assert method.name == 'nat'
+
+    with pytest.raises(Exception) as excinfo:
+        method.setup_firewall(
+            1024, 1026,
+            [(10, u'2404:6800:4004:80c::33')],
+            10,
+            [(10, 64, False, u'2404:6800:4004:80c::'),
+                (10, 128, True, u'2404:6800:4004:80c::101f')],
+            True)
+    assert str(excinfo.value) \
+        == 'Address family "AF_INET6" unsupported by nat method_name'
+    assert mock_ipt_chain_exists.mock_calls == []
+    assert mock_ipt_ttl.mock_calls == []
+    assert mock_ipt.mock_calls == []
+
+    with pytest.raises(Exception) as excinfo:
+        method.setup_firewall(
+            1025, 1027,
+            [(2, u'1.2.3.33')],
+            2,
+            [(2, 24, False, u'1.2.3.0'), (2, 32, True, u'1.2.3.66')],
+            True)
+    assert str(excinfo.value) == 'UDP not supported by nat method_name'
+    assert mock_ipt_chain_exists.mock_calls == []
+    assert mock_ipt_ttl.mock_calls == []
+    assert mock_ipt.mock_calls == []
+
+    method.setup_firewall(
+        1025, 1027,
+        [(2, u'1.2.3.33')],
+        2,
+        [(2, 24, False, u'1.2.3.0'), (2, 32, True, u'1.2.3.66')],
+        False)
+    assert mock_ipt_chain_exists.mock_calls == [
+        call(2, 'nat', 'sshuttle-1025')
+    ]
+    assert mock_ipt_ttl.mock_calls == [
+        call(2, 'nat', '-A', 'sshuttle-1025', '-j', 'REDIRECT',
+             '--dest', u'1.2.3.0/24', '-p', 'tcp', '--to-ports', '1025'),
+        call(2, 'nat', '-A', 'sshuttle-1025', '-j', 'REDIRECT',
+             '--dest', u'1.2.3.33/32', '-p', 'udp',
+             '--dport', '53', '--to-ports', '1027')
+    ]
+    assert mock_ipt.mock_calls == [
+        call(2, 'nat', '-D', 'OUTPUT', '-j', 'sshuttle-1025'),
+        call(2, 'nat', '-D', 'PREROUTING', '-j', 'sshuttle-1025'),
+        call(2, 'nat', '-F', 'sshuttle-1025'),
+        call(2, 'nat', '-X', 'sshuttle-1025'),
+        call(2, 'nat', '-N', 'sshuttle-1025'),
+        call(2, 'nat', '-F', 'sshuttle-1025'),
+        call(2, 'nat', '-I', 'OUTPUT', '1', '-j', 'sshuttle-1025'),
+        call(2, 'nat', '-I', 'PREROUTING', '1', '-j', 'sshuttle-1025'),
+        call(2, 'nat', '-A', 'sshuttle-1025', '-j', 'RETURN',
+             '--dest', u'1.2.3.66/32', '-p', 'tcp')
+    ]
+    mock_ipt_chain_exists.reset_mock()
+    mock_ipt_ttl.reset_mock()
+    mock_ipt.reset_mock()
+
+    method.setup_firewall(1025, 0, [], 2, [], False)
+    assert mock_ipt_chain_exists.mock_calls == [
+        call(2, 'nat', 'sshuttle-1025')
+    ]
+    assert mock_ipt_ttl.mock_calls == []
+    assert mock_ipt.mock_calls == [
+        call(2, 'nat', '-D', 'OUTPUT', '-j', 'sshuttle-1025'),
+        call(2, 'nat', '-D', 'PREROUTING', '-j', 'sshuttle-1025'),
+        call(2, 'nat', '-F', 'sshuttle-1025'),
+        call(2, 'nat', '-X', 'sshuttle-1025')
+    ]
+    mock_ipt_chain_exists.reset_mock()
+    mock_ipt_ttl.reset_mock()
+    mock_ipt.reset_mock()

sshuttle/tests/test_methods_pf.py

@@ -0,0 +1,160 @@
+import pytest
+from mock import Mock, patch, call, ANY
+import socket
+
+from sshuttle.methods import get_method
+
+
+def test_get_supported_features():
+    method = get_method('pf')
+    features = method.get_supported_features()
+    assert not features.ipv6
+    assert not features.udp
+
+
+def test_get_tcp_dstip():
+    sock = Mock()
+    sock.getpeername.return_value = ("127.0.0.1", 1024)
+    sock.getsockname.return_value = ("127.0.0.2", 1025)
+    sock.family = socket.AF_INET
+
+    firewall = Mock()
+    firewall.pfile.readline.return_value = \
+        "QUERY_PF_NAT_SUCCESS 127.0.0.3,1026\n"
+
+    method = get_method('pf')
+    method.set_firewall(firewall)
+    assert method.get_tcp_dstip(sock) == ('127.0.0.3', 1026)
+
+    assert sock.mock_calls == [
+        call.getpeername(),
+        call.getsockname(),
+    ]
+    assert firewall.mock_calls == [
+        call.pfile.write('QUERY_PF_NAT 2,6,127.0.0.1,1024,127.0.0.2,1025\n'),
+        call.pfile.flush(),
+        call.pfile.readline()
+    ]
+
+
+def test_recv_udp():
+    sock = Mock()
+    sock.recvfrom.return_value = "11111", "127.0.0.1"
+    method = get_method('pf')
+    result = method.recv_udp(sock, 1024)
+    assert sock.mock_calls == [call.recvfrom(1024)]
+    assert result == ("127.0.0.1", None, "11111")
+
+
+def test_send_udp():
+    sock = Mock()
+    method = get_method('pf')
+    method.send_udp(sock, None, "127.0.0.1", "22222")
+    assert sock.mock_calls == [call.sendto("22222", "127.0.0.1")]
+
+
+def test_setup_tcp_listener():
+    listener = Mock()
+    method = get_method('pf')
+    method.setup_tcp_listener(listener)
+    assert listener.mock_calls == []
+
+
+def test_setup_udp_listener():
+    listener = Mock()
+    method = get_method('pf')
+    method.setup_udp_listener(listener)
+    assert listener.mock_calls == []
+
+
+def test_check_settings():
+    method = get_method('pf')
+    method.check_settings(True, True)
+    method.check_settings(False, True)
+
+
+@patch('sshuttle.methods.pf.sys.stdout')
+@patch('sshuttle.methods.pf.ioctl')
+@patch('sshuttle.methods.pf.pf_get_dev')
+def test_firewall_command(mock_pf_get_dev, mock_ioctl, mock_stdout):
+    method = get_method('pf')
+    assert not method.firewall_command("somthing")
+
+    command = "QUERY_PF_NAT %d,%d,%s,%d,%s,%d\n" % (
+        socket.AF_INET, socket.IPPROTO_TCP,
+        "127.0.0.1", 1025, "127.0.0.2", 1024)
+    assert method.firewall_command(command)
+
+    assert mock_pf_get_dev.mock_calls == [call()]
+    assert mock_ioctl.mock_calls == [
+        call(mock_pf_get_dev(), 3226747927, ANY),
+    ]
+    assert mock_stdout.mock_calls == [
+        call.write('QUERY_PF_NAT_SUCCESS 0.0.0.0,0\n'),
+        call.flush(),
+    ]
+
+
+# FIXME - test fails with platform=='darwin' due re.search not liking Mock
+# objects.
+@patch('sshuttle.methods.pf.sys.platform', 'not_darwin')
+@patch('sshuttle.methods.pf.pfctl')
+@patch('sshuttle.methods.pf.ioctl')
+@patch('sshuttle.methods.pf.pf_get_dev')
+def test_setup_firewall(mock_pf_get_dev, mock_ioctl, mock_pfctl):
+    method = get_method('pf')
+    assert method.name == 'pf'
+
+    with pytest.raises(Exception) as excinfo:
+        method.setup_firewall(
+            1024, 1026,
+            [(10, u'2404:6800:4004:80c::33')],
+            10,
+            [(10, 64, False, u'2404:6800:4004:80c::'),
+                (10, 128, True, u'2404:6800:4004:80c::101f')],
+            True)
+    assert str(excinfo.value) \
+        == 'Address family "AF_INET6" unsupported by pf method_name'
+    assert mock_pf_get_dev.mock_calls == []
+    assert mock_ioctl.mock_calls == []
+    assert mock_pfctl.mock_calls == []
+
+    with pytest.raises(Exception) as excinfo:
+        method.setup_firewall(
+            1025, 1027,
+            [(2, u'1.2.3.33')],
+            2,
+            [(2, 24, False, u'1.2.3.0'), (2, 32, True, u'1.2.3.66')],
+            True)
+    assert str(excinfo.value) == 'UDP not supported by pf method_name'
+    assert mock_pf_get_dev.mock_calls == []
+    assert mock_ioctl.mock_calls == []
+    assert mock_pfctl.mock_calls == []
+
+    method.setup_firewall(
+        1025, 1027,
+        [(2, u'1.2.3.33')],
+        2,
+        [(2, 24, False, u'1.2.3.0'), (2, 32, True, u'1.2.3.66')],
+        False)
+    assert mock_ioctl.mock_calls == [
+        call(mock_pf_get_dev(), 3295691827, ANY),
+        call(mock_pf_get_dev(), 3424666650, ANY),
+        call(mock_pf_get_dev(), 3424666650, ANY),
+        call(mock_pf_get_dev(), 3295691827, ANY),
+        call(mock_pf_get_dev(), 3424666650, ANY),
+        call(mock_pf_get_dev(), 3424666650, ANY),
+    ]
+    # FIXME - needs more work
+    # print(mock_pfctl.mock_calls)
+    # assert mock_pfctl.mock_calls == []
+    mock_pf_get_dev.reset_mock()
+    mock_ioctl.reset_mock()
+    mock_pfctl.reset_mock()
+
+    method.setup_firewall(1025, 0, [], 2, [], False)
+    assert mock_ioctl.mock_calls == []
+    assert mock_pfctl.mock_calls == [call('-a sshuttle -F all')]
+    mock_pf_get_dev.reset_mock()
+    mock_pfctl.reset_mock()
+    mock_ioctl.reset_mock()

sshuttle/tests/test_methods_tproxy.py

@@ -0,0 +1,272 @@
+from mock import Mock, patch, call
+
+from sshuttle.methods import get_method
+
+
+def test_get_supported_features():
+    method = get_method('tproxy')
+    features = method.get_supported_features()
+    assert features.ipv6
+    assert features.udp
+
+
+def test_get_tcp_dstip():
+    sock = Mock()
+    sock.getsockname.return_value = ('127.0.0.1', 1024)
+    method = get_method('tproxy')
+    assert method.get_tcp_dstip(sock) == ('127.0.0.1', 1024)
+    assert sock.mock_calls == [call.getsockname()]
+
+
+@patch("sshuttle.methods.tproxy.recv_udp")
+def test_recv_udp(mock_recv_udp):
+    mock_recv_udp.return_value = ("127.0.0.1", "127.0.0.2", "11111")
+
+    sock = Mock()
+    method = get_method('tproxy')
+    result = method.recv_udp(sock, 1024)
+    assert sock.mock_calls == []
+    assert mock_recv_udp.mock_calls == [call(sock, 1024)]
+    assert result == ("127.0.0.1", "127.0.0.2", "11111")
+
+
+@patch("sshuttle.methods.socket.socket")
+def test_send_udp(mock_socket):
+    sock = Mock()
+    method = get_method('tproxy')
+    method.send_udp(sock, "127.0.0.2", "127.0.0.1", "2222222")
+    assert sock.mock_calls == []
+    assert mock_socket.mock_calls == [
+        call(sock.family, 2),
+        call().setsockopt(1, 2, 1),
+        call().setsockopt(0, 19, 1),
+        call().bind('127.0.0.2'),
+        call().sendto("2222222", '127.0.0.1'),
+        call().close()
+    ]
+
+
+def test_setup_tcp_listener():
+    listener = Mock()
+    method = get_method('tproxy')
+    method.setup_tcp_listener(listener)
+    assert listener.mock_calls == [
+        call.setsockopt(0, 19, 1)
+    ]
+
+
+def test_setup_udp_listener():
+    listener = Mock()
+    method = get_method('tproxy')
+    method.setup_udp_listener(listener)
+    assert listener.mock_calls == [
+        call.setsockopt(0, 19, 1),
+        call.v4.setsockopt(0, 20, 1),
+        call.v6.setsockopt(41, 74, 1)
+    ]
+
+
+def test_check_settings():
+    method = get_method('tproxy')
+    method.check_settings(True, True)
+    method.check_settings(False, True)
+
+
+def test_firewall_command():
+    method = get_method('tproxy')
+    assert not method.firewall_command("somthing")
+
+
+@patch('sshuttle.methods.tproxy.ipt')
+@patch('sshuttle.methods.tproxy.ipt_ttl')
+@patch('sshuttle.methods.tproxy.ipt_chain_exists')
+def test_setup_firewall(mock_ipt_chain_exists, mock_ipt_ttl, mock_ipt):
+    mock_ipt_chain_exists.return_value = True
+    method = get_method('tproxy')
+    assert method.name == 'tproxy'
+
+    # IPV6
+
+    method.setup_firewall(
+        1024, 1026,
+        [(10, u'2404:6800:4004:80c::33')],
+        10,
+        [(10, 64, False, u'2404:6800:4004:80c::'),
+            (10, 128, True, u'2404:6800:4004:80c::101f')],
+        True)
+    assert mock_ipt_chain_exists.mock_calls == [
+        call(10, 'mangle', 'sshuttle-m-1024'),
+        call(10, 'mangle', 'sshuttle-t-1024'),
+        call(10, 'mangle', 'sshuttle-d-1024')
+    ]
+    assert mock_ipt_ttl.mock_calls == []
+    assert mock_ipt.mock_calls == [
+        call(10, 'mangle', '-D', 'OUTPUT', '-j', 'sshuttle-m-1024'),
+        call(10, 'mangle', '-F', 'sshuttle-m-1024'),
+        call(10, 'mangle', '-X', 'sshuttle-m-1024'),
+        call(10, 'mangle', '-D', 'PREROUTING', '-j', 'sshuttle-t-1024'),
+        call(10, 'mangle', '-F', 'sshuttle-t-1024'),
+        call(10, 'mangle', '-X', 'sshuttle-t-1024'),
+        call(10, 'mangle', '-F', 'sshuttle-d-1024'),
+        call(10, 'mangle', '-X', 'sshuttle-d-1024'),
+        call(10, 'mangle', '-N', 'sshuttle-m-1024'),
+        call(10, 'mangle', '-F', 'sshuttle-m-1024'),
+        call(10, 'mangle', '-N', 'sshuttle-d-1024'),
+        call(10, 'mangle', '-F', 'sshuttle-d-1024'),
+        call(10, 'mangle', '-N', 'sshuttle-t-1024'),
+        call(10, 'mangle', '-F', 'sshuttle-t-1024'),
+        call(10, 'mangle', '-I', 'OUTPUT', '1', '-j', 'sshuttle-m-1024'),
+        call(10, 'mangle', '-I', 'PREROUTING', '1', '-j', 'sshuttle-t-1024'),
+        call(10, 'mangle', '-A', 'sshuttle-d-1024', '-j', 'MARK',
+             '--set-mark', '1'),
+        call(10, 'mangle', '-A', 'sshuttle-d-1024', '-j', 'ACCEPT'),
+        call(10, 'mangle', '-A', 'sshuttle-t-1024', '-m', 'socket',
+             '-j', 'sshuttle-d-1024', '-m', 'tcp', '-p', 'tcp'),
+        call(10, 'mangle', '-A', 'sshuttle-t-1024', '-m', 'socket',
+             '-j', 'sshuttle-d-1024', '-m', 'udp', '-p', 'udp'),
+        call(10, 'mangle', '-A', 'sshuttle-m-1024', '-j', 'MARK',
+             '--set-mark', '1', '--dest', u'2404:6800:4004:80c::33/32',
+             '-m', 'udp', '-p', 'udp', '--dport', '53'),
+        call(10, 'mangle', '-A', 'sshuttle-t-1024', '-j', 'TPROXY',
+             '--tproxy-mark', '0x1/0x1',
+             '--dest', u'2404:6800:4004:80c::33/32',
+             '-m', 'udp', '-p', 'udp', '--dport', '53', '--on-port', '1026'),
+        call(10, 'mangle', '-A', 'sshuttle-m-1024', '-j', 'RETURN',
+             '--dest', u'2404:6800:4004:80c::101f/128',
+             '-m', 'tcp', '-p', 'tcp'),
+        call(10, 'mangle', '-A', 'sshuttle-t-1024', '-j', 'RETURN',
+             '--dest', u'2404:6800:4004:80c::101f/128',
+             '-m', 'tcp', '-p', 'tcp'),
+        call(10, 'mangle', '-A', 'sshuttle-m-1024', '-j', 'RETURN',
+             '--dest', u'2404:6800:4004:80c::101f/128',
+             '-m', 'udp', '-p', 'udp'),
+        call(10, 'mangle', '-A', 'sshuttle-t-1024', '-j', 'RETURN',
+             '--dest', u'2404:6800:4004:80c::101f/128',
+             '-m', 'udp', '-p', 'udp'),
+        call(10, 'mangle', '-A', 'sshuttle-m-1024', '-j', 'MARK',
+             '--set-mark', '1', '--dest', u'2404:6800:4004:80c::/64',
+             '-m', 'tcp', '-p', 'tcp'),
+        call(10, 'mangle', '-A', 'sshuttle-t-1024', '-j', 'TPROXY',
+             '--tproxy-mark', '0x1/0x1', '--dest', u'2404:6800:4004:80c::/64',
+             '-m', 'tcp', '-p', 'tcp', '--on-port', '1024'),
+        call(10, 'mangle', '-A', 'sshuttle-m-1024', '-j', 'MARK',
+             '--set-mark', '1', '--dest', u'2404:6800:4004:80c::/64',
+             '-m', 'udp', '-p', 'udp'),
+        call(10, 'mangle', '-A', 'sshuttle-t-1024', '-j', 'TPROXY',
+             '--tproxy-mark', '0x1/0x1', '--dest', u'2404:6800:4004:80c::/64',
+             '-m', 'udp', '-p', 'udp', '--on-port', '1024')
+    ]
+    mock_ipt_chain_exists.reset_mock()
+    mock_ipt_ttl.reset_mock()
+    mock_ipt.reset_mock()
+
+    method.setup_firewall(1025, 0, [], 10, [], True)
+    assert mock_ipt_chain_exists.mock_calls == [
+        call(10, 'mangle', 'sshuttle-m-1025'),
+        call(10, 'mangle', 'sshuttle-t-1025'),
+        call(10, 'mangle', 'sshuttle-d-1025')
+    ]
+    assert mock_ipt_ttl.mock_calls == []
+    assert mock_ipt.mock_calls == [
+        call(10, 'mangle', '-D', 'OUTPUT', '-j', 'sshuttle-m-1025'),
+        call(10, 'mangle', '-F', 'sshuttle-m-1025'),
+        call(10, 'mangle', '-X', 'sshuttle-m-1025'),
+        call(10, 'mangle', '-D', 'PREROUTING', '-j', 'sshuttle-t-1025'),
+        call(10, 'mangle', '-F', 'sshuttle-t-1025'),
+        call(10, 'mangle', '-X', 'sshuttle-t-1025'),
+        call(10, 'mangle', '-F', 'sshuttle-d-1025'),
+        call(10, 'mangle', '-X', 'sshuttle-d-1025')
+    ]
+    mock_ipt_chain_exists.reset_mock()
+    mock_ipt_ttl.reset_mock()
+    mock_ipt.reset_mock()
+
+    # IPV4
+
+    method.setup_firewall(
+        1025, 1027,
+        [(2, u'1.2.3.33')],
+        2,
+        [(2, 24, False, u'1.2.3.0'), (2, 32, True, u'1.2.3.66')],
+        True)
+    assert mock_ipt_chain_exists.mock_calls == [
+        call(2, 'mangle', 'sshuttle-m-1025'),
+        call(2, 'mangle', 'sshuttle-t-1025'),
+        call(2, 'mangle', 'sshuttle-d-1025')
+    ]
+    assert mock_ipt_ttl.mock_calls == []
+    assert mock_ipt.mock_calls == [
+        call(2, 'mangle', '-D', 'OUTPUT', '-j', 'sshuttle-m-1025'),
+        call(2, 'mangle', '-F', 'sshuttle-m-1025'),
+        call(2, 'mangle', '-X', 'sshuttle-m-1025'),
+        call(2, 'mangle', '-D', 'PREROUTING', '-j', 'sshuttle-t-1025'),
+        call(2, 'mangle', '-F', 'sshuttle-t-1025'),
+        call(2, 'mangle', '-X', 'sshuttle-t-1025'),
+        call(2, 'mangle', '-F', 'sshuttle-d-1025'),
+        call(2, 'mangle', '-X', 'sshuttle-d-1025'),
+        call(2, 'mangle', '-N', 'sshuttle-m-1025'),
+        call(2, 'mangle', '-F', 'sshuttle-m-1025'),
+        call(2, 'mangle', '-N', 'sshuttle-d-1025'),
+        call(2, 'mangle', '-F', 'sshuttle-d-1025'),
+        call(2, 'mangle', '-N', 'sshuttle-t-1025'),
+        call(2, 'mangle', '-F', 'sshuttle-t-1025'),
+        call(2, 'mangle', '-I', 'OUTPUT', '1', '-j', 'sshuttle-m-1025'),
+        call(2, 'mangle', '-I', 'PREROUTING', '1', '-j', 'sshuttle-t-1025'),
+        call(2, 'mangle', '-A', 'sshuttle-d-1025',
+             '-j', 'MARK', '--set-mark', '1'),
+        call(2, 'mangle', '-A', 'sshuttle-d-1025', '-j', 'ACCEPT'),
+        call(2, 'mangle', '-A', 'sshuttle-t-1025', '-m', 'socket',
+             '-j', 'sshuttle-d-1025', '-m', 'tcp', '-p', 'tcp'),
+        call(2, 'mangle', '-A', 'sshuttle-t-1025', '-m', 'socket',
+             '-j', 'sshuttle-d-1025', '-m', 'udp', '-p', 'udp'),
+        call(2, 'mangle', '-A', 'sshuttle-m-1025', '-j', 'MARK',
+             '--set-mark', '1', '--dest', u'1.2.3.33/32',
+             '-m', 'udp', '-p', 'udp', '--dport', '53'),
+        call(2, 'mangle', '-A', 'sshuttle-t-1025', '-j', 'TPROXY',
+             '--tproxy-mark', '0x1/0x1', '--dest', u'1.2.3.33/32',
+             '-m', 'udp', '-p', 'udp', '--dport', '53', '--on-port', '1027'),
+        call(2, 'mangle', '-A', 'sshuttle-m-1025', '-j', 'RETURN',
+             '--dest', u'1.2.3.66/32', '-m', 'tcp', '-p', 'tcp'),
+        call(2, 'mangle', '-A', 'sshuttle-t-1025', '-j', 'RETURN',
+             '--dest', u'1.2.3.66/32', '-m', 'tcp', '-p', 'tcp'),
+        call(2, 'mangle', '-A', 'sshuttle-m-1025', '-j', 'RETURN',
+             '--dest', u'1.2.3.66/32', '-m', 'udp', '-p', 'udp'),
+        call(2, 'mangle', '-A', 'sshuttle-t-1025', '-j', 'RETURN',
+             '--dest', u'1.2.3.66/32', '-m', 'udp', '-p', 'udp'),
+        call(2, 'mangle', '-A', 'sshuttle-m-1025', '-j', 'MARK',
+             '--set-mark', '1', '--dest', u'1.2.3.0/24',
+             '-m', 'tcp', '-p', 'tcp'),
+        call(2, 'mangle', '-A', 'sshuttle-t-1025', '-j', 'TPROXY',
+             '--tproxy-mark', '0x1/0x1', '--dest', u'1.2.3.0/24',
+             '-m', 'tcp', '-p', 'tcp', '--on-port', '1025'),
+        call(2, 'mangle', '-A', 'sshuttle-m-1025', '-j', 'MARK',
+             '--set-mark', '1', '--dest', u'1.2.3.0/24',
+             '-m', 'udp', '-p', 'udp'),
+        call(2, 'mangle', '-A', 'sshuttle-t-1025', '-j', 'TPROXY',
+             '--tproxy-mark', '0x1/0x1', '--dest', u'1.2.3.0/24',
+             '-m', 'udp', '-p', 'udp', '--on-port', '1025')
+    ]
+    mock_ipt_chain_exists.reset_mock()
+    mock_ipt_ttl.reset_mock()
+    mock_ipt.reset_mock()
+
+    method.setup_firewall(1025, 0, [], 2, [], True)
+    assert mock_ipt_chain_exists.mock_calls == [
+        call(2, 'mangle', 'sshuttle-m-1025'),
+        call(2, 'mangle', 'sshuttle-t-1025'),
+        call(2, 'mangle', 'sshuttle-d-1025')
+    ]
+    assert mock_ipt_ttl.mock_calls == []
+    assert mock_ipt.mock_calls == [
+        call(2, 'mangle', '-D', 'OUTPUT', '-j', 'sshuttle-m-1025'),
+        call(2, 'mangle', '-F', 'sshuttle-m-1025'),
+        call(2, 'mangle', '-X', 'sshuttle-m-1025'),
+        call(2, 'mangle', '-D', 'PREROUTING', '-j', 'sshuttle-t-1025'),
+        call(2, 'mangle', '-F', 'sshuttle-t-1025'),
+        call(2, 'mangle', '-X', 'sshuttle-t-1025'),
+        call(2, 'mangle', '-F', 'sshuttle-d-1025'),
+        call(2, 'mangle', '-X', 'sshuttle-d-1025')
+    ]
+    mock_ipt_chain_exists.reset_mock()
+    mock_ipt_ttl.reset_mock()
+    mock_ipt.reset_mock()