Update changes file

Bump attrs from 19.3.0 to 20.1.0

CHANGES.rst

@@ -9,6 +9,50 @@ adheres to `Semantic Versioning`_.
 .. _`Semantic Versioning`: http://semver.org/
 
 
+1.0.3 - 2020-08-24
+------------------
+
+Fixed
+~~~~~
+* Allow Mux() flush/fill to work with python < 3.5
+* Fix parse_hostport to always return string for host.
+* Require -r/--remote parameter.
+* Add missing package in OpenWRT documentation.
+* Fix doc about --listen option.
+* README: add Ubuntu.
+* Increase IP4 ttl to 63 hops instead of 42.
+* Fix formatting in installation.rst
+
+
+1.0.3 - 2020-07-12
+------------------
+
+Fixed
+~~~~~
+* Ask setuptools to require Python 3.5 and above.
+* Add missing import.
+* Fix formatting typos in usage docs
+
+
+1.0.2 - 2020-06-18
+------------------
+
+Fixed
+~~~~~
+* Leave use of default port to ssh command.
+* Remove unwanted references to Python 2.7 in docs.
+* Replace usage of deprecated imp.
+* Fix connection with @ sign in username.
+
+
+1.0.1 - 2020-06-05
+------------------
+
+Fixed
+~~~~~
+* Errors in python long_documentation.
+
+
 1.0.0 - 2020-06-05
 ------------------
 

README.rst

@@ -30,6 +30,10 @@ common case:
 Obtaining sshuttle
 ------------------
 
+- Ubuntu 16.04 or later::
+
+      apt-get install sshuttle
+
 - Debian stretch or later::
 
       apt-get install sshuttle
@@ -98,6 +102,6 @@ https://sshuttle.readthedocs.org/en/latest/
 
 
 Running as a service
--------------
+--------------------
 Sshuttle can also be run as a service and configured using a config management system: 
 https://medium.com/@mike.reider/using-sshuttle-as-a-service-bec2684a65fe

docs/installation.rst

@@ -6,6 +6,7 @@ Installation
       pip install sshuttle
 
 - Debain package manager::
+
       sudo apt install sshuttle
 
 - Clone::
@@ -18,5 +19,6 @@ Installation
 Optionally after installation
 -----------------------------
 
-- Add to sudoers file
+- Add to sudoers file::
+
       sshuttle --sudoers

docs/manpage.rst

@@ -11,7 +11,7 @@ Description
 -----------
 :program:`sshuttle` allows you to create a VPN connection from your
 machine to any remote server that you can connect to via
-ssh, as long as that server has python 2.3 or higher.
+ssh, as long as that server has python 3.5 or higher.
 
 To work, you must have root access on the local machine,
 but you can have a normal account on the server.
@@ -65,7 +65,8 @@ Options
     :program:`sshuttle`, e.g. ``--listen localhost``.
 
     For the tproxy and pf methods this can be an IPv6 address. Use this option 
-    twice if required, to provide both IPv4 and IPv6 addresses.
+    with comma separated values if required, to provide both IPv4 and IPv6
+    addresses, e.g. ``--listen 127.0.0.1:0,[::1]:0``.
 
 .. option:: -H, --auto-hosts
 

docs/openwrt.rst

@@ -3,6 +3,6 @@ OpenWRT
 
 Run::
 
-    opkg install python3 python3-pip iptables-mod-nat-extra iptables-mod-ipopt
+    opkg install python3 python3-pip iptables-mod-extra iptables-mod-nat-extra iptables-mod-ipopt
     python3 /usr/bin/pip3 install sshuttle
     sshuttle -l 0.0.0.0 -r <IP> -x 192.168.1.1 0/0

docs/requirements.rst

@@ -57,9 +57,8 @@ cmd.exe with Administrator access. See :doc:`windows` for more information.
 
 Server side Requirements
 ------------------------
-The server can run in any version of Python between 2.4 and 3.6.
-However it is recommended that you use Python 2.7, Python 3.5 or later whenever
-possible as support for older versions might be dropped in the future.
+
+- Python 3.5 or greater.
 
 
 Additional Suggested Software

docs/usage.rst

@@ -21,7 +21,7 @@ Forward all traffic::
       sshuttle -r username@sshserver 0/0
 
 
-- For 'My VPN broke and need a temporary solution FAST to access local IPv4 addresses': 
+- For 'My VPN broke and need a temporary solution FAST to access local IPv4 addresses'::
 
       sshuttle --dns -NHr username@sshserver 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
 
@@ -70,7 +70,7 @@ Sudoers File
 sshuttle can auto-generate the proper sudoers.d file using the current user 
 for Linux and OSX. Doing this will allow sshuttle to run without asking for
 the local sudo password and to give users who do not have sudo access
-ability to run sshuttle.
+ability to run sshuttle::
 
   sshuttle --sudoers
 
@@ -82,14 +82,14 @@ option:`sshuttle --sudoers --sudoers-username {user_descriptor}` option. Valid
 values for this vary based on how your system is configured. Values such as 
 usernames, groups pre-pended with `%` and sudoers user aliases will work. See
 the sudoers manual for more information on valid user specif actions.
-The options must be used with `--sudoers`
+The options must be used with `--sudoers`::
 
   sshuttle --sudoers --sudoers-user mike
   sshuttle --sudoers --sudoers-user %sudo
 
 The name of the file to be added to sudoers.d can be configured as well. This
 is mostly not necessary but can be useful for giving more than one user
-access to sshuttle. The default is `sshuttle_auto`
+access to sshuttle. The default is `sshuttle_auto`::
 
   sshuttle --sudoer --sudoers-filename sshuttle_auto_mike
   sshuttle --sudoer --sudoers-filename sshuttle_auto_tommy
@@ -97,11 +97,11 @@ access to sshuttle. The default is `sshuttle_auto`
 You can also see what configuration will be added to your system without
 modifying anything. This can be helpfull is the auto feature does not work, or
 you want more control. This option also works with `--sudoers-username`.
-`--sudoers-filename` has no effect with this option.
+`--sudoers-filename` has no effect with this option::
 
   sshuttle --sudoers-no-modify
 
-This will simply sprint the generated configuration to STDOUT. Example
+This will simply sprint the generated configuration to STDOUT. Example::
 
   08:40 PM william$ sshuttle --sudoers-no-modify
 

requirements-tests.txt

@@ -1,7 +1,7 @@
 -r requirements.txt
-attrs==19.3.0
-pytest==5.4.3
-pytest-cov==2.9.0
+attrs==20.1.0
+pytest==6.0.1
+pytest-cov==2.10.1
 mock==2.0.0
-flake8==3.8.2
+flake8==3.8.3
 pyflakes==2.2.0

setup.py

@@ -41,6 +41,7 @@ setup(
     packages=find_packages(),
     license="LGPL2.1+",
     long_description=open('README.rst').read(),
+    long_description_content_type="text/x-rst",
     classifiers=[
         "Development Status :: 5 - Production/Stable",
         "Intended Audience :: Developers",
@@ -60,6 +61,7 @@ setup(
             'sshuttle = sshuttle.cmdline:main',
         ],
     },
+    python_requires='>=3.5',
     tests_require=[
         'pytest',
         'pytest-cov',

sshuttle/assembler.py

@@ -1,6 +1,6 @@
 import sys
 import zlib
-import imp
+import types
 
 verbosity = verbosity  # noqa: F821 must be a previously defined global
 z = zlib.decompressobj()
@@ -15,7 +15,7 @@ while 1:
                              % (name, nbytes))
         content = z.decompress(sys.stdin.read(nbytes))
 
-        module = imp.new_module(name)
+        module = types.ModuleType(name)
         parents = name.rsplit(".", 1)
         if len(parents) == 2:
             parent, parent_name = parents

sshuttle/client.py

@@ -557,6 +557,11 @@ def main(listenip_v6, listenip_v4,
          subnets_include, subnets_exclude, daemon, to_nameserver, pidfile,
          user, sudo_pythonpath):
 
+    if not remotename:
+        # XXX: We can't make it required at the argparse level,
+        #      because sshuttle calls out to itself in FirewallClient.
+        raise Fatal("You must specify -r/--remote.")
+
     if daemon:
         try:
             check_daemon(pidfile)

sshuttle/linux.py

@@ -71,10 +71,10 @@ def ipt_ttl(family, *args):
     global _no_ttl_module
     if not _no_ttl_module:
         # we avoid infinite loops by generating server-side connections
-        # with ttl 42.  This makes the client side not recapture those
+        # with ttl 63.  This makes the client side not recapture those
         # connections, in case client == server.
         try:
-            argsplus = list(args) + ['-m', 'ttl', '!', '--ttl', '42']
+            argsplus = list(args) + ['-m', 'ttl', '!', '--ttl', '63']
             ipt(family, *argsplus)
         except Fatal:
             ipt(family, *args)

sshuttle/methods/ipfw.py

@@ -70,7 +70,7 @@ def ipfw_rule_exists(n):
     found = False
     for line in p.stdout:
         if line.startswith(b'%05d ' % n):
-            if not ('ipttl 42' in line or 'check-state' in line):
+            if not ('ipttl 63' in line or 'check-state' in line):
                 log('non-sshuttle ipfw rule: %r\n' % line.strip())
                 raise Fatal('non-sshuttle ipfw rule #%d already exists!' % n)
             found = True
@@ -185,7 +185,7 @@ class Method(BaseMethod):
         sender.setsockopt(socket.SOL_IP, IP_BINDANY, 1)
         sender.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
         sender.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
-        sender.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
+        sender.setsockopt(socket.SOL_IP, socket.IP_TTL, 63)
         sender.bind(srcip)
         sender.sendto(data, dstip)
         sender.close()
@@ -224,7 +224,7 @@ class Method(BaseMethod):
         ipfw('add', '1', 'fwd', '127.0.0.1,%d' % port,
              'tcp',
              'from', 'any', 'to', 'table(126)',
-             'not', 'ipttl', '42', 'keep-state', 'setup')
+             'not', 'ipttl', '63', 'keep-state', 'setup')
 
         ipfw_noexit('table', '124', 'flush')
         dnscount = 0
@@ -235,11 +235,11 @@ class Method(BaseMethod):
             ipfw('add', '1', 'fwd', '127.0.0.1,%d' % dnsport,
                  'udp',
                  'from', 'any', 'to', 'table(124)',
-                 'not', 'ipttl', '42')
+                 'not', 'ipttl', '63')
         ipfw('add', '1', 'allow',
              'udp',
              'from', 'any', 'to', 'any',
-             'ipttl', '42')
+             'ipttl', '63')
 
         if subnets:
             # create new subnet entries

sshuttle/methods/nft.py

@@ -50,17 +50,17 @@ class Method(BaseMethod):
                      'ip daddr %s/%s' % (snet, swidth), 'return')))
             else:
                 _nft('add rule', chain, *(tcp_ports + (
-                     'ip daddr %s/%s' % (snet, swidth), 'ip ttl != 42',
+                     'ip daddr %s/%s' % (snet, swidth), 'ip ttl != 63',
                      ('redirect to :' + str(port)))))
 
         for _, ip in [i for i in nslist if i[0] == family]:
             if family == socket.AF_INET:
                 _nft('add rule', chain, 'ip protocol udp ip daddr %s' % ip,
-                     'udp dport { 53 }', 'ip ttl != 42',
+                     'udp dport { 53 }', 'ip ttl != 63',
                      ('redirect to :' + str(dnsport)))
             elif family == socket.AF_INET6:
                 _nft('add rule', chain, 'ip6 protocol udp ip6 daddr %s' % ip,
-                     'udp dport { 53 }', 'ip ttl != 42',
+                     'udp dport { 53 }', 'ip ttl != 63',
                      ('redirect to :' + str(dnsport)))
 
     def restore_firewall(self, port, family, udp, user):

sshuttle/server.py

@@ -195,7 +195,7 @@ class DnsProxy(Handler):
 
         family, sockaddr = self._addrinfo(peer, port)
         sock = socket.socket(family, socket.SOCK_DGRAM)
-        sock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
+        sock.setsockopt(socket.SOL_IP, socket.IP_TTL, 63)
         sock.connect(sockaddr)
 
         self.peers[sock] = peer
@@ -252,7 +252,7 @@ class UdpProxy(Handler):
         self.chan = chan
         self.sock = sock
         if family == socket.AF_INET:
-            self.sock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
+            self.sock.setsockopt(socket.SOL_IP, socket.IP_TTL, 63)
 
     def send(self, dstip, data):
         debug2('UDP: sending to %r port %d\n' % dstip)

sshuttle/ssh.py

@@ -3,7 +3,8 @@ import os
 import re
 import socket
 import zlib
-import imp
+import importlib
+import importlib.util
 import subprocess as ssubprocess
 import shlex
 from shlex import quote
@@ -14,43 +15,15 @@ import sshuttle.helpers as helpers
 from sshuttle.helpers import debug2
 
 
-def readfile(name):
-    tokens = name.split(".")
-    f = None
-
-    token = tokens[0]
-    token_name = [token]
-    token_str = ".".join(token_name)
-
-    try:
-        f, pathname, description = imp.find_module(token_str)
-
-        for token in tokens[1:]:
-            module = imp.load_module(token_str, f, pathname, description)
-            if f is not None:
-                f.close()
-
-            token_name.append(token)
-            token_str = ".".join(token_name)
-
-            f, pathname, description = imp.find_module(
-                token, module.__path__)
-
-        if f is not None:
-            contents = f.read()
-        else:
-            contents = ""
-
-    finally:
-        if f is not None:
-            f.close()
-
-    return contents.encode("UTF8")
+def get_module_source(name):
+    spec = importlib.util.find_spec(name)
+    with open(spec.origin, "rt") as f:
+        return f.read().encode("utf-8")
 
 
 def empackage(z, name, data=None):
     if not data:
-        data = readfile(name)
+        data = get_module_source(name)
     content = z.compress(data)
     content += z.flush(zlib.Z_SYNC_FLUSH)
 
@@ -68,17 +41,17 @@ def parse_hostport(rhostport):
 
     and returns a tuple (username, password, port, host)
     """
-    # default port for SSH is TCP port 22
-    port = 22
+    # leave use of default port to ssh command to prevent overwriting
+    # ports configured in ~/.ssh/config when no port is given
+    port = None
     username = None
     password = None
     host = rhostport
 
     if "@" in host:
         # split username (and possible password) from the host[:port]
-        username, host = host.split("@")
+        username, host = host.rsplit("@", 1)
         # Fix #410 bad username error detect
-        # username cannot contain an @ sign in this scenario
         if ":" in username:
             # this will even allow for the username to be empty
             username, password = username.split(":")
@@ -92,12 +65,12 @@ def parse_hostport(rhostport):
         try:
             # try to parse host as an IP adress,
             # if that works it is an IPv6 address
-            host = ipaddress.ip_address(host)
+            host = str(ipaddress.ip_address(host))
         except ValueError:
             # if that fails parse as URL to get the port
             parsed = urlparse('//{}'.format(host))
             try:
-                host = ipaddress.ip_address(parsed.hostname)
+                host = str(ipaddress.ip_address(parsed.hostname))
             except ValueError:
                 # else if both fails, we have a hostname with port
                 host = parsed.hostname
@@ -117,7 +90,7 @@ def connect(ssh_cmd, rhostport, python, stderr, options):
         rhost = host
 
     z = zlib.compressobj(1)
-    content = readfile('sshuttle.assembler')
+    content = get_module_source('sshuttle.assembler')
     optdata = ''.join("%s=%r\n" % (k, v) for (k, v) in list(options.items()))
     optdata = optdata.encode("UTF8")
     content2 = (empackage(z, 'sshuttle') +
@@ -145,6 +118,10 @@ def connect(ssh_cmd, rhostport, python, stderr, options):
             sshl = shlex.split(ssh_cmd)
         else:
             sshl = ['ssh']
+        if port is not None:
+            portl = ["-p", str(port)]
+        else:
+            portl = []
         if python:
             pycmd = "'%s' -c '%s'" % (python, pyscript)
         else:
@@ -155,12 +132,12 @@ def connect(ssh_cmd, rhostport, python, stderr, options):
         if password is not None:
             os.environ['SSHPASS'] = str(password)
             argv = (["sshpass", "-e"] + sshl +
-                    ["-p", str(port)] +
+                    portl +
                     [rhost, '--', pycmd])
 
         else:
             argv = (sshl +
-                    ["-p", str(port)] +
+                    portl +
                     [rhost, '--', pycmd])
     (s1, s2) = socket.socketpair()
 

sshuttle/ssnet.py

@@ -4,21 +4,19 @@ import socket
 import errno
 import select
 import os
+import fcntl
 
 from sshuttle.helpers import b, log, debug1, debug2, debug3, Fatal
 
 MAX_CHANNEL = 65535
 LATENCY_BUFFER_SIZE = 32768
 
-# these don't exist in the socket module in python 2.3!
 SHUT_RD = 0
 SHUT_WR = 1
 SHUT_RDWR = 2
 
-
 HDR_LEN = 8
 
-
 CMD_EXIT = 0x4200
 CMD_PING = 0x4201
 CMD_PONG = 0x4202
@@ -439,7 +437,13 @@ class Mux(Handler):
                 callback(cmd, data)
 
     def flush(self):
+        try:
             os.set_blocking(self.wfile.fileno(), False)
+        except AttributeError:
+            # python < 3.5
+            flags = fcntl.fcntl(self.wfile.fileno(), fcntl.F_GETFL)
+            flags |= os.O_NONBLOCK
+            flags = fcntl.fcntl(self.wfile.fileno(), fcntl.F_SETFL, flags)
         if self.outbuf and self.outbuf[0]:
             wrote = _nb_clean(os.write, self.wfile.fileno(), self.outbuf[0])
             debug2('mux wrote: %r/%d\n' % (wrote, len(self.outbuf[0])))
@@ -449,7 +453,13 @@ class Mux(Handler):
             self.outbuf[0:1] = []
 
     def fill(self):
+        try:
             os.set_blocking(self.rfile.fileno(), False)
+        except AttributeError:
+            # python < 3.5
+            flags = fcntl.fcntl(self.rfile.fileno(), fcntl.F_GETFL)
+            flags |= os.O_NONBLOCK
+            flags = fcntl.fcntl(self.rfile.fileno(), fcntl.F_SETFL, flags)
         try:
             read = _nb_clean(os.read, self.rfile.fileno(), LATENCY_BUFFER_SIZE)
         except OSError:
@@ -573,7 +583,7 @@ class MuxWrapper(SockWrapper):
 def connect_dst(family, ip, port):
     debug2('Connecting to %s:%d\n' % (ip, port))
     outsock = socket.socket(family)
-    outsock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
+    outsock.setsockopt(socket.SOL_IP, socket.IP_TTL, 63)
     return SockWrapper(outsock, outsock,
                        connect_to=(ip, port),
                        peername='%s:%d' % (ip, port))

tests/ssh/test_parse_hostport.py

@@ -0,0 +1,20 @@
+from sshuttle.ssh import parse_hostport
+
+
+def test_host_only():
+    assert parse_hostport("host") == (None, None, None, "host")
+    assert parse_hostport("1.2.3.4") == (None, None, None, "1.2.3.4")
+    assert parse_hostport("2001::1") == (None, None, None, "2001::1")
+    assert parse_hostport("[2001::1]") == (None, None, None, "2001::1")
+
+
+def test_host_and_port():
+    assert parse_hostport("host:22") == (None, None, 22, "host")
+    assert parse_hostport("1.2.3.4:22") == (None, None, 22, "1.2.3.4")
+    assert parse_hostport("[2001::1]:22") == (None, None, 22, "2001::1")
+
+
+def test_username_and_host():
+    assert parse_hostport("user@host") == ("user", None, None, "host")
+    assert parse_hostport("user:@host") == ("user", None, None, "host")
+    assert parse_hostport("user:pass@host") == ("user", "pass", None, "host")