Release version 1.0.5

Add an "is_supported()" function to the different methods so that each
method can include whatever logic they wish to indicate if they are
supported on a particular machine. Previously, methods/__init__.py
contained all of the logic for selecting individual methods. Now, it
iterates through a list of possible options and stops on the first
method that it finds that is_supported().

Currently, the decision is made based on the presence of programs in
the PATH. In the future, things such as the platform sshuttle is
running on could be considered.

.github/workflows/pythonpackage.yml

@@ -5,9 +5,11 @@ name: Python package
 
 on:
   push:
-    branches: [ master ]
+    branches: [ master, tproxy_mark_param ]
   pull_request:
-    branches: [ master ]
+    branches: [ master, tproxy_mark_param ]
+  workflow_dispatch:
+    branches: [ tproxy_mark_param ]
 
 jobs:
   build:
@@ -15,7 +17,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.5, 3.6, 3.7, 3.8]
+        python-version: [3.6, 3.7, 3.8, 3.9]
 
     steps:
     - uses: actions/checkout@v2

.gitignore

@@ -15,3 +15,4 @@
 /.redo
 /.pytest_cache/
 /.python-version
+.vscode/

CHANGES.rst

@@ -9,6 +9,53 @@ adheres to `Semantic Versioning`_.
 .. _`Semantic Versioning`: http://semver.org/
 
 
+1.0.5 - 2020-12-29
+------------------
+
+Added
+~~~~~
+* IPv6 support in nft method.
+* Intercept DNS requests sent by systemd-resolved.
+* Set default tmark.
+* Fix python2 server compatibility.
+* Python 3.9 support.
+
+Fixed
+~~~~~
+* Change license text to LGPL-2.1
+* Fix #494 sshuttle caught in infinite select() loop.
+* Include sshuttle version in verbose output.
+* Add psutil as dependency in setup.py
+* When subnets and excludes are specified with hostnames, use all IPs.
+* Update/document client's handling of IPv4 and IPv6.
+* Update sdnotify.py documentation.
+* Allow no remote to work.
+* Make prefixes in verbose output more consistent.
+* Make nat and nft rules consistent; improve rule ordering.
+* Make server and client handle resolv.conf differently.
+* Fix handling OSError in FirewallClient#__init__
+* Refactor automatic method selection.
+
+Removed
+~~~~~~~
+* Drop testing of Python 3.5
+
+
+1.0.4 - 2020-08-24
+------------------
+
+Fixed
+~~~~~
+* Allow Mux() flush/fill to work with python < 3.5
+* Fix parse_hostport to always return string for host.
+* Require -r/--remote parameter.
+* Add missing package in OpenWRT documentation.
+* Fix doc about --listen option.
+* README: add Ubuntu.
+* Increase IP4 ttl to 63 hops instead of 42.
+* Fix formatting in installation.rst
+
+
 1.0.3 - 2020-07-12
 ------------------
 

LICENSE

@@ -1,13 +1,14 @@
-                  GNU LIBRARY GENERAL PUBLIC LICENSE
+                  GNU LESSER GENERAL PUBLIC LICENSE
-                       Version 2, June 1991
+                       Version 2.1, February 1999
 
- Copyright (C) 1991 Free Software Foundation, Inc.
+ Copyright (C) 1991, 1999 Free Software Foundation, Inc.
  51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
  Everyone is permitted to copy and distribute verbatim copies
  of this license document, but changing it is not allowed.
 
-[This is the first released version of the library GPL.  It is
+[This is the first released version of the Lesser GPL.  It also counts
- numbered 2 because it goes with version 2 of the ordinary GPL.]
+ as the successor of the GNU Library Public License, version 2, hence
+ the version number 2.1.]
 
                             Preamble
 
@@ -16,97 +17,109 @@ freedom to share and change it.  By contrast, the GNU General Public
 Licenses are intended to guarantee your freedom to share and change
 free software--to make sure the software is free for all its users.
 
-  This license, the Library General Public License, applies to some
+  This license, the Lesser General Public License, applies to some
-specially designated Free Software Foundation software, and to any
+specially designated software packages--typically libraries--of the
-other libraries whose authors decide to use it.  You can use it for
+Free Software Foundation and other authors who decide to use it.  You
-your libraries, too.
+can use it too, but we suggest you first think carefully about whether
+this license or the ordinary General Public License is the better
+strategy to use in any particular case, based on the explanations below.
 
-  When we speak of free software, we are referring to freedom, not
+  When we speak of free software, we are referring to freedom of use,
-price.  Our General Public Licenses are designed to make sure that you
+not price.  Our General Public Licenses are designed to make sure that
-have the freedom to distribute copies of free software (and charge for
+you have the freedom to distribute copies of free software (and charge
-this service if you wish), that you receive source code or can get it
+for this service if you wish); that you receive source code or can get
-if you want it, that you can change the software or use pieces of it
+it if you want it; that you can change the software and use pieces of
-in new free programs; and that you know you can do these things.
+it in new free programs; and that you are informed that you can do
+these things.
 
   To protect your rights, we need to make restrictions that forbid
-anyone to deny you these rights or to ask you to surrender the rights.
+distributors to deny you these rights or to ask you to surrender these
-These restrictions translate to certain responsibilities for you if
+rights.  These restrictions translate to certain responsibilities for
-you distribute copies of the library, or if you modify it.
+you if you distribute copies of the library or if you modify it.
 
   For example, if you distribute copies of the library, whether gratis
 or for a fee, you must give the recipients all the rights that we gave
 you.  You must make sure that they, too, receive or can get the source
-code.  If you link a program with the library, you must provide
+code.  If you link other code with the library, you must provide
-complete object files to the recipients so that they can relink them
+complete object files to the recipients, so that they can relink them
-with the library, after making changes to the library and recompiling
+with the library after making changes to the library and recompiling
 it.  And you must show them these terms so they know their rights.
 
-  Our method of protecting your rights has two steps: (1) copyright
+  We protect your rights with a two-step method: (1) we copyright the
-the library, and (2) offer you this license which gives you legal
+library, and (2) we offer you this license, which gives you legal
 permission to copy, distribute and/or modify the library.
 
-  Also, for each distributor's protection, we want to make certain
+  To protect each distributor, we want to make it very clear that
-that everyone understands that there is no warranty for this free
+there is no warranty for the free library.  Also, if the library is
-library.  If the library is modified by someone else and passed on, we
+modified by someone else and passed on, the recipients should know
-want its recipients to know that what they have is not the original
+that what they have is not the original version, so that the original
-version, so that any problems introduced by others will not reflect on
+author's reputation will not be affected by problems that might be
-the original authors' reputations.
+introduced by others.
 
-  Finally, any free program is threatened constantly by software
+  Finally, software patents pose a constant threat to the existence of
-patents.  We wish to avoid the danger that companies distributing free
+any free program.  We wish to make sure that a company cannot
-software will individually obtain patent licenses, thus in effect
+effectively restrict the users of a free program by obtaining a
-transforming the program into proprietary software.  To prevent this,
+restrictive license from a patent holder.  Therefore, we insist that
-we have made it clear that any patent must be licensed for everyone's
+any patent license obtained for a version of the library must be
-free use or not licensed at all.
+consistent with the full freedom of use specified in this license.
 
-  Most GNU software, including some libraries, is covered by the ordinary
+  Most GNU software, including some libraries, is covered by the
-GNU General Public License, which was designed for utility programs.  This
+ordinary GNU General Public License.  This license, the GNU Lesser
-license, the GNU Library General Public License, applies to certain
+General Public License, applies to certain designated libraries, and
-designated libraries.  This license is quite different from the ordinary
+is quite different from the ordinary General Public License.  We use
-one; be sure to read it in full, and don't assume that anything in it is
+this license for certain libraries in order to permit linking those
-the same as in the ordinary license.
+libraries into non-free programs.
 
-  The reason we have a separate public license for some libraries is that
+  When a program is linked with a library, whether statically or using
-they blur the distinction we usually make between modifying or adding to a
+a shared library, the combination of the two is legally speaking a
-program and simply using it.  Linking a program with a library, without
+combined work, a derivative of the original library.  The ordinary
-changing the library, is in some sense simply using the library, and is
+General Public License therefore permits such linking only if the
-analogous to running a utility program or application program.  However, in
+entire combination fits its criteria of freedom.  The Lesser General
-a textual and legal sense, the linked executable is a combined work, a
+Public License permits more lax criteria for linking other code with
-derivative of the original library, and the ordinary General Public License
+the library.
-treats it as such.
 
-  Because of this blurred distinction, using the ordinary General
+  We call this license the "Lesser" General Public License because it
-Public License for libraries did not effectively promote software
+does Less to protect the user's freedom than the ordinary General
-sharing, because most developers did not use the libraries.  We
+Public License.  It also provides other free software developers Less
-concluded that weaker conditions might promote sharing better.
+of an advantage over competing non-free programs.  These disadvantages
+are the reason we use the ordinary General Public License for many
+libraries.  However, the Lesser license provides advantages in certain
+special circumstances.
 
-  However, unrestricted linking of non-free programs would deprive the
+  For example, on rare occasions, there may be a special need to
-users of those programs of all benefit from the free status of the
+encourage the widest possible use of a certain library, so that it becomes
-libraries themselves.  This Library General Public License is intended to
+a de-facto standard.  To achieve this, non-free programs must be
-permit developers of non-free programs to use free libraries, while
+allowed to use the library.  A more frequent case is that a free
-preserving your freedom as a user of such programs to change the free
+library does the same job as widely used non-free libraries.  In this
-libraries that are incorporated in them.  (We have not seen how to achieve
+case, there is little to gain by limiting the free library to free
-this as regards changes in header files, but we have achieved it as regards
+software only, so we use the Lesser General Public License.
-changes in the actual functions of the Library.)  The hope is that this
+
-will lead to faster development of free libraries.
+  In other cases, permission to use a particular library in non-free
+programs enables a greater number of people to use a large body of
+free software.  For example, permission to use the GNU C Library in
+non-free programs enables many more people to use the whole GNU
+operating system, as well as its variant, the GNU/Linux operating
+system.
+
+  Although the Lesser General Public License is Less protective of the
+users' freedom, it does ensure that the user of a program that is
+linked with the Library has the freedom and the wherewithal to run
+that program using a modified version of the Library.
 
   The precise terms and conditions for copying, distribution and
 modification follow.  Pay close attention to the difference between a
 "work based on the library" and a "work that uses the library".  The
-former contains code derived from the library, while the latter only
+former contains code derived from the library, whereas the latter must
-works together with the library.
+be combined with the library in order to run.
-
-  Note that it is possible for a library to be covered by the ordinary
-General Public License rather than by this special one.
 
-                  GNU LIBRARY GENERAL PUBLIC LICENSE
+                  GNU LESSER GENERAL PUBLIC LICENSE
    TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
 
-  0. This License Agreement applies to any software library which
+  0. This License Agreement applies to any software library or other
-contains a notice placed by the copyright holder or other authorized
+program which contains a notice placed by the copyright holder or
-party saying it may be distributed under the terms of this Library
+other authorized party saying it may be distributed under the terms of
-General Public License (also called "this License").  Each licensee is
+this Lesser General Public License (also called "this License").
-addressed as "you".
+Each licensee is addressed as "you".
 
   A "library" means a collection of software functions and/or data
 prepared so as to be conveniently linked with application programs
@@ -255,7 +268,7 @@ distribute the object code for the work under the terms of Section 6.
 Any executables containing that work also fall under Section 6,
 whether or not they are linked directly with the Library itself.
 
-  6. As an exception to the Sections above, you may also compile or
+  6. As an exception to the Sections above, you may also combine or
 link a "work that uses the Library" with the Library to produce a
 work containing portions of the Library, and distribute that work
 under terms of your choice, provided that the terms permit
@@ -282,23 +295,31 @@ of these things:
     Library will not necessarily be able to recompile the application
     to use the modified definitions.)
 
-    b) Accompany the work with a written offer, valid for at
+    b) Use a suitable shared library mechanism for linking with the
+    Library.  A suitable mechanism is one that (1) uses at run time a
+    copy of the library already present on the user's computer system,
+    rather than copying library functions into the executable, and (2)
+    will operate properly with a modified version of the library, if
+    the user installs one, as long as the modified version is
+    interface-compatible with the version that the work was made with.
+
+    c) Accompany the work with a written offer, valid for at
     least three years, to give the same user the materials
     specified in Subsection 6a, above, for a charge no more
     than the cost of performing this distribution.
 
-    c) If distribution of the work is made by offering access to copy
+    d) If distribution of the work is made by offering access to copy
     from a designated place, offer equivalent access to copy the above
     specified materials from the same place.
 
-    d) Verify that the user has already received a copy of these
+    e) Verify that the user has already received a copy of these
     materials or that you have already sent this user a copy.
 
   For an executable, the required form of the "work that uses the
 Library" must include any data and utility programs needed for
 reproducing the executable from it.  However, as a special exception,
-the source code distributed need not include anything that is normally
+the materials to be distributed need not include anything that is
-distributed (in either source or binary form) with the major
+normally distributed (in either source or binary form) with the major
 components (compiler, kernel, and so on) of the operating system on
 which the executable runs, unless that component itself accompanies
 the executable.
@@ -347,7 +368,7 @@ Library), the recipient automatically receives a license from the
 original licensor to copy, distribute, link with or modify the Library
 subject to these terms and conditions.  You may not impose any further
 restrictions on the recipients' exercise of the rights granted herein.
-You are not responsible for enforcing compliance by third parties to
+You are not responsible for enforcing compliance by third parties with
 this License.
 
   11. If, as a consequence of a court judgment or allegation of patent
@@ -390,7 +411,7 @@ excluded.  In such case, this License incorporates the limitation as if
 written in the body of this License.
 
   13. The Free Software Foundation may publish revised and/or new
-versions of the Library General Public License from time to time.
+versions of the Lesser General Public License from time to time.
 Such new versions will be similar in spirit to the present version,
 but may differ in detail to address new problems or concerns.
 
@@ -453,16 +474,16 @@ convey the exclusion of warranty; and each file should have at least the
     Copyright (C) <year>  <name of author>
 
     This library is free software; you can redistribute it and/or
-    modify it under the terms of the GNU Library General Public
+    modify it under the terms of the GNU Lesser General Public
     License as published by the Free Software Foundation; either
-    version 2 of the License, or (at your option) any later version.
+    version 2.1 of the License, or (at your option) any later version.
 
     This library is distributed in the hope that it will be useful,
     but WITHOUT ANY WARRANTY; without even the implied warranty of
     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-    Library General Public License for more details.
+    Lesser General Public License for more details.
 
-    You should have received a copy of the GNU Library General Public
+    You should have received a copy of the GNU Lesser General Public
     License along with this library; if not, write to the Free Software
     Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
 

README.rst

@@ -30,6 +30,10 @@ common case:
 Obtaining sshuttle
 ------------------
 
+- Ubuntu 16.04 or later::
+
+      apt-get install sshuttle
+
 - Debian stretch or later::
 
       apt-get install sshuttle

docs/installation.rst

@@ -6,6 +6,7 @@ Installation
       pip install sshuttle
 
 - Debain package manager::
+
       sudo apt install sshuttle
 
 - Clone::
@@ -18,5 +19,6 @@ Installation
 Optionally after installation
 -----------------------------
 
-- Add to sudoers file
+- Add to sudoers file::
+
       sshuttle --sudoers

docs/manpage.rst

@@ -11,7 +11,7 @@ Description
 -----------
 :program:`sshuttle` allows you to create a VPN connection from your
 machine to any remote server that you can connect to via
-ssh, as long as that server has python 3.5 or higher.
+ssh, as long as that server has python 3.6 or higher.
 
 To work, you must have root access on the local machine,
 but you can have a normal account on the server.
@@ -40,11 +40,15 @@ Options
     that has as the destination port 8000 of 1.2.3.4 and
     1.2.3.0/24:8000-9000 will tunnel traffic going to any port between
     8000 and 9000 (inclusive) for all IPs in the 1.2.3.0/24 subnet.
-    It is also possible to use a name in which case the first IP it resolves
+    A hostname can be provided instead of an IP address. If the
-    to during startup will be routed over the VPN. Valid examples are
+    hostname resolves to multiple IPs, all of the IPs are included.
-    example.com, example.com:8000 and example.com:8000-9000.
+    If a width is provided with a hostname that the width is applied
+    to all of the hostnames IPs (if they are all either IPv4 or IPv6).
+    Widths cannot be supplied to hostnames that resolve to both IPv4
+    and IPv6. Valid examples are example.com, example.com:8000,
+    example.com/24, example.com/24:8000 and example.com:8000-9000.
 
-.. option:: --method <auto|nat|nft|tproxy|pf>
+.. option:: --method <auto|nat|nft|tproxy|pf|ipfw>
 
    Which firewall method should sshuttle use? For auto, sshuttle attempts to
    guess the appropriate method depending on what it can find in PATH. The
@@ -64,8 +68,9 @@ Options
     You can use any name resolving to an IP address of the machine running
     :program:`sshuttle`, e.g. ``--listen localhost``.
 
-    For the tproxy and pf methods this can be an IPv6 address. Use this option 
+    For the nft, tproxy and pf methods this can be an IPv6 address. Use 
-    twice if required, to provide both IPv4 and IPv6 addresses.
+    this option with comma separated values if required, to provide both 
+    IPv4 and IPv6 addresses, e.g. ``--listen 127.0.0.1:0,[::1]:0``.
 
 .. option:: -H, --auto-hosts
 
@@ -91,14 +96,20 @@ Options
     are taken automatically from the server's routing
     table.
 
+    This feature does not detect IPv6 routes. Specify IPv6 subnets
+    manually. For example, specify the ``::/0`` subnet on the command
+    line to route all IPv6 traffic.
+
 .. option:: --dns
 
     Capture local DNS requests and forward to the remote DNS
     server. All queries to any of the local system's DNS
-    servers (/etc/resolv.conf) will be intercepted and
+    servers (/etc/resolv.conf and, if it exists, 
+    /run/systemd/resolve/resolv.conf) will be intercepted and
     resolved on the remote side of the tunnel instead, there
     using the DNS specified via the :option:`--to-ns` option,
-    if specified.
+    if specified. Only plain DNS traffic sent to these servers
+    on port 53 are captured.
 
 .. option:: --ns-hosts=<server1[,server2[,server3[...]]]>
 
@@ -121,9 +132,9 @@ Options
 
 .. option:: --python
 
-    Specify the name/path of the remote python interpreter.
+    Specify the name/path of the remote python interpreter. The
-    The default is just ``python``, which means to use the
+    default is to use ``python3`` (or ``python``, if ``python3``
-    default python interpreter on the remote system's PATH.
+    fails) in the remote system's PATH.
 
 .. option:: -r <[username@]sshserver[:port]>, --remote=<[username@]sshserver[:port]>
 
@@ -201,6 +212,11 @@ Options
     Automatically fork into the background after connecting
     to the remote server.  Implies :option:`--syslog`.
 
+.. option:: -s <file>, --subnets=<file>
+
+    Include the subnets specified in a file instead of on the
+    command line. One subnet per line.
+
 .. option:: --syslog
 
     after connecting, send all log messages to the
@@ -215,7 +231,8 @@ Options
 
 .. option:: --disable-ipv6
 
-    If using tproxy or pf methods, this will disable IPv6 support.
+    Disable IPv6 support for methods that support it (nft, tproxy, and
+    pf).
 
 .. option:: --firewall
 
@@ -257,6 +274,10 @@ Options
     Set the file name for the sudoers.d file to be added. Default is
     "sshuttle_auto". Only works with --sudoers.
 
+.. option:: -t, --tmark
+
+    Transproxy optional traffic mark with provided MARK value.
+
 .. option:: --version
 
     Print program version.

docs/openwrt.rst

@@ -3,6 +3,6 @@ OpenWRT
 
 Run::
 
-    opkg install python3 python3-pip iptables-mod-nat-extra iptables-mod-ipopt
+    opkg install python3 python3-pip iptables-mod-extra iptables-mod-nat-extra iptables-mod-ipopt
     python3 /usr/bin/pip3 install sshuttle
     sshuttle -l 0.0.0.0 -r <IP> -x 192.168.1.1 0/0

docs/requirements.rst

@@ -6,7 +6,7 @@ Client side Requirements
 
 - sudo, or root access on your client machine.
   (The server doesn't need admin access.)
-- Python 3.5 or greater.
+- Python 3.6 or greater.
 
 
 Linux with NAT method
@@ -20,6 +20,18 @@ Requires:
 
 * iptables DNAT, REDIRECT, and ttl modules.
 
+Linux with nft method
+~~~~~~~~~~~~~~~~~~~~~
+Supports
+
+* IPv4 TCP
+* IPv4 DNS
+* IPv6 TCP
+* IPv6 DNS
+
+Requires:
+
+* nftables
 
 Linux with TPROXY method
 ~~~~~~~~~~~~~~~~~~~~~~~~
@@ -58,7 +70,7 @@ cmd.exe with Administrator access. See :doc:`windows` for more information.
 Server side Requirements
 ------------------------
 
-- Python 3.5 or greater.
+- Python 3.6 or greater.
 
 
 Additional Suggested Software

docs/tproxy.rst

@@ -8,9 +8,11 @@ There are some things you need to consider for TPROXY to work:
   done once after booting up::
 
       ip route add local default dev lo table 100
-      ip rule add fwmark 1 lookup 100
+      ip rule add fwmark {TMARK} lookup 100
       ip -6 route add local default dev lo table 100
-      ip -6 rule add fwmark 1 lookup 100
+      ip -6 rule add fwmark {TMARK} lookup 100
+  
+  where {TMARK} is the identifier mark passed with -t or --tmark flag (default value is 1).
 
 - The ``--auto-nets`` feature does not detect IPv6 routes automatically. Add IPv6
   routes manually. e.g. by adding ``'::/0'`` to the end of the command line.

requirements-tests.txt

@@ -1,7 +1,7 @@
 -r requirements.txt
-attrs==19.3.0
+attrs==20.3.0
-pytest==5.4.3
+pytest==6.2.1
-pytest-cov==2.10.0
+pytest-cov==2.10.1
-mock==2.0.0
+mock==4.0.3
-flake8==3.8.3
+flake8==3.8.4
 pyflakes==2.2.0

requirements.txt

@@ -1 +1,2 @@
-setuptools-scm==4.1.2
+setuptools-scm==5.0.1
+psutil

setup.py

@@ -49,10 +49,10 @@ setup(
         "License :: OSI Approved :: "
             "GNU Lesser General Public License v2 or later (LGPLv2+)",
         "Operating System :: OS Independent",
-        "Programming Language :: Python :: 3.5",
         "Programming Language :: Python :: 3.6",
         "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.9",
         "Topic :: System :: Networking",
     ],
     scripts=['bin/sudoers-add'],
@@ -61,7 +61,10 @@ setup(
             'sshuttle = sshuttle.cmdline:main',
         ],
     },
-    python_requires='>=3.5',
+    python_requires='>=3.6',
+    install_requires=[
+        'psutil',
+    ],
     tests_require=[
         'pytest',
         'pytest-cov',

sshuttle/assembler.py

@@ -7,11 +7,14 @@ z = zlib.decompressobj()
 while 1:
     name = sys.stdin.readline().strip()
     if name:
+        # python2 compat: in python2 sys.stdin.readline().strip() -> str
+        #                 in python3 sys.stdin.readline().strip() -> bytes
+        # (see #481)
+        if sys.version_info >= (3, 0):
             name = name.decode("ASCII")
-
         nbytes = int(sys.stdin.readline())
         if verbosity >= 2:
-            sys.stderr.write('server: assembling %r (%d bytes)\n'
+            sys.stderr.write(' s: assembling %r (%d bytes)\n'
                              % (name, nbytes))
         content = z.decompress(sys.stdin.read(nbytes))
 

sshuttle/client.py

@@ -6,6 +6,7 @@ import subprocess as ssubprocess
 import os
 import sys
 import platform
+import psutil
 
 import sshuttle.helpers as helpers
 import sshuttle.ssnet as ssnet
@@ -14,8 +15,9 @@ import sshuttle.ssyslog as ssyslog
 import sshuttle.sdnotify as sdnotify
 from sshuttle.ssnet import SockWrapper, Handler, Proxy, Mux, MuxWrapper
 from sshuttle.helpers import log, debug1, debug2, debug3, Fatal, islocal, \
-    resolvconf_nameservers
+    resolvconf_nameservers, which
 from sshuttle.methods import get_method, Features
+from sshuttle import __version__
 try:
     from pwd import getpwnam
 except ImportError:
@@ -55,7 +57,7 @@ def check_daemon(pidfile):
         if e.errno == errno.ENOENT:
             return  # no pidfile, ok
         else:
-            raise Fatal("can't read %s: %s" % (_pidname, e))
+            raise Fatal("c : can't read %s: %s" % (_pidname, e))
     if not oldpid:
         os.unlink(_pidname)
         return  # invalid pidfile, ok
@@ -194,11 +196,19 @@ class FirewallClient:
                     ['--firewall'])
         if ssyslog._p:
             argvbase += ['--syslog']
-        # Default to sudo unless on OpenBSD in which case use built in `doas`
+
+        # Determine how to prefix the command in order to elevate privileges.
         if platform.platform().startswith('OpenBSD'):
-            elev_prefix = ['doas']
+            elev_prefix = ['doas']  # OpenBSD uses built in `doas`
         else:
             elev_prefix = ['sudo', '-p', '[local sudo] Password: ']
+
+        # Look for binary and switch to absolute path if we can find
+        # it.
+        path = which(elev_prefix[0])
+        if path:
+            elev_prefix[0] = path
+
         if sudo_pythonpath:
             elev_prefix += ['/usr/bin/env',
                             'PYTHONPATH=%s' % python_path]
@@ -213,7 +223,6 @@ class FirewallClient:
         def setup():
             # run in the child process
             s2.close()
-        e = None
         if os.getuid() == 0:
             argv_tries = argv_tries[-1:]  # last entry only
         for argv in argv_tries:
@@ -222,16 +231,13 @@ class FirewallClient:
                     sys.stderr.write('[local su] ')
                 self.p = ssubprocess.Popen(argv, stdout=s1, preexec_fn=setup)
                 # No env: Talking to `FirewallClient.start`, which has no i18n.
-                e = None
                 break
-            except OSError:
+            except OSError as e:
-                pass
+                log('Spawning firewall manager: %r\n' % argv)
+                raise Fatal(e)
         self.argv = argv
         s1.close()
         self.pfile = s2.makefile('rwb')
-        if e:
-            log('Spawning firewall manager: %r\n' % self.argv)
-            raise Fatal(e)
         line = self.pfile.readline()
         self.check()
         if line[0:5] != b'READY':
@@ -242,7 +248,7 @@ class FirewallClient:
 
     def setup(self, subnets_include, subnets_exclude, nslist,
               redirectport_v6, redirectport_v4, dnsport_v6, dnsport_v4, udp,
-              user):
+              user, tmark):
         self.subnets_include = subnets_include
         self.subnets_exclude = subnets_exclude
         self.nslist = nslist
@@ -252,6 +258,7 @@ class FirewallClient:
         self.dnsport_v4 = dnsport_v4
         self.udp = udp
         self.user = user
+        self.tmark = tmark
 
     def check(self):
         rv = self.p.poll()
@@ -299,8 +306,8 @@ class FirewallClient:
             raise Fatal('%r expected STARTED, got %r' % (self.argv, line))
 
     def sethostip(self, hostname, ip):
-        assert(not re.search(rb'[^-\w\.]', hostname))
+        assert(not re.search(br'[^-\w\.]', hostname))
-        assert(not re.search(rb'[^0-9.]', ip))
+        assert(not re.search(br'[^0-9.]', ip))
         self.pfile.write(b'HOST %s,%s\n' % (hostname, ip))
         self.pfile.flush()
 
@@ -417,7 +424,13 @@ def ondns(listener, method, mux, handlers):
     if t is None:
         return
     srcip, dstip, data = t
-    debug1('DNS request from %r to %r: %d bytes\n' % (srcip, dstip, len(data)))
+    # dstip is None if we are using a method where we can't determine
+    # the destination IP of the DNS request that we captured from the client.
+    if dstip is None:
+        debug1('DNS request from %r: %d bytes\n' % (srcip, len(data)))
+    else:
+        debug1('DNS request from %r to %r: %d bytes\n' %
+               (srcip, dstip, len(data)))
     chan = mux.next_channel()
     dnsreqs[chan] = now + 30
     mux.send(chan, ssnet.CMD_DNS_REQ, data)
@@ -431,17 +444,14 @@ def _main(tcp_listener, udp_listener, fw, ssh_cmd, remotename,
           dns_listener, seed_hosts, auto_hosts, auto_nets, daemon,
           to_nameserver):
 
+    helpers.logprefix = 'c : '
     debug1('Starting client with Python version %s\n'
            % platform.python_version())
 
     method = fw.method
 
     handlers = []
-    if helpers.verbose >= 1:
+    debug1('Connecting to server...\n')
-        helpers.logprefix = 'c : '
-    else:
-        helpers.logprefix = 'client: '
-    debug1('connecting to server...\n')
 
     try:
         (serverproc, serversock) = ssh.connect(
@@ -453,7 +463,7 @@ def _main(tcp_listener, udp_listener, fw, ssh_cmd, remotename,
                          auto_nets=auto_nets))
     except socket.error as e:
         if e.args[0] == errno.EPIPE:
-            raise Fatal("failed to establish ssh session (1)")
+            raise Fatal("c : failed to establish ssh session (1)")
         else:
             raise
     mux = Mux(serversock.makefile("rb"), serversock.makefile("wb"))
@@ -471,18 +481,18 @@ def _main(tcp_listener, udp_listener, fw, ssh_cmd, remotename,
         initstring = serversock.recv(len(expected))
     except socket.error as e:
         if e.args[0] == errno.ECONNRESET:
-            raise Fatal("failed to establish ssh session (2)")
+            raise Fatal("c : failed to establish ssh session (2)")
         else:
             raise
 
     rv = serverproc.poll()
     if rv:
-        raise Fatal('server died with error code %d' % rv)
+        raise Fatal('c : server died with error code %d' % rv)
 
     if initstring != expected:
-        raise Fatal('expected server init string %r; got %r'
+        raise Fatal('c : expected server init string %r; got %r'
                     % (expected, initstring))
-    log('Connected.\n')
+    log('Connected to server.\n')
     sys.stdout.flush()
     if daemon:
         daemonize()
@@ -541,11 +551,23 @@ def _main(tcp_listener, udp_listener, fw, ssh_cmd, remotename,
         debug1('seed_hosts: %r\n' % seed_hosts)
         mux.send(0, ssnet.CMD_HOST_REQ, str.encode('\n'.join(seed_hosts)))
 
-    while 1:
+    def check_ssh_alive():
+        if daemon:
+            # poll() won't tell us when process exited since the
+            # process is no longer our child (it returns 0 all the
+            # time).
+            if not psutil.pid_exists(serverproc.pid):
+                raise Fatal('ssh connection to server (pid %d) exited.' %
+                            serverproc.pid)
+        else:
             rv = serverproc.poll()
-        if rv:
+            # poll returns None if process hasn't exited.
-            raise Fatal('server died with error code %d' % rv)
+            if rv is not None:
+                raise Fatal('ssh connection to server (pid %d) exited '
+                            'with returncode %d' % (serverproc.pid, rv))
 
+    while 1:
+        check_ssh_alive()
         ssnet.runonce(handlers, mux)
         if latency_control:
             mux.check_fullness()
@@ -555,7 +577,12 @@ def main(listenip_v6, listenip_v4,
          ssh_cmd, remotename, python, latency_control, dns, nslist,
          method_name, seed_hosts, auto_hosts, auto_nets,
          subnets_include, subnets_exclude, daemon, to_nameserver, pidfile,
-         user, sudo_pythonpath):
+         user, sudo_pythonpath, tmark):
+
+    if not remotename:
+        print("WARNING: You must specify -r/--remote to securely route "
+              "traffic to a remote machine. Running without -r/--remote "
+              "is only recommended for testing.")
 
     if daemon:
         try:
@@ -563,35 +590,81 @@ def main(listenip_v6, listenip_v4,
         except Fatal as e:
             log("%s\n" % e)
             return 5
-    debug1('Starting sshuttle proxy.\n')
+    debug1('Starting sshuttle proxy (version %s).\n' % __version__)
+    helpers.logprefix = 'c : '
 
     fw = FirewallClient(method_name, sudo_pythonpath)
 
-    # Get family specific subnet lists
+    # If --dns is used, store the IP addresses that the client
+    # normally uses for DNS lookups in nslist. The firewall needs to
+    # redirect packets outgoing to this server to the remote host
+    # instead.
     if dns:
-        nslist += resolvconf_nameservers()
+        nslist += resolvconf_nameservers(True)
         if to_nameserver is not None:
             to_nameserver = "%s@%s" % tuple(to_nameserver[1:])
     else:
         # option doesn't make sense if we aren't proxying dns
+        if to_nameserver and len(to_nameserver) > 0:
+            print("WARNING: --to-ns option is ignored because --dns was not "
+                  "used.")
         to_nameserver = None
 
-    subnets = subnets_include + subnets_exclude  # we don't care here
+    # Get family specific subnet lists. Also, the user may not specify
-    subnets_v6 = [i for i in subnets if i[0] == socket.AF_INET6]
+    # any subnets if they use --auto-nets. In this case, our subnets
-    nslist_v6 = [i for i in nslist if i[0] == socket.AF_INET6]
+    # list will be empty and the forwarded subnets will be determined
-    subnets_v4 = [i for i in subnets if i[0] == socket.AF_INET]
+    # later by the server.
+    subnets_v4 = [i for i in subnets_include if i[0] == socket.AF_INET]
+    subnets_v6 = [i for i in subnets_include if i[0] == socket.AF_INET6]
     nslist_v4 = [i for i in nslist if i[0] == socket.AF_INET]
+    nslist_v6 = [i for i in nslist if i[0] == socket.AF_INET6]
 
-    # Check features available
+    # Get available features from the firewall method
     avail = fw.method.get_supported_features()
+
+    # A feature is "required" if the user supplies us parameters which
+    # implies that the feature is needed.
     required = Features()
 
+    # Select the default addresses to bind to / listen to.
+
+    # Assume IPv4 is always available and should always be enabled. If
+    # a method doesn't provide IPv4 support or if we wish to run
+    # ipv6-only, changes to this code are required.
+    assert avail.ipv4
+    required.ipv4 = True
+
+    # listenip_v4 contains user specified value or it is set to "auto".
+    if listenip_v4 == "auto":
+        listenip_v4 = ('127.0.0.1', 0)
+
+    # listenip_v6 is...
+    #    None when IPv6 is disabled.
+    #    "auto" when listen address is unspecified.
+    #    The user specified address if provided by user
+    if listenip_v6 is None:
+        debug1("IPv6 disabled by --disable-ipv6\n")
     if listenip_v6 == "auto":
         if avail.ipv6:
+            debug1("IPv6 enabled: Using default IPv6 listen address ::1\n")
             listenip_v6 = ('::1', 0)
         else:
+            debug1("IPv6 disabled since it isn't supported by method "
+                   "%s.\n" % fw.method.name)
             listenip_v6 = None
 
+    # Make final decision about enabling IPv6:
+    required.ipv6 = False
+    if listenip_v6:
+        required.ipv6 = True
+
+    # If we get here, it is possible that listenip_v6 was user
+    # specified but not supported by the current method.
+    if required.ipv6 and not avail.ipv6:
+        raise Fatal("An IPv6 listen address was supplied, but IPv6 is "
+                    "disabled at your request or is unsupported by the %s "
+                    "method." % fw.method.name)
+
     if user is not None:
         if getpwnam is None:
             raise Fatal("Routing by user not available on this system.")
@@ -599,38 +672,66 @@ def main(listenip_v6, listenip_v4,
             user = getpwnam(user).pw_uid
         except KeyError:
             raise Fatal("User %s does not exist." % user)
-
-    if fw.method.name != 'nat':
-        required.ipv6 = len(subnets_v6) > 0 or listenip_v6 is not None
-        required.ipv4 = len(subnets_v4) > 0 or listenip_v4 is not None
-    else:
-        required.ipv6 = None
-        required.ipv4 = None
-
-    required.udp = avail.udp
-    required.dns = len(nslist) > 0
     required.user = False if user is None else True
 
-    # if IPv6 not supported, ignore IPv6 DNS servers
+    if not required.ipv6 and len(subnets_v6) > 0:
-    if not required.ipv6:
+        print("WARNING: IPv6 subnets were ignored because IPv6 is disabled "
+              "in sshuttle.")
+        subnets_v6 = []
+        subnets_include = subnets_v4
+
+    required.udp = avail.udp  # automatically enable UDP if it is available
+    required.dns = len(nslist) > 0
+
+    # Remove DNS servers using IPv6.
+    if required.dns:
+        if not required.ipv6 and len(nslist_v6) > 0:
+            print("WARNING: Your system is configured to use an IPv6 DNS "
+                  "server but sshuttle is not using IPv6. Therefore DNS "
+                  "traffic your system sends to the IPv6 DNS server won't "
+                  "be redirected via sshuttle to the remote machine.")
             nslist_v6 = []
             nslist = nslist_v4
 
+        if len(nslist) == 0:
+            raise Fatal("Can't redirect DNS traffic since IPv6 is not "
+                        "enabled in sshuttle and all of the system DNS "
+                        "servers are IPv6.")
+
+    # If we aren't using IPv6, we can safely ignore excluded IPv6 subnets.
+    if not required.ipv6:
+        orig_len = len(subnets_exclude)
+        subnets_exclude = [i for i in subnets_exclude
+                           if i[0] == socket.AF_INET]
+        if len(subnets_exclude) < orig_len:
+            print("WARNING: Ignoring one or more excluded IPv6 subnets "
+                  "because IPv6 is not enabled.")
+
+    # This will print error messages if we required a feature that
+    # isn't available by the current method.
     fw.method.assert_features(required)
 
-    if required.ipv6 and listenip_v6 is None:
-        raise Fatal("IPv6 required but not listening.")
-
     # display features enabled
-    debug1("IPv6 enabled: %r\n" % required.ipv6)
+    def feature_status(label, enabled, available):
-    debug1("UDP enabled: %r\n" % required.udp)
+        msg = label + ": "
-    debug1("DNS enabled: %r\n" % required.dns)
+        if enabled:
-    debug1("User enabled: %r\n" % required.user)
+            msg += "on"
+        else:
+            msg += "off "
+            if available:
+                msg += "(available)"
+            else:
+                msg += "(not available with %s method)" % fw.method.name
+        debug1(msg + "\n")
 
-    # bind to required ports
+    debug1("Method: %s\n" % fw.method.name)
-    if listenip_v4 == "auto":
+    feature_status("IPv4", required.ipv4, avail.ipv4)
-        listenip_v4 = ('127.0.0.1', 0)
+    feature_status("IPv6", required.ipv6, avail.ipv6)
+    feature_status("UDP ", required.udp, avail.udp)
+    feature_status("DNS ", required.dns, avail.dns)
+    feature_status("User", required.user, avail.user)
 
+    # Exclude traffic destined to our listen addresses.
     if required.ipv4 and \
             not any(listenip_v4[0] == sex[1] for sex in subnets_v4):
         subnets_exclude.append((socket.AF_INET, listenip_v4[0], 32, 0, 0))
@@ -639,6 +740,25 @@ def main(listenip_v6, listenip_v4,
             not any(listenip_v6[0] == sex[1] for sex in subnets_v6):
         subnets_exclude.append((socket.AF_INET6, listenip_v6[0], 128, 0, 0))
 
+    # We don't print the IP+port of where we are listening here
+    # because we do that below when we have identified the ports to
+    # listen on.
+    debug1("Subnets to forward through remote host (type, IP, cidr mask "
+           "width, startPort, endPort):\n")
+    for i in subnets_include:
+        debug1("  "+str(i)+"\n")
+    if auto_nets:
+        debug1("NOTE: Additional subnets to forward may be added below by "
+               "--auto-nets.\n")
+    debug1("Subnets to exclude from forwarding:\n")
+    for i in subnets_exclude:
+        debug1("  "+str(i)+"\n")
+    if required.dns:
+        debug1("DNS requests normally directed at these servers will be "
+               "redirected to remote:\n")
+        for i in nslist:
+            debug1("  "+str(i)+"\n")
+
     if listenip_v6 and listenip_v6[1] and listenip_v4 and listenip_v4[1]:
         # if both ports given, no need to search for a spare port
         ports = [0, ]
@@ -654,9 +774,8 @@ def main(listenip_v6, listenip_v4,
     redirectport_v6 = 0
     redirectport_v4 = 0
     bound = False
-    debug2('Binding redirector:')
     for port in ports:
-        debug2(' %d' % port)
+        debug2('Trying to bind redirector on port %d\n' % port)
         tcp_listener = MultiListener()
 
         if required.udp:
@@ -698,7 +817,6 @@ def main(listenip_v6, listenip_v4,
             else:
                 raise e
 
-    debug2('\n')
     if not bound:
         assert(last_e)
         raise last_e
@@ -710,10 +828,9 @@ def main(listenip_v6, listenip_v4,
     bound = False
     if required.dns:
         # search for spare port for DNS
-        debug2('Binding DNS:')
         ports = range(12300, 9000, -1)
         for port in ports:
-            debug2(' %d' % port)
+            debug2('Trying to bind DNS redirector on port %d\n' % port)
             if port in used_ports:
                 continue
 
@@ -744,7 +861,7 @@ def main(listenip_v6, listenip_v4,
                     used_ports.append(port)
                 else:
                     raise e
-        debug2('\n')
+
         dns_listener.print_listening("DNS")
         if not bound:
             assert(last_e)
@@ -786,7 +903,7 @@ def main(listenip_v6, listenip_v4,
     # start the firewall
     fw.setup(subnets_include, subnets_exclude, nslist,
              redirectport_v6, redirectport_v4, dnsport_v6, dnsport_v4,
-             required.udp, user)
+             required.udp, user, tmark)
 
     # start the client process
     try:

sshuttle/cmdline.py

@@ -47,8 +47,16 @@ def main():
         elif opt.hostwatch:
             return hostwatch.hw_main(opt.subnets, opt.auto_hosts)
         else:
-            includes = opt.subnets + opt.subnets_file
+            # parse_subnetports() is used to create a list of includes
-            excludes = opt.exclude
+            # and excludes. It is called once for each parameter and
+            # returns a list of one or more items for each subnet (it
+            # can return more than one item when a hostname in the
+            # parameter resolves to multiple IP addresses. Here, we
+            # flatten these lists.
+            includes = [item for sublist in opt.subnets+opt.subnets_file
+                        for item in sublist]
+            excludes = [item for sublist in opt.exclude for item in sublist]
+
             if not includes and not opt.auto_nets:
                 parser.error('at least one subnet, subnet file, '
                              'or -N expected')
@@ -99,7 +107,8 @@ def main():
                                       opt.to_ns,
                                       opt.pidfile,
                                       opt.user,
-                                      opt.sudo_pythonpath)
+                                      opt.sudo_pythonpath,
+                                      opt.tmark)
 
             if return_code == 0:
                 log('Normal exit code, exiting...')

sshuttle/firewall.py

@@ -7,6 +7,7 @@ import platform
 import traceback
 
 import sshuttle.ssyslog as ssyslog
+import sshuttle.helpers as helpers
 from sshuttle.helpers import debug1, debug2, Fatal
 from sshuttle.methods import get_auto_method, get_method
 
@@ -47,14 +48,18 @@ def rewrite_etc_hosts(hostmap, port):
     os.rename(tmpname, HOSTSFILE)
 
 
-def restore_etc_hosts(port):
+def restore_etc_hosts(hostmap, port):
+    # Only restore if we added hosts to /etc/hosts previously.
+    if len(hostmap) > 0:
+        debug2('undoing /etc/hosts changes.\n')
         rewrite_etc_hosts({}, port)
 
 
 # Isolate function that needs to be replaced for tests
 def setup_daemon():
     if os.getuid() != 0:
-        raise Fatal('you must be root (or enable su/sudo) to set the firewall')
+        raise Fatal('fw: '
+                    'You must be root (or enable su/sudo) to set the firewall')
 
     # don't disappear if our controlling terminal or stdout/stderr
     # disappears; we still have to clean up.
@@ -96,8 +101,8 @@ def subnet_weight(s):
 def main(method_name, syslog):
     stdin, stdout = setup_daemon()
     hostmap = {}
-
+    helpers.logprefix = 'fw: '
-    debug1('firewall manager: Starting firewall with Python version %s\n'
+    debug1('Starting firewall with Python version %s\n'
            % platform.python_version())
 
     if method_name == "auto":
@@ -109,7 +114,12 @@ def main(method_name, syslog):
         ssyslog.start_syslog()
         ssyslog.stderr_to_syslog()
 
-    debug1('firewall manager: ready method name %s.\n' % method.name)
+    if not method.is_supported():
+        raise Fatal("The %s method is not supported on this machine. "
+                    "Check that the appropriate programs are in your "
+                    "PATH." % method_name)
+
+    debug1('ready method name %s.\n' % method.name)
     stdout.write('READY %s\n' % method.name)
     stdout.flush()
 
@@ -122,18 +132,18 @@ def main(method_name, syslog):
 
     subnets = []
     if line != 'ROUTES\n':
-        raise Fatal('firewall: expected ROUTES but got %r' % line)
+        raise Fatal('expected ROUTES but got %r' % line)
     while 1:
         line = stdin.readline(128)
         if not line:
-            raise Fatal('firewall: expected route but got %r' % line)
+            raise Fatal('fw: expected route but got %r' % line)
         elif line.startswith("NSLIST\n"):
             break
         try:
             (family, width, exclude, ip, fport, lport) = \
                     line.strip().split(',', 5)
         except BaseException:
-            raise Fatal('firewall: expected route or NSLIST but got %r' % line)
+            raise Fatal('fw: expected route or NSLIST but got %r' % line)
         subnets.append((
             int(family),
             int(width),
@@ -141,31 +151,31 @@ def main(method_name, syslog):
             ip,
             int(fport),
             int(lport)))
-    debug2('firewall manager: Got subnets: %r\n' % subnets)
+    debug2('Got subnets: %r\n' % subnets)
 
     nslist = []
     if line != 'NSLIST\n':
-        raise Fatal('firewall: expected NSLIST but got %r' % line)
+        raise Fatal('fw: expected NSLIST but got %r' % line)
     while 1:
         line = stdin.readline(128)
         if not line:
-            raise Fatal('firewall: expected nslist but got %r' % line)
+            raise Fatal('fw: expected nslist but got %r' % line)
         elif line.startswith("PORTS "):
             break
         try:
             (family, ip) = line.strip().split(',', 1)
         except BaseException:
-            raise Fatal('firewall: expected nslist or PORTS but got %r' % line)
+            raise Fatal('fw: expected nslist or PORTS but got %r' % line)
         nslist.append((int(family), ip))
-        debug2('firewall manager: Got partial nslist: %r\n' % nslist)
+        debug2('Got partial nslist: %r\n' % nslist)
-    debug2('firewall manager: Got nslist: %r\n' % nslist)
+    debug2('Got nslist: %r\n' % nslist)
 
     if not line.startswith('PORTS '):
-        raise Fatal('firewall: expected PORTS but got %r' % line)
+        raise Fatal('fw: expected PORTS but got %r' % line)
     _, _, ports = line.partition(" ")
     ports = ports.split(",")
     if len(ports) != 4:
-        raise Fatal('firewall: expected 4 ports but got %d' % len(ports))
+        raise Fatal('fw: expected 4 ports but got %d' % len(ports))
     port_v6 = int(ports[0])
     port_v4 = int(ports[1])
     dnsport_v6 = int(ports[2])
@@ -180,21 +190,21 @@ def main(method_name, syslog):
     assert(dnsport_v4 >= 0)
     assert(dnsport_v4 <= 65535)
 
-    debug2('firewall manager: Got ports: %d,%d,%d,%d\n'
+    debug2('Got ports: %d,%d,%d,%d\n'
            % (port_v6, port_v4, dnsport_v6, dnsport_v4))
 
     line = stdin.readline(128)
     if not line:
-        raise Fatal('firewall: expected GO but got %r' % line)
+        raise Fatal('fw: expected GO but got %r' % line)
     elif not line.startswith("GO "):
-        raise Fatal('firewall: expected GO but got %r' % line)
+        raise Fatal('fw: expected GO but got %r' % line)
 
     _, _, args = line.partition(" ")
     udp, user = args.strip().split(" ", 1)
     udp = bool(int(udp))
     if user == '-':
         user = None
-    debug2('firewall manager: Got udp: %r, user: %r\n' % (udp, user))
+    debug2('Got udp: %r, user: %r\n' % (udp, user))
 
     subnets_v6 = [i for i in subnets if i[0] == socket.AF_INET6]
     nslist_v6 = [i for i in nslist if i[0] == socket.AF_INET6]
@@ -202,17 +212,17 @@ def main(method_name, syslog):
     nslist_v4 = [i for i in nslist if i[0] == socket.AF_INET]
 
     try:
-        debug1('firewall manager: setting up.\n')
+        debug1('setting up.\n')
 
         if subnets_v6 or nslist_v6:
-            debug2('firewall manager: setting up IPv6.\n')
+            debug2('setting up IPv6.\n')
             method.setup_firewall(
                 port_v6, dnsport_v6, nslist_v6,
                 socket.AF_INET6, subnets_v6, udp,
                 user)
 
         if subnets_v4 or nslist_v4:
-            debug2('firewall manager: setting up IPv4.\n')
+            debug2('setting up IPv4.\n')
             method.setup_firewall(
                 port_v4, dnsport_v4, nslist_v4,
                 socket.AF_INET, subnets_v4, udp,
@@ -235,27 +245,26 @@ def main(method_name, syslog):
             if line.startswith('HOST '):
                 (name, ip) = line[5:].strip().split(',', 1)
                 hostmap[name] = ip
-                debug2('firewall manager: setting up /etc/hosts.\n')
+                debug2('setting up /etc/hosts.\n')
                 rewrite_etc_hosts(hostmap, port_v6 or port_v4)
             elif line:
                 if not method.firewall_command(line):
-                    raise Fatal('firewall: expected command, got %r' % line)
+                    raise Fatal('fw: expected command, got %r' % line)
             else:
                 break
     finally:
         try:
-            debug1('firewall manager: undoing changes.\n')
+            debug1('undoing changes.\n')
         except BaseException:
             debug2('An error occurred, ignoring it.')
 
         try:
             if subnets_v6 or nslist_v6:
-                debug2('firewall manager: undoing IPv6 changes.\n')
+                debug2('undoing IPv6 changes.\n')
                 method.restore_firewall(port_v6, socket.AF_INET6, udp, user)
         except BaseException:
             try:
-                debug1("firewall manager: "
+                debug1("Error trying to undo IPv6 firewall.\n")
-                       "Error trying to undo IPv6 firewall.\n")
                 for line in traceback.format_exc().splitlines():
                     debug1("---> %s\n" % line)
             except BaseException:
@@ -263,25 +272,23 @@ def main(method_name, syslog):
 
         try:
             if subnets_v4 or nslist_v4:
-                debug2('firewall manager: undoing IPv4 changes.\n')
+                debug2('undoing IPv4 changes.\n')
                 method.restore_firewall(port_v4, socket.AF_INET, udp, user)
         except BaseException:
             try:
-                debug1("firewall manager: "
+                debug1("Error trying to undo IPv4 firewall.\n")
-                       "Error trying to undo IPv4 firewall.\n")
                 for line in traceback.format_exc().splitlines():
-                    debug1("firewall manager: ---> %s\n" % line)
+                    debug1("---> %s\n" % line)
             except BaseException:
                 debug2('An error occurred, ignoring it.')
 
         try:
-            debug2('firewall manager: undoing /etc/hosts changes.\n')
+            # debug2() message printed in restore_etc_hosts() function.
-            restore_etc_hosts(port_v6 or port_v4)
+            restore_etc_hosts(hostmap, port_v6 or port_v4)
         except BaseException:
             try:
-                debug1("firewall manager: "
+                debug1("Error trying to undo /etc/hosts changes.\n")
-                       "Error trying to undo /etc/hosts changes.\n")
                 for line in traceback.format_exc().splitlines():
-                    debug1("firewall manager: ---> %s\n" % line)
+                    debug1("---> %s\n" % line)
             except BaseException:
                 debug2('An error occurred, ignoring it.')

sshuttle/helpers.py

@@ -1,6 +1,7 @@
 import sys
 import socket
 import errno
+import os
 
 logprefix = ''
 verbose = 0
@@ -48,17 +49,64 @@ class Fatal(Exception):
     pass
 
 
-def resolvconf_nameservers():
+def resolvconf_nameservers(systemd_resolved):
-    lines = []
+    """Retrieves a list of tuples (address type, address as a string) of
-    for line in open('/etc/resolv.conf'):
+    the DNS servers used by the system to resolve hostnames.
+
+    If parameter is False, DNS servers are retrieved from only
+    /etc/resolv.conf. This behavior makes sense for the sshuttle
+    server.
+
+    If parameter is True, we retrieve information from both
+    /etc/resolv.conf and /run/systemd/resolve/resolv.conf (if it
+    exists). This behavior makes sense for the sshuttle client.
+
+    """
+
+    # Historically, we just needed to read /etc/resolv.conf.
+    #
+    # If systemd-resolved is active, /etc/resolv.conf will point to
+    # localhost and the actual DNS servers that systemd-resolved uses
+    # are stored in /run/systemd/resolve/resolv.conf. For programs
+    # that use the localhost DNS server, having sshuttle read
+    # /etc/resolv.conf is sufficient. However, resolved provides other
+    # ways of resolving hostnames (such as via dbus) that may not
+    # route requests through localhost. So, we retrieve a list of DNS
+    # servers that resolved uses so we can intercept those as well.
+    #
+    # For more information about systemd-resolved, see:
+    # https://www.freedesktop.org/software/systemd/man/systemd-resolved.service.html
+    #
+    # On machines without systemd-resolved, we expect opening the
+    # second file will fail.
+    files = ['/etc/resolv.conf']
+    if systemd_resolved:
+        files += ['/run/systemd/resolve/resolv.conf']
+
+    nsservers = []
+    for f in files:
+        this_file_nsservers = []
+        try:
+            for line in open(f):
                 words = line.lower().split()
                 if len(words) >= 2 and words[0] == 'nameserver':
-            lines.append(family_ip_tuple(words[1]))
+                    this_file_nsservers.append(family_ip_tuple(words[1]))
-    return lines
+            debug2("Found DNS servers in %s: %s\n" %
+                   (f, [n[1] for n in this_file_nsservers]))
+            nsservers += this_file_nsservers
+        except OSError as e:
+            debug3("Failed to read %s when looking for DNS servers: %s\n" %
+                   (f, e.strerror))
+
+    return nsservers
 
 
-def resolvconf_random_nameserver():
+def resolvconf_random_nameserver(systemd_resolved):
-    lines = resolvconf_nameservers()
+    """Return a random nameserver selected from servers produced by
+    resolvconf_nameservers(). See documentation for
+    resolvconf_nameservers() for a description of the parameter.
+    """
+    lines = resolvconf_nameservers(systemd_resolved)
     if lines:
         if len(lines) > 1:
             # don't import this unless we really need it
@@ -99,3 +147,75 @@ def family_to_string(family):
         return "AF_INET"
     else:
         return str(family)
+
+
+def get_env():
+    """An environment for sshuttle subprocesses. See get_path()."""
+    env = {
+        'PATH': get_path(),
+        'LC_ALL': "C",
+    }
+    return env
+
+
+def get_path():
+    """Returns a string of paths separated by os.pathsep.
+
+    Users might not have all of the programs sshuttle needs in their
+    PATH variable (i.e., some programs might be in /sbin). Use PATH
+    and a hardcoded set of paths to search through. This function is
+    used by our which() and get_env() functions. If which() and the
+    subprocess environments differ, programs that which() finds might
+    not be found at run time (or vice versa).
+    """
+    path = []
+    if "PATH" in os.environ:
+        path += os.environ["PATH"].split(os.pathsep)
+    # Python default paths.
+    path += os.defpath.split(os.pathsep)
+    # /sbin, etc are not in os.defpath and may not be in PATH either.
+    # /bin/ and /usr/bin below are probably redundant.
+    path += ['/bin', '/usr/bin', '/sbin', '/usr/sbin']
+
+    # Remove duplicates. Not strictly necessary.
+    path_dedup = []
+    for i in path:
+        if i not in path_dedup:
+            path_dedup.append(i)
+
+    return os.pathsep.join(path_dedup)
+
+
+if sys.version_info >= (3, 3):
+    from shutil import which as _which
+else:
+    # Although sshuttle does not officially support older versions of
+    # Python, some still run the sshuttle server on remote machines
+    # with old versions of python.
+    def _which(file, mode=os.F_OK | os.X_OK, path=None):
+        if path is not None:
+            search_paths = path.split(os.pathsep)
+        elif "PATH" in os.environ:
+            search_paths = os.environ["PATH"].split(os.pathsep)
+        else:
+            search_paths = os.defpath.split(os.pathsep)
+
+        for p in search_paths:
+            filepath = os.path.join(p, file)
+            if os.path.exists(filepath) and os.access(filepath, mode):
+                return filepath
+        return None
+
+
+def which(file, mode=os.F_OK | os.X_OK):
+    """A wrapper around shutil.which() that searches a predictable set of
+    paths and is more verbose about what is happening. See get_path()
+    for more information.
+    """
+    path = get_path()
+    rv = _which(file, mode, path)
+    if rv:
+        debug2("which() found '%s' at %s\n" % (file, rv))
+    else:
+        debug2("which() could not find '%s' in %s\n" % (file, path))
+    return rv

sshuttle/hostwatch.py

@@ -9,7 +9,7 @@ import platform
 
 import subprocess as ssubprocess
 import sshuttle.helpers as helpers
-from sshuttle.helpers import log, debug1, debug2, debug3
+from sshuttle.helpers import log, debug1, debug2, debug3, get_env
 
 POLL_TIME = 60 * 15
 NETSTAT_POLL_TIME = 30
@@ -125,14 +125,10 @@ def _check_dns(hostname):
 
 def _check_netstat():
     debug2(' > netstat\n')
-    env = {
-        'PATH': os.environ['PATH'],
-        'LC_ALL': "C",
-    }
     argv = ['netstat', '-n']
     try:
         p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, stderr=null,
-                              env=env)
+                              env=get_env())
         content = p.stdout.read().decode("ASCII")
         p.wait()
     except OSError:
@@ -151,14 +147,10 @@ def _check_smb(hostname):
     if not _smb_ok:
         return
     debug2(' > smb: %s\n' % hostname)
-    env = {
-        'PATH': os.environ['PATH'],
-        'LC_ALL': "C",
-    }
     argv = ['smbclient', '-U', '%', '-L', hostname]
     try:
         p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, stderr=null,
-                              env=env)
+                              env=get_env())
         lines = p.stdout.readlines()
         p.wait()
     except OSError:
@@ -214,14 +206,10 @@ def _check_nmb(hostname, is_workgroup, is_master):
     if not _nmb_ok:
         return
     debug2(' > n%d%d: %s\n' % (is_workgroup, is_master, hostname))
-    env = {
-        'PATH': os.environ['PATH'],
-        'LC_ALL': "C",
-    }
     argv = ['nmblookup'] + ['-M'] * is_master + ['--', hostname]
     try:
         p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, stderr=null,
-                              env=env)
+                              env=get_env)
         lines = p.stdout.readlines()
         rv = p.wait()
     except OSError:

sshuttle/linux.py

@@ -1,14 +1,13 @@
-import os
 import socket
 import subprocess as ssubprocess
-from sshuttle.helpers import log, debug1, Fatal, family_to_string
+from sshuttle.helpers import log, debug1, Fatal, family_to_string, get_env
 
 
 def nonfatal(func, *args):
     try:
         func(*args)
     except Fatal as e:
-        log('error: %s\n' % e)
+        log('fw: error: %s\n' % e)
 
 
 def ipt_chain_exists(family, table, name):
@@ -19,17 +18,13 @@ def ipt_chain_exists(family, table, name):
     else:
         raise Exception('Unsupported family "%s"' % family_to_string(family))
     argv = [cmd, '-t', table, '-nL']
-    env = {
-        'PATH': os.environ['PATH'],
-        'LC_ALL': "C",
-    }
     try:
-        output = ssubprocess.check_output(argv, env=env)
+        output = ssubprocess.check_output(argv, env=get_env())
         for line in output.decode('ASCII').split('\n'):
             if line.startswith('Chain %s ' % name):
                 return True
     except ssubprocess.CalledProcessError as e:
-        raise Fatal('%r returned %d' % (argv, e.returncode))
+        raise Fatal('fw: %r returned %d' % (argv, e.returncode))
 
 
 def ipt(family, table, *args):
@@ -39,14 +34,10 @@ def ipt(family, table, *args):
         argv = ['iptables', '-t', table] + list(args)
     else:
         raise Exception('Unsupported family "%s"' % family_to_string(family))
-    debug1('>> %s\n' % ' '.join(argv))
+    debug1('%s\n' % ' '.join(argv))
-    env = {
+    rv = ssubprocess.call(argv, env=get_env())
-        'PATH': os.environ['PATH'],
-        'LC_ALL': "C",
-    }
-    rv = ssubprocess.call(argv, env=env)
     if rv:
-        raise Fatal('%r returned %d' % (argv, rv))
+        raise Fatal('fw: %r returned %d' % (argv, rv))
 
 
 def nft(family, table, action, *args):
@@ -54,14 +45,10 @@ def nft(family, table, action, *args):
         argv = ['nft', action, 'inet', table] + list(args)
     else:
         raise Exception('Unsupported family "%s"' % family_to_string(family))
-    debug1('>> %s\n' % ' '.join(argv))
+    debug1('%s\n' % ' '.join(argv))
-    env = {
+    rv = ssubprocess.call(argv, env=get_env())
-        'PATH': os.environ['PATH'],
-        'LC_ALL': "C",
-    }
-    rv = ssubprocess.call(argv, env=env)
     if rv:
-        raise Fatal('%r returned %d' % (argv, rv))
+        raise Fatal('fw: %r returned %d' % (argv, rv))
 
 
 _no_ttl_module = False
@@ -71,15 +58,15 @@ def ipt_ttl(family, *args):
     global _no_ttl_module
     if not _no_ttl_module:
         # we avoid infinite loops by generating server-side connections
-        # with ttl 42.  This makes the client side not recapture those
+        # with ttl 63.  This makes the client side not recapture those
         # connections, in case client == server.
         try:
-            argsplus = list(args) + ['-m', 'ttl', '!', '--ttl', '42']
+            argsplus = list(args)
             ipt(family, *argsplus)
         except Fatal:
             ipt(family, *args)
             # we only get here if the non-ttl attempt succeeds
-            log('sshuttle: warning: your iptables is missing '
+            log('fw: WARNING: your iptables is missing '
                 'the ttl module.\n')
             _no_ttl_module = True
     else:

sshuttle/methods/__init__.py

@@ -1,26 +1,35 @@
-import os
 import importlib
 import socket
 import struct
 import errno
+import ipaddress
 from sshuttle.helpers import Fatal, debug3
 
 
 def original_dst(sock):
+    ip = "0.0.0.0"
+    port = -1
     try:
+        family = sock.family
         SO_ORIGINAL_DST = 80
+
+        if family == socket.AF_INET:
             SOCKADDR_MIN = 16
             sockaddr_in = sock.getsockopt(socket.SOL_IP,
                                           SO_ORIGINAL_DST, SOCKADDR_MIN)
-        (proto, port, a, b, c, d) = struct.unpack('!HHBBBB', sockaddr_in[:8])
+            port, raw_ip = struct.unpack_from('!2xH4s', sockaddr_in[:8])
-        # FIXME: decoding is IPv4 only.
+            ip = str(ipaddress.IPv4Address(raw_ip))
-        assert(socket.htons(proto) == socket.AF_INET)
+        elif family == socket.AF_INET6:
-        ip = '%d.%d.%d.%d' % (a, b, c, d)
+            sockaddr_in = sock.getsockopt(41, SO_ORIGINAL_DST, 64)
-        return (ip, port)
+            port, raw_ip = struct.unpack_from("!2xH4x16s", sockaddr_in)
+            ip = str(ipaddress.IPv6Address(raw_ip))
+        else:
+            raise Fatal("fw: Unknown family type.")
     except socket.error as e:
         if e.args[0] == errno.ENOPROTOOPT:
             return sock.getsockname()
         raise
+    return (ip, port)
 
 
 class Features(object):
@@ -38,12 +47,19 @@ class BaseMethod(object):
     @staticmethod
     def get_supported_features():
         result = Features()
+        result.ipv4 = True
         result.ipv6 = False
         result.udp = False
         result.dns = True
         result.user = False
         return result
 
+    @staticmethod
+    def is_supported():
+        """Returns true if it appears that this method will work on this
+        machine."""
+        return False
+
     @staticmethod
     def get_tcp_dstip(sock):
         return original_dst(sock)
@@ -68,7 +84,7 @@ class BaseMethod(object):
 
     def assert_features(self, features):
         avail = self.get_supported_features()
-        for key in ["udp", "dns", "ipv6", "user"]:
+        for key in ["udp", "dns", "ipv6", "ipv4", "user"]:
             if getattr(features, key) and not getattr(avail, key):
                 raise Fatal(
                     "Feature %s not supported with method %s.\n" %
@@ -86,30 +102,21 @@ class BaseMethod(object):
         return False
 
 
-def _program_exists(name):
-    paths = (os.getenv('PATH') or os.defpath).split(os.pathsep)
-    for p in paths:
-        fn = '%s/%s' % (p, name)
-        if os.path.exists(fn):
-            return not os.path.isdir(fn) and os.access(fn, os.X_OK)
-
-
 def get_method(method_name):
     module = importlib.import_module("sshuttle.methods.%s" % method_name)
     return module.Method(method_name)
 
 
 def get_auto_method():
-    if _program_exists('iptables'):
+    debug3("Selecting a method automatically...\n")
-        method_name = "nat"
+    # Try these methods, in order:
-    elif _program_exists('nft'):
+    methods_to_try = ["nat", "nft", "pf", "ipfw"]
-        method_name = "nft"
+    for m in methods_to_try:
-    elif _program_exists('pfctl'):
+        method = get_method(m)
-        method_name = "pf"
+        if method.is_supported():
-    elif _program_exists('ipfw'):
+            debug3("Method '%s' was automatically selected.\n" % m)
-        method_name = "ipfw"
+            return method
-    else:
-        raise Fatal(
-            "can't find either iptables, nft or pfctl; check your PATH")
 
-    return get_method(method_name)
+    raise Fatal("Unable to automatically find a supported method. Check that "
+                "the appropriate programs are in your PATH. We tried "
+                "methods: %s" % str(methods_to_try))

sshuttle/methods/ipfw.py

@@ -1,8 +1,8 @@
 import os
 import subprocess as ssubprocess
 from sshuttle.methods import BaseMethod
-from sshuttle.helpers import log, debug1, debug3, \
+from sshuttle.helpers import log, debug1, debug2, debug3, \
-    Fatal, family_to_string
+    Fatal, family_to_string, get_env, which
 
 recvmsg = None
 try:
@@ -61,16 +61,12 @@ else:
 
 def ipfw_rule_exists(n):
     argv = ['ipfw', 'list']
-    env = {
+    p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, env=get_env())
-        'PATH': os.environ['PATH'],
-        'LC_ALL': "C",
-    }
-    p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, env=env)
 
     found = False
     for line in p.stdout:
         if line.startswith(b'%05d ' % n):
-            if not ('ipttl 42' in line or 'check-state' in line):
+            if not ('ipttl 63' in line or 'check-state' in line):
                 log('non-sshuttle ipfw rule: %r\n' % line.strip())
                 raise Fatal('non-sshuttle ipfw rule #%d already exists!' % n)
             found = True
@@ -85,11 +81,7 @@ _oldctls = {}
 
 def _fill_oldctls(prefix):
     argv = ['sysctl', prefix]
-    env = {
+    p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, env=get_env())
-        'PATH': os.environ['PATH'],
-        'LC_ALL': "C",
-    }
-    p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, env=env)
     for line in p.stdout:
         line = line.decode()
         assert(line[-1] == '\n')
@@ -105,7 +97,7 @@ def _fill_oldctls(prefix):
 def _sysctl_set(name, val):
     argv = ['sysctl', '-w', '%s=%s' % (name, val)]
     debug1('>> %s\n' % ' '.join(argv))
-    return ssubprocess.call(argv, stdout=open(os.devnull, 'w'))
+    return ssubprocess.call(argv, stdout=open(os.devnull, 'w'), env=get_env())
     # No env: No output. (Or error that won't be parsed.)
 
 
@@ -139,7 +131,7 @@ def sysctl_set(name, val, permanent=False):
 def ipfw(*args):
     argv = ['ipfw', '-q'] + list(args)
     debug1('>> %s\n' % ' '.join(argv))
-    rv = ssubprocess.call(argv)
+    rv = ssubprocess.call(argv, env=get_env())
     # No env: No output. (Or error that won't be parsed.)
     if rv:
         raise Fatal('%r returned %d' % (argv, rv))
@@ -148,7 +140,7 @@ def ipfw(*args):
 def ipfw_noexit(*args):
     argv = ['ipfw', '-q'] + list(args)
     debug1('>> %s\n' % ' '.join(argv))
-    ssubprocess.call(argv)
+    ssubprocess.call(argv, env=get_env())
     # No env: No output. (Or error that won't be parsed.)
 
 
@@ -185,7 +177,7 @@ class Method(BaseMethod):
         sender.setsockopt(socket.SOL_IP, IP_BINDANY, 1)
         sender.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
         sender.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
-        sender.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
+        sender.setsockopt(socket.SOL_IP, socket.IP_TTL, 63)
         sender.bind(srcip)
         sender.sendto(data, dstip)
         sender.close()
@@ -224,7 +216,7 @@ class Method(BaseMethod):
         ipfw('add', '1', 'fwd', '127.0.0.1,%d' % port,
              'tcp',
              'from', 'any', 'to', 'table(126)',
-             'not', 'ipttl', '42', 'keep-state', 'setup')
+             'not', 'ipttl', '63', 'keep-state', 'setup')
 
         ipfw_noexit('table', '124', 'flush')
         dnscount = 0
@@ -235,11 +227,11 @@ class Method(BaseMethod):
             ipfw('add', '1', 'fwd', '127.0.0.1,%d' % dnsport,
                  'udp',
                  'from', 'any', 'to', 'table(124)',
-                 'not', 'ipttl', '42')
+                 'not', 'ipttl', '63')
         ipfw('add', '1', 'allow',
              'udp',
              'from', 'any', 'to', 'any',
-             'ipttl', '42')
+             'ipttl', '63')
 
         if subnets:
             # create new subnet entries
@@ -261,3 +253,10 @@ class Method(BaseMethod):
         ipfw_noexit('table', '124', 'flush')
         ipfw_noexit('table', '125', 'flush')
         ipfw_noexit('table', '126', 'flush')
+
+    def is_supported(self):
+        if which("ipfw"):
+            return True
+        debug2("ipfw method not supported because 'ipfw' command is "
+               "missing.\n")
+        return False

sshuttle/methods/nat.py

@@ -1,6 +1,6 @@
 import socket
 from sshuttle.firewall import subnet_weight
-from sshuttle.helpers import family_to_string
+from sshuttle.helpers import family_to_string, which, debug2
 from sshuttle.linux import ipt, ipt_ttl, ipt_chain_exists, nonfatal
 from sshuttle.methods import BaseMethod
 
@@ -50,17 +50,24 @@ class Method(BaseMethod):
         _ipt('-I', 'OUTPUT', '1', *args)
         _ipt('-I', 'PREROUTING', '1', *args)
 
-        # Firstly we always skip all LOCAL addtrype address, i.e. avoid
+        # This TTL hack allows the client and server to run on the
-        # tunnelling the traffic designated to all local TCP/IP addresses.
+        # same host. The connections the sshuttle server makes will
+        # have TTL set to 63.
+        _ipt_ttl('-A', chain, '-j', 'RETURN',  '-m', 'ttl', '--ttl', '63')
+
+        # Redirect DNS traffic as requested. This includes routing traffic
+        # to localhost DNS servers through sshuttle.
+        for _, ip in [i for i in nslist if i[0] == family]:
+            _ipt('-A', chain, '-j', 'REDIRECT',
+                 '--dest', '%s/32' % ip,
+                 '-p', 'udp',
+                 '--dport', '53',
+                 '--to-ports', str(dnsport))
+
+        # Don't route any remaining local traffic through sshuttle.
         _ipt('-A', chain, '-j', 'RETURN',
              '-m', 'addrtype',
-             '--dst-type', 'LOCAL',
+             '--dst-type', 'LOCAL')
-             '!', '-p', 'udp')
-        # Skip LOCAL traffic if it's not DNS.
-        _ipt('-A', chain, '-j', 'RETURN',
-             '-m', 'addrtype',
-             '--dst-type', 'LOCAL',
-             '-p', 'udp', '!', '--dport', '53')
 
         # create new subnet entries.
         for _, swidth, sexclude, snet, fport, lport \
@@ -74,17 +81,10 @@ class Method(BaseMethod):
                      '--dest', '%s/%s' % (snet, swidth),
                      *tcp_ports)
             else:
-                _ipt_ttl('-A', chain, '-j', 'REDIRECT',
+                _ipt('-A', chain, '-j', 'REDIRECT',
                      '--dest', '%s/%s' % (snet, swidth),
                      *(tcp_ports + ('--to-ports', str(port))))
 
-        for _, ip in [i for i in nslist if i[0] == family]:
-            _ipt_ttl('-A', chain, '-j', 'REDIRECT',
-                     '--dest', '%s/32' % ip,
-                     '-p', 'udp',
-                     '--dport', '53',
-                     '--to-ports', str(dnsport))
-
     def restore_firewall(self, port, family, udp, user):
         # only ipv4 supported with NAT
         if family != socket.AF_INET:
@@ -124,3 +124,10 @@ class Method(BaseMethod):
         result = super(Method, self).get_supported_features()
         result.user = True
         return result
+
+    def is_supported(self):
+        if which("iptables"):
+            return True
+        debug2("nat method not supported because 'iptables' command "
+               "is missing.\n")
+        return False

sshuttle/methods/nft.py

@@ -2,6 +2,7 @@ import socket
 from sshuttle.firewall import subnet_weight
 from sshuttle.linux import nft, nonfatal
 from sshuttle.methods import BaseMethod
+from sshuttle.helpers import debug2, which
 
 
 class Method(BaseMethod):
@@ -16,7 +17,10 @@ class Method(BaseMethod):
         if udp:
             raise Exception("UDP not supported by nft")
 
-        table = 'sshuttle-%s' % port
+        if family == socket.AF_INET:
+            table = 'sshuttle-ipv4-%s' % port
+        if family == socket.AF_INET6:
+            table = 'sshuttle-ipv6-%s' % port
 
         def _nft(action, *args):
             return nft(family, table, action, *args)
@@ -34,43 +38,85 @@ class Method(BaseMethod):
         _nft('add rule', 'output jump %s' % chain)
         _nft('add rule', 'prerouting jump %s' % chain)
 
+        # setup_firewall() gets called separately for ipv4 and ipv6. Make sure
+        # we only handle the version that we expect to.
+        if family == socket.AF_INET:
+            _nft('add rule', chain, 'meta', 'nfproto', '!=', 'ipv4', 'return')
+        else:
+            _nft('add rule', chain, 'meta', 'nfproto', '!=', 'ipv6', 'return')
+
+        # This TTL hack allows the client and server to run on the
+        # same host. The connections the sshuttle server makes will
+        # have TTL set to 63.
+        if family == socket.AF_INET:
+            _nft('add rule', chain, 'ip ttl == 63 return')
+        elif family == socket.AF_INET6:
+            _nft('add rule', chain, 'ip6 hoplimit == 63 return')
+
+        # Strings to use below to simplify our code
+        if family == socket.AF_INET:
+            ip_version_l = 'ipv4'
+            ip_version = 'ip'
+        elif family == socket.AF_INET6:
+            ip_version_l = 'ipv6'
+            ip_version = 'ip6'
+
+        # Redirect DNS traffic as requested. This includes routing traffic
+        # to localhost DNS servers through sshuttle.
+        for _, ip in [i for i in nslist if i[0] == family]:
+            _nft('add rule', chain, ip_version,
+                 'daddr %s' % ip, 'udp dport 53',
+                 ('redirect to :' + str(dnsport)))
+
+        # Don't route any remaining local traffic through sshuttle
+        _nft('add rule', chain, 'fib daddr type local return')
+
         # create new subnet entries.
         for _, swidth, sexclude, snet, fport, lport \
                 in sorted(subnets, key=subnet_weight, reverse=True):
-            tcp_ports = ('ip', 'protocol', 'tcp')
+
+            # match using nfproto as described at
+            # https://superuser.com/questions/1560376/match-ipv6-protocol-using-nftables
             if fport and fport != lport:
-                tcp_ports = \
+                tcp_ports = ('meta', 'nfproto', ip_version_l, 'tcp',
-                    tcp_ports + \
+                             'dport', '{ %d-%d }' % (fport, lport))
-                    ('tcp', 'dport', '{ %d-%d }' % (fport, lport))
             elif fport and fport == lport:
-                tcp_ports = tcp_ports + ('tcp', 'dport', '%d' % (fport))
+                tcp_ports = ('meta', 'nfproto', ip_version_l, 'tcp',
+                             'dport', '%d' % (fport))
+            else:
+                tcp_ports = ('meta', 'nfproto', ip_version_l,
+                             'meta', 'l4proto', 'tcp')
 
             if sexclude:
                 _nft('add rule', chain, *(tcp_ports + (
-                     'ip daddr %s/%s' % (snet, swidth), 'return')))
+                     ip_version, 'daddr %s/%s' % (snet, swidth), 'return')))
             else:
                 _nft('add rule', chain, *(tcp_ports + (
-                     'ip daddr %s/%s' % (snet, swidth), 'ip ttl != 42',
+                    ip_version, 'daddr %s/%s' % (snet, swidth),
                     ('redirect to :' + str(port)))))
 
-        for _, ip in [i for i in nslist if i[0] == family]:
-            if family == socket.AF_INET:
-                _nft('add rule', chain, 'ip protocol udp ip daddr %s' % ip,
-                     'udp dport { 53 }', 'ip ttl != 42',
-                     ('redirect to :' + str(dnsport)))
-            elif family == socket.AF_INET6:
-                _nft('add rule', chain, 'ip6 protocol udp ip6 daddr %s' % ip,
-                     'udp dport { 53 }', 'ip ttl != 42',
-                     ('redirect to :' + str(dnsport)))
-
     def restore_firewall(self, port, family, udp, user):
         if udp:
             raise Exception("UDP not supported by nft method_name")
 
-        table = 'sshuttle-%s' % port
+        if family == socket.AF_INET:
+            table = 'sshuttle-ipv4-%s' % port
+        if family == socket.AF_INET6:
+            table = 'sshuttle-ipv6-%s' % port
 
         def _nft(action, *args):
             return nft(family, table, action, *args)
 
         # basic cleanup/setup of chains
         nonfatal(_nft, 'delete table', '')
+
+    def get_supported_features(self):
+        result = super(Method, self).get_supported_features()
+        result.ipv6 = True
+        return result
+
+    def is_supported(self):
+        if which("nft"):
+            return True
+        debug2("nft method not supported because 'nft' command is missing.\n")
+        return False

sshuttle/methods/pf.py

@@ -11,7 +11,8 @@ from fcntl import ioctl
 from ctypes import c_char, c_uint8, c_uint16, c_uint32, Union, Structure, \
     sizeof, addressof, memmove
 from sshuttle.firewall import subnet_weight
-from sshuttle.helpers import debug1, debug2, debug3, Fatal, family_to_string
+from sshuttle.helpers import debug1, debug2, debug3, Fatal, family_to_string, \
+    get_env, which
 from sshuttle.methods import BaseMethod
 
 
@@ -179,7 +180,7 @@ class FreeBsd(Generic):
         return freebsd
 
     def enable(self):
-        returncode = ssubprocess.call(['kldload', 'pf'])
+        returncode = ssubprocess.call(['kldload', 'pf'], env=get_env())
         # No env: No output.
         super(FreeBsd, self).enable()
         if returncode == 0:
@@ -189,7 +190,7 @@ class FreeBsd(Generic):
         super(FreeBsd, self).disable(anchor)
         if _pf_context['loaded_by_sshuttle'] and \
                 _pf_context['started_by_sshuttle'] == 0:
-            ssubprocess.call(['kldunload', 'pf'])
+            ssubprocess.call(['kldunload', 'pf'], env=get_env())
             # No env: No output.
 
     def add_anchors(self, anchor):
@@ -386,15 +387,10 @@ else:
 def pfctl(args, stdin=None):
     argv = ['pfctl'] + shlex.split(args)
     debug1('>> %s\n' % ' '.join(argv))
-
-    env = {
-        'PATH': os.environ['PATH'],
-        'LC_ALL': "C",
-    }
     p = ssubprocess.Popen(argv, stdin=ssubprocess.PIPE,
                           stdout=ssubprocess.PIPE,
                           stderr=ssubprocess.PIPE,
-                          env=env)
+                          env=get_env())
     o = p.communicate(stdin)
     if p.returncode:
         raise Fatal('%r returned %d' % (argv, p.returncode))
@@ -495,3 +491,9 @@ class Method(BaseMethod):
             return True
         else:
             return False
+
+    def is_supported(self):
+        if which("pfctl"):
+            return True
+        debug2("pf method not supported because 'pfctl' command is missing.\n")
+        return False

sshuttle/methods/tproxy.py

@@ -3,7 +3,7 @@ from sshuttle.firewall import subnet_weight
 from sshuttle.helpers import family_to_string
 from sshuttle.linux import ipt, ipt_ttl, ipt_chain_exists
 from sshuttle.methods import BaseMethod
-from sshuttle.helpers import debug1, debug3, Fatal
+from sshuttle.helpers import debug1, debug2, debug3, Fatal, which
 
 recvmsg = None
 try:
@@ -152,6 +152,16 @@ class Method(BaseMethod):
 
     def setup_firewall(self, port, dnsport, nslist, family, subnets, udp,
                        user):
+        if self.firewall is None:
+            tmark = '1'
+        else:
+            tmark = self.firewall.tmark
+
+        self.setup_firewall_tproxy(port, dnsport, nslist, family, subnets, udp,
+                                   user, tmark)
+
+    def setup_firewall_tproxy(self, port, dnsport, nslist, family, subnets,
+                              udp, user, tmark):
         if family not in [socket.AF_INET, socket.AF_INET6]:
             raise Exception(
                 'Address family "%s" unsupported by tproxy method'
@@ -182,9 +192,9 @@ class Method(BaseMethod):
         _ipt('-F', divert_chain)
         _ipt('-N', tproxy_chain)
         _ipt('-F', tproxy_chain)
-        _ipt('-I', 'OUTPUT', '1', '-j', mark_chain)
+        _ipt('-I', 'OUTPUT', tmark, '-j', mark_chain)
-        _ipt('-I', 'PREROUTING', '1', '-j', tproxy_chain)
+        _ipt('-I', 'PREROUTING', tmark, '-j', tproxy_chain)
-        _ipt('-A', divert_chain, '-j', 'MARK', '--set-mark', '1')
+        _ipt('-A', divert_chain, '-j', 'MARK', '--set-mark', tmark)
         _ipt('-A', divert_chain, '-j', 'ACCEPT')
         _ipt('-A', tproxy_chain, '-m', 'socket', '-j', divert_chain,
              '-m', 'tcp', '-p', 'tcp')
@@ -194,11 +204,11 @@ class Method(BaseMethod):
                  '-m', 'udp', '-p', 'udp')
 
         for _, ip in [i for i in nslist if i[0] == family]:
-            _ipt('-A', mark_chain, '-j', 'MARK', '--set-mark', '1',
+            _ipt('-A', mark_chain, '-j', 'MARK', '--set-mark', tmark,
                  '--dest', '%s/32' % ip,
                  '-m', 'udp', '-p', 'udp', '--dport', '53')
             _ipt('-A', tproxy_chain, '-j', 'TPROXY',
-                 '--tproxy-mark', '0x1/0x1',
+                 '--tproxy-mark', '0x'+tmark+'/0x'+tmark,
                  '--dest', '%s/32' % ip,
                  '-m', 'udp', '-p', 'udp', '--dport', '53',
                  '--on-port', str(dnsport))
@@ -218,12 +228,12 @@ class Method(BaseMethod):
                      '-m', 'tcp',
                      *tcp_ports)
             else:
-                _ipt('-A', mark_chain, '-j', 'MARK', '--set-mark', '1',
+                _ipt('-A', mark_chain, '-j', 'MARK', '--set-mark', tmark,
                      '--dest', '%s/%s' % (snet, swidth),
                      '-m', 'tcp',
                      *tcp_ports)
                 _ipt('-A', tproxy_chain, '-j', 'TPROXY',
-                     '--tproxy-mark', '0x1/0x1',
+                     '--tproxy-mark', '0x'+tmark+'/0x'+tmark,
                      '--dest', '%s/%s' % (snet, swidth),
                      '-m', 'tcp',
                      *(tcp_ports + ('--on-port', str(port))))
@@ -242,12 +252,12 @@ class Method(BaseMethod):
                          '-m', 'udp',
                          *udp_ports)
                 else:
-                    _ipt('-A', mark_chain, '-j', 'MARK', '--set-mark', '1',
+                    _ipt('-A', mark_chain, '-j', 'MARK', '--set-mark', tmark,
                          '--dest', '%s/%s' % (snet, swidth),
                          '-m', 'udp',
                          *udp_ports)
                     _ipt('-A', tproxy_chain, '-j', 'TPROXY',
-                         '--tproxy-mark', '0x1/0x1',
+                         '--tproxy-mark', '0x'+tmark+'/0x'+tmark,
                          '--dest', '%s/%s' % (snet, swidth),
                          '-m', 'udp',
                          *(udp_ports + ('--on-port', str(port))))
@@ -284,3 +294,10 @@ class Method(BaseMethod):
         if ipt_chain_exists(family, table, divert_chain):
             _ipt('-F', divert_chain)
             _ipt('-X', divert_chain)
+
+    def is_supported(self):
+        if which("iptables") and which("ip6tables"):
+            return True
+        debug2("tproxy method not supported because 'iptables' "
+               "or 'ip6tables' commands are missing.\n")
+        return False

sshuttle/options.py

@@ -28,7 +28,14 @@ def parse_subnetport_file(s):
 # 1.2.3.4/5:678, 1.2.3.4:567, 1.2.3.4/16 or just 1.2.3.4
 # [1:2::3/64]:456, [1:2::3]:456, 1:2::3/64 or just 1:2::3
 # example.com:123 or just example.com
+#
+# In addition, the port number can be specified as a range:
+# 1.2.3.4:8000-8080.
+#
+# Can return multiple matches if the domain name used in the request
+# has multiple IP addresses.
 def parse_subnetport(s):
+
     if s.count(':') > 1:
         rx = r'(?:\[?([\w\:]+)(?:/(\d+))?]?)(?::(\d+)(?:-(\d+))?)?$'
     else:
@@ -38,19 +45,56 @@ def parse_subnetport(s):
     if not m:
         raise Fatal('%r is not a valid address/mask:port format' % s)
 
-    addr, width, fport, lport = m.groups()
+    # Ports range from fport to lport. If only one port is specified,
+    # fport is defined and lport is None.
+    #
+    # cidr is the mask defined with the slash notation
+    host, cidr, fport, lport = m.groups()
     try:
-        addrinfo = socket.getaddrinfo(addr, 0, 0, socket.SOCK_STREAM)
+        addrinfo = socket.getaddrinfo(host, 0, 0, socket.SOCK_STREAM)
     except socket.gaierror:
-        raise Fatal('Unable to resolve address: %s' % addr)
+        raise Fatal('Unable to resolve address: %s' % host)
 
-    family, _, _, _, addr = min(addrinfo)
+    # If the address is a domain with multiple IPs and a mask is also
-    max_width = 32 if family == socket.AF_INET else 128
+    # provided, proceed cautiously:
-    width = int(width or max_width)
+    if cidr is not None:
-    if not 0 <= width <= max_width:
+        addr_v6 = [a for a in addrinfo if a[0] == socket.AF_INET6]
-        raise Fatal('width %d is not between 0 and %d' % (width, max_width))
+        addr_v4 = [a for a in addrinfo if a[0] == socket.AF_INET]
 
-    return (family, addr[0], width, int(fport or 0), int(lport or fport or 0))
+        # Refuse to proceed if IPv4 and IPv6 addresses are present:
+        if len(addr_v6) > 0 and len(addr_v4) > 0:
+            raise Fatal("%s has IPv4 and IPv6 addresses, so the mask "
+                        "of /%s is not supported. Specify the IP "
+                        "addresses directly if you wish to specify "
+                        "a mask." % (host, cidr))
+
+        # Warn if a domain has multiple IPs of the same type (IPv4 vs
+        # IPv6) and the mask is applied to all of the IPs.
+        if len(addr_v4) > 1 or len(addr_v6) > 1:
+            print("WARNING: %s has multiple IP addresses. The "
+                  "mask of /%s is applied to all of the addresses."
+                  % (host, cidr))
+
+    rv = []
+    for a in addrinfo:
+        family, _, _, _, addr = a
+
+        # Largest possible slash value we can use with this IP:
+        max_cidr = 32 if family == socket.AF_INET else 128
+
+        if cidr is None:  # if no mask, use largest mask
+            cidr_to_use = max_cidr
+        else:   # verify user-provided mask is appropriate
+            cidr_to_use = int(cidr)
+            if not 0 <= cidr_to_use <= max_cidr:
+                raise Fatal('Slash in CIDR notation (/%d) is '
+                            'not between 0 and %d'
+                            % (cidr_to_use, max_cidr))
+
+        rv.append((family, addr[0], cidr_to_use,
+                   int(fport or 0), int(lport or fport or 0)))
+
+    return rv
 
 
 # 1.2.3.4:567 or just 1.2.3.4 or just 567
@@ -69,16 +113,21 @@ def parse_ipport(s):
     if not m:
         raise Fatal('%r is not a valid IP:port format' % s)
 
-    ip, port = m.groups()
+    host, port = m.groups()
-    ip = ip or '0.0.0.0'
+    host = host or '0.0.0.0'
     port = int(port or 0)
 
     try:
-        addrinfo = socket.getaddrinfo(ip, port, 0, socket.SOCK_STREAM)
+        addrinfo = socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM)
     except socket.gaierror:
-        raise Fatal('%r is not a valid IP:port format' % s)
+        raise Fatal('Unable to resolve address: %s' % host)
+
+    if len(addrinfo) > 1:
+        print("WARNING: Host %s has more than one IP, only using one of them."
+              % host)
 
     family, _, _, _, addr = min(addrinfo)
+    # Note: addr contains (ip, port)
     return (family,) + addr[:2]
 
 
@@ -360,3 +409,11 @@ parser.add_argument(
     do not set PYTHONPATH when invoking sudo
     """
 )
+parser.add_argument(
+    "-t", "--tmark",
+    metavar="[MARK]",
+    default="1",
+    help="""
+    transproxy optional traffic mark with provided MARK value
+    """
+)

sshuttle/sdnotify.py

@@ -1,3 +1,13 @@
+"""When sshuttle is run via a systemd service file, we can communicate
+to systemd about the status of the sshuttle process. In particular, we
+can send READY status to tell systemd that sshuttle has completed
+startup and send STOPPING to indicate that sshuttle is beginning
+shutdown.
+
+For details, see:
+https://www.freedesktop.org/software/systemd/man/sd_notify.html
+"""
+
 import socket
 import os
 
@@ -5,6 +15,7 @@ from sshuttle.helpers import debug1
 
 
 def _notify(message):
+    """Send a notification message to systemd."""
     addr = os.environ.get("NOTIFY_SOCKET", None)
 
     if not addr or len(addr) == 1 or addr[0] not in ('/', '@'):
@@ -31,16 +42,22 @@ def _notify(message):
 
 
 def send(*messages):
+    """Send multiple messages to systemd."""
     return _notify(b'\n'.join(messages))
 
 
 def ready():
+    """Constructs a message that is appropriate to send upon completion of
+sshuttle startup."""
     return b"READY=1"
 
 
 def stop():
+    """Constructs a message that is appropriate to send when sshuttle is
+beginning to shutdown."""
     return b"STOPPING=1"
 
 
 def status(message):
+    """Constructs a status message to be sent to systemd."""
     return b"STATUS=%s" % message.encode('utf8')

sshuttle/server.py

@@ -6,7 +6,7 @@ import time
 import sys
 import os
 import platform
-from shutil import which
+
 
 import sshuttle.ssnet as ssnet
 import sshuttle.helpers as helpers
@@ -14,7 +14,7 @@ import sshuttle.hostwatch as hostwatch
 import subprocess as ssubprocess
 from sshuttle.ssnet import Handler, Proxy, Mux, MuxWrapper
 from sshuttle.helpers import b, log, debug1, debug2, debug3, Fatal, \
-    resolvconf_random_nameserver
+    resolvconf_random_nameserver, which, get_env
 
 
 def _ipmatch(ipstr):
@@ -82,11 +82,7 @@ def _route_iproute(line):
 
 def _list_routes(argv, extract_route):
     # FIXME: IPv4 only
-    env = {
+    p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, env=get_env())
-        'PATH': os.environ['PATH'],
-        'LC_ALL': "C",
-    }
-    p = ssubprocess.Popen(argv, stdout=ssubprocess.PIPE, env=env)
     routes = []
     for line in p.stdout:
         if not line.strip():
@@ -101,7 +97,6 @@ def _list_routes(argv, extract_route):
     rv = p.wait()
     if rv != 0:
         log('WARNING: %r returned %d\n' % (argv, rv))
-        log('WARNING: That prevents --auto-nets from working.\n')
 
     return routes
 
@@ -112,7 +107,8 @@ def list_routes():
     elif which('netstat'):
         routes = _list_routes(['netstat', '-rn'], _route_netstat)
     else:
-        log('WARNING: Neither ip nor netstat were found on the server.\n')
+        log('WARNING: Neither "ip" nor "netstat" were found on the server. '
+            '--auto-nets feature will not work.\n')
         routes = []
 
     for (family, ip, width) in routes:
@@ -187,7 +183,7 @@ class DnsProxy(Handler):
         self.tries += 1
 
         if self.to_nameserver is None:
-            _, peer = resolvconf_random_nameserver()
+            _, peer = resolvconf_random_nameserver(False)
             port = 53
         else:
             peer = self.to_ns_peer
@@ -195,7 +191,7 @@ class DnsProxy(Handler):
 
         family, sockaddr = self._addrinfo(peer, port)
         sock = socket.socket(family, socket.SOCK_DGRAM)
-        sock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
+        sock.setsockopt(socket.SOL_IP, socket.IP_TTL, 63)
         sock.connect(sockaddr)
 
         self.peers[sock] = peer
@@ -252,15 +248,15 @@ class UdpProxy(Handler):
         self.chan = chan
         self.sock = sock
         if family == socket.AF_INET:
-            self.sock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
+            self.sock.setsockopt(socket.SOL_IP, socket.IP_TTL, 63)
 
     def send(self, dstip, data):
-        debug2('UDP: sending to %r port %d\n' % dstip)
+        debug2(' s: UDP: sending to %r port %d\n' % dstip)
         try:
             self.sock.sendto(data, dstip)
         except socket.error:
             _, e = sys.exc_info()[:2]
-            log('UDP send to %r port %d: %s\n' % (dstip[0], dstip[1], e))
+            log(' s: UDP send to %r port %d: %s\n' % (dstip[0], dstip[1], e))
             return
 
     def callback(self, sock):
@@ -268,21 +264,18 @@ class UdpProxy(Handler):
             data, peer = sock.recvfrom(4096)
         except socket.error:
             _, e = sys.exc_info()[:2]
-            log('UDP recv from %r port %d: %s\n' % (peer[0], peer[1], e))
+            log(' s: UDP recv from %r port %d: %s\n' % (peer[0], peer[1], e))
             return
-        debug2('UDP response: %d bytes\n' % len(data))
+        debug2(' s: UDP response: %d bytes\n' % len(data))
         hdr = b("%s,%r," % (peer[0], peer[1]))
         self.mux.send(self.chan, ssnet.CMD_UDP_DATA, hdr + data)
 
 
 def main(latency_control, auto_hosts, to_nameserver, auto_nets):
-    debug1('Starting server with Python version %s\n'
+    debug1(' s: Starting server with Python version %s\n'
            % platform.python_version())
 
-    if helpers.verbose >= 1:
     helpers.logprefix = ' s: '
-    else:
-        helpers.logprefix = 'server: '
     debug1('latency control setting = %r\n' % latency_control)
 
     # synchronization header
@@ -323,7 +316,7 @@ def main(latency_control, auto_hosts, to_nameserver, auto_nets):
                 hw.leftover = b('')
             mux.send(0, ssnet.CMD_HOST_LIST, b('\n').join(lines))
         else:
-            raise Fatal('hostwatch process died')
+            raise Fatal(' s: hostwatch process died')
 
     def got_host_req(data):
         if not hw.pid:
@@ -377,7 +370,7 @@ def main(latency_control, auto_hosts, to_nameserver, auto_nets):
         family = int(data)
         mux.channels[channel] = lambda cmd, data: udp_req(channel, cmd, data)
         if channel in udphandlers:
-            raise Fatal('UDP connection channel %d already open' % channel)
+            raise Fatal(' s: UDP connection channel %d already open' % channel)
         else:
             h = UdpProxy(mux, channel, family)
             handlers.append(h)

sshuttle/ssh.py

@@ -12,7 +12,7 @@ import ipaddress
 from urllib.parse import urlparse
 
 import sshuttle.helpers as helpers
-from sshuttle.helpers import debug2
+from sshuttle.helpers import debug2, which, get_path, Fatal
 
 
 def get_module_source(name):
@@ -43,6 +43,8 @@ def parse_hostport(rhostport):
     """
     # leave use of default port to ssh command to prevent overwriting
     # ports configured in ~/.ssh/config when no port is given
+    if rhostport is None or len(rhostport) == 0:
+        return None, None, None, None
     port = None
     username = None
     password = None
@@ -65,12 +67,12 @@ def parse_hostport(rhostport):
         try:
             # try to parse host as an IP adress,
             # if that works it is an IPv6 address
-            host = ipaddress.ip_address(host)
+            host = str(ipaddress.ip_address(host))
         except ValueError:
             # if that fails parse as URL to get the port
             parsed = urlparse('//{}'.format(host))
             try:
-                host = ipaddress.ip_address(parsed.hostname)
+                host = str(ipaddress.ip_address(parsed.hostname))
             except ValueError:
                 # else if both fails, we have a hostname with port
                 host = parsed.hostname
@@ -139,6 +141,17 @@ def connect(ssh_cmd, rhostport, python, stderr, options):
             argv = (sshl +
                     portl +
                     [rhost, '--', pycmd])
+
+    # Our which() function searches for programs in get_path()
+    # directories (which include PATH). This step isn't strictly
+    # necessary if ssh is already in the user's PATH, but it makes the
+    # error message friendlier if the user incorrectly passes in a
+    # custom ssh command that we cannot find.
+    abs_path = which(argv[0])
+    if abs_path is None:
+        raise Fatal("Failed to find '%s' in path %s" % (argv[0], get_path()))
+    argv[0] = abs_path
+
     (s1, s2) = socket.socketpair()
 
     def setup():
@@ -146,6 +159,7 @@ def connect(ssh_cmd, rhostport, python, stderr, options):
         s2.close()
     s1a, s1b = os.dup(s1.fileno()), os.dup(s1.fileno())
     s1.close()
+
     debug2('executing: %r\n' % argv)
     p = ssubprocess.Popen(argv, stdin=s1a, stdout=s1b, preexec_fn=setup,
                           close_fds=True, stderr=stderr)

sshuttle/ssnet.py

@@ -4,6 +4,7 @@ import socket
 import errno
 import select
 import os
+import fcntl
 
 from sshuttle.helpers import b, log, debug1, debug2, debug3, Fatal
 
@@ -436,7 +437,13 @@ class Mux(Handler):
                 callback(cmd, data)
 
     def flush(self):
+        try:
             os.set_blocking(self.wfile.fileno(), False)
+        except AttributeError:
+            # python < 3.5
+            flags = fcntl.fcntl(self.wfile.fileno(), fcntl.F_GETFL)
+            flags |= os.O_NONBLOCK
+            flags = fcntl.fcntl(self.wfile.fileno(), fcntl.F_SETFL, flags)
         if self.outbuf and self.outbuf[0]:
             wrote = _nb_clean(os.write, self.wfile.fileno(), self.outbuf[0])
             debug2('mux wrote: %r/%d\n' % (wrote, len(self.outbuf[0])))
@@ -446,7 +453,13 @@ class Mux(Handler):
             self.outbuf[0:1] = []
 
     def fill(self):
+        try:
             os.set_blocking(self.rfile.fileno(), False)
+        except AttributeError:
+            # python < 3.5
+            flags = fcntl.fcntl(self.rfile.fileno(), fcntl.F_GETFL)
+            flags |= os.O_NONBLOCK
+            flags = fcntl.fcntl(self.rfile.fileno(), fcntl.F_SETFL, flags)
         try:
             read = _nb_clean(os.read, self.rfile.fileno(), LATENCY_BUFFER_SIZE)
         except OSError:
@@ -570,7 +583,7 @@ class MuxWrapper(SockWrapper):
 def connect_dst(family, ip, port):
     debug2('Connecting to %s:%d\n' % (ip, port))
     outsock = socket.socket(family)
-    outsock.setsockopt(socket.SOL_IP, socket.IP_TTL, 42)
+    outsock.setsockopt(socket.SOL_IP, socket.IP_TTL, 63)
     return SockWrapper(outsock, outsock,
                        connect_to=(ip, port),
                        peername='%s:%d' % (ip, port))

tests/client/test_firewall.py

@@ -55,7 +55,7 @@ def test_rewrite_etc_hosts(tmpdir):
         assert line == ""
 
     with patch('sshuttle.firewall.HOSTSFILE', new=str(new_hosts)):
-        sshuttle.firewall.restore_etc_hosts(10)
+        sshuttle.firewall.restore_etc_hosts(hostmap, 10)
     assert orig_hosts.computehash() == new_hosts.computehash()
 
 
@@ -116,6 +116,8 @@ def test_main(mock_get_method, mock_setup_daemon, mock_rewrite_etc_hosts):
     assert mock_setup_daemon.mock_calls == [call()]
     assert mock_get_method.mock_calls == [
         call('not_auto'),
+        call().is_supported(),
+        call().is_supported().__bool__(),
         call().setup_firewall(
             1024, 1026,
             [(AF_INET6, u'2404:6800:4004:80c::33')],

tests/client/test_helpers.py

@@ -131,7 +131,7 @@ nameserver 2404:6800:4004:80c::3
 nameserver 2404:6800:4004:80c::4
 """)
 
-    ns = sshuttle.helpers.resolvconf_nameservers()
+    ns = sshuttle.helpers.resolvconf_nameservers(False)
     assert ns == [
         (AF_INET, u'192.168.1.1'), (AF_INET, u'192.168.2.1'),
         (AF_INET, u'192.168.3.1'), (AF_INET, u'192.168.4.1'),
@@ -156,7 +156,7 @@ nameserver 2404:6800:4004:80c::2
 nameserver 2404:6800:4004:80c::3
 nameserver 2404:6800:4004:80c::4
 """)
-    ns = sshuttle.helpers.resolvconf_random_nameserver()
+    ns = sshuttle.helpers.resolvconf_random_nameserver(False)
     assert ns in [
         (AF_INET, u'192.168.1.1'), (AF_INET, u'192.168.2.1'),
         (AF_INET, u'192.168.3.1'), (AF_INET, u'192.168.4.1'),

tests/client/test_methods_nat.py

@@ -18,12 +18,22 @@ def test_get_supported_features():
 
 def test_get_tcp_dstip():
     sock = Mock()
+    sock.family = AF_INET
     sock.getsockopt.return_value = struct.pack(
         '!HHBBBB', socket.ntohs(AF_INET), 1024, 127, 0, 0, 1)
     method = get_method('nat')
     assert method.get_tcp_dstip(sock) == ('127.0.0.1', 1024)
     assert sock.mock_calls == [call.getsockopt(0, 80, 16)]
 
+    sock = Mock()
+    sock.family = AF_INET6
+    sock.getsockopt.return_value = struct.pack(
+        '!HH4xBBBBBBBBBBBBBBBB', socket.ntohs(AF_INET6),
+        1024, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1)
+    method = get_method('nft')
+    assert method.get_tcp_dstip(sock) == ('::1', 1024)
+    assert sock.mock_calls == [call.getsockopt(41, 80, 64)]
+
 
 def test_recv_udp():
     sock = Mock()
@@ -123,12 +133,8 @@ def test_setup_firewall(mock_ipt_chain_exists, mock_ipt_ttl, mock_ipt):
         call(AF_INET, 'nat', 'sshuttle-1025')
     ]
     assert mock_ipt_ttl.mock_calls == [
-        call(AF_INET, 'nat', '-A', 'sshuttle-1025', '-j', 'REDIRECT',
+        call(AF_INET, 'nat', '-A', 'sshuttle-1025', '-j', 'RETURN',
-             '--dest', u'1.2.3.0/24', '-p', 'tcp', '--dport', '8000:9000',
+             '-m', 'ttl', '--ttl', '63')
-             '--to-ports', '1025'),
-        call(AF_INET, 'nat', '-A', 'sshuttle-1025', '-j', 'REDIRECT',
-             '--dest', u'1.2.3.33/32', '-p', 'udp',
-             '--dport', '53', '--to-ports', '1027')
     ]
     assert mock_ipt.mock_calls == [
         call(AF_INET, 'nat', '-D', 'OUTPUT', '-j', 'sshuttle-1025'),
@@ -139,14 +145,16 @@ def test_setup_firewall(mock_ipt_chain_exists, mock_ipt_ttl, mock_ipt):
         call(AF_INET, 'nat', '-F', 'sshuttle-1025'),
         call(AF_INET, 'nat', '-I', 'OUTPUT', '1', '-j', 'sshuttle-1025'),
         call(AF_INET, 'nat', '-I', 'PREROUTING', '1', '-j', 'sshuttle-1025'),
+        call(AF_INET, 'nat', '-A', 'sshuttle-1025', '-j', 'REDIRECT',
+             '--dest', u'1.2.3.33/32', '-p', 'udp',
+             '--dport', '53', '--to-ports', '1027'),
         call(AF_INET, 'nat', '-A', 'sshuttle-1025', '-j', 'RETURN',
-             '-m', 'addrtype', '--dst-type', 'LOCAL',
+             '-m', 'addrtype', '--dst-type', 'LOCAL'),
-             '!', '-p', 'udp'),
         call(AF_INET, 'nat', '-A', 'sshuttle-1025', '-j', 'RETURN',
-             '-m', 'addrtype', '--dst-type', 'LOCAL',
+             '--dest', u'1.2.3.66/32', '-p', 'tcp', '--dport', '8080:8080'),
-             '-p', 'udp', '!', '--dport', '53'),
+        call(AF_INET, 'nat', '-A', 'sshuttle-1025', '-j', 'REDIRECT',
-        call(AF_INET, 'nat', '-A', 'sshuttle-1025', '-j', 'RETURN',
+             '--dest', u'1.2.3.0/24', '-p', 'tcp', '--dport', '8000:9000',
-             '--dest', u'1.2.3.66/32', '-p', 'tcp', '--dport', '8080:8080')
+             '--to-ports', '1025')
     ]
     mock_ipt_chain_exists.reset_mock()
     mock_ipt_ttl.reset_mock()

tests/client/test_methods_pf.py

@@ -4,7 +4,7 @@ from socket import AF_INET, AF_INET6
 import pytest
 from mock import Mock, patch, call, ANY
 from sshuttle.methods import get_method
-from sshuttle.helpers import Fatal
+from sshuttle.helpers import Fatal, get_env
 from sshuttle.methods.pf import FreeBsd, Darwin, OpenBsd
 
 
@@ -316,7 +316,8 @@ def test_setup_firewall_freebsd(mock_pf_get_dev, mock_ioctl, mock_pfctl,
              b'to <dns_servers> port 53 keep state\n'),
         call('-e'),
     ]
-    assert call(['kldload', 'pf']) in mock_subprocess_call.mock_calls
+    assert call(['kldload', 'pf'], env=get_env()) in \
+        mock_subprocess_call.mock_calls
     mock_pf_get_dev.reset_mock()
     mock_ioctl.reset_mock()
     mock_pfctl.reset_mock()

tests/client/test_options.py

@@ -13,7 +13,6 @@ _ip4_reprs = {
         '3098282570': '184.172.10.74',
         '0xb8.0xac.0x0a.0x4a': '184.172.10.74',
         '0270.0254.0012.0112': '184.172.10.74',
-        'localhost': '127.0.0.1'
 }
 
 _ip4_swidths = (1, 8, 22, 27, 32)
@@ -31,7 +30,7 @@ _ip6_swidths = (48, 64, 96, 115, 128)
 def test_parse_subnetport_ip4():
     for ip_repr, ip in _ip4_reprs.items():
         assert sshuttle.options.parse_subnetport(ip_repr) \
-                == (socket.AF_INET, ip, 32, 0, 0)
+                == [(socket.AF_INET, ip, 32, 0, 0)]
     with pytest.raises(Fatal) as excinfo:
         sshuttle.options.parse_subnetport('10.256.0.0')
     assert str(excinfo.value) == 'Unable to resolve address: 10.256.0.0'
@@ -42,34 +41,35 @@ def test_parse_subnetport_ip4_with_mask():
         for swidth in _ip4_swidths:
             assert sshuttle.options.parse_subnetport(
                     '/'.join((ip_repr, str(swidth)))
-                    ) == (socket.AF_INET, ip, swidth, 0, 0)
+                    ) == [(socket.AF_INET, ip, swidth, 0, 0)]
     assert sshuttle.options.parse_subnetport('0/0') \
-        == (socket.AF_INET, '0.0.0.0', 0, 0, 0)
+        == [(socket.AF_INET, '0.0.0.0', 0, 0, 0)]
     with pytest.raises(Fatal) as excinfo:
         sshuttle.options.parse_subnetport('10.0.0.0/33')
-    assert str(excinfo.value) == 'width 33 is not between 0 and 32'
+    assert str(excinfo.value) \
+        == 'Slash in CIDR notation (/33) is not between 0 and 32'
 
 
 def test_parse_subnetport_ip4_with_port():
     for ip_repr, ip in _ip4_reprs.items():
         assert sshuttle.options.parse_subnetport(':'.join((ip_repr, '80'))) \
-            == (socket.AF_INET, ip, 32, 80, 80)
+            == [(socket.AF_INET, ip, 32, 80, 80)]
         assert sshuttle.options.parse_subnetport(':'.join((ip_repr, '80-90')))\
-            == (socket.AF_INET, ip, 32, 80, 90)
+            == [(socket.AF_INET, ip, 32, 80, 90)]
 
 
 def test_parse_subnetport_ip4_with_mask_and_port():
     for ip_repr, ip in _ip4_reprs.items():
         assert sshuttle.options.parse_subnetport(ip_repr + '/32:80') \
-            == (socket.AF_INET, ip, 32, 80, 80)
+            == [(socket.AF_INET, ip, 32, 80, 80)]
         assert sshuttle.options.parse_subnetport(ip_repr + '/16:80-90') \
-            == (socket.AF_INET, ip, 16, 80, 90)
+            == [(socket.AF_INET, ip, 16, 80, 90)]
 
 
 def test_parse_subnetport_ip6():
     for ip_repr, ip in _ip6_reprs.items():
         assert sshuttle.options.parse_subnetport(ip_repr) \
-                == (socket.AF_INET6, ip, 128, 0, 0)
+                == [(socket.AF_INET6, ip, 128, 0, 0)]
 
 
 def test_parse_subnetport_ip6_with_mask():
@@ -77,25 +77,26 @@ def test_parse_subnetport_ip6_with_mask():
         for swidth in _ip4_swidths + _ip6_swidths:
             assert sshuttle.options.parse_subnetport(
                     '/'.join((ip_repr, str(swidth)))
-                    ) == (socket.AF_INET6, ip, swidth, 0, 0)
+                    ) == [(socket.AF_INET6, ip, swidth, 0, 0)]
     assert sshuttle.options.parse_subnetport('::/0') \
-        == (socket.AF_INET6, '::', 0, 0, 0)
+        == [(socket.AF_INET6, '::', 0, 0, 0)]
     with pytest.raises(Fatal) as excinfo:
         sshuttle.options.parse_subnetport('fc00::/129')
-    assert str(excinfo.value) == 'width 129 is not between 0 and 128'
+    assert str(excinfo.value) \
+        == 'Slash in CIDR notation (/129) is not between 0 and 128'
 
 
 def test_parse_subnetport_ip6_with_port():
     for ip_repr, ip in _ip6_reprs.items():
         assert sshuttle.options.parse_subnetport('[' + ip_repr + ']:80') \
-            == (socket.AF_INET6, ip, 128, 80, 80)
+            == [(socket.AF_INET6, ip, 128, 80, 80)]
         assert sshuttle.options.parse_subnetport('[' + ip_repr + ']:80-90') \
-            == (socket.AF_INET6, ip, 128, 80, 90)
+            == [(socket.AF_INET6, ip, 128, 80, 90)]
 
 
 def test_parse_subnetport_ip6_with_mask_and_port():
     for ip_repr, ip in _ip6_reprs.items():
         assert sshuttle.options.parse_subnetport('[' + ip_repr + '/128]:80') \
-            == (socket.AF_INET6, ip, 128, 80, 80)
+            == [(socket.AF_INET6, ip, 128, 80, 80)]
         assert sshuttle.options.parse_subnetport('[' + ip_repr + '/16]:80-90')\
-            == (socket.AF_INET6, ip, 16, 80, 90)
+            == [(socket.AF_INET6, ip, 16, 80, 90)]

tests/ssh/test_parse_hostport.py

@@ -0,0 +1,20 @@
+from sshuttle.ssh import parse_hostport
+
+
+def test_host_only():
+    assert parse_hostport("host") == (None, None, None, "host")
+    assert parse_hostport("1.2.3.4") == (None, None, None, "1.2.3.4")
+    assert parse_hostport("2001::1") == (None, None, None, "2001::1")
+    assert parse_hostport("[2001::1]") == (None, None, None, "2001::1")
+
+
+def test_host_and_port():
+    assert parse_hostport("host:22") == (None, None, 22, "host")
+    assert parse_hostport("1.2.3.4:22") == (None, None, 22, "1.2.3.4")
+    assert parse_hostport("[2001::1]:22") == (None, None, 22, "2001::1")
+
+
+def test_username_and_host():
+    assert parse_hostport("user@host") == ("user", None, None, "host")
+    assert parse_hostport("user:@host") == ("user", None, None, "host")
+    assert parse_hostport("user:pass@host") == ("user", "pass", None, "host")

tox.ini

@@ -5,12 +5,14 @@ envlist =
     py36,
     py37,
     py38,
+    py39,
 
 [testenv]
 basepython =
     py36: python3.6
     py37: python3.7
     py38: python3.8
+    py39: python3.9
 commands =
     pip install -e .
     # actual flake8 test