The list of subnets to route over VPN and the list of subnets to
exclude are parsed in option.py parse_subnetport(). Hostnames or IP
addresses are supported. If a hostname was provided, only the first IP
address was considered. This could result in some traffic not
traversing the VPN that the user might expect should traverse it from
the arguments passed to sshuttle.
This patch makes the function handle all of the IPs if a hostname is
provided. If a user provides a hostname with a CIDR mask, problems can
occur and we warn the user about the issue.
If the user includes a hostname with both an IPv4 and an IPv6 address,
and the underlying method doesn't support IPv6, then this patch will
cause sshuttle to fail. I plan to provide a future patch where failure
won't occur if the only place IPv6 addresses appear is in the exclude
list. In that case it should be safe to ignore the IPv6 address.
This patch also changes parse_ipport() which is used by the --to-ns
option. If the user provides a hostname here, we just use the first IP
from the hostname and warn the user that only one is being used.
Some bug reports include verbose sshuttle output but lack the version
that is being used. Including the sshuttle version in the output may
make it easier to handle future bug reports.
Improve detection of when the ssh process exits in both daemon and
foreground modes. Previously, sshuttle could infinite loop with 100%
cpu usage if the ssh process died. On machines that use suspend, the
ssh connection might not resume after wakeup. Now, this situation is
detected and sshuttle exits. The fix involves changing the return
value we check for when we call poll() and using a psutil function to
detect when the process exits if we are running sshuttle as a daemon.
Python2 ignores the byte string qualification (b’foo’) but falls over for the combination rb for this regexp. Switching the qualification to br appears to fix this and works in both python2 and python3.
Fixes #469. We replace python3 exclusive code with a check for python3 and a compatibility fix. Note that the switch from os.set_nonblocking to fcntl.fcntl in 98d052d (fixing #503) also fixes python2 compatibility.
* Use types instead of imp.new_module.
I can follow up with https://docs.python.org/3/library/importlib.html#importlib.util.module_from_spec if need be.
* use source loader from importlib
* Revert "use source loader from importlib"
This reverts commit 1f255704f7.
* use inspect.getsource, but alas
* placate linter
* use find_spec to resolve a module spec to a file path
* better function naming
* remove outdated comment
* Fix parsing of hostnames to allow ssh aliases defined in ssh configs)
* nicer formatting, pep8 applied
* Properly parse IPv6 addresses with port specification
* Now also handles hostnames with port specified and IPv6 addresses without port properly
* Updated parameter description for the remotehost specification
* Make the urlparse import backwards compatible to python2
Co-authored-by: Tobi <tobi-git@grimm.gr>
* Add auto password prompt
Add auto password with sshpass
use user:password@host or user:password:port@host
* Update ssh.py
* Fix for IPv4 only
* Delete print sorry bad commit
* ipv4 fix
* Fix IPv4 args
* Fix for ipv6
* Fix ipv6 no password
* Add function parse_hostport
* Fix minor bug detect port
* Fix minor bug password detect
* Clear Code
* bad write "=" replace with "=="
* Rewrite code for more understand logical and fix minor bug
* add default define port
* delete old variable unused
* replace "==" per "is" try fix code reviews
* reback
* try define password with len
* Fix consistence variable password PR
* simplify function split ipv4 or ipv6
* clear code
* added sudoers options to command line arguments
* added sudoers options to command line arguments
* template for sudoers file
* Added option for GUI sudo
* added support for GUI sudo
* script for auto adding sudo file
* sudoers auto add works and validates
* small change
* Clean up for CI
* removed code that belongs in another PR
* added path for package bins
* added sudoers bin
* added sudoers-add to setup file
* fixed issue with sudoers bash script
* auto sudoers now works
* added --sudoers-no-modify option
* bin now works with ./run
* removed debug print
* Updated sudoers-add script
* Fixed error passing sudoers config to script
* more dynamic building of sudoers file
* added option to specify sudoers.d file name
* fixed indent issue
* fixed indent issue
* indent issue
* clean up
* formating
* docs
* fix for flags
* Update usage.rst
* removed shell=true
* cleared CI errors
* cleared CI errors
* removed random
* cleared linter issue
* cleared linter issue
* cleared linter issue
* updated sudoers-add script
* safer temp file
* moved bin directory
* moved bin directory
* removed print
* fixed spacing issue
* sudoers commands must only containe upper case latters
* Make hostwatch locale-independent
See #377: hostwatch used to call netstat and parse the result,
without setting the locale.
The problem is converting the binary output to a unicode string,
as the locale may be utf-8, latin-1, or literally anything.
Setting the locale to C avoids this issue, as netstat's source
strings to not use non-ASCII characters.
* Break line, check all other invocations
This commit resolves#297, allowing the buffers used in the latency control to be changed with a command line option ‘--latency-buffer-size’.
We do this by changing a module variable in ssnet.py (similar to the MAX_CHANNEL variable) which seems to be the simplest code change without extensive hacking.
Documentation is also updated.
The changes in a765aa32 removed a more complex pieced of code for parsing which sudo command to use. The %(eb)s no longer refers to any variable and is directly printed to the command line.
%(eb)s is now replaced with ‘sudo’.
Regression was introduced in #337 that is skipping all local traffic,
including DNS. This change makes UDP port 53 (DNS) LOCAL traffic to be
treated as special case.
Fixes#357
* re-organized imports according to pep8
* fixed all remaining pep8 issues
* moved common config into setup.cfg, additionally test `tests`
* removed --select=X -- the errors selected where by default not in
flake8's --ignore list so effectively had no effect
* update .travis.yml to reflect changes in tox.ini
* make travis just use tox in order to avoid code duplaction
* replace py.test with pytest
* fixed .travis.yml
* try different pypy toxenv
* hopefully fixed testenv for pypy
* added pypy basepython, removed unused python2.6
* install dev package before testing (fixes missing coverage)
* fixed empty exception pass blocks with noqa
* Added dummy log message on empty try-except-pass blocks to make dodacy happy :(
* Replaced Exception with BaseException
There's a known issue that makes sshuttle crash if there are too
many routes on the remote host (that don't fit in 64KB). This patch
requests the routes only if auto-nets is specified on the command
line.
This was susceptible to the same deadlock issue that ipt_chain_exists
had and was fixed in d43db80 where if the command returned a significant
amount of output, it wouldn't all be read in, resulting in the
subprocess hanging waiting for the output to be read.
When running sshuttle with a large list of routes it's failing to clean
them up at exit. It returns the following:
$ sshuttle -r user@host.example.com -s /tmp/aws-cidrs.txt
user@host.example.com's password:
client: Connected.
^CAnother app is currently holding the xtables lock; still -9s 0us time ahead to have a chance to grab the lock...
Another app is currently holding the xtables lock; still -19s 0us time ahead to have a chance to grab the lock...
Another app is currently holding the xtables lock; still -29s 0us time ahead to have a chance to grab the lock...
This continues indefinitely. Looking in ps reveals that there are 2
iptables processes running. Killing -9 the first one, allows sshuttle to
continue and clean up successfully.
The problem lies with the use of Popen here. The function currently
returns as soon as it finds a match without consuming everything from
stdout. This means that if there's more output from iptables than will
fit in the buffer it doesn't exit, and therefore doesn't release the
kernel xtables lock.
Before this change, in pf, exclusions used a pass out quick which gave
them higher precedence than any other rule independent of subnet width.
As reported in #265 this causes exclusion from one instance of sshuttle
to also take effect on other instances because quick aborts the
evaluation of rules across all anchors.
This commit changes the precedence of rules so quick can now be
dropped. The new order is defined by the following rule, from
subnet_weight:
"We need to go from smaller, more specific, port ranges, to larger,
less-specific, port ranges. At each level, we order by subnet
width, from most-specific subnets (largest swidth) to
least-specific. On ties, excludes come first."
* Fixes support for OpenBSD (6.1+)
As reported in #219, new versions of OpenBSD ship with a different
pfioc_rule struct. This commit adjusts the offset to match the new struct.
* Fixes tests for OpenBSD 6.1+
Having the tests in a `tests` directory in root is the most common
approach. Also moved pytest's conftest.py into `tests` making the
fixture available for client and server tests.
This provides a way to avoid setting PYTHONPATH when invoking the
privileged part of sshuttle with sudo. This is useful if running
sshuttle as a PEX archive, as Telepresence does, as it enables
sshuttle's sudo access to be securely locked down.
PEX archives will extract themselves into the invoking user's home
directory, which means that the invoking user has full control over
the code in them. This makes restricting sudo access with
PYTHONPATH set completely pointless in this scenario -- an attacker
could put any code into ~/.pex and gain full root access anyway.
On the other hand, if sshuttle is a PEX archive, the privileged
invocation will simply extract itself into /root/.pex anyway, so
there is no need to set PYTHONPATH in this case.
Starting sshuttle without having to type in one's password requires to
put the sudo-ed command in the `/etc/sudoers` file. However, sshuttle
sets an environment variable, which cannot be done as-is in the sudoers
file. This fix prepend the /usr/bin/env command, which allows one to
pass fixed environment variables to a sudo-ed command.
In practice, the sub-command:
```
sudo PYTHONPATH=/usr/lib/python3/dist-packages -- \
/usr/bin/python3 /usr/bin/sshuttle --method auto --firewall
```
becomes
```
sudo /usr/bin/env PYTHONPATH=/usr/lib/python3/dist-packages \
/usr/bin/python3 /usr/bin/sshuttle --method auto --firewall
```
This small change will allow a file path to be passed as argument from which
the command line options will be loaded.
Extra command line options can be passed (in addition to those already in the
file) and existing ones can be overriden.
Example sshuttle.conf file:
192.168.0.0/16
--remote
user@example.com
Example sshuttle call:
sshuttle @/path/to/sshuttle.conf
Example sshuttle call with verbose flags added:
sshuttle @/path/to/sshuttle.conf -vvv
Example sshuttle call overriding the remote server:
sshuttle @/path/to/sshuttle.conf -r otheruser@test.example.com
When I starting sshuttle with option `--seed-hosts example.com`, got the following error:
```
hostwatch: Starting hostwatch with Python version 3.5.2
hostwatch: Traceback (most recent call last):
---> File "sshuttle.server", line 144, in start_hostwatch
---> File "sshuttle.hostwatch", line 272, in hw_main
---> File "sshuttle.hostwatch", line 234, in check_host
---> File "sshuttle.hostwatch", line 32, in _is_ip
---> File "/usr/lib/python3.5/re.py", line 163, in match
---> return _compile(pattern, flags).match(string)
---> TypeError: cannot use a string pattern on a bytes-like object
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "assembler.py", line 37, in <module>
File "sshuttle.server", line 393, in main
File "sshuttle.ssnet", line 596, in runonce
File "sshuttle.server", line 324, in hostwatch_ready
sshuttle.helpers.Fatal: hostwatch process died
```
It seems like the list of hosts is not properly decoded on the server side. This is an attempt to fix that.
As suggested by @colinmkeith the UDP and DNS proxies should listen on different
ports otherwise the DNS proxy can get traffic intended to the UDP proxy (or
vice-versa) and handle it incorrectly as reported in #178.
At first sight it seems that we had the code in place to try another port if
the one we are binding is already bound, however, with UDP and REUSEADDR the
OS will not refuse to bind two sockets to the same socket address, so both
the UDP proxy and DNS proxy were being bound to the same pair.
Some Linux distros, like Alpine, Arch, etc and some BSDs, like FreeBSD, are
now shipping with python3.6 as the default python3. Both the client and the
server are failing to run in this distros, because we are specifically looking
for python3.5.
These changes make the run shell script use python3 if the version is greater
than 3.5, otherwise falling back as usual.
On the server any version of python3 will do, use it before falling back to
python, as the server code can run with any version of python3.
When the pf module is not loaded our calls to pfctl will fail with
unhelpful messages.
This change spares the user the pain of decrypting those messages and manually
enabling pf. It also keeps track if pf was loaded by sshuttle and unloads on
exit if that was the case.
Also fixed the case where both ipv4 and ipv6 anchors were added by sshuttle
but the first call of disable would disable pf before the second call had the
chance of cleaning it's anchor.
If we receive no routes from server or if, for some reason, we receive
some empty lines, we should skip them instead of crashing.
Fixes on of the problems in #147.
Currently hostwatch only adds hostnames even when FQDNs are available.
This commit changes found_host so that when the name is a FQDN, both the FQDN
and an hostname are added, e.g., given api.foo.com both api and api.foo.com
will be added.
Fixes#151 if merged.
N.B.: I rarely use hostwatch, it would probably be a good idea to get feedback
from people who actually use it before merging. Not too sure about this...
While with AF_INET sockaddr is a 2-tuple composed by (address, port),
with AF_INET6 it is a 4-tuple with (address, port, flow info, scope id).
We were always passing a 2-tuple to socket.connect which would fail whenever
the address was, for instance, a link-local IPv6 address that needs a scope id.
With this change we now use getaddrinfo to correctly compute the full tuple.
Fixes#156.
When doing port forwarding on lo0 avoid the special case where the
traffic on lo0 did not came from sshuttle pass out rule but from the lo0
address itself. Fixes#159.
This change makes the subnets with the most specific port ranges come
before subnets with larger, least specific, port ranges. Before this
change subnets with smaller swidth would always come first and only for
subnets with the same width would the size of the port range be
considered.
Example:
188.0.0.0/8 -x 0.0.0.0/0:443
Before: 188.0.0.0/8 would come first meaning that all ports would be
routed through the VPN for the subnet 188.0.0.0/8
After: 0.0.0.0/0:443 comes first, meaning that port 443 will be
excluded for all subnets, including 188.0.0.0/8. All other ports of
188.0.0.0/8 will be routed.
* Adds support for tunneling specific port ranges
This set of changes implements the ability of specifying a port or port
range for an IP or subnet to only tunnel those ports for that subnet.
Also supports excluding a port or port range for a given IP or subnet.
When, for a given subnet, there are intercepting ranges being added and
excluded, the most specific, i.e., smaller range, takes precedence. In
case of a tie the exclusion wins.
For different subnets, the most specific, i.e., largest swidth, takes
precedence independent of any eventual port ranges.
Examples:
Tunnels all traffic to the 188.0.0.0/8 subnet except those to port 443.
```
sshuttle -r <server> 188.0.0.0/8 -x 188.0.0.0/8:443
```
Only tunnels traffic to port 80 of the 188.0.0.0/8 subnet.
```
sshuttle -r <server> 188.0.0.0/8:80
```
Tunnels traffic to the 188.0.0.0/8 subnet and the port range that goes
from 80 to 89.
```
sshuttle -r <server> 188.0.0.0/8:80-89 -x 188.0.0.0/8:80-90
```
* Allow subnets to be specified with domain names
Simplifies the implementation of address parsing by using
socket.getaddrinfo(), which can handle domain resolution, IPv4 and IPv6
addresses. This was proposed and mostly implemented by @DavidBuchanan314
in #146.
Signed-off-by: David Buchanan <DavidBuchanan314@users.noreply.github.com>
Signed-off-by: João Vieira <vieira@yubo.be>
* Also use getaddrinfo for parsing listen addr:port
* Fixes tests for tunneling a port range
* Updates documentation to include port/port range
Adds some examples with subnet:port and subnet:port-port.
Also clarifies the versions of Python supported on the server while
maintaining the recommendation for Python 2.7, 3.5 or later.
Mentions support for pfSense.
* In Py2 only named arguments may follow *expression
Fixes issue in Python 2.7 where *expression may only be followed by
named arguments.
* Use right regex to extract ip4/6, mask and ports
* Tests for parse_subnetport
netstat outputs some headers in BSD (that the Linux version does not)
that are not tabular and were breaking our 'split line into columns
and get nth column' logic. We now skip such headers.
Should fix#141.
There was runtime failure on UDP or DNS processing, because "socket" was redefined to PyXAPI's socket_ext in tproxy.py, but still was plain Python's socket in client.py
Fixed https://github.com/sshuttle/sshuttle/issues/134 for me
`netstat` has been deprecated for some time and some distros might
start shipping without it in the near future. This commit adds support
for `ip route` and uses it when available.
PfSense is based on FreeBSD and its pf is pretty close to the one
FreeBSD ships, however some structures have different fields and two
offsets had to be fixed.
We set it to true when we enable pf, but do not set it back to False
after disabling. When using IPv4 and IPv6 we end up trying to disable
twice which procudes an error while undoing changes in FreeBSD 11.
These changes introduce support for sdnotify allowing sshuttle to notify
systemd when it finishes connecting to the server and installing
firewall rules, and is ready to tunnel requests.
This should fix an issue introduced in #117 where when no subnets are
given via file (-s file) the variable is None instead of an empty list
and the concatenation with the subnets given as positional parameters
fails.
By just splitting at spaces, multi-word arguments are torn apart even if
quoted. In case of custom ssh-cmd, this makes it practically impossible
to set certian options through `ssh -o`.
shlex splits arguments like a shell and e.g. respects quotes.
This should fix#116. Handling this while still having the positional
arguments and -s both write to the same list turned out to be more
complicated than it's worth so each writes to their own variable and we
merge them at the end.
AF_INET is the same constant on Linux and BSD but AF_INET6
is different. As the client and server can be running on
different platforms we can not just set the socket family
to what comes in the wire.
A possible implementation for the change requested in #94, so that seed
hosts can be used without auto hosts. In this scenario only the
specified hosts (or ips) will be looked up (or rev looked up).
We shouldn't come up with a fatal error because of a ENETUNREACH when
trying to contact the DNS server. Although this error shouldn't happen
either.
Fixes#89.
Previously the sshuttle shell script would pass the python to use as the
first argument of the command. The new run script no longer does this.
Instead we can obtain the python being used via sys.executable.
Fixes#88.
It is often the case that the user has no administrative control over
the server that is being used. As such it is important to support as
many versions as possible, at least on the remote server end. These
fixes will allow sshuttle to be used with servers that have only
python 2.4 or python 2.6 installed while hopefully not breaking the
compatibility with 2.7 and 3.5.
When passing multiple subnet files, e.g., by using -s/--subnets
multiple times or by using it together with subnets passed as positional
arguments append the content from all sources instead of only using the
subnets from the last source. This makes the behaviour of -s/--subnets
consistent with -x/--exclude.
This allows disabling all client tests using a conftest.py file, if for
example #56 gets merged and the server supports more python versions
then the server.
The server side tests are very incomplete.
