First, check if TTL indicates we should ignore packet (instead of
checking in multiple rules later). Also, nft method didn't do this at
all. Now, nft matches the behavior of nat.
Second, forward DNS traffic (we may need to intercept traffic to
localhost if a DNS server is running on localhost).
Third, ignore any local traffic packets. (Previously, we ignored local
traffic except DNS and then had the DNS rules). The nft method didn't
do this previously at all. It now matches the behavior of nat.
Lastly, list the subnets to redirect and/or exclude. This step is left
unchanged. Excluding the local port that we are listening on is
redundant with the third step, but should cause no harm.
In summary, this ordering simplifies the rules in nat and eliminates
differences that previously existed between nat and nft.
Use 'c' prefix for client, 's' prefix for server, and 'fw' prefix for
firewall messages. The 'c' and 's' prefixes were used sometimes but
not consistently. The firewall printed messages prefixed with
"firewall manager:" or "firewall:" or ">>" previously.
This patch also fixes a couple of print() calls that should have been
debug1()---a bug introduced in a recent commit.
Without this patch, sshuttle 'restores' /etc/hosts even if it didn't
make any modifications to it. This can be confirmed by running without
--auto-hosts and confirming that the modification time of /etc/hosts
is unchanged while sshuttle is running, but is updated when sshuttle
exits (and a debug2() message is printed indicating the file is
written).
I'm not aware of the previous behavior causing problems. However,
writing an important file unnecessarily as root should be avoided.
Pull request #502 made -r/--remote required. However, the
documentation still indicates that using no remote is a valid way to
test sshuttle (see Examples section of man page). I think this mode
might be useful for testing performance local without ssh, local with
ssh, and remote with ssh.
This patch adds a warning when -r/--remote is missing but restores the
previous behavior.
Additional comments, checks, warning messages, and diagnostic
information is printed out when the client starts.
We assume IPv4 is always present and enabled. We assume IPv6 is not
supported when it is disabled at the command line or when it is not
supported by the firewall method. Warn if IPv6 is disabled but the
user specified IPv6 subnets, IPv6 DNS servers, or IPv6 excludes that
are effectively ignored.
Instead of indicating which features are on/off, we also indicate if
features are available in the verbose output.
We also more clearly print the subnets that we forward, excludes, and
any redirected DNS servers to the terminal output.
These changes should help handling bug reports and make it clearer to
users what is happening. It should also make it more graceful when a
user specifies a subnet/exclude with hostname that resolves to both
IPv4 and IPv6 (but IPv6 is disabled in sshuttle).
The list of subnets to route over VPN and the list of subnets to
exclude are parsed in option.py parse_subnetport(). Hostnames or IP
addresses are supported. If a hostname was provided, only the first IP
address was considered. This could result in some traffic not
traversing the VPN that the user might expect should traverse it from
the arguments passed to sshuttle.
This patch makes the function handle all of the IPs if a hostname is
provided. If a user provides a hostname with a CIDR mask, problems can
occur and we warn the user about the issue.
If the user includes a hostname with both an IPv4 and an IPv6 address,
and the underlying method doesn't support IPv6, then this patch will
cause sshuttle to fail. I plan to provide a future patch where failure
won't occur if the only place IPv6 addresses appear is in the exclude
list. In that case it should be safe to ignore the IPv6 address.
This patch also changes parse_ipport() which is used by the --to-ns
option. If the user provides a hostname here, we just use the first IP
from the hostname and warn the user that only one is being used.
Some bug reports include verbose sshuttle output but lack the version
that is being used. Including the sshuttle version in the output may
make it easier to handle future bug reports.
Improve detection of when the ssh process exits in both daemon and
foreground modes. Previously, sshuttle could infinite loop with 100%
cpu usage if the ssh process died. On machines that use suspend, the
ssh connection might not resume after wakeup. Now, this situation is
detected and sshuttle exits. The fix involves changing the return
value we check for when we call poll() and using a psutil function to
detect when the process exits if we are running sshuttle as a daemon.
Python2 ignores the byte string qualification (b’foo’) but falls over for the combination rb for this regexp. Switching the qualification to br appears to fix this and works in both python2 and python3.
Fixes #469. We replace python3 exclusive code with a check for python3 and a compatibility fix. Note that the switch from os.set_nonblocking to fcntl.fcntl in 98d052d (fixing #503) also fixes python2 compatibility.
* Use types instead of imp.new_module.
I can follow up with https://docs.python.org/3/library/importlib.html#importlib.util.module_from_spec if need be.
* use source loader from importlib
* Revert "use source loader from importlib"
This reverts commit 1f255704f7.
* use inspect.getsource, but alas
* placate linter
* use find_spec to resolve a module spec to a file path
* better function naming
* remove outdated comment
* Fix parsing of hostnames to allow ssh aliases defined in ssh configs)
* nicer formatting, pep8 applied
* Properly parse IPv6 addresses with port specification
* Now also handles hostnames with port specified and IPv6 addresses without port properly
* Updated parameter description for the remotehost specification
* Make the urlparse import backwards compatible to python2
Co-authored-by: Tobi <tobi-git@grimm.gr>
* Add auto password prompt
Add auto password with sshpass
use user:password@host or user:password:port@host
* Update ssh.py
* Fix for IPv4 only
* Delete print sorry bad commit
* ipv4 fix
* Fix IPv4 args
* Fix for ipv6
* Fix ipv6 no password
* Add function parse_hostport
* Fix minor bug detect port
* Fix minor bug password detect
* Clear Code
* bad write "=" replace with "=="
* Rewrite code for more understand logical and fix minor bug
* add default define port
* delete old variable unused
* replace "==" per "is" try fix code reviews
* reback
* try define password with len
* Fix consistence variable password PR
* simplify function split ipv4 or ipv6
* clear code
* added sudoers options to command line arguments
* added sudoers options to command line arguments
* template for sudoers file
* Added option for GUI sudo
* added support for GUI sudo
* script for auto adding sudo file
* sudoers auto add works and validates
* small change
* Clean up for CI
* removed code that belongs in another PR
* added path for package bins
* added sudoers bin
* added sudoers-add to setup file
* fixed issue with sudoers bash script
* auto sudoers now works
* added --sudoers-no-modify option
* bin now works with ./run
* removed debug print
* Updated sudoers-add script
* Fixed error passing sudoers config to script
* more dynamic building of sudoers file
* added option to specify sudoers.d file name
* fixed indent issue
* fixed indent issue
* indent issue
* clean up
* formating
* docs
* fix for flags
* Update usage.rst
* removed shell=true
* cleared CI errors
* cleared CI errors
* removed random
* cleared linter issue
* cleared linter issue
* cleared linter issue
* updated sudoers-add script
* safer temp file
* moved bin directory
* moved bin directory
* removed print
* fixed spacing issue
* sudoers commands must only containe upper case latters
* Make hostwatch locale-independent
See #377: hostwatch used to call netstat and parse the result,
without setting the locale.
The problem is converting the binary output to a unicode string,
as the locale may be utf-8, latin-1, or literally anything.
Setting the locale to C avoids this issue, as netstat's source
strings to not use non-ASCII characters.
* Break line, check all other invocations
This commit resolves#297, allowing the buffers used in the latency control to be changed with a command line option ‘--latency-buffer-size’.
We do this by changing a module variable in ssnet.py (similar to the MAX_CHANNEL variable) which seems to be the simplest code change without extensive hacking.
Documentation is also updated.
The changes in a765aa32 removed a more complex pieced of code for parsing which sudo command to use. The %(eb)s no longer refers to any variable and is directly printed to the command line.
%(eb)s is now replaced with ‘sudo’.
Regression was introduced in #337 that is skipping all local traffic,
including DNS. This change makes UDP port 53 (DNS) LOCAL traffic to be
treated as special case.
Fixes#357
* re-organized imports according to pep8
* fixed all remaining pep8 issues
* moved common config into setup.cfg, additionally test `tests`
* removed --select=X -- the errors selected where by default not in
flake8's --ignore list so effectively had no effect
* update .travis.yml to reflect changes in tox.ini
* make travis just use tox in order to avoid code duplaction
* replace py.test with pytest
* fixed .travis.yml
* try different pypy toxenv
* hopefully fixed testenv for pypy
* added pypy basepython, removed unused python2.6
* install dev package before testing (fixes missing coverage)
* fixed empty exception pass blocks with noqa
* Added dummy log message on empty try-except-pass blocks to make dodacy happy :(
* Replaced Exception with BaseException
There's a known issue that makes sshuttle crash if there are too
many routes on the remote host (that don't fit in 64KB). This patch
requests the routes only if auto-nets is specified on the command
line.
This was susceptible to the same deadlock issue that ipt_chain_exists
had and was fixed in d43db80 where if the command returned a significant
amount of output, it wouldn't all be read in, resulting in the
subprocess hanging waiting for the output to be read.
When running sshuttle with a large list of routes it's failing to clean
them up at exit. It returns the following:
$ sshuttle -r user@host.example.com -s /tmp/aws-cidrs.txt
user@host.example.com's password:
client: Connected.
^CAnother app is currently holding the xtables lock; still -9s 0us time ahead to have a chance to grab the lock...
Another app is currently holding the xtables lock; still -19s 0us time ahead to have a chance to grab the lock...
Another app is currently holding the xtables lock; still -29s 0us time ahead to have a chance to grab the lock...
This continues indefinitely. Looking in ps reveals that there are 2
iptables processes running. Killing -9 the first one, allows sshuttle to
continue and clean up successfully.
The problem lies with the use of Popen here. The function currently
returns as soon as it finds a match without consuming everything from
stdout. This means that if there's more output from iptables than will
fit in the buffer it doesn't exit, and therefore doesn't release the
kernel xtables lock.
Before this change, in pf, exclusions used a pass out quick which gave
them higher precedence than any other rule independent of subnet width.
As reported in #265 this causes exclusion from one instance of sshuttle
to also take effect on other instances because quick aborts the
evaluation of rules across all anchors.
This commit changes the precedence of rules so quick can now be
dropped. The new order is defined by the following rule, from
subnet_weight:
"We need to go from smaller, more specific, port ranges, to larger,
less-specific, port ranges. At each level, we order by subnet
width, from most-specific subnets (largest swidth) to
least-specific. On ties, excludes come first."
* Fixes support for OpenBSD (6.1+)
As reported in #219, new versions of OpenBSD ship with a different
pfioc_rule struct. This commit adjusts the offset to match the new struct.
* Fixes tests for OpenBSD 6.1+
Having the tests in a `tests` directory in root is the most common
approach. Also moved pytest's conftest.py into `tests` making the
fixture available for client and server tests.
This provides a way to avoid setting PYTHONPATH when invoking the
privileged part of sshuttle with sudo. This is useful if running
sshuttle as a PEX archive, as Telepresence does, as it enables
sshuttle's sudo access to be securely locked down.
PEX archives will extract themselves into the invoking user's home
directory, which means that the invoking user has full control over
the code in them. This makes restricting sudo access with
PYTHONPATH set completely pointless in this scenario -- an attacker
could put any code into ~/.pex and gain full root access anyway.
On the other hand, if sshuttle is a PEX archive, the privileged
invocation will simply extract itself into /root/.pex anyway, so
there is no need to set PYTHONPATH in this case.
Starting sshuttle without having to type in one's password requires to
put the sudo-ed command in the `/etc/sudoers` file. However, sshuttle
sets an environment variable, which cannot be done as-is in the sudoers
file. This fix prepend the /usr/bin/env command, which allows one to
pass fixed environment variables to a sudo-ed command.
In practice, the sub-command:
```
sudo PYTHONPATH=/usr/lib/python3/dist-packages -- \
/usr/bin/python3 /usr/bin/sshuttle --method auto --firewall
```
becomes
```
sudo /usr/bin/env PYTHONPATH=/usr/lib/python3/dist-packages \
/usr/bin/python3 /usr/bin/sshuttle --method auto --firewall
```
This small change will allow a file path to be passed as argument from which
the command line options will be loaded.
Extra command line options can be passed (in addition to those already in the
file) and existing ones can be overriden.
Example sshuttle.conf file:
192.168.0.0/16
--remote
user@example.com
Example sshuttle call:
sshuttle @/path/to/sshuttle.conf
Example sshuttle call with verbose flags added:
sshuttle @/path/to/sshuttle.conf -vvv
Example sshuttle call overriding the remote server:
sshuttle @/path/to/sshuttle.conf -r otheruser@test.example.com
When I starting sshuttle with option `--seed-hosts example.com`, got the following error:
```
hostwatch: Starting hostwatch with Python version 3.5.2
hostwatch: Traceback (most recent call last):
---> File "sshuttle.server", line 144, in start_hostwatch
---> File "sshuttle.hostwatch", line 272, in hw_main
---> File "sshuttle.hostwatch", line 234, in check_host
---> File "sshuttle.hostwatch", line 32, in _is_ip
---> File "/usr/lib/python3.5/re.py", line 163, in match
---> return _compile(pattern, flags).match(string)
---> TypeError: cannot use a string pattern on a bytes-like object
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "assembler.py", line 37, in <module>
File "sshuttle.server", line 393, in main
File "sshuttle.ssnet", line 596, in runonce
File "sshuttle.server", line 324, in hostwatch_ready
sshuttle.helpers.Fatal: hostwatch process died
```
It seems like the list of hosts is not properly decoded on the server side. This is an attempt to fix that.
