On high latency links, the PING/PONG round trip triggered by fullness
checking could kill the bandwidth. Disabling it could result in >10x
bandwidth increase in some setups where the existing latency is already high
and the available bandwidth is also high.
We were doing it as soon as the other end was shut_read, but we didn't
confirm first that the other end's read buffer had been totally emptied.
This caused problems on twitter.com, at least, where they create a zillion
short-lived connections.
I don't actually need these lines at all, as it turns out, because we were
already calling nowrite() correctly on the other end after a buffer got
emptied.
When the server disconnected, we were forwarding that information to the
client. But we weren't forwarding back the other way when the client
disconnected since there was no callback in place to do that.
Relatedly, when we failed entirely to connect to the server, we didn't notify the
client right away. Now we do.
Thanks to 'Roger' on the mailing list for pointing out these bugs.
(Note by apenwarr: I used Roger's original patch as the basis for this one,
but implemented it a different way. All errors are thus my fault, but Roger
gets the credit for actually tracking down the circular reference that
caused the memory leak.)
(Note by apenwarr: seems to still work for me. The reason the
problem occurred is that reassigning 'handlers' doesn't change it in its
parent; it creates a whole new list, and the caller still owns the old one
with all the dead sockets in it. The problem seems to have been introduced
in commit 84376284db when I factored the
runonce() functionality out of the client and server but didn't notice this
reassignment.)
Mercifully, socket.socket.shutdown() still does, but it uses hardcoded
integer parameters - and the integers correspond to the SHUT_RD and SHUT_WR
definitions in later versions - so let's just hardcode them ourselves.
See the carnage for yourself:
http://docs.python.org/release/2.3.5/lib/socket-objects.html
EPIPE is a serious error from these places, so we have to actually do
something. Otherwise the client ends up busy waiting when the server
disconnects by surprise.
Bug noticed in a log from Chetan Kunte.
- If you tried to connect to a server that didn't exist, then disconnected
the client during the 60-second connection timeout, the server would
busy wait for 60 seconds.
- If you connected to a server and then sent data, but then the server
disconnected before reading all your data, the server would busy wait.
(example: yes | telnet servername 80)
If you 'telnet localhost 12300' weird things happen; someday we should
probably auto-detect and avoid that altogether. But meanwhile, catch EPIPE
if it happens (it's irrelevant) and don't barf with a %d data type for a
value that can apparently sometimes be None.
Now if you use --auto-hosts (-H), the client will ask the server to spawn a
hostwatcher to add names. That, in turn, will send names back to the
server, which sends them back to the client, which sends them to the
firewall subprocess, which will write them to /etc/hosts. Whew!
Only the firewall process can write to /etc/hosts, of course, because only
he's running as root.
Since the name discovery process is kind of slow, we cache the names in
~/.sshuttle.hosts on the remote server.
Right now, most of the names are discovered using nmblookup and smbclient,
as well as by reading the existing entries in /etc/hosts. What would really
be nice would be to query active directory or mdns somehow... but I don't
really know how those work, so this is what you get for now :) It's pretty
neat, at least.
Now if you do
./sshuttle -Nr username@myservername
It'll automatically route the "local" subnets (ie., stuff in the routing
table) from myservername. This is (hopefully a reasonable default setting
for most people.
Instead, grab our source code, send it over the link, and have python eval
it and then start the server.py main() function.
Strangely, there's now *less* horrible stuff in ssh.py, because we no longer
have to munge around with the PATH environment variable. And this
significantly reduces the setup required to get sshuttle going.
Based on a suggestion from Wayne Scott.
Basic forwarding now works on MacOS, assuming you set up ipfw correctly
(ha ha). I wasted a few hours today trying to figure this out, and I'm *so
very close*, but unfortunately it just didn't work. Think you can figure it
out?
Related changes:
- don't die if iptables is unavailable
- BSD uses getsockname() instead of SO_ORIGINAL_DST
- non-blocking connect() returns EISCONN once it's connected
- you can't setsockopt IP_TTL more than once
It seems ssh is kind of stupid and uses a really big SO_SNDBUF (hundreds of
kbytes). Thus, we can't depend on the socket's output buffer to limit our
latency down to something reasonable. Instead, limit the amount of data we
can send in a single round trip.
