Commit Graph

9 Commits

Author SHA1 Message Date
Christian Schwarz
69bfcb7bed daemon/active: implement watchdog to handle stuck replication / pruners
ActiveSide.do() can only run sequentially, i.e. we cannot run
replication and pruning in parallel. Why?

* go-streamrpc only allows one active request at a time
(this is bad design and should be fixed at some point)
* replication and pruning are implemented independently, but work on the
same resources (snapshots)

A: pruning might destroy a snapshot that is planned to be replicated
B: replication might replicate snapshots that should be pruned

We do not have any resource management / locking for A and B, but we
have a use case where users don't want their machine fill up with
snapshots if replication does not work.
That means we _have_ to run the pruners.

A further complication is that we cannot just cancel the replication
context after a timeout and move on to the pruner: it could be initial
replication and we don't know how long it will take.
(And we don't have resumable send & recv yet).

With the previous commits, we can implement the watchdog using context
cancellation.
Note that the 'MadeProgress()' calls can only be placed right before
non-error state transition. Otherwise, we could end up in a live-lock.
2018-10-19 17:23:00 +02:00
Anton Schirg
6ca11a7391 byte counter for status 2018-08-30 12:54:30 +02:00
Christian Schwarz
e30ae972f4 gofmt 2018-08-25 21:30:25 +02:00
Christian Schwarz
a0b320bfeb streamrpc now requires net.Conn => use it instead of rwc everywhere 2018-08-08 13:09:51 +02:00
Christian Schwarz
b69089a527 Puller: refactor + use Task API
* drop rx byte count functionality
* will be re-added to Task as necessary

refs #10
2017-12-27 14:39:47 +01:00
Christian Schwarz
8eb4a2ba44 Rudimentary progress reporting on send / recv side. 2017-08-06 16:21:54 +02:00
Christian Schwarz
5f84d30972 util/ReadWriteCloserLogger: handle unset readlog | writelog 2017-05-20 19:39:32 +02:00
Christian Schwarz
6f84bf665d cmd: support logging reads & writes from sshbytestream to a file. 2017-05-13 15:34:28 +02:00
Christian Schwarz
74719ad846 rpc: chunk JSON parts of communication + refactoring
JSONDecoder was buffering more of connection data than just the JSON.
=> Unchunker didn't bother and just started unchunking.

While chaining JSONDecoder.Buffered() and the connection using
ChainedReader works, it's still not a clean architecture.

=> Every JSON message is now wrapped in a chunked stream
   (chunked and unchunked)
   => no special-cases
=> Keep ChainedReader, might be useful later on...
2017-05-13 15:33:46 +02:00