Issue #56 shows zombie SSH processes.
We fix this by actually Close()ing the RWC in job pull.
If this fixes#56 it also fixes#6 --- it's the same issue.
Additionally, debugging around this revealed another issue: just
Close()ing the sshbytestream in job source will apparently outpace the
normal data stream of stdin and stdout (URG or PUSH flags?). leading
to ugly errors in the logs.
With proper TCP connections, we would simply set the connection to
linger and close it, letting the kernel handle the final timeout. Meh.
refs #56
refs #6
- make TCPOutlet fully asynchronous, dropping messages if connection is
not fast enough
- syslog is just fine for now, local anyways
- stdout same thing
refs #26
An instance of Task tracks a single thread of activity that is part of a Job.
While the docs already use this terminology of tasks being composed of jobs,
the code did not have an object to represent these semantics.
Now it does:
* A task t is initialized with a root activity, which is its name
* t can t.Enter() and t.Finish() an activity, building
a stack of activities
* t's code can get a logger t.Log() whose logTaskField is set to the
concatenated stack of activities
* t's code can update IO progress it made since leaving idle state
* t's code's log output vie t.Log() is captured since leaving idle
state
* FIXME: find a way to bound that buffer
refs #10
refs #48
Version is autodetected on build using git
If it cannot be detected with git, an override must be provided.
For tracability of distros, the distroy packagers should override as
well, which is why I added a README entry for package mainatiners.
refs #35
Abandons stderr special-casing:
* looks weird on shell and IO redirection to same file because of
interleaving of stdout and stderr
* better than a separate dedicated outlet because it does not require
additional configuration
fixes#28
BREAK SEMANTICS CONFIG
In contrast to any 'something<' mapping, a '<' mapping cannot be unique
Thus, '<' mappings are thus just an append to target, which is exactly
what we get when trimming empty prefix ''.
Otherwise, given mapping
{ "<": "storage/backups/app-srv" }
Before (clearly a conflict)
zroot => storage/backups/app-srv
storage => storage/backups/app-srv
After:
zroot => storage/backups/app-srv/zroot
storage => storage/backups/app-srv/storage
However, mapping directly with subtree wildcard is still possible, just
not with the root wildcard
{
"<" "storage/backups/app-srv"
"zroot/var/db<": "storage/db_replication/app-srv"
}
fixes#22
While filesystems is also not the right term (since it excludes ZVOLs),
we want to stay consistent with comments & terminology used in docs.
BREAK CONFIG
fixes#17
We lost the nice context-stack [jobname][taskname][...] at the beginning
of each log line when switching to logrus.
Define some field names that define these contexts.
Write a human-friendly formatter that presents these field names like
the solution we had before logrus.
Write some other formatters for logfmt and json output along the way.
Limit ourselves to stdout logging for now.
Implement
* pruning on source side
* local job
* test subcommand for doing a dry-run of a prune policy
* use a non-blocking callback from autosnap to trigger the depending
jobs -> avoids races, looks saner in the debug log
Done:
* implement autosnapper that asserts interval between snapshots
* implement pruner
* job pull: pulling + pruning
* job source: autosnapping + serving
TODO
* job source: pruning
* job local: everything
* fatal errors such as serve that cannot bind socket must be more
visible
* couldn't things that need a snapshotprefix just use a interface
Prefixer() instead? then we could have prefixsnapshotfilter and not
duplicate it every time...
* either go full context.Context or not at all...? just wait because
community climate around it isn't that great and we only need it for
cancellation? roll our own?
How it works:
`zrepl stdinserver CLIENT_IDENTITY`
* connects to the socket in $global.serve.stdinserver.sockdir/CLIENT_IDENTITY
* sends its stdin / stdout file descriptors to the `zrepl daemon` process (see cmsg(3))
* does nothing more
This enables a setup where `zrepl daemon` is not directly exposed to the
internet but instead all traffic is tunnelled through SSH.
The server with the source job has an authorized_keys file entry for the
public key used by the corresponding pull job
command="/mnt/zrepl stdinserver CLIENT_IDENTITY" ssh-ed25519 AAAAC3NzaC1E... zrepl@pullingserver
