Christian Schwarz
4b373fbd95
zfs & replication: explicit conflict types for FilesystemDiff + handling in repl
2017-07-08 13:13:16 +02:00
Christian Schwarz
8e378d76b9
scratchpad: repeat: run a command in a certain interval or as soon as it finishes
2017-07-08 12:08:34 +02:00
Christian Schwarz
2c13fbe6ec
config: rename 'pools' section to 'remotes'
2017-07-08 12:08:34 +02:00
Christian Schwarz
e951beaef5
Simplify CLI by requiring explicit job names.
...
Job names are derived from job type + user-defined name in config file
CLI now has subcommands corresponding 1:1 to the config file sections:
push,pull,autosnap,prune
A subcommand always expects a job name, thus executes exactly one job.
Dict-style syntax also used for PullACL and Sink sections.
jobrun package is currently only used for autosnap, all others need to
be invoked repeatedly via external tool.
Plan is to re-integrate jobrun in an explicit daemon-mode (subcommand).
2017-07-08 11:13:50 +02:00
Christian Schwarz
b44a005bbb
Switch to using https://github.com/spf13/cobra for CLI.
...
Use opportunity to structure project by subcommands.
2017-07-06 13:36:55 +02:00
Christian Schwarz
655b3ab55f
implement automatic snapshotting feature
2017-07-02 00:02:33 +02:00
Christian Schwarz
8c8a6ee905
implement snapshot pruning feature
2017-07-02 00:02:33 +02:00
Christian Schwarz
e0d39ddf11
Implement RetentionGrid structure.
2017-07-01 23:19:31 +02:00
Christian Schwarz
c7f140a00f
zfs: support destroy
2017-07-01 23:19:31 +02:00
Christian Schwarz
c22190e981
zfs: extract filesystem version code to separate file & add filtering support
2017-07-01 23:19:31 +02:00
Christian Schwarz
2b6f3ece6b
jobrun: fix timing issue and minor printing issues
...
Would add offset by 1 sec
(possibly to avoid being scheduled just a little bit too early).
Turns out this leads to delays for jobs with interval < 2s :)
2017-07-01 23:19:31 +02:00
Christian Schwarz
2c50c8fd63
cmd: run: flag for running jobs only once
2017-07-01 23:19:31 +02:00
Christian Schwarz
4f86fa8332
cmd: support for pprof over http
2017-07-01 23:19:31 +02:00
Christian Schwarz
af2aa9dfe1
cmd/jobrun: repeat strategies as part of jobrun
2017-07-01 23:19:25 +02:00
Christian Schwarz
93d098162e
cmd: run: select job to run
2017-06-09 20:54:01 +02:00
Christian Schwarz
d8adce6110
zfs: Support foo/bar/* globs
2017-05-20 19:50:24 +02:00
Christian Schwarz
5f84d30972
util/ReadWriteCloserLogger: handle unset readlog | writelog
2017-05-20 19:39:32 +02:00
Christian Schwarz
3b1cac1ea2
cmd: make --logfile global parameter
2017-05-20 18:17:08 +02:00
Christian Schwarz
35dcfc234e
Implement push support.
...
Pushing is achieved by inverting the roles on the established
connection, i.e. the client tells the server what data it should pull
from the client (PullMeRequest).
Role inversion is achieved by moving the server loop to the serverLoop
function of ByteStreamRPC, which can be called from both the Listen()
function (server-side) and the PullMeRequest() client-side function.
A donwside of this PullMe approach is that the replication policies
become part of the rpc, because the puller must follow the policy.
2017-05-20 18:17:08 +02:00
Christian Schwarz
c7161cf8e6
handler: remove PushMapping, rename PullMapping to PullACL
2017-05-20 17:43:49 +02:00
Christian Schwarz
3c7f782dac
rpc: remove FilesystemRequest.Direction (unused)
2017-05-20 17:43:49 +02:00
Christian Schwarz
40fe7e643d
cmd: Move replication logic to separate file.
2017-05-20 17:29:37 +02:00
Christian Schwarz
7ad2ed5956
Rename sink -> stdinserver subcommand.
2017-05-16 16:43:39 +02:00
Christian Schwarz
f36ef41c39
scratchpad/chunker: fix api breakage
2017-05-14 15:58:33 +02:00
Christian Schwarz
defe134c8b
sshbytestream: default ServerAliveInterval
2017-05-14 14:16:12 +02:00
Christian Schwarz
b1a3a57623
cmd close RPC with timeout
2017-05-14 14:11:19 +02:00
Christian Schwarz
48a4e8033a
rpc: close outgoing SSH connection on exit.
2017-05-14 14:11:19 +02:00
Christian Schwarz
04206ebd8b
util.IOCommand: Close() gracefully via SIGTERM
2017-05-14 14:11:19 +02:00
Christian Schwarz
ee570bb060
refactor: consolidate ForkReader-like implementations to IOCommand
2017-05-14 12:27:15 +02:00
Christian Schwarz
ee8b0d3781
cmd: dup2(logfile, stderr) if logfile set
2017-05-13 15:35:19 +02:00
Christian Schwarz
6f84bf665d
cmd: support logging reads & writes from sshbytestream to a file.
2017-05-13 15:34:28 +02:00
Christian Schwarz
74719ad846
rpc: chunk JSON parts of communication + refactoring
...
JSONDecoder was buffering more of connection data than just the JSON.
=> Unchunker didn't bother and just started unchunking.
While chaining JSONDecoder.Buffered() and the connection using
ChainedReader works, it's still not a clean architecture.
=> Every JSON message is now wrapped in a chunked stream
(chunked and unchunked)
=> no special-cases
=> Keep ChainedReader, might be useful later on...
2017-05-13 15:33:46 +02:00
Christian Schwarz
feabf1abcd
rpc: logging for bytestream listener
2017-05-13 15:25:09 +02:00
Christian Schwarz
53b3a940ec
WIP: main: tree traversal
2017-05-13 15:25:09 +02:00
Christian Schwarz
5bc6d460cf
WIP: sink & pull implementation
2017-05-13 15:25:09 +02:00
Christian Schwarz
b2de658270
util: fix package name of chunking
2017-05-13 15:25:09 +02:00
Christian Schwarz
6524cfce0c
create file if it doesn't exist
2017-05-13 15:25:09 +02:00
Christian Schwarz
dd6dd60e98
Simplify "fork then io.Reader" abstractions
2017-05-13 15:23:37 +02:00
Christian Schwarz
54778c0374
zfs: remove erronous output from ForkReader
2017-05-13 15:23:28 +02:00
Christian Schwarz
c762502f6e
zfs: implement DatasetPath top down walk
2017-05-13 15:23:28 +02:00
Christian Schwarz
2407556f15
main: implement request handler.
2017-05-07 12:28:03 +02:00
Christian Schwarz
996be3704d
zfs: implement ZFSListFilesystemExists()
2017-05-07 12:26:41 +02:00
Christian Schwarz
6a420e872e
zfs: implement ZFSSet
2017-05-07 12:23:12 +02:00
Christian Schwarz
d37bdc3d7a
zfs: implement ZFSRecv
2017-05-07 12:22:57 +02:00
Christian Schwarz
030bd7affe
zfs: implement ZFSSend
2017-05-07 12:20:56 +02:00
Christian Schwarz
1a92717894
zfs: FilesytemDiff: len(IncrementalPath) < 2 means same most recent version
...
bonus: style + typo fixes
2017-05-07 12:20:56 +02:00
Christian Schwarz
d0c0e2510b
zfs: FilesystemDiff: support empty left list = no common ancestor
2017-05-07 12:20:56 +02:00
Christian Schwarz
cd8796aed4
rpc: Initial|IncrementalTransferRequest transfer zfs data structures
2017-05-07 12:20:56 +02:00
Christian Schwarz
fa97d3d98a
config: parse InitialReplPolicy with default to most_recent
2017-05-07 12:00:34 +02:00
Christian Schwarz
c71be910f9
rpc: fix incremental transfer request handling
2017-05-07 11:53:47 +02:00
