Christian Schwarz
4e45b4090b
pull log output: optimize to be readable by humans
2017-08-06 18:28:05 +02:00
Christian Schwarz
cba083cadf
Make zfs.DatasetPath json.Marshaler and json.Unmarshaler
...
Had to resort to using pointers to zfs.DatasetPath everywhere... Should
find a better solution for that.
2017-08-06 16:22:15 +02:00
Christian Schwarz
2ce07c9342
rework filters & mappings
...
config defines a single datastructure that can act both as a Map and as a Filter
(DatasetMapFilter)
Cleanup wildcard syntax along the way (also changes semantics).
2017-08-06 16:21:54 +02:00
Christian Schwarz
3fac6a67df
extract PullACL check into function
2017-08-06 16:21:54 +02:00
Christian Schwarz
4732fdd4cc
Implement placeholder filesystems.
...
Note the docs on the placeholder user property introduced with this
commit. The solution is not really satisfying but couldn't think of a
better one OTOMH
2017-08-06 16:21:54 +02:00
Christian Schwarz
8eb4a2ba44
Rudimentary progress reporting on send / recv side.
2017-08-06 16:21:54 +02:00
Christian Schwarz
d1999fc17c
Remove months as a possible time interval unit as it is too volatile.
...
Thanks to @erdgeist for pointing that out.
refs #2
2017-07-09 00:38:16 +02:00
Dirk Engling
5afbedbd87
Shrink the 'monthly' interval from 32 weeks to 32 days
2017-07-09 00:11:02 +02:00
Christian Schwarz
4b373fbd95
zfs & replication: explicit conflict types for FilesystemDiff + handling in repl
2017-07-08 13:13:16 +02:00
Christian Schwarz
2c13fbe6ec
config: rename 'pools' section to 'remotes'
2017-07-08 12:08:34 +02:00
Christian Schwarz
e951beaef5
Simplify CLI by requiring explicit job names.
...
Job names are derived from job type + user-defined name in config file
CLI now has subcommands corresponding 1:1 to the config file sections:
push,pull,autosnap,prune
A subcommand always expects a job name, thus executes exactly one job.
Dict-style syntax also used for PullACL and Sink sections.
jobrun package is currently only used for autosnap, all others need to
be invoked repeatedly via external tool.
Plan is to re-integrate jobrun in an explicit daemon-mode (subcommand).
2017-07-08 11:13:50 +02:00
Christian Schwarz
b44a005bbb
Switch to using https://github.com/spf13/cobra for CLI.
...
Use opportunity to structure project by subcommands.
2017-07-06 13:36:55 +02:00
Christian Schwarz
655b3ab55f
implement automatic snapshotting feature
2017-07-02 00:02:33 +02:00
Christian Schwarz
8c8a6ee905
implement snapshot pruning feature
2017-07-02 00:02:33 +02:00
Christian Schwarz
c22190e981
zfs: extract filesystem version code to separate file & add filtering support
2017-07-01 23:19:31 +02:00
Christian Schwarz
2c50c8fd63
cmd: run: flag for running jobs only once
2017-07-01 23:19:31 +02:00
Christian Schwarz
4f86fa8332
cmd: support for pprof over http
2017-07-01 23:19:31 +02:00
Christian Schwarz
af2aa9dfe1
cmd/jobrun: repeat strategies as part of jobrun
2017-07-01 23:19:25 +02:00
Christian Schwarz
93d098162e
cmd: run: select job to run
2017-06-09 20:54:01 +02:00
Christian Schwarz
3b1cac1ea2
cmd: make --logfile global parameter
2017-05-20 18:17:08 +02:00
Christian Schwarz
35dcfc234e
Implement push support.
...
Pushing is achieved by inverting the roles on the established
connection, i.e. the client tells the server what data it should pull
from the client (PullMeRequest).
Role inversion is achieved by moving the server loop to the serverLoop
function of ByteStreamRPC, which can be called from both the Listen()
function (server-side) and the PullMeRequest() client-side function.
A donwside of this PullMe approach is that the replication policies
become part of the rpc, because the puller must follow the policy.
2017-05-20 18:17:08 +02:00
Christian Schwarz
c7161cf8e6
handler: remove PushMapping, rename PullMapping to PullACL
2017-05-20 17:43:49 +02:00
Christian Schwarz
3c7f782dac
rpc: remove FilesystemRequest.Direction (unused)
2017-05-20 17:43:49 +02:00
Christian Schwarz
40fe7e643d
cmd: Move replication logic to separate file.
2017-05-20 17:29:37 +02:00
Christian Schwarz
7ad2ed5956
Rename sink -> stdinserver subcommand.
2017-05-16 16:43:39 +02:00
Christian Schwarz
b1a3a57623
cmd close RPC with timeout
2017-05-14 14:11:19 +02:00
Christian Schwarz
48a4e8033a
rpc: close outgoing SSH connection on exit.
2017-05-14 14:11:19 +02:00
Christian Schwarz
ee8b0d3781
cmd: dup2(logfile, stderr) if logfile set
2017-05-13 15:35:19 +02:00
Christian Schwarz
6f84bf665d
cmd: support logging reads & writes from sshbytestream to a file.
2017-05-13 15:34:28 +02:00
Christian Schwarz
feabf1abcd
rpc: logging for bytestream listener
2017-05-13 15:25:09 +02:00
Christian Schwarz
53b3a940ec
WIP: main: tree traversal
2017-05-13 15:25:09 +02:00
Christian Schwarz
5bc6d460cf
WIP: sink & pull implementation
2017-05-13 15:25:09 +02:00
Christian Schwarz
2407556f15
main: implement request handler.
2017-05-07 12:28:03 +02:00
Christian Schwarz
cd8796aed4
rpc: Initial|IncrementalTransferRequest transfer zfs data structures
2017-05-07 12:20:56 +02:00
Christian Schwarz
fa97d3d98a
config: parse InitialReplPolicy with default to most_recent
2017-05-07 12:00:34 +02:00
Christian Schwarz
22454738af
application-wide logging through Logger interface
2017-05-03 18:32:11 +02:00
Christian Schwarz
55463e5f26
jobrun: per-job logger
2017-05-03 18:28:04 +02:00
Christian Schwarz
3b6d79ec67
jobrun: log through abstract logger interface instead of stderr
2017-05-03 18:27:55 +02:00
Christian Schwarz
77f749112c
main: remove global handler and unused structs
2017-05-03 18:27:23 +02:00
Christian Schwarz
f005ce318d
Purge model package, not really used anyways.
2017-05-03 17:26:45 +02:00
Christian Schwarz
43f67d2b7c
rpc: add FilesystemVersionsRequest
2017-05-03 17:12:15 +02:00
Christian Schwarz
301be177ea
config: fix broken parsing of direct mapping
...
Would only parse wildcard ('|') DirectMapping but no specific direct
mappings.
2017-05-01 20:08:20 +02:00
Christian Schwarz
6da2deb96e
config: fix mapping parser
2017-04-30 23:47:11 +02:00
Christian Schwarz
3e0c758d7f
config: PushACLs, sinks are also just ClientMappings & LOCAL_TRANSPORT_IDENTITY
2017-04-30 23:47:11 +02:00
Christian Schwarz
2e6dc26993
sshbytestream: IdentityFile and custom SSHCommand.
2017-04-30 23:46:59 +02:00
Christian Schwarz
2d57e15936
Change transport format in zrepl config & parse it.
2017-04-29 20:10:09 +02:00
Christian Schwarz
9edc2005ea
config; pointer to pools for pull and push jobs
2017-04-29 19:07:47 +02:00
Christian Schwarz
526255a9ef
Implement jobrun package, abstraction for cron-like goroutines.
...
Unlike cron, there is no overtaking though.
2017-04-29 18:29:15 +02:00
Christian Schwarz
d9ecfc8eb4
Gofmt megacommit.
2017-04-26 20:29:54 +02:00
Christian Schwarz
9750bf3123
Wireframe main executable.
2017-04-26 20:22:17 +02:00
