Christian Schwarz
75e42fd860
pruner: implement Report method + display in status command
2018-09-24 19:25:40 +02:00
Christian Schwarz
75ba5874a5
active side: track activities in Run() as atomically updated member
2018-09-24 19:23:53 +02:00
Christian Schwarz
1ce0c69e4f
implement local replication using new local transport
...
The new local transport uses socketpair() and a switchboard based on
client identities.
The special local job type is gone, which is good since it does not fit
into the 'Active/Passive side ' + 'mode' concept used to implement the
duality of push/sink | pull/source.
2018-09-24 14:43:53 +02:00
Christian Schwarz
f3e8eda04d
fixup 4e04f8d3d2: snapper with separate stopped state for clean shutdown
...
would tight loop in ErrorWait
2018-09-24 14:40:47 +02:00
Christian Schwarz
4e04f8d3d2
snapper: make error mode an error wait mode
...
Just because taking one snapshot fails does not mean snapper needs to
stop for all others.
Since users are advised to monitor error logs, snapshot-taking errors
can still be addressed.
The ErrorWait mode allows a potential future Report / Status command to
distinguish normal waits from error waits.
2018-09-24 12:36:10 +02:00
Christian Schwarz
d04b9713c4
implement pull + sink modes for active and passive side
2018-09-24 12:36:10 +02:00
Christian Schwarz
ffe33aff3d
fix pruner: protobuf one-ofs require non-zero value, even if no public fields
2018-09-24 12:36:10 +02:00
Christian Schwarz
e3be120d88
refactor push + source into active + passive 'sides' with push and source 'modes'
2018-09-24 12:36:10 +02:00
Christian Schwarz
9446b51a1f
status: infra for reporting jobs instead of just replication.Report
2018-09-23 21:11:33 +02:00
Christian Schwarz
4a6160baf3
update to streamrpc 0.4 & adjust config (not breaking)
2018-09-23 20:28:30 +02:00
Christian Schwarz
7f9eb62640
sink: concurrent connection handling
2018-09-18 22:44:00 +02:00
Christian Schwarz
6c3f442f13
daemon control / jsonclient: fix connection leak due to open request body
...
Also:
- Defensive measures in control http server (1s timeouts)
(prevent the leak, even if request body is not closed)
- Add prometheus metrics to track control socket latencies
(were used for debugging)
2018-09-13 12:44:46 +02:00
Christian Schwarz
fa47667f31
bring back prometheus metrics, with new metrics for replication state machine
2018-09-07 22:22:34 -07:00
Christian Schwarz
7836ea36fc
serve TLS: validate client CNs against whitelist in config file
2018-09-06 13:34:39 -07:00
Christian Schwarz
0f75677e59
daemon/pruner: fix exercise (don't call it test)
2018-09-05 21:47:44 -07:00
Christian Schwarz
2c25f28972
simplify mapping & filtering in endpoints (re-rooting only)
2018-09-05 19:51:06 -07:00
Christian Schwarz
975fdee217
replication & pruning: ditch replicated-property, use bookmark as cursor instead
...
A bookmark with a well-known name is used to track which version was
last successfully received by the receiver.
The createtxg that can be retrieved from the bookmark using `zfs get` is
used to set the Replicated attribute of each snap on the sender:
If the snap's CreateTXG > the cursor's, it is not yet replicated,
otherwise it has been.
There is an optional config option to change the behvior to
`CreateTXG >= the cursor's`, and the implementation defaults to that.
The reason: While things work just fine with `CreateTXG > the cursor's`,
ZFS does not provide size estimates in a `zfs send` dry run
(see acd2418
2018-09-05 19:51:06 -07:00
Christian Schwarz
9eca269ad8
fixup 308e5e35fb: remove fprintf debug output
2018-09-05 08:35:31 -07:00
Christian Schwarz
c21222ef13
serve/tls: use handshake timeout from config
2018-09-05 08:32:59 -07:00
Christian Schwarz
adab06405b
make go vet happy
2018-09-04 17:25:10 -07:00
Christian Schwarz
bfc631f6a6
fix broken pruner exercise (don't call it test...)
2018-09-04 17:19:59 -07:00
Christian Schwarz
308e5e35fb
Multi-client servers + bring back stdinserver support
2018-09-04 16:43:55 -07:00
Christian Schwarz
e161347e47
Implement periodic snapshotting.
2018-09-04 16:43:55 -07:00
Christian Schwarz
754b253043
config: no-field for replication anymore
...
It's closer to the original config and we don't want users to specify
'filesystems' and similar multiple times in a single job definition.
2018-09-04 14:44:45 -07:00
Christian Schwarz
0c4a3f8dc4
pruning/history: properly communicate via rpc if snapshot does not exist
2018-09-04 14:01:48 -07:00
Christian Schwarz
8799108b55
fixup b95e983d0d: prunerFactory: fix duplicate logger fields
2018-09-03 13:19:56 -07:00
Christian Schwarz
03f9f81cb5
fixup 3d8e552c6a: validate streamrpc config in factory constructors
2018-09-03 13:17:53 -07:00
Christian Schwarz
3d8e552c6a
streamrpc 0.3 + config from daemon/config
2018-09-02 15:46:42 -07:00
Christian Schwarz
d55a271ac7
WIP adopt updated yaml-config with 'fromdefaults' struct tag
2018-09-02 15:46:03 -07:00
Christian Schwarz
b95e983d0d
bump go-streamrpc to 0.2, cleanup logging
...
logging should be user-friendly in INFO mode
2018-09-02 15:45:18 -07:00
Anton Schirg
32391adf4f
build pruner in factory and check prune rules
2018-08-30 19:20:14 +02:00
Anton Schirg
c0a3e1f121
wrap error in buildJob with job name
2018-08-30 17:40:02 +02:00
Christian Schwarz
1690339440
colorized stdout logger if stdout is tty
2018-08-30 13:33:28 +02:00
Anton Schirg
6cedd0a2e8
add status command
2018-08-30 12:54:30 +02:00
Christian Schwarz
7dd49b835a
finish pruning implementation in push job
2018-08-30 11:52:05 +02:00
Christian Schwarz
12dd240b5f
fixup pruner
2018-08-30 11:49:06 +02:00
Christian Schwarz
a2aa8e7bd7
finish pruner implementation
2018-08-29 19:00:45 +02:00
Christian Schwarz
c69ebd3806
WIP rewrite the daemon
...
cmd subdir does not build on purpose, it's only left in tree to grab old
code and move it to github.com/zrepl/zrepl/daemon
2018-08-27 22:22:44 +02:00
Christian Schwarz
df6e1bc64d
privatize pprofServer
2018-08-27 19:13:35 +02:00
Christian Schwarz
89dc267780
start implementing new daemon in package daemon
2018-08-27 19:10:55 +02:00
