ab3e783168
rpc: treat protocol handshake errors as permanent
...
treat handshake errors as permanent on the client
The issue was observed by 100% CPU usage due to lack ofrate-limiting in
dataconn.ReqPing retries=> safeguard that
2019-03-15 16:18:01 +01:00
7584c66bdb
pruner: remove retry handling + fix early give-up
...
Retry handling is broken since the gRPC changes (wrong error classification).
Will come back at some point, hopefully by merging the replication
driver retry infrastructure.
However, the simpler architecture allows an easy fix for the problem
that the pruner practically gave up on the first error it encountered.
fixes #123
2019-03-13 21:04:39 +01:00
d78d20e2d0
pruner: skip placeholders + FSes without correspondents on source
...
fixes #126
2019-03-13 20:42:37 +01:00
b85ec52387
rpc/ctrl: nicer perr info debug log messages
2019-03-13 19:20:04 +01:00
edcd258cc9
replication: more elaborate messages for Conflict errors
2019-03-13 18:46:04 +01:00
d5250bbf51
client/status: fix wrap for multiline strings with leading space
2019-03-13 18:46:04 +01:00
d50e553ebb
handle changes to placeholder state correctly
...
We assumed that `zfs recv -F FS` would basically replace FS inplace, leaving its children untouched.
That is in fact not the case, it only works if `zfs send -R` is set, which we don't do.
Thus, implement the required functionality manually.
This solves a `zfs recv` error that would occur when a filesystem previously created as placeholder on the receiving side becomes a non-placeholder filesystem (likely due to config change on the sending side):
zfs send pool1/foo@1 | zfs recv -F pool1/bar
cannot receive new filesystem stream:
destination has snapshots (eg. pool1/bar)
must destroy them to overwrite it
2019-03-13 18:46:04 +01:00
1eb0f12a61
replication: add diff test case
2019-03-13 18:45:40 +01:00
8129ed91f1
zfs + replication: migrate dead zfs/diff_test.go to replication/logic/diff
...
(and remove the dead code from package zfs)
2019-03-13 16:39:10 +01:00
c87759affe
replication/driver: automatic retries on connectivity-related errors
2019-03-13 15:00:40 +01:00
07b43bffa4
replication: refactor driving logic (no more explicit state machine)
2019-03-13 15:00:40 +01:00
0230c6321f
rpc/dataconn: microbenchmark
2019-03-13 13:57:21 +01:00
796c5ad42d
rpc rewrite: control RPCs using gRPC + separate RPC for data transfer
...
transport/ssh: update go-netssh to new version
=> supports CloseWrite and Deadlines
=> build: require Go 1.11 (netssh requires it)
2019-03-13 13:53:48 +01:00
d281fb00e3
socketpair: directly export *net.UnixConn (and add test for that behavior)
2019-03-13 11:36:34 +01:00
76a6c623f3
tlsconf and transport/tls: support NSS-formatted keylog file for debugging
...
... via env variable
2019-03-13 00:28:38 +01:00
25c974f0b5
envconst: support for int64
2019-03-13 00:07:33 +01:00
ea719f5b5a
build: use 'git describe --always' to determine ZREPL_VERSION
2019-03-13 00:07:33 +01:00
3105fa4ff8
build: use dep's required feature for dev tools
2019-03-12 23:43:39 +01:00
f724480c7b
Add documentation regarding using a certificate chain
2019-01-22 10:09:24 -08:00
5c5e8c0baf
Documentation changes mostly as requested
2019-01-22 16:46:34 +01:00
bb5278fe9b
Permit peers to provide a cert chain (multiple certs). fixes #103
2019-01-09 10:10:37 -08:00
38b0bd76f5
build: just use go {test,vet} ./... for targets vet, test and generate
2018-12-11 22:00:03 +01:00
c1aab0bee9
config: update yaml-config and use zeropositive constraint for timeouts
2018-12-11 21:54:36 +01:00
ef3283638a
logger: add stderrlogger (sometimes useful)
2018-12-11 21:24:54 +01:00
68b62a5c00
tlsconf: clear handshake deadline after completed handshake
2018-12-11 21:24:26 +01:00
7a75a4d384
util/iocommand: timeout kill on close + other hardening
2018-12-11 21:06:54 +01:00
1aae7b222f
docs: fix confusing description of the role of client identity for sink jobs
2018-12-01 15:19:59 +01:00
3535b251ab
freeze Go build dependencies in Gopkg.lock
...
* use pseudo-depdencies in build/build.go to convince dep
* update Travis, Dockerfile and Docs
* build.Dockerfile image now contains the Go build dependencies
* => faster builds
* bump pdu file after protoc update
fixes #106
2018-12-01 14:36:40 +01:00
707f070a3c
build: fix dirty detection at the end of release build
...
was using Bashisms
2018-12-01 14:36:40 +01:00
160a3b6d32
more gofmt, drop snapjob.go_prefmt after it was accidentally added
2018-11-21 22:14:43 +01:00
d977796f18
Add SnapJob docs
2018-11-21 16:59:46 +01:00
3cef76d463
Refactor snapJob() to snapJobFromConfig()
2018-11-21 14:37:03 +01:00
e9564a7e5c
Inlined a couple legacy leftover functions from the mode copypasta
2018-11-21 14:35:40 +01:00
b79ad3ddc3
Honour PruneKeepNotReplicated.KeepSnashotAtCursor in SinglePrunerFactory
2018-11-21 14:17:38 +01:00
d0f898751f
Gofmt snapjob.go
2018-11-21 14:02:21 +01:00
22d9830baa
Fix prometheus with multiple jobs
2018-11-21 04:26:03 +01:00
c4e23862cd
Added status view for SnapJob.
2018-11-21 04:06:13 +01:00
e10dc129de
Make getPruner() private
2018-11-21 03:39:03 +01:00
dd11fc96db
Touchups in job.go
2018-11-21 03:27:39 +01:00
7de3c0a09a
Removed the references to a pruning 'side' in the singlepruner logging code and the snapjob prometheus thing.
2018-11-21 02:52:33 +01:00
141e49727c
Missed a last reference to tasks
2018-11-21 02:51:23 +01:00
442d61918b
remove most of the watchdog machinery
2018-11-21 02:42:13 +01:00
58dcc07430
Added SnapJobStatus
2018-11-21 02:08:39 +01:00
19d0916e34
remove snapMode, rename snap_ActiveSide to SnapJob
2018-11-21 01:54:56 +01:00
1265cc7934
pruned unused lines and comments ;)
2018-11-21 01:34:50 +01:00
3d2688e959
Ugly but working inital snapjob implementation
2018-11-20 19:30:15 +01:00
7ab51fad0d
zfs: add 'received' property source, handle 'any' source correctly and use 'any' for placeholder FS detection
...
we want was first noticed in zfs 0.8rc1
Upstream doc PR: https://github.com/zfsonlinux/zfs/pull/8134
2018-11-16 13:07:13 +01:00
3472145df6
pruner + proto change: better handling of missing replication cursor
...
- don't treat missing replication cursor as an error in protocol
- treat it as a per-fs planning error instead
2018-11-16 12:21:54 +01:00
5e1ea21f85
pruning: add 'Negate' option to KeepRegex and expose it in config
2018-11-16 12:21:54 +01:00
2db3977408
cli: add 'test placeholder' subcommand for placeholder debugging
2018-11-16 12:21:54 +01:00
