extern/zrepl
1
0
Fork 0
mirror of https://github.com/zrepl/zrepl.git synced 2024-12-22 15:11:16 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
604 Commits 48 Branches 23 Tags 3.7 MiB
4ee00091d6
Commit Graph

604 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Christian Schwarz
4ee00091d6 pull job: support manual-only invocation 2019-03-16 14:24:05 +01:00
Christian Schwarz
c655622bf7 build: travis: allow_failures of go:master 2019-03-15 22:43:18 +01:00
Christian Schwarz
2c3b3c093d rpc: do not leak grpc state change logger goroutine 2019-03-15 16:18:01 +01:00
Christian Schwarz
ab3e783168 rpc: treat protocol handshake errors as permanent 
treat handshake errors as permanent on the client

The issue was observed by 100% CPU usage due to lack ofrate-limiting in
dataconn.ReqPing retries=> safeguard that
 2019-03-15 16:18:01 +01:00
Christian Schwarz
7584c66bdb pruner: remove retry handling + fix early give-up 
Retry handling is broken since the gRPC changes (wrong error classification).
Will come back at some point, hopefully by merging the replication
driver retry infrastructure.

However, the simpler architecture allows an easy fix for the problem
that the pruner practically gave up on the first error it encountered.

fixes #123
 2019-03-13 21:04:39 +01:00
Christian Schwarz
d78d20e2d0 pruner: skip placeholders + FSes without correspondents on source 
fixes #126
 2019-03-13 20:42:37 +01:00
Christian Schwarz
b85ec52387 rpc/ctrl: nicer perr info debug log messages 2019-03-13 19:20:04 +01:00
Christian Schwarz
edcd258cc9 replication: more elaborate messages for Conflict errors 2019-03-13 18:46:04 +01:00
Christian Schwarz
d5250bbf51 client/status: fix wrap for multiline strings with leading space 2019-03-13 18:46:04 +01:00
Christian Schwarz
d50e553ebb handle changes to placeholder state correctly 
We assumed that `zfs recv -F FS` would basically replace FS inplace, leaving its children untouched.
That is in fact not the case, it only works if `zfs send -R` is set, which we don't do.

Thus, implement the required functionality manually.

This solves a `zfs recv` error that would occur when a filesystem previously created as placeholder on the receiving side becomes a non-placeholder filesystem (likely due to config change on the sending side):

  zfs send pool1/foo@1 | zfs recv -F pool1/bar
  cannot receive new filesystem stream:
  destination has snapshots (eg. pool1/bar)
  must destroy them to overwrite it
 2019-03-13 18:46:04 +01:00
Christian Schwarz
1eb0f12a61 replication: add diff test case 2019-03-13 18:45:40 +01:00
Christian Schwarz
8129ed91f1 zfs + replication: migrate dead zfs/diff_test.go to replication/logic/diff 
(and remove the dead code from package zfs)
 2019-03-13 16:39:10 +01:00
Christian Schwarz
c87759affe replication/driver: automatic retries on connectivity-related errors 2019-03-13 15:00:40 +01:00
Christian Schwarz
07b43bffa4 replication: refactor driving logic (no more explicit state machine) 2019-03-13 15:00:40 +01:00
Christian Schwarz
0230c6321f rpc/dataconn: microbenchmark 2019-03-13 13:57:21 +01:00
Christian Schwarz
796c5ad42d rpc rewrite: control RPCs using gRPC + separate RPC for data transfer 
transport/ssh: update go-netssh to new version
    => supports CloseWrite and Deadlines
    => build: require Go 1.11 (netssh requires it)
 2019-03-13 13:53:48 +01:00
Christian Schwarz
d281fb00e3 socketpair: directly export *net.UnixConn (and add test for that behavior) 2019-03-13 11:36:34 +01:00
Christian Schwarz
76a6c623f3 tlsconf and transport/tls: support NSS-formatted keylog file for debugging 
... via env variable
 2019-03-13 00:28:38 +01:00
Christian Schwarz
25c974f0b5 envconst: support for int64 2019-03-13 00:07:33 +01:00
Christian Schwarz
ea719f5b5a build: use 'git describe --always' to determine ZREPL_VERSION 2019-03-13 00:07:33 +01:00
Christian Schwarz
3105fa4ff8 build: use dep's required feature for dev tools 2019-03-12 23:43:39 +01:00
Christian Schwarz
38b0bd76f5 build: just use go {test,vet} ./... for targets vet, test and generate 2018-12-11 22:00:03 +01:00
Christian Schwarz
c1aab0bee9 config: update yaml-config and use zeropositive constraint for timeouts 2018-12-11 21:54:36 +01:00
Christian Schwarz
ef3283638a logger: add stderrlogger (sometimes useful) 2018-12-11 21:24:54 +01:00
Christian Schwarz
68b62a5c00 tlsconf: clear handshake deadline after completed handshake 2018-12-11 21:24:26 +01:00
Christian Schwarz
7a75a4d384 util/iocommand: timeout kill on close + other hardening 2018-12-11 21:06:54 +01:00
Christian Schwarz
1aae7b222f docs: fix confusing description of the role of client identity for sink jobs 2018-12-01 15:19:59 +01:00
Christian Schwarz
3535b251ab freeze Go build dependencies in Gopkg.lock 
* use pseudo-depdencies in build/build.go to convince dep
* update Travis, Dockerfile and Docs
* build.Dockerfile image now contains the Go build dependencies
* => faster builds
* bump pdu file after protoc update

fixes #106
 2018-12-01 14:36:40 +01:00
Christian Schwarz
707f070a3c build: fix dirty detection at the end of release build 
was using Bashisms
 2018-12-01 14:36:40 +01:00
Christian Schwarz
7ab51fad0d zfs: add 'received' property source, handle 'any' source correctly and use 'any' for placeholder FS detection 
we want was first noticed in zfs 0.8rc1

Upstream doc PR: https://github.com/zfsonlinux/zfs/pull/8134
 2018-11-16 13:07:13 +01:00
Christian Schwarz
3472145df6 pruner + proto change: better handling of missing replication cursor 
- don't treat missing replication cursor as an error in protocol
- treat it as a per-fs planning error instead
 2018-11-16 12:21:54 +01:00
Christian Schwarz
5e1ea21f85 pruning: add 'Negate' option to KeepRegex and expose it in config 2018-11-16 12:21:54 +01:00
Christian Schwarz
2db3977408 cli: add 'test placeholder' subcommand for placeholder debugging 2018-11-16 12:21:54 +01:00
Christian Schwarz
ca6d5d3bb5 build: Travis CI configuration 2018-11-16 12:10:58 +01:00
Christian Schwarz
163c2bc533 docs: update requirements.txt 2018-11-16 12:10:58 +01:00
Christian Schwarz
dd286aa12e client: fix status bytes per second measurement 
still far from perfect, but better than incorrect values
 2018-11-05 01:37:51 +01:00
Christian Schwarz
80babe3ab4 docs/README: update package hierarchy overview 2018-10-26 22:05:57 +02:00
Christian Schwarz
ca0cab0a15 docs/tutorial: fix headlines 2018-10-26 21:52:49 +02:00
JMoVS
ad8be226fd fix small typo 2018-10-22 11:32:37 +02:00
Christian Schwarz
9b3e5c38e2 docs: fix changelog + invocations of wakeup subcommand 2018-10-22 11:27:00 +02:00
Christian Schwarz
7e1c5f5d1f docs: discourage use of ssh+stdinserver transport due to inferior error handling 2018-10-22 11:25:16 +02:00
Christian Schwarz
98bc8d1717 daemon/job: explicit notice of ZREPL_JOB_WATCHDOG_TIMEOUT environment variable on cancellation 2018-10-22 11:03:31 +02:00
Christian Schwarz
2889a5d5ff client/status: current bytes/second + spinning progress bar 2018-10-21 23:15:21 +02:00
Christian Schwarz
0b8c19c620 docs/tutorial: switch to push setup & use mutual TLS (2 machines) 2018-10-21 22:20:35 +02:00
Christian Schwarz
a62b475f46 docs/transport/tls: document self-signed certs procedure for 2-machine setup 2018-10-21 22:20:07 +02:00
Christian Schwarz
1691839c6b replication: handle context cancellation errors as GlobalError 2018-10-21 19:06:35 +02:00
Christian Schwarz
36265ff349 fixup 438f950be3: forgotten ErrorCount in printf 2018-10-21 18:37:57 +02:00
Christian Schwarz
94427d334b replication + pruner + watchdog: adjust timeouts based on practical experience 2018-10-21 18:37:57 +02:00
Christian Schwarz
b2844569c8 replication: rewrite error handling + simplify state machines 
* Remove explicity state machine code for all but replication.Replication
* Introduce explicit error types that satisfy interfaces which provide
  sufficient information for replication.Replication to make intelligent
  retry + queuing decisions

  * Temporary()
  * LocalToFS()

* Remove the queue and replace it with a simple array that we sort each
  time (yay no generics :( )
 2018-10-21 18:37:57 +02:00
Christian Schwarz
ae5e60b1ae client/status: display problems as wrapped + indented if they do not fit the current line 2018-10-21 17:50:08 +02:00