extern/zrepl
1
0
Fork 0
mirror of https://github.com/zrepl/zrepl.git synced 2024-11-26 02:14:44 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
774 Commits 48 Branches 23 Tags 3.6 MiB
5e17d7ba80
Commit Graph

121 Commits

Author SHA1 Message Date
Christian Schwarz
ad0b055245 daemon/prometheus: fix crash if listener cannot be created 
refs #238

  zrepl version=v0.2.0-11-gdc39c81 GOOS=linux GOARCH=amd64 Compiler=gc
 starting daemon
 [pull_source]: starting job
 [_prometheus]: starting job
 [connection_loss_tidyup]: starting job
 [connection_loss_tidyup]: wait for wakeups
 [_control]: starting job
 [_prometheus]: cannot listen err="listen tcp 10.0.0.200:9091: bind: cannot assign requested add
 [_prometheus]: job exited
 panic: runtime error: invalid memory address or nil pointer dereference
         panic: runtime error: invalid memory address or nil pointer dereference
 [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x81ea4d]
 goroutine 25 [running]:
 net/http.(*onceCloseListener).close(...)
         /usr/local/go/src/net/http/server.go:3330
 sync.(*Once).doSlow(0xc00018b060, 0xc0000c7bc0)
         /usr/local/go/src/sync/once.go:66 +0xe3
 sync.(*Once).Do(...)
         /usr/local/go/src/sync/once.go:57
 net/http.(*onceCloseListener).Close(0xc00018b050, 0xc0003a6000, 0xe0)
         /usr/local/go/src/net/http/server.go:3326 +0x77
 panic(0xb1d1c0, 0x11d7d90)
         /usr/local/go/src/runtime/panic.go:679 +0x1b2
 net/http.(*onceCloseListener).Accept(0xc00018b050, 0xc000120020, 0xb0fd20, 0x11d7ce0, 0xbee6e0)
         <autogenerated>:1 +0x32
 net/http.(*Server).Serve(0xc0003a6000, 0x0, 0x0, 0x0, 0x0)
         /usr/local/go/src/net/http/server.go:2896 +0x286
 net/http.Serve(...)
         /usr/local/go/src/net/http/server.go:2468
 github.com/zrepl/zrepl/daemon.(*prometheusJob).Run(0xc000109940, 0xd19ec0, 0xc00018a930)
         /go/src/github.com/zrepl/zrepl/daemon/prometheus.go:75 +0x23e
 github.com/zrepl/zrepl/daemon.(*jobs).start.func1(0xc0000f68c0, 0xd22c40, 0xc000116ee0, 0xd1ba4
         /go/src/github.com/zrepl/zrepl/daemon/daemon.go:220 +0x121
 created by github.com/zrepl/zrepl/daemon.(*jobs).start
         /go/src/github.com/zrepl/zrepl/daemon/daemon.go:216 +0x52e
 zrepl.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
 zrepl.service: Failed with result 'exit-code'.
 2019-11-16 22:11:13 +01:00
Christian Schwarz
b5ff1a9926 snapper + client/status: snapshotting reports 2019-09-27 21:31:00 +02:00
Ross Williams
729c83ee72 pre- and post-snapshot hooks 
* stack-based execution model, documented in documentation
* circbuf for capturing hook output
* built-in hooks for postgres and mysql
* refactor docs, too much info on the jobs page, too difficult
  to discover snapshotting & hooks

Co-authored-by: Ross Williams <ross@ross-williams.net>
Co-authored-by: Christian Schwarz <me@cschwarz.com>

fixes #74
 2019-09-27 21:25:59 +02:00
Ross Williams
00434f4ac9 daemon/logging: format human: treat 'subsystem' field prefixed 
logging.Subsystem != string
=> typecast failed

(Also, nobody guarantees that e.Fields contains `field`)
 2019-09-27 20:39:43 +02:00
Christian Schwarz
a6497b2c6e add platformtest: infrastructure for ZFS compatiblity testing 2019-09-14 13:43:46 +02:00
Christian Schwarz
77d3a1ad4d build: drop go Dep, switch to modules, support Go 1.13 
bump enumer to v1.1.1
bump golangci-lint to v1.17.1

no `go mod tidy` because 1.13 and 1.12 seem to alter each other's output

fixes #112
 2019-09-14 13:36:44 +02:00
Christian Schwarz
921b34235e daemon: env var for autostarting pprof endpoint 2019-09-07 19:50:57 +02:00
Christian Schwarz
b167de3096 filters: add some basic test cases for DatasetMapFilter 2019-07-14 18:28:06 +02:00
Christian Schwarz
f31b54582f daemon/job: fix receiving-job root-fs overlap detection 
fixup for 7756c9a5
fixes #160
 2019-03-27 13:12:26 +01:00
Christian Schwarz
5b97953bfb run golangci-lint and apply suggested fixes 2019-03-27 13:12:26 +01:00
Christian Schwarz
afed762774 format source tree using goimports 2019-03-22 19:41:12 +01:00
Christian Schwarz
0ab62f197a Merge branch 'problame/overlapping-dataset-hierarchy-improvements' into 'master' 
closes #153
 2019-03-22 11:02:57 +01:00
Christian Schwarz
9a2d1b3074 Merge branch 'problame/138-snapshotter-fails-silently' into 'master' 
closes #152
 2019-03-22 10:59:58 +01:00
Christian Schwarz
26ec29d8b2 snapper: retry on errors during syncUp and log them 
fixes #138
 2019-03-21 17:17:10 +01:00
Christian Schwarz
c0028c1c44 daemon/logging: add replication logic logger 2019-03-21 17:03:34 +01:00
Christian Schwarz
7756c9a55c config + job: forbid non-verlapping receiver root_fs 
refs #136
refs #140
 2019-03-21 12:07:55 +01:00
Christian Schwarz
6f7467e8d8 Merge branch 'InsanePrawn-master' into 'master' 2019-03-20 19:45:00 +01:00
Christian Schwarz
5aefc47f71 daemon: remove last traces of watchdog mechanism 2019-03-19 18:15:34 +01:00
Christian Schwarz
158d1175e3 rename SinglePruner to LocalPruner 2019-03-17 21:18:25 +01:00
Christian Schwarz
b25da7b9b0 job: snap: comment fix 2019-03-17 21:07:42 +01:00
Christian Schwarz
5cd2593f52 job: snap: workaround for replication cursor requirement 2019-03-17 21:07:01 +01:00
Christian Schwarz
d8d9e34914 pruner: single: remove unused member considerSnapAtCursorReplicated 2019-03-17 20:57:34 +01:00
Christian Schwarz
17818439a0 Merge branch 'problame/replication_refactor' into InsanePrawn-master 2019-03-17 17:33:51 +01:00
Christian Schwarz
da3ba50a2c Merge remote-tracking branch 'origin/master' into problame/replication_refactor 2019-03-16 14:48:01 +01:00
Christian Schwarz
4ee00091d6 pull job: support manual-only invocation 2019-03-16 14:24:05 +01:00
Christian Schwarz
aff639e87a Merge remote-tracking branch 'origin/master' into InsanePrawn-master 2019-03-15 21:05:20 +01:00
Christian Schwarz
a0f301d700 syslog logging: fix priority parsing + add test for default facility 2019-03-15 18:18:16 +01:00
Ximalas
fc311a9fd6 syslog logging: support setting facility in config 2019-03-15 17:55:11 +01:00
Christian Schwarz
7584c66bdb pruner: remove retry handling + fix early give-up 
Retry handling is broken since the gRPC changes (wrong error classification).
Will come back at some point, hopefully by merging the replication
driver retry infrastructure.

However, the simpler architecture allows an easy fix for the problem
that the pruner practically gave up on the first error it encountered.

fixes #123
 2019-03-13 21:04:39 +01:00
Christian Schwarz
d78d20e2d0 pruner: skip placeholders + FSes without correspondents on source 
fixes #126
 2019-03-13 20:42:37 +01:00
Christian Schwarz
c87759affe replication/driver: automatic retries on connectivity-related errors 2019-03-13 15:00:40 +01:00
Christian Schwarz
07b43bffa4 replication: refactor driving logic (no more explicit state machine) 2019-03-13 15:00:40 +01:00
Christian Schwarz
796c5ad42d rpc rewrite: control RPCs using gRPC + separate RPC for data transfer 
transport/ssh: update go-netssh to new version
    => supports CloseWrite and Deadlines
    => build: require Go 1.11 (netssh requires it)
 2019-03-13 13:53:48 +01:00
InsanePrawn
160a3b6d32 more gofmt, drop snapjob.go_prefmt after it was accidentally added 2018-11-21 22:14:43 +01:00
InsanePrawn
3cef76d463 Refactor snapJob() to snapJobFromConfig() 2018-11-21 14:37:03 +01:00
InsanePrawn
e9564a7e5c Inlined a couple legacy leftover functions from the mode copypasta 2018-11-21 14:35:40 +01:00
InsanePrawn
b79ad3ddc3 Honour PruneKeepNotReplicated.KeepSnashotAtCursor in SinglePrunerFactory 2018-11-21 14:17:38 +01:00
InsanePrawn
d0f898751f Gofmt snapjob.go 2018-11-21 14:02:21 +01:00
InsanePrawn
22d9830baa Fix prometheus with multiple jobs 2018-11-21 04:26:03 +01:00
InsanePrawn
e10dc129de Make getPruner() private 2018-11-21 03:39:03 +01:00
InsanePrawn
dd11fc96db Touchups in job.go 2018-11-21 03:27:39 +01:00
InsanePrawn
7de3c0a09a Removed the references to a pruning 'side' in the singlepruner logging code and the snapjob prometheus thing. 2018-11-21 02:52:33 +01:00
InsanePrawn
141e49727c Missed a last reference to tasks 2018-11-21 02:51:23 +01:00
InsanePrawn
442d61918b remove most of the watchdog machinery 2018-11-21 02:42:13 +01:00
InsanePrawn
58dcc07430 Added SnapJobStatus 2018-11-21 02:08:39 +01:00
InsanePrawn
19d0916e34 remove snapMode, rename snap_ActiveSide to SnapJob 2018-11-21 01:54:56 +01:00
InsanePrawn
1265cc7934 pruned unused lines and comments ;) 2018-11-21 01:34:50 +01:00
InsanePrawn
3d2688e959 Ugly but working inital snapjob implementation 2018-11-20 19:30:15 +01:00
Christian Schwarz
3472145df6 pruner + proto change: better handling of missing replication cursor 
- don't treat missing replication cursor as an error in protocol
- treat it as a per-fs planning error instead
 2018-11-16 12:21:54 +01:00
Christian Schwarz
5e1ea21f85 pruning: add 'Negate' option to KeepRegex and expose it in config 2018-11-16 12:21:54 +01:00