extern/zrepl
1
0
Fork 0
mirror of https://github.com/zrepl/zrepl.git synced 2024-11-30 04:15:30 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
373 Commits 48 Branches 23 Tags 3.6 MiB
d677cde6d0
Commit Graph

197 Commits

Author SHA1 Message Date
Christian Schwarz
d677cde6d0 implement tcp and tcp+tls transports 2018-08-25 12:58:17 +02:00
Christian Schwarz
54c9dcb7c1 move replication policy constants to package replication 2018-08-22 10:11:14 +02:00
Christian Schwarz
9b537ec704 simplify naming in endpoint package 2018-08-22 10:05:21 +02:00
Christian Schwarz
70aad0940f cmd: move replication endpoints into subpackage 2018-08-22 00:43:58 +02:00
Christian Schwarz
7b3a84e2a3 move replication package to project root (independent of cmd package) 2018-08-22 00:19:03 +02:00
Christian Schwarz
301c7b2dd5 restructure and rename, making mainfsm the replication package itself 2018-08-22 00:14:12 +02:00
Christian Schwarz
2f205d205b remove EndpointPair abstraction 2018-08-21 22:15:00 +02:00
Christian Schwarz
38532abf45 enforce encapsulation by breaking up replication package into packages 
not perfect yet, public shouldn't be required to use 'common' package to
use replication package
 2018-08-16 21:05:21 +02:00
Christian Schwarz
c7d28fee8f gofmt 2018-08-16 14:02:33 +02:00
Christian Schwarz
bf1e626b9a proper queue abstraction 2018-08-16 14:02:16 +02:00
Christian Schwarz
93929b61e4 propert locking on FSReplication 2018-08-16 12:01:51 +02:00
Christian Schwarz
5479463783 always use ReplicationState, and have a map from that to the rsfs 2018-08-16 11:02:34 +02:00
Christian Schwarz
094eced2c7 WIP: states with updater func instead of direct locking 2018-08-16 01:26:09 +02:00
Christian Schwarz
991f13a3da Reporting 2018-08-15 20:29:49 +02:00
Christian Schwarz
7303d91abf WIP state-machine based replication 2018-08-11 12:19:10 +02:00
Christian Schwarz
c1f3076eb3 WIP2 logging done somewhat 2018-08-10 17:06:00 +02:00
Christian Schwarz
74445a0017 fixup 2018-08-08 13:12:50 +02:00
Christian Schwarz
a0b320bfeb streamrpc now requires net.Conn => use it instead of rwc everywhere 2018-08-08 13:09:51 +02:00
Christian Schwarz
1826535e6f WIP 2018-07-15 17:36:53 +02:00
Christian Schwarz
1a8d2c5ebe replication: context support and propert closing of stale readers 2018-07-08 23:31:46 +02:00
Christian Schwarz
8cca0a8547 Initial working version 
Summary:
* Logging is still bad
* test output in a lot of placed
* FIXMEs every where

Test Plan: None, just review

Differential Revision: https://phabricator.cschwarz.com/D2
 2018-06-24 10:44:00 +02:00
Christian Schwarz
0918ef6815 WIP: diffing and replication algorithm 2018-05-02 21:26:24 +02:00
Christian Schwarz
9d7110eaad config: fix shadowed error return values 2018-04-14 11:25:12 +02:00
Christian Schwarz
82ea535692 daemon: expose prometheus in new global.monitoring config section + document it 
refs #67
 2018-04-14 11:24:47 +02:00
Christian Schwarz
a4da029105 cmd: prometheus job type and Task instrumentation 
refs #67
 2018-04-13 23:37:53 +02:00
Christian Schwarz
aa3865d0a3 daemon: Job types as dedicated type 
refs #67
 2018-04-05 22:22:55 +02:00
Christian Schwarz
0895e02844 daemon: Task: track relation to parent job 
refs #67
 2018-04-05 22:18:22 +02:00
Christian Schwarz
26b436463d ssh+stdinserver: connect: dial_timeout 
This  is a follow-up to ccd062e
 2018-03-04 17:19:41 +01:00
Christian Schwarz
aa92261ea7 bookmarking: prune policy for bookmarks 
refs #34
 2018-02-17 20:48:31 +01:00
Christian Schwarz
8e34843eb1 autosnap: do not treat zero fs filter results as fatal 2018-02-17 19:27:00 +01:00
Christian Schwarz
bfaf6fdfbb daemon: fix missing newline on parse error 2018-02-17 17:43:55 +01:00
Christian Schwarz
f992fed968 control pprof rewrite: expose pprof metrics via HTTP server controlled from CLI 2018-02-17 16:20:10 +01:00
Christian Schwarz
94967b596c docs: document changes to ssh+stdinserver transport implementation: ccd062e 2018-02-17 15:16:29 +01:00
Christian Schwarz
f3d3a7f5f8 stdinserver: fixup ccd062e: assert socket is in private directory 2018-02-17 14:12:44 +01:00
Christian Schwarz
ccd062e238 ssh+stdinserver: dump sshbytestream for github.com/problame/go-netssh 
Cleaner abstractions + underlying go-rwccmd package does proper handling
of asynchronous exits, etc.
 2018-02-17 01:08:15 +01:00
Christian Schwarz
6b5bd0a43c job pull + source: fix broken connection teardown 
Issue #56 shows zombie SSH processes.
We fix this by actually Close()ing the RWC in job pull.
If this fixes #56 it also fixes #6 --- it's the same issue.

Additionally, debugging around this revealed another issue: just
Close()ing the sshbytestream in job source will apparently outpace the
normal data stream of stdin and stdout (URG or PUSH flags?).  leading
to ugly errors in the logs.
With proper TCP connections, we would simply set the connection to
linger and close it, letting the kernel handle the final timeout. Meh.

refs #56
refs #6
 2018-02-16 20:57:27 +01:00
Christian Schwarz
921bccb960 job source: use task logger 2018-02-15 23:51:57 +01:00
Christian Schwarz
5f2c14adab zfs: use custom datatype to pass ZFS properties in ZFSSet 
refs #55
 2018-01-05 18:42:10 +01:00
Christian Schwarz
787675aee8 control status command: only show verbose logs on user request 2017-12-30 13:53:19 +01:00
Christian Schwarz
01e0519b7b control status subcommand: fix typo in usage 2017-12-30 13:44:55 +01:00
Christian Schwarz
8742b7f763 handler: fix typo in log message 2017-12-30 13:29:04 +01:00
Christian Schwarz
56f13741f9 test pattern subcommand: better example command 2017-12-29 22:45:38 +01:00
Christian Schwarz
61842988b9 Task & TaskStatus: DeepCopy(): actually copy lastUpdate field 
otherwise, only changes to activity level would udpate TaskStatus
LastUpdate field

refs #10
 2017-12-29 21:43:12 +01:00
Christian Schwarz
be7176bee7 Puller: fix wrong filesystem log field usage 
was introduced in 9465b593
 2017-12-29 21:25:42 +01:00
Christian Schwarz
839eccf513 logger.Outlet: WriteEntry must not block 
- make TCPOutlet fully asynchronous, dropping messages if connection is
  not fast enough
- syslog is just fine for now, local anyways
- stdout same thing

refs #26
 2017-12-29 17:21:58 +01:00
Christian Schwarz
acd9aedb98 cmd control status: unify job logs, option to show only one job & always show logs 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
835cf6b12f cmd control status: warn about inactive tasks 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
4b3d83ec1f TaskStatus: add LastUpdate field 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
d13c6e3fc3 job local: refactor + use Task API 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
63fa7a67e9 job source: refactor + use Task API 
refs #10
 2017-12-27 18:34:24 +01:00