Commit Graph

400 Commits

Author SHA1 Message Date
Anton Schirg
b4ea5f56b2 ssh config 2018-08-26 23:46:59 +02:00
Anton Schirg
5e51595d7f local job config 2018-08-26 23:29:57 +02:00
Anton Schirg
48a08e4f4d config for pull and source 2018-08-26 23:13:16 +02:00
Anton Schirg
e2bf557d17 use optional and default feature of yaml-config 2018-08-26 22:06:47 +02:00
Anton Schirg
fbb8a25320 add prometheus monitoring to config 2018-08-26 22:06:47 +02:00
Anton Schirg
add1b69809 move retentiongrid to own package 2018-08-26 22:06:47 +02:00
Anton Schirg
cd9a428841 rename variable 2018-08-26 22:06:47 +02:00
Anton Schirg
4ec5e23457 set channel buffer to prevent leaking goroutine 2018-08-26 22:06:47 +02:00
Anton Schirg
13dc63bd23 build logger from new config 2018-08-26 22:06:47 +02:00
Anton Schirg
38bb78b642 WIP new config format 2018-08-26 22:03:57 +02:00
Christian Schwarz
6425c26b1b start refactoring: move daemon into subpackage 2018-08-26 21:58:58 +02:00
Christian Schwarz
428339e1ad move version info to separate package 2018-08-26 21:58:20 +02:00
Christian Schwarz
ee5445777d logging format 'human': continue printing prefixed fields if some are missing 2018-08-26 19:13:09 +02:00
Christian Schwarz
a0f72b585b remove JobStatus, Task abstraction and 'control status' subcommand
Control status will be replaced by job-specific output at some point.

Task was not useful anymore with state machine, may reintroduce
something similar at a later point, but consider alternatives:

- opentracing.io
- embedding everything in ctx
	- activity stack would work easily
	- log entries via proxy logger.Logger object
- progress reporting should be in status reports of individial jobs
2018-08-26 19:08:30 +02:00
Christian Schwarz
7ff72fb6d9 replication: document most important aspects of Endpoint interface 2018-08-26 15:12:43 +02:00
Christian Schwarz
f6be5b776b cmd: clean up usage of contextKeyLog through getter and setter functions 2018-08-26 14:58:57 +02:00
Christian Schwarz
666ead2646 make go vet happy 2018-08-26 14:51:20 +02:00
Christian Schwarz
cf01086df5 build: pin protoc version and update protobuf + regenerate 2018-08-26 14:35:18 +02:00
Christian Schwarz
ea0e3a29e4 fixup 88de8ba8bb: gofmt 2018-08-25 22:30:44 +02:00
Christian Schwarz
71203ab325 move various timeouts to package-level variables 2018-08-25 22:30:16 +02:00
Christian Schwarz
88de8ba8bb initial repl policy: get rid of unimplemented options 2018-08-25 22:23:47 +02:00
Christian Schwarz
861e5f8313 special logging fields: from now on only 'job', 'task', 'subsystem' 2018-08-25 22:15:37 +02:00
Christian Schwarz
e30ae972f4 gofmt 2018-08-25 21:30:25 +02:00
Christian Schwarz
e082816de5 fixup d677cde6d0: unused import 2018-08-25 15:16:38 +02:00
Christian Schwarz
b56e236874 add go-streamrpc to Gopkg.toml 2018-08-25 15:14:27 +02:00
Christian Schwarz
f46d1bc338 fixup 70aad0940f: fix broken config_test.go 2018-08-25 13:02:38 +02:00
Christian Schwarz
51cfcfe79b job source: do not stop listener on accept() errors
refs #77
2018-08-25 13:00:51 +02:00
Christian Schwarz
d677cde6d0 implement tcp and tcp+tls transports 2018-08-25 12:58:17 +02:00
Christian Schwarz
873c64ecc3 update README to reflect restructuring 2018-08-22 10:15:27 +02:00
Christian Schwarz
54c9dcb7c1 move replication policy constants to package replication 2018-08-22 10:11:14 +02:00
Christian Schwarz
9b537ec704 simplify naming in endpoint package 2018-08-22 10:05:21 +02:00
Christian Schwarz
70aad0940f cmd: move replication endpoints into subpackage 2018-08-22 00:43:58 +02:00
Christian Schwarz
7b3a84e2a3 move replication package to project root (independent of cmd package) 2018-08-22 00:19:03 +02:00
Christian Schwarz
301c7b2dd5 restructure and rename, making mainfsm the replication package itself 2018-08-22 00:14:12 +02:00
Christian Schwarz
2f205d205b remove EndpointPair abstraction 2018-08-21 22:15:00 +02:00
Christian Schwarz
38532abf45 enforce encapsulation by breaking up replication package into packages
not perfect yet, public shouldn't be required to use 'common' package to
use replication package
2018-08-16 21:05:21 +02:00
Christian Schwarz
c7d28fee8f gofmt 2018-08-16 14:02:33 +02:00
Christian Schwarz
bf1e626b9a proper queue abstraction 2018-08-16 14:02:16 +02:00
Christian Schwarz
93929b61e4 propert locking on FSReplication 2018-08-16 12:01:51 +02:00
Christian Schwarz
5479463783 always use ReplicationState, and have a map from that to the rsfs 2018-08-16 11:02:34 +02:00
Christian Schwarz
094eced2c7 WIP: states with updater func instead of direct locking 2018-08-16 01:26:09 +02:00
Christian Schwarz
991f13a3da Reporting 2018-08-15 20:29:49 +02:00
Christian Schwarz
7303d91abf WIP state-machine based replication 2018-08-11 12:19:10 +02:00
Christian Schwarz
c1f3076eb3 WIP2 logging done somewhat 2018-08-10 17:06:00 +02:00
Christian Schwarz
74445a0017 fixup 2018-08-08 13:12:50 +02:00
Christian Schwarz
a0b320bfeb streamrpc now requires net.Conn => use it instead of rwc everywhere 2018-08-08 13:09:51 +02:00
Christian Schwarz
1826535e6f WIP 2018-07-15 17:36:53 +02:00
Christian Schwarz
1a8d2c5ebe replication: context support and propert closing of stale readers 2018-07-08 23:31:46 +02:00
Christian Schwarz
8cca0a8547 Initial working version
Summary:
* Logging is still bad
* test output in a lot of placed
* FIXMEs every where

Test Plan: None, just review

Differential Revision: https://phabricator.cschwarz.com/D2
2018-06-24 10:44:00 +02:00
Christian Schwarz
fa6426f803 WIP: zfs: hacky resume token parsing 2018-05-02 21:26:56 +02:00