Commit Graph

36 Commits

Author SHA1 Message Date
Christian Schwarz
a0f72b585b remove JobStatus, Task abstraction and 'control status' subcommand
Control status will be replaced by job-specific output at some point.

Task was not useful anymore with state machine, may reintroduce
something similar at a later point, but consider alternatives:

- opentracing.io
- embedding everything in ctx
	- activity stack would work easily
	- log entries via proxy logger.Logger object
- progress reporting should be in status reports of individial jobs
2018-08-26 19:08:30 +02:00
Christian Schwarz
f6be5b776b cmd: clean up usage of contextKeyLog through getter and setter functions 2018-08-26 14:58:57 +02:00
Christian Schwarz
666ead2646 make go vet happy 2018-08-26 14:51:20 +02:00
Christian Schwarz
ea0e3a29e4 fixup 88de8ba8bb: gofmt 2018-08-25 22:30:44 +02:00
Christian Schwarz
88de8ba8bb initial repl policy: get rid of unimplemented options 2018-08-25 22:23:47 +02:00
Christian Schwarz
e30ae972f4 gofmt 2018-08-25 21:30:25 +02:00
Christian Schwarz
54c9dcb7c1 move replication policy constants to package replication 2018-08-22 10:11:14 +02:00
Christian Schwarz
9b537ec704 simplify naming in endpoint package 2018-08-22 10:05:21 +02:00
Christian Schwarz
70aad0940f cmd: move replication endpoints into subpackage 2018-08-22 00:43:58 +02:00
Christian Schwarz
7b3a84e2a3 move replication package to project root (independent of cmd package) 2018-08-22 00:19:03 +02:00
Christian Schwarz
301c7b2dd5 restructure and rename, making mainfsm the replication package itself 2018-08-22 00:14:12 +02:00
Christian Schwarz
38532abf45 enforce encapsulation by breaking up replication package into packages
not perfect yet, public shouldn't be required to use 'common' package to
use replication package
2018-08-16 21:05:21 +02:00
Christian Schwarz
7303d91abf WIP state-machine based replication 2018-08-11 12:19:10 +02:00
Christian Schwarz
c1f3076eb3 WIP2 logging done somewhat 2018-08-10 17:06:00 +02:00
Christian Schwarz
1826535e6f WIP 2018-07-15 17:36:53 +02:00
Christian Schwarz
8cca0a8547 Initial working version
Summary:
* Logging is still bad
* test output in a lot of placed
* FIXMEs every where

Test Plan: None, just review

Differential Revision: https://phabricator.cschwarz.com/D2
2018-06-24 10:44:00 +02:00
Christian Schwarz
aa3865d0a3 daemon: Job types as dedicated type
refs #67
2018-04-05 22:22:55 +02:00
Christian Schwarz
0895e02844 daemon: Task: track relation to parent job
refs #67
2018-04-05 22:18:22 +02:00
Christian Schwarz
aa92261ea7 bookmarking: prune policy for bookmarks
refs #34
2018-02-17 20:48:31 +01:00
Christian Schwarz
d13c6e3fc3 job local: refactor + use Task API
refs #10
2017-12-27 18:34:24 +01:00
Christian Schwarz
b69089a527 Puller: refactor + use Task API
* drop rx byte count functionality
* will be re-added to Task as necessary

refs #10
2017-12-27 14:39:47 +01:00
Christian Schwarz
59e34942d1 Puller: make main interface public
refs #10
2017-12-27 14:39:46 +01:00
Christian Schwarz
58ee796394 adopt Task API: infect datastructures
refs #10
2017-12-27 14:39:46 +01:00
Christian Schwarz
8c7e373049 daemon: DaemonStatus + JobStatus + dummy implementation
refs #10
2017-12-27 14:39:46 +01:00
Christian Schwarz
583a63a68f refactor: encapsulate pulling in a struct
refs #10
2017-12-24 15:23:28 +01:00
Christian Schwarz
51af880701 refactor: parametrize PrefixFilter VersionType check
refs #34
2017-11-13 10:59:22 +01:00
Christian Schwarz
83d450b1f2 config: support days (d) and weeks (w) in durations
fixes #18
2017-10-05 15:17:37 +02:00
Christian Schwarz
c31ec8c646 convert more code to structured logging 2017-09-23 17:52:29 +02:00
Christian Schwarz
9465b593f9 cmd: configurable logrus formatters
We lost the nice context-stack [jobname][taskname][...] at the beginning
of each log line when switching to logrus.

Define some field names that define these contexts.
Write a human-friendly formatter that presents these field names like
the solution we had before logrus.

Write some other formatters for logfmt and json output along the way.

Limit ourselves to stdout logging for now.
2017-09-23 11:24:36 +02:00
Christian Schwarz
bfcba7b281 cmd: logging using logrus 2017-09-22 17:01:54 +02:00
Christian Schwarz
9cd83399d3 cmd: remove global state in main.go
* refactoring
* Now supporting default config locations
2017-09-17 18:32:00 +02:00
Christian Schwarz
6a05e101cf WIP daemon:
Implement
* pruning on source side
* local job
* test subcommand for doing a dry-run of a prune policy

* use a non-blocking callback from autosnap to trigger the depending
jobs -> avoids races, looks saner in the debug log
2017-09-16 21:13:19 +02:00
Christian Schwarz
e3ec093d53 cmd: handler: check FilesystemVersionFilter as part of ACL 2017-09-16 20:24:46 +02:00
Christian Schwarz
e70b6f3071 WIP: recurring jobs
Done:

* implement autosnapper that asserts interval between snapshots
* implement pruner

* job pull: pulling + pruning
* job source: autosnapping + serving

TODO

* job source: pruning
* job local: everything
* fatal errors such as serve that cannot bind socket must be more
visible
* couldn't things that need a snapshotprefix just use a interface
Prefixer() instead? then we could have prefixsnapshotfilter and not
duplicate it every time...
* either go full context.Context or not at all...? just wait because
community climate around it isn't that great and we only need it for
cancellation? roll our own?
2017-09-15 19:35:19 +02:00
Christian Schwarz
1deaa459c8 config: unify job debugging options 2017-09-11 15:45:10 +02:00
Christian Schwarz
73c9033583 WIP: Switch to new config format.
Don't use jobrun for daemon, just call JobDo() once, the job must
organize stuff itself.

Sacrifice all the oneshot commands, they will be reintroduced as
client-calls to the daemon.
2017-09-10 17:53:54 +02:00