Commit Graph

544 Commits

Author SHA1 Message Date
Christian Schwarz
d13c6e3fc3 job local: refactor + use Task API
refs #10
2017-12-27 18:34:24 +01:00
Christian Schwarz
63fa7a67e9 job source: refactor + use Task API
refs #10
2017-12-27 18:34:24 +01:00
Christian Schwarz
7d89d1fb00 job pull: refactor + use Task API
refs #10
2017-12-27 18:34:24 +01:00
Christian Schwarz
b69089a527 Puller: refactor + use Task API
* drop rx byte count functionality
* will be re-added to Task as necessary

refs #10
2017-12-27 14:39:47 +01:00
Christian Schwarz
59e34942d1 Puller: make main interface public
refs #10
2017-12-27 14:39:46 +01:00
Christian Schwarz
91c4a97f72 Pruner: refactor + use Task API
refs #10
2017-12-27 14:39:46 +01:00
Christian Schwarz
13562b48ed IntervalAutosnap: refactor + use Task API
refs #10
2017-12-27 14:39:46 +01:00
Christian Schwarz
58ee796394 adopt Task API: infect datastructures
refs #10
2017-12-27 14:39:46 +01:00
Christian Schwarz
ce351146cf job control: implement JobStatus 2017-12-27 14:39:46 +01:00
Christian Schwarz
14b8d69a63 cmd control status + expose DaemonStatus via control API
refs #10
2017-12-27 14:39:46 +01:00
Christian Schwarz
8c7e373049 daemon: DaemonStatus + JobStatus + dummy implementation
refs #10
2017-12-27 14:39:46 +01:00
Christian Schwarz
2c87b15e83 daemon: Task abstraction + TaskStatus
An instance of Task tracks a single thread of activity that is part of a Job.

While the docs already use this terminology of tasks being composed of jobs,
the code did not have an object to represent these semantics.
Now it does:

* A task t is initialized with a root activity, which is its name
* t can t.Enter() and t.Finish() an activity, building
  a stack of activities
* t's code can get a logger t.Log() whose logTaskField is set to the
  concatenated stack of activities
* t's code can update IO progress it made since leaving idle state
* t's code's log output vie t.Log() is captured since leaving idle
  state
  * FIXME: find a way to bound that buffer

refs #10
refs #48
2017-12-27 14:39:46 +01:00
Christian Schwarz
d7f3fb93ae bash completions: hidden subcommand + integrate into Makefile 2017-12-27 14:39:46 +01:00
Christian Schwarz
ebf209427a logging: support ignoring fields in HumanFormatter
should be refactored to logger one day so the implementation of ignoring
is not duplicated to each outlet.

refs #10
2017-12-27 14:39:46 +01:00
Christian Schwarz
51377a8ffb logger: support replacing fields
do no delete() from array since this could lead to resizing

refs #10
2017-12-27 14:39:43 +01:00
Christian Schwarz
f14dc3107f logger: implement json.Unmarshaler
refs #10
2017-12-27 13:50:07 +01:00
Christian Schwarz
261d095108 logger: support forking of outlets
refs #10
2017-12-27 13:50:07 +01:00
Christian Schwarz
583a63a68f refactor: encapsulate pulling in a struct
refs #10
2017-12-24 15:23:28 +01:00
Christian Schwarz
2716c75ad5 build: target for go library dependencies
Didn't notice it because vendor/ was already populated on my dev
machine, but did notice it in Docker build.

Docker build now consumes devsetup like regular user, so this should
catch future problems.

Remove remaining curl|shit functionality from lazy.sh (no checkout logic
needed anymore).

refs #35
2017-11-19 12:34:01 +01:00
Christian Schwarz
e8facfe9fa docs: sphinx-versioning would not build master
sphinx-versioning only build branches / commits with a 'docs/conf.py',
otherwise:

    => Gathering info about the remote git repository...
    => Getting list of all remote branches/tags...
    => Found: docs_theme master resumable_send_recv 0.0.1 0.0.2
    => With docs: 0.0.2
    => Root ref master not found in: 0.0.2

refs #35
2017-11-18 21:28:10 +01:00
Christian Schwarz
d424e800c8 docs: publish.sh check if sphinx-versioning is installed
refs #35
2017-11-18 21:16:54 +01:00
Christian Schwarz
896f31bbf3 'zrepl version' and 'zrepl control version' subcommand + maintainer README
Version is autodetected on build using git
If it cannot be detected with git, an override must be provided.

For tracability of distros, the distroy packagers should override as
well, which is why I added a README entry for package mainatiners.

refs #35
2017-11-18 21:12:48 +01:00
Christian Schwarz
d59426a8cc document new build infra in README.md
refs #35
2017-11-18 19:17:59 +01:00
Christian Schwarz
903fbff710 Add Docker build image, modularize lazy.sh and adjust build from source instructions
refs #35
2017-11-18 19:11:14 +01:00
Christian Schwarz
bc4b129536 lazy.sh: support non-terminal outputs 2017-11-18 17:02:43 +01:00
Christian Schwarz
b4b1bebb5c rename clone_and_build.sh to lazy.sh
refs #35
2017-11-18 17:02:11 +01:00
Christian Schwarz
445a280aa2 build: include docs in release artifacts + use sphinxcontrib-versioning
refs #35
2017-11-18 16:28:06 +01:00
Christian Schwarz
b276787dd4 Makefile: use ARTIFACTDIR variable everywhere
refs #35
2017-11-18 16:20:14 +01:00
Christian Schwarz
bfbab9382e fixup: remove unused StdoutOutlet function
refs #28
2017-11-17 00:36:48 +01:00
Christian Schwarz
2bfcfa5be8 logging: first outlet receives logger error message
Abandons stderr special-casing:

* looks weird on shell and IO redirection to same file because of
interleaving of stdout and stderr
* better than a separate dedicated outlet because it does not require
additional configuration

fixes #28

BREAK SEMANTICS CONFIG
2017-11-17 00:25:38 +01:00
Christian Schwarz
a7f70a566d logger: write internal / outlet errors to an error outlet
refs #28
2017-11-16 23:49:47 +01:00
Christian Schwarz
f5ead68586 README: remove obsolete dirs in developer docs 2017-11-16 21:49:49 +01:00
Christian Schwarz
bf6c58425a README: fix typo 2017-11-16 21:48:21 +01:00
Christian Schwarz
8249a5d1b7 docs: tutorial: fix indentation of sample config 2017-11-16 09:14:01 +01:00
Christian Schwarz
b576253ea8 logging: fixup 4763486: implementation would parse 'date' instead of 'time' field in config 2017-11-15 11:14:20 +01:00
Christian Schwarz
476348689a logging: stdout outlet: include time in output if tty or forced through config 2017-11-15 11:04:34 +01:00
Christian Schwarz
ed68bffea5 bookmark every snapshot
replication logic already supports bookmarks \o/

refs #34
2017-11-13 10:59:46 +01:00
Christian Schwarz
51af880701 refactor: parametrize PrefixFilter VersionType check
refs #34
2017-11-13 10:59:22 +01:00
Christian Schwarz
cef63ac176 logging: stdout formatter: use logfmt package to format non-special stdout fields + handle errors
refs #40
2017-11-13 10:58:07 +01:00
Christian Schwarz
9e48c70f58 Makefile: fix default goal 2017-11-12 21:41:34 +01:00
Christian Schwarz
fe40352f8e docs: link to github 2017-11-12 16:45:11 +01:00
Christian Schwarz
fd123fc6c4 docs: add warning about lack of async TCP outlet
refs #26
2017-11-12 16:41:25 +01:00
Christian Schwarz
961500cc2c clone_and_build.sh: move set -e out of sheband to work on curl pipe bash 2017-11-12 16:27:10 +01:00
Christian Schwarz
47726ad877 improve install from source
* Idempotent clone_and_build.sh does everything
* Add documentation for how to build in Docker

Had to sacrificy go generate because stringer apparently can't handle
vendor directory used by go dep, fails with error
on go generate rpc/frame_layer.go

refs #37
2017-11-12 16:15:12 +01:00
Christian Schwarz
3b6cede108 go dep: run dep ensure, apparently cut off all unused transitive dependencies 2017-11-12 14:19:53 +01:00
Christian Schwarz
2cad13f27b docs: add changelog 2017-11-12 14:12:57 +01:00
Christian Schwarz
b5475921a8 docs: fixup wrong fieldname in source-job
3e647c1 config: source job: rename field 'datasets' to 'filesystems'

BREAK CONFIG
2017-11-12 14:11:48 +01:00
Christian Schwarz
88c684e1d0 README.md: document procedure for breaking changes 2017-11-12 14:10:16 +01:00
Christian Schwarz
a4d28701d9 docs: fix publish.sh script (was not pushing changes to master) 2017-11-12 13:33:34 +01:00
Christian Schwarz
000c8b4186 README.md: some documentation on how to build the docs 2017-11-11 23:33:09 +01:00