extern/zrepl
1
0
Fork 0
mirror of https://github.com/zrepl/zrepl.git synced 2024-11-25 09:54:47 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
329 Commits 48 Branches 23 Tags 3.6 MiB
4cf910874d
Commit Graph

329 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Christian Schwarz
4cf910874d rpc: make DataType a stringer, fixing debug messages 2018-02-18 13:33:53 +01:00
Christian Schwarz
3ba3648f0f zfs: use channel as iterator for ZFSList results 
The old approach with ZFSList would keep the two-dimensional array of
lines and their fields in memory (for a short time), which could easily
consume 100s of MiB with > 10000 snapshots / bookmarks (see #34)

fixes #61
 2018-02-18 13:28:46 +01:00
Christian Schwarz
aa92261ea7 bookmarking: prune policy for bookmarks 
refs #34
 2018-02-17 20:48:31 +01:00
Christian Schwarz
8e34843eb1 autosnap: do not treat zero fs filter results as fatal 2018-02-17 19:27:00 +01:00
Christian Schwarz
bfaf6fdfbb daemon: fix missing newline on parse error 2018-02-17 17:43:55 +01:00
Christian Schwarz
f992fed968 control pprof rewrite: expose pprof metrics via HTTP server controlled from CLI 2018-02-17 16:20:10 +01:00
Christian Schwarz
94967b596c docs: document changes to ssh+stdinserver transport implementation: ccd062e 2018-02-17 15:16:29 +01:00
Christian Schwarz
759dae4552 build: further fixups of ccd062e: remove ref to deleted sshbytestream subpkg 2018-02-17 14:28:04 +01:00
Christian Schwarz
f3d3a7f5f8 stdinserver: fixup ccd062e: assert socket is in private directory 2018-02-17 14:12:44 +01:00
Christian Schwarz
ccd062e238 ssh+stdinserver: dump sshbytestream for github.com/problame/go-netssh 
Cleaner abstractions + underlying go-rwccmd package does proper handling
of asynchronous exits, etc.
 2018-02-17 01:08:15 +01:00
Christian Schwarz
fc1c46ffd7 logger: fix ReplaceWith: would case parent field to be nil 
Now WithField and ReplaceWith are wrappers around a common
forkLogger routine

regression introduced in 51377a8
 2018-02-16 21:19:15 +01:00
Christian Schwarz
6b5bd0a43c job pull + source: fix broken connection teardown 
Issue #56 shows zombie SSH processes.
We fix this by actually Close()ing the RWC in job pull.
If this fixes #56 it also fixes #6 --- it's the same issue.

Additionally, debugging around this revealed another issue: just
Close()ing the sshbytestream in job source will apparently outpace the
normal data stream of stdin and stdout (URG or PUSH flags?).  leading
to ugly errors in the logs.
With proper TCP connections, we would simply set the connection to
linger and close it, letting the kernel handle the final timeout. Meh.

refs #56
refs #6
 2018-02-16 20:57:27 +01:00
Christian Schwarz
921bccb960 job source: use task logger 2018-02-15 23:51:57 +01:00
Christian Schwarz
24b29a0865 Gopkg: remove unused dependencies + cleanup Gopkg.toml 2018-02-15 22:18:32 +01:00
Christian Schwarz
5f2c14adab zfs: use custom datatype to pass ZFS properties in ZFSSet 
refs #55
 2018-01-05 18:42:10 +01:00
Christian Schwarz
787675aee8 control status command: only show verbose logs on user request 2017-12-30 13:53:19 +01:00
Christian Schwarz
6f68c98c16 logger.Levle: implement flag.Value 2017-12-30 13:52:51 +01:00
Christian Schwarz
01e0519b7b control status subcommand: fix typo in usage 2017-12-30 13:44:55 +01:00
Christian Schwarz
8742b7f763 handler: fix typo in log message 2017-12-30 13:29:04 +01:00
Christian Schwarz
710bf79f7e logger.Logger: fix WithFields() dropping all but last field 2017-12-30 13:00:23 +01:00
Christian Schwarz
a622ef1487 docs: promote test subcommand 2017-12-29 22:53:33 +01:00
Christian Schwarz
56f13741f9 test pattern subcommand: better example command 2017-12-29 22:45:38 +01:00
Christian Schwarz
746fb4ff88 build: include generate step in release build + warn of dirty git working copy 2017-12-29 22:34:14 +01:00
Christian Schwarz
8473462adf build: adjust wrong path of zrepl source dir in build.Dockerfile 
was symlinking /zrepl to /go/src/github.com/zrepl/zrepl earlier, forgot
to change that apparently

see 47726ad877

refs #38
 2017-12-29 22:25:48 +01:00
Christian Schwarz
61842988b9 Task & TaskStatus: DeepCopy(): actually copy lastUpdate field 
otherwise, only changes to activity level would udpate TaskStatus
LastUpdate field

refs #10
 2017-12-29 21:43:12 +01:00
Christian Schwarz
be7176bee7 Puller: fix wrong filesystem log field usage 
was introduced in 9465b593
 2017-12-29 21:25:42 +01:00
Christian Schwarz
c403e56835 fixup: broken test case for logger 
refs #26
 2017-12-29 21:14:49 +01:00
Christian Schwarz
839eccf513 logger.Outlet: WriteEntry must not block 
- make TCPOutlet fully asynchronous, dropping messages if connection is
  not fast enough
- syslog is just fine for now, local anyways
- stdout same thing

refs #26
 2017-12-29 17:21:58 +01:00
Christian Schwarz
9a19615fd4 docs: document bookmarking + remove warning about replication lag 
refs #34
 2017-12-28 13:24:25 +01:00
Christian Schwarz
03ba2bb7c8 docs: move config files + runtime dir doc to new configuration/preface 2017-12-27 18:34:24 +01:00
Christian Schwarz
7ac2821147 docs: small usage section mentioning CLI 2017-12-27 18:34:24 +01:00
Christian Schwarz
e6554b77c0 docs: mention control status command in tutorial 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
acd9aedb98 cmd control status: unify job logs, option to show only one job & always show logs 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
835cf6b12f cmd control status: warn about inactive tasks 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
4b3d83ec1f TaskStatus: add LastUpdate field 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
d13c6e3fc3 job local: refactor + use Task API 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
63fa7a67e9 job source: refactor + use Task API 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
7d89d1fb00 job pull: refactor + use Task API 
refs #10
 2017-12-27 18:34:24 +01:00
Christian Schwarz
b69089a527 Puller: refactor + use Task API 
* drop rx byte count functionality
* will be re-added to Task as necessary

refs #10
 2017-12-27 14:39:47 +01:00
Christian Schwarz
59e34942d1 Puller: make main interface public 
refs #10
 2017-12-27 14:39:46 +01:00
Christian Schwarz
91c4a97f72 Pruner: refactor + use Task API 
refs #10
 2017-12-27 14:39:46 +01:00
Christian Schwarz
13562b48ed IntervalAutosnap: refactor + use Task API 
refs #10
 2017-12-27 14:39:46 +01:00
Christian Schwarz
58ee796394 adopt Task API: infect datastructures 
refs #10
 2017-12-27 14:39:46 +01:00
Christian Schwarz
ce351146cf job control: implement JobStatus 2017-12-27 14:39:46 +01:00
Christian Schwarz
14b8d69a63 cmd control status + expose DaemonStatus via control API 
refs #10
 2017-12-27 14:39:46 +01:00
Christian Schwarz
8c7e373049 daemon: DaemonStatus + JobStatus + dummy implementation 
refs #10
 2017-12-27 14:39:46 +01:00
Christian Schwarz
2c87b15e83 daemon: Task abstraction + TaskStatus 
An instance of Task tracks a single thread of activity that is part of a Job.

While the docs already use this terminology of tasks being composed of jobs,
the code did not have an object to represent these semantics.
Now it does:

* A task t is initialized with a root activity, which is its name
* t can t.Enter() and t.Finish() an activity, building
  a stack of activities
* t's code can get a logger t.Log() whose logTaskField is set to the
  concatenated stack of activities
* t's code can update IO progress it made since leaving idle state
* t's code's log output vie t.Log() is captured since leaving idle
  state
  * FIXME: find a way to bound that buffer

refs #10
refs #48
 2017-12-27 14:39:46 +01:00
Christian Schwarz
d7f3fb93ae bash completions: hidden subcommand + integrate into Makefile 2017-12-27 14:39:46 +01:00
Christian Schwarz
ebf209427a logging: support ignoring fields in HumanFormatter 
should be refactored to logger one day so the implementation of ignoring
is not duplicated to each outlet.

refs #10
 2017-12-27 14:39:46 +01:00
Christian Schwarz
51377a8ffb logger: support replacing fields 
do no delete() from array since this could lead to resizing

refs #10
 2017-12-27 14:39:43 +01:00