Commit Graph

26 Commits

Author SHA1 Message Date
InsanePrawn
180c3d9ae1 Reformat all files with make format.
Signed-off-by: InsanePrawn <insane.prawny@gmail.com>
2020-08-31 23:57:45 +02:00
Christian Schwarz
e500d9ee26 package trace: track activeChildTasks in a set if debugEnabled=true
refs #358
2020-08-23 20:13:58 +02:00
Christian Schwarz
10a14a8c50 [#307] add package trace, integrate it with logging, and adopt it throughout zrepl
package trace:

- introduce the concept of tasks and spans, tracked as linked list within ctx
    - see package-level docs for an overview of the concepts
    - **main feature 1**: unique stack of task and span IDs
        - makes it easy to follow a series of log entries in concurrent code
    - **main feature 2**: ability to produce a chrome://tracing-compatible trace file
        - either via an env variable or a `zrepl pprof` subcommand
        - this is not a CPU profile, we already have go pprof for that
        - but it is very useful to visually inspect where the
          replication / snapshotter / pruner spends its time
          ( fixes #307 )

usage in package daemon/logging:

- goal: every log entry should have a trace field with the ID stack from package trace

- make `logging.GetLogger(ctx, Subsys)` the authoritative `logger.Logger` factory function
    - the context carries a linked list of injected fields which
      `logging.GetLogger` adds to the logger it returns
    - `logging.GetLogger` also uses package `trace` to get the
      task-and-span-stack and injects it into the returned logger's fields
2020-05-19 11:30:02 +02:00
Christian Schwarz
1336c91865 zfs: introduce pkg zfs/zfscmd for command logging, status, prometheus metrics
refs #196
2020-04-05 20:47:25 +02:00
InsanePrawn
44bd354eae Spellcheck all files
Signed-off-by: InsanePrawn <insane.prawny@gmail.com>
2020-02-24 16:06:09 +01:00
Ross Williams
729c83ee72 pre- and post-snapshot hooks
* stack-based execution model, documented in documentation
* circbuf for capturing hook output
* built-in hooks for postgres and mysql
* refactor docs, too much info on the jobs page, too difficult
  to discover snapshotting & hooks

Co-authored-by: Ross Williams <ross@ross-williams.net>
Co-authored-by: Christian Schwarz <me@cschwarz.com>

fixes #74
2019-09-27 21:25:59 +02:00
Ross Williams
00434f4ac9 daemon/logging: format human: treat 'subsystem' field prefixed
logging.Subsystem != string
=> typecast failed

(Also, nobody guarantees that e.Fields contains `field`)
2019-09-27 20:39:43 +02:00
Christian Schwarz
a6497b2c6e add platformtest: infrastructure for ZFS compatiblity testing 2019-09-14 13:43:46 +02:00
Christian Schwarz
5b97953bfb run golangci-lint and apply suggested fixes 2019-03-27 13:12:26 +01:00
Christian Schwarz
afed762774 format source tree using goimports 2019-03-22 19:41:12 +01:00
Christian Schwarz
c0028c1c44 daemon/logging: add replication logic logger 2019-03-21 17:03:34 +01:00
Christian Schwarz
da3ba50a2c Merge remote-tracking branch 'origin/master' into problame/replication_refactor 2019-03-16 14:48:01 +01:00
Christian Schwarz
a0f301d700 syslog logging: fix priority parsing + add test for default facility 2019-03-15 18:18:16 +01:00
Ximalas
fc311a9fd6 syslog logging: support setting facility in config 2019-03-15 17:55:11 +01:00
Christian Schwarz
c87759affe replication/driver: automatic retries on connectivity-related errors 2019-03-13 15:00:40 +01:00
Christian Schwarz
07b43bffa4 replication: refactor driving logic (no more explicit state machine) 2019-03-13 15:00:40 +01:00
Christian Schwarz
796c5ad42d rpc rewrite: control RPCs using gRPC + separate RPC for data transfer
transport/ssh: update go-netssh to new version
    => supports CloseWrite and Deadlines
    => build: require Go 1.11 (netssh requires it)
2019-03-13 13:53:48 +01:00
Christian Schwarz
be962998ba move serve and connecter into transports package 2018-10-11 21:21:46 +02:00
Christian Schwarz
308e5e35fb Multi-client servers + bring back stdinserver support 2018-09-04 16:43:55 -07:00
Christian Schwarz
e161347e47 Implement periodic snapshotting. 2018-09-04 16:43:55 -07:00
Christian Schwarz
d55a271ac7 WIP adopt updated yaml-config with 'fromdefaults' struct tag 2018-09-02 15:46:03 -07:00
Christian Schwarz
b95e983d0d bump go-streamrpc to 0.2, cleanup logging
logging should be user-friendly in INFO mode
2018-09-02 15:45:18 -07:00
Christian Schwarz
1690339440 colorized stdout logger if stdout is tty 2018-08-30 13:33:28 +02:00
Christian Schwarz
a2aa8e7bd7 finish pruner implementation 2018-08-29 19:00:45 +02:00
Christian Schwarz
c69ebd3806 WIP rewrite the daemon
cmd subdir does not build on purpose, it's only left in tree to grab old
code and move it to github.com/zrepl/zrepl/daemon
2018-08-27 22:22:44 +02:00
Christian Schwarz
89dc267780 start implementing new daemon in package daemon 2018-08-27 19:10:55 +02:00