Commit Graph

34 Commits

Author SHA1 Message Date
Christian Schwarz
3cb1865909 chore: trace spans: use crypto/rand for generating them
math/rand.Read is deprecated in newer Go versions.

Also, it appears that crypto/rand is faster when used from multiple
goroutines: https://gist.github.com/problame/0699acd6f99db4163f26f0b8a61569f3
2024-09-08 23:19:45 +00:00
Christian Schwarz
def510abfd chore: require go 1.22/1.23, upgrade protobuf, upgrade all deps
Go upgrade:
- Go 1.23 is current => use that for release builds
- Go 1.22 is less than one year old, it's desirable to support it.
- The [`Go Toolchains`](https://go.dev/doc/toolchain) stuff is available
  in both of these (would also be in Go 1.21). That is quite nice stuff,
  but required some changes to how we versions we use in CircleCI and
  the `release-docker` Makefile target.

Protobuf upgrade:
- Go to protobuf GH release website
- Download latest locally
- run `sha256sum`
- replace existing pinned hashes
- `make generate`

Deps upgrade:
- `go get -t -u all`
- repository moves aren't handled well automatically, fix manually
- repeat until no changes
2024-09-08 20:49:09 +00:00
Christian Schwarz
a6aa610165 run go1.19 gofmt and make adjustments as needed
(Go 1.19 expanded doc comment syntax)
2022-10-24 22:22:41 +02:00
Christian Schwarz
c1e2c9826f trace: hint debug env var in error when crashing due to active child tasks
refs https://github.com/zrepl/zrepl/issues/542
2021-12-05 18:57:43 +01:00
Mathias Fredriksson
6ac537210b daemon: avoid math/rand race by using global source
Unless we're using the global source for math/rand, (*rand.Rand).Read
should not be called concurrently. We seed the rng in daemon.Run to
avoid ambiguity or hiding global side effects inside packages.

closes #414
2021-01-25 00:16:01 +01:00
Christian Schwarz
0d96627ffb [#388] trace: make WithTaskGroup actually concurrent 2021-01-24 22:28:47 +01:00
Christian Schwarz
fecc9416ab [#347] package trace: envconst-configurable debug mode 2020-09-02 22:45:44 +02:00
Christian Schwarz
a7915db4c3 [#347] package trace: printing debugString before instead of at panic (fixup e500d9e) 2020-09-02 22:45:44 +02:00
InsanePrawn
180c3d9ae1 Reformat all files with make format.
Signed-off-by: InsanePrawn <insane.prawny@gmail.com>
2020-08-31 23:57:45 +02:00
Christian Schwarz
e500d9ee26 package trace: track activeChildTasks in a set if debugEnabled=true
refs #358
2020-08-23 20:13:58 +02:00
Christian Schwarz
10a14a8c50 [#307] add package trace, integrate it with logging, and adopt it throughout zrepl
package trace:

- introduce the concept of tasks and spans, tracked as linked list within ctx
    - see package-level docs for an overview of the concepts
    - **main feature 1**: unique stack of task and span IDs
        - makes it easy to follow a series of log entries in concurrent code
    - **main feature 2**: ability to produce a chrome://tracing-compatible trace file
        - either via an env variable or a `zrepl pprof` subcommand
        - this is not a CPU profile, we already have go pprof for that
        - but it is very useful to visually inspect where the
          replication / snapshotter / pruner spends its time
          ( fixes #307 )

usage in package daemon/logging:

- goal: every log entry should have a trace field with the ID stack from package trace

- make `logging.GetLogger(ctx, Subsys)` the authoritative `logger.Logger` factory function
    - the context carries a linked list of injected fields which
      `logging.GetLogger` adds to the logger it returns
    - `logging.GetLogger` also uses package `trace` to get the
      task-and-span-stack and injects it into the returned logger's fields
2020-05-19 11:30:02 +02:00
Christian Schwarz
1336c91865 zfs: introduce pkg zfs/zfscmd for command logging, status, prometheus metrics
refs #196
2020-04-05 20:47:25 +02:00
InsanePrawn
44bd354eae Spellcheck all files
Signed-off-by: InsanePrawn <insane.prawny@gmail.com>
2020-02-24 16:06:09 +01:00
Ross Williams
729c83ee72 pre- and post-snapshot hooks
* stack-based execution model, documented in documentation
* circbuf for capturing hook output
* built-in hooks for postgres and mysql
* refactor docs, too much info on the jobs page, too difficult
  to discover snapshotting & hooks

Co-authored-by: Ross Williams <ross@ross-williams.net>
Co-authored-by: Christian Schwarz <me@cschwarz.com>

fixes #74
2019-09-27 21:25:59 +02:00
Ross Williams
00434f4ac9 daemon/logging: format human: treat 'subsystem' field prefixed
logging.Subsystem != string
=> typecast failed

(Also, nobody guarantees that e.Fields contains `field`)
2019-09-27 20:39:43 +02:00
Christian Schwarz
a6497b2c6e add platformtest: infrastructure for ZFS compatiblity testing 2019-09-14 13:43:46 +02:00
Christian Schwarz
5b97953bfb run golangci-lint and apply suggested fixes 2019-03-27 13:12:26 +01:00
Christian Schwarz
afed762774 format source tree using goimports 2019-03-22 19:41:12 +01:00
Christian Schwarz
c0028c1c44 daemon/logging: add replication logic logger 2019-03-21 17:03:34 +01:00
Christian Schwarz
da3ba50a2c Merge remote-tracking branch 'origin/master' into problame/replication_refactor 2019-03-16 14:48:01 +01:00
Christian Schwarz
a0f301d700 syslog logging: fix priority parsing + add test for default facility 2019-03-15 18:18:16 +01:00
Ximalas
fc311a9fd6 syslog logging: support setting facility in config 2019-03-15 17:55:11 +01:00
Christian Schwarz
c87759affe replication/driver: automatic retries on connectivity-related errors 2019-03-13 15:00:40 +01:00
Christian Schwarz
07b43bffa4 replication: refactor driving logic (no more explicit state machine) 2019-03-13 15:00:40 +01:00
Christian Schwarz
796c5ad42d rpc rewrite: control RPCs using gRPC + separate RPC for data transfer
transport/ssh: update go-netssh to new version
    => supports CloseWrite and Deadlines
    => build: require Go 1.11 (netssh requires it)
2019-03-13 13:53:48 +01:00
Christian Schwarz
be962998ba move serve and connecter into transports package 2018-10-11 21:21:46 +02:00
Christian Schwarz
308e5e35fb Multi-client servers + bring back stdinserver support 2018-09-04 16:43:55 -07:00
Christian Schwarz
e161347e47 Implement periodic snapshotting. 2018-09-04 16:43:55 -07:00
Christian Schwarz
d55a271ac7 WIP adopt updated yaml-config with 'fromdefaults' struct tag 2018-09-02 15:46:03 -07:00
Christian Schwarz
b95e983d0d bump go-streamrpc to 0.2, cleanup logging
logging should be user-friendly in INFO mode
2018-09-02 15:45:18 -07:00
Christian Schwarz
1690339440 colorized stdout logger if stdout is tty 2018-08-30 13:33:28 +02:00
Christian Schwarz
a2aa8e7bd7 finish pruner implementation 2018-08-29 19:00:45 +02:00
Christian Schwarz
c69ebd3806 WIP rewrite the daemon
cmd subdir does not build on purpose, it's only left in tree to grab old
code and move it to github.com/zrepl/zrepl/daemon
2018-08-27 22:22:44 +02:00
Christian Schwarz
89dc267780 start implementing new daemon in package daemon 2018-08-27 19:10:55 +02:00