Commit Graph

1140 Commits

Author SHA1 Message Date
90b35da295 Revert "add metric zrepl_daemon_job_up"
This reverts commit 43ef8f5bf0.
2025-01-06 19:00:25 +00:00
160e702043 replace zrepl_job_name with internal dashboard variable zrepl_replication_jobid
This makes the dashboard change not depend on the new
metric zrepl_daemon_job_up
2025-01-06 18:49:42 +00:00
75d2071a52 seach-replace grafanacloud datasource names to sth more standard 2025-01-06 18:32:01 +00:00
8e871ea599 multiple changes (sorry)
* use `zrepl_daemon_job_up` to populate zrepl_job_name variable
* filter by `zrepl_job_name` variable where appropriate
* create sections About, Top, Replication, Internals
* consistently force selection of a single prom_job_name (no
  multi-value)
2025-01-06 18:28:23 +00:00
43ef8f5bf0 add metric zrepl_daemon_job_up 2025-01-06 17:06:04 +00:00
f3840d6457 import into grafanacloud, fixup
https://github.com/zrepl/zrepl/pull/847#discussion_r1904343361, export
2025-01-06 16:35:45 +00:00
4d8358abcf Change dashboard name to something more readable 2024-11-07 00:19:25 +01:00
418311df2a Fix units and layout for Grafana 11 2024-11-07 00:17:19 +01:00
c8090f73a4 Update Grafana Dashboard 2024-11-07 00:00:43 +01:00
d7ede3f82c zrepl status: switch back to tview from cview & upgrade to latest (#846)
While investigating https://github.com/zrepl/zrepl/issues/700
I checked in on `zrepl status` dependencies and found that
`cview`, which was/is a fork of tview, appears to be unmaintained.

We switched to it 4.5 years ago in a58ce74.

Checking now, `github.com/rivo/tview` seems to be somewhat maintained
again.
I also checked what k9s uses because that tool came to mind as a Go
terminal UI app.
It does use `tview`, but, a fork that has diverged substantially.

Maybe in another 4.5 years stuff the ecosystem has consolidated...

refs https://github.com/zrepl/zrepl/issues/700
2024-11-05 21:35:30 +01:00
bca37c741c README: document patch release procedure (#845)
fixes https://github.com/zrepl/zrepl/issues/374
2024-11-04 11:26:17 +01:00
98073c7dca build: simplify make format, remove make formatcheck (#843) 2024-11-04 11:25:30 +01:00
f1608aba15 hearbeatconn: convert dead integration test into Go test (#842)
fixes https://github.com/zrepl/zrepl/issues/417
2024-11-04 11:10:04 +01:00
dc05cd00f2 lint: add lint checking for time.Equal (#841)
No issues found, tested that the lint works by changing code locally.

fixes https://github.com/zrepl/zrepl/issues/5
2024-11-02 15:45:09 +01:00
2923009689 deb: place bash completion at correct path, using debhelper (#839)
fixes https://github.com/zrepl/zrepl/issues/794
2024-10-28 00:46:23 +01:00
cc0f3b1f13 dist/systemd: remove various Protect* settings (#831)
It pains me to do it, but, especially with hooks, the Protect
settings are too restrictive.

I wish there were a systemd API that allowed us to self-sandbox,
using these settings, _after_ parsing the config.

fixes https://github.com/zrepl/zrepl/issues/735
2024-10-27 21:43:50 +01:00
20abaa2e7f devex: manage devtools in a project-local subdirectory + cleanup README (#829) 2024-10-21 21:36:34 +02:00
908807bd59 move implementation to internal/ directory (#828) 2024-10-18 19:21:17 +02:00
b9b9ad10cf snapshotting: ability to specify timestamp location != UTC (#801)
This PR adds a new field optional field `timestamp_location` that allows
the user to specify a timezone different than the default UTC for use in
the snapshot suffix.

I took @mjasnik 's PR https://github.com/zrepl/zrepl/pull/785 and
refactored+extended it as follows:
* move all formatting logic into its own package
* disallow `dense` and `human` with formats != UTC to protect users from
stupidity
* document behavior more clearly
* regression test for existing users
2024-10-18 15:12:41 +02:00
904c1512a3 Merge pull request #819 from zrepl/problame/dependency-upgrades
dependency upgrades
2024-10-18 15:00:40 +02:00
3df5e223be take notes for changelog 2024-09-08 23:19:45 +00:00
b8f55a97ba build: circleci: use large class for release-build job 2024-09-08 23:19:45 +00:00
82adb2b9f5 build: circleci: remove obsolete script
The binary packaging workflow has long since
been moved to this repo (I don't think the external
workflow work ever completed).
2024-09-08 23:19:45 +00:00
7b6adab6b1 build: circleci: only archive artifacts/release 2024-09-08 23:19:45 +00:00
e390aa0c5a build: circleci: update VM image used for release builds
Doesn't matter much because everything happens inside Docker.
2024-09-08 23:19:45 +00:00
5a8f0b9a24 build: make release: check toolchain GOVERSION matches expecations (and refactor/extend Makefile a bit) 2024-09-08 23:19:45 +00:00
3cb1865909 chore: trace spans: use crypto/rand for generating them
math/rand.Read is deprecated in newer Go versions.

Also, it appears that crypto/rand is faster when used from multiple
goroutines: https://gist.github.com/problame/0699acd6f99db4163f26f0b8a61569f3
2024-09-08 23:19:45 +00:00
0ab92d4861 build: avoid compiling platformtest test list generator
This also fixes a deprecation warning.
2024-09-08 23:19:45 +00:00
740ab4b1b2 chore: io/ioutil has been deprecated 2024-09-08 23:19:45 +00:00
48c5b60024 chore: grpc.DialContext has been deprecated 2024-09-08 23:19:45 +00:00
40fd700855 chore: newer staticcheck complains about useless fmt.Sprintf 2024-09-08 20:57:09 +00:00
def510abfd chore: require go 1.22/1.23, upgrade protobuf, upgrade all deps
Go upgrade:
- Go 1.23 is current => use that for release builds
- Go 1.22 is less than one year old, it's desirable to support it.
- The [`Go Toolchains`](https://go.dev/doc/toolchain) stuff is available
  in both of these (would also be in Go 1.21). That is quite nice stuff,
  but required some changes to how we versions we use in CircleCI and
  the `release-docker` Makefile target.

Protobuf upgrade:
- Go to protobuf GH release website
- Download latest locally
- run `sha256sum`
- replace existing pinned hashes
- `make generate`

Deps upgrade:
- `go get -t -u all`
- repository moves aren't handled well automatically, fix manually
- repeat until no changes
2024-09-08 20:49:09 +00:00
08769a8752 fix: accidental use of wrong logging package 2024-09-08 12:57:58 +00:00
affe00aefe docs: draw attention to risks of not_replicated (#810)
Co-authored-by: Christian Schwarz <me@cschwarz.com>
2024-09-05 23:56:59 +02:00
5615f4929a fix: replication of placeholder filesystems (#744)
fixes https://github.com/zrepl/zrepl/issues/742

Before this PR, when chaining replication from
A => B => C, if B had placeholders and the `filesystems`
included these placeholders, we'd incorrectly
fail the planning phase with error
`sender does not have any versions`.

The non-placeholder child filesystems of these placeholders
would then fail to replicate because of the
initial-replication-dependency-tracking that we do, i.e.,
their parent failed to initially replication, hence
they fail to replicate as well
(`parent(s) failed during initial replication`).

We can do better than that because we have the information
whether a sender-side filesystem is a placeholder.
This PR makes the planner act on that information.
The outcome is that placeholders are replicated as
placeholders (albeit the receiver remains in control
of how these placeholders are created, i.e., `recv.placeholders`)
The mechanism to do it is:
1. Don't plan any replication steps for filesystems that
   are placeholders on the sender.
2. Ensure that, if a receiving-side filesystem exists, it
   is indeed a placeholder.

Check (2) may seem overly restrictive, but, the goal here
is not just to mirror all non-placeholder filesystems, but
also to mirror the hierarchy.

Testing performed:
- [x] confirm with issue reporter that this PR fixes their issue
- [x] add a regression test that fails without the changes in this PR
2024-09-05 23:26:42 +02:00
440b07443f Remove dead Bountysource link (#806)
Bountysource is dead.


https://www.theblockchain-group.com/wp-content/uploads/2023/11/TBG-CP17112023.pdf
2024-07-28 20:04:07 +02:00
e2fcf9ff5b docs: add missing newline for codeblock in docs/compile-from-source.rst (#768)
to make it render correctly
2024-07-13 20:42:50 +02:00
a5f6bc3697 github: disable dependabot for docs (#800) 2024-07-13 18:22:25 +02:00
9c63736489 treat empty jobs & empty YAML as valid & ship empty jobs in deb/rpm (#788)
fixes https://github.com/zrepl/zrepl/issues/784
obsoletes https://github.com/zrepl/zrepl/pull/787
2024-05-14 19:18:22 +02:00
830536715e docs: use $zrepl_apt_repo_file on installation snippet (#783) 2024-05-08 00:30:50 +02:00
ebc46cf1c0 Fix last_n keep rule (#691) (#750)
From https://github.com/zrepl/zrepl/issues/691

The last_n prune rule keeps everything, regardless of if it matches the
regex or not, if there are less than count snapshot. The expectation
would be to never keep non-regex snapshots, regardless of number.
2023-12-22 13:38:14 +01:00
27012e5623 Allow same root_fs for different jobs: sinks and so on (#752)
Because some jobs add client identity to root_fs and other jobs don't do
that,
we can't reliable detect overlapping of filesystems. And and the same
time we
need an ability to use equal or overlapped root_fs for different jobs.
For
instance see this config:

```
  - name: "zdisk"
    type: "sink"
    root_fs: "zdisk/zrepl"
    serve:
      type: "local"
      listener_name: "zdisk"
```
and
```
  - name: "remote-to-zdisk"
    type: "pull"
    connect:
      type: "tls"
    root_fs: "zdisk/zrepl/remote"
```

As you can see, two jobs have overlapped root_fs, but actually datasets
are not
overlapped, because job `zdisk` save everything under
`zdisk/zrepl/localhost`,
because it adds client identity. So they actually use two different
filesystems:
`zdisk/zrepl/localhost` and `zdisk/zrepl/remote`. And we can't detect
this
situation during config check. So let's just remove this check, because
it's
admin's duty to configure correct root_fs's.

---------

Co-authored-by: Christian Schwarz <me@cschwarz.com>
2023-11-01 00:12:54 +01:00
30faaec26a build: ci: fix quickcheck-docs for external PRs (#763)
fixes https://github.com/zrepl/zrepl/issues/762
2023-11-01 00:12:23 +01:00
21e0ae63a6 build: fix rpm builds, broken by ef9a63b: support package revisions 2023-10-07 18:46:28 +00:00
370f40881d build: wrap-and-checksum didn't include .deb files
fixup of 9d5c892023
2023-10-07 17:03:27 +00:00
fb71a7e4b0 build: forward ZREPL_VERSION and ZREPL_PACKAGE_RELEASE to docker targets 2023-10-07 16:36:43 +00:00
ef9a63b075 build: support package revisions 2023-10-07 16:36:43 +00:00
faef059edf build: get rid of bins-all target special case, bring back test vet lint steps of release target 2023-10-07 16:36:43 +00:00
ad9fbf7b6d build: generic _impl target to run a make target for all GOOS/GOARCH combinations 2023-10-07 16:26:53 +00:00
3bd17b8069 build: remove GO_SUPPORTS_ILLUMOS cruft
illumos is supported by all Go versions that can build zrepl
2023-10-07 16:26:53 +00:00