From bdf99f6bb4a946a1910845e84282897a2da4f809 Mon Sep 17 00:00:00 2001 From: Christian Schwarz Date: Mon, 18 Mar 2019 12:45:27 +0100 Subject: [PATCH 1/7] docs: document material in `dist/` --- README.md | 4 +++- docs/changelog.rst | 3 +++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index b7fd784..dd329aa 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,8 @@ Check out the *Coding Workflow* section below for details. * Make sure your distro is compatible with the paths in `docs/installation.rst`. * Ship a default config that adheres to your distro's `hier` and logging system. * Ship a service manager file and _please_ try to upstream it to this repository. - * `dist/systemd` contains a Systemd unit template + * `dist/systemd` contains a Systemd unit template. +* Ship other material provided in `./dist`, e.g. in `/usr/share/zrepl/`. * Use `make release ZREPL_VERSION='mydistro-1.2.3_1'` * Your distro's name and any versioning supplemental to zrepl's (e.g. package revision) should be in this string * Make sure you are informed about new zrepl versions, e.g. by subscribing to GitHub's release RSS feed. @@ -62,6 +63,7 @@ Make sure to develop an understanding how zrepl is typically used by studying th │   ├── streamrpcconfig # abstraction for configuration of go-streamrpc │   └── transport # transports implementation ├── docs # sphinx-based documentation +├── dist # supplemental material for users & package maintainers │   ├── **/*.rst # documentation in reStructuredText │   ├── sphinxconf │   │   └── conf.py # sphinx config (see commit 445a280 why its not in docs/) diff --git a/docs/changelog.rst b/docs/changelog.rst index 672cd61..de5cb2c 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -94,10 +94,13 @@ Changes * |feature| ``zrepl status`` for live-updating replication progress (it's really cool!) * |feature| :issue:`67`: Expose `Prometheus `_ metrics via HTTP (:ref:`config docs `) + * Compatible Grafana dashboard shipping in ``dist/grafana`` + * |break_config| Logging outlet types must be specified using the ``type`` instead of ``outlet`` key * |break| :issue:`53`: CLI: ``zrepl control *`` subcommands have been made direct subcommands of ``zrepl *`` * |bugfix| Goroutine leak on ssh transport connection timeouts * |bugfix| :issue:`81` :issue:`77` : handle failed accepts correctly (``source`` job) +* |feature| Systemd unit file in ``dist/systemd`` .. |lastrelease| replace:: 0.0.3 From cef865f5ce26d634c7f7a6a8536fa125a628c40e Mon Sep 17 00:00:00 2001 From: Christian Schwarz Date: Mon, 18 Mar 2019 12:46:16 +0100 Subject: [PATCH 2/7] docs: changelog 0.1: document forgotten bugfixes & features --- docs/changelog.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/changelog.rst b/docs/changelog.rst index de5cb2c..c94ed1d 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -100,6 +100,8 @@ Changes * |break| :issue:`53`: CLI: ``zrepl control *`` subcommands have been made direct subcommands of ``zrepl *`` * |bugfix| Goroutine leak on ssh transport connection timeouts * |bugfix| :issue:`81` :issue:`77` : handle failed accepts correctly (``source`` job) +* |bugfix| :issue:`100`: fix incompatibility with ZoL 0.8 +* |feature| :issue:`115`: logging: configurable syslog facility * |feature| Systemd unit file in ``dist/systemd`` .. |lastrelease| replace:: 0.0.3 From 84019238dfafba752dbcb5515a1fa7956c4883de Mon Sep 17 00:00:00 2001 From: Christian Schwarz Date: Mon, 18 Mar 2019 12:46:52 +0100 Subject: [PATCH 3/7] docs: changelog: remove whitespace & churn --- docs/changelog.rst | 5 ----- 1 file changed, 5 deletions(-) diff --git a/docs/changelog.rst b/docs/changelog.rst index c94ed1d..5230fcb 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -112,8 +112,3 @@ Previous Releases .. NOTE:: Due to limitations in our documentation system, we only show the changelog since the last release and the time this documentation is built. For the changelog of previous releases, use the version selection in the hosted version of these docs at `zrepl.github.io `_. - - - - -W From 53dc0c3c5e6bec957ae9a50d9d0d4975407320dc Mon Sep 17 00:00:00 2001 From: Christian Schwarz Date: Mon, 18 Mar 2019 14:40:46 +0100 Subject: [PATCH 4/7] docs: document existence of zrepl:placeholder property fixes #129 --- docs/configuration/jobs.rst | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/docs/configuration/jobs.rst b/docs/configuration/jobs.rst index 077784f..2b6edfd 100644 --- a/docs/configuration/jobs.rst +++ b/docs/configuration/jobs.rst @@ -91,7 +91,8 @@ The following steps take place during replication and can be monitored using the * Perform replication steps in the following order: Among all filesystems with pending replication steps, pick the filesystem whose next replication step's snapshot is the oldest. - * After a successful replication step, update the replication cursor bookmark (see below) + * Create placeholder filesystems on the receiving side to mirror the dataset paths on the sender to ``root_fs/${client_identity}``. + * After a successful replication step, update the replication cursor bookmark (see below). The idea behind the execution order of replication steps is that if the sender snapshots all filesystems simultaneously at fixed intervals, the receiver will have all filesystems snapshotted at time ``T1`` before the first snapshot at ``T2 = T1 + $interval`` is replicated. @@ -102,6 +103,14 @@ It is a bookmark of the most recent successfully replicated snapshot to the rece It is is used by the :ref:`not_replicated ` keep rule to identify all snapshots that have not yet been replicated to the receiving side. Regardless of whether that keep rule is used, the bookmark ensures that replication can always continue incrementally. +**Placeholder filesystems** on the receiving side are regular ZFS filesystems with the placeholder property ``zrepl:placeholder``. +Placeholders allow the receiving side to mirror the sender's ZFS dataset hierachy without replicating every filesystem at every intermediary dataset path component. +Consider the following example: ``S/H/J`` shall be replicated to ``R/sink/job/S/H/J``, but neither ``S/H`` nor ``S`` shall be replicated. +ZFS requires the existence of ``R/sink/job/S`` and ``R/sink/job/S/H`` in order to receive into ``R/sink/job/S/H/J``. +Thus, zrepl creates the parent filesystems as placeholders on the receiving side. +If at some point ``S/H`` and ``S`` shall be replicated, the receiving side invalidates the placeholder flag automatically. +The ``zrepl test placeholder`` command can be used to check whether a filesystem is a placeholder. + .. ATTENTION:: Currently, zrepl does not replicate filesystem properties. From fb999c8617b3e7c1b9dfda2979dcfaccf48d8ce6 Mon Sep 17 00:00:00 2001 From: Christian Schwarz Date: Mon, 18 Mar 2019 14:48:12 +0100 Subject: [PATCH 5/7] docs: drop references to 'main config file' were just confusing in the remaining places where they were used fixes #127 --- docs/configuration/jobs.rst | 2 +- docs/configuration/logging.rst | 2 +- docs/configuration/monitoring.rst | 2 +- docs/global.rst.inc | 3 ++- docs/installation.rst | 2 -- 5 files changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/configuration/jobs.rst b/docs/configuration/jobs.rst index 2b6edfd..9a2ea89 100644 --- a/docs/configuration/jobs.rst +++ b/docs/configuration/jobs.rst @@ -14,7 +14,7 @@ Job Types & Replication Overview & Terminology ---------------------- -A *job* is the unit of activity tracked by the zrepl daemon and configured in the |mainconfig|. +A *job* is the unit of activity tracked by the zrepl daemon. Every job has a unique ``name``, a ``type`` and type-dependent fields which are documented on this page. Replication always happens between a pair of jobs: one is the **active side**, and one the **passive side**. diff --git a/docs/configuration/logging.rst b/docs/configuration/logging.rst index a9077bb..7a5bb5a 100644 --- a/docs/configuration/logging.rst +++ b/docs/configuration/logging.rst @@ -7,7 +7,7 @@ Logging zrepl uses structured logging to provide users with easily processable log messages. -Logging outlets are configured in the ``global`` section of the |mainconfig|. +Logging outlets are configured in the ``global`` section of the config file. :: diff --git a/docs/configuration/monitoring.rst b/docs/configuration/monitoring.rst index f0e97dc..138b968 100644 --- a/docs/configuration/monitoring.rst +++ b/docs/configuration/monitoring.rst @@ -5,7 +5,7 @@ Monitoring ========== -Monitoring endpoints are configured in the ``global.monitoring`` section of the |mainconfig|. +Monitoring endpoints are configured in the ``global.monitoring`` section of the config file. .. _monitoring-prometheus: diff --git a/docs/global.rst.inc b/docs/global.rst.inc index bbe9204..0074484 100644 --- a/docs/global.rst.inc +++ b/docs/global.rst.inc @@ -1,3 +1,4 @@ .. global list of substitutions -.. |mainconfig| replace:: :ref:`main configuration file ` +.. + .. |macroname| replace:: macro expansion diff --git a/docs/installation.rst b/docs/installation.rst index a9086ca..a3c8cc5 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -80,8 +80,6 @@ Either way, all build results are located in the ``artifacts/`` directory. It is your job to install the apropriate binary in the zrepl users's ``$PATH``, e.g. ``/usr/local/bin/zrepl``. Otherwise, the examples in the :ref:`tutorial` may need to be adjusted. -.. _mainconfigfile: - What next? ---------- From 7633c1cdf148b83c80197005caff915043ca2fee Mon Sep 17 00:00:00 2001 From: Christian Schwarz Date: Mon, 18 Mar 2019 14:56:32 +0100 Subject: [PATCH 6/7] docs: tutorial: s/pull/push --- docs/tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorial.rst b/docs/tutorial.rst index 0d639b6..c40400d 100644 --- a/docs/tutorial.rst +++ b/docs/tutorial.rst @@ -171,7 +171,7 @@ The Linux equivalent might look like this: :: Summary ------- -Congratulations, you have a working pull backup. Where to go next? +Congratulations, you have a working push backup. Where to go next? * Read more about :ref:`configuration format, options & job types ` * Configure :ref:`logging ` \& :ref:`monitoring `. From 25eeedbce840b787e9fceb62b93cacf7b35ad18b Mon Sep 17 00:00:00 2001 From: Christian Schwarz Date: Tue, 19 Mar 2019 18:22:15 +0100 Subject: [PATCH 7/7] docs: link to good_first_issue and docs category --- docs/index.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/index.rst b/docs/index.rst index 6fe0e7e..8fc5b50 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -77,7 +77,8 @@ We are happy about any help we can get! * These docs live in the ``docs/`` subdirectory * Document any non-obvious / confusing / plain broken behavior you encounter when setting up zrepl for the first time -* Check the *Issues* and *Projects* sections for things to do +* Check the *Issues* and *Projects* sections for things to do. + The `good first issues `_ and `docs `_ are suitable starting points. .. admonition:: Development Workflow :class: note