Merge branch 'problame/0.1-docs-fixes'

README.md

@@ -26,7 +26,8 @@ Check out the *Coding Workflow* section below for details.
 * Make sure your distro is compatible with the paths in `docs/installation.rst`.
 * Ship a default config that adheres to your distro's `hier` and logging system.
 * Ship a service manager file and _please_ try to upstream it to this repository.
-  * `dist/systemd` contains a Systemd unit template
+  * `dist/systemd` contains a Systemd unit template.
+* Ship other material provided in `./dist`, e.g. in `/usr/share/zrepl/`.
 * Use `make release ZREPL_VERSION='mydistro-1.2.3_1'`
     * Your distro's name and any versioning supplemental to zrepl's (e.g. package revision) should be in this string
 * Make sure you are informed about new zrepl versions, e.g. by subscribing to GitHub's release RSS feed.
@@ -62,6 +63,7 @@ Make sure to develop an understanding how zrepl is typically used by studying th
 │   ├── streamrpcconfig     # abstraction for configuration of go-streamrpc
 │   └── transport           # transports implementation
 ├── docs                    # sphinx-based documentation
+├── dist                    # supplemental material for users & package maintainers
 │   ├── **/*.rst            # documentation in reStructuredText
 │   ├── sphinxconf
 │   │   └── conf.py         # sphinx config (see commit 445a280 why its not in docs/)

docs/changelog.rst

@@ -94,10 +94,15 @@ Changes
 * |feature| ``zrepl status`` for live-updating replication progress (it's really cool!)
 * |feature| :issue:`67`: Expose `Prometheus <https://prometheus.io>`_ metrics via HTTP (:ref:`config docs <monitoring-prometheus>`)
 
+  * Compatible Grafana dashboard shipping in ``dist/grafana``
+
 * |break_config| Logging outlet types must be specified using the ``type`` instead of ``outlet`` key
 * |break| :issue:`53`: CLI: ``zrepl control *`` subcommands have been made direct subcommands of ``zrepl *``
 * |bugfix| Goroutine leak on ssh transport connection timeouts
 * |bugfix| :issue:`81` :issue:`77` : handle failed accepts correctly (``source`` job)
+* |bugfix| :issue:`100`: fix incompatibility with ZoL 0.8
+* |feature| :issue:`115`: logging: configurable syslog facility
+* |feature| Systemd unit file in ``dist/systemd``
 
 .. |lastrelease| replace:: 0.0.3
 
@@ -107,8 +112,3 @@ Previous Releases
 .. NOTE::
     Due to limitations in our documentation system, we only show the changelog since the last release and the time this documentation is built.
     For the changelog of previous releases, use the version selection in the hosted version of these docs at `zrepl.github.io <https://zrepl.github.io>`_.
-    
-    
-
-
-W

docs/configuration/jobs.rst

@@ -14,7 +14,7 @@ Job Types & Replication
 Overview & Terminology
 ----------------------
 
-A *job* is the unit of activity tracked by the zrepl daemon and configured in the |mainconfig|.
+A *job* is the unit of activity tracked by the zrepl daemon.
 Every job has a unique ``name``, a ``type`` and type-dependent fields which are documented on this page.
 
 Replication always happens between a pair of jobs: one is the **active side**, and one the **passive side**.
@@ -91,7 +91,8 @@ The following steps take place during replication and can be monitored using the
 
   * Perform replication steps in the following order:
     Among all filesystems with pending replication steps, pick the filesystem whose next replication step's snapshot is the oldest.
-  * After a successful replication step, update the replication cursor bookmark (see below)
+  * Create placeholder filesystems on the receiving side to mirror the dataset paths on the sender to ``root_fs/${client_identity}``.
+  * After a successful replication step, update the replication cursor bookmark (see below).
    
 The idea behind the execution order of replication steps is that if the sender snapshots all filesystems simultaneously at fixed intervals, the receiver will have all filesystems snapshotted at time ``T1`` before the first snapshot at ``T2 = T1 + $interval`` is replicated.
 
@@ -102,6 +103,14 @@ It is a bookmark of the most recent successfully replicated snapshot to the rece
 It is is used by the :ref:`not_replicated <prune-keep-not-replicated>` keep rule to identify all snapshots that have not yet been replicated to the receiving side.
 Regardless of whether that keep rule is used, the bookmark ensures that replication can always continue incrementally.
 
+**Placeholder filesystems** on the receiving side are regular ZFS filesystems with the placeholder property ``zrepl:placeholder``.
+Placeholders allow the receiving side to mirror the sender's ZFS dataset hierachy without replicating every filesystem at every intermediary dataset path component.
+Consider the following example: ``S/H/J`` shall be replicated to ``R/sink/job/S/H/J``, but neither ``S/H`` nor ``S`` shall be replicated.
+ZFS requires the existence of ``R/sink/job/S`` and ``R/sink/job/S/H`` in order to receive into ``R/sink/job/S/H/J``.
+Thus, zrepl creates the parent filesystems as placeholders on the receiving side.
+If at some point ``S/H`` and ``S`` shall be replicated, the receiving side invalidates the placeholder flag automatically.
+The ``zrepl test placeholder`` command can be used to check whether a filesystem is a placeholder.
+
 .. ATTENTION::
 
     Currently, zrepl does not replicate filesystem properties.

docs/configuration/logging.rst

@@ -7,7 +7,7 @@ Logging
 
 zrepl uses structured logging to provide users with easily processable log messages.
 
-Logging outlets are configured in the ``global`` section of the |mainconfig|.
+Logging outlets are configured in the ``global`` section of the config file.
 
 ::
 

docs/configuration/monitoring.rst

@@ -5,7 +5,7 @@
 Monitoring
 ==========
 
-Monitoring endpoints are configured in the ``global.monitoring`` section of the |mainconfig|.
+Monitoring endpoints are configured in the ``global.monitoring`` section of the config file.
 
 .. _monitoring-prometheus:
 

docs/global.rst.inc

@@ -1,3 +1,4 @@
 .. global list of substitutions
 
-.. |mainconfig| replace:: :ref:`main configuration file <mainconfigfile>`
+.. 
+  .. |macroname| replace:: macro expansion

docs/index.rst

@@ -77,7 +77,8 @@ We are happy about any help we can get!
   * These docs live in the ``docs/`` subdirectory
 
 * Document any non-obvious / confusing / plain broken behavior you encounter when setting up zrepl for the first time
-* Check the *Issues* and *Projects* sections for things to do
+* Check the *Issues* and *Projects* sections for things to do.
+  The `good first issues <https://github.com/zrepl/zrepl/labels/good%20first%20issue>`_ and `docs <https://github.com/zrepl/zrepl/labels/docs>`_ are suitable starting points.
 
 .. admonition:: Development Workflow
     :class: note

docs/installation.rst

@@ -80,8 +80,6 @@ Either way, all build results are located in the ``artifacts/`` directory.
     It is your job to install the apropriate binary in the zrepl users's ``$PATH``, e.g. ``/usr/local/bin/zrepl``.
     Otherwise, the examples in the :ref:`tutorial` may need to be adjusted.
 
-.. _mainconfigfile:
-
 What next?
 ----------
 

docs/tutorial.rst

@@ -171,7 +171,7 @@ The Linux equivalent might look like this: ::
 Summary
 -------
 
-Congratulations, you have a working pull backup. Where to go next?
+Congratulations, you have a working push backup. Where to go next?
 
 * Read more about :ref:`configuration format, options & job types <configuration_toc>`
 * Configure :ref:`logging <logging>` \& :ref:`monitoring <monitoring>`.