diff --git a/docs/configuration/jobs.rst b/docs/configuration/jobs.rst index 5c1787d..804b5f1 100644 --- a/docs/configuration/jobs.rst +++ b/docs/configuration/jobs.rst @@ -54,6 +54,7 @@ Example: :sampleconf:`pullbackup/productionhost.yml`. - Snapshotting Task (every ``interval``, |patient|) - A snapshot of filesystems matched by ``filesystems`` is taken every ``interval`` with prefix ``snapshot_prefix``. + - A bookmark of that snapshot is created with the same name. - The ``prune`` policy is triggered on filesystems matched by ``filesystems`` with snapshots matched by ``snapshot_prefix``. - Serve Task @@ -62,11 +63,13 @@ Example: :sampleconf:`pullbackup/productionhost.yml`. A source job is the counterpart to a :ref:`job-pull`. -Note that the prune policy determines the maximum replication lag: -a pull job may stop replication due to link failure, misconfiguration or administrative action. -The source prune policy will eventually destroy the last common snapshot between source and pull job, requiring full replication. Make sure you read the |prune| policy documentation. +Note that zrepl does not prune bookmarks due to the following reason: +a pull job may stop replication due to link failure, misconfiguration or administrative action. +The source prune policy will eventually destroy the last common snapshot between source and pull job. +Without bookmarks, the prune policy would need to perform full replication again. +With bookmarks, we can resume incremental replication, only losing the snapshots pruned since the outage. .. _job-pull: @@ -148,6 +151,7 @@ Example: :sampleconf:`localbackup/host1.yml` #. Evaluate ``mapping`` for local filesystems, those with a *target filesystem* are called *mapped filesystems*. #. Snapshot *mapped filesystems* with ``snapshot_prefix``. + #. Bookmark the snapshot created above. #. Replicate *mapped filesystems* to their respective *target filesystems*: #. Only snapshots with prefix ``snapshot_prefix`` are replicated. @@ -159,6 +163,8 @@ Example: :sampleconf:`localbackup/host1.yml` #. The ``prune_rhs`` policy is triggered for all *target filesystems* A local job is combination of source & pull job executed on the same machine. +Note that while snapshots are pruned, bookmarks are not pruned and kept around forever. +Refer to the comments on :ref:`source job ` for the reasoning behind this. Terminology ----------- diff --git a/docs/configuration/prune.rst b/docs/configuration/prune.rst index fab5589..9f2fd62 100644 --- a/docs/configuration/prune.rst +++ b/docs/configuration/prune.rst @@ -53,12 +53,11 @@ The following procedure happens during pruning: .. ATTENTION:: - .. TODO this is obsolete as soon as bookmarks are created during snapshotting + The configuration of the first interval (``1x1h(keep=all)`` in the example) determines the **maximum allowable replication lag** because the source and destination pruning policies do not coordinate: + if replication does not work for whatever reason, source will continue to execute the prune policy. + Eventually, source destroys a snapshot that has never been replicated to destination, degrading the temporal resolution of your backup. - The configuration of the first interval (``1x1h(keep=all)`` in the example) determines the **maximum allowable replication lag** between source and destination. - After the first interval, source and destination likely have different retention settings. - This means source and destination may prune different snapshots, prohibiting incremental replication froms snapshots that are not in the first interval. + Thus, **always** configure the first interval to ``1x?(keep=all)``, substituting ``?`` with the maximum time replication may fail due to downtimes, maintenance, connectivity issues, etc. - **Always** configure the first interval to ``1x?(keep=all)``, substituting ``?`` with the maximum time replication may fail due to downtimes, maintenance, connectivity issues, etc. - After outages longer than ``?`` you may be required to perform **full replication** again. +.. We intentionally do not mention that bookmarks are used to bridge the gap between source and dest that are out of sync snapshot-wise. This is an implementation detail.