docs: document job types

The documentation describes intended behavior.

Apparently, there are some bugs regarding *patient* tasks.

refs #8
refs #13

docs/content/configuration/jobs.md

@@ -3,12 +3,107 @@ title = "Job Types"
 weight = 10
 +++
 
-{{% alert theme="warning" %}}Under Construction{{% /alert %}}
+A *job* is the unit of activity tracked by the zrepl daemon and configured in the [configuration file]({{< relref "install/_index.md#configuration-files" >}}).
 
-See {{< sampleconflink >}} for examples.
+Every job has a unique `name`, a `type` and type-dependent fields which are documented on this page.
 
-## Source
+Check out the [tutorial]({{< relref "tutorial/_index.md" >}}) and {{< sampleconflink >}} for examples on how job types are actually used.
 
-## Pull
+## Job Types
 
-## Local
+### Source
+
+|Parameter|Default|Description / Example|
+|-----|-------|-------|
+|`type`||`source`|
+|`name`||unique name of the job|
+|`serve`||{{< zrepl-transport "serve transport" >}} specification|
+|`datasets`||{{< zrepl-filter >}} for datasets to expose to client|
+|`snapshot_prefix`||prefix for ZFS snapshots taken by this job|
+|`interval`||snapshotting interval|
+|`prune`||{{< zrepl-prune >}} policy for datasets in `datasets` with prefix `snapshot_prefix`|
+
+* Snapshotting Task (every `interval`, {{% zrepl-job-patient %}})
+    1. A snapshot of filesystems matched by `datasets` is taken every `interval` with prefix `snapshot_prefix`.
+    1. The `prune` policy is triggered on datasets matched by `datasets` with snapshots matched by `snapshot_prefix`.
+* Serve Task
+    * Wait for connections from pull job using `serve`
+
+A source job is the counterpart to a [pull job]({{< relref "#pull" >}}).
+
+Example: {{< sampleconflink "pullbackup/productionhost.yml" >}}
+
+### Pull
+
+|Parameter|Default|Description / Example|
+|-----|-------|-------|
+|`type`||`pull`|
+|`name`||unique name of the job|
+|`connect`||{{< zrepl-transport "connect transport" >}} specification|
+|`interval`||Interval between pull attempts|
+|`mapping`||{{< zrepl-mapping >}} for remote to local filesystems|
+|`initial_repl_policy`|`most_recent`|initial replication policy|
+|`snapshot_prefix`||prefix filter used for replication & pruning|
+|`prune`||{{< zrepl-prune >}} policy for local filesystems reachable by `mapping`|
+
+* Main Task (every `interval`, {{% zrepl-job-patient %}})
+    1. A connection to the remote source job is established using the strategy in `connect`
+    1. `mapping` maps filesystems presented by the remote side to local *target filesystems*
+    1. Those remote filesystems with a local *target filesystem* are replicated:
+        1. Only snapshots with prefix `snapshot_prefix` are replicated.
+        1. If possible, incremental replication takes place.
+        1. If the local target filesystem does not exist, `initial_repl_policy` is used.
+        1. On conflicts, an error is logged but replication of other filesystems with mapping continues.
+    1. The `prune` policy is triggered for all *target filesystems*
+
+
+A pull job is the counterpart to a [source job]({{< relref "#source" >}}).
+
+Example: {{< sampleconflink "pullbackup/backuphost.yml" >}}
+
+### Local
+
+|Parameter|Default|Description / Example|
+|-----|-------|-------|
+|`type`||`local`|
+|`name`||unique name of the job|
+|`mapping`||{{<zrepl-mapping>}} from source to target filesystem (both local)|
+|`snapshot_prefix`||prefix for ZFS snapshots taken by this job|
+|`interval`|snapshotting & replication interval|
+|`initial_repl_policy`|`most_recent`|initial replication policy|
+|`prune_lhs`||pruning policy on left-hand-side (source)|
+|`prune_rhs`||pruning policy on right-hand-side (target)|
+
+* Main Task (every `interval`, {{% zrepl-job-patient %}})
+    1. Evaluate `mapping` for local filesystems, those with a *target filesystem* are called *mapped filesystems*.
+    1. Snapshot *mapped filesystems* with `snapshot_prefix`.
+    1. Replicate *mapped filesystems* to their respective *target filesystems*:
+        1. Only snapshots with prefix `snapshot_prefix` are replicated.
+        1. If possible, incremental replication takes place.
+        1. If the *target filesystem* does not exist, `initial_repl_policy` is used.
+        1. On conflicts, an error is logged but replication of other *mapped filesystems* continues.
+    1. The `prune_lhs` policy is triggered for all *mapped filesystems*
+    1. The `prune_rhs` policy is triggered for all *target filesystems*
+
+A local job is combination of source & pull job executed on the same machine.
+
+Example: {{< sampleconflink "localbackup/host1.yml" >}}
+
+## Glossary
+
+#### Task
+
+A job consists of one or more tasks and a task consists of one or more steps.
+Some tasks may be periodic while others wait for an event to occur.
+
+#### patient
+
+it is supposed to execute some task every `interval`.
+We call the start of the task an *invocation*.
+
+* If the task completes in less than `interval`, the task is restarted at `last_invocation + interval`.
+* Otherwise, a patient job
+    * logs a warning as soon as a task exceeds its configured `interval`
+    * waits for the last invocation to finish
+    * logs a warning with the effective task duration
+    * immediately starts a new invocation of the task

docs/layouts/shortcodes/zrepl-filter.html

@@ -0,0 +1,6 @@
+{{ $this := .}}
+{{with .Site.GetPage "page" "configuration" "map_filter_syntax.md" }}
+<a href="{{.URL}}#filters">{{default "filter" ($this.Get 0) }}</a>
+{{else}}
+{{.PageDoesNotExistError}}
+{{end}}

docs/layouts/shortcodes/zrepl-job-patient.html

@@ -0,0 +1 @@
+{{ "[patient](#patient)" | markdownify }}

docs/layouts/shortcodes/zrepl-mapping.html

@@ -0,0 +1,6 @@
+{{ $this := .}}
+{{with .Site.GetPage "page" "configuration" "map_filter_syntax.md" }}
+<a href="{{.URL}}#mappings">{{default "mapping" ($this.Get 0) }}</a>
+{{else}}
+{{.PageDoesNotExistError}}
+{{end}}

docs/layouts/shortcodes/zrepl-prune.html

@@ -0,0 +1,6 @@
+{{ $this := .}}
+{{with .Site.GetPage "page" "configuration" "prune.md" }}
+<a href="{{.URL}}">{{default "prune policy" ($this.Get 0) }}</a>
+{{else}}
+{{.PageDoesNotExistError}}
+{{end}}

docs/layouts/shortcodes/zrepl-transport.html

@@ -0,0 +1,6 @@
+{{ $this := .}}
+{{with .Site.GetPage "page" "configuration" "transports.md" }}
+<a href="{{.URL}}">{{default "transport" ($this.Get 0) }}</a>
+{{else}}
+{{.PageDoesNotExistError}}
+{{end}}

docs/themes/docdock

@@ -1 +1 @@
-Subproject commit 1e35d34235f01ada2fae96802510fd3937dfdb54
+Subproject commit 1ee44353686c9efd84509ef38e384e603661d3a5