.. include:: ../global.rst.inc .. |patient| replace:: :ref:`patient ` .. |serve-transport| replace:: :ref:`serve transport` .. |connect-transport| replace:: :ref:`connect transport` .. |mapping| replace:: :ref:`mapping ` .. |filter| replace:: :ref:`filter ` .. |prune| replace:: :ref:`prune ` .. _job: Job Types ========= A *job* is the unit of activity tracked by the zrepl daemon and configured in the |mainconfig|. Every job has a unique ``name``, a ``type`` and type-dependent fields which are documented on this page. Check out the :ref:`tutorial` and :sampleconf:`/` for examples on how job types are actually used. .. ATTENTION:: Currently, zrepl does not replicate filesystem properties. Whe receiving a filesystem, it is never mounted (`-u` flag) and `mountpoint=none` is set. This is temporary and being worked on :issue:`24`. .. _job-source: Source Job ---------- Example: :sampleconf:`pullbackup/productionhost.yml`. .. list-table:: :widths: 20 80 :header-rows: 1 * - Parameter - Comment * - ``type`` - = ``source`` * - ``name`` - unique name of the job * - ``serve`` - |serve-transport| * - ``filesystems`` - |filter| for filesystems to expose to client * - ``snapshot_prefix`` - prefix for ZFS snapshots taken by this job * - ``interval`` - snapshotting interval * - ``prune`` - |prune| for versions of filesytems in ``filesystems``, versions prefixed with ``snapshot_prefix`` - Snapshotting Task (every ``interval``, |patient|) - A snapshot of filesystems matched by ``filesystems`` is taken every ``interval`` with prefix ``snapshot_prefix``. - A bookmark of that snapshot is created with the same name. - The ``prune`` policy is evaluated for versions of filesystems matched by ``filesystems``, versions prefixed with ``snapshot_prefix``. - Serve Task - Wait for connections from pull job using ``serve``. A source job is the counterpart to a :ref:`job-pull`. Make sure you read the |prune| policy documentation. .. _job-pull: Pull Job -------- Example: :sampleconf:`pullbackup/backuphost.yml` .. list-table:: :widths: 20 80 :header-rows: 1 * - Parameter - Comment * - ``type`` - = ``pull`` * - ``name`` - unqiue name of the job * - ``connect`` - |connect-transport| * - ``interval`` - Interval between pull attempts * - ``mapping`` - |mapping| for remote to local filesystems * - ``initial_repl_policy`` - default = ``most_recent``, initial replication policy * - ``snapshot_prefix`` - prefix snapshots must match to be considered for replication & pruning * - ``prune`` - |prune| policy for versions of filesystems of local filesystems reachable by ``mapping``, versions prefixed with ``snapshot_prefix`` * Main Task (every ``interval``, |patient|) #. A connection to the remote source job is established using the strategy in ``connect`` #. ``mapping`` maps filesystems presented by the remote side to local *target filesystems* #. Those remote filesystems with a local *target filesystem* are replicated #. Only snapshots with prefix ``snapshot_prefix`` are replicated. #. If possible, incremental replication takes place. #. If the local target filesystem does not exist, ``initial_repl_policy`` is used. #. On conflicts, an error is logged but replication of other filesystems with mapping continues. #. The ``prune`` policy is evaluated for all *target filesystems* A pull job is the counterpart to a :ref:`job-source`. Make sure you read the |prune| policy documentation. .. _job-local: Local Job --------- Example: :sampleconf:`localbackup/host1.yml` .. list-table:: :widths: 20 80 :header-rows: 1 * - Parameter - Comment * - ``type`` - = ``local`` * - ``name`` - unqiue name of the job * - ``mapping`` - |mapping| from source to target filesystem (both local) * - ``snapshot_prefix`` - prefix for ZFS snapshots taken by this job * - ``interval`` - snapshotting & replication interval * - ``initial_repl_policy`` - default = ``most_recent``, initial replication policy * - ``prune_lhs`` - pruning policy on left-hand-side (source) * - ``prune_rhs`` - pruning policy on right-hand-side (target) * Main Task (every ``interval``, |patient|) #. Evaluate ``mapping`` for local filesystems, those with a *target filesystem* are called *mapped filesystems*. #. Snapshot *mapped filesystems* with ``snapshot_prefix``. #. Bookmark the snapshot created above. #. Replicate *mapped filesystems* to their respective *target filesystems*: #. Only snapshots with prefix ``snapshot_prefix`` are replicated. #. If possible, incremental replication takes place. #. If the *target filesystem* does not exist, ``initial_repl_policy`` is used. #. On conflicts, an error is logged but replication of other *mapped filesystems* continues. #. The ``prune_lhs`` policy is triggered for all *mapped filesystems* #. The ``prune_rhs`` policy is triggered for all *target filesystems* A local job is combination of source & pull job executed on the same machine. Terminology ----------- task A job consists of one or more tasks and a task consists of one or more steps. Some tasks may be periodic while others wait for an event to occur. patient task .. _job-term-patient: A patient task is supposed to execute some task every `interval`. We call the start of the task an *invocation*. * If the task completes in less than `interval`, the task is restarted at `last_invocation + interval`. * Otherwise, a patient job * logs a warning as soon as a task exceeds its configured `interval` * waits for the last invocation to finish * logs a warning with the effective task duration * immediately starts a new invocation of the task filesystem version A snapshot or a bookmark.