mirror of
https://github.com/zrepl/zrepl.git
synced 2025-06-05 01:17:25 +02:00
50cc2fdb77
**Note: This is a WIP PR that presently only adds the documentation for
the features so that proposal can be discussed.**
Background
==========
While trying to use zrepl in my ansible driven home lab deployment I ran
into an interesting problem. My ZFS based servers subscribe to different
sometimes overlapping roles.
Example role distribution between two servers:
```
serverA:
- common
- web
- file
serverB:
- common
- git
- file
```
Each role wants to create and manage a ZFS dataset with its own
replication / backup policies:
- web: pool/web
- git: pool/git
- file pool/file
At present the creation of a ZFS dataset from each role role is somewhat
very easy, so to is the creation of the basic zrepl configuration file
from the "common" role.
However, when each role tries to register it's job(s) into the singular
zrepl configuration files things get tricky.
I could try adding a role at the end that hardcodes the datasets that
need to be backed up but that seems a bit hacky.
I could also use ansible's `lineinfile` task to try to idempotently add
each dataset's snapshot jobs to the zrepl configuration files but that
causes a problem:
Everytime the "common" role gets run the basic zrepl configuration file
gets re-created causing the web, git, and file roles to all register
changes as they have to re-insert all jobs back into the singular
configuration file.
Proposed Solution
=================
The proposed solution is to allow for the distribution of zrepl job
definition between multiple different YAML files that can be included
from the main zrepl configuration files.
```
global: ...
jobs:
include: jobs.d
```
This directive would be only acceptable in the main configuration file
and is mutually exclusive with any other job definitions in the file.
To keep things lean there will be no conflict resolution provided to
users, job names must be unique across all included job YAML files.
With this feature, the above problem becomes much simpler:
- Common: Sets up the global zrepl configuration and the include
directive
- web/git/file: Each manage their own datasets and create their
jobs.d/web.yml, jobs.d/git.yml, and jobs.d/file.yml.
98 lines
2.4 KiB
ReStructuredText
98 lines
2.4 KiB
ReStructuredText
.. _miscellaneous:
|
|
|
|
Miscellaneous
|
|
=============
|
|
|
|
.. _conf-runtime-directories:
|
|
|
|
Runtime Directories & UNIX Sockets
|
|
----------------------------------
|
|
|
|
The zrepl daemon needs to open various UNIX sockets in a runtime directory:
|
|
|
|
* a ``control`` socket that the CLI commands use to interact with the daemon
|
|
* the :ref:`transport-ssh+stdinserver` listener opens one socket per configured client, named after ``client_identity`` parameter
|
|
|
|
There is no authentication on these sockets except the UNIX permissions.
|
|
The zrepl daemon will refuse to bind any of the above sockets in a directory that is world-accessible.
|
|
|
|
The following sections of the ``global`` config shows the default paths.
|
|
The shell script below shows how the default runtime directory can be created.
|
|
|
|
::
|
|
|
|
global:
|
|
control:
|
|
sockpath: /var/run/zrepl/control
|
|
serve:
|
|
stdinserver:
|
|
sockdir: /var/run/zrepl/stdinserver
|
|
|
|
|
|
::
|
|
|
|
mkdir -p /var/run/zrepl/stdinserver
|
|
chmod -R 0700 /var/run/zrepl
|
|
|
|
|
|
Durations & Intervals
|
|
---------------------
|
|
|
|
Interval & duration fields in job definitions, pruning configurations, etc. must match the following regex:
|
|
|
|
::
|
|
|
|
var durationStringRegex *regexp.Regexp = regexp.MustCompile(`^\s*(\d+)\s*(s|m|h|d|w)\s*$`)
|
|
// s = second, m = minute, h = hour, d = day, w = week (7 days)
|
|
|
|
|
|
.. _conf-include-directories:
|
|
|
|
Including Configuration Directories
|
|
-----------------------------------
|
|
|
|
It is possible to distribute zrepl job definitions over multiple YAML files. This is
|
|
achieved by using the `include` key in the configuration file.
|
|
|
|
The directive is mutually exclusive with any other jobs definition and can only exist
|
|
in the configuration file:
|
|
|
|
::
|
|
global: ...
|
|
|
|
jobs:
|
|
include: jobs.d
|
|
|
|
.. NOTE::
|
|
The included directory path is treated as absolute when starting with `/` else it
|
|
is treated as a relative path from the directory of the loaded configuration file.
|
|
|
|
Included YAML job files must end with the `.yml` extension and can only contain
|
|
contain the `jobs` key. Additionally, job names must be unique across all job YAML
|
|
files.
|
|
|
|
Examples
|
|
^^^^^^^^
|
|
|
|
::
|
|
> /etc/zrepl/zrepl.yml
|
|
|
|
jobs:
|
|
include: jobs.d
|
|
|
|
> /etc/zrepl/zrepl.d/MyDataSet.yml
|
|
jobs:
|
|
- name: snapjob
|
|
type: snap
|
|
filesystems: {
|
|
"MyPool/MyDataset<": true,
|
|
}
|
|
snapshotting:
|
|
type: periodic
|
|
interval: 2m
|
|
prefix: zrepl_snapjob_
|
|
pruning:
|
|
keep:
|
|
- type: last_n
|
|
count: 60
|