extern/zrepl
1
0
Fork 0
mirror of https://github.com/zrepl/zrepl.git synced 2025-06-20 01:37:45 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity

docs: tutorial: unambiguous hostnames

Browse Source 
s/backups/backup-srv
s/prod1/app-srv

Also fixes wrong hostname in Analysis section.

fixes #15
This commit is contained in:
Christian Schwarz 2017-10-02 13:18:22 +02:00
parent 164f77d80c
commit 3f394a8960
1 changed files with 42 additions and 39 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

81
docs/content/tutorial/_index.md
View File

@ -5,40 +5,41 @@ weight: 1
This tutorial shows how zrepl can be used to implement a ZFS-based pull backup. This tutorial shows how zrepl can be used to implement a ZFS-based pull backup.
We assume the following scenario We assume the following scenario:
* Production server `prod1` with filesystems to back up * Production server `app-srv` with filesystems to back up:
* `zroot/var/db` * `zroot/var/db`
* `zroot/usr/home` and all its child filesystems * `zroot/usr/home` and all its child filesystems
* **except** `zroot/usr/home/paranoid` belonging to a user doing backups themselves * **except** `zroot/usr/home/paranoid` belonging to a user doing backups themselves
* Backup server `backups` with * Backup server `backup-srv` with
* Filesystem `storage/zrepl/pull/prod1` + children dedicated to backups of `prod1` * Filesystem `storage/zrepl/pull/app-srv` + children dedicated to backups of `app-srv`
Our backup solution should fulfill the following requirements: Our backup solution should fulfill the following requirements:
* Periodically snapshot the filesystems on `prod1` *every 10 minutes* * Periodically snapshot the filesystems on `app-srv` *every 10 minutes*
* Incrementally replicate these snapshots to `storage/zrepl/pull/prod1/*` on `backups` * Incrementally replicate these snapshots to `storage/zrepl/pull/app-srv/*` on `backup-srv`
* Keep only very few snapshots on `prod1` to save disk space * Keep only very few snapshots on `app-srv` to save disk space
* Keep a fading history (24 hourly, 30 daily, 6 monthly) of snapshots on `backups` * Keep a fading history (24 hourly, 30 daily, 6 monthly) of snapshots on `backup-srv`
## Analysis ## Analysis
We can model this situation as two jobs: We can model this situation as two jobs:
* A **source job** on `prod1` * A **source job** on `app-srv`
* Creates the snapshots * Creates the snapshots
* Keeps a few snapshots that are also on `prod1` to enable incremental replication * Keeps a short history of snapshots to enable incremental replication to `backup-srv`
* A **pull job** on `prod1` * Accepts connections from `backup-srv`
* Pulls the snapshots * A **pull job** on `backup-srv`
* Fades out snapshots as they age * Connects to the `zrepl daemon` process on `app-srv`
* Pulls the snapshots to `storage/zrepl/pull/app-srv/*`
* Fades out snapshots in `storage/zrepl/pull/app-srv/*` as they age
{{%expand "Side note: why doesn't `backups` take the snapshots right before replication?" %}} {{%expand "Side note: why doesn't the **pull job** create the snapshots?" %}}
After all, a little `ssh prod1 'zfs snapshot...'` wouldn't be so bad, right?
As is the case with all distributed systems, the link between `prod1` and `backups` might be down for an hour or two. As is the case with all distributed systems, the link between `app-srv` and `backup-srv` might be down for an hour or two.
We do not want to sacrifice our required backup resolution of 10 minute intervals for a temporary connection outage. We do not want to sacrifice our required backup resolution of 10 minute intervals for a temporary connection outage.
When the link comes up again, `backups` will happily catch up the 12 snapshots taken by `prod1` in the meantime, without When the link comes up again, `backup-srv` will happily catch up the 12 snapshots taken by `app-srv` in the meantime, without
a gap in our backup history. a gap in our backup history.
{{%/expand%}} {{%/expand%}}
@ -46,23 +47,23 @@ a gap in our backup history.
Follow the [OS-specific installation instructions]({{< relref "install/_index.md" >}}) and come back here. Follow the [OS-specific installation instructions]({{< relref "install/_index.md" >}}) and come back here.
## Configure `backups` ## Configure `backup-srv`
We define a **pull job** named `pull_prod1` in the [main configuration file]({{< relref "install/_index.md#configuration-files" >}} ): We define a **pull job** named `pull_app-srv` in the [main configuration file]({{< relref "install/_index.md#configuration-files" >}} ):
```yaml ```yaml
jobs: jobs:
- name: pull_prod1 - name: pull_app-srv
type: pull type: pull
connect: connect:
type: ssh+stdinserver type: ssh+stdinserver
host: prod1.example.com host: app-srv.example.com
user: root user: root
port: 22 port: 22
identity_file: /etc/zrepl/ssh/prod1 identity_file: /etc/zrepl/ssh/app-srv
interval: 10m interval: 10m
mapping: { mapping: {
"<":"storage/zrepl/pull/prod1" "<":"storage/zrepl/pull/app-srv"
} }
initial_repl_policy: most_recent initial_repl_policy: most_recent
snapshot_prefix: zrepl_pull_backup_ snapshot_prefix: zrepl_pull_backup_
@ -71,22 +72,22 @@ jobs:
grid: 1x1h(keep=all) | 24x1h | 35x1d | 6x30d grid: 1x1h(keep=all) | 24x1h | 35x1d | 6x30d
``` ```
The `connect` section instructs zrepl to use the `stdinserver` transport: instead of directly exposing zrepl on `prod1` The `connect` section instructs zrepl to use the `stdinserver` transport: instead of directly exposing zrepl on `app-srv`
to the internet, `backups` starts the `zrepl stdinserver` on `prod1` via SSH. to the internet, `backup-srv` starts the `zrepl stdinserver` subcommand on `app-srv` via SSH.
(You can learn more about what happens [here]({{< relref "configuration/transports.md#stdinserver" >}}), or just continue following this tutorial.) (You can learn more about what happens [here]({{< relref "configuration/transports.md#stdinserver" >}}), or just continue following this tutorial.)
Thus, we need to create the SSH key pair `/etc/zrepl/ssh/prod1{,.pub}` and later pass the public part to `prod1` Thus, we need to create the SSH key pair `/etc/zrepl/ssh/app-srv{,.pub}` which identifies `backup-srv` toward `app-srv`.
which will use it to authenticate `backups`. Execute the following commands on `backups` as the root user: Execute the following commands on `backup-srv` as the root user:
```bash ```bash
cd /etc/zrepl cd /etc/zrepl
mkdir -p ssh mkdir -p ssh
chmod 0700 ssh chmod 0700 ssh
ssh-keygen -t ed25519 -N '' -f /etc/zrepl/ssh/prod1 ssh-keygen -t ed25519 -N '' -f /etc/zrepl/ssh/app-srv
``` ```
You can learn more about the [**pull job** format here]({{< relref "configuration/jobs.md#pull" >}}) but for now we are good to go. You can learn more about the [**pull job** format here]({{< relref "configuration/jobs.md#pull" >}}) but for now we are good to go.
## Configure `prod1` ## Configure `app-srv`
We define a corresponding **source job** named `pull_backup` in the [main configuration file]({{< relref "install/_index.md#configuration-files" >}}) We define a corresponding **source job** named `pull_backup` in the [main configuration file]({{< relref "install/_index.md#configuration-files" >}})
`zrepl.yml`: `zrepl.yml`:
@ -98,7 +99,7 @@ jobs:
type: source type: source
serve: serve:
type: stdinserver type: stdinserver
client_identity: backups.example.com client_identity: backup-srv.example.com
datasets: { datasets: {
"zroot/var/db": "ok", "zroot/var/db": "ok",
"zroot/usr/home<": "ok", "zroot/usr/home<": "ok",
@ -112,28 +113,30 @@ jobs:
``` ```
The `serve` section corresponds to the `connect` section in the configuration of `backups`. The `serve` section corresponds to the `connect` section in the configuration of `backup-srv`.
We need to allow the SSH key on `backups` to execute `zrepl stdinserver backups.example.com` on As mentioned before, the SSH key `app-srv.pub` created in the section before identifies `backup-srv` toward `app-srv`.
`prod1`. For good measure, we will in fact enforce that only this command can be executed. We enforce that by limiting `backup-srv` to execute exactly `zrepl stdinserver CLIENT_IDENTITY` when connecting to `app-srv`.
Open `/root/.ssh/authorized_keys` and add either of the the following lines, replacing BACKUPS_SSH_PUBKEY at the end Open `/root/.ssh/authorized_keys` and add either of the the following lines.<br />
of the line with the contents of `/etc/zrepl/ssh/prod1.pub` (note the **.pub** !) from `backups`.
``` ```
# for OpenSSH >= 7.2 # for OpenSSH >= 7.2
command="zrepl stdinserver backups.example.com",restrict BACKUPS_SSH_PUBKEY command="zrepl stdinserver backup-srv.example.com",restrict PULLING_SSH_KEY
# for older OpenSSH versions # for older OpenSSH versions
command="zrepl stdinserver backups.example.com",no-port-forwarding,no-X11-forwarding,no-pty,no-agent-forwarding,no-user-rc BACKUPS_SSH_PUBKEY command="zrepl stdinserver backup-srv.example.com",no-port-forwarding,no-X11-forwarding,no-pty,no-agent-forwarding,no-user-rc PULLING_SSH_KEY
``` ```
{{% alert theme="info" %}}The entries **must** be on a single line, including the replaced BACKUPS_SSH_PUBKEY{{% /alert %}} {{% notice info %}}
Replace PULLING_SSH_KEY with the contents of `app-srv.pub`.<br/>
The entries **must** be on a single line, including the replaced PULLING_SSH_KEY.
{{% /notice %}}
Again, you can learn more about the [**source job** format here]({{< relref "configuration/jobs.md#source" >}}). Again, you can learn more about the [**source job** format here]({{< relref "configuration/jobs.md#source" >}}).
## Apply Configuration Changes ## Apply Configuration Changes
We need to restart the zrepl daemon on **both** `prod1` and `backups`. We need to restart the zrepl daemon on **both** `app-srv` and `backup-srv`.
This is [OS-specific]({{< relref "install/_index.md#restarting" >}}). This is [OS-specific]({{< relref "install/_index.md#restarting" >}}).