mirror of
https://github.com/zrepl/zrepl.git
synced 2025-02-16 10:29:54 +01:00
docs: tutorial: unambiguous hostnames
s/backups/backup-srv s/prod1/app-srv Also fixes wrong hostname in Analysis section. fixes #15
This commit is contained in:
parent
164f77d80c
commit
3f394a8960
@ -5,40 +5,41 @@ weight: 1
|
||||
|
||||
This tutorial shows how zrepl can be used to implement a ZFS-based pull backup.
|
||||
|
||||
We assume the following scenario
|
||||
We assume the following scenario:
|
||||
|
||||
* Production server `prod1` with filesystems to back up
|
||||
* Production server `app-srv` with filesystems to back up:
|
||||
* `zroot/var/db`
|
||||
* `zroot/usr/home` and all its child filesystems
|
||||
* **except** `zroot/usr/home/paranoid` belonging to a user doing backups themselves
|
||||
* Backup server `backups` with
|
||||
* Filesystem `storage/zrepl/pull/prod1` + children dedicated to backups of `prod1`
|
||||
* Backup server `backup-srv` with
|
||||
* Filesystem `storage/zrepl/pull/app-srv` + children dedicated to backups of `app-srv`
|
||||
|
||||
Our backup solution should fulfill the following requirements:
|
||||
|
||||
* Periodically snapshot the filesystems on `prod1` *every 10 minutes*
|
||||
* Incrementally replicate these snapshots to `storage/zrepl/pull/prod1/*` on `backups`
|
||||
* Keep only very few snapshots on `prod1` to save disk space
|
||||
* Keep a fading history (24 hourly, 30 daily, 6 monthly) of snapshots on `backups`
|
||||
* Periodically snapshot the filesystems on `app-srv` *every 10 minutes*
|
||||
* Incrementally replicate these snapshots to `storage/zrepl/pull/app-srv/*` on `backup-srv`
|
||||
* Keep only very few snapshots on `app-srv` to save disk space
|
||||
* Keep a fading history (24 hourly, 30 daily, 6 monthly) of snapshots on `backup-srv`
|
||||
|
||||
## Analysis
|
||||
|
||||
We can model this situation as two jobs:
|
||||
|
||||
* A **source job** on `prod1`
|
||||
* A **source job** on `app-srv`
|
||||
* Creates the snapshots
|
||||
* Keeps a few snapshots that are also on `prod1` to enable incremental replication
|
||||
* A **pull job** on `prod1`
|
||||
* Pulls the snapshots
|
||||
* Fades out snapshots as they age
|
||||
* Keeps a short history of snapshots to enable incremental replication to `backup-srv`
|
||||
* Accepts connections from `backup-srv`
|
||||
* A **pull job** on `backup-srv`
|
||||
* Connects to the `zrepl daemon` process on `app-srv`
|
||||
* Pulls the snapshots to `storage/zrepl/pull/app-srv/*`
|
||||
* Fades out snapshots in `storage/zrepl/pull/app-srv/*` as they age
|
||||
|
||||
{{%expand "Side note: why doesn't `backups` take the snapshots right before replication?" %}}
|
||||
After all, a little `ssh prod1 'zfs snapshot...'` wouldn't be so bad, right?
|
||||
{{%expand "Side note: why doesn't the **pull job** create the snapshots?" %}}
|
||||
|
||||
As is the case with all distributed systems, the link between `prod1` and `backups` might be down for an hour or two.
|
||||
As is the case with all distributed systems, the link between `app-srv` and `backup-srv` might be down for an hour or two.
|
||||
We do not want to sacrifice our required backup resolution of 10 minute intervals for a temporary connection outage.
|
||||
|
||||
When the link comes up again, `backups` will happily catch up the 12 snapshots taken by `prod1` in the meantime, without
|
||||
When the link comes up again, `backup-srv` will happily catch up the 12 snapshots taken by `app-srv` in the meantime, without
|
||||
a gap in our backup history.
|
||||
{{%/expand%}}
|
||||
|
||||
@ -46,23 +47,23 @@ a gap in our backup history.
|
||||
|
||||
Follow the [OS-specific installation instructions]({{< relref "install/_index.md" >}}) and come back here.
|
||||
|
||||
## Configure `backups`
|
||||
## Configure `backup-srv`
|
||||
|
||||
We define a **pull job** named `pull_prod1` in the [main configuration file]({{< relref "install/_index.md#configuration-files" >}} ):
|
||||
We define a **pull job** named `pull_app-srv` in the [main configuration file]({{< relref "install/_index.md#configuration-files" >}} ):
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
- name: pull_prod1
|
||||
- name: pull_app-srv
|
||||
type: pull
|
||||
connect:
|
||||
type: ssh+stdinserver
|
||||
host: prod1.example.com
|
||||
host: app-srv.example.com
|
||||
user: root
|
||||
port: 22
|
||||
identity_file: /etc/zrepl/ssh/prod1
|
||||
identity_file: /etc/zrepl/ssh/app-srv
|
||||
interval: 10m
|
||||
mapping: {
|
||||
"<":"storage/zrepl/pull/prod1"
|
||||
"<":"storage/zrepl/pull/app-srv"
|
||||
}
|
||||
initial_repl_policy: most_recent
|
||||
snapshot_prefix: zrepl_pull_backup_
|
||||
@ -71,22 +72,22 @@ jobs:
|
||||
grid: 1x1h(keep=all) | 24x1h | 35x1d | 6x30d
|
||||
```
|
||||
|
||||
The `connect` section instructs zrepl to use the `stdinserver` transport: instead of directly exposing zrepl on `prod1`
|
||||
to the internet, `backups` starts the `zrepl stdinserver` on `prod1` via SSH.
|
||||
The `connect` section instructs zrepl to use the `stdinserver` transport: instead of directly exposing zrepl on `app-srv`
|
||||
to the internet, `backup-srv` starts the `zrepl stdinserver` subcommand on `app-srv` via SSH.
|
||||
(You can learn more about what happens [here]({{< relref "configuration/transports.md#stdinserver" >}}), or just continue following this tutorial.)
|
||||
|
||||
Thus, we need to create the SSH key pair `/etc/zrepl/ssh/prod1{,.pub}` and later pass the public part to `prod1`
|
||||
which will use it to authenticate `backups`. Execute the following commands on `backups` as the root user:
|
||||
Thus, we need to create the SSH key pair `/etc/zrepl/ssh/app-srv{,.pub}` which identifies `backup-srv` toward `app-srv`.
|
||||
Execute the following commands on `backup-srv` as the root user:
|
||||
|
||||
```bash
|
||||
cd /etc/zrepl
|
||||
mkdir -p ssh
|
||||
chmod 0700 ssh
|
||||
ssh-keygen -t ed25519 -N '' -f /etc/zrepl/ssh/prod1
|
||||
ssh-keygen -t ed25519 -N '' -f /etc/zrepl/ssh/app-srv
|
||||
```
|
||||
You can learn more about the [**pull job** format here]({{< relref "configuration/jobs.md#pull" >}}) but for now we are good to go.
|
||||
|
||||
## Configure `prod1`
|
||||
## Configure `app-srv`
|
||||
|
||||
We define a corresponding **source job** named `pull_backup` in the [main configuration file]({{< relref "install/_index.md#configuration-files" >}})
|
||||
`zrepl.yml`:
|
||||
@ -98,7 +99,7 @@ jobs:
|
||||
type: source
|
||||
serve:
|
||||
type: stdinserver
|
||||
client_identity: backups.example.com
|
||||
client_identity: backup-srv.example.com
|
||||
datasets: {
|
||||
"zroot/var/db": "ok",
|
||||
"zroot/usr/home<": "ok",
|
||||
@ -112,28 +113,30 @@ jobs:
|
||||
|
||||
```
|
||||
|
||||
The `serve` section corresponds to the `connect` section in the configuration of `backups`.
|
||||
The `serve` section corresponds to the `connect` section in the configuration of `backup-srv`.
|
||||
|
||||
We need to allow the SSH key on `backups` to execute `zrepl stdinserver backups.example.com` on
|
||||
`prod1`. For good measure, we will in fact enforce that only this command can be executed.
|
||||
As mentioned before, the SSH key `app-srv.pub` created in the section before identifies `backup-srv` toward `app-srv`.
|
||||
We enforce that by limiting `backup-srv` to execute exactly `zrepl stdinserver CLIENT_IDENTITY` when connecting to `app-srv`.
|
||||
|
||||
Open `/root/.ssh/authorized_keys` and add either of the the following lines, replacing BACKUPS_SSH_PUBKEY at the end
|
||||
of the line with the contents of `/etc/zrepl/ssh/prod1.pub` (note the **.pub** !) from `backups`.
|
||||
Open `/root/.ssh/authorized_keys` and add either of the the following lines.<br />
|
||||
|
||||
```
|
||||
# for OpenSSH >= 7.2
|
||||
command="zrepl stdinserver backups.example.com",restrict BACKUPS_SSH_PUBKEY
|
||||
command="zrepl stdinserver backup-srv.example.com",restrict PULLING_SSH_KEY
|
||||
# for older OpenSSH versions
|
||||
command="zrepl stdinserver backups.example.com",no-port-forwarding,no-X11-forwarding,no-pty,no-agent-forwarding,no-user-rc BACKUPS_SSH_PUBKEY
|
||||
command="zrepl stdinserver backup-srv.example.com",no-port-forwarding,no-X11-forwarding,no-pty,no-agent-forwarding,no-user-rc PULLING_SSH_KEY
|
||||
```
|
||||
|
||||
{{% alert theme="info" %}}The entries **must** be on a single line, including the replaced BACKUPS_SSH_PUBKEY{{% /alert %}}
|
||||
{{% notice info %}}
|
||||
Replace PULLING_SSH_KEY with the contents of `app-srv.pub`.<br/>
|
||||
The entries **must** be on a single line, including the replaced PULLING_SSH_KEY.
|
||||
{{% /notice %}}
|
||||
|
||||
Again, you can learn more about the [**source job** format here]({{< relref "configuration/jobs.md#source" >}}).
|
||||
|
||||
## Apply Configuration Changes
|
||||
|
||||
We need to restart the zrepl daemon on **both** `prod1` and `backups`.
|
||||
We need to restart the zrepl daemon on **both** `app-srv` and `backup-srv`.
|
||||
|
||||
This is [OS-specific]({{< relref "install/_index.md#restarting" >}}).
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user