Commit Graph

40 Commits

Author SHA1 Message Date
Juergen Hoetzel
d35e2400b2 transport/{TCP,TLS}: optional IP_FREEBIND / IP_BINDANY bind socketops
Allows to bind to an address even if it is not actually (yet or ever)
configured. Fixes #238

Rationale:
https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/#whatdoesthismeanformeadeveloper
2020-01-04 17:21:48 +01:00
Christian Schwarz
7e743c74dc docs + samples: adjust ssh 'Compression' arg in examples 2019-11-20 18:19:16 +01:00
Christian Schwarz
5c95c21727 transport/local: configurable dial_timeout for connect, default 2s 2019-09-29 19:05:54 +02:00
Christian Schwarz
f976212ec9 config: validate presence of port in addresses
fixes #213
2019-09-28 14:25:14 +02:00
Ross Williams
729c83ee72 pre- and post-snapshot hooks
* stack-based execution model, documented in documentation
* circbuf for capturing hook output
* built-in hooks for postgres and mysql
* refactor docs, too much info on the jobs page, too difficult
  to discover snapshotting & hooks

Co-authored-by: Ross Williams <ross@ross-williams.net>
Co-authored-by: Christian Schwarz <me@cschwarz.com>

fixes #74
2019-09-27 21:25:59 +02:00
Christian Schwarz
5b97953bfb run golangci-lint and apply suggested fixes 2019-03-27 13:12:26 +01:00
Christian Schwarz
afed762774 format source tree using goimports 2019-03-22 19:41:12 +01:00
Christian Schwarz
17818439a0 Merge branch 'problame/replication_refactor' into InsanePrawn-master 2019-03-17 17:33:51 +01:00
Christian Schwarz
da3ba50a2c Merge remote-tracking branch 'origin/master' into problame/replication_refactor 2019-03-16 14:48:01 +01:00
Christian Schwarz
4ee00091d6 pull job: support manual-only invocation 2019-03-16 14:24:05 +01:00
Christian Schwarz
aff639e87a Merge remote-tracking branch 'origin/master' into InsanePrawn-master 2019-03-15 21:05:20 +01:00
Christian Schwarz
a0f301d700 syslog logging: fix priority parsing + add test for default facility 2019-03-15 18:18:16 +01:00
Ximalas
fc311a9fd6 syslog logging: support setting facility in config 2019-03-15 17:55:11 +01:00
Christian Schwarz
796c5ad42d rpc rewrite: control RPCs using gRPC + separate RPC for data transfer
transport/ssh: update go-netssh to new version
    => supports CloseWrite and Deadlines
    => build: require Go 1.11 (netssh requires it)
2019-03-13 13:53:48 +01:00
Christian Schwarz
c1aab0bee9 config: update yaml-config and use zeropositive constraint for timeouts 2018-12-11 21:54:36 +01:00
InsanePrawn
3d2688e959 Ugly but working inital snapjob implementation 2018-11-20 19:30:15 +01:00
Christian Schwarz
5e1ea21f85 pruning: add 'Negate' option to KeepRegex and expose it in config 2018-11-16 12:21:54 +01:00
Christian Schwarz
8cfeeee23a config: fixup 1f072936c5: broken test 2018-10-19 17:23:00 +02:00
Christian Schwarz
1f072936c5 fix default stdout outlet 2018-10-18 15:48:24 +02:00
Christian Schwarz
63169c51b7 add 'test filesystems' subcommand for testing filesystem filters 2018-10-13 16:22:19 +02:00
Christian Schwarz
125b561df3 rename root_dataset to root_fs for receiving-side jobs 2018-10-11 18:03:18 +02:00
Christian Schwarz
0c3a694470 fixup: add test for global section 2018-10-11 17:52:19 +02:00
Christian Schwarz
4e16952ad9 snapshotting: support 'periodic' and 'manual' mode
1. Change config format to support multiple types
   of snapshotting modes.
2. Implement a hacky way to support periodic or completely
   manual snaphots.

In manual mode, the user has to trigger replication using the wakeup
mechanism after they took snapshots using their own tooling.

As indicated by the comment, a more general solution would be desirable,
but we want to get the release out and 'manual' mode is a feature that
some people requested...
2018-10-11 15:59:23 +02:00
Christian Schwarz
14febbeb4c config: skip files that do not end in .yml 2018-10-11 13:09:04 +02:00
Christian Schwarz
93c90cd705 pruning: fix YAML representation of PruneKeepRegex 2018-10-11 13:07:52 +02:00
Christian Schwarz
01668a989e transport local: named listeners + struct renaming 2018-10-11 13:06:47 +02:00
Christian Schwarz
9e941d5be5 pruning: implement 'grid' keep rule 2018-09-24 17:33:16 +02:00
Christian Schwarz
1ce0c69e4f implement local replication using new local transport
The new local transport uses socketpair() and a switchboard based on
client identities.
The special local job type is gone, which is good since it does not fit
into the 'Active/Passive side ' + 'mode' concept used to implement the
duality of push/sink | pull/source.
2018-09-24 14:43:53 +02:00
Christian Schwarz
cf5d63ee88 config: fix broken tests + reduce example configs 2018-09-24 12:41:39 +02:00
Christian Schwarz
e3be120d88 refactor push + source into active + passive 'sides' with push and source 'modes' 2018-09-24 12:36:10 +02:00
Christian Schwarz
4a6160baf3 update to streamrpc 0.4 & adjust config (not breaking) 2018-09-23 20:28:30 +02:00
Christian Schwarz
975fdee217 replication & pruning: ditch replicated-property, use bookmark as cursor instead
A bookmark with a well-known name is used to track which version was
last successfully received by the receiver.
The createtxg that can be retrieved from the bookmark using `zfs get` is
used to set the Replicated attribute of each snap on the sender:
If the snap's CreateTXG > the cursor's, it is not yet replicated,
otherwise it has been.

There is an optional config option to change the behvior to
`CreateTXG >= the cursor's`, and the implementation defaults to that.

The reason: While things work just fine with `CreateTXG > the cursor's`,
ZFS does not provide size estimates in a `zfs send` dry run
(see acd2418).
However, to enable the use case of keeping the snapshot only around for
the replication, the config flag exists.
2018-09-05 19:51:06 -07:00
Christian Schwarz
adab06405b make go vet happy 2018-09-04 17:25:10 -07:00
Christian Schwarz
308e5e35fb Multi-client servers + bring back stdinserver support 2018-09-04 16:43:55 -07:00
Christian Schwarz
754b253043 config: no-field for replication anymore
It's closer to the original config and we don't want users to specify
'filesystems' and similar multiple times in a single job definition.
2018-09-04 14:44:45 -07:00
Christian Schwarz
3d8e552c6a streamrpc 0.3 + config from daemon/config 2018-09-02 15:46:42 -07:00
Christian Schwarz
d55a271ac7 WIP adopt updated yaml-config with 'fromdefaults' struct tag 2018-09-02 15:46:03 -07:00
Christian Schwarz
1690339440 colorized stdout logger if stdout is tty 2018-08-30 13:33:28 +02:00
Christian Schwarz
d684302864 pruning: fix tests + implement 'not_replicated' and 'keep_regex' keep rule
tests expected that a KeepRule returns a *keep* list whereas it
actually returns a *destroy* list.
2018-08-30 11:46:47 +02:00
Christian Schwarz
c69ebd3806 WIP rewrite the daemon
cmd subdir does not build on purpose, it's only left in tree to grab old
code and move it to github.com/zrepl/zrepl/daemon
2018-08-27 22:22:44 +02:00