extern/zrepl
1
0
Fork 0
mirror of https://github.com/zrepl/zrepl.git synced 2024-11-24 17:35:01 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
526 Commits 48 Branches 23 Tags 3.6 MiB
e7497ab3d0
Commit Graph

58 Commits

Author SHA1 Message Date
Christian Schwarz
074f989547 Merge branch 'replication_rewrite' (in fact it's a 90% rewrite) 2018-10-13 16:26:23 +02:00
Christian Schwarz
fa47667f31 bring back prometheus metrics, with new metrics for replication state machine 2018-09-07 22:22:34 -07:00
Christian Schwarz
1323a30a0c zfs: ability to specify sources for zfsGet 
fix use for Placeholder, leave rest as previous behavior
 2018-09-05 19:51:06 -07:00
Christian Schwarz
975fdee217 replication & pruning: ditch replicated-property, use bookmark as cursor instead 
A bookmark with a well-known name is used to track which version was
last successfully received by the receiver.
The createtxg that can be retrieved from the bookmark using `zfs get` is
used to set the Replicated attribute of each snap on the sender:
If the snap's CreateTXG > the cursor's, it is not yet replicated,
otherwise it has been.

There is an optional config option to change the behvior to
`CreateTXG >= the cursor's`, and the implementation defaults to that.

The reason: While things work just fine with `CreateTXG > the cursor's`,
ZFS does not provide size estimates in a `zfs send` dry run
(see acd2418).
However, to enable the use case of keeping the snapshot only around for
the replication, the config flag exists.
 2018-09-05 19:51:06 -07:00
Christian Schwarz
acd2418803 handle DryRun send size estimate errors with bookmarks 2018-09-05 17:41:25 -07:00
Christian Schwarz
4b39a18178 zfs: disable resume token test because it doesn't work in docker 2018-09-04 17:31:46 -07:00
Christian Schwarz
1e27720b99 zfs: skip test with ZFS_BINARY mock (doesn't work in parallel) 2018-09-04 17:02:02 -07:00
Christian Schwarz
4336af295f fixup 22ca80eb7e: scraping regex was broken and potentially mixed with stdout 2018-09-04 14:01:48 -07:00
Christian Schwarz
f0860767f5 zfs: include stderr of command in ZFSError.Error() 
Since we don't implement screen-scraping of ZFS output ATM, this is
better than nothing, as user's may be able to figure out what' sthe
problem from the logs / status reports.
 2018-09-02 15:46:42 -07:00
Anton Schirg
f387e23214 fix: at least two snapshots were needed to start replication 2018-08-30 19:20:18 +02:00
Anton Schirg
98f3f3dfd8 show expected size of current send 
Needs to be changed to send sizes for all planned steps
 2018-08-30 12:58:13 +02:00
Anton Schirg
b2f01e454f bug in ZFSListFilesystemVersions? 2018-08-30 12:54:30 +02:00
Christian Schwarz
22ca80eb7e remote snapshot destruction & replication status zfs property 2018-08-30 11:51:47 +02:00
Christian Schwarz
666ead2646 make go vet happy 2018-08-26 14:51:20 +02:00
Christian Schwarz
e30ae972f4 gofmt 2018-08-25 21:30:25 +02:00
Christian Schwarz
1a8d2c5ebe replication: context support and propert closing of stale readers 2018-07-08 23:31:46 +02:00
Christian Schwarz
8cca0a8547 Initial working version 
Summary:
* Logging is still bad
* test output in a lot of placed
* FIXMEs every where

Test Plan: None, just review

Differential Revision: https://phabricator.cschwarz.com/D2
 2018-06-24 10:44:00 +02:00
Moritz Fago
302d77cf4a zfs: allow spaces in zfs names. 
Space is a allowed character in zfs names accoring to
https://github.com/zfsonlinux/zfs/issues/439.
 2018-06-20 14:14:04 +02:00
Christian Schwarz
fa6426f803 WIP: zfs: hacky resume token parsing 2018-05-02 21:26:56 +02:00
Christian Schwarz
0764f8824e zfs: prometheus metrics 
refs #67
 2018-04-05 22:12:25 +02:00
Christian Schwarz
75fd21e454 make generate: stringer was updated and now uses strconv instead of fmt 
bd4635fd25 (diff-0415b5286e4cf3e373f349d917e5e039)
 2018-04-01 15:30:04 +02:00
Christian Schwarz
3ba3648f0f zfs: use channel as iterator for ZFSList results 
The old approach with ZFSList would keep the two-dimensional array of
lines and their fields in memory (for a short time), which could easily
consume 100s of MiB with > 10000 snapshots / bookmarks (see #34)

fixes #61
 2018-02-18 13:28:46 +01:00
Christian Schwarz
5f2c14adab zfs: use custom datatype to pass ZFS properties in ZFSSet 
refs #55
 2018-01-05 18:42:10 +01:00
Christian Schwarz
ed68bffea5 bookmark every snapshot 
replication logic already supports bookmarks \o/

refs #34
 2017-11-13 10:59:46 +01:00
Christian Schwarz
8934d7cb64 zfs: was not returning error on invalid dataset path 2017-09-30 16:39:15 +02:00
Christian Schwarz
0cbee78b40 fix unreachable code & missing stringer-generated code 2017-09-30 16:31:55 +02:00
Christian Schwarz
4e45b4090b pull log output: optimize to be readable by humans 2017-08-06 18:28:05 +02:00
Christian Schwarz
cba083cadf Make zfs.DatasetPath json.Marshaler and json.Unmarshaler 
Had to resort to using pointers to zfs.DatasetPath everywhere... Should
find a better solution for that.
 2017-08-06 16:22:15 +02:00
Christian Schwarz
2ce07c9342 rework filters & mappings 
config defines a single datastructure that can act both as a Map and as a Filter
(DatasetMapFilter)

Cleanup wildcard syntax along the way (also changes semantics).
 2017-08-06 16:21:54 +02:00
Christian Schwarz
4732fdd4cc Implement placeholder filesystems. 
Note the docs on the placeholder user property introduced with this
commit. The solution is not really satisfying but couldn't think of a
better one OTOMH
 2017-08-06 16:21:54 +02:00
Christian Schwarz
9ab6f18f82 zfs: fix/update tests for diffs for createtxg & guid 2017-07-09 00:08:50 +02:00
Christian Schwarz
4b373fbd95 zfs & replication: explicit conflict types for FilesystemDiff + handling in repl 2017-07-08 13:13:16 +02:00
Christian Schwarz
655b3ab55f implement automatic snapshotting feature 2017-07-02 00:02:33 +02:00
Christian Schwarz
8c8a6ee905 implement snapshot pruning feature 2017-07-02 00:02:33 +02:00
Christian Schwarz
c7f140a00f zfs: support destroy 2017-07-01 23:19:31 +02:00
Christian Schwarz
c22190e981 zfs: extract filesystem version code to separate file & add filtering support 2017-07-01 23:19:31 +02:00
Christian Schwarz
d8adce6110 zfs: Support foo/bar/* globs 2017-05-20 19:50:24 +02:00
Christian Schwarz
ee570bb060 refactor: consolidate ForkReader-like implementations to IOCommand 2017-05-14 12:27:15 +02:00
Christian Schwarz
dd6dd60e98 Simplify "fork then io.Reader" abstractions 2017-05-13 15:23:37 +02:00
Christian Schwarz
54778c0374 zfs: remove erronous output from ForkReader 2017-05-13 15:23:28 +02:00
Christian Schwarz
c762502f6e zfs: implement DatasetPath top down walk 2017-05-13 15:23:28 +02:00
Christian Schwarz
996be3704d zfs: implement ZFSListFilesystemExists() 2017-05-07 12:26:41 +02:00
Christian Schwarz
6a420e872e zfs: implement ZFSSet 2017-05-07 12:23:12 +02:00
Christian Schwarz
d37bdc3d7a zfs: implement ZFSRecv 2017-05-07 12:22:57 +02:00
Christian Schwarz
030bd7affe zfs: implement ZFSSend 2017-05-07 12:20:56 +02:00
Christian Schwarz
1a92717894 zfs: FilesytemDiff: len(IncrementalPath) < 2 means same most recent version 
bonus: style + typo fixes
 2017-05-07 12:20:56 +02:00
Christian Schwarz
d0c0e2510b zfs: FilesystemDiff: support empty left list = no common ancestor 2017-05-07 12:20:56 +02:00
Christian Schwarz
cd8796aed4 rpc: Initial|IncrementalTransferRequest transfer zfs data structures 2017-05-07 12:20:56 +02:00
Christian Schwarz
44d20ac256 zfs: ZFSListMapping fix broken logic 2017-05-06 23:45:33 +02:00
Christian Schwarz
aa696dd0ce zfs: use createtxg and guid properties for FilesystemVersion 2017-05-06 10:58:01 +02:00