Commit Graph

32 Commits

Author SHA1 Message Date
Christian Schwarz
1323a30a0c zfs: ability to specify sources for zfsGet
fix use for Placeholder, leave rest as previous behavior
2018-09-05 19:51:06 -07:00
Christian Schwarz
975fdee217 replication & pruning: ditch replicated-property, use bookmark as cursor instead
A bookmark with a well-known name is used to track which version was
last successfully received by the receiver.
The createtxg that can be retrieved from the bookmark using `zfs get` is
used to set the Replicated attribute of each snap on the sender:
If the snap's CreateTXG > the cursor's, it is not yet replicated,
otherwise it has been.

There is an optional config option to change the behvior to
`CreateTXG >= the cursor's`, and the implementation defaults to that.

The reason: While things work just fine with `CreateTXG > the cursor's`,
ZFS does not provide size estimates in a `zfs send` dry run
(see acd2418).
However, to enable the use case of keeping the snapshot only around for
the replication, the config flag exists.
2018-09-05 19:51:06 -07:00
Christian Schwarz
acd2418803 handle DryRun send size estimate errors with bookmarks 2018-09-05 17:41:25 -07:00
Christian Schwarz
4336af295f fixup 22ca80eb7e: scraping regex was broken and potentially mixed with stdout 2018-09-04 14:01:48 -07:00
Christian Schwarz
f0860767f5 zfs: include stderr of command in ZFSError.Error()
Since we don't implement screen-scraping of ZFS output ATM, this is
better than nothing, as user's may be able to figure out what' sthe
problem from the logs / status reports.
2018-09-02 15:46:42 -07:00
Anton Schirg
98f3f3dfd8 show expected size of current send
Needs to be changed to send sizes for all planned steps
2018-08-30 12:58:13 +02:00
Christian Schwarz
22ca80eb7e remote snapshot destruction & replication status zfs property 2018-08-30 11:51:47 +02:00
Christian Schwarz
1a8d2c5ebe replication: context support and propert closing of stale readers 2018-07-08 23:31:46 +02:00
Christian Schwarz
8cca0a8547 Initial working version
Summary:
* Logging is still bad
* test output in a lot of placed
* FIXMEs every where

Test Plan: None, just review

Differential Revision: https://phabricator.cschwarz.com/D2
2018-06-24 10:44:00 +02:00
Christian Schwarz
0764f8824e zfs: prometheus metrics
refs #67
2018-04-05 22:12:25 +02:00
Christian Schwarz
3ba3648f0f zfs: use channel as iterator for ZFSList results
The old approach with ZFSList would keep the two-dimensional array of
lines and their fields in memory (for a short time), which could easily
consume 100s of MiB with > 10000 snapshots / bookmarks (see #34)

fixes #61
2018-02-18 13:28:46 +01:00
Christian Schwarz
5f2c14adab zfs: use custom datatype to pass ZFS properties in ZFSSet
refs #55
2018-01-05 18:42:10 +01:00
Christian Schwarz
ed68bffea5 bookmark every snapshot
replication logic already supports bookmarks \o/

refs #34
2017-11-13 10:59:46 +01:00
Christian Schwarz
cba083cadf Make zfs.DatasetPath json.Marshaler and json.Unmarshaler
Had to resort to using pointers to zfs.DatasetPath everywhere... Should
find a better solution for that.
2017-08-06 16:22:15 +02:00
Christian Schwarz
2ce07c9342 rework filters & mappings
config defines a single datastructure that can act both as a Map and as a Filter
(DatasetMapFilter)

Cleanup wildcard syntax along the way (also changes semantics).
2017-08-06 16:21:54 +02:00
Christian Schwarz
655b3ab55f implement automatic snapshotting feature 2017-07-02 00:02:33 +02:00
Christian Schwarz
c7f140a00f zfs: support destroy 2017-07-01 23:19:31 +02:00
Christian Schwarz
ee570bb060 refactor: consolidate ForkReader-like implementations to IOCommand 2017-05-14 12:27:15 +02:00
Christian Schwarz
dd6dd60e98 Simplify "fork then io.Reader" abstractions 2017-05-13 15:23:37 +02:00
Christian Schwarz
54778c0374 zfs: remove erronous output from ForkReader 2017-05-13 15:23:28 +02:00
Christian Schwarz
6a420e872e zfs: implement ZFSSet 2017-05-07 12:23:12 +02:00
Christian Schwarz
d37bdc3d7a zfs: implement ZFSRecv 2017-05-07 12:22:57 +02:00
Christian Schwarz
030bd7affe zfs: implement ZFSSend 2017-05-07 12:20:56 +02:00
Christian Schwarz
cd8796aed4 rpc: Initial|IncrementalTransferRequest transfer zfs data structures 2017-05-07 12:20:56 +02:00
Christian Schwarz
aa696dd0ce zfs: use createtxg and guid properties for FilesystemVersion 2017-05-06 10:58:01 +02:00
Christian Schwarz
644d3dd06b zfs: public ZFSList() function + consolidation of mapping code 2017-05-01 20:35:04 +02:00
Christian Schwarz
28f0b90c25 zfs: check for forbidden dataset path characters 2017-05-01 20:09:35 +02:00
Christian Schwarz
ac5a911c64 zfs: glob mapping for complete wildcard ("*") 2017-04-30 23:47:11 +02:00
Christian Schwarz
d9ecfc8eb4 Gofmt megacommit. 2017-04-26 20:29:54 +02:00
Christian Schwarz
00231ecb73 Implement config parser. 2017-04-26 19:57:40 +02:00
Christian Schwarz
40f3b530e1 Implement DatasetMapping + basic ZFS list functionality. 2017-04-26 17:39:16 +02:00
Christian Schwarz
123becbd22 Interface wireframe 2017-04-14 19:26:32 +02:00