mirror of
https://github.com/rclone/rclone.git
synced 2024-12-13 10:45:16 +01:00
810644e873
Before this change, the path1 version of a file always prevailed during --resync, and many users requested options to automatically select the winner based on characteristics such as newer, older, larger, and smaller. This change adds support for such options. Note that ideally this feature would have been implemented by allowing the existing `--resync` flag to optionally accept string values such as `--resync newer`. However, this would have been a breaking change, as the existing flag is a `bool` and it does not seem to be possible to have a `string` flag that accepts both `--resync newer` and `--resync` (with no argument.) (`NoOptDefVal` does not work for this, as it would force an `=` like `--resync=newer`.) So instead, the best compromise to avoid a breaking change was to add a new `--resync-mode CHOICE` flag that implies `--resync`, while maintaining the existing behavior of `--resync` (which implies `--resync-mode path1`. i.e. both flags are now valid, and either can be used without the other. --resync-mode CHOICE In the event that a file differs on both sides during a `--resync`, `--resync-mode` controls which version will overwrite the other. The supported options are similar to `--conflict-resolve`. For all of the following options, the version that is kept is referred to as the "winner", and the version that is overwritten (deleted) is referred to as the "loser". The options are named after the "winner": - `path1` - (the default) - the version from Path1 is unconditionally considered the winner (regardless of `modtime` and `size`, if any). This can be useful if one side is more trusted or up-to-date than the other, at the time of the `--resync`. - `path2` - same as `path1`, except the path2 version is considered the winner. - `newer` - the newer file (by `modtime`) is considered the winner, regardless of which side it came from. This may result in having a mix of some winners from Path1, and some winners from Path2. (The implementation is analagous to running `rclone copy --update` in both directions.) - `older` - same as `newer`, except the older file is considered the winner, and the newer file is considered the loser. - `larger` - the larger file (by `size`) is considered the winner (regardless of `modtime`, if any). This can be a useful option for remotes without `modtime` support, or with the kinds of files (such as logs) that tend to grow but not shrink, over time. - `smaller` - the smaller file (by `size`) is considered the winner (regardless of `modtime`, if any). For all of the above options, note the following: - If either of the underlying remotes lacks support for the chosen method, it will be ignored and will fall back to the default of `path1`. (For example, if `--resync-mode newer` is set, but one of the paths uses a remote that doesn't support `modtime`.) - If a winner can't be determined because the chosen method's attribute is missing or equal, it will be ignored, and bisync will instead try to determine whether the files differ by looking at the other `--compare` methods in effect. (For example, if `--resync-mode newer` is set, but the Path1 and Path2 modtimes are identical, bisync will compare the sizes.) If bisync concludes that they differ, preference is given to whichever is the "source" at that moment. (In practice, this gives a slight advantage to Path2, as the 2to1 copy comes before the 1to2 copy.) If the files _do not_ differ, nothing is copied (as both sides are already correct). - These options apply only to files that exist on both sides (with the same name and relative path). Files that exist *only* on one side and not the other are *always* copied to the other, during `--resync` (this is one of the main differences between resync and non-resync runs.). - `--conflict-resolve`, `--conflict-loser`, and `--conflict-suffix` do not apply during `--resync`, and unlike these flags, nothing is renamed during `--resync`. When a file differs on both sides during `--resync`, one version always overwrites the other (much like in `rclone copy`.) (Consider using `--backup-dir` to retain a backup of the losing version.) - Unlike for `--conflict-resolve`, `--resync-mode none` is not a valid option (or rather, it will be interpreted as "no resync", unless `--resync` has also been specified, in which case it will be ignored.) - Winners and losers are decided at the individual file-level only (there is not currently an option to pick an entire winning directory atomically, although the `path1` and `path2` options typically produce a similar result.) - To maintain backward-compatibility, the `--resync` flag implies `--resync-mode path1` unless a different `--resync-mode` is explicitly specified. Similarly, all `--resync-mode` options (except `none`) imply `--resync`, so it is not necessary to use both the `--resync` and `--resync-mode` flags simultaneously -- either one is sufficient without the other.
9.6 KiB
9.6 KiB
(01) : test extended-char-paths
(02) : fix-names {path1/}測試_Русский_{spc}_{spc}_ě_áñ/
(03) : fix-names {path2/}測試_Русский_{spc}_{spc}_ě_áñ/
(04) : fix-names {path1/}測試_Русский_{spc}_{spc}_ě_áñ/測試_check{spc}file
(05) : fix-names {path2/}測試_Русский_{spc}_{spc}_ě_áñ/測試_check{spc}file
(06) : fix-names {path1/}測試_Русский_{spc}_{spc}_ě_áñ/filename_contains_ě_.txt
(07) : fix-names {path2/}測試_Русский_{spc}_{spc}_ě_áñ/filename_contains_ě_.txt
(08) : fix-names {path1/}測試_check{spc}file
(09) : fix-names {path2/}測試_check{spc}file
(10) : fix-names {path1/}測試_Русский_{spc}_{spc}_ě_áñ/測試_check{spc}file
(11) : fix-names {path2/}測試_Русский_{spc}_{spc}_ě_áñ/測試_check{spc}file
(12) : test resync subdirs with extended chars
(13) : bisync subdir=測試_Русский_{spc}_{spc}_ě_áñ resync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}測試_Русский_ _ _ě_áñ/" with Path2 "{path2/}測試_Русский_ _ _ě_áñ/"
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}測試_Русский_ _ _ě_áñ/" vs Path2 "{path2/}測試_Русский_ _ _ě_áñ/"
INFO : Bisync successful
(14) : copy-listings resync
(15) : test place new files with extended chars on each side
(16) : touch-glob 2001-01-02 {datadir/} file1.txt
(17) : copy-as {datadir/}file1.txt {path1/}測試_Русский_{spc}_{spc}_ě_áñ 測試_file1p1
(18) : copy-as {datadir/}file1.txt {path2/}測試_Русский_{spc}_{spc}_ě_áñ 測試_file1p2
(19) : test normal sync of subdirs with extended chars
(20) : bisync subdir=測試_Русский_{spc}_{spc}_ě_áñ
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}測試_Русский_ _ _ě_áñ/" with Path2 "{path2/}測試_Русский_ _ _ě_áñ/"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : - Path1 File is new - 測試_file1p1
INFO : Path1: 1 changes: 1 new, 0 modified, 0 deleted
INFO : Path2 checking for diffs
INFO : - Path2 File is new - 測試_file1p2
INFO : Path2: 1 changes: 1 new, 0 modified, 0 deleted
INFO : Applying changes
INFO : - Path1 Queue copy to Path2 - {path2/}測試_Русский_ _ _ě_áñ/測試_file1p1
INFO : - Path2 Queue copy to Path1 - {path1/}測試_Русский_ _ _ě_áñ/測試_file1p2
INFO : - Path2 Do queued copies to - Path1
INFO : - Path1 Do queued copies to - Path2
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}測試_Русский_ _ _ě_áñ/" vs Path2 "{path2/}測試_Русский_ _ _ě_áñ/"
INFO : Bisync successful
(21) : move-listings normal-sync
(22) : test check-filename with extended chars. check should fail.
(23) : bisync resync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(24) : delete-file {path1/}測試_Русский_{spc}_{spc}_ě_áñ/測試_check{spc}file
(25) : bisync check-access check-filename=測試_check{spc}file
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : - Path1 File was deleted - 測試_Русский_ _ _ě_áñ/測試_check file
INFO : Path1: 1 changes: 0 new, 0 modified, 1 deleted
INFO : Path2 checking for diffs
INFO : Checking access health
ERROR : Access test failed: Path1 count 1, Path2 count 2 - 測試_check file
ERROR : - Access test failed: Path2 file not found in Path1 - 測試_Русский_ _ _ě_áñ/測試_check file
ERROR : Bisync critical error: check file check failed
ERROR : Bisync aborted. Must run --resync to recover.
Bisync error: bisync aborted
(26) : copy-listings check-access-fail
(27) : test check-filename with extended chars. check should pass.
(28) : bisync resync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(29) : bisync check-access check-filename=測試_check{spc}file
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : Path2 checking for diffs
INFO : Checking access health
INFO : Found 2 matching "測試_check file" files on both paths
INFO : No changes found
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(30) : move-listings check-access-pass
(31) : test filters-file path with extended chars - masks /fileZ.txt
(32) : copy-file {datadir/}測試_filtersfile.txt {workdir/}
(33) : bisync filters-file={workdir/}測試_filtersfile.txt resync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Using filters file {workdir/}測試_filtersfile.txt
INFO : Storing filters file hash to {workdir/}測試_filtersfile.txt.md5
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(34) : copy-as {datadir/}file1.txt {path1/} fileZ.txt
(35) : bisync filters-file={workdir/}測試_filtersfile.txt
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Using filters file {workdir/}測試_filtersfile.txt
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : Path2 checking for diffs
INFO : No changes found
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(02) : fix-names {path1/}測試_Русский_{spc}_{spc}_ě_áñ/
(03) : fix-names {path2/}測試_Русский_{spc}_{spc}_ě_áñ/
(04) : fix-names {path1/}測試_Русский_{spc}_{spc}_ě_áñ/測試_check{spc}file
(05) : fix-names {path2/}測試_Русский_{spc}_{spc}_ě_áñ/測試_check{spc}file
(06) : fix-names {path1/}測試_Русский_{spc}_{spc}_ě_áñ/filename_contains_ě_.txt
(07) : fix-names {path2/}測試_Русский_{spc}_{spc}_ě_áñ/filename_contains_ě_.txt
(08) : fix-names {path1/}測試_check{spc}file
(09) : fix-names {path2/}測試_check{spc}file
(10) : fix-names {path1/}測試_Русский_{spc}_{spc}_ě_áñ/測試_check{spc}file
(11) : fix-names {path2/}測試_Русский_{spc}_{spc}_ě_áñ/測試_check{spc}file
(12) : test resync subdirs with extended chars
(13) : bisync subdir=測試_Русский_{spc}_{spc}_ě_áñ resync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}測試_Русский_ _ _ě_áñ/" with Path2 "{path2/}測試_Русский_ _ _ě_áñ/"
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}測試_Русский_ _ _ě_áñ/" vs Path2 "{path2/}測試_Русский_ _ _ě_áñ/"
INFO : Bisync successful
(14) : copy-listings resync
(15) : test place new files with extended chars on each side
(16) : touch-glob 2001-01-02 {datadir/} file1.txt
(17) : copy-as {datadir/}file1.txt {path1/}測試_Русский_{spc}_{spc}_ě_áñ 測試_file1p1
(18) : copy-as {datadir/}file1.txt {path2/}測試_Русский_{spc}_{spc}_ě_áñ 測試_file1p2
(19) : test normal sync of subdirs with extended chars
(20) : bisync subdir=測試_Русский_{spc}_{spc}_ě_áñ
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}測試_Русский_ _ _ě_áñ/" with Path2 "{path2/}測試_Русский_ _ _ě_áñ/"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : - Path1 File is new - 測試_file1p1
INFO : Path1: 1 changes: 1 new, 0 modified, 0 deleted
INFO : Path2 checking for diffs
INFO : - Path2 File is new - 測試_file1p2
INFO : Path2: 1 changes: 1 new, 0 modified, 0 deleted
INFO : Applying changes
INFO : - Path1 Queue copy to Path2 - {path2/}測試_Русский_ _ _ě_áñ/測試_file1p1
INFO : - Path2 Queue copy to Path1 - {path1/}測試_Русский_ _ _ě_áñ/測試_file1p2
INFO : - Path2 Do queued copies to - Path1
INFO : - Path1 Do queued copies to - Path2
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}測試_Русский_ _ _ě_áñ/" vs Path2 "{path2/}測試_Русский_ _ _ě_áñ/"
INFO : Bisync successful
(21) : move-listings normal-sync
(22) : test check-filename with extended chars. check should fail.
(23) : bisync resync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(24) : delete-file {path1/}測試_Русский_{spc}_{spc}_ě_áñ/測試_check{spc}file
(25) : bisync check-access check-filename=測試_check{spc}file
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : - Path1 File was deleted - 測試_Русский_ _ _ě_áñ/測試_check file
INFO : Path1: 1 changes: 0 new, 0 modified, 1 deleted
INFO : Path2 checking for diffs
INFO : Checking access health
ERROR : Access test failed: Path1 count 1, Path2 count 2 - 測試_check file
ERROR : - Access test failed: Path2 file not found in Path1 - 測試_Русский_ _ _ě_áñ/測試_check file
ERROR : Bisync critical error: check file check failed
ERROR : Bisync aborted. Must run --resync to recover.
Bisync error: bisync aborted
(26) : copy-listings check-access-fail
(27) : test check-filename with extended chars. check should pass.
(28) : bisync resync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(29) : bisync check-access check-filename=測試_check{spc}file
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : Path2 checking for diffs
INFO : Checking access health
INFO : Found 2 matching "測試_check file" files on both paths
INFO : No changes found
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(30) : move-listings check-access-pass
(31) : test filters-file path with extended chars - masks /fileZ.txt
(32) : copy-file {datadir/}測試_filtersfile.txt {workdir/}
(33) : bisync filters-file={workdir/}測試_filtersfile.txt resync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Using filters file {workdir/}測試_filtersfile.txt
INFO : Storing filters file hash to {workdir/}測試_filtersfile.txt.md5
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(34) : copy-as {datadir/}file1.txt {path1/} fileZ.txt
(35) : bisync filters-file={workdir/}測試_filtersfile.txt
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Using filters file {workdir/}測試_filtersfile.txt
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : Path2 checking for diffs
INFO : No changes found
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful