rclone/fs/operations
nielash fd8faeb0e6 vfs: fix unicode normalization on macOS - fixes #7072
Before this change, the VFS layer did not properly handle unicode normalization,
which caused problems particularly for users of macOS. While attempts were made
to handle it with various `-o modules=iconv` combinations, this was an imperfect
solution, as no one combination allowed both NFC and NFD content to
simultaneously be both visible and editable via Finder.

After this change, the VFS supports `--no-unicode-normalization` (default `false`)
via the existing `--vfs-case-insensitive` logic, which is extended to apply to both
case insensitivity and unicode normalization form.

This change also adds an additional flag, `--vfs-block-norm-dupes`, to address a
probably rare but potentially possible scenario where a directory contains
multiple duplicate filenames after applying case and unicode normalization
settings. In such a scenario, this flag (disabled by default) hides the
duplicates. This comes with a performance tradeoff, as rclone will have to scan
the entire directory for duplicates when listing a directory. For this reason,
it is recommended to leave this disabled if not needed. However, macOS users may
wish to consider using it, as otherwise, if a remote directory contains both NFC
and NFD versions of the same filename, an odd situation will occur: both
versions of the file will be visible in the mount, and both will appear to be
editable, however, editing either version will actually result in only the NFD
version getting edited under the hood. `--vfs-block-norm-dupes` prevents this
confusion by detecting this scenario, hiding the duplicates, and logging an
error, similar to how this is handled in `rclone sync`.
2024-03-06 16:12:13 +00:00
..
operationsflags sync: report list of synced paths to file -- see #7282 2024-01-20 14:50:08 -05:00
check_test.go operations: skip backends incapable of testing TestApplyTransforms - #7591 2024-02-13 15:07:41 -05:00
check.go vfs: fix unicode normalization on macOS - fixes #7072 2024-03-06 16:12:13 +00:00
copy_test.go operations: fix invalid UTF-8 when truncating file names when not using --inplace 2023-10-29 14:04:37 +00:00
copy.go rc: add srcFs and dstFs to core/stats and core/transferred stats 2024-02-02 11:43:10 +00:00
dedupe_test.go test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
dedupe.go accounting: Make checkers show what they are doing 2023-03-01 11:10:38 +00:00
listdirsorted_test.go test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
logger.go sync: report list of synced paths to file -- see #7282 2024-01-20 14:50:08 -05:00
lsjson_test.go lsjson,lsf: make sure metadata appears for directories 2024-02-28 16:26:14 +00:00
lsjson.go lsjson,lsf: make sure metadata appears for directories 2024-02-28 16:26:14 +00:00
multithread_test.go rc: add srcFs and dstFs to core/stats and core/transferred stats 2024-02-02 11:43:10 +00:00
multithread.go operations: use built in io.OffsetWriter for go1.20 2024-01-15 16:22:07 +00:00
operations_internal_test.go fs: deglobalise the config #4685 2020-11-26 16:40:12 +00:00
operations_test.go operations: fix TestSetDirModTime for backends with SetDirModTime but not Metadata 2024-03-01 11:39:21 +00:00
operations.go operations: fix TestMkdirModTime test 2024-03-01 11:18:24 +00:00
rc_test.go operations: add operations/hashsum to the rc as rclone hashsum equivalent 2024-02-29 16:21:42 +00:00
rc.go operations: add operations/hashsum to the rc as rclone hashsum equivalent 2024-02-29 16:21:42 +00:00
reopen_test.go operations: make Open() return an io.ReadSeekCloser #7350 2023-11-20 18:07:05 +00:00
reopen.go operations: make Open() return an io.ReadSeekCloser #7350 2023-11-20 18:07:05 +00:00