rclone/vfs
nielash fd8faeb0e6 vfs: fix unicode normalization on macOS - fixes #7072
Before this change, the VFS layer did not properly handle unicode normalization,
which caused problems particularly for users of macOS. While attempts were made
to handle it with various `-o modules=iconv` combinations, this was an imperfect
solution, as no one combination allowed both NFC and NFD content to
simultaneously be both visible and editable via Finder.

After this change, the VFS supports `--no-unicode-normalization` (default `false`)
via the existing `--vfs-case-insensitive` logic, which is extended to apply to both
case insensitivity and unicode normalization form.

This change also adds an additional flag, `--vfs-block-norm-dupes`, to address a
probably rare but potentially possible scenario where a directory contains
multiple duplicate filenames after applying case and unicode normalization
settings. In such a scenario, this flag (disabled by default) hides the
duplicates. This comes with a performance tradeoff, as rclone will have to scan
the entire directory for duplicates when listing a directory. For this reason,
it is recommended to leave this disabled if not needed. However, macOS users may
wish to consider using it, as otherwise, if a remote directory contains both NFC
and NFD versions of the same filename, an odd situation will occur: both
versions of the file will be visible in the mount, and both will appear to be
editable, however, editing either version will actually result in only the NFD
version getting edited under the hood. `--vfs-block-norm-dupes` prevents this
confusion by detecting this scenario, hiding the duplicates, and logging an
error, similar to how this is handled in `rclone sync`.
2024-03-06 16:12:13 +00:00
..
test_vfs build: remove random.Seed since random generator is seeded automatically in go1.20 2024-01-15 16:22:07 +00:00
vfscache rc: add srcFs and dstFs to core/stats and core/transferred stats 2024-02-02 11:43:10 +00:00
vfscommon vfs: fix unicode normalization on macOS - fixes #7072 2024-03-06 16:12:13 +00:00
vfsflags vfs: fix unicode normalization on macOS - fixes #7072 2024-03-06 16:12:13 +00:00
vfstest nfsmount: New mount command to provide mount mechanism on macOS without FUSE 2023-10-06 14:08:20 +01:00
dir_handle_test.go test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
dir_handle.go Spelling fixes 2020-10-14 15:21:31 +01:00
dir_test.go vfs: Update parent directory modtimes on vfs actions 2023-09-08 17:19:52 +01:00
dir.go vfs: fix unicode normalization on macOS - fixes #7072 2024-03-06 16:12:13 +00:00
errors_test.go vfs: add tests and subsequent fixes 2017-11-04 10:24:10 +00:00
errors.go build: drop support for go1.8 2019-04-15 21:49:58 +01:00
file_test.go mockfs: make it so it can be registered as an Fs 2023-04-28 14:31:05 +01:00
file.go vfs: use atomic types 2023-08-22 12:52:13 +01:00
make_open_tests.go Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
open_test.go vfs: bring DO NOT EDIT comments in line with "go help generate" 2020-04-30 12:24:44 +01:00
rc_test.go test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
rc.go vfs: add vfs/stats remote control to show statistics - fixes #5816 2021-11-23 18:00:21 +00:00
read_test.go vfs: [bugfix] Implement Name() method in WriteFileHandle and ReadFileHandle 2023-10-06 14:08:20 +01:00
read_write_test.go vfs: fix stale data when using --vfs-cache-mode full 2024-01-15 11:12:59 +00:00
read_write.go vfs: Add go-billy dependency and make sure vfs.Handle implements billy.File 2023-10-06 14:08:20 +01:00
read.go rc: add srcFs and dstFs to core/stats and core/transferred stats 2024-02-02 11:43:10 +00:00
vfs_case_test.go vfs: fix unicode normalization on macOS - fixes #7072 2024-03-06 16:12:13 +00:00
vfs_test.go vfs: add MkdirAll function to make a directory and all beneath 2023-04-13 18:00:22 +01:00
vfs.go docs: factor large docs into separate .md files to make them easier to maintain. 2023-11-13 16:27:09 +00:00
vfs.md vfs: fix unicode normalization on macOS - fixes #7072 2024-03-06 16:12:13 +00:00
vfstest_test.go nfsmount: New mount command to provide mount mechanism on macOS without FUSE 2023-10-06 14:08:20 +01:00
vstate_string.go vfs: fix virtual entries causing deleted files to still appear 2020-11-10 16:47:25 +00:00
write_test.go vfs: [bugfix] Implement Name() method in WriteFileHandle and ReadFileHandle 2023-10-06 14:08:20 +01:00
write.go vfs: [bugfix] Implement Name() method in WriteFileHandle and ReadFileHandle 2023-10-06 14:08:20 +01:00