rclone/fs/list
Nick Craig-Wood 59a5530ce7 sync: implement --list-cutoff to allow on disk sorting for reduced memory use
Before this change, rclone had to load an entire directory into RAM in
order to sort it so it could be synced.

With directories with millions of entries, this used too much memory.

This fixes the probem by using an on disk sort when there are more
than --list-cutoff entries in a directory.

Fixes #7974
2024-12-18 15:30:13 +00:00
..
helpers_test.go list: add WithListP helper to implement List for ListP backends 2024-12-18 15:30:12 +00:00
helpers.go list: add WithListP helper to implement List for ListP backends 2024-12-18 15:30:12 +00:00
list_test.go fs: allow Metadata calls to be called with Directory or Object 2024-02-27 10:56:19 +00:00
list.go sync: implement --list-cutoff to allow on disk sorting for reduced memory use 2024-12-18 15:30:13 +00:00
sorter_test.go sync: implement --list-cutoff to allow on disk sorting for reduced memory use 2024-12-18 15:30:13 +00:00
sorter.go sync: implement --list-cutoff to allow on disk sorting for reduced memory use 2024-12-18 15:30:13 +00:00