extern/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2024-11-26 02:14:42 +01:00
Code Issues Packages Projects Releases Wiki Activity
381b845307
rclone/fs
History
Nick Craig-Wood 48cdedc97b Re-implement sync routine to work a directory at a time 
Multiple directories (up to --checkers worth) are scanned at once.

This uses much less memory than the previous scheme - only the amount
of memory needed to hold an entire directory listing of objects.

For directory based remotes the speed is unchanged.

For bucket based remotes, instead of doing one API call to list the
whole bucket, it does multiple calls, one for each pseudo directory.
However these are done in parallel so in practice this seems to speed
up directory listings.

This replaces the existing sync method as it performs faster and uses
less memory.

The old sync method is available with the temporary --old-sync-method
flag.

Fixes #517
Fixes #439
Fixes #236
Fixes #1067
2017-02-04 10:30:25 +00:00
..
all Basic SFTP support, Issue #521 2017-02-04 10:29:18 +00:00
testdata Add configuration file encryption 2016-02-16 16:32:05 +01:00
accounting_other.go Fix non-windows/non-unix builds for bwlimit/SIGUSR2 feature and add a mutex 2016-10-30 19:20:16 +00:00
accounting_unix.go Fix non-windows/non-unix builds for bwlimit/SIGUSR2 feature and add a mutex 2016-10-30 19:20:16 +00:00
accounting.go Only start bandwidth ticker when necessary. 2017-01-04 19:03:49 -08:00
buffer_test.go Convert more tests to use assert/require 2016-06-30 15:45:30 +01:00
buffer.go Convert to using github.com/pkg/errors everywhere 2016-06-13 17:43:03 +01:00
closed_conn_win.go Unwrap errors properly for patform specific connection retry code. 2016-06-10 13:48:41 +01:00
closed_conn.go Retry errors which indicate the connection closed prematurely. 2016-04-29 17:29:34 +01:00
config_read_password_unsupported.go Stub out ReadPassword on plan9 and solaris to fix compilation 2016-02-21 10:31:53 +00:00
config_read_password.go Stub out ReadPassword on plan9 and solaris to fix compilation 2016-02-21 10:31:53 +00:00
config_test.go Allow all options to be set from environment variables 2017-01-03 22:42:47 +00:00
config.go Make --delete-after the default and refactor --delete-{before,during,after} parsing 2017-02-04 10:30:25 +00:00
driveletter_windows.go windows: Stop drive letters (eg C:) getting mixed up with remotes (eg drive:) 2015-02-07 22:32:51 +00:00
driveletter.go windows: Stop drive letters (eg C:) getting mixed up with remotes (eg drive:) 2015-02-07 22:32:51 +00:00
error.go Make sure wrapped retry/fatal errors are never nil to avoid panic 2016-12-13 16:02:14 +00:00
filter_test.go Fix filters to add ** rules to directory rules 2017-01-05 19:33:49 +00:00
filter.go Fix filters to add ** rules to directory rules 2017-01-05 19:33:49 +00:00
flags_test.go Allow all options to be set from environment variables 2017-01-03 22:42:47 +00:00
flags.go Allow all options to be set from environment variables 2017-01-03 22:42:47 +00:00
fs.go Define a new Features() method for Fs 2017-01-16 17:33:25 +00:00
glob_test.go Convert to using github.com/pkg/errors everywhere 2016-06-13 17:43:03 +01:00
glob.go Convert to using github.com/pkg/errors everywhere 2016-06-13 17:43:03 +01:00
hash_test.go Convert more tests to use assert/require 2016-06-30 15:45:30 +01:00
hash.go Fix incomplete local hashes. 2016-06-19 16:51:49 +02:00
http_new.go Redo http Transport code 2016-09-12 17:50:19 +01:00
http_old.go Redo http Transport code 2016-09-12 17:50:19 +01:00
http_test.go Remove Authorization: headers from --dump-headers output 2016-11-02 15:53:43 +00:00
http.go Remove Authorization: headers from --dump-headers output 2016-11-02 15:53:43 +00:00
lister_test.go Stop single file and --files-from operations iterating through the source bucket. 2016-10-07 11:39:39 +01:00
lister.go Fix race in Lister.Finished which was causing the tests to be unreliable 2017-01-19 20:11:17 +00:00
make_test_files.go A script to make a directory heirarchy for testing 2016-03-05 20:26:15 +00:00
operations_test.go Add ListDirSorted function to list a directory 2017-02-04 10:30:25 +00:00
operations.go Add ListDirSorted function to list a directory 2017-02-04 10:30:25 +00:00
options.go Add options for Open and implement Range for all remotes 2016-10-05 21:03:56 +01:00
sync_test.go Re-implement sync routine to work a directory at a time 2017-02-04 10:30:25 +00:00
sync.go Re-implement sync routine to work a directory at a time 2017-02-04 10:30:25 +00:00
test_all.go Complete SFTP remote #521 2017-02-04 10:29:46 +00:00
version.go Version v1.35 2017-01-02 15:33:06 +00:00
versioncheck.go Move versioncheck so it happens earlier in the compile process. 2016-09-12 17:50:19 +01:00