rclone/fs/operations
Nick Craig-Wood 2db0e23584 backends: change OpenChunkWriter interface to allow backend concurrency override
Before this change the concurrency used for an upload was rather
inconsistent.

- if size below `--backend-upload-cutoff` (default 200M) do single part upload.

- if size below `--multi-thread-cutoff` (default 256M) or using streaming
  uploads (eg `rclone rcat) do multipart upload using
  `--backend-upload-concurrency` to set the concurrency used by the uploader.

- otherwise do multipart upload using `--multi-thread-streams` to set the
  concurrency.

This change makes the default for the concurrency used be the
`--backend-upload-concurrency`. If `--multi-thread-streams` is set and larger
than the `--backend-upload-concurrency` then that will be used instead.

This means that if the user sets `--backend-upload-concurrency` then it will be
obeyed for all multipart/multi-thread transfers and the user can override them
all with `--multi-thread-streams`.

See: #7056
2023-09-03 11:47:05 +01:00
..
check_test.go test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
check.go fs: use atomic types 2023-08-22 12:52:13 +01:00
dedupe_test.go test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
dedupe.go accounting: Make checkers show what they are doing 2023-03-01 11:10:38 +00:00
listdirsorted_test.go test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
lsjson_test.go rc: fix operations/stat with trailing / 2023-03-22 16:22:45 +00:00
lsjson.go lsjson: make --stat more efficient 2023-04-28 14:31:05 +01:00
multithread_test.go multithread: refactor multithread operation to use OpenChunkWriter if available #7056 2023-08-12 17:55:01 +01:00
multithread.go backends: change OpenChunkWriter interface to allow backend concurrency override 2023-09-03 11:47:05 +01:00
operations_internal_test.go fs: deglobalise the config #4685 2020-11-26 16:40:12 +00:00
operations_test.go rmdirs: remove directories concurrently controlled by --checkers 2023-08-18 12:05:15 +01:00
operations.go operations: document multi-thread copy and tweak defaults 2023-08-24 12:39:27 +01:00
rc_test.go test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
rc.go doc: Clarify the srcFs and dstRs when using local filesystem 2023-03-24 11:25:39 +00:00
reopen_test.go Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
reopen.go operations: reopen downloads on error when using check --download and cat 2023-06-10 14:42:29 +01:00