rclone/lib
Nick Craig-Wood 2db0e23584 backends: change OpenChunkWriter interface to allow backend concurrency override
Before this change the concurrency used for an upload was rather
inconsistent.

- if size below `--backend-upload-cutoff` (default 200M) do single part upload.

- if size below `--multi-thread-cutoff` (default 256M) or using streaming
  uploads (eg `rclone rcat) do multipart upload using
  `--backend-upload-concurrency` to set the concurrency used by the uploader.

- otherwise do multipart upload using `--multi-thread-streams` to set the
  concurrency.

This change makes the default for the concurrency used be the
`--backend-upload-concurrency`. If `--multi-thread-streams` is set and larger
than the `--backend-upload-concurrency` then that will be used instead.

This means that if the user sets `--backend-upload-concurrency` then it will be
obeyed for all multipart/multi-thread transfers and the user can override them
all with `--multi-thread-streams`.

See: #7056
2023-09-03 11:47:05 +01:00
..
atexit lib: use atomic types 2023-08-22 12:52:13 +01:00
bucket bucket: add Join function for a simplified path.Join 2023-03-03 15:51:00 +00:00
buildinfo build: add snap installation 2023-08-18 11:57:25 +01:00
cache lib/cache: fix alias backend shutting down too soon 2022-11-09 16:29:23 +00:00
daemonize build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
debug build: update to go1.21rc3 and make go1.19 the minimum required version 2023-07-16 10:09:25 +01:00
dircache lib: fix typos found by codespell 2023-03-24 20:40:52 +00:00
encoder Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
env docs: punctuation cleanup 2021-10-20 22:56:19 +02:00
errors lib: fix typos found by codespell 2023-03-24 20:40:52 +00:00
exitcode --max-transfer - add new exit code (10) 2023-07-18 09:51:31 +01:00
file lib: use atomic types 2023-08-22 12:52:13 +01:00
http docs: group the global flags and make them appear on command and flags pages 2023-08-02 12:53:09 +01:00
israce build: apply gofmt from golang 1.17 2021-09-09 20:43:59 +03:00
jwtutil Use jwt-go (golang-jwt) instead of deprecated jws (x/oauth2/jws) 2023-03-26 19:20:50 +02:00
kv build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
mmap build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
multipart backends: change OpenChunkWriter interface to allow backend concurrency override 2023-09-03 11:47:05 +01:00
oauthutil lib: use atomic types 2023-08-22 12:52:13 +01:00
pacer build: reformat comments to pass go1.19 vet 2022-08-05 16:35:41 +01:00
plugin Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
pool lib/pool: add DelayAccounting() to fix accounting when reading hashes 2023-08-24 16:42:09 +01:00
proxy build: fix lint errors when re-enabling revive exported & package-comments 2023-08-29 13:03:13 +01:00
random Remove github.com/pkg/errors and replace with std library version 2021-11-07 11:53:30 +00:00
ranges Spelling fixes 2020-10-14 15:21:31 +01:00
readers lib/readers: add NoSeeker to adapt io.Reader to io.ReadSeeker 2023-08-24 12:39:27 +01:00
rest lib/rest: remove unnecessary nil check 2023-08-09 19:17:42 +09:00
structs test: do not test deprecated and unused Dial and DialTLS functions on http Transport type 2023-03-26 14:28:15 +02:00
terminal lib: fix typos found by codespell 2023-03-24 20:40:52 +00:00
version Fix lint issues reported by staticcheck 2022-06-13 21:13:50 +02:00