rclone/backend
Nick Craig-Wood 351fc609b1 b2: fix uploading files bigger than 1TiB
Before this change when uploading files bigger than 1TiB, the chunk
calculator would work out that the chunk size needed to be bigger than
the default 100 MiB to fit within the 10,000 parts limit.

However the uploader was still using the memory pool for the old chunk
size and this caused errors like

    panic: runtime error: slice bounds out of range [:122683392] with capacity 100663296

The fix for this is to make a temporary pool with the larger chunk
size and use it during the upload of the large file.

See: https://forum.rclone.org/t/rclone-cannot-complete-upload-to-b2-restarts-upload-frequently/35617/
2023-01-22 12:46:23 +00:00
..
alias build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
all Add a native backend for oracle object storage - fixes #6299 2022-10-13 13:04:56 +01:00
amazonclouddrive build: reformat comments to pass go1.19 vet 2022-08-05 16:35:41 +01:00
azureblob azureblob: remove workarounds for SDK bugs after v0.6.1 update 2023-01-16 11:19:16 +00:00
b2 b2: fix uploading files bigger than 1TiB 2023-01-22 12:46:23 +00:00
box Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
cache test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
chunker rcat: preserve metadata when Copy falls back to Rcat 2022-11-10 12:04:35 +00:00
combine fs: move operations.NewOverrideRemote to fs.NewOverrideRemote 2022-11-21 08:02:09 +00:00
compress Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
crypt crypt: fix for unencrypted directory names on case insensitive remotes 2023-01-11 16:32:40 +00:00
drive drive: handle shared drives with leading/trailing space in name (related to #6618) 2022-12-14 10:18:12 +01:00
dropbox all: fix spelling across the project 2022-08-30 11:16:26 +02:00
fichier all: fix spelling across the project 2022-08-30 11:16:26 +02:00
filefabric Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
ftp ftp: Improve performance to speed up --files-from and NewObject 2022-12-05 16:19:04 +00:00
googlecloudstorage Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
googlephotos fs: move operations.NewOverrideRemote to fs.NewOverrideRemote 2022-11-21 08:02:09 +00:00
hasher Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
hdfs build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
hidrive all: fix spelling across the project 2022-08-30 11:16:26 +02:00
http test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
internetarchive all: fix spelling across the project 2022-08-30 11:16:26 +02:00
jottacloud Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
koofr all: fix spelling across the project 2022-08-30 11:16:26 +02:00
local backend/local: fix %!w(<nil>) in "failed to read directory" error 2022-12-26 18:37:32 +01:00
mailru mailru: allow timestamps to be before the epoch 1970-01-01 2022-11-10 11:27:01 +00:00
mega all: fix spelling across the project 2022-08-30 11:16:26 +02:00
memory Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
netstorage Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
onedrive onedrive: disable change notify in China region since it is not supported 2022-09-16 16:57:29 +01:00
opendrive build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
oracleobjectstorage oracle-object-storage: expose the storage_tier option in config 2023-01-15 16:20:55 +00:00
pcloud all: fix spelling across the project 2022-08-30 11:16:26 +02:00
premiumizeme build: reformat comments to pass go1.19 vet 2022-08-05 16:35:41 +01:00
putio build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
qingstor build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
s3 s3/storj: update endpoints 2022-12-22 15:46:49 +00:00
seafile seafile: renew library password - fixes #6662 2023-01-15 16:26:29 +00:00
sftp sftp: fix NewObject with leading / 2022-12-15 12:40:59 +00:00
sharefile Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
sia build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
smb smb: fix issue where spurious dot directory is created 2022-11-20 17:12:02 +00:00
storj storj: implement purge 2023-01-15 16:23:49 +00:00
sugarsync Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
swift Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
union Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
uptobox Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
webdav webdav: fix tests after go1.20 upgrade 2023-01-16 11:19:16 +00:00
yandex build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
zoho Replace deprecated ioutil 2022-11-07 11:41:47 +00:00