rclone/backend
Nick Craig-Wood 5b5fdc6bc5 s3: add provider quirk --s3-might-gzip to fix corrupted on transfer: sizes differ
Before this change, some files were giving this error when downloaded
from Cloudflare and other providers.

    ERROR corrupted on transfer: sizes differ NNN vs MMM

This is because these providers auto gzips the object when rclone
wasn't expecting it to. (AWS does not gzip objects without their being
uploaded gzipped).

This patch adds a quirk to for fix the problem and a flag to control
it. The quirk `might_gzip` is set to `true` for all providers except
AWS.

See: https://forum.rclone.org/t/s3-error-corrupted-on-transfer-sizes-differ-nnn-vs-mmm/33694/
Fixes: #6533
2022-11-04 16:53:32 +00:00
..
alias build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
all Add a native backend for oracle object storage - fixes #6299 2022-10-13 13:04:56 +01:00
amazonclouddrive build: reformat comments to pass go1.19 vet 2022-08-05 16:35:41 +01:00
azureblob build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
b2 all: fix spelling across the project 2022-08-30 11:16:26 +02:00
box all: fix spelling across the project 2022-08-30 11:16:26 +02:00
cache build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
chunker all: fix spelling across the project 2022-08-30 11:16:26 +02:00
combine all: fix spelling across the project 2022-08-30 11:16:26 +02:00
compress compress: add extra debugging in case we have a repeat of #6434 2022-09-18 11:31:11 +01:00
crypt all: fix spelling across the project 2022-08-30 11:16:26 +02:00
drive filter: Fix incorrect filtering with UseFilter context flag and wrapping backends 2022-09-05 16:19:50 +01:00
dropbox all: fix spelling across the project 2022-08-30 11:16:26 +02:00
fichier all: fix spelling across the project 2022-08-30 11:16:26 +02:00
filefabric all: fix spelling across the project 2022-08-30 11:16:26 +02:00
ftp ftp: Fix hang when using ExplicitTLS to certain servers. 2022-10-14 12:10:03 +01:00
googlecloudstorage all: fix spelling across the project 2022-08-30 11:16:26 +02:00
googlephotos onedrive: cleanup brand name 2022-08-30 10:23:29 +02:00
hasher hasher: support metadata 2022-06-29 17:30:37 +01:00
hdfs build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
hidrive all: fix spelling across the project 2022-08-30 11:16:26 +02:00
http docs: skip "Connection" suffix from FTP, SSH/SFTP and HTTP backend names 2022-06-21 23:43:00 +02:00
internetarchive all: fix spelling across the project 2022-08-30 11:16:26 +02:00
jottacloud all: fix spelling across the project 2022-08-30 11:16:26 +02:00
koofr all: fix spelling across the project 2022-08-30 11:16:26 +02:00
local local: clean absolute paths - fixes #6493 2022-10-25 21:09:56 +02:00
mailru build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
mega all: fix spelling across the project 2022-08-30 11:16:26 +02:00
memory build: reformat comments to pass go1.19 vet 2022-08-05 16:35:41 +01:00
netstorage all: fix spelling across the project 2022-08-30 11:16:26 +02:00
onedrive onedrive: disable change notify in China region since it is not supported 2022-09-16 16:57:29 +01:00
opendrive build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
oracleobjectstorage oracle-object-storage: doc fix 2022-10-28 20:32:17 +02:00
pcloud all: fix spelling across the project 2022-08-30 11:16:26 +02:00
premiumizeme build: reformat comments to pass go1.19 vet 2022-08-05 16:35:41 +01:00
putio build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
qingstor build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
s3 s3: add provider quirk --s3-might-gzip to fix corrupted on transfer: sizes differ 2022-11-04 16:53:32 +00:00
seafile onedrive: cleanup brand name 2022-08-30 10:23:29 +02:00
sftp sftp: add configuration options to set ssh Ciphers / MACs / KeyExchange 2022-11-03 17:11:28 +00:00
sharefile build: reformat comments to pass go1.19 vet 2022-08-05 16:35:41 +01:00
sia build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
smb smb: backend to support SMB - fixes #2042 2022-09-30 16:10:57 +01:00
storj build: reformat comments to pass go1.19 vet 2022-08-05 16:35:41 +01:00
sugarsync onedrive: cleanup brand name 2022-08-30 10:23:29 +02:00
swift swift: add --swift-no-large-objects to reduce HEAD requests 2022-10-13 11:58:19 +01:00
union union: propagate SlowHash feature 2022-10-10 07:58:01 +01:00
uptobox all: fix spelling across the project 2022-08-30 11:16:26 +02:00
webdav onedrive: cleanup brand name 2022-08-30 10:23:29 +02:00
yandex build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
zoho all: fix spelling across the project 2022-08-30 11:16:26 +02:00