rclone/backend/s3
Nick Craig-Wood 5b5fdc6bc5 s3: add provider quirk --s3-might-gzip to fix corrupted on transfer: sizes differ
Before this change, some files were giving this error when downloaded
from Cloudflare and other providers.

    ERROR corrupted on transfer: sizes differ NNN vs MMM

This is because these providers auto gzips the object when rclone
wasn't expecting it to. (AWS does not gzip objects without their being
uploaded gzipped).

This patch adds a quirk to for fix the problem and a flag to control
it. The quirk `might_gzip` is set to `true` for all providers except
AWS.

See: https://forum.rclone.org/t/s3-error-corrupted-on-transfer-sizes-differ-nnn-vs-mmm/33694/
Fixes: #6533
2022-11-04 16:53:32 +00:00
..
gen_setfrom.go s3: refactor to use generated code instead of reflection to copy structs 2022-08-05 16:45:23 +01:00
s3_internal_test.go all: fix spelling across the project 2022-08-30 11:16:26 +02:00
s3_test.go s3: fix SetModTime on GLACIER/ARCHIVE objects and implement set/get tier 2019-09-14 09:18:55 +01:00
s3.go s3: add provider quirk --s3-might-gzip to fix corrupted on transfer: sizes differ 2022-11-04 16:53:32 +00:00
setfrom.go s3: refactor to use generated code instead of reflection to copy structs 2022-08-05 16:45:23 +01:00
v2sign.go s3: fix copy multipart with v2 auth failing with 'SignatureDoesNotMatch' 2021-01-27 14:43:02 +00:00