extern/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2024-12-22 23:22:08 +01:00
Code Issues Packages Projects Releases Wiki Activity

s3: fix corrupted on transfer: sizes differ 0 vs xxxx with Ceph

Browse Source 
In this commit, released in 1.56.0 we started reading the size of the
object from the Content-Length header as returned by the GET request
to read the object.

4401d180aa s3: add --s3-no-head-object

However some object storage systems, notably Ceph, don't return a
Content-Length header.

The new code correctly calls the setMetaData function with a nil
pointer to the ContentLength.

However due to this commit from 2014, released in v1.18, the
setMetaData function was not ignoring the size as it should have done.

0da6f24221  s3: use official github.com/aws/aws-sdk-go including multipart upload #101

This commit correctly ignores the content length if not set.

Fixes #5732
This commit is contained in:
Nick Craig-Wood 2021-10-29 10:23:41 +01:00
parent e6e1c49b58
commit cf2c2792e6
1 changed files with 1 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
backend/s3/s3.go
View File

@ -2892,14 +2892,12 @@ func (o *Object) readMetaData(ctx context.Context) (err error) {
}
func (o *Object) setMetaData(etag *string, contentLength *int64, lastModified *time.Time, meta map[string]*string, mimeType *string, storageClass *string) {
var size int64
// Ignore missing Content-Length assuming it is 0
// Some versions of ceph do this due their apache proxies
if contentLength != nil {
size = *contentLength
o.bytes = *contentLength
}
o.setMD5FromEtag(aws.StringValue(etag))
o.bytes = size
o.meta = meta
if o.meta == nil {
o.meta = map[string]*string{}