13 Commits

Author	SHA1	Message	Date
Nick Craig-Wood	93955b755f	operations: fix retries downloading too much data with certain backends ... Before this fix if more than one retry happened on a file that rclone had opened for read with a backend that uses fs.FixRangeOption then rclone would read too much data and the transfer would fail. Backends affected: - azureblob, azurefiles, b2, box, dropbox, fichier, filefabric - googlecloudstorage, hidrive, imagekit, jottacloud, koofr, netstorage - onedrive, opendrive, oracleobjectstorage, pikpak, premiumizeme - protondrive, qingstor, quatrix, s3, sharefile, sugarsync, swift - uptobox, webdav, zoho This was because rclone was emitting Range requests for the wrong data range on the second and subsequent retries. This was caused by fs.FixRangeOption modifying the options and the reopen code relying on them not being modified. This fix makes a copy of the fs.FixRangeOption in the reopen code to fix the problem. In future it might be best to change fs.FixRangeOption so it returns a new options slice. Fixes #7759	2024-04-13 19:25:15 +01:00
Nick Craig-Wood	a4fc5edc5e	operations: add more assertions to ReOpen tests to check seek positions	2024-04-13 19:25:15 +01:00
Nick Craig-Wood	c0fb9ebfce	operations: make Open() return an io.ReadSeekCloser #7350 ... As part of reducing memory usage in rclone, we need to have a raw handle to an object we can seek with.	2023-11-20 18:07:05 +00:00
albertony	5d6b8141ec	Replace deprecated ioutil ... As of Go 1.16, the same functionality is now provided by package io or package os, and those implementations should be preferred in new code.	2022-11-07 11:41:47 +00:00
Nick Craig-Wood	e43b5ce5e5	Remove github.com/pkg/errors and replace with std library version ... This is possible now that we no longer support go1.12 and brings rclone into line with standard practices in the Go world. This also removes errors.New and errors.Errorf from lib/errors and prefers the stdlib errors package over lib/errors.	2021-11-07 11:53:30 +00:00
Martin Michlmayr	4aee962233	doc: fix typos throughout docs and code	2020-05-20 15:54:51 +01:00
Tim Gallant	9bf3d3da4c	fs: add UploadHeaders, DownloadHeaders to Update/Put/Open options	2020-04-23 11:07:21 +01:00
Nick Craig-Wood	7f15cc9556	operations: make ReOpen and NewReOpen public for re-use elsewhere	2020-04-19 15:18:49 +01:00
Nick Craig-Wood	cd3c699f28	lib/readers: factor ErrorReader from multiple sources	2020-04-19 15:18:49 +01:00
Nick Craig-Wood	57d5de6fba	build: fix up package paths after repo move ... git grep -l github.com/ncw/rclone | xargs -d'\n' perl -i~ -lpe 's|github.com/ncw/rclone|github.com/rclone/rclone|g' goimports -w `find . -name \*.go`	2019-07-28 18:47:38 +01:00
Aleksandar Jankovic	f78cd1e043	Add context propagation to rclone ... - Change rclone/fs interfaces to accept context.Context - Update interface implementations to use context.Context - Change top level usage to propagate context to lover level functions Context propagation is needed for stopping transfers and passing other request-scoped values.	2019-06-19 11:59:46 +01:00
Nick Craig-Wood	0655738da6	operations: re-work reopen framework so it can take a RangeOption #2252 ... This is in preparation for multipart downloads.	2019-05-11 23:35:19 +01:00
Nick Craig-Wood	076d3da825	operations: resume downloads if the reader fails in copy - fixes #2108 ... This puts a shim on the reader opened by Copy so that if an error is returned, the reader is re-opened at the correct seek point. This should make downloading very large files more reliable.	2019-02-12 11:47:57 +00:00