This was removed as part of #1716 to fix rclone uploads taking double
the space.
7f744033d8 onedrive: Removed upload cutoff and always do session uploads
As far as I can see, two revisions are still being created for single
part uploads so the default for this flag is set to -1, off.
However it may be useful for experimentation.
See: #8545
Before this change, sometimes, perhaps on heavily loaded sharepoint
servers, uploads would sometimes fail with the error:
{"error":{"code":"itemNotFound","message":"The upload session was not found"}}
This retries the upload after a 5 second delay up to --low-level-retries times.
Fixes#8545
As part of changes to the Google Photos APIs the scopes rclone used
for accessing Google photos have been removed.
This commit replaces the scopes with updated ones.
These aren't as powerful as the old scopes - this means rclone will
only be able to download photos it uploaded from March 31, 2025.
To use these new scopes do `rclone reconnect yourgooglephotosremote:`
Fixes#8434
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
We lost a previous documentation fix (#7077) detailing how to restore
single objects from AWS S3 Glacier.
Also make clearer that rclone provides restore functionality natively.
Co-authored-by: danielkrajnik <dan94kra@gmail.com>
This retries propfind on 425 status
In ownCloud Infinite Scale, files might be in that state if
postprocessing is still ongoing. All metadata are available anyway
Allow item status 425 "too early" for items when changing metadata
Fixes the upload behavior with ownCloud Infinite Scale
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Klaas Freitag <kraft@freisturz.de>
This was formalized in
c69eb84573
But it appears that we forgot to update `http`, and the `FsRoot` test didn't
catch it because we don't currently have an http integration test.
This commit modernizes Go usage. This was done with:
go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...
Then files needed to be `go fmt`ed and a few comments needed to be
restored.
The modernizations include replacing
- if/else conditional assignment by a call to the built-in min or max functions added in go1.21
- sort.Slice(x, func(i, j int) bool) { return s[i] < s[j] } by a call to slices.Sort(s), added in go1.21
- interface{} by the 'any' type added in go1.18
- append([]T(nil), s...) by slices.Clone(s) or slices.Concat(s), added in go1.21
- loop around an m[k]=v map update by a call to one of the Collect, Copy, Clone, or Insert functions from the maps package, added in go1.21
- []byte(fmt.Sprintf...) by fmt.Appendf(nil, ...), added in go1.19
- append(s[:i], s[i+1]...) by slices.Delete(s, i, i+1), added in go1.21
- a 3-clause for i := 0; i < n; i++ {} loop by for i := range n {}, added in go1.22
Sometimes the Azure blob servers reply with 503 ServerBusy errors and
these should be retried.
Before this change, when testing to see if a single part server side
copy was done, ServerBusy errors were returned to the user rather than
being retried.
Wrapping the call in the pacer fixes the problem and ensures it is
retried properly using the --low-level-retries mechanism.
Found via `codespell -q 3 -S "./docs/static,./fs/rc/params_test.go" -L aadd,afile,alledges,bbefore,bu,buda,copys,couldn,crashers,crypted,ddelete,deriver,failre,goup,hashin,hel,inbraces,keep-alives,ket,medias,ment,mis,nd,nin,notin,ois,ot,parth,re-use,re-using,responser,rin,sav,splited,streamin,synching,te,twoo,ue,unknwon,wasn`
These files must be "exported" to be useful. The export process
is controlled by the --dropbox-export-formats flag and the ancilliary flags
--dropbox-skip-exports and --dropbox-show-all-exports modeled on the
Google drive equivalents
* Lower pacer minSleep to establish new connections faster
* Use Echo requests to check whether connections are working (required an upgrade of go-smb2)
* Only remount shares when needed
* Use context for connection establishment
* When returning a connection to the pool, only check the ones that encountered errors
* Close connections in parallel
When doing a multipart upload or copy, if a InvalidBlobOrBlock error
is received, it can mean that there are uncomitted blocks from a
previous failed attempt with a different length of ID.
This patch makes rclone attempt to clear the uncomitted blocks and
retry if it receives this error.
This implements multipart server side copy to improve copying from one
azure region to another by orders of magnitude (from 30s for a 100M
file to 10s for a 10G file with --azureblob-upload-concurrency 500).
- Add `--azureblob-copy-cutoff` to control the cutoff from single to multipart copy
- Add `--azureblob-copy-concurrency` to control the copy concurrency
- Add ServerSideAcrossConfigs flag as this now works properly
- Implement multipart copy using put block list API
- Shortcut multipart copy for same storage account
- Override with `--azureblob-use-copy-blob`
Fixes#8249
This speeds up server side copies for small files which need the check
the copy status by using an exponential ramp up of time to check the
copy status endpoint.
Before this change, if a multipart upload was aborted, then rclone
would leave uncommitted blocks lying around. Azure has a limit of
100,000 uncommitted blocks per storage account, so when you then try
to upload other stuff into that account, or simply the same file
again, you can run into this limit. This causes errors like the
following:
BlockCountExceedsLimit: The uncommitted block count cannot exceed the
maximum limit of 100,000 blocks.
This change removes the uncommitted blocks if a multipart upload is
aborted or fails.
If there was an existing destination file, it takes care not to
overwrite it by recomitting already comitted blocks.
This means that the scheme for allocating block IDs had to change to
make them different for each block and each upload.
Fixes#5583
Added the ability to include item's metadata on uploads via the
Internet Archive backend using the `--internetarchive-metadata="key=value"`
argument. This is hidden from the configurator as should only
really be used on the command line.
Before this change, metadata had to be manually added after uploads.
With this new feature, users can specify metadata directly during the
upload process.
