Before this change, small objects uploaded with SSE-AWS/SSE-C would
not have MD5 sums.
This change adds metadata for these objects in the same way that the
metadata is stored for multipart uploaded objects.
See: #1824#2827
If rclone is configured for server side encryption - either aws:kms or
sse-c (but not sse-s3) then don't treat the ETags returned on objects
as MD5 hashes.
This fixes being able to upload small files.
Fixes#1824
Before this change stable releases updated the current beta which mean
confusingly the current beta release would jump backwards from
1.54.0-beta to 1.53.3-beta say.
This commit stops any tagged build making a current beta release. They
will still make beta releases, they just won't update the
rclone*current*.zip and version.txt files.
This also means that a .0 release will not make a current beta like it
does at the moment.
This shouldn't be read as encouraging the use of math/rand instead of
crypto/rand in security sensitive contexts, rather as a safer default
if that does happen by accident.
For some reason the API started returning some integers as strings in
JSON. This is probably OK in Javascript but it upsets Go.
This is easily fixed with the `json:"name,size"` struct tag.
Before this change if a file was uploaded through a mount, then
deleted externally, trying to upload that file again could give EEXIST
"file already exists".
This was because the file already existing in the cache was confusing
rclone into thinking it already had the file.
The fix is to check that if rclone has a stale cache file then to
ignore it in this situation.
See: https://forum.rclone.org/t/rclone-cant-reuse-filenames/20400
This caches all the objects returned from the List call. This makes
opening them much quicker so speeds up prune and restores. It also
uses fewer transactions. It can be disabled with
`--cache-objects=false`.
This was discovered when using the B2 backend when the budget was
being blown on list object calls which can avoided with a bit of
caching.
For typical 1 million file backup for a latop or server this will only
use a small amount more memory.
Before this change a circular symlink would cause rclone to error out from the listings.
After this change rclone will skip a circular symlink and carry on the listing,
producing an error at the end.
Fixes#4743
Before this change the tests were run against the previous stable
rclone/rclone docker image.
This unfortunately masked errors in the integration test server.
This change uses the currently installed rclone to run "rclone serve
ftp" etc. This is installed out of the current code by the integration
test server so will make a better test.
Before this change, if a file was created on a remote but deleted
externally from that remote then there was potential for the delete to
never be noticed.
The sequence of events was:
- Create file on VFS - creates virtual directory entry
- File deleted externally to remote before the directory refreshed
- Now the file has a virtual add but is not in the listings so will never disappear
This patch fixes it by removing all virtual directory entries except
the following when the directory is re-read.
- On remotes which can't have empty directories: virtual directory
adds are not flushed. These will remain virtual as long as the
directory is empty.
- For virtual file add: files that are in the process of being
uploaded are not flushed
This patch also adds the distinction between virtually added files and
directories.
It also refactors the virtual directory logic to make it easier to follow.
Fixes#4446
This adds a context.Context parameter to NewFs and related calls.
This is necessary as part of reading config from the context -
backends need to be able to read the global config.
Fix the copy and move operations that broke in 127f0fc when copying directly
to a remote without a specific destination.
Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
It seems that when doing chunked uploads to onedrive, if the chunks
take more than 3 minutes or so to upload then they may timeout with
error 504 Gateway Timeout.
This change produces an error (just once) suggesting lowering
`--onedrive-chunk-size` or decreasing `--transfers`.
This is easy to replicate with:
rclone copy -Pvv --bwlimit 0.05M 20M onedrive:20M
See: https://forum.rclone.org/t/default-onedrive-chunk-size-does-not-work/20010/
