Create a full loop of documentation for rclone about, backends overview
and individual backend pages.
Discussion:
https://github.com/rclone/rclone/pull/4774 relates
Previously pull was requested, in part, under ref
https://github.com/rclone/rclone/pull/4801
Notes:
Introduce a tentative draft see-link format the end of sections to try
rather than lots of in-para links.
Update about.go incl link to list of backends not supporting about.
In list of backends not supporting about, include link to about command
reference.
I appreciate there may be decisions to make going forward about whether
command links should be code formatted, and using proper pretty url
links, but I have fudged that for now.
Update backend pages that do not support about with wording used
previously for ftp - it is in passive voice but I can live with it. (my
own wording and fault). The note is applied to a limitations section. If
one does not already exist it is created (even if there are other
limitations with their own sections)
This is done by making fs.Config private and attaching it to the
context instead.
The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
Before this change, small objects uploaded with SSE-AWS/SSE-C would
not have MD5 sums.
This change adds metadata for these objects in the same way that the
metadata is stored for multipart uploaded objects.
See: #1824#2827
If rclone is configured for server side encryption - either aws:kms or
sse-c (but not sse-s3) then don't treat the ETags returned on objects
as MD5 hashes.
This fixes being able to upload small files.
Fixes#1824
Before this change stable releases updated the current beta which mean
confusingly the current beta release would jump backwards from
1.54.0-beta to 1.53.3-beta say.
This commit stops any tagged build making a current beta release. They
will still make beta releases, they just won't update the
rclone*current*.zip and version.txt files.
This also means that a .0 release will not make a current beta like it
does at the moment.
This shouldn't be read as encouraging the use of math/rand instead of
crypto/rand in security sensitive contexts, rather as a safer default
if that does happen by accident.
For some reason the API started returning some integers as strings in
JSON. This is probably OK in Javascript but it upsets Go.
This is easily fixed with the `json:"name,size"` struct tag.
Before this change if a file was uploaded through a mount, then
deleted externally, trying to upload that file again could give EEXIST
"file already exists".
This was because the file already existing in the cache was confusing
rclone into thinking it already had the file.
The fix is to check that if rclone has a stale cache file then to
ignore it in this situation.
See: https://forum.rclone.org/t/rclone-cant-reuse-filenames/20400
This caches all the objects returned from the List call. This makes
opening them much quicker so speeds up prune and restores. It also
uses fewer transactions. It can be disabled with
`--cache-objects=false`.
This was discovered when using the B2 backend when the budget was
being blown on list object calls which can avoided with a bit of
caching.
For typical 1 million file backup for a latop or server this will only
use a small amount more memory.
Before this change a circular symlink would cause rclone to error out from the listings.
After this change rclone will skip a circular symlink and carry on the listing,
producing an error at the end.
Fixes#4743
Before this change the tests were run against the previous stable
rclone/rclone docker image.
This unfortunately masked errors in the integration test server.
This change uses the currently installed rclone to run "rclone serve
ftp" etc. This is installed out of the current code by the integration
test server so will make a better test.
Before this change, if a file was created on a remote but deleted
externally from that remote then there was potential for the delete to
never be noticed.
The sequence of events was:
- Create file on VFS - creates virtual directory entry
- File deleted externally to remote before the directory refreshed
- Now the file has a virtual add but is not in the listings so will never disappear
This patch fixes it by removing all virtual directory entries except
the following when the directory is re-read.
- On remotes which can't have empty directories: virtual directory
adds are not flushed. These will remain virtual as long as the
directory is empty.
- For virtual file add: files that are in the process of being
uploaded are not flushed
This patch also adds the distinction between virtually added files and
directories.
It also refactors the virtual directory logic to make it easier to follow.
Fixes#4446
This adds a context.Context parameter to NewFs and related calls.
This is necessary as part of reading config from the context -
backends need to be able to read the global config.
