From 78150e82a29fd1c912b81e960aea002626ba606f Mon Sep 17 00:00:00 2001 From: Nick Craig-Wood Date: Sun, 4 Aug 2019 11:43:10 +0100 Subject: [PATCH] docs: update bugs and limitations document --- docs/content/bugs.md | 49 +++++++++++++++++++++++++++----------------- 1 file changed, 30 insertions(+), 19 deletions(-) diff --git a/docs/content/bugs.md b/docs/content/bugs.md index 8929a69bc..13323e6be 100644 --- a/docs/content/bugs.md +++ b/docs/content/bugs.md @@ -1,31 +1,42 @@ --- title: "Bugs" description: "Rclone Bugs and Limitations" -date: "2014-06-16" +date: "2019-08-04" --- -Bugs and Limitations --------------------- +# Bugs and Limitations -### Empty directories are left behind / not created ## +## Limitations -With remotes that have a concept of directory, eg Local and Drive, -empty directories may be left behind, or not created when one was -expected. +### Directory timestamps aren't preserved -This is because rclone doesn't have a concept of a directory - it only -works on objects. Most of the object storage systems can't actually -store a directory so there is nowhere for rclone to store anything -about directories. +Rclone doesn't currently preserve the timestamps of directories. This +is because rclone only really considers objects when syncing. -You can work round this to some extent with the`purge` command which -will delete everything under the path, **inluding** empty directories. +### Rclone struggles with millions of files in a directory -This may be fixed at some point in -[Issue #100](https://github.com/rclone/rclone/issues/100) +Currently rclone loads each directory entirely into memory before +using it. Since each Rclone object takes 0.5k-1k of memory this can +take a very long time and use an extremely large amount of memory. -### Directory timestamps aren't preserved ## +Millions of files in a directory tend caused by software writing cloud +storage (eg S3 buckets). + +### Bucket based remotes and folders + +Bucket based remotes (eg S3/GCS/Swift/B2) do not have a concept of +directories. Rclone therefore cannot create directories in them which +means that empty directories on a bucket based remote will tend to +disappear. + +Some software creates empty keys ending in `/` as directory markers. +Rclone doesn't do this as it potentially creates more objects and +costs more. It may do in future (probably with a flag). + +## Bugs + +Bugs are stored in rclone's Github project: + +* [Reported bugs](https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+label%3Abug) +* [Known issues](https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Known+Problem%22) -For the same reason as the above, rclone doesn't have a concept of a -directory - it only works on objects, therefore it can't preserve the -timestamps of directories.