mirror of
https://github.com/rclone/rclone.git
synced 2024-11-22 16:34:30 +01:00
drive: add docs for --fast-list and add to integration tests
This commit is contained in:
parent
dc5a734522
commit
f7c0b2407d
@ -312,6 +312,45 @@ d) Delete this remote
|
|||||||
y/e/d> y
|
y/e/d> y
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### --fast-list ###
|
||||||
|
|
||||||
|
This remote supports `--fast-list` which allows you to use fewer
|
||||||
|
transactions in exchange for more memory. See the [rclone
|
||||||
|
docs](/docs/#fast-list) for more details.
|
||||||
|
|
||||||
|
It does this by combining multiple `list` calls into a single API request.
|
||||||
|
|
||||||
|
This works by combining many `'%s' in parents` filters into one expression.
|
||||||
|
To list the contents of directories a, b and c, the the following requests will be send by the regular `List` function:
|
||||||
|
```
|
||||||
|
trashed=false and 'a' in parents
|
||||||
|
trashed=false and 'b' in parents
|
||||||
|
trashed=false and 'c' in parents
|
||||||
|
```
|
||||||
|
These can now be combined into a single request:
|
||||||
|
```
|
||||||
|
trashed=false and ('a' in parents or 'b' in parents or 'c' in parents)
|
||||||
|
```
|
||||||
|
|
||||||
|
The implementation of `ListR` will put up to 50 `parents` filters into one request.
|
||||||
|
It will use the `--checkers` value to specify the number of requests to run in parallel.
|
||||||
|
|
||||||
|
In tests, these batch requests were up to 20x faster than the regular method.
|
||||||
|
Running the following command against different sized folders gives:
|
||||||
|
```
|
||||||
|
rclone lsjson -vv -R --checkers=6 gdrive:folder
|
||||||
|
```
|
||||||
|
|
||||||
|
small folder (220 directories, 700 files):
|
||||||
|
|
||||||
|
- without `--fast-list`: 38s
|
||||||
|
- with `--fast-list`: 10s
|
||||||
|
|
||||||
|
large folder (10600 directories, 39000 files):
|
||||||
|
|
||||||
|
- without `--fast-list`: 22:05 min
|
||||||
|
- with `--fast-list`: 58s
|
||||||
|
|
||||||
### Modified time ###
|
### Modified time ###
|
||||||
|
|
||||||
Google drive stores modification times accurate to 1 ms.
|
Google drive stores modification times accurate to 1 ms.
|
||||||
|
@ -134,7 +134,7 @@ operations more efficient.
|
|||||||
| Dropbox | Yes | Yes | Yes | Yes | No [#575](https://github.com/ncw/rclone/issues/575) | No | Yes | Yes | Yes |
|
| Dropbox | Yes | Yes | Yes | Yes | No [#575](https://github.com/ncw/rclone/issues/575) | No | Yes | Yes | Yes |
|
||||||
| FTP | No | No | Yes | Yes | No | No | Yes | No [#2178](https://github.com/ncw/rclone/issues/2178) | No |
|
| FTP | No | No | Yes | Yes | No | No | Yes | No [#2178](https://github.com/ncw/rclone/issues/2178) | No |
|
||||||
| Google Cloud Storage | Yes | Yes | No | No | No | Yes | Yes | No [#2178](https://github.com/ncw/rclone/issues/2178) | No |
|
| Google Cloud Storage | Yes | Yes | No | No | No | Yes | Yes | No [#2178](https://github.com/ncw/rclone/issues/2178) | No |
|
||||||
| Google Drive | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes |
|
| Google Drive | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
|
||||||
| HTTP | No | No | No | No | No | No | No | No [#2178](https://github.com/ncw/rclone/issues/2178) | No |
|
| HTTP | No | No | No | No | No | No | No | No [#2178](https://github.com/ncw/rclone/issues/2178) | No |
|
||||||
| Hubic | Yes † | Yes | No | No | No | Yes | Yes | No [#2178](https://github.com/ncw/rclone/issues/2178) | Yes |
|
| Hubic | Yes † | Yes | No | No | No | Yes | Yes | No [#2178](https://github.com/ncw/rclone/issues/2178) | Yes |
|
||||||
| Mega | Yes | No | Yes | Yes | No | No | No | No [#2178](https://github.com/ncw/rclone/issues/2178) | Yes |
|
| Mega | Yes | No | Yes | Yes | No | No | No | No [#2178](https://github.com/ncw/rclone/issues/2178) | Yes |
|
||||||
|
@ -45,7 +45,7 @@ var (
|
|||||||
{
|
{
|
||||||
Name: "TestCryptDrive:",
|
Name: "TestCryptDrive:",
|
||||||
SubDir: false,
|
SubDir: false,
|
||||||
FastList: false,
|
FastList: true,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
Name: "TestCryptSwift:",
|
Name: "TestCryptSwift:",
|
||||||
@ -55,7 +55,7 @@ var (
|
|||||||
{
|
{
|
||||||
Name: "TestDrive:",
|
Name: "TestDrive:",
|
||||||
SubDir: false,
|
SubDir: false,
|
||||||
FastList: false,
|
FastList: true,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
Name: "TestDropbox:",
|
Name: "TestDropbox:",
|
||||||
|
Loading…
Reference in New Issue
Block a user