diff --git a/MANUAL.html b/MANUAL.html
index 4b87ec4b1..b16cb8800 100644
--- a/MANUAL.html
+++ b/MANUAL.html
@@ -19,7 +19,7 @@
Oct 21, 2022 Nov 17, 2022rclone(1) User Manual
-
Fetch the correct binary for your processor type by clicking on these links. If not sure, use the first link.
Open this file in the Explorer and extract rclone.exe
. Rclone is a portable executable so you can place it wherever is convenient.
Open a CMD window (or powershell) and run the binary. Note that rclone does not launch a GUI by default, it runs in the CMD Window.
@@ -8081,13 +8081,13 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB totalThese flags are available for every command. They control the backends and may be set in the config file.
@@ -8848,6 +8848,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total --s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000) --s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s) --s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool + --s3-might-gzip Tristate Set this if the backend might gzip objects (default unset) --s3-no-check-bucket If set, don't attempt to check the bucket exists or create it --s3-no-head If set, don't HEAD uploaded objects to check integrity --s3-no-head-object If set, do not do HEAD before GET when getting objects @@ -12315,15 +12316,39 @@ $ rclone -q --s3-versions ls s3:cleanup-testSet this if the backend might gzip objects.
+Normally providers will not alter objects when they are downloaded. If an object was not uploaded with Content-Encoding: gzip
then it won't be set on download.
However some providers may gzip objects even if they weren't uploaded with Content-Encoding: gzip
(eg Cloudflare).
A symptom of this would be receiving errors like
+ERROR corrupted on transfer: sizes differ NNN vs MMM
+If you set this flag and rclone downloads an object with Content-Encoding: gzip set and chunked transfer encoding, then rclone will decompress the object on the fly.
+If this is set to unset (the default) then rclone will choose according to the provider setting what to apply, but you can override rclone's choice here.
+Properties:
+Suppress setting and reading of system metadata
Properties:
@@ -22458,7 +22506,17 @@ y/e/d> yHere is an example of making a mailru configuration. First create a Mail.ru Cloud account and choose a tariff, then run
+Here is an example of making a mailru configuration.
+First create a Mail.ru Cloud account and choose a tariff.
+You will need to log in and create an app password for rclone. Rclone will not work with your normal username and password - it will give an error like oauth2: server response missing access_token
.
Now run
rclone config
This will guide you through an interactive setup process:
No remotes found, make a new one?
@@ -22480,6 +22538,10 @@ User name (usually email)
Enter a string value. Press Enter for the default ("").
user> username@mail.ru
Password
+
+This must be an app password - rclone will not work with your normal
+password. See the Configuration section in the docs for how to make an
+app password.
y) Yes type in my own password
g) Generate random password
y/g> y
@@ -22598,6 +22660,7 @@ y/e/d> y
Password.
+This must be an app password - rclone will not work with your normal password. See the Configuration section in the docs for how to make an app password.
NB Input to this must be obscured - see rclone obscure.
Properties:
remove
in rclone_serve_restic command (Joda Stößer)--s3-might-gzip
to fix corrupted on transfer: sizes differ (Nick Craig-Wood)Failed to sync: context canceled
at the end of syncs (Nick Craig-Wood)It would be possible to make a sync system which stored binary diffs instead of whole objects like rclone does, but that would break the 1:1 mapping of files on your hard disk to objects in the remote cloud storage system.
All the cloud storage systems support partial downloads of content, so it would be possible to make partial downloads work. However to make this work efficiently this would require storing a significant amount of metadata, which breaks the desired 1:1 mapping of files to objects.
No, not at present. rclone only does uni-directional sync from A -> B. It may do in the future though since it has all the primitives - it just requires writing the algorithm to do it.
+Yes, since rclone v1.58.0, bidirectional cloud sync is available.
Yes. rclone will follow the standard environment variables for proxies, similar to cURL and other programs.
In general the variables are called http_proxy
(for services reached over http
) and https_proxy
(for services reached over https
). Most public services will be using https
, but you may wish to set both.
Note: If the proxy server requires a username and password, then use
+export http_proxy=http://username:password@proxyserver:12345
+export https_proxy=$http_proxy
+export HTTP_PROXY=$http_proxy
+export HTTPS_PROXY=$http_proxy
The NO_PROXY
allows you to disable the proxy for specific hosts. Hosts must be comma separated, and can contain domains or parts. For instance "foo.com" also matches "bar.foo.com".
e.g.
export no_proxy=localhost,127.0.0.0/8,my.host.name
diff --git a/MANUAL.md b/MANUAL.md
index c7030525b..741d83e0d 100644
--- a/MANUAL.md
+++ b/MANUAL.md
@@ -1,6 +1,6 @@
% rclone(1) User Manual
% Nick Craig-Wood
-% Oct 21, 2022
+% Nov 17, 2022
# Rclone syncs your files to cloud storage
@@ -315,9 +315,9 @@ The simplest fix is to run
Fetch the correct binary for your processor type by clicking on these
links. If not sure, use the first link.
-- [Intel/AMD - 64 Bit](https://downloads.rclone.org/rclone-current-linux-amd64.zip)
-- [Intel/AMD - 32 Bit](https://downloads.rclone.org/rclone-current-linux-386.zip)
-- [ARM - 64 Bit](https://downloads.rclone.org/rclone-current-linux-arm64.zip)
+- [Intel/AMD - 64 Bit](https://downloads.rclone.org/rclone-current-windows-amd64.zip)
+- [Intel/AMD - 32 Bit](https://downloads.rclone.org/rclone-current-windows-386.zip)
+- [ARM - 64 Bit](https://downloads.rclone.org/rclone-current-windows-arm64.zip)
Open this file in the Explorer and extract `rclone.exe`. Rclone is a
portable executable so you can place it wherever is convenient.
@@ -14241,7 +14241,7 @@ upon backend-specific capabilities.
| Microsoft OneDrive | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | Yes |
| OpenDrive | Yes | Yes | Yes | Yes | No | No | No | No | No | Yes |
| OpenStack Swift | Yes † | Yes | No | No | No | Yes | Yes | No | Yes | No |
-| Oracle Object Storage | Yes | Yes | No | No | Yes | Yes | No | No | No | No |
+| Oracle Object Storage | No | Yes | No | No | Yes | Yes | Yes | No | No | No |
| pCloud | Yes | Yes | Yes | Yes | Yes | No | No | Yes | Yes | Yes |
| premiumize.me | Yes | No | Yes | Yes | No | No | No | Yes | Yes | Yes |
| put.io | Yes | No | Yes | Yes | Yes | No | Yes | No | Yes | Yes |
@@ -14501,7 +14501,7 @@ These flags are available for every command.
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
- --user-agent string Set the user-agent to a specified string (default "rclone/v1.60.0")
+ --user-agent string Set the user-agent to a specified string (default "rclone/v1.60.1")
-v, --verbose count Print lots more stuff (repeat for more)
```
@@ -14884,6 +14884,7 @@ and may be set in the config file.
--s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
--s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
--s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
+ --s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
--s3-no-check-bucket If set, don't attempt to check the bucket exists or create it
--s3-no-head If set, don't HEAD uploaded objects to check integrity
--s3-no-head-object If set, do not do HEAD before GET when getting objects
@@ -18940,15 +18941,31 @@ Properties:
- "s3.ap-southeast-1.lyvecloud.seagate.com"
- Seagate Lyve Cloud AP Southeast 1 (Singapore)
- "s3.wasabisys.com"
- - Wasabi US East endpoint
+ - Wasabi US East 1 (N. Virginia)
+ - "s3.us-east-2.wasabisys.com"
+ - Wasabi US East 2 (N. Virginia)
+ - "s3.us-central-1.wasabisys.com"
+ - Wasabi US Central 1 (Texas)
- "s3.us-west-1.wasabisys.com"
- - Wasabi US West endpoint
+ - Wasabi US West 1 (Oregon)
+ - "s3.ca-central-1.wasabisys.com"
+ - Wasabi CA Central 1 (Toronto)
- "s3.eu-central-1.wasabisys.com"
- - Wasabi EU Central endpoint
+ - Wasabi EU Central 1 (Amsterdam)
+ - "s3.eu-central-2.wasabisys.com"
+ - Wasabi EU Central 2 (Frankfurt)
+ - "s3.eu-west-1.wasabisys.com"
+ - Wasabi EU West 1 (London)
+ - "s3.eu-west-2.wasabisys.com"
+ - Wasabi EU West 2 (Paris)
- "s3.ap-northeast-1.wasabisys.com"
- Wasabi AP Northeast 1 (Tokyo) endpoint
- "s3.ap-northeast-2.wasabisys.com"
- Wasabi AP Northeast 2 (Osaka) endpoint
+ - "s3.ap-southeast-1.wasabisys.com"
+ - Wasabi AP Southeast 1 (Singapore)
+ - "s3.ap-southeast-2.wasabisys.com"
+ - Wasabi AP Southeast 2 (Sydney)
- "s3.ir-thr-at1.arvanstorage.com"
- ArvanCloud Tehran Iran (Asiatech) endpoint
@@ -20167,6 +20184,37 @@ Properties:
- Type: bool
- Default: false
+#### --s3-might-gzip
+
+Set this if the backend might gzip objects.
+
+Normally providers will not alter objects when they are downloaded. If
+an object was not uploaded with `Content-Encoding: gzip` then it won't
+be set on download.
+
+However some providers may gzip objects even if they weren't uploaded
+with `Content-Encoding: gzip` (eg Cloudflare).
+
+A symptom of this would be receiving errors like
+
+ ERROR corrupted on transfer: sizes differ NNN vs MMM
+
+If you set this flag and rclone downloads an object with
+Content-Encoding: gzip set and chunked transfer encoding, then rclone
+will decompress the object on the fly.
+
+If this is set to unset (the default) then rclone will choose
+according to the provider setting what to apply, but you can override
+rclone's choice here.
+
+
+Properties:
+
+- Config: might_gzip
+- Env Var: RCLONE_S3_MIGHT_GZIP
+- Type: Tristate
+- Default: unset
+
#### --s3-no-system-metadata
Suppress setting and reading of system metadata
@@ -32271,8 +32319,21 @@ Currently it is recommended to disable 2FA on Mail.ru accounts intended for rclo
## Configuration
-Here is an example of making a mailru configuration. First create a Mail.ru Cloud
-account and choose a tariff, then run
+Here is an example of making a mailru configuration.
+
+First create a Mail.ru Cloud account and choose a tariff.
+
+You will need to log in and create an app password for rclone. Rclone
+**will not work** with your normal username and password - it will
+give an error like `oauth2: server response missing access_token`.
+
+- Click on your user icon in the top right
+- Go to Security / "Пароль и безопасность"
+- Click password for apps / "Пароли для внешних приложений"
+- Add the password - give it a name - eg "rclone"
+- Copy the password and use this password below - your normal login password won't work.
+
+Now run
rclone config
@@ -32298,6 +32359,10 @@ User name (usually email)
Enter a string value. Press Enter for the default ("").
user> username@mail.ru
Password
+
+This must be an app password - rclone will not work with your normal
+password. See the Configuration section in the docs for how to make an
+app password.
y) Yes type in my own password
g) Generate random password
y/g> y
@@ -32420,6 +32485,11 @@ Properties:
Password.
+This must be an app password - rclone will not work with your normal
+password. See the Configuration section in the docs for how to make an
+app password.
+
+
**NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/).
Properties:
@@ -40954,6 +41024,40 @@ Options:
# Changelog
+## v1.60.1 - 2022-11-17
+
+[See commits](https://github.com/rclone/rclone/compare/v1.60.0...v1.60.1)
+
+* Bug Fixes
+ * lib/cache: Fix alias backend shutting down too soon (Nick Craig-Wood)
+ * wasm: Fix walltime link error by adding up-to-date wasm_exec.js (João Henrique Franco)
+ * docs
+ * Update faq.md with bisync (Samuel Johnson)
+ * Corrected download links in windows install docs (coultonluke)
+ * Add direct download link for windows arm64 (albertony)
+ * Remove link to rclone slack as it is no longer supported (Nick Craig-Wood)
+ * Faq: how to use a proxy server that requires a username and password (asdffdsazqqq)
+ * Oracle-object-storage: doc fix (Manoj Ghosh)
+ * Fix typo `remove` in rclone_serve_restic command (Joda Stößer)
+ * Fix character that was incorrectly interpreted as markdown (Clément Notin)
+* VFS
+ * Fix deadlock caused by cache cleaner and upload finishing (Nick Craig-Wood)
+* Local
+ * Clean absolute paths (albertony)
+ * Fix -L/--copy-links with filters missing directories (Nick Craig-Wood)
+* Mailru
+ * Note that an app password is now needed (Nick Craig-Wood)
+ * Allow timestamps to be before the epoch 1970-01-01 (Nick Craig-Wood)
+* S3
+ * Add provider quirk `--s3-might-gzip` to fix corrupted on transfer: sizes differ (Nick Craig-Wood)
+ * Allow Storj to server side copy since it seems to work now (Nick Craig-Wood)
+ * Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)
+ * Add additional Wasabi locations (techknowlogick)
+* Smb
+ * Fix `Failed to sync: context canceled` at the end of syncs (Nick Craig-Wood)
+* WebDAV
+ * Fix Move/Copy/DirMove when using -server-side-across-configs (Nick Craig-Wood)
+
## v1.60.0 - 2022-10-21
[See commits](https://github.com/rclone/rclone/compare/v1.59.0...v1.60.0)
@@ -45237,9 +45341,8 @@ of metadata, which breaks the desired 1:1 mapping of files to objects.
### Can rclone do bi-directional sync? ###
-No, not at present. rclone only does uni-directional sync from A ->
-B. It may do in the future though since it has all the primitives - it
-just requires writing the algorithm to do it.
+Yes, since rclone v1.58.0, [bidirectional cloud sync](https://rclone.org/bisync/) is
+available.
### Can I use rclone with an HTTP proxy? ###
@@ -45264,6 +45367,14 @@ possibilities. So, on Linux, you may end up with code similar to
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy
+
+Note: If the proxy server requires a username and password, then use
+
+ export http_proxy=http://username:password@proxyserver:12345
+ export https_proxy=$http_proxy
+ export HTTP_PROXY=$http_proxy
+ export HTTPS_PROXY=$http_proxy
+
The `NO_PROXY` allows you to disable the proxy for specific hosts.
Hosts must be comma separated, and can contain domains or parts.
For instance "foo.com" also matches "bar.foo.com".
diff --git a/MANUAL.txt b/MANUAL.txt
index 6cdd6ee0a..2ed638a05 100644
--- a/MANUAL.txt
+++ b/MANUAL.txt
@@ -1,6 +1,6 @@
rclone(1) User Manual
Nick Craig-Wood
-Oct 21, 2022
+Nov 17, 2022
Rclone syncs your files to cloud storage
@@ -13781,7 +13781,7 @@ upon backend-specific capabilities.
Microsoft OneDrive Yes Yes Yes Yes Yes No No Yes Yes Yes
OpenDrive Yes Yes Yes Yes No No No No No Yes
OpenStack Swift Yes † Yes No No No Yes Yes No Yes No
- Oracle Object Storage Yes Yes No No Yes Yes No No No No
+ Oracle Object Storage No Yes No No Yes Yes Yes No No No
pCloud Yes Yes Yes Yes Yes No No Yes Yes Yes
premiumize.me Yes No Yes Yes No No No Yes Yes Yes
put.io Yes No Yes Yes Yes No Yes No Yes Yes
@@ -14038,7 +14038,7 @@ These flags are available for every command.
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
- --user-agent string Set the user-agent to a specified string (default "rclone/v1.60.0")
+ --user-agent string Set the user-agent to a specified string (default "rclone/v1.60.1")
-v, --verbose count Print lots more stuff (repeat for more)
Backend Flags
@@ -14419,6 +14419,7 @@ and may be set in the config file.
--s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
--s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
--s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
+ --s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
--s3-no-check-bucket If set, don't attempt to check the bucket exists or create it
--s3-no-head If set, don't HEAD uploaded objects to check integrity
--s3-no-head-object If set, do not do HEAD before GET when getting objects
@@ -18425,15 +18426,31 @@ Properties:
- "s3.ap-southeast-1.lyvecloud.seagate.com"
- Seagate Lyve Cloud AP Southeast 1 (Singapore)
- "s3.wasabisys.com"
- - Wasabi US East endpoint
+ - Wasabi US East 1 (N. Virginia)
+ - "s3.us-east-2.wasabisys.com"
+ - Wasabi US East 2 (N. Virginia)
+ - "s3.us-central-1.wasabisys.com"
+ - Wasabi US Central 1 (Texas)
- "s3.us-west-1.wasabisys.com"
- - Wasabi US West endpoint
+ - Wasabi US West 1 (Oregon)
+ - "s3.ca-central-1.wasabisys.com"
+ - Wasabi CA Central 1 (Toronto)
- "s3.eu-central-1.wasabisys.com"
- - Wasabi EU Central endpoint
+ - Wasabi EU Central 1 (Amsterdam)
+ - "s3.eu-central-2.wasabisys.com"
+ - Wasabi EU Central 2 (Frankfurt)
+ - "s3.eu-west-1.wasabisys.com"
+ - Wasabi EU West 1 (London)
+ - "s3.eu-west-2.wasabisys.com"
+ - Wasabi EU West 2 (Paris)
- "s3.ap-northeast-1.wasabisys.com"
- Wasabi AP Northeast 1 (Tokyo) endpoint
- "s3.ap-northeast-2.wasabisys.com"
- Wasabi AP Northeast 2 (Osaka) endpoint
+ - "s3.ap-southeast-1.wasabisys.com"
+ - Wasabi AP Southeast 1 (Singapore)
+ - "s3.ap-southeast-2.wasabisys.com"
+ - Wasabi AP Southeast 2 (Sydney)
- "s3.ir-thr-at1.arvanstorage.com"
- ArvanCloud Tehran Iran (Asiatech) endpoint
@@ -19665,6 +19682,36 @@ Properties:
- Type: bool
- Default: false
+--s3-might-gzip
+
+Set this if the backend might gzip objects.
+
+Normally providers will not alter objects when they are downloaded. If
+an object was not uploaded with Content-Encoding: gzip then it won't be
+set on download.
+
+However some providers may gzip objects even if they weren't uploaded
+with Content-Encoding: gzip (eg Cloudflare).
+
+A symptom of this would be receiving errors like
+
+ ERROR corrupted on transfer: sizes differ NNN vs MMM
+
+If you set this flag and rclone downloads an object with
+Content-Encoding: gzip set and chunked transfer encoding, then rclone
+will decompress the object on the fly.
+
+If this is set to unset (the default) then rclone will choose according
+to the provider setting what to apply, but you can override rclone's
+choice here.
+
+Properties:
+
+- Config: might_gzip
+- Env Var: RCLONE_S3_MIGHT_GZIP
+- Type: Tristate
+- Default: unset
+
--s3-no-system-metadata
Suppress setting and reading of system metadata
@@ -31679,8 +31726,22 @@ Features highlights
Configuration
-Here is an example of making a mailru configuration. First create a
-Mail.ru Cloud account and choose a tariff, then run
+Here is an example of making a mailru configuration.
+
+First create a Mail.ru Cloud account and choose a tariff.
+
+You will need to log in and create an app password for rclone. Rclone
+will not work with your normal username and password - it will give an
+error like oauth2: server response missing access_token.
+
+- Click on your user icon in the top right
+- Go to Security / "Пароль и безопасность"
+- Click password for apps / "Пароли для внешних приложений"
+- Add the password - give it a name - eg "rclone"
+- Copy the password and use this password below - your normal login
+ password won't work.
+
+Now run
rclone config
@@ -31705,6 +31766,10 @@ This will guide you through an interactive setup process:
Enter a string value. Press Enter for the default ("").
user> username@mail.ru
Password
+
+ This must be an app password - rclone will not work with your normal
+ password. See the Configuration section in the docs for how to make an
+ app password.
y) Yes type in my own password
g) Generate random password
y/g> y
@@ -31826,6 +31891,10 @@ Properties:
Password.
+This must be an app password - rclone will not work with your normal
+password. See the Configuration section in the docs for how to make an
+app password.
+
NB Input to this must be obscured - see rclone obscure.
Properties:
@@ -40428,6 +40497,53 @@ Options:
Changelog
+v1.60.1 - 2022-11-17
+
+See commits
+
+- Bug Fixes
+ - lib/cache: Fix alias backend shutting down too soon (Nick
+ Craig-Wood)
+ - wasm: Fix walltime link error by adding up-to-date wasm_exec.js
+ (João Henrique Franco)
+ - docs
+ - Update faq.md with bisync (Samuel Johnson)
+ - Corrected download links in windows install docs
+ (coultonluke)
+ - Add direct download link for windows arm64 (albertony)
+ - Remove link to rclone slack as it is no longer supported
+ (Nick Craig-Wood)
+ - Faq: how to use a proxy server that requires a username and
+ password (asdffdsazqqq)
+ - Oracle-object-storage: doc fix (Manoj Ghosh)
+ - Fix typo remove in rclone_serve_restic command (Joda Stößer)
+ - Fix character that was incorrectly interpreted as markdown
+ (Clément Notin)
+- VFS
+ - Fix deadlock caused by cache cleaner and upload finishing (Nick
+ Craig-Wood)
+- Local
+ - Clean absolute paths (albertony)
+ - Fix -L/--copy-links with filters missing directories (Nick
+ Craig-Wood)
+- Mailru
+ - Note that an app password is now needed (Nick Craig-Wood)
+ - Allow timestamps to be before the epoch 1970-01-01 (Nick
+ Craig-Wood)
+- S3
+ - Add provider quirk --s3-might-gzip to fix corrupted on transfer:
+ sizes differ (Nick Craig-Wood)
+ - Allow Storj to server side copy since it seems to work now (Nick
+ Craig-Wood)
+ - Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)
+ - Add additional Wasabi locations (techknowlogick)
+- Smb
+ - Fix Failed to sync: context canceled at the end of syncs (Nick
+ Craig-Wood)
+- WebDAV
+ - Fix Move/Copy/DirMove when using -server-side-across-configs
+ (Nick Craig-Wood)
+
v1.60.0 - 2022-10-21
See commits
@@ -45966,9 +46082,7 @@ metadata, which breaks the desired 1:1 mapping of files to objects.
Can rclone do bi-directional sync?
-No, not at present. rclone only does uni-directional sync from A -> B.
-It may do in the future though since it has all the primitives - it just
-requires writing the algorithm to do it.
+Yes, since rclone v1.58.0, bidirectional cloud sync is available.
Can I use rclone with an HTTP proxy?
@@ -45993,6 +46107,13 @@ set all possibilities. So, on Linux, you may end up with code similar to
export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy
+Note: If the proxy server requires a username and password, then use
+
+ export http_proxy=http://username:password@proxyserver:12345
+ export https_proxy=$http_proxy
+ export HTTP_PROXY=$http_proxy
+ export HTTPS_PROXY=$http_proxy
+
The NO_PROXY allows you to disable the proxy for specific hosts. Hosts
must be comma separated, and can contain domains or parts. For instance
"foo.com" also matches "bar.foo.com".
diff --git a/docs/content/changelog.md b/docs/content/changelog.md
index f68c8f5a0..2c844f256 100644
--- a/docs/content/changelog.md
+++ b/docs/content/changelog.md
@@ -5,6 +5,40 @@ description: "Rclone Changelog"
# Changelog
+## v1.60.1 - 2022-11-17
+
+[See commits](https://github.com/rclone/rclone/compare/v1.60.0...v1.60.1)
+
+* Bug Fixes
+ * lib/cache: Fix alias backend shutting down too soon (Nick Craig-Wood)
+ * wasm: Fix walltime link error by adding up-to-date wasm_exec.js (João Henrique Franco)
+ * docs
+ * Update faq.md with bisync (Samuel Johnson)
+ * Corrected download links in windows install docs (coultonluke)
+ * Add direct download link for windows arm64 (albertony)
+ * Remove link to rclone slack as it is no longer supported (Nick Craig-Wood)
+ * Faq: how to use a proxy server that requires a username and password (asdffdsazqqq)
+ * Oracle-object-storage: doc fix (Manoj Ghosh)
+ * Fix typo `remove` in rclone_serve_restic command (Joda Stößer)
+ * Fix character that was incorrectly interpreted as markdown (Clément Notin)
+* VFS
+ * Fix deadlock caused by cache cleaner and upload finishing (Nick Craig-Wood)
+* Local
+ * Clean absolute paths (albertony)
+ * Fix -L/--copy-links with filters missing directories (Nick Craig-Wood)
+* Mailru
+ * Note that an app password is now needed (Nick Craig-Wood)
+ * Allow timestamps to be before the epoch 1970-01-01 (Nick Craig-Wood)
+* S3
+ * Add provider quirk `--s3-might-gzip` to fix corrupted on transfer: sizes differ (Nick Craig-Wood)
+ * Allow Storj to server side copy since it seems to work now (Nick Craig-Wood)
+ * Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)
+ * Add additional Wasabi locations (techknowlogick)
+* Smb
+ * Fix `Failed to sync: context canceled` at the end of syncs (Nick Craig-Wood)
+* WebDAV
+ * Fix Move/Copy/DirMove when using -server-side-across-configs (Nick Craig-Wood)
+
## v1.60.0 - 2022-10-21
[See commits](https://github.com/rclone/rclone/compare/v1.59.0...v1.60.0)
diff --git a/docs/content/commands/rclone_serve_restic.md b/docs/content/commands/rclone_serve_restic.md
index 40a4ed6d8..fff9d2f20 100644
--- a/docs/content/commands/rclone_serve_restic.md
+++ b/docs/content/commands/rclone_serve_restic.md
@@ -11,7 +11,7 @@ Serve the remote for restic's REST API.
## Synopsis
-Run a basic web server to serve a remote over restic's REST backend
+Run a basic web server to serve a remove over restic's REST backend
API over HTTP. This allows restic to use rclone as a data storage
mechanism for cloud providers that restic does not support directly.
diff --git a/docs/content/flags.md b/docs/content/flags.md
index 1cdce0e1d..bfe400969 100644
--- a/docs/content/flags.md
+++ b/docs/content/flags.md
@@ -162,7 +162,7 @@ These flags are available for every command.
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
- --user-agent string Set the user-agent to a specified string (default "rclone/v1.60.0")
+ --user-agent string Set the user-agent to a specified string (default "rclone/v1.60.1")
-v, --verbose count Print lots more stuff (repeat for more)
```
@@ -545,6 +545,7 @@ and may be set in the config file.
--s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
--s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
--s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
+ --s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
--s3-no-check-bucket If set, don't attempt to check the bucket exists or create it
--s3-no-head If set, don't HEAD uploaded objects to check integrity
--s3-no-head-object If set, do not do HEAD before GET when getting objects
diff --git a/docs/content/local.md b/docs/content/local.md
index 8b7920c91..f123aa74b 100644
--- a/docs/content/local.md
+++ b/docs/content/local.md
@@ -428,8 +428,8 @@ Properties:
Don't check to see if the files change during upload.
Normally rclone checks the size and modification time of files as they
-are being uploaded and aborts with a message which starts "can't copy -
-source file is being updated" if the file changes during upload.
+are being uploaded and aborts with a message which starts "can't copy
+- source file is being updated" if the file changes during upload.
However on some file systems this modification time check may fail (e.g.
[Glusterfs #2206](https://github.com/rclone/rclone/issues/2206)) so this
diff --git a/docs/content/mailru.md b/docs/content/mailru.md
index 5814283c8..57b7fd21c 100644
--- a/docs/content/mailru.md
+++ b/docs/content/mailru.md
@@ -190,6 +190,11 @@ Properties:
Password.
+This must be an app password - rclone will not work with your normal
+password. See the Configuration section in the docs for how to make an
+app password.
+
+
**NB** Input to this must be obscured - see [rclone obscure](/commands/rclone_obscure/).
Properties:
diff --git a/docs/content/s3.md b/docs/content/s3.md
index 4ffa23922..c9fbfaada 100644
--- a/docs/content/s3.md
+++ b/docs/content/s3.md
@@ -1631,15 +1631,31 @@ Properties:
- "s3.ap-southeast-1.lyvecloud.seagate.com"
- Seagate Lyve Cloud AP Southeast 1 (Singapore)
- "s3.wasabisys.com"
- - Wasabi US East endpoint
+ - Wasabi US East 1 (N. Virginia)
+ - "s3.us-east-2.wasabisys.com"
+ - Wasabi US East 2 (N. Virginia)
+ - "s3.us-central-1.wasabisys.com"
+ - Wasabi US Central 1 (Texas)
- "s3.us-west-1.wasabisys.com"
- - Wasabi US West endpoint
+ - Wasabi US West 1 (Oregon)
+ - "s3.ca-central-1.wasabisys.com"
+ - Wasabi CA Central 1 (Toronto)
- "s3.eu-central-1.wasabisys.com"
- - Wasabi EU Central endpoint
+ - Wasabi EU Central 1 (Amsterdam)
+ - "s3.eu-central-2.wasabisys.com"
+ - Wasabi EU Central 2 (Frankfurt)
+ - "s3.eu-west-1.wasabisys.com"
+ - Wasabi EU West 1 (London)
+ - "s3.eu-west-2.wasabisys.com"
+ - Wasabi EU West 2 (Paris)
- "s3.ap-northeast-1.wasabisys.com"
- Wasabi AP Northeast 1 (Tokyo) endpoint
- "s3.ap-northeast-2.wasabisys.com"
- Wasabi AP Northeast 2 (Osaka) endpoint
+ - "s3.ap-southeast-1.wasabisys.com"
+ - Wasabi AP Southeast 1 (Singapore)
+ - "s3.ap-southeast-2.wasabisys.com"
+ - Wasabi AP Southeast 2 (Sydney)
- "s3.ir-thr-at1.arvanstorage.com"
- ArvanCloud Tehran Iran (Asiatech) endpoint
@@ -2858,6 +2874,37 @@ Properties:
- Type: bool
- Default: false
+#### --s3-might-gzip
+
+Set this if the backend might gzip objects.
+
+Normally providers will not alter objects when they are downloaded. If
+an object was not uploaded with `Content-Encoding: gzip` then it won't
+be set on download.
+
+However some providers may gzip objects even if they weren't uploaded
+with `Content-Encoding: gzip` (eg Cloudflare).
+
+A symptom of this would be receiving errors like
+
+ ERROR corrupted on transfer: sizes differ NNN vs MMM
+
+If you set this flag and rclone downloads an object with
+Content-Encoding: gzip set and chunked transfer encoding, then rclone
+will decompress the object on the fly.
+
+If this is set to unset (the default) then rclone will choose
+according to the provider setting what to apply, but you can override
+rclone's choice here.
+
+
+Properties:
+
+- Config: might_gzip
+- Env Var: RCLONE_S3_MIGHT_GZIP
+- Type: Tristate
+- Default: unset
+
#### --s3-no-system-metadata
Suppress setting and reading of system metadata
diff --git a/rclone.1 b/rclone.1
index fd0219cdd..0fdf4ad99 100644
--- a/rclone.1
+++ b/rclone.1
@@ -1,7 +1,7 @@
.\"t
.\" Automatically generated by Pandoc 2.9.2.1
.\"
-.TH "rclone" "1" "Oct 21, 2022" "User Manual" ""
+.TH "rclone" "1" "Nov 17, 2022" "User Manual" ""
.hy
.SH Rclone syncs your files to cloud storage
.PP
@@ -490,13 +490,13 @@ links.
If not sure, use the first link.
.IP \[bu] 2
Intel/AMD - 64
-Bit (https://downloads.rclone.org/rclone-current-linux-amd64.zip)
+Bit (https://downloads.rclone.org/rclone-current-windows-amd64.zip)
.IP \[bu] 2
Intel/AMD - 32
-Bit (https://downloads.rclone.org/rclone-current-linux-386.zip)
+Bit (https://downloads.rclone.org/rclone-current-windows-386.zip)
.IP \[bu] 2
ARM - 64
-Bit (https://downloads.rclone.org/rclone-current-linux-arm64.zip)
+Bit (https://downloads.rclone.org/rclone-current-windows-arm64.zip)
.PP
Open this file in the Explorer and extract \f[C]rclone.exe\f[R].
Rclone is a portable executable so you can place it wherever is
@@ -19313,7 +19313,7 @@ T}
T{
Oracle Object Storage
T}@T{
-Yes
+No
T}@T{
Yes
T}@T{
@@ -19325,7 +19325,7 @@ Yes
T}@T{
Yes
T}@T{
-No
+Yes
T}@T{
No
T}@T{
@@ -19924,7 +19924,7 @@ These flags are available for every command.
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
- --user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.60.0\[dq])
+ --user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.60.1\[dq])
-v, --verbose count Print lots more stuff (repeat for more)
\f[R]
.fi
@@ -20308,6 +20308,7 @@ They control the backends and may be set in the config file.
--s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
--s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
--s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
+ --s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
--s3-no-check-bucket If set, don\[aq]t attempt to check the bucket exists or create it
--s3-no-head If set, don\[aq]t HEAD uploaded objects to check integrity
--s3-no-head-object If set, do not do HEAD before GET when getting objects
@@ -26648,19 +26649,57 @@ Seagate Lyve Cloud AP Southeast 1 (Singapore)
\[dq]s3.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
-Wasabi US East endpoint
+Wasabi US East 1 (N.
+Virginia)
+.RE
+.IP \[bu] 2
+\[dq]s3.us-east-2.wasabisys.com\[dq]
+.RS 2
+.IP \[bu] 2
+Wasabi US East 2 (N.
+Virginia)
+.RE
+.IP \[bu] 2
+\[dq]s3.us-central-1.wasabisys.com\[dq]
+.RS 2
+.IP \[bu] 2
+Wasabi US Central 1 (Texas)
.RE
.IP \[bu] 2
\[dq]s3.us-west-1.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
-Wasabi US West endpoint
+Wasabi US West 1 (Oregon)
+.RE
+.IP \[bu] 2
+\[dq]s3.ca-central-1.wasabisys.com\[dq]
+.RS 2
+.IP \[bu] 2
+Wasabi CA Central 1 (Toronto)
.RE
.IP \[bu] 2
\[dq]s3.eu-central-1.wasabisys.com\[dq]
.RS 2
.IP \[bu] 2
-Wasabi EU Central endpoint
+Wasabi EU Central 1 (Amsterdam)
+.RE
+.IP \[bu] 2
+\[dq]s3.eu-central-2.wasabisys.com\[dq]
+.RS 2
+.IP \[bu] 2
+Wasabi EU Central 2 (Frankfurt)
+.RE
+.IP \[bu] 2
+\[dq]s3.eu-west-1.wasabisys.com\[dq]
+.RS 2
+.IP \[bu] 2
+Wasabi EU West 1 (London)
+.RE
+.IP \[bu] 2
+\[dq]s3.eu-west-2.wasabisys.com\[dq]
+.RS 2
+.IP \[bu] 2
+Wasabi EU West 2 (Paris)
.RE
.IP \[bu] 2
\[dq]s3.ap-northeast-1.wasabisys.com\[dq]
@@ -26675,6 +26714,18 @@ Wasabi AP Northeast 1 (Tokyo) endpoint
Wasabi AP Northeast 2 (Osaka) endpoint
.RE
.IP \[bu] 2
+\[dq]s3.ap-southeast-1.wasabisys.com\[dq]
+.RS 2
+.IP \[bu] 2
+Wasabi AP Southeast 1 (Singapore)
+.RE
+.IP \[bu] 2
+\[dq]s3.ap-southeast-2.wasabisys.com\[dq]
+.RS 2
+.IP \[bu] 2
+Wasabi AP Southeast 2 (Sydney)
+.RE
+.IP \[bu] 2
\[dq]s3.ir-thr-at1.arvanstorage.com\[dq]
.RS 2
.IP \[bu] 2
@@ -28823,6 +28874,42 @@ Env Var: RCLONE_S3_DECOMPRESS
Type: bool
.IP \[bu] 2
Default: false
+.SS --s3-might-gzip
+.PP
+Set this if the backend might gzip objects.
+.PP
+Normally providers will not alter objects when they are downloaded.
+If an object was not uploaded with \f[C]Content-Encoding: gzip\f[R] then
+it won\[aq]t be set on download.
+.PP
+However some providers may gzip objects even if they weren\[aq]t
+uploaded with \f[C]Content-Encoding: gzip\f[R] (eg Cloudflare).
+.PP
+A symptom of this would be receiving errors like
+.IP
+.nf
+\f[C]
+ERROR corrupted on transfer: sizes differ NNN vs MMM
+\f[R]
+.fi
+.PP
+If you set this flag and rclone downloads an object with
+Content-Encoding: gzip set and chunked transfer encoding, then rclone
+will decompress the object on the fly.
+.PP
+If this is set to unset (the default) then rclone will choose according
+to the provider setting what to apply, but you can override rclone\[aq]s
+choice here.
+.PP
+Properties:
+.IP \[bu] 2
+Config: might_gzip
+.IP \[bu] 2
+Env Var: RCLONE_S3_MIGHT_GZIP
+.IP \[bu] 2
+Type: Tristate
+.IP \[bu] 2
+Default: unset
.SS --s3-no-system-metadata
.PP
Suppress setting and reading of system metadata
@@ -44213,7 +44300,32 @@ supported by rclone)
.SS Configuration
.PP
Here is an example of making a mailru configuration.
-First create a Mail.ru Cloud account and choose a tariff, then run
+.PP
+First create a Mail.ru Cloud account and choose a tariff.
+.PP
+You will need to log in and create an app password for rclone.
+Rclone \f[B]will not work\f[R] with your normal username and password -
+it will give an error like
+\f[C]oauth2: server response missing access_token\f[R].
+.IP \[bu] 2
+Click on your user icon in the top right
+.IP \[bu] 2
+Go to Security / \[dq]\[u041F]\[u0430]\[u0440]\[u043E]\[u043B]\[u044C]
+\[u0438]
+\[u0431]\[u0435]\[u0437]\[u043E]\[u043F]\[u0430]\[u0441]\[u043D]\[u043E]\[u0441]\[u0442]\[u044C]\[dq]
+.IP \[bu] 2
+Click password for apps /
+\[dq]\[u041F]\[u0430]\[u0440]\[u043E]\[u043B]\[u0438]
+\[u0434]\[u043B]\[u044F]
+\[u0432]\[u043D]\[u0435]\[u0448]\[u043D]\[u0438]\[u0445]
+\[u043F]\[u0440]\[u0438]\[u043B]\[u043E]\[u0436]\[u0435]\[u043D]\[u0438]\[u0439]\[dq]
+.IP \[bu] 2
+Add the password - give it a name - eg \[dq]rclone\[dq]
+.IP \[bu] 2
+Copy the password and use this password below - your normal login
+password won\[aq]t work.
+.PP
+Now run
.IP
.nf
\f[C]
@@ -44244,6 +44356,10 @@ User name (usually email)
Enter a string value. Press Enter for the default (\[dq]\[dq]).
user> username\[at]mail.ru
Password
+
+This must be an app password - rclone will not work with your normal
+password. See the Configuration section in the docs for how to make an
+app password.
y) Yes type in my own password
g) Generate random password
y/g> y
@@ -44441,6 +44557,11 @@ Required: true
.PP
Password.
.PP
+This must be an app password - rclone will not work with your normal
+password.
+See the Configuration section in the docs for how to make an app
+password.
+.PP
\f[B]NB\f[R] Input to this must be obscured - see rclone
obscure (https://rclone.org/commands/rclone_obscure/).
.PP
@@ -56024,6 +56145,93 @@ Options:
.IP \[bu] 2
\[dq]error\[dq]: return an error based on option value
.SH Changelog
+.SS v1.60.1 - 2022-11-17
+.PP
+See commits (https://github.com/rclone/rclone/compare/v1.60.0...v1.60.1)
+.IP \[bu] 2
+Bug Fixes
+.RS 2
+.IP \[bu] 2
+lib/cache: Fix alias backend shutting down too soon (Nick Craig-Wood)
+.IP \[bu] 2
+wasm: Fix walltime link error by adding up-to-date wasm_exec.js
+(Jo\[~a]o Henrique Franco)
+.IP \[bu] 2
+docs
+.RS 2
+.IP \[bu] 2
+Update faq.md with bisync (Samuel Johnson)
+.IP \[bu] 2
+Corrected download links in windows install docs (coultonluke)
+.IP \[bu] 2
+Add direct download link for windows arm64 (albertony)
+.IP \[bu] 2
+Remove link to rclone slack as it is no longer supported (Nick
+Craig-Wood)
+.IP \[bu] 2
+Faq: how to use a proxy server that requires a username and password
+(asdffdsazqqq)
+.IP \[bu] 2
+Oracle-object-storage: doc fix (Manoj Ghosh)
+.IP \[bu] 2
+Fix typo \f[C]remove\f[R] in rclone_serve_restic command (Joda
+St\[:o]\[ss]er)
+.IP \[bu] 2
+Fix character that was incorrectly interpreted as markdown (Cl\['e]ment
+Notin)
+.RE
+.RE
+.IP \[bu] 2
+VFS
+.RS 2
+.IP \[bu] 2
+Fix deadlock caused by cache cleaner and upload finishing (Nick
+Craig-Wood)
+.RE
+.IP \[bu] 2
+Local
+.RS 2
+.IP \[bu] 2
+Clean absolute paths (albertony)
+.IP \[bu] 2
+Fix -L/--copy-links with filters missing directories (Nick Craig-Wood)
+.RE
+.IP \[bu] 2
+Mailru
+.RS 2
+.IP \[bu] 2
+Note that an app password is now needed (Nick Craig-Wood)
+.IP \[bu] 2
+Allow timestamps to be before the epoch 1970-01-01 (Nick Craig-Wood)
+.RE
+.IP \[bu] 2
+S3
+.RS 2
+.IP \[bu] 2
+Add provider quirk \f[C]--s3-might-gzip\f[R] to fix corrupted on
+transfer: sizes differ (Nick Craig-Wood)
+.IP \[bu] 2
+Allow Storj to server side copy since it seems to work now (Nick
+Craig-Wood)
+.IP \[bu] 2
+Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)
+.IP \[bu] 2
+Add additional Wasabi locations (techknowlogick)
+.RE
+.IP \[bu] 2
+Smb
+.RS 2
+.IP \[bu] 2
+Fix \f[C]Failed to sync: context canceled\f[R] at the end of syncs (Nick
+Craig-Wood)
+.RE
+.IP \[bu] 2
+WebDAV
+.RS 2
+.IP \[bu] 2
+Fix Move/Copy/DirMove when using -server-side-across-configs (Nick
+Craig-Wood)
+.RE
.SS v1.60.0 - 2022-10-21
.PP
See commits (https://github.com/rclone/rclone/compare/v1.59.0...v1.60.0)
@@ -67037,10 +67245,8 @@ significant amount of metadata, which breaks the desired 1:1 mapping of
files to objects.
.SS Can rclone do bi-directional sync?
.PP
-No, not at present.
-rclone only does uni-directional sync from A -> B.
-It may do in the future though since it has all the primitives - it just
-requires writing the algorithm to do it.
+Yes, since rclone v1.58.0, bidirectional cloud
+sync (https://rclone.org/bisync/) is available.
.SS Can I use rclone with an HTTP proxy?
.PP
Yes.
@@ -67073,6 +67279,17 @@ export HTTPS_PROXY=$http_proxy
\f[R]
.fi
.PP
+Note: If the proxy server requires a username and password, then use
+.IP
+.nf
+\f[C]
+export http_proxy=http://username:password\[at]proxyserver:12345
+export https_proxy=$http_proxy
+export HTTP_PROXY=$http_proxy
+export HTTPS_PROXY=$http_proxy
+\f[R]
+.fi
+.PP
The \f[C]NO_PROXY\f[R] allows you to disable the proxy for specific
hosts.
Hosts must be comma separated, and can contain domains or parts.