extern/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2024-11-23 00:43:49 +01:00
Code Issues Packages Projects Releases Wiki Activity
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
5,245 Commits 177 Branches 114 Tags 315 MiB
Go 98.6%
HTML 0.4%
Shell 0.3%
Python 0.3%
JavaScript 0.2%
Other 0.1%
f7e3115955
Go to file
Nick Craig-Wood f7e3115955 s3: fix Wasabi HEAD requests returning stale data by using only 1 transport 
In this commit

fc5b14b620 s3: Added `--s3-disable-http2` to disable http/2

We created our own transport so we could disable http/2. However the
added function is called twice meaning that we create two HTTP
transports. This didn't happen with the original code because the
default transport is cached by fshttp.

Rclone normally does a PUT followed by a HEAD request to check an
upload has been successful.

With the two transports, the PUT and the HEAD were being done on
different HTTP transports. This means that it wasn't re-using the same
HTTP connection, so the HEAD request showed the previous object value.
This caused rclone to declare the upload was corrupted, delete the
object and try again.

This patch makes sure we only create one transport and use it for both
PUT and HEAD requests which fixes the problem with Wasabi.

See: https://forum.rclone.org/t/each-time-rclone-is-run-1-3-fails-2-3-succeeds/22545
2021-03-05 15:34:56 +00:00
.github build: replace go 1.16-rc1 by 1.16.x (#5036) 2021-02-17 03:37:30 +03:00
backend s3: fix Wasabi HEAD requests returning stale data by using only 1 transport 2021-03-05 15:34:56 +00:00
bin test: add makefiles test command (converted from script) 2021-02-13 14:24:43 +00:00
cmd cmount: fix mount dropping on macOS by setting --daemon-timeout 10m 2021-02-21 12:56:19 +00:00
contrib/docker Contrib: Add sample WebDAV server Docker Compose manifest. 2019-09-15 16:06:54 +01:00
docs Add Maxwell Calman to contributors 2021-03-05 15:34:56 +00:00
fs dropbox,ftp,onedrive,yandex: make --timeout 0 work properly 2021-03-01 12:08:58 +00:00
fstest chunker: partially implement no-rename transactions (#4675) 2021-02-28 10:49:17 +00:00
graphics build: Add file properties and icon to Windows executable (fixes #4304) 2020-06-11 09:26:14 +01:00
lib dropbox: add scopes to oauth request and optionally "members.read" 2021-02-13 12:35:24 +00:00
vfs vfs: --vfs-used-is-size to report used space using recursive scan (#4043) 2021-02-17 23:36:13 +03:00
.gitattributes build: add azure pipelines build 2019-08-06 10:31:32 +01:00
.gitignore lib: Add file name compression 2020-08-13 16:14:11 +01:00
.golangci.yml build: move linter build tags into Makefile to fix golangci-lint 2019-04-12 15:48:36 +01:00
CONTRIBUTING.md CONTRIBUTING.md: recommend to push feature branch with -u + minor tuneups 2021-02-24 20:24:59 +00:00
COPYING Initial commit - some small parts working 2012-11-18 17:32:31 +00:00
Dockerfile build: add an rclone user to the Docker image but don't use it by default 2021-02-11 17:45:44 +00:00
go.mod ftp: update dependency jlaffaye/ftp (#5097) 2021-03-05 15:58:04 +03:00
go.sum ftp: update dependency jlaffaye/ftp (#5097) 2021-03-05 15:58:04 +03:00
MAINTAINERS.md compress: finish docs 2020-12-02 16:30:02 +01:00
Makefile build: ensure go1.16 go gets the correct versions of tools 2021-02-04 17:34:30 +00:00
MANUAL.html docs: replace #file-caching with #vfs-file-caching 2021-02-13 12:56:25 +00:00
MANUAL.md docs: replace #file-caching with #vfs-file-caching 2021-02-13 12:56:25 +00:00
MANUAL.txt Version v1.54.0 2021-02-02 14:17:09 +00:00
notes.txt docs: spelling: etc. 2020-10-28 18:16:23 +00:00
rclone.1 Version v1.54.0 2021-02-02 14:17:09 +00:00
rclone.go lib: add plugin support 2019-09-29 11:05:10 +01:00
README.md HDFS (Hadoop Distributed File System) implementation - #42 2021-01-07 09:48:51 +00:00
RELEASE.md build: ensure go1.16 go gets the correct versions of tools 2021-02-04 17:34:30 +00:00
VERSION Start v1.55.0-DEV development 2021-02-02 16:30:58 +00:00

README.md

rclone logo

Website | Documentation | Download | Contributing | Changelog | Installation | Forum

Build Status Go Report Card GoDoc Docker Pulls

Rclone

Rclone ("rsync for cloud storage") is a command line program to sync files and directories to and from different cloud storage providers.

Storage providers

Please see the full list of all storage providers and their features

Features

  • MD5/SHA-1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Check mode to check for file hash equality
  • Can sync to and from network, e.g. two different cloud accounts
  • Optional large file chunking (Chunker)
  • Optional transparent compression (Compress)
  • Optional encryption (Crypt)
  • Optional cache (Cache)
  • Optional FUSE mount (rclone mount)
  • Multi-threaded downloads to local disk
  • Can serve local or remote files over HTTP/WebDav/FTP/SFTP/dlna

Installation & documentation

Please see the rclone website for:

Downloads

License

This is free software under the terms of MIT the license (check the COPYING file included in this package).