extern/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2024-12-23 07:29:35 +01:00
Code Issues Packages Projects Releases Wiki Activity
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
5,240 Commits 182 Branches 114 Tags 334 MiB
Go 98.6%
HTML 0.4%
Shell 0.3%
Python 0.3%
JavaScript 0.2%
Other 0.1%
9cc8ff4dd4
Go to file
Maxwell Calman 9cc8ff4dd4 chunker: partially implement no-rename transactions (#4675) 
Some storage providers e.g. S3 don't have an efficient rename operation.
Before this change, when chunker finished an upload, the server-side copy
and delete operations that renamed temporary chunks to their final names
could take a significant amount of time.
This PR records transaction identifier (versioning) in the metadata of
chunker composite objects striving to remove the need for rename
operations on such backends.
This approach will be triggered be the new "transactions" configuration
option, which can be "rename" (the default) or "norename".
We implement the new approach for uploads (Put operations).
The chunker Move operation still uses the rename operation of
underlying backend. Filling this gap is left for a later PR.

Co-authored-by: Ivan Andreev <ivandeex@gmail.com>
2021-02-28 10:49:17 +00:00
.github build: replace go 1.16-rc1 by 1.16.x (#5036) 2021-02-17 03:37:30 +03:00
backend chunker: partially implement no-rename transactions (#4675) 2021-02-28 10:49:17 +00:00
bin test: add makefiles test command (converted from script) 2021-02-13 14:24:43 +00:00
cmd cmount: fix mount dropping on macOS by setting --daemon-timeout 10m 2021-02-21 12:56:19 +00:00
contrib/docker Contrib: Add sample WebDAV server Docker Compose manifest. 2019-09-15 16:06:54 +01:00
docs chunker: partially implement no-rename transactions (#4675) 2021-02-28 10:49:17 +00:00
fs rc: implement passing filter config with _filter parameter 2021-02-23 10:54:40 +00:00
fstest chunker: partially implement no-rename transactions (#4675) 2021-02-28 10:49:17 +00:00
graphics build: Add file properties and icon to Windows executable (fixes #4304) 2020-06-11 09:26:14 +01:00
lib dropbox: add scopes to oauth request and optionally "members.read" 2021-02-13 12:35:24 +00:00
vfs vfs: --vfs-used-is-size to report used space using recursive scan (#4043) 2021-02-17 23:36:13 +03:00
.gitattributes build: add azure pipelines build 2019-08-06 10:31:32 +01:00
.gitignore lib: Add file name compression 2020-08-13 16:14:11 +01:00
.golangci.yml build: move linter build tags into Makefile to fix golangci-lint 2019-04-12 15:48:36 +01:00
CONTRIBUTING.md CONTRIBUTING.md: recommend to push feature branch with -u + minor tuneups 2021-02-24 20:24:59 +00:00
COPYING Initial commit - some small parts working 2012-11-18 17:32:31 +00:00
Dockerfile build: add an rclone user to the Docker image but don't use it by default 2021-02-11 17:45:44 +00:00
go.mod dropbox: add polling support - fixes #2949 2021-02-24 09:33:31 +00:00
go.sum dropbox: add polling support - fixes #2949 2021-02-24 09:33:31 +00:00
MAINTAINERS.md compress: finish docs 2020-12-02 16:30:02 +01:00
Makefile build: ensure go1.16 go gets the correct versions of tools 2021-02-04 17:34:30 +00:00
MANUAL.html docs: replace #file-caching with #vfs-file-caching 2021-02-13 12:56:25 +00:00
MANUAL.md docs: replace #file-caching with #vfs-file-caching 2021-02-13 12:56:25 +00:00
MANUAL.txt Version v1.54.0 2021-02-02 14:17:09 +00:00
notes.txt docs: spelling: etc. 2020-10-28 18:16:23 +00:00
rclone.1 Version v1.54.0 2021-02-02 14:17:09 +00:00
rclone.go lib: add plugin support 2019-09-29 11:05:10 +01:00
README.md HDFS (Hadoop Distributed File System) implementation - #42 2021-01-07 09:48:51 +00:00
RELEASE.md build: ensure go1.16 go gets the correct versions of tools 2021-02-04 17:34:30 +00:00
VERSION Start v1.55.0-DEV development 2021-02-02 16:30:58 +00:00

README.md

rclone logo

Website | Documentation | Download | Contributing | Changelog | Installation | Forum

Build Status Go Report Card GoDoc Docker Pulls

Rclone

Rclone ("rsync for cloud storage") is a command line program to sync files and directories to and from different cloud storage providers.

Storage providers

Please see the full list of all storage providers and their features

Features

  • MD5/SHA-1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Check mode to check for file hash equality
  • Can sync to and from network, e.g. two different cloud accounts
  • Optional large file chunking (Chunker)
  • Optional transparent compression (Compress)
  • Optional encryption (Crypt)
  • Optional cache (Cache)
  • Optional FUSE mount (rclone mount)
  • Multi-threaded downloads to local disk
  • Can serve local or remote files over HTTP/WebDav/FTP/SFTP/dlna

Installation & documentation

Please see the rclone website for:

Downloads

License

This is free software under the terms of MIT the license (check the COPYING file included in this package).