extern/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2024-12-22 23:22:08 +01:00
Code Issues Packages Projects Releases Wiki Activity
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
7,310 Commits 182 Branches 114 Tags 334 MiB
Go 98.6%
HTML 0.4%
Shell 0.3%
Python 0.3%
JavaScript 0.2%
Other 0.1%
2db0e23584
Go to file
Nick Craig-Wood 2db0e23584 backends: change OpenChunkWriter interface to allow backend concurrency override 
Before this change the concurrency used for an upload was rather
inconsistent.

- if size below `--backend-upload-cutoff` (default 200M) do single part upload.

- if size below `--multi-thread-cutoff` (default 256M) or using streaming
  uploads (eg `rclone rcat) do multipart upload using
  `--backend-upload-concurrency` to set the concurrency used by the uploader.

- otherwise do multipart upload using `--multi-thread-streams` to set the
  concurrency.

This change makes the default for the concurrency used be the
`--backend-upload-concurrency`. If `--multi-thread-streams` is set and larger
than the `--backend-upload-concurrency` then that will be used instead.

This means that if the user sets `--backend-upload-concurrency` then it will be
obeyed for all multipart/multi-thread transfers and the user can override them
all with `--multi-thread-streams`.

See: #7056
2023-09-03 11:47:05 +01:00
.github ci: change Winget Releaser job to ubuntu-latest 2023-08-17 11:36:28 +01:00
backend backends: change OpenChunkWriter interface to allow backend concurrency override 2023-09-03 11:47:05 +01:00
bin docs: remove contributor's old email 2023-08-23 12:31:48 +01:00
cmd serve dlna: fix MIME type if backend can't identify it 2023-09-01 18:09:44 +01:00
cmdtest cmdtest: fix typos found by codespell 2023-03-24 20:44:25 +00:00
contrib docker volume plugin: add missing fuse3 dependency #6844 2023-03-15 15:57:53 +00:00
docs backends: change OpenChunkWriter interface to allow backend concurrency override 2023-09-03 11:47:05 +01:00
fs backends: change OpenChunkWriter interface to allow backend concurrency override 2023-09-03 11:47:05 +01:00
fstest fstests: add backend integration test for purging root directory #7247 2023-08-25 17:39:07 +01:00
graphics build: Add file properties and icon to Windows executable (fixes #4304) 2020-06-11 09:26:14 +01:00
lib backends: change OpenChunkWriter interface to allow backend concurrency override 2023-09-03 11:47:05 +01:00
librclone librclone:Added example on using librclone with golang. 2023-03-17 12:00:27 +00:00
vfs vfs: make sure struct field is aligned for atomic access 2023-08-22 12:52:13 +01:00
.gitattributes build: add azure pipelines build 2019-08-06 10:31:32 +01:00
.gitignore gitignore: add .DS_Store and remove *.log 2023-08-12 17:24:21 +01:00
.golangci.yml build: re-enable revive linters 2023-08-29 13:01:15 +01:00
CONTRIBUTING.md docs: add some more docs on making your own backend 2023-08-09 20:00:46 +09:00
COPYING Initial commit - some small parts working 2012-11-18 17:32:31 +00:00
Dockerfile docker: add missing fuse3 dependency - fixes #6844 2023-03-15 10:54:30 +00:00
go.mod serve ftp: update to goftp.io/server v2.0.1 - fixes #7237 2023-08-22 17:24:05 +01:00
go.sum serve ftp: update to goftp.io/server v2.0.1 - fixes #7237 2023-08-22 17:24:05 +01:00
MAINTAINERS.md build: add snap installation 2023-08-18 11:57:25 +01:00
Makefile build: fix build failure installing nfpm 2023-06-07 15:41:52 +01:00
MANUAL.html Version v1.63.0 2023-06-30 14:11:17 +01:00
MANUAL.md Version v1.63.0 2023-06-30 14:11:17 +01:00
MANUAL.txt Version v1.63.0 2023-06-30 14:11:17 +01:00
notes.txt docs: spelling: etc. 2020-10-28 18:16:23 +00:00
rclone.1 Version v1.63.0 2023-06-30 14:11:17 +01:00
rclone.go lib: add plugin support 2019-09-29 11:05:10 +01:00
README.md Add Object storage to Leviia on README.md 2023-07-18 09:52:05 +01:00
RELEASE.md build: update release docs to be more careful with the tag 2023-03-15 14:53:21 +00:00
VERSION Start v1.64.0-DEV development 2023-06-30 15:39:03 +01:00

README.md

rclone logo rclone logo

Website | Documentation | Download | Contributing | Changelog | Installation | Forum

Build Status Go Report Card GoDoc Docker Pulls

Rclone

Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.

Storage providers

Please see the full list of all storage providers and their features

Virtual storage providers

These backends adapt or modify other storage providers

  • Alias: rename existing remotes 📄
  • Cache: cache remotes (DEPRECATED) 📄
  • Chunker: split large files 📄
  • Combine: combine multiple remotes into a directory tree 📄
  • Compress: compress files 📄
  • Crypt: encrypt files 📄
  • Hasher: hash files 📄
  • Union: join multiple remotes to work together 📄

Features

  • MD5/SHA-1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Check mode to check for file hash equality
  • Can sync to and from network, e.g. two different cloud accounts
  • Optional large file chunking (Chunker)
  • Optional transparent compression (Compress)
  • Optional encryption (Crypt)
  • Optional FUSE mount (rclone mount)
  • Multi-threaded downloads to local disk
  • Can serve local or remote files over HTTP/WebDAV/FTP/SFTP/DLNA

Installation & documentation

Please see the rclone website for:

Downloads

License

This is free software under the terms of the MIT license (check the COPYING file included in this package).