"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
Go to file
Nick Craig-Wood ab803d1278 b2: implement OpenChunkWriter and multi-thread uploads #7056
This implements the OpenChunkWriter interface for b2 which
enables multi-thread uploads.

This makes the memory controls of the s3 backend inoperative; they are
replaced with the global ones.

    --b2-memory-pool-flush-time
    --b2-memory-pool-use-mmap

By using the buffered reader this fixes excessive memory use when
uploading large files as it will share memory pages between all
readers.
2023-08-24 12:39:27 +01:00
.github ci: change Winget Releaser job to ubuntu-latest 2023-08-17 11:36:28 +01:00
backend b2: implement OpenChunkWriter and multi-thread uploads #7056 2023-08-24 12:39:27 +01:00
bin docs: remove contributor's old email 2023-08-23 12:31:48 +01:00
cmd serve ftp: fix race condition when using the auth proxy 2023-08-23 15:11:47 +01:00
cmdtest cmdtest: fix typos found by codespell 2023-03-24 20:44:25 +00:00
contrib docker volume plugin: add missing fuse3 dependency #6844 2023-03-15 15:57:53 +00:00
docs operations: document multi-thread copy and tweak defaults 2023-08-24 12:39:27 +01:00
fs operations: document multi-thread copy and tweak defaults 2023-08-24 12:39:27 +01:00
fstest fs: add context.Ctx to ChunkWriter methods 2023-08-24 12:39:27 +01:00
graphics build: Add file properties and icon to Windows executable (fixes #4304) 2020-06-11 09:26:14 +01:00
lib s3: factor generic multipart upload into lib/multipart #7056 2023-08-24 12:39:27 +01:00
librclone librclone:Added example on using librclone with golang. 2023-03-17 12:00:27 +00:00
vfs vfs: make sure struct field is aligned for atomic access 2023-08-22 12:52:13 +01:00
.gitattributes build: add azure pipelines build 2019-08-06 10:31:32 +01:00
.gitignore gitignore: add .DS_Store and remove *.log 2023-08-12 17:24:21 +01:00
.golangci.yml Use jwt-go (golang-jwt) instead of deprecated jws (x/oauth2/jws) 2023-03-26 19:20:50 +02:00
CONTRIBUTING.md docs: add some more docs on making your own backend 2023-08-09 20:00:46 +09:00
COPYING Initial commit - some small parts working 2012-11-18 17:32:31 +00:00
Dockerfile docker: add missing fuse3 dependency - fixes #6844 2023-03-15 10:54:30 +00:00
go.mod serve ftp: update to goftp.io/server v2.0.1 - fixes #7237 2023-08-22 17:24:05 +01:00
go.sum serve ftp: update to goftp.io/server v2.0.1 - fixes #7237 2023-08-22 17:24:05 +01:00
MAINTAINERS.md build: add snap installation 2023-08-18 11:57:25 +01:00
Makefile build: fix build failure installing nfpm 2023-06-07 15:41:52 +01:00
MANUAL.html Version v1.63.0 2023-06-30 14:11:17 +01:00
MANUAL.md Version v1.63.0 2023-06-30 14:11:17 +01:00
MANUAL.txt Version v1.63.0 2023-06-30 14:11:17 +01:00
notes.txt docs: spelling: etc. 2020-10-28 18:16:23 +00:00
rclone.1 Version v1.63.0 2023-06-30 14:11:17 +01:00
rclone.go lib: add plugin support 2019-09-29 11:05:10 +01:00
README.md Add Object storage to Leviia on README.md 2023-07-18 09:52:05 +01:00
RELEASE.md build: update release docs to be more careful with the tag 2023-03-15 14:53:21 +00:00
VERSION Start v1.64.0-DEV development 2023-06-30 15:39:03 +01:00

rclone logo rclone logo

Website | Documentation | Download | Contributing | Changelog | Installation | Forum

Build Status Go Report Card GoDoc Docker Pulls

Rclone

Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.

Storage providers

Please see the full list of all storage providers and their features

Virtual storage providers

These backends adapt or modify other storage providers

  • Alias: rename existing remotes 📄
  • Cache: cache remotes (DEPRECATED) 📄
  • Chunker: split large files 📄
  • Combine: combine multiple remotes into a directory tree 📄
  • Compress: compress files 📄
  • Crypt: encrypt files 📄
  • Hasher: hash files 📄
  • Union: join multiple remotes to work together 📄

Features

  • MD5/SHA-1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Check mode to check for file hash equality
  • Can sync to and from network, e.g. two different cloud accounts
  • Optional large file chunking (Chunker)
  • Optional transparent compression (Compress)
  • Optional encryption (Crypt)
  • Optional FUSE mount (rclone mount)
  • Multi-threaded downloads to local disk
  • Can serve local or remote files over HTTP/WebDAV/FTP/SFTP/DLNA

Installation & documentation

Please see the rclone website for:

Downloads

License

This is free software under the terms of the MIT license (check the COPYING file included in this package).