extern/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2024-11-23 00:43:49 +01:00
Code Issues Packages Projects Releases Wiki Activity
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
6,697 Commits 177 Branches 114 Tags 315 MiB
Go 98.6%
HTML 0.4%
Shell 0.3%
Python 0.3%
JavaScript 0.2%
Other 0.1%
351fc609b1
Go to file
Nick Craig-Wood 351fc609b1 b2: fix uploading files bigger than 1TiB 
Before this change when uploading files bigger than 1TiB, the chunk
calculator would work out that the chunk size needed to be bigger than
the default 100 MiB to fit within the 10,000 parts limit.

However the uploader was still using the memory pool for the old chunk
size and this caused errors like

    panic: runtime error: slice bounds out of range [:122683392] with capacity 100663296

The fix for this is to make a temporary pool with the larger chunk
size and use it during the upload of the large file.

See: https://forum.rclone.org/t/rclone-cannot-complete-upload-to-b2-restarts-upload-frequently/35617/
2023-01-22 12:46:23 +00:00
.github build: update to go1.20rc3 and make go1.17 the minimum required version 2023-01-16 11:19:16 +00:00
backend b2: fix uploading files bigger than 1TiB 2023-01-22 12:46:23 +00:00
bin docs: Add minimum versions to all the backend pages and some of the other pages 2022-11-18 14:41:24 +00:00
cmd mount,mount2,cmount: fix --allow-non-empty #3562 2023-01-20 15:39:54 +00:00
cmdtest test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
contrib serve/docker: build docker plugin for multiple platforms #5668 2021-10-21 13:01:23 +03:00
docs Add alankrit to contributors 2023-01-20 15:39:49 +00:00
fs fs:Added multiple ca certificate support. 2023-01-17 12:16:11 +00:00
fstest seafile: renew library password - fixes #6662 2023-01-15 16:26:29 +00:00
graphics build: Add file properties and icon to Windows executable (fixes #4304) 2020-06-11 09:26:14 +01:00
lib lib/errors: add support for unwrapping go1.20 multi errors 2023-01-17 11:35:19 +00:00
librclone config: add config/setpath for setting config path via rc/librclone 2022-12-15 12:41:30 +00:00
vfs test: replace defer cleanup with t.Cleanup 2022-12-09 14:38:05 +00:00
.gitattributes build: add azure pipelines build 2019-08-06 10:31:32 +01:00
.gitignore chunker: fix md5all test for no-meta test remotes 2021-10-18 18:04:07 +03:00
.golangci.yml build: fix lint option max-issues-per-linter 2022-09-14 12:11:54 +01:00
CONTRIBUTING.md all: fix spelling across the project 2022-08-30 11:16:26 +02:00
COPYING Initial commit - some small parts working 2012-11-18 17:32:31 +00:00
Dockerfile build: add an rclone user to the Docker image but don't use it by default 2021-02-11 17:45:44 +00:00
go.mod build: update to go1.20rc3 and make go1.17 the minimum required version 2023-01-16 11:19:16 +00:00
go.sum build: update to go1.20rc3 and make go1.17 the minimum required version 2023-01-16 11:19:16 +00:00
MAINTAINERS.md storj: rename tardigrade backend to storj backend #5616 2022-02-11 11:04:15 +00:00
Makefile Add compiletest target to compile all the tests only 2022-12-15 12:38:09 +00:00
MANUAL.html Version v1.61.0 2022-12-20 17:16:14 +00:00
MANUAL.md Version v1.61.0 2022-12-20 17:16:14 +00:00
MANUAL.txt Version v1.61.0 2022-12-20 17:16:14 +00:00
notes.txt docs: spelling: etc. 2020-10-28 18:16:23 +00:00
rclone.1 Version v1.61.0 2022-12-20 17:16:14 +00:00
rclone.go lib: add plugin support 2019-09-29 11:05:10 +01:00
README.md s3: add Liara LOS to provider list 2022-12-06 12:25:23 +00:00
RELEASE.md docs: correct RELEASE procedure for stable branch 2022-12-23 12:34:04 +00:00
VERSION Start v1.62.0-DEV development 2022-12-20 18:33:59 +00:00

README.md

rclone logo rclone logo

Website | Documentation | Download | Contributing | Changelog | Installation | Forum

Build Status Go Report Card GoDoc Docker Pulls

Rclone

Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.

Storage providers

Please see the full list of all storage providers and their features

Virtual storage providers

These backends adapt or modify other storage providers

  • Alias: rename existing remotes 📄
  • Cache: cache remotes (DEPRECATED) 📄
  • Chunker: split large files 📄
  • Combine: combine multiple remotes into a directory tree 📄
  • Compress: compress files 📄
  • Crypt: encrypt files 📄
  • Hasher: hash files 📄
  • Union: join multiple remotes to work together 📄

Features

  • MD5/SHA-1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Check mode to check for file hash equality
  • Can sync to and from network, e.g. two different cloud accounts
  • Optional large file chunking (Chunker)
  • Optional transparent compression (Compress)
  • Optional encryption (Crypt)
  • Optional FUSE mount (rclone mount)
  • Multi-threaded downloads to local disk
  • Can serve local or remote files over HTTP/WebDAV/FTP/SFTP/DLNA

Installation & documentation

Please see the rclone website for:

Downloads

License

This is free software under the terms of the MIT license (check the COPYING file included in this package).