extern/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2024-11-22 00:13:49 +01:00
Code Issues Packages Projects Releases Wiki Activity
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
8,018 Commits 177 Branches 114 Tags 314 MiB
Go 98.6%
HTML 0.4%
Shell 0.3%
Python 0.3%
JavaScript 0.2%
Other 0.1%
d068e0b1a9
Go to file
Nick Craig-Wood d068e0b1a9 operations: fix hashing problem in integration tests 
Before this change backends which supported more than one hash (eg
pcloud) or backends which wrapped backends supporting more than one
hash (combine) would fail the TestMultithreadCopy and
TestMultithreadCopyAbort with an error like

    Failed to make new multi hasher: requested set 000001ff contains unknown hash types

This was caused by the tests limiting the globally available hashes to
the first hash supplied by the backend.

This was added in this commit

d5d28a7513 operations: fix overwrite of destination when multi-thread transfer fails

to overcome the tests taking >100s on the local backend because they
made every single hash that the local backend. It brought this time
down to 20s.

This commit fixes the problem and retains the CPU speedup by only
applying the fix from the original commit if the destination backend
is the local backend. This fixes the common case (testing on the local
backend). This does not fix the problem for a backend which wraps the
local backend (eg combine) but this is run only on the integration
test machine and not on all the CI.
2024-06-12 11:11:54 +01:00
.github build: update issue label notification machinery 2024-05-15 15:07:45 +01:00
backend pikpak: implement configurable chunk size for multipart upload 2024-06-12 13:19:25 +09:00
bin tree-wide: replace /bin/bash with /usr/bin/env bash 2024-06-11 12:47:47 +01:00
cmd tree-wide: replace /bin/bash with /usr/bin/env bash 2024-06-11 12:47:47 +01:00
cmdtest cmdtest: fix typos found by codespell 2023-03-24 20:44:25 +00:00
contrib docker volume plugin: add missing fuse3 dependency #6844 2023-03-15 15:57:53 +00:00
docs Add Bill Fraser to contributors 2024-06-12 11:11:54 +01:00
fs operations: fix hashing problem in integration tests 2024-06-12 11:11:54 +01:00
fstest pikpak: remove PublicLink from integration tests 2024-06-12 04:43:49 +09:00
graphics Version v1.66.0 2024-03-10 11:22:43 +00:00
lib windows: make rclone work with SeBackupPrivilege and/or SeRestorePrivilege 2024-06-07 13:26:30 +01:00
librclone docs/librclone: the newer and recommended ucrt64 subsystem of msys2 can now be used for building on windows 2024-01-01 21:56:45 +01:00
vfs vfs: fix renaming a directory 2024-06-11 12:09:19 +01:00
.gitattributes Add go mod and sum to gitattributes for consistent line endings 2024-04-13 11:16:42 +02:00
.gitignore rcserver: set ModTime for dirs and files served by --rc-serve 2024-04-02 12:10:45 +01:00
.golangci.yml build: re-enable revive linters 2023-08-29 13:01:15 +01:00
CONTRIBUTING.md Refresh CONTRIBUTING.md 2023-11-19 12:48:27 +00:00
COPYING Initial commit - some small parts working 2012-11-18 17:32:31 +00:00
Dockerfile build: fix docker build on arm/v6 2024-01-03 17:43:23 +00:00
go.mod build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azidentity 2024-06-12 09:33:04 +01:00
go.sum build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azidentity 2024-06-12 09:33:04 +01:00
MAINTAINERS.md Add @nielash as bisync maintainer 2023-09-05 04:05:39 -04:00
Makefile build: Inject rclone version tag when testing 2024-05-13 18:44:31 +01:00
MANUAL.html Version v1.66.0 2024-03-10 11:22:43 +00:00
MANUAL.md Version v1.66.0 2024-03-10 11:22:43 +00:00
MANUAL.txt Version v1.66.0 2024-03-10 11:22:43 +00:00
notes.txt docs: spelling: etc. 2020-10-28 18:16:23 +00:00
rclone.1 Version v1.66.0 2024-03-10 11:22:43 +00:00
rclone.go lib: add plugin support 2019-09-29 11:05:10 +01:00
README.md s3: Add Magalu S3 Object Storage as provider 2024-06-06 11:25:45 +01:00
RELEASE.md build: update all dependencies 2024-05-22 15:50:31 +01:00
VERSION Start v1.67.0-DEV development 2024-03-10 12:14:00 +00:00

README.md


rclone logo rclone logo

Website | Documentation | Download | Contributing | Changelog | Installation | Forum

Build Status Go Report Card GoDoc Docker Pulls

Rclone

Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.

Storage providers

  • 1Fichier 📄
  • Akamai Netstorage 📄
  • Alibaba Cloud (Aliyun) Object Storage System (OSS) 📄
  • Amazon S3 📄
  • ArvanCloud Object Storage (AOS) 📄
  • Backblaze B2 📄
  • Box 📄
  • Ceph 📄
  • China Mobile Ecloud Elastic Object Storage (EOS) 📄
  • Cloudflare R2 📄
  • Citrix ShareFile 📄
  • DigitalOcean Spaces 📄
  • Digi Storage 📄
  • Dreamhost 📄
  • Dropbox 📄
  • Enterprise File Fabric 📄
  • Fastmail Files 📄
  • FTP 📄
  • Google Cloud Storage 📄
  • Google Drive 📄
  • Google Photos 📄
  • HDFS (Hadoop Distributed Filesystem) 📄
  • HiDrive 📄
  • HTTP 📄
  • Huawei Cloud Object Storage Service(OBS) 📄
  • ImageKit 📄
  • Internet Archive 📄
  • Jottacloud 📄
  • IBM COS S3 📄
  • IONOS Cloud 📄
  • Koofr 📄
  • Leviia Object Storage 📄
  • Liara Object Storage 📄
  • Linkbox 📄
  • Linode Object Storage 📄
  • Magalu Object Storage 📄
  • Mail.ru Cloud 📄
  • Memset Memstore 📄
  • Mega 📄
  • Memory 📄
  • Microsoft Azure Blob Storage 📄
  • Microsoft Azure Files Storage 📄
  • Microsoft OneDrive 📄
  • Minio 📄
  • Nextcloud 📄
  • OVH 📄
  • Blomp Cloud Storage 📄
  • OpenDrive 📄
  • OpenStack Swift 📄
  • Oracle Cloud Storage 📄
  • Oracle Object Storage 📄
  • ownCloud 📄
  • pCloud 📄
  • Petabox 📄
  • PikPak 📄
  • premiumize.me 📄
  • put.io 📄
  • Proton Drive 📄
  • QingStor 📄
  • Qiniu Cloud Object Storage (Kodo) 📄
  • Quatrix 📄
  • Rackspace Cloud Files 📄
  • RackCorp Object Storage 📄
  • Scaleway 📄
  • Seafile 📄
  • SeaweedFS 📄
  • SFTP 📄
  • SMB / CIFS 📄
  • StackPath 📄
  • Storj 📄
  • SugarSync 📄
  • Synology C2 Object Storage 📄
  • Tencent Cloud Object Storage (COS) 📄
  • Uloz.to 📄
  • Wasabi 📄
  • WebDAV 📄
  • Yandex Disk 📄
  • Zoho WorkDrive 📄
  • The local filesystem 📄

Please see the full list of all storage providers and their features

Virtual storage providers

These backends adapt or modify other storage providers

  • Alias: rename existing remotes 📄
  • Cache: cache remotes (DEPRECATED) 📄
  • Chunker: split large files 📄
  • Combine: combine multiple remotes into a directory tree 📄
  • Compress: compress files 📄
  • Crypt: encrypt files 📄
  • Hasher: hash files 📄
  • Union: join multiple remotes to work together 📄

Features

  • MD5/SHA-1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Bisync (two way) to keep two directories in sync bidirectionally
  • Check mode to check for file hash equality
  • Can sync to and from network, e.g. two different cloud accounts
  • Optional large file chunking (Chunker)
  • Optional transparent compression (Compress)
  • Optional encryption (Crypt)
  • Optional FUSE mount (rclone mount)
  • Multi-threaded downloads to local disk
  • Can serve local or remote files over HTTP/WebDAV/FTP/SFTP/DLNA

Installation & documentation

Please see the rclone website for:

Downloads

License

This is free software under the terms of the MIT license (check the COPYING file included in this package).