extern/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2024-12-22 23:22:08 +01:00
Code Issues Packages Projects Releases Wiki Activity
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
5,910 Commits 182 Branches 114 Tags 334 MiB
Go 98.6%
HTML 0.4%
Shell 0.3%
Python 0.3%
JavaScript 0.2%
Other 0.1%
b91c349cd5
Go to file
Nick Craig-Wood b91c349cd5 local: fix hash invalidation which caused errors with local crypt mount 
Before this fix if a file was updated, but to the same length and
timestamp then the local backend would return the wrong (cached)
hashes for the object.

This happens regularly on a crypted local disk mount when the VFS
thinks files have been changed but actually their contents are
identical to that written previously. This is because when files are
uploaded their nonce changes so the contents of the file changes but
the timestamp and size remain the same because the file didn't
actually change.

This causes errors like this:

    ERROR: file: Failed to copy: corrupted on transfer: md5 crypted
    hash differ "X" vs "Y"

This turned out to be because the local backend wasn't clearing its
cache of hashes when the file was updated.

This fix clears the hash cache for Update and Remove.

It also puts a src and destination in the crypt message to make future
debugging easier.

Fixes #4031
2021-11-24 12:09:34 +00:00
.github build: raise minimum go version to go1.15 2021-11-10 16:11:12 +00:00
backend local: fix hash invalidation which caused errors with local crypt mount 2021-11-24 12:09:34 +00:00
bin Remove github.com/pkg/errors and replace with std library version 2021-11-07 11:53:30 +00:00
cmd serve sftp: update docs on --stdio 2021-11-12 10:49:35 +00:00
cmdtest config: support hyphen in remote name from environment variable 2021-10-01 15:18:04 +02:00
contrib serve/docker: build docker plugin for multiple platforms #5668 2021-10-21 13:01:23 +03:00
docs Add GGG KILLER to contributors 2021-11-23 18:00:21 +00:00
fs fs/operations: add server-side moves to stats 2021-11-18 12:20:56 +00:00
fstest fstests: treat accountUpgradeRequired as success for OneDrive PublicLink 2021-11-17 17:35:17 +00:00
graphics build: Add file properties and icon to Windows executable (fixes #4304) 2020-06-11 09:26:14 +01:00
lib Remove github.com/pkg/errors and replace with std library version 2021-11-07 11:53:30 +00:00
librclone Remove github.com/pkg/errors and replace with std library version 2021-11-07 11:53:30 +00:00
vfs vfs: add vfs/stats remote control to show statistics - fixes #5816 2021-11-23 18:00:21 +00:00
.gitattributes build: add azure pipelines build 2019-08-06 10:31:32 +01:00
.gitignore chunker: fix md5all test for no-meta test remotes 2021-10-18 18:04:07 +03:00
.golangci.yml build: increase timeout for golangci-lint to 10 minutes 2021-10-15 15:58:52 +01:00
CONTRIBUTING.md Remove github.com/pkg/errors and replace with std library version 2021-11-07 11:53:30 +00:00
COPYING Initial commit - some small parts working 2012-11-18 17:32:31 +00:00
Dockerfile build: add an rclone user to the Docker image but don't use it by default 2021-02-11 17:45:44 +00:00
go.mod crypt: add base64 and base32768 filename encoding options #5801 2021-11-15 17:57:02 +00:00
go.sum crypt: add base64 and base32768 filename encoding options #5801 2021-11-15 17:57:02 +00:00
MAINTAINERS.md docs: improve grammar and fix typos (#5361) 2021-11-04 12:50:43 +01:00
Makefile build: more docs on upgrading dependencies 2021-11-10 10:34:56 +00:00
MANUAL.html Version v1.57.0 2021-11-01 15:45:40 +00:00
MANUAL.md Version v1.57.0 2021-11-01 15:45:40 +00:00
MANUAL.txt Version v1.57.0 2021-11-01 15:45:40 +00:00
notes.txt docs: spelling: etc. 2020-10-28 18:16:23 +00:00
rclone.1 Version v1.57.0 2021-11-01 15:45:40 +00:00
rclone.go lib: add plugin support 2019-09-29 11:05:10 +01:00
README.md s3: Add RackCorp object storage to providers 2021-11-09 11:46:58 +00:00
RELEASE.md build: more docs on upgrading dependencies 2021-11-10 10:34:56 +00:00
VERSION Start v1.58.0-DEV development 2021-11-01 16:54:17 +00:00

README.md

rclone logo

Website | Documentation | Download | Contributing | Changelog | Installation | Forum

Build Status Go Report Card GoDoc Docker Pulls

Rclone

Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.

Storage providers

Please see the full list of all storage providers and their features

Features

  • MD5/SHA-1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Check mode to check for file hash equality
  • Can sync to and from network, e.g. two different cloud accounts
  • Optional large file chunking (Chunker)
  • Optional transparent compression (Compress)
  • Optional encryption (Crypt)
  • Optional FUSE mount (rclone mount)
  • Multi-threaded downloads to local disk
  • Can serve local or remote files over HTTP/WebDav/FTP/SFTP/dlna

Installation & documentation

Please see the rclone website for:

Downloads

License

This is free software under the terms of MIT the license (check the COPYING file included in this package).