rclone/backend
Nick Craig-Wood 1648c1a0f3 crypt: calculate hashes for uploads from local disk
Before this change crypt would not calculate hashes for files it was
uploading. This is because, in the general case, they have to be
downloaded, encrypted and hashed which is too resource intensive.

However this causes backends which need the hash first before
uploading (eg s3/b2 when uploading chunked files) not to have a hash
of the file. This causes cryptcheck to complain about missing hashes
on large files uploaded via s3/b2.

This change calculates hashes for the upload if the upload is coming
from a local filesystem. It does this by encrypting and hashing the
local file re-using the code used by cryptcheck. For a local disk this
is not a lot more intensive than calculating the hash.

See: https://forum.rclone.org/t/strange-output-for-cryptcheck/15437
Fixes: #2809
2020-04-22 11:33:48 +01:00
..
alias build: fix up package paths after repo move 2019-07-28 18:47:38 +01:00
all The memory backend 2020-01-18 10:41:08 +00:00
amazonclouddrive backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
azureblob backend/azureblob: fmt nit 2020-02-20 15:50:53 +01:00
b2 b2: ignore directory markers at the root also 2020-03-31 11:46:17 +01:00
box backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
cache cache: move methods used for testing into test file 2020-03-16 18:41:32 +00:00
chunker chunker: reduce length of temporary suffix 2019-12-09 16:56:32 +00:00
crypt crypt: calculate hashes for uploads from local disk 2020-04-22 11:33:48 +01:00
drive drive: don't delete files with multiple parents to avoid data loss 2020-03-31 17:28:32 +01:00
dropbox dropbox: make error insufficient space to be fatal 2020-03-26 16:19:50 +00:00
fichier backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
ftp ftp: attempt to work-around pureftp sending spurious 150 messages 2020-03-01 09:17:51 +00:00
googlecloudstorage gcs: ignore directory markers at the root also 2020-03-31 11:46:10 +01:00
googlephotos googlephotos: make the start year configurable - fixes #3630 2020-04-15 18:08:07 +01:00
http fstest: create AssertTimeEqualWithPrecision from CheckTimeEqualWithPrecision 2020-04-22 11:33:00 +01:00
hubic Add context to all http.NewRequest #3257 2019-09-09 23:27:07 +01:00
jottacloud jottacloud: implement cleanup 2020-04-11 16:42:25 +02:00
koofr backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
local local: factor PreAllocate and SetSparse to lib/file 2020-04-19 15:18:49 +01:00
mailru backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
mega backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
memory The memory backend 2020-01-18 10:41:08 +00:00
onedrive onedrive: make error quotaLimitReached to be fatal - Fixes #4089 2020-03-26 16:19:50 +00:00
opendrive opendrive: implement --opendrive-chunk-size #3707 2020-01-18 11:56:01 +00:00
pcloud backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
premiumizeme backend/premiumizeme: prune unused functions 2020-02-11 12:17:35 +00:00
putio backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
qingstor qingstor: make rclone cleanup remove pending multipart uploads older than 24h 2020-03-18 12:49:21 +00:00
s3 s3: report errors on bucket creation (mkdir) correctly 2020-04-15 13:13:13 +01:00
sftp build: implement a framework for starting test servers during tests 2020-01-18 16:47:37 +00:00
sharefile backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
sugarsync sugarsync: new backend - fixes #622 2020-01-17 17:39:34 +00:00
swift swift: Update OVH API endpoint - Fixes #3890 2020-01-24 17:43:00 +00:00
union union: Implement multiple writable remotes 2020-03-21 18:11:24 +00:00
webdav webdav: report full and consistent usage with about 2020-03-05 15:10:19 +00:00
yandex backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00