Introduce cipherVersion on the object level, so it can then be used in Size() function to determine the correct decrypted size based on explicit object version; Modify DecryptDataSeek and newDecrypterSeek, to include 'o *Object' parameter, so it can then be passed to newDecrypter where version detection happens based on magic bytes.
In V2 version each file encrypted using separate CEK (wrapped with AES Keywrap RFC3394). Truncation protection uses last byte of nonce to indicate last encryption block.
Added MD5 hash support for crypt.go (only if V2 cipher is used), MD5 of plaintext is encrypted and authenticated at the end of file.
Added 4 reserved bytes at the beginning for future use.
Added 1 hash type byte in the footer to allow upgrade to different than MD5 hash.
cipher_version setting determines hash version used for writes.
For reads we determine cipher version by reading header magic bytes, that's regardless of cipher_version setting. A
Added TestNewEncrypterV2 test to demonstrate the outputs and make sure that outputs match the expected hardcoded values.
Modified other tests to satisfy newly added params (e.g. cek).
Related to: https://github.com/rclone/rclone/issues/7192
This changes crypt's use of sync.Pool: Instead of storing slices
it now stores pointers pointers fixed sized arrays.
This issue was reported by staticcheck:
SA6002 - Storing non-pointer values in sync.Pool allocates memory
A sync.Pool is used to avoid unnecessary allocations and reduce
the amount of work the garbage collector has to do.
When passing a value that is not a pointer to a function that accepts
an interface, the value needs to be placed on the heap, which means
an additional allocation. Slices are a common thing to put in sync.Pools,
and they're structs with 3 fields (length, capacity, and a pointer to
an array). In order to avoid the extra allocation, one should store
a pointer to the slice instead.
See: https://staticcheck.io/docs/checks#SA6002
Before this change the code wasn't taking into account the error
io.ErrUnexpectedEOF that io.ReadFull can return properly. Sometimes
that error was being returned instead of a more specific and useful
error.
To fix this, io.ReadFull was replaced with the simpler
readers.ReadFill which is much easier to use correctly.
This is possible now that we no longer support go1.12 and brings
rclone into line with standard practices in the Go world.
This also removes errors.New and errors.Errorf from lib/errors and
prefers the stdlib errors package over lib/errors.
With the file version format standardized in lib/version, `crypt` can
now treat the version strings separately from the encrypted/decrypted
file names. This allows --b2-versions to work with `crypt`.
Fixes#1627
Co-authored-by: Luc Ritchie <luc.ritchie@gmail.com>
- Change rclone/fs interfaces to accept context.Context
- Update interface implementations to use context.Context
- Change top level usage to propagate context to lover level functions
Context propagation is needed for stopping transfers and passing other
request-scoped values.
