Commit Graph

1677 Commits

Author SHA1 Message Date
Nick Craig-Wood
6001f05a12 union: the root folder shows the modtime configured by --default-time #6986 2023-05-04 15:03:11 +01:00
Nick Craig-Wood
f7b87a8049 koofr: ensure folders show the modtime configured by --default-time #6986 2023-05-04 15:03:11 +01:00
Nick Craig-Wood
d379641021 http: ensure folders show the modtime configured by --default-time #6986 2023-05-04 15:03:11 +01:00
Nick Craig-Wood
84281c9089 dropbox: ensure folders show the modtime configured by --default-time #6986 2023-05-04 15:03:11 +01:00
Nick Craig-Wood
61d6f538b3 onedrive: add --onedrive-av-override flag to download files flagged as virus
This also produces a warning when rclone detects files have been
blocked because of virus content

    server reports this file is infected with a virus - use --onedrive-av-override to download anyway

Fixes #557
2023-05-03 15:21:30 +01:00
Nick Craig-Wood
65b2e378e0 drive: fix incorrect remote after Update on object
Before this change, when Object.Update was called in the drive
backend, it overwrote the remote with that of the object info.

This is incorrect - the remote doesn't change on Update and this patch
fixes that and introduces a new test to make sure it is correct for
all backends.

This was noticed when doing Update of objects in a nested combine
backend.

See: https://forum.rclone.org/t/rclone-runtime-goroutine-stack-exceeds-1000000000-byte-limit/37912
2023-05-03 13:51:27 +01:00
Nick Craig-Wood
dea6bdf3df combine: fix goroutine stack overflow on bad object
If the Remote() call failed to do its path adjustment, then it would
recursively call Remote() as part of logging the failure and cause a
stack overflow.

This fixes it by logging the underlying object instead.

See: https://forum.rclone.org/t/rclone-runtime-goroutine-stack-exceeds-1000000000-byte-limit/37912
2023-05-03 13:51:27 +01:00
Adam K
5f138dd822 dropbox: syncing documentation with source for dropbox default batch_timeout - fixes #6984 2023-05-02 17:04:32 +01:00
WeidiDeng
ae6874170f
webdav: set modtime using propset for owncloud and nextcloud 2023-04-28 17:38:49 +01:00
Nick Craig-Wood
f5bab284c3 s3: fix missing "tier" metadata
Before this change if the storage class wasn't set on the object, we
didn't set the "tier" metadata.

This made it impossible to filter on tier using the metadata filters.

This returns the "tier" metadata as STANDARD if the storage class
isn't set on the object.

See: https://forum.rclone.org/t/copy-from-s3-to-another-s3-filter-by-storage-class/37861
2023-04-28 14:33:01 +01:00
Nick Craig-Wood
066e00b470 gcs: empty directory markers #3453
- Report correct feature flag
- Fix test failures due to that
- don't output the root directory marker
- Don't create the directory marker if it is the bucket or root
- Create directories when uploading files
2023-04-28 14:31:05 +01:00
Jānis Bebrītis
e0c445d36e gcs: empty directory markers - #3453 2023-04-28 14:31:05 +01:00
Nick Craig-Wood
74652bf318 s3: empty directory markers further work #3453
- Report correct feature flag
- Fix test failures due to that
- don't output the root directory marker
- Don't create the directory marker if it is the bucket or root
- Create directories when uploading files
2023-04-28 14:31:05 +01:00
Jānis Bebrītis
b6a95c70e9 s3: empty directory markers - #3453 2023-04-28 14:31:05 +01:00
Nick Craig-Wood
aca7d0fd22 s3: fix potential crash in integration tests 2023-04-28 14:31:05 +01:00
Tobias Gion
c4a9e480c9
ftp: lower log message priority when SetModTime is not supported to debug
See: https://forum.rclone.org/t/ftp-fritz-box-setmodtime-is-not-supported/37781
2023-04-25 16:31:42 +02:00
Nick Craig-Wood
232d304c13 drive: fix trailing slash mis-identificaton of folder as file
Before this change, drive would mistakenly identify a folder with a
training slash as a file when passed to NewObject.

This was picked up by the integration tests
2023-04-25 12:10:15 +01:00
Nick Craig-Wood
bb6cfe109d crypt: fix reading 0 length files
In an earlier patch

d5afcf9e34 crypt: try not to return "unexpected EOF" error

This introduced a bug for 0 length files which this fixes which only
manifests if the io.Reader returns data and EOF which not all readers
do.

This was failing in the integration tests.
2023-04-24 16:54:40 +01:00
WeidiDeng
864eb89a67
webdav: fix server side copy/move not overwriting - fixes #6964 2023-04-24 14:35:42 +01:00
Nick Craig-Wood
db8c007983 swift: ignore 404 error when deleting an object
See: https://forum.rclone.org/t/rclone-should-optionally-ignore-404-for-delete/37592
2023-04-22 10:49:10 +01:00
jladbrook
281a007b1a
crypt: add suffix option to set a custom suffix for encrypted files - fixes #6392 2023-04-20 17:28:13 +01:00
Brian Starkey
589b7b4873 s3: update Scaleway storage classes
There are now 3 classes:
 * "STANDARD" - Multi-AZ, all regions
 * "ONEZONE_IA" - Single-AZ, FR-PAR only
 * "GLACIER" - Archive, FR-PAR and NL-AMS only
2023-04-19 17:20:30 +01:00
Nick Craig-Wood
04d2781fda fichier: add cdn option to use CDN for download - Fixes #6943 2023-04-18 17:35:21 +01:00
Nick Craig-Wood
451f4c2a8f onedrive: fix quickxorhash on 32 bit architectures
Before this fix quickxorhash would sometimes crash with an error like
this:

    panic: runtime error: slice bounds out of range [-1248:]

This was caused by an incorrect cast of a 64 bit number to a 32 bit
one on 32 bit platforms.

See: https://forum.rclone.org/t/panic-runtime-error-slice-bounds-out-of-range/37548
2023-04-13 15:14:46 +01:00
Roel Arents
a79db20bcd azureblob: send nil tier if empty string 2023-04-05 15:08:32 +01:00
Nick Craig-Wood
037a6bd1b0 crypt: recommend Dropbox for base32768 encoding
See: https://forum.rclone.org/t/base32768-filename-encoding-with-crypt-dropbox-remote/37375
2023-04-05 14:51:21 +01:00
wiserain
243bcc9d07
pikpak: new backend
Fixes #6429
2023-04-04 16:33:48 +01:00
Nick Craig-Wood
64cf9ac911 local: fix /path/to/file.rclonelink when -l/--links is in use
Before this change using /path/to/file.rclonelink would not find the
file when using -l/--links.

This fixes the problem by doing another stat call if the file wasn't
found without the suffix if -l/--links is in use.

It will also give an error if you refer to a symlink without its
suffix which will not work because the limit to a single file
filtering will be using the file name without the .rclonelink suffix.

    need ".rclonelink" suffix to refer to symlink when using -l/--links

Before this change it would use the symlink as a directory which then
would fail when listed.

See: #6855
2023-04-04 10:22:00 +01:00
Nick Craig-Wood
15a3ec8fa1 local: fix filtering of symlinks with -l/--links flag
Before this fix, with the -l flag, the `.rclonelink` suffix wasn't
being added to the file names before filtering by name.

See #6855
2023-04-04 10:22:00 +01:00
Joel
f4c787ab74
sftp: add --sftp-host-key-algorithms to allow specifying SSH host key algorithms 2023-03-30 18:00:54 +01:00
ed
3f0bec2ee9 webdav: make pacer minSleep configurable
This adds the config argument --webdav-pacer-min-sleep which specifies
the http-request rate limit. Lowering this from the default 10ms can
greatly improve performance when synchronizing small files.

See: https://forum.rclone.org/t/webdav-with-persistent-connections/37024/10
2023-03-27 15:30:02 +02:00
albertony
0374ea2c79 Use jwt-go (golang-jwt) instead of deprecated jws (x/oauth2/jws)
golang.org/x/oauth2/jws is deprecated: this package is not intended for public use and
might be removed in the future. It exists for internal use only. Please switch to another
JWS package or copy this package into your own source tree.

github.com/golang-jwt/jwt/v4 seems to be a good alternative, and was already
an implicit dependency.
2023-03-26 19:20:50 +02:00
albertony
9172c9b3dd crypt: reduce allocations
This changes crypt's use of sync.Pool: Instead of storing slices
it now stores pointers pointers fixed sized arrays.

This issue was reported by staticcheck:

SA6002 - Storing non-pointer values in sync.Pool allocates memory

A sync.Pool is used to avoid unnecessary allocations and reduce
the amount of work the garbage collector has to do.

When passing a value that is not a pointer to a function that accepts
an interface, the value needs to be placed on the heap, which means
an additional allocation. Slices are a common thing to put in sync.Pools,
and they're structs with 3 fields (length, capacity, and a pointer to
an array). In order to avoid the extra allocation, one should store
a pointer to the slice instead.

See: https://staticcheck.io/docs/checks#SA6002
2023-03-26 14:28:15 +02:00
albertony
78deab05f9 netstorage: ignore false positive from the staticcheck linter regarding header name not being canonical 2023-03-26 14:28:15 +02:00
albertony
5f3c276d0a zoho: remove unused code (fixes issue reported by the unused linter) 2023-03-26 14:28:15 +02:00
albertony
9cea493f58 union: remove unused code (fixes issue reported by the unused linter) 2023-03-26 14:28:15 +02:00
albertony
400d1a4468 swift: remove unused code (fixes issue reported by the unused linter) 2023-03-26 14:28:15 +02:00
albertony
851ce0f4fe seafile: remove unused code for legacy API v2 (fixes issue reported by the unused linter) 2023-03-26 14:28:15 +02:00
albertony
cc885bd39a hidrive: remove unused code (fixes issue reported by the unused linter) 2023-03-26 14:28:15 +02:00
albertony
a1a8c21c70 dropbox: remove unused code (fixes issue reported by the unused linter) 2023-03-26 14:28:15 +02:00
albertony
6ef4bd8c45 cache: remove unused code (fixes issue reported by the unused linter) 2023-03-26 14:28:15 +02:00
albertony
fb316123ec azureblob: remove unused code (fixes issue reported by the unused linter) 2023-03-26 14:28:15 +02:00
albertony
270af61665 smb: code cleanup to avoid overwriting ctx before first use (fixes issue reported by the staticcheck linter) 2023-03-26 14:28:15 +02:00
albertony
930574c6e9 oracleobjectstorage: remove empty branch (fixes issue reported by the staticcheck linter) 2023-03-26 14:28:15 +02:00
albertony
c1586a9866 onedrive: report any list errors during cleanup 2023-03-26 14:28:15 +02:00
albertony
b2ce7c9aa6 hidrive: error strings should not be capitalized 2023-03-26 14:28:15 +02:00
Dimitri Papadopoulos
55a12bd639 backend: fix repeated words typos 2023-03-25 09:31:36 +01:00
Dimitri Papadopoulos
9183618082 backend: fix typos found by codespell 2023-03-24 20:42:45 +00:00
Dimitri Papadopoulos
bfe272bf67 backend: fix typos found by codespell 2023-03-24 11:34:14 +00:00
Peter Brunner
c27e6a89b0
drive: add env_auth to drive provider
This change provides the ability to pass `env_auth` as a parameter to
the drive provider. This enables the provider to pull IAM
credentials from the environment or instance metadata. Previously if no
auth method was given it would default to requesting oauth.
2023-03-24 11:11:21 +00:00