Commit Graph

1210 Commits

Author SHA1 Message Date
Nick Craig-Wood
bb0c4ad2d8 union: fix rename not working with union of local disk and bucket based remote
Before this change the union's feature flags were a strict AND of the
underlying remotes. This means that a union of a local disk (which can
Move but not Copy) and a bucket based remote (which can Copy but not
Move) could neither Move nor Copy.

This fix advertises Move in the union if all the remotes can Move or
Copy. It also implements Move as Copy+Delete (like rclone does
normally) if the underlying union does not support Move.

This enables renames to work with unions of local disk and bucket
based remotes expected.

Fixes #5632
2021-09-30 20:09:02 +01:00
albertony
b389b84685 jottacloud: refactor all file state checks into common functions 2021-09-30 19:34:48 +02:00
albertony
b0f06d9920 jottacloud: improved error handling with SetModTime and corrupt files in general 2021-09-30 19:34:48 +02:00
albertony
159229527d jottacloud: implement SetModTime to support modtime-only changes - #5627 2021-09-30 19:34:48 +02:00
Jonta
d8d621c175
docs: grammar/readability (#5633) 2021-09-30 15:34:00 +02:00
Fred
00c6642fad seafile: fix 2fa state machine 2021-09-18 12:44:59 +01:00
Nick Craig-Wood
badefdb060 pcloud: try harder to delete a failed upload
This fixes the integration tests when testing errored uploads
2021-09-17 10:43:45 +01:00
Nick Craig-Wood
9c2533821d pcloud: return an early error when Put is called with an unknown size
This stops the 10 minute pause in the integration tests
2021-09-17 10:43:45 +01:00
Nick Craig-Wood
c718fe4330 pcloud: fix sha256 hashes #5496
This was started in

3626f10f26 pcloud: add sha256 support - fixes #5496

But this support turned out to be incomplete and caused the
integration tests to fail.
2021-09-17 10:43:45 +01:00
Ivan Andreev
24de896df2 build: apply gofmt from golang 1.17 2021-09-09 20:43:59 +03:00
Tatsuya Noyori
05f128868f
azureblob: add --azureblob-no-head-object 2021-09-06 10:41:54 +01:00
albertony
f1f923a986 Change byte unit format from MiByte to MiB 2021-08-31 09:57:27 +02:00
Nick Craig-Wood
6a9ef27b09 cache: don't run failing tests on windows/386
After updating rclone's dependencies these tests started failing on
windows/386

- TestInternalDoubleWrittenContentMatches
- TestInternalMaxChunkSizeRespected

The failures look like this. The root cause is unknown. The `Wait(n=1)
would exceed context deadline` errors come from golang.org/x/time/rate
but it isn't clear what is calling them.

2021/08/20 21:57:16 ERROR : worker-0 <one>: object open failed 0: rate: Wait(n=1) would exceed context deadline
[snip ~10 duplicates]
2021/08/20 21:57:56 ERROR : tidwcm1629496636/one: (0/26) error (chunk not found 0) response
2021/08/20 21:58:02 ERROR : worker-0 <one>: object open failed 0: rate: Wait(n=1) would exceed context deadline
--- FAIL: TestInternalDoubleWrittenContentMatches (45.77s)
    cache_internal_test.go:310:
        	Error Trace:	cache_internal_test.go:310
        	Error:      	Not equal:
        	            	expected: "one content updated double"
        	            	actual  : ""

        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-one content updated double
        	            	+
        	Test:       	TestInternalDoubleWrittenContentMatches
2021/08/20 21:58:03 original size: 23592960
2021/08/20 21:58:03 updated size: 12
2021-08-20 23:28:18 +01:00
Nick Craig-Wood
2cefae51a1 build: make go1.14 the minimum supported Go for building 2021-08-20 19:04:29 +01:00
albertony
9a2811f0b2 local: refactor default os encoding out from local backend into shared encoder lib 2021-08-19 20:14:50 +02:00
Parth Shukla
60323dc5e2 googlephotos: Use encoder for album names 2021-08-19 16:38:31 +01:00
Nick Craig-Wood
890b6a45b5 sugarsync: fix initial connection after config re-arrangement - Fixes #5525
In this commit the config system was re-arranged

    94dbfa4ea fs: change Config callback into state based callback #3455

This passed the password as a temporary config parameter but forgot to
reveal it in the API call.
2021-08-14 12:53:36 +01:00
Ken Enrique Morel
3626f10f26 pcloud: add sha256 support - fixes #5496 2021-08-14 12:48:25 +01:00
hota
839c20bb35 s3: add Wasabi's AP-Northeast endpoint info
* Wasabi starts to provide AP Northeast (Tokyo) endpoint for all customers, so add it to the list

Signed-off-by: lindwurm <lindwurm.q@gmail.com>
2021-08-01 14:56:52 +01:00
Nick Craig-Wood
6545755758 sftp: remove spurious error message on --sftp-disable-concurrent-reads 2021-07-31 11:04:45 +01:00
Alex Chen
63a24255f8
onedrive: handle HTTP 400 better in PublicLink() (#5419) 2021-07-27 17:55:57 +08:00
Nick Craig-Wood
76f8095bc5 hdfs: fix documentation so it doesn't have HTML in 2021-07-20 19:36:30 +01:00
Nick Craig-Wood
d38f6bb0ab gphotos: fix read only scope not being used properly
Before this change the read only scope was being ignored and rclone
was asking for a read-write scope.

https://forum.rclone.org/t/google-photos-copy-sync-errors/25153
2021-07-20 16:57:55 +01:00
Nick Craig-Wood
807102ada2 drive: fix config system overwriting team drive ID - fixes #5454 2021-07-20 16:51:59 +01:00
buengese
8652cfe575 jottacloud: add no versions option 2021-07-15 17:29:30 +02:00
Nick Craig-Wood
94b1439299 drive: fix some google docs being treated as files - fixes #5455
At some point some google docs files started having sizes returned in
their listing information.

This then caused rclone to treat the docs as files which caused
downloads to fail.

The API docs now state that google docs may have sizes (whereas I'm
pretty sure it didn't earlier).

This fix removes the check for size, so google docs are identified
solely by not having an MD5 checksum.
2021-07-14 11:40:58 +01:00
Chuan Zh
ba836d45ff
s3: update Alibaba OSS endpoints 2021-07-08 12:03:04 +01:00
Ivan Andreev
4680c0776d
backend/local: skip entries removed concurrently with List() (#5297)
This change fixes the bug described below:
if a file is removed while the local backend List() runs,
the call will flag an accounting error.
The bug manifests itself if local backend is the Sync target
due to intrinsic concurrency.
The odds to hit this bug depend on --checkers and --transfers.
Chunker over local backend is affected even more because
updating a composite object with a smaller size content
translates into removing chunks on the underlying file system
and involves a number of List() calls.
2021-07-07 16:50:19 +03:00
buengese
fb305b5976 fichier: check that we actually got a download token and retry if we didn't 2021-07-06 14:58:50 +02:00
Dmitry Sitnikov
b439199578
azureblob: Fix typo in Azure Blob help
Change the command to create RBAC file to the correct one
`az ad sp create-for-rbac`
Add the link to the command documentation
https://docs.microsoft.com/en-us/cli/azure/ad/sp?view=azure-cli-latest#az_ad_sp_create_for_rbac
2021-07-05 15:58:41 +01:00
Nick Craig-Wood
76ea716abf ftp: make upload error 250 indicate success
Some servers seem to send return code 250 to indicate successful
upload - previously rclone was treating this as an error.

See: https://forum.rclone.org/t/transfer-on-mega-in-ftp-mode-is-not-working/24642/
2021-07-05 10:35:02 +01:00
Xuanchen Wu
0cb973f127
onedrive: Make link return direct download link (#5417)
Co-authored-by: Cnly <minecnly@gmail.com>
2021-06-22 21:25:08 +08:00
Nick Craig-Wood
3349b055f5 fichier: fix move of files in the same directory
See: https://forum.rclone.org/t/1fichier-rclone-does-not-allow-to-rename-files-and-folders-when-you-mount-a-1fichier-disk-drive/24726/24
2021-06-11 14:21:23 +01:00
Nick Craig-Wood
bef0c23e00 fichier: make error messages report text from the API
See: https://forum.rclone.org/t/1fichier-rclone-does-not-allow-to-rename-files-and-folders-when-you-mount-a-1fichier-disk-drive/24726/24
2021-06-11 14:21:23 +01:00
Nick Craig-Wood
84201ed891 zoho: improve wording for region - fixes #5377 2021-06-11 14:21:23 +01:00
Florian Penzkofer
92b3518c78 fichier: support downloading password protected files and folders 2021-06-10 19:00:26 +02:00
Ivan Andreev
062919e08c
deprecate cache backend (#5382) 2021-06-10 19:52:55 +03:00
Chris Lu
1f846c18d4
s3: Add SeaweedFS 2021-06-08 09:59:57 +01:00
albertony
41f561bf26 jottacloud: fix invalid url in output from link command
Fixes #5370
2021-05-31 10:40:21 +02:00
database64128
df60e6323c 🧹 GCS: Clean up time format constants 2021-05-28 14:44:50 +01:00
database64128
ee2fac1855 🕰️ GCS: Compatible with gsutil's mtime metadata
- Write `goog-reserved-file-mtime` in addition to `mtime`.
- Fallback to `goog-reserved-file-mtime` if `mtime` doesn't exist.
- ref rclone/rclone#5331
2021-05-28 14:44:50 +01:00
Ivan Andreev
5b6f637461
fs/hash: align hashsum names and update documentation (#5339)
- Unify all hash names as lowercase alphanumerics without punctuation.
- Legacy names continue to work but disappear from docs, they can be depreciated or dropped later.
- Make rclone hashsum print supported hash list in case of wrong spelling.
- Update documentation.

Fixes #5071
Fixes #4841
2021-05-21 17:32:33 +03:00
Nick Craig-Wood
1b44035e45 filefabric: fix listing after change of from field from "int" to int. 2021-05-18 17:11:16 +01:00
Nick Craig-Wood
c0cda087a8 s3: don't check to see if remote is object if it ends with /
Before this change, rclone would always check the root to see if it
was an object.

This change doesn't check to see if the root is an object if the path
ends with a /

This avoids a transaction where rclone HEADs the path to see if it
exists.

See #4990
2021-05-17 16:43:34 +01:00
Nick Craig-Wood
04308dcaa1 local: add --local-unicode-normalization (and remove --local-no-unicode-normalization)
macOS stores files in NFD form and transferring them like this to some
systems causes the Korean language to display incorrectly.

This adds the flag --local-unicode-normalization to optionally
normalize the file names to NFC.

This also removes the (long deprecated) --local-no-unicode-normalization flag

See: https://forum.rclone.org/t/support-for-korean-jaso-conversion/19435
2021-05-17 16:34:25 +01:00
Nick Craig-Wood
6555d3eb33 onedrive: fix failed to configure: empty token found error #3455
This bug was caused as part of the config rework
2021-05-17 12:10:58 +01:00
Alexey Ivanov
2fa7a3c0fb dropbox: simplify chunked uploads
Signed-off-by: Alexey Ivanov <rbtz@dropbox.com>
2021-05-14 14:07:44 +01:00
Nick Craig-Wood
75c417ad93 dropbox: fix async batch missing the last few entries 2021-05-14 14:07:44 +01:00
Nick Craig-Wood
5ee646f264 dropbox: make batcher retry all errors so it doesn't exit early
See: https://forum.rclone.org/t/dropbox-too-many-requests-or-write-operations-trying-again-in-15-seconds/23316/18
2021-05-14 14:07:44 +01:00
Nick Craig-Wood
4a4aca4da7 dropbox: fix deadlock in batch Commit 2021-05-14 14:07:44 +01:00