buengese
a2a0388036
zoho: make upload cutoff configurable
2024-09-17 20:40:42 +01:00
buengese
48543d38e8
zoho: add support for private spaces
2024-09-17 20:40:42 +01:00
buengese
eceb390152
zoho: try to handle rate limits a bit better
2024-09-17 20:40:42 +01:00
buengese
f4deffdc96
zoho: print clear error message when missing oauth scope
2024-09-17 20:40:42 +01:00
buengese
c172742cef
zoho: switch to large file upload API for larger files, fix missing URL encoding of filenames for the upload API
2024-09-17 20:40:42 +01:00
buengese
7daed30754
zoho: use download server to accelerate downloads
...
Co-authored-by: rishi.sridhar <rishi.sridhar@zohocorp.com>
2024-09-17 20:40:42 +01:00
Nick Craig-Wood
802a938bd1
zoho: fix inefficiencies uploading with new API to avoid throttling
...
Before this fix, rclone queried the uploaded object to find its size
and modtime after upload as the API did not return these items.
Zoho have subsequently modified the API to return these items so
rclone uses them to avoid an API call.
This should help with rclone being throttled by Zoho.
See: https://forum.rclone.org/t/second-followup-on-the-older-topic-rclone-invokes-more-number-of-workdrive-s-files-listing-api-calls-which-exceeds-the-throttling-limit/45697/20
2024-09-04 10:45:47 +01:00
Nick Craig-Wood
3fef8016b5
zoho: sleep for 60 seconds if rate limit error received
2024-06-12 16:34:30 +01:00
Nick Craig-Wood
edf6537c61
zoho: remove simple file names complication which is no longer needed
2024-06-12 16:34:27 +01:00
Nick Craig-Wood
00f0e9df9d
zoho: retry reading info if size wasn't returned
2024-06-12 16:34:24 +01:00
Nick Craig-Wood
e6ab644350
zoho: fix throttling problem when uploading files
...
Before this change rclone checked to see if a file existed before
uploading it. It did this to avoid making duplicate files. This
involved listing the destination directory to see if the file existed
which was rate limited by Zoho.
However Zoho can't have duplicate files anyway so this fix just
removes that check and the PutUnchecked method which isn't needed.
See: https://forum.rclone.org/t/second-followup-on-the-older-topic-rclone-invokes-more-number-of-workdrive-s-files-listing-api-calls-which-exceeds-the-throttling-limit/45697
See: https://forum.rclone.org/t/followup-on-the-older-topic-rclone-invokes-more-number-of-workdrive-s-files-listing-api-calls-which-exceeds-the-throttling-limit/44794
2024-06-12 16:34:18 +01:00
Nick Craig-Wood
61c18e3b60
zoho: use cursor listing for improved performance
...
Cursor listing enables us to list up to 1,000 items per call
(previously it was 10) and uses one less transaction per call.
See: https://forum.rclone.org/t/second-followup-on-the-older-topic-rclone-invokes-more-number-of-workdrive-s-files-listing-api-calls-which-exceeds-the-throttling-limit/45697/4
2024-06-12 16:34:11 +01:00
Nick Craig-Wood
b750c50bfd
zoho: remove Range requests workarounds to fix integration tests
...
Zoho are now responding to Range requests properly. The remnants of
our old workaround was breaking the integration tests so this removes
them.
2023-09-05 18:21:15 +01:00
Nick Craig-Wood
4f8dab8bce
zoho: fix downloads with Range: header returning the wrong data
...
Zoho has started returning the results from Range: requests with a 200
response code rather than the technically correct 206 error code.
Before this change this triggered workaround code to deal with Zoho
not obeying Range: requests properly.
This fix tests the returned header for a Content-Range: header and if
it exists assumes it is a valid reply to the Range: request despite
the status being 200.
This problem was spotted by the integration tests.
2023-06-14 17:43:26 +01:00
albertony
5f3c276d0a
zoho: remove unused code (fixes issue reported by the unused linter)
2023-03-26 14:28:15 +02:00
albertony
5d6b8141ec
Replace deprecated ioutil
...
As of Go 1.16, the same functionality is now provided by package io or
package os, and those implementations should be preferred in new code.
2022-11-07 11:41:47 +00:00
Josh Soref
ce3b65e6dc
all: fix spelling across the project
...
* abcdefghijklmnopqrstuvwxyz
* accounting
* additional
* allowed
* almost
* already
* appropriately
* arise
* bandwidth
* behave
* bidirectional
* brackets
* cached
* characters
* cloud
* committing
* concatenating
* configured
* constructs
* current
* cutoff
* deferred
* different
* directory
* disposition
* dropbox
* either way
* error
* excess
* experiments
* explicitly
* externally
* files
* github
* gzipped
* hierarchies
* huffman
* hyphen
* implicitly
* independent
* insensitive
* integrity
* libraries
* literally
* metadata
* mimics
* missing
* modification
* multipart
* multiple
* nightmare
* nonexistent
* number
* obscure
* ourselves
* overridden
* potatoes
* preexisting
* priority
* received
* remote
* replacement
* represents
* reproducibility
* response
* satisfies
* sensitive
* separately
* separator
* specifying
* string
* successful
* synchronization
* syncing
* šenfeld
* take
* temporarily
* testcontents
* that
* the
* themselves
* throttling
* timeout
* transaction
* transferred
* unnecessary
* using
* webbrowser
* which
* with
* workspace
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-08-30 11:16:26 +02:00
YFdyh000
b5818454f7
onedrive: cleanup brand name
2022-08-30 10:23:29 +02:00
Nick Craig-Wood
6fd9e3d717
build: reformat comments to pass go1.19 vet
...
See: https://go.dev/doc/go1.19#go-doc
2022-08-05 16:35:41 +01:00
albertony
ec117593f1
Fix lint issues reported by staticcheck
...
Used staticcheck 2022.1.2 (v0.3.2)
See: staticcheck.io
2022-06-13 21:13:50 +02:00
buengese
cee79f27ee
zoho: add Japan and China regions
2022-06-12 15:37:30 +02:00
Nick Craig-Wood
e43b5ce5e5
Remove github.com/pkg/errors and replace with std library version
...
This is possible now that we no longer support go1.12 and brings
rclone into line with standard practices in the Go world.
This also removes errors.New and errors.Errorf from lib/errors and
prefers the stdlib errors package over lib/errors.
2021-11-07 11:53:30 +00:00
Nick Craig-Wood
3fbaa4c0b0
backends: make NewObject return fs.ErrorIsDir if possible
...
This changes the interface to NewObject so that if NewObject is called
on a directory then it should return fs.ErrorIsDir if possible without
doing any extra work, otherwise fs.ErrorObjectNotFound.
Tested on integration test server with:
go run integration-test.go -tests backend -run TestIntegration/FsMkdir/FsPutFiles/FsNewObjectDir -branch fix-stat -maxtries 1
2021-10-14 17:15:50 +01:00
Nick Craig-Wood
84201ed891
zoho: improve wording for region - fixes #5377
2021-06-11 14:21:23 +01:00
Nick Craig-Wood
f122808d86
fs: add names to each config parameter so we can override them #3455
2021-05-14 14:07:44 +01:00
Nick Craig-Wood
94dbfa4ea6
fs: change Config callback into state based callback #3455
...
This is a very large change which turns the post Config function in
backends into a state based call and response system so that
alternative user interfaces can be added.
The existing config logic has been converted, but it is quite
complicated and folloup commits will likely be needed to fix it!
Follow up commits will add a command line and API based way of using
this configuration system.
2021-05-14 14:07:44 +01:00
Nick Craig-Wood
b78c9a65fa
backends: remove log.Fatal and replace with error returns #5234
...
This changes the Config interface so that it returns an error.
2021-04-27 18:18:08 +01:00
buengese
c114695a66
zoho: do not ask for mountpoint twice when using headless setup
2021-04-08 00:23:27 +02:00
buengese
0caf417779
zoho: fix error when region isn't set
2021-04-05 15:11:30 +02:00
Nick Craig-Wood
fc57648b75
lib/rest: fix multipart uploads stopping on context cancel
...
Before this change when the context was cancelled (due to
--max-duration for example) this could deadlock when uploading
multipart uploads.
This change fixes the problem by introducing another go routine to
monitor the context and close the pipe with an error when the context
errors.
2021-03-29 19:09:47 +01:00
buengese
da5b0cb611
zoho: add forgotten setupRegion() to NewFs
...
- this finally fixes regions other than eu
2021-03-21 02:15:22 +01:00
buengese
0187bc494a
zoho: replace client id
2021-03-21 02:15:22 +01:00
Nick Craig-Wood
4013bc4a4c
Fix excessive retries missing --max-duration timeout - fixes #4504
...
This change checks the context whenever rclone might retry, and
doesn't retry if the current context has an error.
This fixes the pathological behaviour of `--max-duration` refusing to
exit because all the context deadline exceeded errors were being
retried.
This unfortunately meant changing the shouldRetry logic in every
backend and doing a lot of context propagation.
See: https://forum.rclone.org/t/add-flag-to-exit-immediately-when-max-duration-reached/22723
2021-03-13 09:25:44 +00:00
buengese
333faa6c68
zoho: fix custom client id's
2021-02-23 11:27:05 +00:00
buengese
66c3f2f31f
new backend: zoho workdrive - fixes #4533
2020-12-30 17:56:08 +00:00
