Commit Graph

8686 Commits

Author SHA1 Message Date
326c43ab3f s3: add IDrive e2 to provider list 2022-06-28 09:12:36 +01:00
32006033e6 docs: note wider impact of --checkers=N on parallelism #6280 2022-06-27 19:33:07 +01:00
517e7d9271 accounting: fix unknown length file transfers count 3 transfers each #6213
This was caused by nested calls to NewTransfer/Done.

This fixes the problem by only incrementing transfers if the remote is
present in the transferMap which means we only increment it once.
2022-06-27 17:56:03 +01:00
fdd2f8e6d2 Error strings should not be capitalized
Reported by staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
2022-06-23 23:26:02 +02:00
027746ef6e drive: moved rclone_folder_id to advanced section - fixes #3463 2022-06-23 22:08:09 +02:00
53f831f40a docs: add missing code section formatting to commands and flags 2022-06-21 23:43:00 +02:00
9f81b4df4f docs/lsjson: fix code block indentation 2022-06-21 23:43:00 +02:00
bf54c909c9 docs: improved capitalization 2022-06-21 23:43:00 +02:00
dbf1234edf docs: skip "Connection" suffix from FTP, SSH/SFTP and HTTP backend names 2022-06-21 23:43:00 +02:00
70d9d75801 docs: improve serve command descriptions 2022-06-21 23:43:00 +02:00
bc70a95fca docs: consistent capitalization of WebDAV, DLNA, HTTP 2022-06-21 23:43:00 +02:00
ee87e919c5 docs/ncdu: note that refresh screen shortcut will fix screen corruption 2022-06-21 23:43:00 +02:00
4f0eae366f docs/ncdu: fix inconsistency in key map help text
Ctrl+L was listed as ^L, while Ctrl+c was listed as c-C. Changed the latter to ^c.
2022-06-21 23:43:00 +02:00
de5ccaab8e docs: cross link doc pages for related commands 2022-06-21 23:43:00 +02:00
4b7dc35cf4 Fix sync docs incorrect merge
This copies the changes from an autogenerated section in the following commit:
f2a15a174f
2022-06-21 23:43:00 +02:00
bc705e14d8 vfscache: fix fatal error: sync: unlock of unlocked mutex error
This message is a double panic and was actually caused by an assertion
panic in:

vfs/vfscache/downloaders/downloaders.go

This is triggered by the code added relatively recently to fix a bug
with renaming files:

ec72432cec vfs: fix failed to _ensure cache internal error: downloaders is nil error

So it appears that item.o may be nil at this point.

This patch detects item.o being nil and fetches it again with NewObject.

Fixes #6190 Fixes #6235
2022-06-21 14:28:53 +01:00
ea5bb79366 drive: document export for google apps scripts better
This also adds some more mime types from the code.

See: https://forum.rclone.org/t/how-do-i-copy-google-apps-scripts-with-rclone/31373
2022-06-21 14:28:53 +01:00
e95dff2fa1 drive: add backend commands exportformats and importformats for debugging 2022-06-21 14:28:53 +01:00
99dfe1eeae Add Martin Czygan to contributors 2022-06-21 14:28:53 +01:00
ed92bf335d Add J-P Treen to contributors 2022-06-21 14:28:53 +01:00
3d55b537c6 Add Caleb to contributors 2022-06-21 14:28:53 +01:00
d03fffdf8d Add eNV25 to contributors 2022-06-21 14:28:53 +01:00
7a909ebfb0 fs/cache: fix cache unpin 2022-06-20 12:14:58 +01:00
ac0dc9922e copyurl: add tests for the option to honor the HTTP header filename directive 2022-06-20 12:06:09 +01:00
8b8802a078 copyurl: Add option to honor the HTTP header filename directive.
Implemented --header-filename for use with copyurl.

For specifically setting preferred download filenames for HTTP requests, RFC 6226
specifies a 'filename' directive, available within 'Content-Disposition'
header. We can handle with 'mime.ParseMediaType'.

See below for details:
https://httpwg.org/specs/rfc6266.html#disposition.parameter.filename
https://httpwg.org/specs/rfc6266.html#advice.generating

Co-authored-by: buengese <buengese@protonmail.com>
2022-06-20 12:06:09 +01:00
f2a15a174f docs: grammatical clarification in sync docs 2022-06-19 15:26:53 +02:00
21c746a56c fichier: parse api error codes and them accordingly 2022-06-19 15:07:33 +02:00
36add0afbf ncdu: replace termbox with tcell's termbox wrapper
The https://github.com/nsf/termbox-go library is no longer maintained
so this change replaces it with the maintained
github.com/gdamore/tcell library which has a termbox backwards
compatibility layer.

There are a few minor changes from the termbox library:

- Using Clear with fg bg ColorDefault resulted in a white background for some reason.
    - Clear with fg ColorWhite bg ColorBlack was used instead.
- tcell's termbox wrapper doesn't support ColorLightYellow.
    - ColorYellow + 8 was used instead.
2022-06-19 11:22:45 +01:00
14e0396fcb test_all: allow internet archive backend more time 2022-06-18 15:13:18 +01:00
100acc570a test_all: fix -clean so it works on remotes with paths 2022-06-18 15:10:09 +01:00
b9de37af80 test_all: Only run backend tests for Internet Archive as it is too slow 2022-06-17 16:52:30 +01:00
f7c36ce0f9 s3: unwrap SDK errors to reveal underlying errors on upload
The SDK doesn't wrap errors in a Go standard way so they can't be
unwrapped and tested for - eg fatal error.

The code looks for a Serialization or RequestError and returns the
unwrapped underlying error if possible.

This fixes the fs/operations integration tests checking for fatal
errors being returned.
2022-06-17 16:52:30 +01:00
f829ded456 Add Phil Shackleton to contributors 2022-06-17 16:52:30 +01:00
2fac8fdde6 Add Scott Grimes to contributors 2022-06-17 16:52:30 +01:00
8e2d9a4cb9 drive: update Internal OAuth consent screen docs
Updated instructions in `Making your own client_id` section to record process for selecting "Internal" OAuth consent screen.
2022-06-17 16:50:01 +01:00
295006f662 opendrive: resolve lag and truncate bugs fixes #5936
Co-authored-by: buengese <buengese@protonmail.com>
2022-06-17 16:48:03 +01:00
dcc128c70d docs/onedrive: document creation of client ID for OneDrive Business 2022-06-17 07:33:56 +02:00
c85fbebce6 s3: simplify PutObject code to use the Request.SetStreamingBody method
In this commit

e5974ac4b0 s3: use PutObject from the aws SDK to upload single part objects

rclone was made to upload objects to s3 using PUT requests rather than
using signed uploads.

However this change missed the fact that there is a supported way to
do this in the SDK using the SetStreamingBody method on the Request.

This therefore reverts a lot of the previous commit to do with making
an unsigned connection and other complication and uses the SDK
facility.
2022-06-16 23:26:19 +01:00
e59801c69b Add Maciej Radzikowski to contributors 2022-06-16 23:26:19 +01:00
5697dbc80f local: fix parsing of --local-nounc flag 2022-06-16 22:13:50 +01:00
7d3648dc46 serve ftp: check --passive-port arguments are correct
See: https://forum.rclone.org/t/serve-ftp-passive-port-validity-check/27458
2022-06-16 22:13:50 +01:00
a6ca4b3817 test info: check file name lengths using 1,2,3,4 byte unicode characters 2022-06-16 22:13:50 +01:00
e57fe14b61 mount: log IO errors at ERROR level - fixes #6217 2022-06-16 22:13:50 +01:00
115f1c2cc9 operations: speed up hash checking by aborting the other hash if first returns nothing
This speeds up hash checks when a Hash() function returns "" - this
means that the hash can be canceled for the other side.

In the common case of local hash vs remote hash empty this saves a lot
of time.

See: https://forum.rclone.org/t/rclone-s3-backend-copy-is-2x-slower-than-aws-s3-cp/27321/9
2022-06-16 22:13:50 +01:00
5e4caa69ce local: make Hash function cancelable via context 2022-06-16 22:13:50 +01:00
e7483b40b3 fshttp: add --disable-http-keep-alives to disable HTTP Keep Alives
See: https://forum.rclone.org/t/getting-rate-limited-before-advertised-limit-on-s3-compatible-object-storage/31010/
2022-06-16 22:13:50 +01:00
fa48b880c2 s3: retry RequestTimeout errors
See: https://forum.rclone.org/t/s3-failed-upload-large-files-bad-request-400/27695
2022-06-16 22:13:50 +01:00
4ac875a811 sync: fix --max-duration and --cutoff-mode soft
Before this change using --max-duration and --cutoff-mode soft would
work like --cutoff-mode hard.

This bug was introduced in this commit which made transfers be
cancelable - before that transfers couldn't be canceled.

122a47fba6 accounting: Allow transfers to be canceled with context #3257

This change adds the timeout to the input context for reading files
rather than the transfer context so the files transfers themselves
aren't canceled if --cutoff-mode soft is in action.

This also adds a test for cutoff mode soft and max duration which was
missing.

See: https://forum.rclone.org/t/max-duration-and-retries-not-working-correctly/27738
2022-06-16 22:13:50 +01:00
3f61869179 cmount: add tracing for *xattr FUSE callbacks
See: https://github.com/winfsp/cgofuse/issues/66
See: https://forum.rclone.org/t/cannot-copy-files-to-mounted-azure-storage-windows/30092
2022-06-16 22:13:50 +01:00
60d87185e1 sftp: add --sftp-set-env option to set environment variables
Fixes #6094
2022-06-16 22:13:50 +01:00