Commit Graph

8494 Commits

Author SHA1 Message Date
c4451bc43a fs: add --metadata-set flag to specify metadata for uploads 2022-06-29 14:29:36 +01:00
0652ec95db fs: implement MetadataInfo to show info about metadata in help and rc
Info about this will appear in operations/fsinfo and in the backend
help (`rclone help backend s3`).
2022-06-29 14:29:36 +01:00
6a0e021dac fs: implement optional Metadata interface for Objects #111
This implements integration tests for the feature also.
2022-06-29 11:21:29 +01:00
461d041c4d fstest: remove spurious contents return from PutTestContents and friends 2022-06-29 11:18:02 +01:00
35f24d5b84 Add vyloy to contributors 2022-06-29 10:55:03 +01:00
370c8fa220 Add mirekphd to contributors 2022-06-29 10:55:03 +01:00
0fca4d2c86 Add buda to contributors 2022-06-29 10:55:03 +01:00
5de9278650 fs/cache: make sure we call the Shutdown method on backends
This change ensures we call the Shutdown method on backends when
they drop out of the fs/cache and at program exit.

Some backends implement the optional fs.Shutdowner interface. Until now,
Shutdown is only checked and called, when a backend is wrapped (e.g.
crypt, compress, ...).

To have a general way to perform operations at the end of the backend
lifecycle with proper error handling, we can call Shutdown at cache
clear time.

We add a finalize hook to the cache which will be called when values
drop out of the cache.

Previous discussion: https://forum.rclone.org/t/31336
2022-06-28 12:51:59 +01:00
326c43ab3f s3: add IDrive e2 to provider list 2022-06-28 09:12:36 +01:00
32006033e6 docs: note wider impact of --checkers=N on parallelism #6280 2022-06-27 19:33:07 +01:00
517e7d9271 accounting: fix unknown length file transfers count 3 transfers each #6213
This was caused by nested calls to NewTransfer/Done.

This fixes the problem by only incrementing transfers if the remote is
present in the transferMap which means we only increment it once.
2022-06-27 17:56:03 +01:00
fdd2f8e6d2 Error strings should not be capitalized
Reported by staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
2022-06-23 23:26:02 +02:00
027746ef6e drive: moved rclone_folder_id to advanced section - fixes #3463 2022-06-23 22:08:09 +02:00
53f831f40a docs: add missing code section formatting to commands and flags 2022-06-21 23:43:00 +02:00
9f81b4df4f docs/lsjson: fix code block indentation 2022-06-21 23:43:00 +02:00
bf54c909c9 docs: improved capitalization 2022-06-21 23:43:00 +02:00
dbf1234edf docs: skip "Connection" suffix from FTP, SSH/SFTP and HTTP backend names 2022-06-21 23:43:00 +02:00
70d9d75801 docs: improve serve command descriptions 2022-06-21 23:43:00 +02:00
bc70a95fca docs: consistent capitalization of WebDAV, DLNA, HTTP 2022-06-21 23:43:00 +02:00
ee87e919c5 docs/ncdu: note that refresh screen shortcut will fix screen corruption 2022-06-21 23:43:00 +02:00
4f0eae366f docs/ncdu: fix inconsistency in key map help text
Ctrl+L was listed as ^L, while Ctrl+c was listed as c-C. Changed the latter to ^c.
2022-06-21 23:43:00 +02:00
de5ccaab8e docs: cross link doc pages for related commands 2022-06-21 23:43:00 +02:00
4b7dc35cf4 Fix sync docs incorrect merge
This copies the changes from an autogenerated section in the following commit:
f2a15a174f
2022-06-21 23:43:00 +02:00
bc705e14d8 vfscache: fix fatal error: sync: unlock of unlocked mutex error
This message is a double panic and was actually caused by an assertion
panic in:

vfs/vfscache/downloaders/downloaders.go

This is triggered by the code added relatively recently to fix a bug
with renaming files:

ec72432cec vfs: fix failed to _ensure cache internal error: downloaders is nil error

So it appears that item.o may be nil at this point.

This patch detects item.o being nil and fetches it again with NewObject.

Fixes #6190 Fixes #6235
2022-06-21 14:28:53 +01:00
ea5bb79366 drive: document export for google apps scripts better
This also adds some more mime types from the code.

See: https://forum.rclone.org/t/how-do-i-copy-google-apps-scripts-with-rclone/31373
2022-06-21 14:28:53 +01:00
e95dff2fa1 drive: add backend commands exportformats and importformats for debugging 2022-06-21 14:28:53 +01:00
99dfe1eeae Add Martin Czygan to contributors 2022-06-21 14:28:53 +01:00
ed92bf335d Add J-P Treen to contributors 2022-06-21 14:28:53 +01:00
3d55b537c6 Add Caleb to contributors 2022-06-21 14:28:53 +01:00
d03fffdf8d Add eNV25 to contributors 2022-06-21 14:28:53 +01:00
7a909ebfb0 fs/cache: fix cache unpin 2022-06-20 12:14:58 +01:00
ac0dc9922e copyurl: add tests for the option to honor the HTTP header filename directive 2022-06-20 12:06:09 +01:00
8b8802a078 copyurl: Add option to honor the HTTP header filename directive.
Implemented --header-filename for use with copyurl.

For specifically setting preferred download filenames for HTTP requests, RFC 6226
specifies a 'filename' directive, available within 'Content-Disposition'
header. We can handle with 'mime.ParseMediaType'.

See below for details:
https://httpwg.org/specs/rfc6266.html#disposition.parameter.filename
https://httpwg.org/specs/rfc6266.html#advice.generating

Co-authored-by: buengese <buengese@protonmail.com>
2022-06-20 12:06:09 +01:00
f2a15a174f docs: grammatical clarification in sync docs 2022-06-19 15:26:53 +02:00
21c746a56c fichier: parse api error codes and them accordingly 2022-06-19 15:07:33 +02:00
36add0afbf ncdu: replace termbox with tcell's termbox wrapper
The https://github.com/nsf/termbox-go library is no longer maintained
so this change replaces it with the maintained
github.com/gdamore/tcell library which has a termbox backwards
compatibility layer.

There are a few minor changes from the termbox library:

- Using Clear with fg bg ColorDefault resulted in a white background for some reason.
    - Clear with fg ColorWhite bg ColorBlack was used instead.
- tcell's termbox wrapper doesn't support ColorLightYellow.
    - ColorYellow + 8 was used instead.
2022-06-19 11:22:45 +01:00
14e0396fcb test_all: allow internet archive backend more time 2022-06-18 15:13:18 +01:00
100acc570a test_all: fix -clean so it works on remotes with paths 2022-06-18 15:10:09 +01:00
b9de37af80 test_all: Only run backend tests for Internet Archive as it is too slow 2022-06-17 16:52:30 +01:00
f7c36ce0f9 s3: unwrap SDK errors to reveal underlying errors on upload
The SDK doesn't wrap errors in a Go standard way so they can't be
unwrapped and tested for - eg fatal error.

The code looks for a Serialization or RequestError and returns the
unwrapped underlying error if possible.

This fixes the fs/operations integration tests checking for fatal
errors being returned.
2022-06-17 16:52:30 +01:00
f829ded456 Add Phil Shackleton to contributors 2022-06-17 16:52:30 +01:00
2fac8fdde6 Add Scott Grimes to contributors 2022-06-17 16:52:30 +01:00
8e2d9a4cb9 drive: update Internal OAuth consent screen docs
Updated instructions in `Making your own client_id` section to record process for selecting "Internal" OAuth consent screen.
2022-06-17 16:50:01 +01:00
295006f662 opendrive: resolve lag and truncate bugs fixes #5936
Co-authored-by: buengese <buengese@protonmail.com>
2022-06-17 16:48:03 +01:00
dcc128c70d docs/onedrive: document creation of client ID for OneDrive Business 2022-06-17 07:33:56 +02:00
c85fbebce6 s3: simplify PutObject code to use the Request.SetStreamingBody method
In this commit

e5974ac4b0 s3: use PutObject from the aws SDK to upload single part objects

rclone was made to upload objects to s3 using PUT requests rather than
using signed uploads.

However this change missed the fact that there is a supported way to
do this in the SDK using the SetStreamingBody method on the Request.

This therefore reverts a lot of the previous commit to do with making
an unsigned connection and other complication and uses the SDK
facility.
2022-06-16 23:26:19 +01:00
e59801c69b Add Maciej Radzikowski to contributors 2022-06-16 23:26:19 +01:00
5697dbc80f local: fix parsing of --local-nounc flag 2022-06-16 22:13:50 +01:00
7d3648dc46 serve ftp: check --passive-port arguments are correct
See: https://forum.rclone.org/t/serve-ftp-passive-port-validity-check/27458
2022-06-16 22:13:50 +01:00
a6ca4b3817 test info: check file name lengths using 1,2,3,4 byte unicode characters 2022-06-16 22:13:50 +01:00