Commit Graph

1577 Commits

Author SHA1 Message Date
8913679d88 accounting: Fix elapsed time not show actual time since beginning
This fixes the elapsed time display in the statistics output in the rc and in the log messages.
2020-07-26 11:59:50 +01:00
aa93b39d9b build: fix tests on go1.15
go1.15 introduced a stricter policy for what you can convert with
`string()` and now `go vet` warns if you try to do `string(int)`.

See: https://github.com/golang/go/issues/32479
2020-07-25 18:51:28 +01:00
744828a4de rc: allow JSON parameters to simplify command line usage
If the parameter being passed is an object then it can be passed as a
JSON string rather than using the `--json` flag which simplifies the
command line.

rclone rc operations/list fs=/tmp remote=test opt='{"showHash": true}'

Rather than

rclone rc operations/list --json '{"fs": "/tmp", "remote": "test", "opt": {"showHash": true}}'
2020-07-22 18:40:52 +01:00
ff84351655 operations: factor Check and related functions into its own files 2020-07-21 22:08:13 +01:00
8b6f2bbb4b check,cryptcheck: add reporting of filenames for same/missing/changed #3264
See: https://forum.rclone.org/t/rclone-check-v-doesnt-show-once-per-minute-update-counts/17402
2020-07-21 22:08:13 +01:00
811b30d116 sync: fix deadlock with --track-renames-strategy modtime - fixes #4427
Before this change we could exit the popRenameMap function with the
lock held.

This fixes the problem by defer-ring the unlock.

See: https://forum.rclone.org/t/track-renames-strategy-modtime-doesnt-work/16992
2020-07-17 17:09:58 +01:00
bcd362fcd5 accounting: fix documentation for speed/speedAvg
Fix the documentation for the very confusingly names speed and
speedAvg stats items.

See: https://github.com/rclone/rclone-webui-react/issues/99
2020-07-17 15:35:15 +01:00
bfa5715017 fs/accounting: sort transfers by start time 2020-07-07 15:42:06 +01:00
e2183ad661 fs/accounting: use transferMap instead of stringSet 2020-07-07 15:42:06 +01:00
8d5bc7f28b fs/cache: fix moveto/copyto remote:file remote:file2
Before this change, if the cache was given a source `remote:file` it
stored `remote:` with the error `fs.ErrorIsFile` attached. This meant
that if it `remote:` was subsequently looked up it would return the
`fs.ErrorIsFile` error.

This broke `moveto remote:file remote:file2` as moveto would lookup
`remote:` from the second argument and erroneously get the
`fs.ErrorIsFile` error.

This likely broke other commands too.

This was broken in

4c9836035 fs/cache: Add Pin and Unpin and canonicalised lookup

Which was released in v1.52.0

The fix is to make a new cache entry for `remote:` with no error
attached in the case that the original call returned `fs.ErrorIsFile`.
2020-07-02 10:55:36 +01:00
99c293a403 log: fix --use-json-log going to stderr not --log-file on Windows - fixes #4367 2020-07-01 20:47:37 +01:00
05ddef117a accounting: add HasBuffer method to Account 2020-06-30 12:03:39 +01:00
ae8bbc63da cache: export Canonicalize method for external use 2020-06-30 12:01:36 +01:00
79f5d940cf fs: add Fingerprint to detect changes in an object 2020-06-30 12:01:36 +01:00
c820576329 fs: define SlowModTime and SlowHash features in the relevant backends 2020-06-30 12:01:36 +01:00
a2dd23efd3 rc: Add tests for operations/uploadfile
rc: Go import file rc_test.go
2020-06-25 11:38:24 +01:00
fa43d02874 rc: Add operations/uploadfile to upload a file through rc using encoding multipart/form-data 2020-06-25 11:38:24 +01:00
d0de39ebcd rc: add NeedsRequest to call. 2020-06-25 11:38:24 +01:00
3e82771413 Start v1.52.2-DEV development 2020-06-24 14:35:12 +01:00
ce394426b0 check: fix successful retries with --download counting errors
See: https://forum.rclone.org/t/tons-of-data-corruption-after-rclone-copy-to-mega-can-rclone-correct-it/17017
2020-06-20 14:52:04 +01:00
40611fc4fc check: retry downloads if they fail when using the --download flag
See: https://forum.rclone.org/t/tons-of-data-corruption-after-rclone-copy-to-mega-can-rclone-correct-it/17017/7
2020-06-18 16:16:19 +01:00
7c4ba9fcb2 check: Fix misleading message which printed errors instead of differences
See: https://forum.rclone.org/t/tons-of-data-corruption-after-rclone-copy-to-mega-can-rclone-correct-it/17017/7
2020-06-18 16:16:19 +01:00
3efdf5e095 operations: implement --refresh-times flag to set modtimes on hashless backends 2020-06-17 10:48:13 +01:00
d174b97af7 errors: add WSAECONNREFUSED and more to the list of retriable Windows errors
This adds the missing WSAECONNREFUSED error to the list of errors we
can retry under Windows.

> Connection refused.  No connection could be made because the target
> computer actively refused it.

It also adds any relevant errors I could see in the error code list.

See: https://forum.rclone.org/t/failing-to-upload-large-file-to-b2/17085
2020-06-17 10:46:22 +01:00
d6a9017298 fs: fix formatting of errInvalidCharacters error message
errInvalidCharacters: 'and space .' -> 'and space.' Remove a space between the word, space, and period.

Co-authored-by: jtagcat <gitlab@c7.ee>
2020-06-16 15:08:09 +01:00
2d88d24881 config: reject remote names starting with a dash. (#4261) 2020-06-16 15:00:34 +01:00
a3f6fe5287 dedupe: fix logging to be easier to understand #4321 2020-06-16 11:41:56 +01:00
b58bb03e95 test: Don't run unreliable tests on CI #4171 2020-06-15 21:34:37 +01:00
763944f673 rcd: fix incorrect prometheus metrics - fixes #4341
This was caused by using the stats group from the context passed in by the rcd
rather than the global stats group.

Signed-off-by: Gary Kim <gary@garykim.dev>
2020-06-14 10:09:24 +01:00
fb06427c69 sync: fix --track-renames-strategy modtime
Before this change `--track-renames-strategy` was broken. The hashing
method it used could declare times that were very close together to be
different.

The time hash was discarded and instead we check the modification time
window on every hash match.

Provided that the user doesn't use `--track-renames-strategy` on a
huge number of identically sized files this will perform just fine.

See: https://forum.rclone.org/t/track-renames-strategy-modtime-doesnt-work/16992/5
2020-06-12 15:38:35 +01:00
d55053098f check: make check do --checkers files concurrently - fixes #4318 2020-06-10 17:20:54 +01:00
63cf0b1cdd check: make check command obey --dry-run/-i/--interactive - fixes #4325 2020-06-10 17:20:54 +01:00
a1382a03aa Start v1.52.1-DEV development 2020-06-10 16:49:55 +01:00
b23cf58a41 operations: Add skip all, do all, quit operations to --interactive - fixes #3886
This also adds SkipDestructive into all the remaing places --dry-run
was used and adds documentation.
2020-06-10 12:33:53 +01:00
ba5eb230fb operations: interactive mode -i/--interactive for destructive operations #3886 2020-06-10 12:33:53 +01:00
b5c654a100 lib/structs: factor reflection based structure manipulation into a library 2020-06-10 12:28:48 +01:00
3c14a893fb asyncreader: Make StopBuffer as well as Abandon and fix confusion in callers 2020-06-09 18:05:12 +01:00
55ad1354b6 link: Add --expire and --unlink flags
This adds expire and unlink fields to the PublicLink interface.

This fixes up the affected backends and removes unlink parameters
where they are present.
2020-06-05 14:51:05 +01:00
b62d08d136 config: set RCLONE_CONFIG_DIR for use in config files and subprocesses
See: https://forum.rclone.org/t/relative-path-in-rclone-config-service-account-json/16693
2020-06-03 17:39:08 +01:00
1cceadaf7c Start v1.52.0-DEV development 2020-05-27 18:36:32 +01:00
6882aeff97 Version v1.52.0 2020-05-27 17:31:10 +01:00
041b201abd doc: fix typos throughout docs and code 2020-05-25 11:23:58 +01:00
518d39815c sync,copy,move: add --check-first to do all checking before starting transfers
See: https://forum.rclone.org/t/rclone-sync-doing-transfer-and-checking-in-paralel/16352/
2020-05-22 17:50:07 +01:00
147f97d1f7 sync: allow --max-backlog to be -ve meaning as large as possible 2020-05-22 17:50:07 +01:00
4aee962233 doc: fix typos throughout docs and code 2020-05-20 15:54:51 +01:00
eb6e9b194a fspath: Stop empty strings being a valid path - fixes #4239
Before this change you could use "" as a valid remote, so `rclone lsf
""` would work. This was treated as the current directory.

This is unexpected and creates a footgun for scripting when an empty
variable is passed to rclone by accident.

This fix returns the error "can't use empty string as a path" instead
of allowing it.
2020-05-19 12:34:23 +01:00
fb169a8b54 doc: fix typos throughout docs 2020-05-19 12:02:44 +01:00
98c34e413d config: add --obscure and --no-obscure flags to config create/update
Before this change there was some ambiguity about whether passwords
were obscured on not passing them into config create or config update.

This change adds the --obscure and --no-obscure flags to make the
intent clear.

It also updates the remote control and the tests.

Fixes #3728
2020-05-15 16:41:37 +01:00
899c8e0697 march: added flag to allow Unicode filenames to remain unique
If your filenames contain two near-identical Unicode characters,
rclone will normalize these, making them identical. This flag
gives you the ability to keep them unique. This might
create unintended side effects, such as duplicating files that
contain certain Unicode characters, when downloading them from
certain cloud providers to a macOS filesystem.

Fixes #4228
2020-05-15 12:28:01 +01:00
044a3b3920 fserrors: Make "tls: use of closed connection" a retriable error
This has happened when uploading very large files to B2. It is
probably a bug in the go runtime but we'll attempt to work-around it
here.

See: https://forum.rclone.org/t/large-file-upload-to-backblaze-failed/16128/5
2020-05-13 11:42:37 +01:00