Commit Graph

1115 Commits

Author SHA1 Message Date
0eaf5475ef Start v1.50.1-DEV development 2019-11-02 15:26:01 +00:00
520ddbcceb config: do not open browser on headless if google fs
On google fs (drive, google photos, and google cloud storage), if
headless is selected, do not open browser.

This also supplies a new option "auth-no-open-browser" for authorize
if the user does not want it.

This should fix #3323.
2019-10-30 14:12:42 +00:00
1ce1ea34aa hash: fix hash names for DropboxHash and CRC-32
These were unintentionally renamed as part of 1dc8bcd48c

Fixes #3679
2019-10-30 12:20:10 +00:00
e6378daadf fshttp: don't print token bucket errors on context cancelled
These happen as a natural part of exceeding --max-transfer and we
don't need to worry the user with them.
2019-10-30 12:20:10 +00:00
e0356f5aae rcd: Adding group parameter to stats 2019-10-29 16:39:37 +00:00
e81eca4055 fshttp: fix error reporting on tpslimit token bucket errors 2019-10-28 22:11:38 +00:00
199ac61bde rc: add methods to turn on blocking and mutex profiling 2019-10-28 22:11:38 +00:00
1868c77e16 rc: fix formatting of docs 2019-10-27 10:43:40 +00:00
daff5a824e Start v1.50.0-DEV development 2019-10-26 12:42:06 +01:00
6fabf476cf Version v1.50.0 2019-10-26 11:04:54 +01:00
0c265713fd rc: added command core/quit 2019-10-17 15:04:22 +01:00
13e46c4b3f accounting: cull the old time ranges when possible to save memory 2019-10-17 11:43:32 +01:00
d40972bf1a accounting: allow up to 100 completed transfers in the accounting list
This fixes the core/transfers rc so it shows items again.
2019-10-16 22:13:17 +01:00
b002ff8d54 accounting: fix total duration calculation
This was broken in e337cae0c5 when we deleted the transfers
immediately.

This is fixed by keeping a merged slice of time ranges of completed
transfers and adding those to the current transfers.
2019-10-16 22:13:17 +01:00
0b6cdb7677 fshttp: allow Transport to be customized #3631 2019-10-16 11:26:08 +01:00
543100070a sync: free objects after they come out of the transfer pipe
This reduces memory when the transfer pipe shrinks

See: https://forum.rclone.org/t/rclone-memory-consumption-increasing-linearly/12244
2019-10-16 10:27:07 +01:00
e337cae0c5 accounting: fix memory leak noticeable for transfers of large numbers of objects
Before this fix we weren't removing transfers from the transfer stats.
For transfers with 1000s of objects this uses a noticeable amount of
memory.

See: https://forum.rclone.org/t/rclone-memory-consumption-increasing-linearly/12244
2019-10-16 10:27:07 +01:00
af05e290cf Fix --files-from without --no-traverse doing a recursive scan
In a28239f005 we made --files-from obey --no-traverse.  In the
process this caused --files-from without --no-traverse to do a
complete recursive scan unecessarily.

This was only noticeable in users of fs/march, so sync/copy/move/etc
not in ls/lsf/etc.

This fix makes sure that we use conventional directory listings in
fs/march unless `--files-from` and `--no-traverse` is set or
`--fast-list` is active.

Fixes #3619
2019-10-15 19:51:01 +01:00
f9f9d5029b fserror: make http2 "stream error:" a retriable error
It was reported that v1.49.4 which was accidentally compiled with
go1.13 instead of go1.12 produced errors like this:

    Failed to get StartPageToken: Get https://www.googleapis.com/drive/v3/changes/startPageToken?XXX: stream error: stream ID 1789; INTERNAL_ERROR
    IO error: open file failed: Get https://www.googleapis.com/drive/v3/files/XXX?alt=media: stream error: stream ID 1781; INTERNAL_ERROR

These are errors from the http2 library.  It appears that go1.13 when
communicating with google drive defaults to http2 whereas with go1.12
it doesn't.

It is unclear what is causing these errors, but retrying them since
they don't happen very often seems like a valid strategy.

This was fixed in v1.49.5 by compiling with go1.12 - this fix is
designed to work with go1.13

See: https://forum.rclone.org/t/1-49-4-plex-internal-errors-on-google-drive/12108/
2019-10-15 19:46:44 +01:00
1b2ffbeca0 cmd: fix environment variables not setting command line flags
Before this fix quite a lot of the commands were ignoring environment
variables intended to set flags.
2019-10-14 17:02:09 +01:00
6e378d7d32 config: fix setting of non top level flags from environment variables
Before this fix, attempting to set a non top level environment
variable would fail with "Couldn't find flag".

This fixes it by passing in the flags that the env var is being set
from.

Fixes #3615
2019-10-14 17:02:09 +01:00
b63e9befe8 rc docs: fix code section not rendering properly due to missing quotes 2019-10-13 12:26:37 +01:00
f3b0f8a9f0 sync: --update/-u not transfer files that haven't changed - fixes #3232
Before this change --update would transfer any file which was newer
than the destination regardless of whether it had changed or not.
This is needlessly wasteful of bandwidth.

After this change --update will only transfer files if they are newer
**and** they are different (checked with checksum and size).
2019-10-12 11:54:56 +01:00
02c777ffbf filter: prevent mix opts when filesfrom is present - fixes #3599 2019-10-11 16:17:02 +01:00
58f280b8a2 fserrors: fix a bug in Cause function 2019-10-10 16:05:15 +01:00
ec09de1628 Change the Debug message in NeedTranser (#3608)
'Couldn't find file - Need to Transfer' changed to 'Need to transfer -
File Not Found at Destination' because while reading the debug logs, it
confuses with failure in operation.
2019-10-10 13:44:05 +01:00
455b9280ba config: use alternating Red/Green in config to make more obvious 2019-10-08 19:59:47 +01:00
7712b780ba operations: display 'Deleted X extra copies' only if dedupe successful - fixes #3551 2019-10-08 16:35:53 +01:00
5c2dfeee46 operations: display 'All duplicates removed' only if dedupe successful -fixes rclone#3550 2019-10-08 16:34:13 +01:00
d15425e8c8 Start v1.49.5-DEV development 2019-10-05 12:42:28 +01:00
5271fe3b3f yandex: use lib/encoder 2019-10-05 10:22:43 +01:00
135717e12b mailru: use lib/encoder 2019-10-02 20:10:01 +01:00
fd51f24906 putio: use lib/encoder
And in the process
- fix a bug with + and & in file name
- fix NewObject returning directories as files
2019-10-02 11:34:08 +01:00
4615343b73 premiumizeme: use lib/encoder 2019-10-02 11:34:08 +01:00
1dc8bcd48c Remove backend dependency from fs/hash 2019-10-01 16:29:58 +01:00
b581f2de26 sharefile: use lib/encoder 2019-09-30 22:00:25 +01:00
8d8fad724b ficher: use lib/encoder 2019-09-30 22:00:25 +01:00
d122d1d191 qingstor: use lib/encoder 2019-09-30 22:00:25 +01:00
35d6ff89bf azureblob: use lib/encoder 2019-09-30 22:00:24 +01:00
53bec33027 swift: use lib/encoder 2019-09-30 22:00:24 +01:00
3304bb7a56 googlecloudstorage: use lib/encoder
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2019-09-30 22:00:24 +01:00
33f129fbbc s3: use lib/encoder
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2019-09-30 22:00:24 +01:00
6ae7bd7914 local: encode invalid UTF-8 on macOS 2019-09-30 22:00:24 +01:00
32af4cd6f3 ftp: use lib/encoder 2019-09-30 22:00:24 +01:00
00b2c02bf4 pcloud: use lib/encoder
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2019-09-30 22:00:24 +01:00
c8d3e57418 encodings: add . and .. to all backends, except Drive 2019-09-30 14:05:49 +01:00
4837bc3546 jottacloud: use lib/encoder 2019-09-30 14:05:49 +01:00
c09b62a088 encodings: add all known backend encodings 2019-09-30 14:05:49 +01:00
97a218903c fstest: remove WinPath from fstest.Item 2019-09-30 14:05:48 +01:00
262ba28dec config: add ReadNonEmptyLine utility function 2019-09-30 11:23:21 +01:00