Commit Graph

1537 Commits

Author SHA1 Message Date
5c2dfeee46 operations: display 'All duplicates removed' only if dedupe successful -fixes rclone#3550 2019-10-08 16:34:13 +01:00
d15425e8c8 Start v1.49.5-DEV development 2019-10-05 12:42:28 +01:00
5271fe3b3f yandex: use lib/encoder 2019-10-05 10:22:43 +01:00
135717e12b mailru: use lib/encoder 2019-10-02 20:10:01 +01:00
fd51f24906 putio: use lib/encoder
And in the process
- fix a bug with + and & in file name
- fix NewObject returning directories as files
2019-10-02 11:34:08 +01:00
4615343b73 premiumizeme: use lib/encoder 2019-10-02 11:34:08 +01:00
1dc8bcd48c Remove backend dependency from fs/hash 2019-10-01 16:29:58 +01:00
b581f2de26 sharefile: use lib/encoder 2019-09-30 22:00:25 +01:00
8d8fad724b ficher: use lib/encoder 2019-09-30 22:00:25 +01:00
d122d1d191 qingstor: use lib/encoder 2019-09-30 22:00:25 +01:00
35d6ff89bf azureblob: use lib/encoder 2019-09-30 22:00:24 +01:00
53bec33027 swift: use lib/encoder 2019-09-30 22:00:24 +01:00
3304bb7a56 googlecloudstorage: use lib/encoder
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2019-09-30 22:00:24 +01:00
33f129fbbc s3: use lib/encoder
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2019-09-30 22:00:24 +01:00
6ae7bd7914 local: encode invalid UTF-8 on macOS 2019-09-30 22:00:24 +01:00
32af4cd6f3 ftp: use lib/encoder 2019-09-30 22:00:24 +01:00
00b2c02bf4 pcloud: use lib/encoder
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2019-09-30 22:00:24 +01:00
c8d3e57418 encodings: add . and .. to all backends, except Drive 2019-09-30 14:05:49 +01:00
4837bc3546 jottacloud: use lib/encoder 2019-09-30 14:05:49 +01:00
c09b62a088 encodings: add all known backend encodings 2019-09-30 14:05:49 +01:00
97a218903c fstest: remove WinPath from fstest.Item 2019-09-30 14:05:48 +01:00
262ba28dec config: add ReadNonEmptyLine utility function 2019-09-30 11:23:21 +01:00
74f6300875 Start v1.49.4-DEV development 2019-09-29 19:47:59 +01:00
ccecfa9cb1 chunker: finish meta-format before release
changes:
- chunker: remove GetTier and SetTier
- remove wdmrcompat metaformat
- remove fastopen strategy
- make hash_type option non-advanced
- adverise hash support when possible
- add metadata field "ver", run strict checks
- describe internal behavior in comments
- improve documentation

note:
wdmrcompat used to write file name in the metadata, so maximum metadata
size was 1K; removing it allows to cap size by 200 bytes now.
2019-09-25 11:03:33 +01:00
c41812fc88 tests: bring memory hungry tests close to end 2019-09-24 12:45:12 +01:00
d98d1be3fe accounting: fix panic due to server-side copy fallback 2019-09-24 12:45:12 +01:00
661dc568f3 fstest: let backends advertise maximum file size 2019-09-24 12:45:12 +01:00
1e4691f951 tests/sync: adjust transfer counts for chunker 2019-09-24 12:45:12 +01:00
557edecd40 log: add Stack() function for debugging who calls what 2019-09-20 11:53:08 +01:00
b242b0a078 lib/cache,rc/jobs: make tests more reliable 2019-09-20 11:53:08 +01:00
56544bb2fd accounting: fix file handle leak on errors - fixes #3547
In 53a1a0e3ef we introduced a problem where if there was an
error on the file being transferred then the file was re-opened and
the old one wasn't closed.

This was partially fixed in bfbddab46b however this didn't
address the case of the old file being closed.

This is now fixed by
- marking the file as open again in UpdateReader
- moving the stopping the accounting machinery to a new method Done
2019-09-19 16:20:07 +01:00
b8b12a4000 Start v1.49.3-DEV development 2019-09-15 18:10:08 +01:00
ef7bfd3f03 fs: Make prefix free backend config read prefix free env var also
Before this change you could only configure the local backend flags
which don't have the local prefix (eg `--copy-links`) with
`RCLONE_LOCAL_COPY_LINKS`.

This change makes `RCLONE_COPY_LINKS` valid too which is much more
logical for the users.

Fixes #3534
2019-09-14 18:26:07 +01:00
6959c997e2 config: remove error: can't use --size-only and --ignore-size together.
As sometimes they are useful together, for example when the remote
changes the sizes of the uploaded files.

See: https://forum.rclone.org/t/files-upload-will-be-auto-compress-how-do-i-sync-a-file-to-remote/10578
2019-09-14 10:13:44 +01:00
bfbddab46b fs/accounting: Fix "file already closed" on transfer retries
This was caused by the recent reworking of the accounting interface.
The Transfer object was recycling the Accounting object without
resetting the stream.

See: https://forum.rclone.org/t/error-file-already-closed/11469/
See: https://forum.rclone.org/t/rclone-b2-sync-post-error-method-not-supported/11718/
2019-09-13 18:35:02 +01:00
934440a9df accounting: fix total duration calculation
Fixes: #3498
2019-09-12 10:29:40 +01:00
bdcd0b4c64 Add mailru hash (mrhash) 2019-09-09 21:34:15 +01:00
94eb9a4014 Start v1.49.2-DEV development 2019-09-08 17:36:21 +01:00
f1347139fa config: check config names more carefully and report errors - fixes #3506
Before this change it was possible to make a remote with an invalid
name in the config file, either manually or with `rclone config
create` (but not with `rclone config`).

When this remote was used, because it was invalid, rclone would
presume this remote name was a local directory for a very suprising
user experience!

This change checks remote names more carefully and returns errors
- when the user tries to use an invalid remote name on the command line
- when an invalid remote name is used in `rclone config create/update/password`
- when the user tries to enter an invalid remote name in `rclone config`

This does not prevent the user entering a remote name with invalid
characters in the config manually, but such a remote will fail
immediately when it is used on the command line.
2019-09-06 12:07:09 +01:00
cf9b973fe4 accounting: fix locking in Transfer to avoid deadlock with --progress
Before this change, using -P occasionally deadlocked on the Transfer
mutex when Transfer.Done() was called with a non nil error and the
StatsInfo mutex since they mutually call each other.

This was fixed by making sure that the Transfer mutex is always
released before calling any StatsInfo methods.

This improves on: 6f87267b34

Fixes #3505
2019-09-06 10:00:44 +01:00
ffa1dac10b build: apply gofmt from go1.13 to change case of number literals 2019-09-05 13:59:06 +01:00
530ba66d35 operations: fix -u/--update with google photos / files of unknown size
Before this change if -u/--update was in effect we compared the size
of the files to see if the transfer should go ahead.  This was
comparing -1 with an actual size so the transfer always proceeded.

After this change we use the existing `sizeDiffers` function which
does the correct comparison with -1 for files of unknown length.

See: https://forum.rclone.org/t/sync-with-google-photos-to-local-drive-will-result-in-recoping/11605
2019-09-04 17:31:17 +01:00
89b6d89077 build: drop support for go1.9 2019-09-04 10:23:48 +01:00
b71ac141cc copyurl: add --auto-filename flag for using file name from url in destination path (#3451) 2019-09-03 17:25:19 +01:00
5932acfee3 rc: fix docs for config/create /update /password 2019-09-03 08:34:15 +01:00
8d296d0e1d Start v1.49.1-DEV development 2019-09-02 13:10:47 +01:00
50a4ed8fc4 operations: fix accounting for server side copies
See: https://forum.rclone.org/t/b2-server-side-copy-doesnt-show-cumulative-progress/11154
2019-09-02 09:26:33 +01:00
693112d57e config: Fix generated passwords being stored as empty password - Fixes #3492 2019-08-28 12:11:03 +01:00
d1a39dcc4b Start v1.49.0-DEV development 2019-08-26 17:44:09 +01:00
a6387e1f81 Version v1.49.0 2019-08-26 15:25:20 +01:00