Commit Graph

5204 Commits

Author SHA1 Message Date
Tim Gallant
c3884aafd9 drive: adds special oauth help test - fixes #4555 2020-09-07 12:48:46 +01:00
Nick Craig-Wood
0a9785a4ff build: don't explicitly set ARM version to fix ARMv5 build #4553
This partially reverts commit f71f6c57d7.
2020-09-07 12:39:26 +01:00
Nick Craig-Wood
8140f67092 check: fix docs
See: https://forum.rclone.org/t/possible-issue-with-documention/18926
2020-09-07 12:10:52 +01:00
Nick Craig-Wood
4a001b8a02 check: add back missing --download flag - fixes #4565
This was accidentally removed when refactoring check and cryptcheck in

8b6f2bbb4b check,cryptcheck: add reporting of filenames for same/missing/changed #3264
2020-09-05 09:29:35 +01:00
Nick Craig-Wood
525433e6dd build: fix "Illegal instruction" error for ARMv6 builds - fixes #4553
Before this change we used `go build -i` to build the releases in parallel.

However this causes the ARMv6 and ARMv7 build to get mixed up somehow,
causing an illegal instruction when running rclone binaries on ARMv6.

See go bug: https://github.com/golang/go/issues/41223

This removes the -i which should have no effect on build times on the
CI and appears to fix the problem.
2020-09-04 16:30:50 +01:00
Nick Craig-Wood
f71f6c57d7 build: explicitly set ARM version to fix build #4553 2020-09-04 16:30:50 +01:00
albertony
e35623c72e docs/jottacloud: mention that uploads from local disk will not need to cache files to disk for md5 calculation 2020-09-04 00:04:09 +02:00
Nick Craig-Wood
344bce7e2a docs: fix formatting of rc docs page
See: https://forum.rclone.org/t/rclone-1-53-release/18880/24
2020-09-03 11:53:24 +01:00
Nick Craig-Wood
3a4322a7ba build: update build for stable branch 2020-09-03 11:30:23 +01:00
Nick Craig-Wood
27b9ae4fc3 vfs: fix spurious error "vfs cache: failed to _ensure cache EOF"
Before this change the error message was produced for every file which
was confusing users.

After this change we check for EOF and return from ReadAt at that
point.

See: https://forum.rclone.org/t/rclone-1-53-release/18880/10
2020-09-03 10:25:00 +01:00
Nick Craig-Wood
7e2488af10 build: include vendor tar ball in release and fix startdev 2020-09-02 17:53:05 +01:00
Nick Craig-Wood
41ecb586c4 Start v1.54.0-DEV development 2020-09-02 17:52:58 +01:00
Nick Craig-Wood
510ac341e1 Version v1.53.0 2020-09-02 17:00:18 +01:00
Nick Craig-Wood
358e2b2665 Revert "docs: make the website navbar stick to top"
Unfortunately this breaks the anchor links - the titles are under the
navbar, so revert this for the moment.

This reverts commit cdfb3f7194.
2020-09-02 16:43:05 +01:00
Nick Craig-Wood
3305079a03 vfs: fix typos in help 2020-09-02 14:24:44 +01:00
Nick Craig-Wood
6ed8471a37 Add Sam Edwards to contributors 2020-09-02 14:12:22 +01:00
Nick Craig-Wood
dc7ce37c32 Add WarpedPixel to contributors 2020-09-02 14:12:22 +01:00
themylogin
57c10babfe drive: Remove --drive-alternate-export in favor of exportLinks
Google engineer confirms that the new official API should works properly:
https://issuetracker.google.com/issues/36761333#comment8
2020-09-02 12:16:25 +01:00
Sam Edwards
23b2c58018 vfs: Quiet removeNotInUse logging to debug when not removing 2020-09-02 11:55:20 +01:00
Evan Harris
78abd21eec docs: Cleaned up and corrected bugs docs verbiage 2020-09-02 11:53:52 +01:00
Evan Harris
841edc729c docs: Updated --track-renames docs 2020-09-02 11:52:01 +01:00
Evan Harris
b03fcbcc12 docs: Updated overview with Opendrive support of dupes 2020-09-02 11:45:11 +01:00
WarpedPixel
b60ac7b66a
onedrive: document workaround for file size download errors #2036 2020-09-02 11:44:05 +01:00
Nick Craig-Wood
725ae91387 s3: reduce the default --s3-copy-cutoff to < 5GB
The maximum value for the --s3--copy-cutoff should be 5GiB as tested
with AWS S3.

However b2 have implemented this as 5GB rather than 5GiB so having the
default at 5 GiB makes the b2s3 server side copy of a large file by
default.

This patch sets the default to 4768 MiB which is slightly less than
5GB.

This should have very little effect on anything.

If in future rclone can lower this limit more if Copy can multithread.

See: https://forum.rclone.org/t/copying-files-within-a-b2-bucket/16680/76
2020-09-01 18:53:29 +01:00
Nick Craig-Wood
b7dd3ce608 s3: preserve metadata when doing multipart copy
Before this change the s3 multipart server side copy was not
preserving the metadata of the object. This was most noticeable
because the modtime was not preserved.

This change fetches the metadata from the object before starting the
copy and overwrites it if requires.

It will also mean any other metadata is preserved.

See: https://forum.rclone.org/t/copying-files-within-a-b2-bucket/16680/70
2020-09-01 18:39:30 +01:00
Nick Craig-Wood
70c8566cb8 fs: Pin created backends until parents are finalized
This attempts to solve the backend lifecycle problem by

- Pinning backends mentioned on the command line into the cache
  indefinitely

- Unpinning backends when the containing structure (VFS, wrapping
  backend) is destroyed

See: https://forum.rclone.org/t/rclone-rc-backend-command-not-working-as-expected/18834
2020-09-01 18:21:03 +01:00
Nick Craig-Wood
0d066bdf46 alias,cache,chunker,crypt: make any created backends be cached to fix rc problems
Before this change, when the above backends created a new backend they
didn't put it into the backend cache.

This meant that rc commands acting on those backends did not work.

This was fixed by making sure the backends use the backend cache.

See: https://forum.rclone.org/t/rclone-rc-backend-command-not-working-as-expected/18834
2020-09-01 18:21:03 +01:00
Nick Craig-Wood
3affc2e066 fspath: make JoinRootPath convert backslashes to slashes on Windows
The function is used for contructing remotes which may have
backslashes in on Windows.
2020-09-01 18:21:03 +01:00
Nick Craig-Wood
23c826db52 union: fix writing with the all policy - fixes #4534
Before this change writing with the all policy deadlocked while
uploading.

This change fixes the problem by fixing the multi reader, closing the
pipes at the correct time with the correct error. This is factored
into a new function as it was used twice.

This patch also adds a new test which tests the all policies.
2020-09-01 18:21:03 +01:00
Nick Craig-Wood
1ae36a4e32 Add Lucas Kanashiro to contributors 2020-09-01 18:21:03 +01:00
Nick Craig-Wood
bc969ad244 Add Egor Margineanu to contributors 2020-09-01 18:21:03 +01:00
Nick Craig-Wood
d7ac1f5b0e Add Aaron Gokaslan to contributors 2020-09-01 18:21:03 +01:00
Nick Craig-Wood
5bf53fe3ac test_all: only run the backend tests for 1fichier
Running all the tests for 1fichier takes too long due to the directory
reading rate limiter.

The backend tests do complete in a reasonable time (21 mins).
2020-09-01 16:13:47 +01:00
Nick Craig-Wood
9cc17cec9a swift: fix missing hash from object returned from upload
Before this fix we were reading the hash from the upload using the
string "ETag", however the go runtime normalises the tag into "Etag"
so we were in fact always reading an empty string.

This bug was introduced in

aeea4430d5 swift: efficiency: slim Object and reduce requests on upload

It was spotted by the integration tests.

The fix was just to use the canonical form "Etag" instead of "ETag".
2020-09-01 16:04:32 +01:00
Nick Craig-Wood
e2816629d0 fstest: fix upwrapping tests for bucket based remotes
TestIntegration/FsRmdirNotFound was failing on crypt wrapping a bucket based remote.

This was spotted by the integration tests.
2020-09-01 15:43:41 +01:00
Nick Craig-Wood
3f0d54daae crypt: fix purge bug introduced by refactor #1891
In this commit

a2afa9aadd fs: Add directory to optional Purge interface

We failed to encrypt the directory name so the Purge failed.

This was spotted by the integration tests.
2020-09-01 15:16:14 +01:00
Aaron Gokaslan
7dcbebf9bc jottacloud: rename unused variable to _ in jottacloud.go 2020-08-31 18:11:36 +01:00
Nick Craig-Wood
c31defbbd3 fs: add debug to show when a backend is being created
See: https://forum.rclone.org/t/rclone-rc-backend-command-not-working-as-expected/18834/
2020-08-31 14:51:06 +01:00
Nick Craig-Wood
e54ce35019 build: change beta numbering to be semver compatible - Fixes #4516
From now on the betas will be numbered for the version that they will
become, so:

v1.53.0-beta.NNNN.CCCCC

Where N is commit number and C is commit. When released this will
become v1.53.0 and the beta will become v1.54.0-beta.NNN.CCCCC.

The commit number is the count of the commits since the root of the
tree since we can no longer use the the git version numbers since the
last tag.

This will simplify building the stable branch but that release
procedure hasn't been revised yet.

This commit also injects the name of the branch for the beta builds
into the download path.
2020-08-31 13:55:04 +01:00
Nick Craig-Wood
75d54d720c version: replace internal code with github.com/coreos/go-semver
We were already importing go-semver so it makes sense to remove the
duplicated semver parsing code and just use go-semver
2020-08-31 13:55:04 +01:00
Nick Craig-Wood
cc0421cb9e rc/webgui: skip AddPlugin and RemovePlugin tests if download fails 2020-08-31 13:45:06 +01:00
Nick Craig-Wood
9c01ac9894 rc/webgui: improve error handling on web fetches 2020-08-31 13:45:06 +01:00
Chaitanya Bankanhal
20300d1f61 plugins: Change failing plugin test to new repo rclone/rclone-test-plugin 2020-08-31 13:45:06 +01:00
Chaitanya Bankanhal
6231beefc5 webui: Fix broken webui because of plugins redirection 2020-08-31 13:45:06 +01:00
Nick Craig-Wood
068cfdaa00 drive: fix "panic: send on closed channel" when recycling dir entries
In this commit:

cbf3d43561 drive: fix missing items when listing using --fast-list / ListR

We introduced a bug where under specific circumstances it could cause
a "panic: send on closed channel".

This was caused by:

- rclone engaging the workaround from the commit above
- one of the listing routines returning an error
- this caused the `in` channel to be closed to stop the readers
- however the workaround was recycling stuff into the `in` channel at the time
- hence the panic on closed channel

This fix factors out the sending to the `in` channel into `sendJob`
and calls this both from the master go routine and the list
runners. `sendJob` detects the `in` channel being closed properly and
also deals correctly with contention on the `in` channel.

Fixes #4511
2020-08-31 11:41:15 +01:00
Nick Craig-Wood
7d62d1fc97 Add aus to contributors 2020-08-31 11:41:15 +01:00
Nick Craig-Wood
e13ac28b8d Add Leo Luan to contributors 2020-08-31 11:41:15 +01:00
Lucas Kanashiro
b30ee57cd9 backend/local/aaaa: remove this unneeded file
This file was introduced as part of f39655093 probably by
mistake. There is no reference for this file in the local
backend directory.

Fixes #4536
2020-08-30 22:35:58 +01:00
Egor Margineanu
921e384c4d
s3: update IBM COS endpoints - fixes #4522 2020-08-30 17:21:11 +01:00
Aaron Gokaslan
bf685f600e webgui: fixes previously unhandled error in JSON marshall in fs/rc/webgui/plugins.go:writeToFile 2020-08-30 17:15:03 +01:00