extern/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2024-11-25 09:54:44 +01:00
Code Issues Packages Projects Releases Wiki Activity
2,240 Commits 177 Branches 114 Tags 315 MiB
1c9572aba1
Commit Graph

2240 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Craig-Wood
ef3bcec76c fs: Extend SizeSuffix to include TB and PB for rclone about 2018-04-17 21:53:42 +01:00
Nick Craig-Wood
1ac6dacf0f about: complete other providers and re-work internals 
* Implement about for:
    * local, crypt, cache, drive, swift, hubic, onedrive, pcloud, dropbox
  * Implement `--json` and `---full` flag for `rclone about`
  * change About interface to return a Usage structure
  * Remove operations.About as it is too thin an interface
  * Implement Integration test

Relates to #1138 and #1564
 2018-04-17 21:53:27 +01:00
a-roussos
94e277d759 about: add new command 'about' to get quota info from a remote 
Implemented for drive only.

Relates to #1138 and #1564.
 2018-04-17 21:50:14 +01:00
Nick Craig-Wood
b83814082b backend/http: if HEAD didn't return Content-Length use -1 as size 
This means that the files will be treated as an unknown length and
will download properly.

Fixes #2247
 2018-04-16 19:40:02 +01:00
Nick Craig-Wood
2b7957cc74 vfs: Only make the VFS cache if --vfs-cache-mode > Off 
This stops the cache cleaner running unnecessarily and saves
resources.

This also helps with issue #2227 which was caused by a second mount
deleting objects in the first mounts cache.
 2018-04-16 17:06:41 +01:00
Nick Craig-Wood
3d5106e52b drive: fix DirMove leaving a hardlinked directory behind #2245 
This bug was introduced by the v3 API conversion in 07f20dd1fd.

The problem was that dircache.FindPath doesn't work for the root directory.

This adds an internal error for dircache.FindPath being called with
the root directory.  This makes a failing test, which the fix to the
drive backend fixes.

This also improves the DirCache integration test.
 2018-04-15 10:12:21 +01:00
Nick Craig-Wood
29ce1c2747 fstest: fix CheckListingWithPrecision with non Windows safe chars 
* Factor WinPath from fstest to fstests
  * Use it to normalize the directory names while checking them
 2018-04-15 10:12:20 +01:00
Nick Craig-Wood
dc247d21ff s3: add in config for all the supported S3 providers #2140 
These are AWS, Ceph, Dreamhost, IBM COS S3, Minio, Wasabi and Other.

This configures endpoints where known and makes sure config doesn't
appear where it isn't valid where possible.
 2018-04-13 16:33:26 +01:00
Nick Craig-Wood
8c3740c2c5 config: Improve the Provider matching to have a negated match #2140 
This makes it easier to make classes of provider in the config.
 2018-04-13 16:06:37 +01:00
Giri Badanahatti
acd5d4377e config,s3: hierarchical configuration support #2140 
This introduces a method of making provider specific configuration
within a remote.  This is useful particularly in s3.

This commit does the basic configuration in S3 for IBM COS.
 2018-04-13 16:05:35 +01:00
Matthew Holt
9e4cd55477 size: Add --json flag 2018-04-13 13:38:06 +01:00
Nick Craig-Wood
2015f98f0c Add Craig Rachel to contributors 2018-04-13 13:36:46 +01:00
Craig Rachel
0e6faa2313 s3: add One Zone Infrequent Access storage class - fixes #2240 2018-04-13 13:36:25 +01:00
Nick Craig-Wood
905e40b3e6 Add Peter Baumgartner to contributors 2018-04-13 13:33:22 +01:00
Peter Baumgartner
1db68571fd s3,swift: Add --use-server-modtime 
`--use-server-modtime` stops s3 and swift retrieving the modtime from metadata which enables a fast sync mode with the `--update` flag.
 2018-04-13 13:32:17 +01:00
Nick Craig-Wood
6b67489133 Add Animosity022 to contributors 2018-04-13 13:26:41 +01:00
Animosity022
27dfcf303c cache: improve docs 
This adds that the cache-chunk-path needs to be cleared manually if chunk-size is changed.
 2018-04-13 13:26:26 +01:00
Nick Craig-Wood
e6d9720d7b Add Mateusz Piotrowski to contributors 2018-04-13 13:25:16 +01:00
Mateusz Piotrowski
196da4d903 dropbox: fix a typo in the docs 2018-04-13 13:24:58 +01:00
Nick Craig-Wood
18317a2747 vendor: update github.com/pkg/sftp because dep insisted 2018-04-13 13:23:55 +01:00
Nick Craig-Wood
ef412c1985 drive: fix misplaced log in dedupe MergeDirs 2018-04-13 13:23:55 +01:00
Nick Craig-Wood
d97fe3b824 fs/operations: make dedupe work with mega 
* factor into its own files
  * remove assumptions about having a given hash type
  * make tests work if the remote has no hash
 2018-04-13 13:23:55 +01:00
Nick Craig-Wood
792c9e185e Add Antoine GIRARD to contributors 2018-04-13 13:23:55 +01:00
Antoine GIRARD
1f681e585b fstests: fix typo 2018-04-13 13:23:08 +01:00
Nick Craig-Wood
e82452ce9a drive: check Open calls for google error messages 
This should also enable Open calls to retry properly
 2018-04-11 20:55:58 +01:00
Nick Craig-Wood
dcf8334673 fs: add --dump goroutines and --dump openfiles 
These are developer flags useful for tracking down resource leaks.
 2018-04-11 20:55:58 +01:00
Nick Craig-Wood
37be78705d fs/fshttp: limit MaxIdleConns and MaxIdleConnsPerHost 
Before this change mega (which uses a different host per download)
would open too many sockets.
 2018-04-11 20:51:28 +01:00
Nick Craig-Wood
4b5ff33125 fstest: retry cleaning the integration test directory if necessary 2018-04-11 20:51:13 +01:00
Nick Craig-Wood
d5b2ec32f1 local: add --local-no-check-updated to disable update checks #2206 
This disables the `can't copy - source file is being updated` checks.
 2018-04-09 15:27:58 +01:00
Nick Craig-Wood
aeedacfb50 Add Michael P. Dubner to contributors 2018-04-09 13:33:27 +01:00
Michael P. Dubner
92b266d361 rc: new call rc/pid - closes #2211 2018-04-09 13:33:04 +01:00
Nick Craig-Wood
05e32cfcf9 dropbox: Fix crypt+obfuscate on dropbox - fixes #2191 
Before this change we lowercased the dropbox root directory.  This was
likely a leftover from when we used to build a dictionary to translate
the cases of dropbox files.  Now with the v2 API we can rely on
dropbox to do that for us, so we no longer need to lowercase the root.

This fixes issues using crypt with name obfuscation on dropbox.
 2018-04-09 11:53:41 +01:00
Nick Craig-Wood
cbec59146a lsf: make sure we use localtime in tests - fixes Box integration tests 
This problem was introduced with eca99b33c0.  It seems Box is the only
remote which converts time zones, so if you give it a GMT time zone,
it returns a PST time zone which represents the same instant.
 2018-04-09 11:46:49 +01:00
Nick Craig-Wood
06e3fa3aba mounttest: reduce duplicated code and improve test output #2154 
The written out list of tests was replaced with a nested test for
mount and cmount. The tests for each VFS cache mode were also replaced
with nested tests which makes the output and the code much cleaner.
 2018-04-08 15:04:14 +01:00
Nick Craig-Wood
0fa700b3cf Make integration tests use go1.7+ nested tests #2154 
* Removed generated code and code generator
  * Updated docs on how to write integration tests
  * Tidied up the actual integration tests
 2018-04-08 15:04:14 +01:00
Nick Craig-Wood
42f0963bf9 local: retry remove on Windows sharing violation error #2202 
Before this change asynchronous closes in cmount could cause sharing
violations under Windows on Remove which manifest themselves
frequently as test failures.

This change lets the Remove be retried on a sharing violation under
Windows.
 2018-04-07 17:36:26 +01:00
Nick Craig-Wood
be54fd8f70 Remove builds conditional on go1.7 since that is now guaranteed #2154 
Old fallback code was deleted and the go1.7 style code inlined where
appropriate.
 2018-04-07 11:42:55 +01:00
Nick Craig-Wood
e5be471ce0 Use io.SeekStart/End/Current constants now for go1.7+ #2154 2018-04-07 11:42:36 +01:00
Nick Craig-Wood
80588a5a6b Replace "golang.org/x/net/context" with "context" for go1.7+ #2154 2018-04-07 11:42:08 +01:00
Nick Craig-Wood
67023f0040 Require go1.7 for compilation #2154 
* Update the travis tests to exclude go1.6
  * Update the compile check to require go1.7+
  * Update misc go1.6 workarounds marked in the source
 2018-04-06 20:18:14 +01:00
Nick Craig-Wood
32e02bd367 fstests: Fix TestObjectRemove failures 
This was failing because TestPublicLink was causing the file to be
modified with Google drive.
 2018-04-06 16:27:19 +01:00
Nick Craig-Wood
c749cf8d99 dropbox: fix repeatedly uploading the same files - fixes #2218 
In #2134 and dfd0f4c5a4 some testing
changes got committed by accident which caused this regression.

This patch reverts it to how it was before.
 2018-04-06 15:34:56 +01:00
Nick Craig-Wood
92cfb57fbd fstest/test_all: make -clean work better with google cloud storage 2018-04-06 14:54:33 +01:00
Nick Craig-Wood
0cb5c4aa73 gcs: detect bucket presence by listing it - fixes #2193 
Doing it like this enables the use of a service account that only has
the "Storage Object Admin" role.
 2018-04-06 12:45:15 +01:00
Nick Craig-Wood
0358e9e724 Add Eri Bastos to contributors 2018-04-05 20:20:53 +01:00
Eri Bastos
a69d8ec93b Fixed typo on ownCloud description 2018-04-05 20:20:31 +01:00
Nick Craig-Wood
92c5aa3786 s3: add --s3-chunk-size option - fixes #2203 2018-04-05 15:40:08 +01:00
Nick Craig-Wood
fbe1c7f1ea dropbox: remove unused code 2018-04-05 15:23:23 +01:00
Nick Craig-Wood
c4531daa43 local: work on spurious "can't copy - source file is being updated" errors #2206 
Update all the time comparisons to use time.Time.Equal instead of ==

Improve the logging for that error so we can see exactly what has changed
 2018-04-05 14:57:30 +01:00
remusb
6e11a25df5 cache: flush the memory cache after close 2018-04-04 23:25:53 +03:00