1b95718460
Fix typos in filter docs and unit test assertions
2015-10-20 09:16:47 +01:00
6a3580c556
Show status of master branch, so it doesn't show the status of the last pushed branch,
2015-10-19 18:56:03 +01:00
16c9fba5de
Fix tests failing on Windows.
...
* ":" is kept when part of a drive.
* Create tests.
* Fix test framework.
2015-10-19 17:36:15 +01:00
4e952af614
Allow spaces in remotes and check remote names for validity at creation time - fixes #171
2015-10-12 17:54:09 +01:00
6344c3051c
Add async readahead buffer
...
This adds an async read buffer of 4x4MB when copying files >10MB.
This fixes #164 and reduces the number of IO operations for copy/move.
2015-10-12 08:30:27 +01:00
ab9f521cbd
Allow '&' and disallow ':' in Windows filenames.
...
Fixes #161
2015-10-05 11:04:25 +02:00
3a900e5bb7
Version v1.23
2015-10-03 16:24:07 +01:00
b4d7741611
Improve output of --dump-headers
2015-10-03 16:04:51 +01:00
95fd79faf9
swift: use Content-Length on uploads - fixes #125
2015-10-03 16:04:51 +01:00
b79dc01016
swift: stop chunked operations logging "Failed to read info: Object Not Found"
2015-10-03 16:04:51 +01:00
bf562d7373
Tweak wording on client/secret ids in acd, gcs and drive - fixes #155
2015-10-03 16:04:45 +01:00
2e9f2ea3d3
oauthutil: fix headless config for drive and gcs - fixes #158
2015-10-03 16:04:21 +01:00
177dbbc29a
Implement rclone size for measuring remotes - fixes #153
2015-10-03 16:04:21 +01:00
4712043e26
Fix race condition in pacer tests - fixes #156
2015-10-03 16:04:21 +01:00
852acd5e4e
oauthutil: tell the user they should try again if the webserver method failed
2015-10-03 16:04:21 +01:00
9f1daabb2c
s3: allow anonymous access to public repositories - fixes #154
...
When setting up the remote, leave both the access key and secret key
blank.
2015-10-03 16:04:21 +01:00
938dd24cc9
Fix typo
2015-09-28 22:51:33 +01:00
57aad81b68
Version v1.22
2015-09-28 19:38:20 +01:00
a91bcaaeb0
Implement rsync like include and exclude - fixes #27
...
* Implement include/exclude
* Implement rsync compatible file globbing
* Implement command line filtering flags
* --delete-excluded - Delete files on dest excluded from sync
* --filter - Add a file-filtering rule
* --filter-from - Read filtering patterns from a file
* --exclude - Exclude files matching pattern
* --exclude-from - Read exclude patterns from file
* --include - Include files matching pattern
* --include-from - Read include patterns from file
* --files-from - Read list of source-file nam
* --min-size - Don't transfer any file smaller than this in k or suffix k|M|G
* --max-size - Don't transfer any file larger than this in k or suffix k|M|G
* Document
2015-09-28 19:18:21 +01:00
d04c21b198
Add Sergey Tolmachev to contributors
2015-09-28 19:12:35 +01:00
4a0a42c2f1
swift: large file upload fixes
...
* Read metadata in file listing for 0 length files to fix syncs
* Ignore non-existent files in isManifestFile to fix errors on copy
* remove nsToSwiftFloatString - experiments with the swift program
indicate that it puts a variable number of points after the
decimal, so might as well use the one in the swift library.
* Make sure segments get deleted properly when move from segmented
to non segmented and vice versa
* Use internal list routine to detect errors on listing
* Remove the _segments container if possible
* Remove manifest first when deleting
2015-09-26 17:58:04 +01:00
cc7b9af50e
swift: support files > 5GB - fixes #46
...
* Write segments to ..._segments container
* Choice of container and segment names compatible with swift tool
* See http://docs.openstack.org/developer/swift/overview_large_objects.html
* Controlled by command line flag --swift-chunk-size
* Segments removed on delete
2015-09-26 13:03:58 +01:00
68fef49c55
swift: fetch headers as the only source of metadata
2015-09-26 13:03:58 +01:00
5d4b149884
Version v1.21
2015-09-22 21:05:11 +01:00
5f20ae707d
Make lsl output times in localtime and fix tests - fixes #141
2015-09-22 19:04:12 +01:00
e9c915e6fe
Fix golint warnings
2015-09-22 18:47:16 +01:00
2ed158aba3
Fixes from go vet and errcheck
2015-09-22 07:31:12 +01:00
05050d53ad
Implement compliant pacing scheme for Amazon Cloud Drive
...
* Implement switchable pacing algorithm
* Add tests for pacer
2015-09-21 21:49:54 +01:00
e391311512
gofmt
2015-09-17 18:42:39 +01:00
3234c28f7c
Minor adjustments to Unc path conversion function.
2015-09-17 18:42:10 +01:00
6fbd9cf24b
Make Google Drive Directory reads concurrent for increased speed.
...
If you use the -v flag then this will show the progress of directory
listing. Fixes #143 .
2015-09-17 18:42:10 +01:00
bc5b63ffef
Move ACD Connection-limiter into Pacer.
2015-09-17 18:42:10 +01:00
788ef76f1c
Show more of the filename and align output.
...
Print more of the file name, and make the output aligned, so it is nicer on frequent updates.
2015-09-17 15:53:45 +02:00
0872ec3204
Allow user to override credentials again in drive, gcs and acd - fixes #139
2015-09-16 20:08:40 +01:00
0a5870208e
Display individual transfer progress
...
Improve progress printing by displaying individual file progress, as well
as a moving average speed with ETA. Example output:
2015/09/15 16:38:21
Transferred: 183599104 Bytes (4646.49 kByte/s)
Errors: 0
Checks: 1
Transferred: 0
Elapsed time: 38.5s
Transferring:
* 01_06_14.mp3: 33% done. avg: 1280.5, cur: 1288.8 kByte/s. ETA: 1m12s
* 01_12_15.mp3: 33% done. avg: 1002.2, cur: 943.4 kByte/s. ETA: 1m17s
* 01_13_14.mp3: 48% done. avg: 1456.8, cur: 1425.2 kByte/s. ETA: 39s
* 01_19_15.mp3: 28% done. avg: 1226.9, cur: 1114.4 kByte/s. ETA: 1m37s
2015-09-16 19:42:46 +01:00
3219334c3e
Add "-race" test to travis.
2015-09-16 12:26:28 +02:00
79fd662676
Protect concurrent read/writes to pacer.
...
Protects all variables in the pacer from concurrent modifications. It is now safe to modify pacer settings while it is running.
I decided to not go for an RWMutex, since all accesses are very short, so the overhead of an RWMutex isn't worth it.
Fixes #138 .
2015-09-16 12:25:55 +02:00
34193fd8d9
Version v1.20
2015-09-15 07:42:31 +01:00
2203766f77
Retry listing in tests to work around eventual consistency
2015-09-14 21:01:25 +01:00
235cbe0e57
amazon cloud drive: retry 409 errors too
2015-09-14 21:00:44 +01:00
f50f353b5d
local: always use UNC paths on Windows - fixes #124 , fixes #130 , fixes #90
...
* Convert all paths to UNC paths on Windows.
* Update local filesystem to always use UNC paths.
* Change tests, so they can work with Windows character replacements.
* Remove "/" suffix on paths.
* Always use path/filepath
2015-09-14 19:58:03 +01:00
00afe6cc96
ACD: Shorten minimum and maximum sleep.
2015-09-14 18:22:21 +01:00
dd48e62b7e
Limit concurrent directory listings.
...
Never run more than "fs.Config.Checkers" directory listings at once.
2015-09-14 18:11:59 +01:00
a1a780e847
Read folders in separate goroutines.
...
As proposed in the FIXME, read folders in parallel.
This appears to fix "Next token is expired" on very big directories.
The only downside is that this doesn't abort at once if an error is found.
I added some logging, so there is some output for "-v".
2015-09-14 18:11:59 +01:00
fa87077211
Retry if a timeout occurs.
...
This happens more rarely than 500, but can still be encountered.
2015-09-14 18:02:35 +01:00
6ac7145d2d
Retry when we get a "500 Internal server error".
...
It seems like this happens randomly, and retrying works fine for that.
2015-09-14 18:02:35 +01:00
f1226f19b2
drive, googlecloudstorage: optionally use auto config for the oauth token
2015-09-12 14:17:39 +01:00
3ecbf2af25
Fix missing link text.
2015-09-12 11:22:17 +02:00
79f2e95bf9
Add Klaus Post to Contributors
2015-09-11 20:20:28 +01:00
faee50b238
List multiple matched commands.
...
* When multiple commands match, list them all.
* Change example to an existing command.
2015-09-11 19:44:50 +01:00
