Commit Graph

6084 Commits

Author SHA1 Message Date
Nick Craig-Wood
83f0d3e03d acd: remove 409 conflict from error codes we will retry
This should fix the very long pauses or getting stuck people have seen
in uploads.
2016-01-23 17:02:09 +00:00
Nick Craig-Wood
eda4130703 Fix integration tests so they can be run independently and out of order - fixes #291
* Make all integration tests start with an empty remote
  * Add an -individual flag so this can be a different bucket/container/directory
  * Fix up tests after changing the hashers
  * Add sha1sum test
  * Make directory checking in tests sleep more to fix acd inconsistencies
  * Factor integration tests to make more maintainable
  * Ensure remote writes have a fstest.CheckItems() before use
    * this fixes eventual consistency on the directory listings later
  * Call fs.Stats.ResetCounters() before every fs.Sync()

Note that the tests shouldn't be run concurrently as fs.Config is global state.
2016-01-23 17:02:09 +00:00
Nick Craig-Wood
ccba859812 Test all available hashes for each remote 2016-01-23 09:10:36 +00:00
Nick Craig-Wood
de3cf5e8d7 Add -verbose flag to unit tests and add some more eventual consistency retries 2016-01-20 20:06:05 +00:00
Nick Craig-Wood
ce305321b6 amazon cloud drive: Fix "Next token is expired" - Fixes #289 Fixes #263
This should also fix the consequent "409 Conflict" name already exists errors.
2016-01-20 20:05:52 +00:00
Nick Craig-Wood
e6117e978e Add Werner Beroux to contributors 2016-01-20 16:33:28 +00:00
Werner Beroux
4b40898743 Update filtering.md
Clarify by removing the extension which makes it confusing if not careful.
2016-01-20 16:16:24 +01:00
Nick Craig-Wood
ae3a0ec27e b2: Don't re-read the SHA1 if we already have it 2016-01-19 08:21:20 +00:00
Nick Craig-Wood
d9458fb4ee b2: return error in Hash from readFileMetadata operation 2016-01-19 08:21:10 +00:00
Nick Craig-Wood
27f67edb1a Fix formatting problem in sha1sum 2016-01-17 13:56:42 +00:00
Nick Craig-Wood
3ffea738e6 Make hash constants start from 1 not 2 2016-01-17 10:47:24 +00:00
Nick Craig-Wood
a63dd6020c onedrive: fix incorrectly decoded SHA-1 2016-01-17 10:46:36 +00:00
Nick Craig-Wood
d0678bc3e5 local: report error on stat in Hash in case file disappeared 2016-01-17 10:46:19 +00:00
klauspost
ce04a073ef Update templates to changes in the latest hugo version
Fixes #295
2016-01-16 14:11:52 +00:00
Nick Craig-Wood
c337a367f3 Make make serve fail if make website would fail 2016-01-16 14:10:57 +00:00
klauspost
7ae40cb352 Update information on revised hash functionality. 2016-01-16 10:17:11 +00:00
Nick Craig-Wood
e8daab7971 Fix integration tests for remotes with unsupported hash schemes 2016-01-16 09:45:15 +00:00
klauspost
78c3a5ccfa Add support for multiple hash types.
Add support for multiple hash types with negotiation of common hash types for comparison.

Manually rebased version of #277 (see discussion there)
2016-01-11 13:39:33 +01:00
Nick Craig-Wood
2142c75846 Add missing docs for options - fixes #278 2016-01-10 12:04:20 +00:00
Nick Craig-Wood
c724d8f614 dropbox: Make file exclusion error controllable with -q #287 2016-01-10 11:49:04 +00:00
Nick Craig-Wood
af5f4ee724 Make --include rules add their implict exclude * at the end of the filter list
This means you can mix `--include` and `--include-from` with the
other filters (eg `--exclude`) but you must include all the files you
want in the include statement.

Fixes #280
2016-01-10 11:42:53 +00:00
Nick Craig-Wood
01aa4394a6 Explain that errored sync doesn't delete files - fixes #285 2016-01-10 10:44:33 +00:00
Nick Craig-Wood
2646519712 Add --memprofile flag 2016-01-09 15:25:48 +00:00
Nick Craig-Wood
5b2efd563a Add Xavier Lucas to contributors 2016-01-08 08:32:52 +00:00
xlucas
e7b7432079 OVH Swift authentication enpoint 2016-01-08 08:30:13 +00:00
Nick Craig-Wood
ea2ef4443b Remove -verbose from errcheck 2016-01-08 08:20:04 +00:00
klauspost
25f22ec561 Add "--ignore-existing" flag.
Add option to completely ignore existing files and not consider them for transfer.

Fixes #274
2016-01-08 08:20:04 +00:00
Nick Craig-Wood
5189231a34 Tweaks to rclone authorize
* Document the headless / remote setup procedure
  * Move Config constants into fs
  * Parse arguments in main for Authorize
2016-01-07 20:31:23 +00:00
klauspost
bcbd30bb8a Add easier headless configuration.
This will allow setting up a remote with copy&paste of values to a headless machine. It will allow copy+pasting a token into the configuration.

This requires rclone to be on a machine with a proper browser. Custom client id and secrets are supported.

To test token generation, use `rclone auth "fs type"`.
2016-01-07 20:31:23 +00:00
Nick Craig-Wood
c245183101 Stop errcheck running for go < 1.5 2016-01-07 16:37:51 +00:00
klauspost
4ce2a84df0 Document workaround for ACD maximum file size.
Document workaround for ACD maximum file size and display a warning in verbose mode before upload starts.

Fixes #215.
2016-01-05 17:12:16 +00:00
klauspost
3c31d711b3 Add local file system option to disable UNC on Windows.
This will add an option to disable UNC conversion on Windows to deal with buggy file system implementations like EncFS.

Fixes #261
2016-01-05 17:08:11 +00:00
Nick Craig-Wood
3f5d8390ba Add Björn Harrtell to contributors 2016-01-05 17:05:31 +00:00
Björn Harrtell
78edafcaac drive: add --drive-auth-owner-only to only consider files owned by the user. 2016-01-05 17:02:04 +00:00
Nick Craig-Wood
1ce3673006 Add -clean flag to test_all.go to clean left over test directories 2016-01-03 21:49:26 +00:00
Nick Craig-Wood
3423de65fa Make canonical place for all fs in fs/all/all.go 2016-01-03 14:12:45 +00:00
Nick Craig-Wood
0c81439bc3 Fix upload_github target 2016-01-02 12:18:32 +00:00
Nick Craig-Wood
77fb8ac240 Version 1.26 2016-01-02 12:04:32 +00:00
Nick Craig-Wood
979dfb8cc6 Add Joseph Spurrier to contributors 2016-01-02 11:50:49 +00:00
Joseph Spurrier
fe0289f2f5 s3: Fix corrupting Content-Type on mod time update
This fixes an issue where updating the modification time resets the
content-type to the S3 default of binary/octet-stream which breaks
static websites that expect an html file to have a content-type of
text/html.
2016-01-02 11:47:52 +00:00
Nick Craig-Wood
6a64567dd7 Add Dmitry Burdeev (dibu) to contributors 2016-01-02 11:45:30 +00:00
Nick Craig-Wood
8de8cd62ca yandex: stop create folder error being fatal 2015-12-30 21:07:42 +00:00
Nick Craig-Wood
cba27d2920 yandex: correct precision to 1ns 2015-12-30 20:47:44 +00:00
Nick Craig-Wood
9ade179407 yandex: Fix socket leaks 2015-12-30 14:30:16 +00:00
Nick Craig-Wood
82b85431bd yandex: Make it use our http client so logging, bwlimit etc works properly 2015-12-30 14:30:16 +00:00
Nick Craig-Wood
98778b1870 Docs for Yandex 2015-12-30 14:30:16 +00:00
Nick Craig-Wood
dfd46c23f9 Fix forgotten update for test_all.go 2015-12-30 12:12:24 +00:00
dibu28
3ac4407b88 Implement Yandex storage backend - fixes #234 2015-12-30 12:11:46 +00:00
Nick Craig-Wood
8ea0d5212f Add -verbose flag to test_all and fix tries count 2015-12-30 11:34:22 +00:00
Nick Craig-Wood
acd350d833 Add retry for eventual consistency in findObject test 2015-12-30 10:46:04 +00:00