Commit Graph

464 Commits

Author SHA1 Message Date
Nick Craig-Wood
2857ed5c35 fs: fix --immutable tests on remotes which don't have modtime 2017-09-28 08:56:30 +01:00
Jacob McNamee
2d8e75cab4 Implement --immutable option 2017-09-24 20:00:00 +01:00
Stefan
85877f3adc config: add show/file subcommands which print the config/its path (fixes #1086) 2017-09-19 17:59:19 +02:00
Nick Craig-Wood
87335de8a8 fs: fix filename normalization issues in the tests when running on OS X 2017-09-17 15:31:22 +01:00
Stefan Breunig
12405f9f41 fuse: re-use rcat to support uploads for all remotes (fixes #1672) 2017-09-16 22:49:08 +02:00
Stefan
234bfae0d5 b2: implement streaming upload of files with unknown length (see #1614) (closes #1686) 2017-09-16 22:43:48 +02:00
Nick Craig-Wood
4ac9a65049 fs: stop normalizing file names but do a normalized compare in the sync
This works by using a transform function to transform file names when
doing a compare when matching file names in a directory.  rclone now
UTF-8 normalizes the file names and does a case insensitive compare if
the destination remote is case insensitive.

This deprecates the --local-no-unicode-normalization flag.

Fixes #1477
2017-09-16 19:49:31 +01:00
Nick Craig-Wood
a8e41f081c fs: re-implement check and cryptcheck using the same traversal as sync
This makes them 100% consistent with sync and also make them use less
memory as they no longer build the whole tree in memory first.

Fixes #1657
2017-09-16 19:49:31 +01:00
Nick Craig-Wood
261c7ad9e4 fs: make syncCopyMove use context for go routine cancellation 2017-09-16 19:49:31 +01:00
Nick Craig-Wood
fe96d5cf0a fs: factor multiple directory traverse out of sync 2017-09-16 19:49:31 +01:00
Nick Craig-Wood
6df12b3f00 fs: improve retriable error detection 2017-09-16 19:48:49 +01:00
Stefan Breunig
9d3f11b493 amazonclouddrive, rcat: ensure rcat integration test passes even with AmazonCloudDrive (fixes: #1680) 2017-09-15 18:09:04 +02:00
Nick Craig-Wood
798502b204 fs: add more errors to be considered temporary errors
This makes a framework for adding temporary errors identified by
syscall number or by error string.

Fixes #1660
2017-09-14 18:01:43 +01:00
Stefan Breunig
56dedc49e3 rcat: properly report if the upload fails 2017-09-13 20:21:52 +02:00
Girish Ramakrishnan
4c45cbea18 copy: error out if dst could not be listed 2017-09-12 09:29:44 +01:00
Stefan Breunig
a91448c83a rcat: honor --dry-run even for small files 2017-09-11 22:28:16 +02:00
Stefan Breunig
80b1f2a494 rcat: configurable small files cutoff and implement proper upload verification 2017-09-11 08:26:53 +02:00
Stefan Breunig
57817397a0 rcat: directly upload small files without streaming them 2017-09-11 08:25:34 +02:00
Nick Craig-Wood
cbea06026a Make check obey --ignore-size - fixes #1643 2017-09-01 11:20:41 +01:00
Nick Craig-Wood
2c8d6e86cc fs: fix gofmt 2017-08-31 16:01:19 +01:00
cbruegg
bb6300b032 Fix bwlimit toggle in conjunction with schedules (Fixes #1607) 2017-08-31 15:33:29 +01:00
Stefan Breunig
a122b9fa7a yandex: implement streaming uploads (see #1614) 2017-08-19 14:07:23 +02:00
Alex McGrath Kraak
2cc9071791 http: add --user-agent option. close #1557 2017-08-17 06:49:27 +02:00
Nick Craig-Wood
265fb8a5e2 fs: Manage empty directories - fixes #100
During the sync we collect a list of directories which should be empty
and attempt to rmdir them at the end of the sync.  If the directories
are not empty then the rmdir will fail, logging a message but not
erroring the sync.
2017-08-09 21:07:00 +01:00
Nick Craig-Wood
8a1a900733 fstest: use Feature.CanHaveEmptyDirectories to sharpen tests
Now we actually test whether the directories are present or not,
filtering out empty directories in the test using the
CanHaveEmptyDirectories flag.
2017-08-09 20:55:08 +01:00
Nick Craig-Wood
20ae7d562b fs: Add CanHaveEmptyDirectories and BucketBased feature flags to all remotes 2017-08-09 20:55:08 +01:00
Nick Craig-Wood
ec2ea37ad2 fs: Add --disable flag to disable optional features - fixes #1551
Eg to disable server side copy use `--disable copy`, to see a list of
what you can disable, `--disable help`.
2017-08-07 21:34:45 +01:00
Nick Craig-Wood
bfe812ea6b dedupe: implement merging of duplicate directories - fixes #1243 2017-08-07 15:36:41 +01:00
Nick Craig-Wood
db1995e63a Add MergeDirs optional interface and implement it for drive 2017-08-07 15:32:47 +01:00
Nick Craig-Wood
81a2ab599f fs: add optional ID to fs.Directory and set it in the remotes which care 2017-08-07 15:31:22 +01:00
Stefan Breunig
40f24e0ea3 config: use absolute ConfigPath to ensure newly written config is on the same mount - fixes #1569 2017-08-05 12:13:25 +02:00
Nick Craig-Wood
92d2e1f8d7 azureblob: rework and complete #801
* Fixup bitrot (rclone and Azure library)
  * Implement Copy
  * Add modtime to metadata under mtime key as RFC3339Nano
  * Make multipart upload work
  * Make it pass the integration tests
  * Fix uploading of zero length blobs
  * Rename to azureblob as it seems likely we will do azurefile
  * Add docs
2017-08-04 22:56:16 +01:00
Andrei Dragomir
036fd61a50 Added Azure Blob storage support #801 2017-08-04 22:54:27 +01:00
wuyu
ec5b72f8d5 Add new QingStor remote
Add new package qingstor to support QingStor API.

Add new unit test for its and tested through; But I commented
on some tests case because of some of the features of QingStor.

Add new docs for it.
2017-08-04 17:25:47 +01:00
Nick Craig-Wood
f682002b84 fs: Make tests create a new bucket rather than purging the old one
This enables QingStor to pass the tests as it has a 2 minute lockout
on deleting the old bucket then creating it again.
2017-08-04 17:09:28 +01:00
Stefan Breunig
28a18303f3 implement rcat – fixes #230, fixes #1001 2017-08-03 21:42:35 +02:00
Nick Craig-Wood
3e3a59768e fs/test_all: fix after fstest factorisation 2017-08-03 20:01:05 +01:00
Nick Craig-Wood
e01741b557 fs: Cleaning up directories in test is no longer needed
..as it is done in the finalise method.
2017-08-03 20:01:05 +01:00
Nick Craig-Wood
71fe046937 fs: Add Find method to DirTree 2017-07-26 16:38:53 +01:00
Nick Craig-Wood
d5ff7104e5 fs: Implement NewDirTree for non --fast-list 2017-07-26 16:38:44 +01:00
Nick Craig-Wood
cd4895690a fstest: Factor test initialisation into Initialise() 2017-07-26 16:38:33 +01:00
Nick Craig-Wood
1ecf2bcbd5 fs: fix typo in --bind description 2017-07-23 23:08:33 +01:00
Nick Craig-Wood
c3d6cc91ec Fix --bind flag changes under go1.6
Correcting 9f24639568
2017-07-23 22:36:32 +01:00
Nick Craig-Wood
9f24639568 Add --bind flag for choosing the local addr on outgoing connections - fixes #1087
Supported by all remotes except FTP.
2017-07-23 16:27:39 +01:00
Nick Craig-Wood
7b81f12dad box: add docs
* reorder remotes so they are in alphabetical order by full name everywhere
  * update CONTRIBUTING doc
2017-07-23 11:32:34 +01:00
Nick Craig-Wood
d279161cee Implement box storage remote - #97 2017-07-23 11:32:34 +01:00
Nick Craig-Wood
db6009126d Fix test failure with new stretchr/testify - fixes #1550 2017-07-23 08:59:07 +01:00
Nick Craig-Wood
c8b5ee1e54 Start v1.37-DEV development 2017-07-22 20:43:06 +01:00
Nick Craig-Wood
a73ecec11f Version v1.37 2017-07-22 20:04:29 +01:00
Nick Craig-Wood
0cf19ef66a Make ListDirSorted check for subdirectories and write test 2017-07-19 09:36:27 +01:00