036fd61a50
Added Azure Blob storage support #801
2017-08-04 22:54:27 +01:00
ec5b72f8d5
Add new QingStor remote
...
Add new package qingstor to support QingStor API.
Add new unit test for its and tested through; But I commented
on some tests case because of some of the features of QingStor.
Add new docs for it.
2017-08-04 17:25:47 +01:00
f682002b84
fs: Make tests create a new bucket rather than purging the old one
...
This enables QingStor to pass the tests as it has a 2 minute lockout
on deleting the old bucket then creating it again.
2017-08-04 17:09:28 +01:00
28a18303f3
implement rcat – fixes #230 , fixes #1001
2017-08-03 21:42:35 +02:00
3e3a59768e
fs/test_all: fix after fstest factorisation
2017-08-03 20:01:05 +01:00
e01741b557
fs: Cleaning up directories in test is no longer needed
...
..as it is done in the finalise method.
2017-08-03 20:01:05 +01:00
71fe046937
fs: Add Find method to DirTree
2017-07-26 16:38:53 +01:00
d5ff7104e5
fs: Implement NewDirTree for non --fast-list
2017-07-26 16:38:44 +01:00
cd4895690a
fstest: Factor test initialisation into Initialise()
2017-07-26 16:38:33 +01:00
1ecf2bcbd5
fs: fix typo in --bind description
2017-07-23 23:08:33 +01:00
c3d6cc91ec
Fix --bind flag changes under go1.6
...
Correcting 9f24639568
2017-07-23 22:36:32 +01:00
9f24639568
Add --bind flag for choosing the local addr on outgoing connections - fixes #1087
...
Supported by all remotes except FTP.
2017-07-23 16:27:39 +01:00
7b81f12dad
box: add docs
...
* reorder remotes so they are in alphabetical order by full name everywhere
* update CONTRIBUTING doc
2017-07-23 11:32:34 +01:00
d279161cee
Implement box storage remote - #97
2017-07-23 11:32:34 +01:00
db6009126d
Fix test failure with new stretchr/testify - fixes #1550
2017-07-23 08:59:07 +01:00
c8b5ee1e54
Start v1.37-DEV development
2017-07-22 20:43:06 +01:00
a73ecec11f
Version v1.37
2017-07-22 20:04:29 +01:00
0cf19ef66a
Make ListDirSorted check for subdirectories and write test
2017-07-19 09:36:27 +01:00
655891170f
Check in ListDirSorted that the directory entries all belong
2017-07-18 23:39:42 +01:00
6f71260acf
Add --tpslimit and --tpslimit-burst to limit transactions per second for HTTP
...
This is useful if you are being rate limited or banned by your cloud
storage provider.
2017-07-16 17:25:39 +01:00
62e28d0a72
Replace token bucket limiter github.com/tsenart/tb with golang.org/x/time/rate
...
In tests tsenart/tb has proved inaccurate at low rates.
2017-07-16 16:14:44 +01:00
72add5ab27
sync: state whether duplicates are objects are directories
2017-07-08 15:42:18 +01:00
69ff009264
Use a stable sort for sorting directory entries
...
This is useful if there are duplicates. Assuming the remote delivers
the entries in a consistent order, this will give the best user
experience in syncing as it will consistently use the first entry for
the sync comparison.
2017-07-06 14:07:26 +01:00
27b157580e
Move make_test_files.go into bin
2017-07-06 11:54:57 +01:00
f0512d1a52
Fix missing fs.Dir -> fs.Directory
2017-07-06 11:31:36 +01:00
575e779b55
Warn about duplicate files when syncing - fixes #1506
...
Error about unsorted directories and test thoroughly
2017-06-30 21:24:13 +01:00
e7d04fc103
Create fs.Directory interface and use it everywhere
2017-06-30 14:26:59 +01:00
e2d7d413ef
fs: rename BasicInfo to DirEntry
2017-06-30 14:26:58 +01:00
e7e9aa0dfa
fs: Remove unused ListFser interface
2017-06-30 14:26:58 +01:00
f88300a153
Don't Mkdir at the start of sync - fixes #1131
...
This is possible now that the bucket based remotes will create the
buckets on demand (9c1e703777
2017-06-29 12:31:53 +01:00
e54087ece1
Fix config tests to save configData which fixes subsequent tests
2017-06-29 12:31:53 +01:00
54561fd2bc
s3: work around eventual consistency in bucket creation
...
Deleting a bucket then testing its existence can give the wrong
result. Work around by keeping a flag as to whether we have deleted
the bucket.
2017-06-29 12:31:52 +01:00
f3c7e1a9dd
Debug directory creation and removal - fixes #1192
2017-06-27 22:19:35 +01:00
aa20486485
Add --stats-log-level so can see --stats without -v - fixes #1180
...
The most common use for this flag is likely to be showing the stats
without using -v by using `--stats-log-level NOTICE`.
2017-06-26 22:50:37 +01:00
52b042971a
keep file permissions and try to keep user/group on supported systems ( fixes #1467 )
2017-06-25 09:05:24 +02:00
2d2778eabf
don't delete remote if name does not change while renaming ( fixes #1495 )
2017-06-25 08:55:54 +02:00
6b89e6c381
add new http remote filesystem
...
Signed-off-by: Vasiliy Tolstov <v.tolstov@selfip.ru >
2017-06-20 21:27:14 +01:00
b76cd4abd2
Fix Range header option
2017-06-20 21:27:14 +01:00
4fd9570332
fs: Use an in place filter in ListDirSorted
2017-06-20 21:27:14 +01:00
5250fcdf08
core: fix data race in walk
...
This was detected by the race detector when the client of Walk() sorted entries.
2017-06-15 16:46:32 +01:00
64662bef8d
Deprecate --old-sync-method it is replaced with --fast-list
...
Remove old sync method code.
2017-06-14 16:49:40 +01:00
0b8d9084fc
test_all: print command line so it can be cut and pasted into bash
2017-06-14 16:49:40 +01:00
8a6a8b9623
Change List interface and add ListR optional interface
...
This simplifies the implementation of remotes. The only required
interface is now `List` which is a simple one level directory list.
Optionally remotes may implement `ListR` if they have an efficient way
of doing a recursive list.
2017-06-14 16:49:40 +01:00
6fc88ff32e
Use --fast-list flag for sync/copy/move - fixes #1277
...
Redo test framework to take a -fast-list flag and test remotes with that flag.
2017-06-14 16:49:40 +01:00
50928a5027
Implement --fast-list flag.
...
This is supported remotes which can do a recursive listing. It will
use more memory.
This is related to #1277 but doesn't fix that issue yet.
2017-06-14 16:49:40 +01:00
53c3e5f0ab
Add placeholder support for ListR interface.
...
The ListR interface will be implemented by remotes that can do a
recursive directory listing more efficiently than just recursing
through the directories. These include the bucket based remotes.
2017-06-14 16:49:40 +01:00
0edb025257
Fixup tests with dirs vs bucket based fs
2017-06-14 16:49:40 +01:00
7e20e16cff
core: Implement Walk directory listing and use in place of Lister
...
This is in preparation for removing the Lister code and replacing the
fundamental operation in the Fs with listing a single directory.
2017-06-14 16:49:40 +01:00
68333d34a1
dropbox: make setting mod time on existing files work properly
...
This is a fix left over from the v2 conversion. Dropbox ignores the
client modification on an incoming file if it was identical to the
existing file. This change deletes the existing file first before
re-uploading the new one.
2017-06-13 13:58:39 +01:00
740b3f6ae2
Fix problems found with ineffassign
2017-06-13 11:52:36 +01:00
