Commit Graph

1340 Commits

Author SHA1 Message Date
Nick Craig-Wood
d981456ddc Add Vasiliy Tolstov to contributors 2017-06-20 21:27:14 +01:00
Nick Craig-Wood
b22c4c4307 http: fix, tidy and rework ready for release
* Fix remaining problems
  * Refactor to make testing easier and add a test suite
  * Make path parsing more robust.
  * Add single file operations
  * Add MimeType reading for objects
  * Add documentation
  * Note go1.7+ is required to build
2017-06-20 21:27:14 +01:00
Nick Craig-Wood
afc8cc550a http: Update interfaces for List/ListR/Put/Update 2017-06-20 21:27:14 +01:00
Vasiliy Tolstov
83b642e98f fix for caddy web server
Signed-off-by: Vasiliy Tolstov <v.tolstov@selfip.ru>
2017-06-20 21:27:14 +01:00
Nick Craig-Wood
d5d635b7f3 http: Fix comments, remove optional methods which don't work 2017-06-20 21:27:14 +01:00
Vasiliy Tolstov
6b89e6c381 add new http remote filesystem
Signed-off-by: Vasiliy Tolstov <v.tolstov@selfip.ru>
2017-06-20 21:27:14 +01:00
Nick Craig-Wood
be0dd09801 vendor: golang.org/x/net/html for http 2017-06-20 21:27:14 +01:00
Nick Craig-Wood
b76cd4abd2 Fix Range header option 2017-06-20 21:27:14 +01:00
Nick Craig-Wood
0dbf1230bc Update CONTRIBUTING with --fast-list 2017-06-20 21:27:14 +01:00
Nick Craig-Wood
4fd9570332 fs: Use an in place filter in ListDirSorted 2017-06-20 21:27:14 +01:00
Harshavardhana
8d77e48190 Minio supports ETags and metadata.
Current doc mentioned lack of ETag and metadata
support which since has been long fixed in many
upstream Minio releases.

Also cleanup the doc to show new startup banner etc.
2017-06-20 08:21:02 +01:00
Nick Craig-Wood
dcce65b2b3 mount/cmount: factor duplicated code into mountlib 2017-06-19 14:36:51 +01:00
Nick Craig-Wood
4ce31555b2 vendor: update github.com/billziss-gh/cgofuse - fixes #1481 2017-06-19 09:53:34 +01:00
Nick Craig-Wood
5ed4bc97f3 travis: reduce number of parallel builds to avoid "Killed" error 2017-06-19 08:16:35 +01:00
Nick Craig-Wood
54e37be591 Only test with -race using go latest 2017-06-19 08:07:50 +01:00
Nick Craig-Wood
eaa717b88a Fix crypt obfuscate tests with Windows 2017-06-18 22:53:19 +01:00
Nick Craig-Wood
bbbc202ee6 Add ftp.md to docs builder and update docs 2017-06-15 20:12:26 +01:00
Nick Craig-Wood
97364fd0b6 ncdu: disable on plan9 and solaris as termbox isn't supported there 2017-06-15 20:10:54 +01:00
Nick Craig-Wood
c34f11a92f rclone ncdu for exploring a remote with a text based user interface. 2017-06-15 17:44:17 +01:00
Nick Craig-Wood
e31fc877e2 vendor: github.com/nsf/termbox-go and dependencies for rclone ncdu 2017-06-15 16:46:32 +01:00
Nick Craig-Wood
e069fc439e crypt: use an in place filter for encrypting directory entries 2017-06-15 16:46:32 +01:00
Nick Craig-Wood
5250fcdf08 core: fix data race in walk
This was detected by the race detector when the client of Walk() sorted entries.
2017-06-15 16:46:32 +01:00
Edward Q. Bridges
9876ba53f8 Updated permissions
As it happens, after testing the `GetObject` permission is also required to do `HEAD` requests on a given object.
2017-06-14 17:29:21 +01:00
Nick Craig-Wood
64662bef8d Deprecate --old-sync-method it is replaced with --fast-list
Remove old sync method code.
2017-06-14 16:49:40 +01:00
Nick Craig-Wood
0b8d9084fc test_all: print command line so it can be cut and pasted into bash 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
7be49249d3 Add lsjson command - fixes #1063 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
8a6a8b9623 Change List interface and add ListR optional interface
This simplifies the implementation of remotes.  The only required
interface is now `List` which is a simple one level directory list.

Optionally remotes may implement `ListR` if they have an efficient way
of doing a recursive list.
2017-06-14 16:49:40 +01:00
Nick Craig-Wood
6fc88ff32e Use --fast-list flag for sync/copy/move - fixes #1277
Redo test framework to take a -fast-list flag and test remotes with that flag.
2017-06-14 16:49:40 +01:00
Nick Craig-Wood
50928a5027 Implement --fast-list flag.
This is supported remotes which can do a recursive listing.  It will
use more memory.

This is related to #1277 but doesn't fix that issue yet.
2017-06-14 16:49:40 +01:00
Nick Craig-Wood
3a431056e2 gcs, swift: increase directory listing chunk to 1000 to increase performance 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
53c3e5f0ab Add placeholder support for ListR interface.
The ListR interface will be implemented by remotes that can do a
recursive directory listing more efficiently than just recursing
through the directories.  These include the bucket based remotes.
2017-06-14 16:49:40 +01:00
Nick Craig-Wood
0edb025257 Fixup tests with dirs vs bucket based fs 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
fded4dbea2 yandex: correct error return for listing empty directory 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
7e20e16cff core: Implement Walk directory listing and use in place of Lister
This is in preparation for removing the Lister code and replacing the
fundamental operation in the Fs with listing a single directory.
2017-06-14 16:49:40 +01:00
Nick Craig-Wood
1e88f0702a dropbox: fix oauth configuration
This was broken in c59a292719
2017-06-14 16:46:46 +01:00
Nick Craig-Wood
68333d34a1 dropbox: make setting mod time on existing files work properly
This is a fix left over from the v2 conversion.  Dropbox ignores the
client modification on an incoming file if it was identical to the
existing file.  This change deletes the existing file first before
re-uploading the new one.
2017-06-13 13:58:39 +01:00
Nick Craig-Wood
740b3f6ae2 Fix problems found with ineffassign 2017-06-13 11:52:36 +01:00
Nick Craig-Wood
28fcc53e45 mount test: retry umount as it fails occasionally
This is because of the background releasing of files which happens
after all the files are closed.
2017-06-13 10:52:10 +01:00
Nick Craig-Wood
2ca477c57f swift: make sensible error if the user forgets the container - fixes #1470 2017-06-10 14:44:56 +01:00
Nick Craig-Wood
9a11d3efd9 Revert "Start Cat tests from 2 as onedrive doesn't support ranging from 1"
Now that https://github.com/OneDrive/onedrive-api-docs/issues/543 is
fixed, this can be reverted.

This reverts commit 320c53eab0.
2017-06-10 13:48:00 +01:00
Nick Craig-Wood
10d5377ed8 acd: remove revoked credentials, allow oauth proxy config and update docs 2017-06-10 12:02:34 +01:00
Nick Craig-Wood
ee14efd3c2 config: fix menu selection when no remotes 2017-06-10 11:39:40 +01:00
Nick Craig-Wood
b4be7d65a6 Update build to go1.8.3 2017-06-09 12:06:28 +01:00
Nick Craig-Wood
52e1bfae2a oauth: Allow auth_url and token_url to be set in the config file
If set in the config file, these override the ones configured into the
remote.  This enables alternative oauth servers to be used for all
oauth remotes.  This can only be altered by editing the config file
for the moment.
2017-06-08 20:35:32 +01:00
Nick Craig-Wood
9c1e703777 swift, b2, gcs, s3: Fix moveto and copyto
We now make sure the container/bucket is created before creating any objects.
2017-06-07 14:34:59 +01:00
Nick Craig-Wood
b49821956a Fix copyto/moveto test error (see #1261) 2017-06-07 14:08:46 +01:00
Nick Craig-Wood
a61ba1e7c4 moveto, copyto: report transfers and checks as per move and copy 2017-06-07 13:02:21 +01:00
Nick Craig-Wood
d30cc1e119 Factor RemoteSplit into fs 2017-06-07 12:27:33 +01:00
Nick Craig-Wood
74a3dfc4e1 Fix TestHashSums 2017-06-06 23:21:47 +01:00
Nick Craig-Wood
3fe9448229 drive, acd, onedrive: Cache the directory IDs when reading the parent directory
This makes directory listings much more efficient (one less
transaction needed) and also fixes #1439 (which was caused by having
to look up a directory name with quotes in which isn't dealt with well
by the list routine) by not doing a directory lookup at all.
2017-06-05 12:26:30 +01:00