Commit Graph

32 Commits

Author SHA1 Message Date
Nick Craig-Wood
b257de4aba Be more constistent with naming in remotes
* External objects are called Fs and Object
  * Object.fs always points to the Fs
2015-11-07 11:14:46 +00:00
Nick Craig-Wood
bf562d7373 Tweak wording on client/secret ids in acd, gcs and drive - fixes #155 2015-10-03 16:04:45 +01:00
Nick Craig-Wood
e9c915e6fe Fix golint warnings 2015-09-22 18:47:16 +01:00
Nick Craig-Wood
2ed158aba3 Fixes from go vet and errcheck 2015-09-22 07:31:12 +01:00
Nick Craig-Wood
c98a51b26c Lightly obscure secrets 2015-09-01 22:33:34 +01:00
Nick Craig-Wood
e2717a031e Implement Mover and DirMover interfaces fixes #115
* unit tests
  * local
  * drive
  * dropbox
2015-09-01 21:49:13 +01:00
Nick Craig-Wood
92745aa950 Add Root() to Fs interface 2015-09-01 21:49:13 +01:00
Nick Craig-Wood
0168f55f3e Switch to spf13 fork of pflag - fixes #116
This supports --long value as well as --long=value which is as
expected for a unix utility.
2015-08-29 18:14:24 +01:00
Nick Craig-Wood
8b60ab86a1 dropbox: force use of our custom transport which makes timeouts work 2015-08-29 17:48:15 +01:00
Nick Craig-Wood
8e3d45d2dc dropbox: increase chunk size to improve upload speeds - fixes #103
Chunks aren't buffered in memory, so chose 128M as the default size as
producing the maximum throughput.  This takes the throughput from 78
kBytes/s to 4MBytes/s a 50x improvement!
2015-08-25 19:01:37 +01:00
Nick Craig-Wood
a96b522958 Implement server side copies if possible - fixes #99
Add optional fs.Copier interface

Implemented for
  * swift
  * s3
  * drive
  * dropbox
  * google cloud storage
2015-08-23 21:18:38 +01:00
Nick Craig-Wood
fedf81c2b7 Add Name() to Fs interface to return name as passed to NewFs 2015-08-23 13:36:38 +01:00
Nick Craig-Wood
bc19e2d84b dropbox: Issue an error message when trying to upload bad file name - fixes #108 2015-08-20 20:46:35 +01:00
Nick Craig-Wood
1d9e76bb0f dropbox: remove datastore - Fixes #55 #84
This means that dropbox no longer stores MD5SUMs and modified times.

Fixup the tests so that blank MD5SUMs are ignored, and that if
Precision is set to a fs.ModTimeNotSupported, ModTimes can be ignored too.

This opens the door for other FSs which don't support metadata easily.
2015-08-17 17:26:36 +01:00
Leonid Shalupov
3fcff32524 do not print stats in quiet mode - fixes #70
...unless had some errors or stats interval requested.

Add fs.ErrorLog to differentiate between Logs which should be
suppressed and errors which shouldn't.
2015-08-10 11:17:54 +01:00
Nick Craig-Wood
472f065ce7 Make sizes in Dropbox 64 bit - fixes #80 2015-08-10 11:17:54 +01:00
Leonid Shalupov
754ce9dec6 dropbox: record names coming from dropbox API, fixes #53 case insensitivity causes duplicated files 2015-06-14 15:23:56 +01:00
Nick Craig-Wood
1d254a3674 Implement --timeout and --conntimeout - fixes #54
NB dropbox still to do
2015-05-10 11:29:55 +01:00
Nick Craig-Wood
51b24a1dc6 dropbox: adapt code to recent library changes 2014-12-23 13:55:22 +00:00
Nick Craig-Wood
c389616657 all: make private functions / variables / constant which shouldn't be public 2014-07-29 17:50:07 +01:00
Nick Craig-Wood
17ffb0855f Fixes after running errcheck 2014-07-25 18:19:49 +01:00
Nick Craig-Wood
282cba20a0 swift, s3, dropbox: fix metadata read on Update()
This was causing changed files to be marked as corrupted on upload
2014-07-20 11:23:05 +01:00
Nick Craig-Wood
2479ce2c8e dropbox: go1.1 compatibility 2014-07-19 15:48:40 +01:00
Nick Craig-Wood
dfc8a375f6 dropbox: Switch to using RFC3339 for time metadata 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
7c9bdb4b7a dropbox: make limited fs work (copy single file) 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
f8bb0d9cc8 dropbox: remove metadata when we remove files 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
b185e104ed dropbox: Fix mkdir on already created directory 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
e57a4c7c0c dropbox: open the datastore in the background 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
d2f187e1a1 dropbox: Use /delta to list objects - much quicker
Also fix major performance problem - re-reading entry each time!
2014-07-15 19:27:42 +01:00
Nick Craig-Wood
c9aca33030 dropbox: Fix concurrent access to Dropbox datastore and Lower case keys in datastore 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
2b0911531c dropbox: basics of metadata in Dropbox datastore working 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
2149185fc2 dropbox: Initial support of full Fs interface
Still missing metadata support (eg SetModTime)
2014-07-15 19:27:42 +01:00