Commit Graph

84 Commits

Author SHA1 Message Date
klauspost
78c3a5ccfa Add support for multiple hash types.
Add support for multiple hash types with negotiation of common hash types for comparison.

Manually rebased version of #277 (see discussion there)
2016-01-11 13:39:33 +01:00
Nick Craig-Wood
c724d8f614 dropbox: Make file exclusion error controllable with -q #287 2016-01-10 11:49:04 +00:00
Nick Craig-Wood
b257de4aba Be more constistent with naming in remotes
* External objects are called Fs and Object
  * Object.fs always points to the Fs
2015-11-07 11:14:46 +00:00
Nick Craig-Wood
bf562d7373 Tweak wording on client/secret ids in acd, gcs and drive - fixes #155 2015-10-03 16:04:45 +01:00
Nick Craig-Wood
e9c915e6fe Fix golint warnings 2015-09-22 18:47:16 +01:00
Nick Craig-Wood
2ed158aba3 Fixes from go vet and errcheck 2015-09-22 07:31:12 +01:00
Nick Craig-Wood
c98a51b26c Lightly obscure secrets 2015-09-01 22:33:34 +01:00
Nick Craig-Wood
e2717a031e Implement Mover and DirMover interfaces fixes #115
* unit tests
  * local
  * drive
  * dropbox
2015-09-01 21:49:13 +01:00
Nick Craig-Wood
92745aa950 Add Root() to Fs interface 2015-09-01 21:49:13 +01:00
Nick Craig-Wood
0168f55f3e Switch to spf13 fork of pflag - fixes #116
This supports --long value as well as --long=value which is as
expected for a unix utility.
2015-08-29 18:14:24 +01:00
Nick Craig-Wood
8b60ab86a1 dropbox: force use of our custom transport which makes timeouts work 2015-08-29 17:48:15 +01:00
Nick Craig-Wood
8e3d45d2dc dropbox: increase chunk size to improve upload speeds - fixes #103
Chunks aren't buffered in memory, so chose 128M as the default size as
producing the maximum throughput.  This takes the throughput from 78
kBytes/s to 4MBytes/s a 50x improvement!
2015-08-25 19:01:37 +01:00
Nick Craig-Wood
a96b522958 Implement server side copies if possible - fixes #99
Add optional fs.Copier interface

Implemented for
  * swift
  * s3
  * drive
  * dropbox
  * google cloud storage
2015-08-23 21:18:38 +01:00
Nick Craig-Wood
fedf81c2b7 Add Name() to Fs interface to return name as passed to NewFs 2015-08-23 13:36:38 +01:00
Nick Craig-Wood
bc19e2d84b dropbox: Issue an error message when trying to upload bad file name - fixes #108 2015-08-20 20:46:35 +01:00
Nick Craig-Wood
1d9e76bb0f dropbox: remove datastore - Fixes #55 #84
This means that dropbox no longer stores MD5SUMs and modified times.

Fixup the tests so that blank MD5SUMs are ignored, and that if
Precision is set to a fs.ModTimeNotSupported, ModTimes can be ignored too.

This opens the door for other FSs which don't support metadata easily.
2015-08-17 17:26:36 +01:00
Leonid Shalupov
3fcff32524 do not print stats in quiet mode - fixes #70
...unless had some errors or stats interval requested.

Add fs.ErrorLog to differentiate between Logs which should be
suppressed and errors which shouldn't.
2015-08-10 11:17:54 +01:00
Nick Craig-Wood
472f065ce7 Make sizes in Dropbox 64 bit - fixes #80 2015-08-10 11:17:54 +01:00
Leonid Shalupov
754ce9dec6 dropbox: record names coming from dropbox API, fixes #53 case insensitivity causes duplicated files 2015-06-14 15:23:56 +01:00
Nick Craig-Wood
1d254a3674 Implement --timeout and --conntimeout - fixes #54
NB dropbox still to do
2015-05-10 11:29:55 +01:00
Nick Craig-Wood
51b24a1dc6 dropbox: adapt code to recent library changes 2014-12-23 13:55:22 +00:00
Nick Craig-Wood
c389616657 all: make private functions / variables / constant which shouldn't be public 2014-07-29 17:50:07 +01:00
Nick Craig-Wood
17ffb0855f Fixes after running errcheck 2014-07-25 18:19:49 +01:00
Nick Craig-Wood
282cba20a0 swift, s3, dropbox: fix metadata read on Update()
This was causing changed files to be marked as corrupted on upload
2014-07-20 11:23:05 +01:00
Nick Craig-Wood
2479ce2c8e dropbox: go1.1 compatibility 2014-07-19 15:48:40 +01:00
Nick Craig-Wood
dfc8a375f6 dropbox: Switch to using RFC3339 for time metadata 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
7c9bdb4b7a dropbox: make limited fs work (copy single file) 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
f8bb0d9cc8 dropbox: remove metadata when we remove files 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
b185e104ed dropbox: Fix mkdir on already created directory 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
e57a4c7c0c dropbox: open the datastore in the background 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
d2f187e1a1 dropbox: Use /delta to list objects - much quicker
Also fix major performance problem - re-reading entry each time!
2014-07-15 19:27:42 +01:00
Nick Craig-Wood
c9aca33030 dropbox: Fix concurrent access to Dropbox datastore and Lower case keys in datastore 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
2b0911531c dropbox: basics of metadata in Dropbox datastore working 2014-07-15 19:27:42 +01:00
Nick Craig-Wood
2149185fc2 dropbox: Initial support of full Fs interface
Still missing metadata support (eg SetModTime)
2014-07-15 19:27:42 +01:00