Nick Craig-Wood
8369b5209f
swift: Make sure we read the size for 0 length files - Fixes #237
...
This was causing a problem with sync for chunked files. The directory
listing would read their size back as 0 and see that the size had
changed and immediately resync it.
2015-12-17 13:30:58 +00:00
Nick Craig-Wood
fcea3777c0
Implement Hubic storage system - fixes #200
2015-11-14 08:08:52 +00:00
Nick Craig-Wood
e8ba43c479
swift: Use ContentType from Object to avoid lookups in listings - fixes #208
2015-11-11 17:19:57 +00:00
Nick Craig-Wood
5bb76cc35c
Stop SetModTime losing metadata (eg X-Object-Manifest) - fixes #203
2015-11-11 17:00:08 +00:00
Nick Craig-Wood
a795d93bc3
swift, s3, googlecloudstorage: Don't delete the container/bucket if fs wasn't at root - fixes #172
2015-11-07 15:32:40 +00:00
Nick Craig-Wood
5df04cb763
swift: ignore directory marker objects where appropriate - fixes #190
...
* When creating a LimitedFs
* When calling List() to list files
* In the Storable() method
* Add a Purge() method to delete the directory marker objects too
This is a partial fix for #172
2015-11-07 15:32:11 +00:00
Nick Craig-Wood
b257de4aba
Be more constistent with naming in remotes
...
* External objects are called Fs and Object
* Object.fs always points to the Fs
2015-11-07 11:14:46 +00:00
Nick Craig-Wood
95fd79faf9
swift: use Content-Length on uploads - fixes #125
2015-10-03 16:04:51 +01:00
Nick Craig-Wood
b79dc01016
swift: stop chunked operations logging "Failed to read info: Object Not Found"
2015-10-03 16:04:51 +01:00
Nick Craig-Wood
4a0a42c2f1
swift: large file upload fixes
...
* Read metadata in file listing for 0 length files to fix syncs
* Ignore non-existent files in isManifestFile to fix errors on copy
* remove nsToSwiftFloatString - experiments with the swift program
indicate that it puts a variable number of points after the
decimal, so might as well use the one in the swift library.
* Make sure segments get deleted properly when move from segmented
to non segmented and vice versa
* Use internal list routine to detect errors on listing
* Remove the _segments container if possible
* Remove manifest first when deleting
2015-09-26 17:58:04 +01:00
Sergey Tolmachev
cc7b9af50e
swift: support files > 5GB - fixes #46
...
* Write segments to ..._segments container
* Choice of container and segment names compatible with swift tool
* See http://docs.openstack.org/developer/swift/overview_large_objects.html
* Controlled by command line flag --swift-chunk-size
* Segments removed on delete
2015-09-26 13:03:58 +01:00
Nick Craig-Wood
68fef49c55
swift: fetch headers as the only source of metadata
2015-09-26 13:03:58 +01:00
Nick Craig-Wood
e9c915e6fe
Fix golint warnings
2015-09-22 18:47:16 +01:00
Nick Craig-Wood
e2717a031e
Implement Mover and DirMover interfaces fixes #115
...
* unit tests
* local
* drive
* dropbox
2015-09-01 21:49:13 +01:00
Nick Craig-Wood
92745aa950
Add Root() to Fs interface
2015-09-01 21:49:13 +01:00
Nick Craig-Wood
a96b522958
Implement server side copies if possible - fixes #99
...
Add optional fs.Copier interface
Implemented for
* swift
* s3
* drive
* dropbox
* google cloud storage
2015-08-23 21:18:38 +01:00
Nick Craig-Wood
fedf81c2b7
Add Name() to Fs interface to return name as passed to NewFs
2015-08-23 13:36:38 +01:00
Leonid Shalupov
3fcff32524
do not print stats in quiet mode - fixes #70
...
...unless had some errors or stats interval requested.
Add fs.ErrorLog to differentiate between Logs which should be
suppressed and errors which shouldn't.
2015-08-10 11:17:54 +01:00
Nick Craig-Wood
1d254a3674
Implement --timeout and --conntimeout - fixes #54
...
NB dropbox still to do
2015-05-10 11:29:55 +01:00
Nick Craig-Wood
c47c9cd440
swift: add region parameter - fixes #38
2015-03-04 17:09:53 +00:00
Nick Craig-Wood
ee6b39aa6c
Add tenant parameter for swift - fixes #13
2014-12-12 15:26:08 +00:00
Nick Craig-Wood
d3c16608e4
Test Listing the Root of each Fs
2014-07-31 23:20:39 +01:00
Nick Craig-Wood
3e27ff1b95
Add Root List test and fs.Limited tests for single files
2014-07-31 21:35:29 +01:00
Nick Craig-Wood
c389616657
all: make private functions / variables / constant which shouldn't be public
2014-07-29 17:50:07 +01:00
Nick Craig-Wood
457ef2c190
Automatically generate the tests files for each Fs
2014-07-29 17:18:22 +01:00
Nick Craig-Wood
b013c58537
swift: return directories without / in ListDir
2014-07-24 23:13:33 +01:00
Nick Craig-Wood
a5b0d88608
Make tests for each Fs
...
Factor tests out of rclonetest
2014-07-24 23:13:32 +01:00
Nick Craig-Wood
282cba20a0
swift, s3, dropbox: fix metadata read on Update()
...
This was causing changed files to be marked as corrupted on upload
2014-07-20 11:23:05 +01:00
Nick Craig-Wood
5a8bce6353
swift: Read metadata after Update or Put
2014-07-19 13:05:06 +01:00
Nick Craig-Wood
c1a245d1c8
Factor UserAgent to fs and move Version to fs
2014-07-13 19:19:58 +01:00
Nick Craig-Wood
99695d57ab
Implement single file operations for all file systems
2014-05-05 22:17:57 +01:00
Nick Craig-Wood
d0ca58bbb1
swift: Support sub container paths
2014-05-05 18:26:37 +01:00
Nick Craig-Wood
02afcb00e9
Factor Fs.Put into Object.Update and call Update rather than Put if possible
2014-04-18 17:49:01 +01:00
Nick Craig-Wood
92ec29fe3f
Factor the generic code into fs and add some more intefaces
2014-03-28 22:26:42 +00:00
Nick Craig-Wood
2e5f0ef258
Rename filing systems modules (so I can tell them apart in emacs buffers!)
2014-03-16 10:09:55 +00:00
Nick Craig-Wood
0a108832e2
Remove filesystem flags and put in config file with editor
2014-03-15 16:06:11 +00:00
Nick Craig-Wood
a34bf2ae5e
Make imports absolute and use own copy of goamz pending patch inclusion
2013-07-11 17:37:33 +01:00
Nick Craig-Wood
f1a08ad6bf
Namespace the swift options
2013-06-28 18:24:06 +01:00
Nick Craig-Wood
b1f681b912
Rename fs.FsX to fs.X
2013-06-28 08:57:32 +01:00
Nick Craig-Wood
6085dc1b5a
Refactor into sub filesystems
2013-06-27 22:17:51 +01:00
