Nick Craig-Wood
11da2a6c9b
Break the fs package up into smaller parts.
...
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.
The new code layout is documented in CONTRIBUTING.
2018-01-15 17:51:14 +00:00
Nick Craig-Wood
b8b620f5c2
Move all backends into backend directory
2018-01-12 17:07:38 +00:00
Nick Craig-Wood
29d34426bc
vfs: fix deletion of in use directories #1860
...
This was causing errors if the cache cleaner was called between the
Open and the pendingOpen of a RW file.
The fix was to move the cache open to the Open from the openPending.
2017-12-15 15:42:49 +00:00
Nick Craig-Wood
38b85e94ea
vfs: rename --cache-* options to --vfs-cache-* to save confusion
...
..as the backend cache options are all called --cache-* too. Adjust
docs to point out what the vfs cache does vs the backend cache.
2017-12-03 12:14:15 +00:00
Nick Craig-Wood
c16ac697a9
vfs: keep track of directories in the cache also #1860
...
This makes managing empty directories more reliable.
2017-11-28 20:39:23 +00:00
Nick Craig-Wood
0978957a2e
vfs: make sure all 96 combinations of flags for Open work
2017-11-28 20:39:23 +00:00
Nick Craig-Wood
d1b19f975d
vfs: remove items from cache when deleted #1860
...
Also fixes Error message when items have been deleted from the cache
(eg when Moved) when the cache reaper comes to delete them.
2017-11-28 16:13:58 +00:00
Nick Craig-Wood
3fb4fe31d2
vfs: make sure write only handles never truncate files they shouldn't
2017-11-20 19:42:25 +00:00
Nick Craig-Wood
76b151984c
vfs: cache the size of the object in the read handle
2017-11-20 17:57:13 +00:00
Nick Craig-Wood
60945d0a37
vfs: remove misleading comment
2017-11-20 12:46:23 +00:00
Nick Craig-Wood
b4083b4371
vfs: rename Fsync to Sync and implement Sync on Node and Handle
2017-11-20 12:46:23 +00:00
Nick Craig-Wood
9738f8532b
vfs: Add FlushDirCache method
2017-11-20 12:46:23 +00:00
Nick Craig-Wood
a5b034a992
vfs: add WaitForWriters to wait until all writers have finished
2017-11-20 12:46:23 +00:00
Nick Craig-Wood
321b6da7af
vfs: don't remove file from writers until it is transferred
...
This means that the list of active writers is up to date
2017-11-20 12:46:23 +00:00
Nick Craig-Wood
1b22ee5b93
vfs: fix error handling in openPending so it returns the correct error
2017-11-20 12:46:23 +00:00
Nick Craig-Wood
eab55ce882
vfs: add open files to directories
2017-11-20 12:46:23 +00:00
Nick Craig-Wood
c560017934
vfs: add Path method to Node and use it to stop reading nil DirEntry
...
All DirEntry calls now have been checked for nil or converted to use Path.
2017-11-20 12:46:22 +00:00
Nick Craig-Wood
992647b157
vfs: Don't error a r/w file open without cache; delay error until Read called
...
If we open a file for r/w without the cache we now always return a
handle and return an error if the file is ever read from. This fixes
incompatibility with cmount under windows.
2017-11-20 12:46:22 +00:00
Nick Craig-Wood
dec21ccf63
vfs, cmount: make truncate work properly in the presence or otherwise of open files
2017-11-20 12:46:22 +00:00
Nick Craig-Wood
e7f2935333
vfs: decode flags in Open/OpenFile for debug
2017-11-20 12:46:22 +00:00
Nick Craig-Wood
8951875c21
vfs,mount,cmount,mountlib: allow flags to be overriden by environment variables
2017-11-20 12:46:21 +00:00
Nick Craig-Wood
05a1e1532b
vfs,mount,cmount,serve: Add documentation for vfs caching modes
2017-11-20 12:46:21 +00:00
Nick Craig-Wood
7f20e1d7f3
vfs: add read write files and caching #711
...
This adds new flags to mount, cmount, serve *
--cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
--cache-mode string Cache mode off|minimal|writes|full (default "off")
--cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
2017-11-20 12:36:50 +00:00
Nick Craig-Wood
a0cfa0929b
vfs: remove un-needed (after introduction of rcat) createInfo struct
2017-11-20 12:00:32 +00:00
Nick Craig-Wood
3fb1e96988
vfs: factor Open logic from Dir.Create into vfs.OpenFile
2017-11-20 12:00:32 +00:00
Fabian Möller
5acb6f47e7
mountlib: log when poll-interval is ineffective
...
Notify the user in case poll-interval is used on a unsupported remote
2017-11-19 20:08:14 +00:00
Nick Craig-Wood
c31556c6d1
vfs: Make sure all public methods are locked in Read and Write Handle
2017-11-04 10:24:10 +00:00
Nick Craig-Wood
2083ac6e2a
vfs: add ECLOSED and tidy errors
2017-11-04 10:24:10 +00:00
Nick Craig-Wood
22ee839d05
cmount,vfs: unify Read and Write handles and File and Dir where possible
2017-11-04 10:24:10 +00:00
Nick Craig-Wood
5634659ea3
mount,vfs: unify Read and Write handles in preparation for ReadWrite handles
2017-11-04 10:24:10 +00:00
Nick Craig-Wood
e18122e88b
vfs: add tests and subsequent fixes
...
* Tests for VFS layer
* Small fixes found during testing
* Fix Close, Flush and Release behaviour for ReadFileHandle and WriteFileHandle
* Fix nil object bugs on File
2017-11-04 10:24:10 +00:00
Nick Craig-Wood
8184ec4b70
vfs: add EPERM to errors
2017-11-03 13:00:00 +00:00
Nick Craig-Wood
190367d917
vfs: factor duplicated Open code into vfs from mount/cmount
2017-11-03 13:00:00 +00:00
Nick Craig-Wood
a5dc62f6c1
vfs: Make file handles compatible with OS
...
* Implement directory handles
* Unify OpenFile
* Add all the methods to match *os.File
* Add StatParent and Rename methods to VFS
2017-11-03 13:00:00 +00:00
Nick Craig-Wood
3e0c91ba4b
vfs: Move DefaultOpt to vfs and make some methods private
2017-11-03 13:00:00 +00:00
Nick Craig-Wood
7e065440fb
vfs: rename Lookup to Stat to be more in keeping with os
2017-11-03 12:59:59 +00:00
Nick Craig-Wood
e8883e9fdb
vfs: factor flags into vfsflags and remove global variables
2017-11-03 12:59:59 +00:00
Nick Craig-Wood
1a8f824bad
vfs: use os package errors where possible
2017-11-03 12:59:59 +00:00
Nick Craig-Wood
c1aaff220d
Factor new vfs module out of cmd/mountlib
...
This is an OS style file system abstraction with directory caching
used in mount, cmount, serve webdav and serve http.
2017-11-03 12:59:59 +00:00
