Commit Graph

86 Commits

Author SHA1 Message Date
57d5de6fba build: fix up package paths after repo move
git grep -l github.com/ncw/rclone | xargs -d'\n' perl -i~ -lpe 's|github.com/ncw/rclone|github.com/rclone/rclone|g'
goimports -w `find . -name \*.go`
2019-07-28 18:47:38 +01:00
f78cd1e043 Add context propagation to rclone
- Change rclone/fs interfaces to accept context.Context
- Update interface implementations to use context.Context
- Change top level usage to propagate context to lover level functions

Context propagation is needed for stopping transfers and passing other
request-scoped values.
2019-06-19 11:59:46 +01:00
a43ed567ee vfs: implement --vfs-cache-max-size to limit the total size of the cache 2019-02-02 12:30:10 +00:00
554ee0d963 vfs: add --dir-perms and --file-perms flags - fixes #2897
This allows files to be shown with the execute bit which allows
binaries to be run under Windows and Linux.
2019-01-08 17:29:38 +00:00
1d14972e41 vfs: reduce directory cache cleared by poll-interval
Reduce the number of nodes purged from the dir-cache when ForgetPath is
called. This is done by only forgetting the cache of the received path
and invalidating the parent folder cache by resetting *Dir.read.

The parent will read the listing on the next access and reuse the
dir-cache of entries in *Dir.items.
2018-10-02 10:21:14 +01:00
1eec59e091 fs: update ChangeNotifier interface
This introduces a channel to the ChangeNotify function, which can be
used to update the poll-interval and cleanly exit the polling function.
2018-09-25 14:01:13 +02:00
7983b6bdca vfs: enable vfs-read-chunk-size by default 2018-07-29 18:17:05 +01:00
98bf65c43b vfs: fix ChangeNotify for new or changed folders
Fixes #2251
2018-05-05 12:54:03 +01:00
bd3ad1ac3e vfs: add option to read source files in chunks 2018-05-05 12:49:42 +01:00
2b855751fc vfs,mount,cmount: use About to return the correct disk total/used/free
Disks total, used, free now shows correctly for mount and cmount (eg
`df` for Unix or in the Windows explorer).
2018-04-18 18:27:34 +01:00
2b7957cc74 vfs: Only make the VFS cache if --vfs-cache-mode > Off
This stops the cache cleaner running unnecessarily and saves
resources.

This also helps with issue #2227 which was caused by a second mount
deleting objects in the first mounts cache.
2018-04-16 17:06:41 +01:00
80588a5a6b Replace "golang.org/x/net/context" with "context" for go1.7+ #2154 2018-04-07 11:42:08 +01:00
2db0c4dd95 vfs: add remote control for directory cache flushing 2018-03-17 10:34:02 +00:00
70f07fd3ac fs: add ChangeNotify and backend support for it (#2094)
* fs: rename DirChangeNotify to ChangeNotify

* cache: switch to ChangeNotify

* ChangeNotify: keep order of notifications
2018-03-08 22:03:34 +02:00
54deb01f00 vfs: Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC
Before this change Open("name", os.O_RDONLY|os.O_TRUNC) would have
truncated the file.  This is what Linux does, but is counterintuitive.
POSIX states this is undefined, so return an error in this case
instead.  This preserves the invariant O_RDONLY => file is not
changed.
2018-02-26 17:04:27 +00:00
04a0a7406b vfs: downgrade "poll-interval is not supported" message to Info
...to save confusion as it isn't very important
2018-02-09 07:57:50 +00:00
a80d8a21dc vfs: add flags parameter to Dir.Create 2018-01-19 13:18:40 +00:00
11da2a6c9b Break the fs package up into smaller parts.
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.

The new code layout is documented in CONTRIBUTING.
2018-01-15 17:51:14 +00:00
b4083b4371 vfs: rename Fsync to Sync and implement Sync on Node and Handle 2017-11-20 12:46:23 +00:00
9738f8532b vfs: Add FlushDirCache method 2017-11-20 12:46:23 +00:00
a5b034a992 vfs: add WaitForWriters to wait until all writers have finished 2017-11-20 12:46:23 +00:00
c560017934 vfs: add Path method to Node and use it to stop reading nil DirEntry
All DirEntry calls now have been checked for nil or converted to use Path.
2017-11-20 12:46:22 +00:00
e7f2935333 vfs: decode flags in Open/OpenFile for debug 2017-11-20 12:46:22 +00:00
7f20e1d7f3 vfs: add read write files and caching #711
This adds new flags to mount, cmount, serve *

    --cache-max-age duration         Max age of objects in the cache. (default 1h0m0s)
    --cache-mode string              Cache mode off|minimal|writes|full (default "off")
    --cache-poll-interval duration   Interval to poll the cache for stale objects. (default 1m0s)
2017-11-20 12:36:50 +00:00
3fb1e96988 vfs: factor Open logic from Dir.Create into vfs.OpenFile 2017-11-20 12:00:32 +00:00
5acb6f47e7 mountlib: log when poll-interval is ineffective
Notify the user in case poll-interval is used on a unsupported remote
2017-11-19 20:08:14 +00:00
2083ac6e2a vfs: add ECLOSED and tidy errors 2017-11-04 10:24:10 +00:00
22ee839d05 cmount,vfs: unify Read and Write handles and File and Dir where possible 2017-11-04 10:24:10 +00:00
5634659ea3 mount,vfs: unify Read and Write handles in preparation for ReadWrite handles 2017-11-04 10:24:10 +00:00
e18122e88b vfs: add tests and subsequent fixes
* Tests for VFS layer
  * Small fixes found during testing
  * Fix Close, Flush and Release behaviour for ReadFileHandle and WriteFileHandle
  * Fix nil object bugs on File
2017-11-04 10:24:10 +00:00
190367d917 vfs: factor duplicated Open code into vfs from mount/cmount 2017-11-03 13:00:00 +00:00
a5dc62f6c1 vfs: Make file handles compatible with OS
* Implement directory handles
  * Unify OpenFile
  * Add all the methods to match *os.File
  * Add StatParent and Rename methods to VFS
2017-11-03 13:00:00 +00:00
3e0c91ba4b vfs: Move DefaultOpt to vfs and make some methods private 2017-11-03 13:00:00 +00:00
7e065440fb vfs: rename Lookup to Stat to be more in keeping with os 2017-11-03 12:59:59 +00:00
e8883e9fdb vfs: factor flags into vfsflags and remove global variables 2017-11-03 12:59:59 +00:00
c1aaff220d Factor new vfs module out of cmd/mountlib
This is an OS style file system abstraction with directory caching
used in mount, cmount, serve webdav and serve http.
2017-11-03 12:59:59 +00:00