Commit Graph

23 Commits

Author SHA1 Message Date
Nick Craig-Wood
cb5bd47e61 build: fix errors spotted by ineffassign linter
These were mostly caused by shadowing err and a good fraction of them
will have caused errors not to be propagated properly.
2018-05-05 17:32:41 +01:00
Nick Craig-Wood
c19e675ca6 vfs: unify locking for RWFileHandle.openPending,.close and File.Delete #2141
Without this fix the cached file can be removed as the file is being
uploaded or downloaded.  This can cause the directory listings to
become inconsistent (this issue) or data loss (if a retry was needed
in the Copy).

Remove file needs to be excluded from running at the same time as both
openPending and close so it makes sense to unify the locking between
all 3.
2018-03-15 20:49:07 +00:00
Nick Craig-Wood
5795bd7db6 vfs: update cached copy if we know it has changed even if pending opens
This fixes a problem with Caddy serving corrupted files out of the VFS
cache when the file on the remote changed.
2018-03-04 11:20:22 +00:00
Nick Craig-Wood
9b011ce7e4 vfs: keep track of number of open RWHandles 2018-03-04 11:20:22 +00:00
Nick Craig-Wood
7fb53a031c vfs: don't cache the object in read and read/write handles
This should help with inconsistent reads when the source object changes.
2018-03-04 11:20:22 +00:00
Nick Craig-Wood
b91bd32489 vfs: Fix TestWriteFileDoubleClose with --vfs-cache-mode >= writes
This was causing the file to be closed on Flush() instead of Release()
when the file was opened with O_TRUNC.
2018-02-26 21:26:32 +00:00
Nick Craig-Wood
c3d0f68923 vfs: fix truncation work-around on Windows
This no longer needs to deal with O_RDONLY and O_TRUNC since we
disallow this earlier.  This also fixes the code to just do it for
O_APPEND, not for everything.
2018-02-26 19:46:38 +00:00
Nick Craig-Wood
f57e92b9a5 vfs: fix creation of files when truncating #2083
As spotted by @B4dM4n
2018-02-26 19:37:58 +00:00
Nick Craig-Wood
baf9ee5cf7 vfs: update cached copy if we know it has changed before using it
Before this change we would have to wait for the --vfs-cache-max-age
to expire before getting an update.
2018-02-26 18:00:51 +00:00
Nick Craig-Wood
354f1ad722 vfs: Use operations.Copy instead of CopyFile for efficiency 2018-02-26 17:54:18 +00:00
Nick Craig-Wood
9252224d82 vfs: don't open the file when using a RW handle for a null Seek
Background: cmd/mount/file.go Open() function does a Seek(0, 1) to see
if the file handle is seekable to set a FUSE hint.  Before this change
the file was downloaded before it needed to be which was inefficient
(and broke beta.rclone.org because HEAD requests caused downloads!).
2018-02-22 17:28:21 +00:00
Fabian Möller
62e72801be
vfs: fix race between multiple RWFileHandle (#2052)
Fixes #2034
2018-02-18 14:12:26 +01:00
Nick Craig-Wood
11da2a6c9b Break the fs package up into smaller parts.
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.

The new code layout is documented in CONTRIBUTING.
2018-01-15 17:51:14 +00:00
Nick Craig-Wood
29d34426bc vfs: fix deletion of in use directories #1860
This was causing errors if the cache cleaner was called between the
Open and the pendingOpen of a RW file.

The fix was to move the cache open to the Open from the openPending.
2017-12-15 15:42:49 +00:00
Nick Craig-Wood
c16ac697a9 vfs: keep track of directories in the cache also #1860
This makes managing empty directories more reliable.
2017-11-28 20:39:23 +00:00
Nick Craig-Wood
0978957a2e vfs: make sure all 96 combinations of flags for Open work 2017-11-28 20:39:23 +00:00
Nick Craig-Wood
b4083b4371 vfs: rename Fsync to Sync and implement Sync on Node and Handle 2017-11-20 12:46:23 +00:00
Nick Craig-Wood
321b6da7af vfs: don't remove file from writers until it is transferred
This means that the list of active writers is up to date
2017-11-20 12:46:23 +00:00
Nick Craig-Wood
1b22ee5b93 vfs: fix error handling in openPending so it returns the correct error 2017-11-20 12:46:23 +00:00
Nick Craig-Wood
eab55ce882 vfs: add open files to directories 2017-11-20 12:46:23 +00:00
Nick Craig-Wood
dec21ccf63 vfs, cmount: make truncate work properly in the presence or otherwise of open files 2017-11-20 12:46:22 +00:00
Nick Craig-Wood
e7f2935333 vfs: decode flags in Open/OpenFile for debug 2017-11-20 12:46:22 +00:00
Nick Craig-Wood
7f20e1d7f3 vfs: add read write files and caching #711
This adds new flags to mount, cmount, serve *

    --cache-max-age duration         Max age of objects in the cache. (default 1h0m0s)
    --cache-mode string              Cache mode off|minimal|writes|full (default "off")
    --cache-poll-interval duration   Interval to poll the cache for stale objects. (default 1m0s)
2017-11-20 12:36:50 +00:00