Fabian Möller
1a40bceb1d
backend: unify NewFs path handling for wrapping remotes
...
Use the same function to join the root paths for the wrapping remotes
alias, cache and crypt.
The new function fspath.JoinRootPath is equivalent to path.Join, but if
the first non empty element starts with "//", this is preserved to allow
Windows network path to be used in these remotes.
2018-10-10 17:50:27 +01:00
Nick Craig-Wood
a9273c5da5
docs: move documentation for options from docs/content into backends
...
In the following commit, the documentation will be autogenerated.
2018-10-06 11:47:46 +01:00
dcpu
c0084f43dd
cache: Remove entries that no longer exist in the source
...
list directory with 25k files
before(1.43.1)
5m24s
after
3m21s
2018-10-06 11:23:33 +01:00
Fabian Möller
1eec59e091
fs: update ChangeNotifier interface
...
This introduces a channel to the ChangeNotify function, which can be
used to update the poll-interval and cleanly exit the polling function.
2018-09-25 14:01:13 +02:00
Fabian Möller
3c44ef788a
cache: add plex_insecure option to skip certificate validation
...
Fixes #2215
2018-09-10 21:19:25 +01:00
Fabian Möller
f6ee0795ac
cache: preserve leading / in wrapped remote path
...
When combining the remote value and the root path, preserve the absence
or presence of the / at the beginning of the wrapped remote path.
e.g. a remote "cloud:" and root path "dir" becomes "cloud:dir" instead
of "cloud:/dir".
Fixes #2553
2018-09-10 20:35:50 +01:00
Anagh Kumar Baranwal
5a6750e1cd
cache: documentation fix for cache-chunk-total-size - Fixes #2519
...
Signed-off-by: Anagh Kumar Baranwal <anaghk.dos@gmail.com>
2018-09-04 16:16:35 +01:00
Fabian Möller
deda093637
cache: fix error return value of cache/fetch rc method
2018-09-03 17:32:11 +02:00
dcpu
a4c4019032
cache: improve performance by not sending info requests for cached chunks
2018-09-03 15:41:06 +01:00
Fabian Möller
a0b3fd3a33
cache: fix worker scale down
...
Ensure that calling scaleWorkers will create/destroy the right amount of
workers.
2018-09-03 12:29:35 +02:00
Fabian Möller
cdbe3691b7
cache: add cache/fetch rc function
2018-09-03 12:29:35 +02:00
Dr. Tobias Quathamer
76450c01f3
cache: remove accidentally committed files
...
* Delete cache_upload_test.go.orig
* Delete cache_upload_test.go.rej
2018-09-02 15:51:22 +01:00
Nick Craig-Wood
6156bc5806
cache: fix nil pointer deref - fixes #2448
2018-08-07 21:33:13 +01:00
Fabian Möller
eb6aafbd14
cache: implement fs.ObjectUnWrapper
2018-08-07 09:28:55 +01:00
Nick Craig-Wood
ffd11662ba
cache: fix nil pointer deref when using lsjson on cached directory
...
This stops embedding the fs.Directory into the cache.Directory because
it can be nil and implements an ID method which checks the nil status.
See: https://forum.rclone.org/t/runtime-error-with-cache-remote-and-lsjson/6305
2018-08-05 09:42:31 +01:00
Nick Craig-Wood
f3f48d7d49
Implement new backend config system
...
This unifies the 3 methods of reading config
* command line
* environment variable
* config file
And allows them all to be configured in all places. This is done by
making the []fs.Option in the backend registration be the master
source of what the backend options are.
The backend changes are:
* Use the new configmap.Mapper parameter
* Use configstruct to parse it into an Options struct
* Add all config to []fs.Option including defaults and help
* Remove all uses of pflag
* Remove all uses of config.FileGet
2018-07-16 21:20:47 +01:00
remusb
aaa8591661
cache: add non cached dirs on notifications - #2155
2018-06-13 23:57:26 +03:00
remusb
4df1794932
cache: fix panic when running without plex configs
2018-06-13 15:06:14 +03:00
remusb
339fbf0df5
cache: reconnect plex websocket on failures
2018-06-12 22:58:15 +03:00
remusb
13ccb39819
cache: allow root to be expired from rc - #2237
2018-06-12 22:19:03 +03:00
remusb
f9a1a7e700
cache: fix root folder caching
2018-06-10 21:54:20 +03:00
remusb
4e8e5fed7d
cache: clean remaining empty folders from temp upload path
2018-06-09 14:52:31 +03:00
remusb
c0f772bc14
cache: update internal tests and small fixes
2018-06-08 23:34:38 +03:00
remusb
8a84975993
cache: cache lists using batch writes
2018-06-04 21:04:45 +03:00
John Clayton
71b4f1ccab
cache: use secure websockets for HTTPS Plex addresses
2018-05-22 11:47:57 +01:00
Nick Craig-Wood
e5ff375948
Use config.FileGet instead of fs.ConfigFileGet
2018-05-22 09:43:24 +01:00
Nick Craig-Wood
1320e84bc2
build: remove unused code spotted by the deadcode linter
2018-05-05 17:32:41 +01:00
Nick Craig-Wood
cb5bd47e61
build: fix errors spotted by ineffassign linter
...
These were mostly caused by shadowing err and a good fraction of them
will have caused errors not to be propagated properly.
2018-05-05 17:32:41 +01:00
Nick Craig-Wood
9ab2521ef2
rc: autogenerate and tidy the docs and commands
...
* Rename rc/pid -> core/pid
* Sort the output of `rc list`
* Make a script to autogenerate the docs
* Tidy docs
2018-04-23 20:57:17 +01:00
Nick Craig-Wood
1ac6dacf0f
about: complete other providers and re-work internals
...
* Implement about for:
* local, crypt, cache, drive, swift, hubic, onedrive, pcloud, dropbox
* Implement `--json` and `---full` flag for `rclone about`
* change About interface to return a Usage structure
* Remove operations.About as it is too thin an interface
* Implement Integration test
Relates to #1138 and #1564
2018-04-17 21:53:27 +01:00
a-roussos
94e277d759
about: add new command 'about' to get quota info from a remote
...
Implemented for drive only.
Relates to #1138 and #1564 .
2018-04-17 21:50:14 +01:00
Nick Craig-Wood
0fa700b3cf
Make integration tests use go1.7+ nested tests #2154
...
* Removed generated code and code generator
* Updated docs on how to write integration tests
* Tidied up the actual integration tests
2018-04-08 15:04:14 +01:00
Nick Craig-Wood
be54fd8f70
Remove builds conditional on go1.7 since that is now guaranteed #2154
...
Old fallback code was deleted and the go1.7 style code inlined where
appropriate.
2018-04-07 11:42:55 +01:00
Nick Craig-Wood
e5be471ce0
Use io.SeekStart/End/Current constants now for go1.7+ #2154
2018-04-07 11:42:36 +01:00
Nick Craig-Wood
80588a5a6b
Replace "golang.org/x/net/context" with "context" for go1.7+ #2154
2018-04-07 11:42:08 +01:00
remusb
6e11a25df5
cache: flush the memory cache after close
2018-04-04 23:25:53 +03:00
remusb
1dea99ab20
cache: purge file data on notification
2018-04-03 23:24:45 +03:00
remusb
e42cee5e02
cache: always forget parent dir for notifications - for #2117
2018-03-31 12:44:09 +03:00
Stefan
a8267d1628
link: allow creating public link to files and folders - closes #1562
2018-03-29 09:10:19 +02:00
remusb
b52e34ef5e
cache: add info log on notification - for #2150
2018-03-23 22:41:01 +02:00
remusb
0ed0d9a7bc
cache: integrate with Plex websocket
2018-03-22 21:21:03 +02:00
seuffert
d9c13bff83
add rc cache/stats
2018-03-22 21:16:16 +02:00
remusb
4924ac2f17
cache: reduce log level for plex api - for #2102
2018-03-17 11:57:36 +02:00
remusb
078d705dbe
cache: notify vfs and support crypt in rpc - #2111
2018-03-15 11:39:16 +02:00
remusb
677971643c
cache: add support for rc
2018-03-14 22:58:20 +02:00
Remus Bunduc
d76da1f5fd
cache: fix dir cache issue - #2117
2018-03-14 11:08:30 +02:00
Nick Craig-Wood
a7dbf32c53
cache: Implement --cache-db-wait-time flag
...
This can be used to make the cache wait for other running cache
backends to finish rather than erroring after 1 second.
2018-03-08 21:14:09 +00:00
Remus Bunduc
70f07fd3ac
fs: add ChangeNotify and backend support for it ( #2094 )
...
* fs: rename DirChangeNotify to ChangeNotify
* cache: switch to ChangeNotify
* ChangeNotify: keep order of notifications
2018-03-08 22:03:34 +02:00
Nick Craig-Wood
fe8eeec5b5
cache: improve efficiency with RangeOption and RangeSeek #1825
...
* All remotes now support RangeOption so remove SeekOption
* Correct off by one error as RangeOption arguments are inclusive.
* Use RangeSeek in preference to Seek if available
2018-03-02 17:10:56 +00:00
Fabian Möller
aeefa34f62
fstests: add TestInternal ( #2085 )
...
TestInternal allows to perform a custom test on the backend using the
optional InternalTester interface.
2018-02-25 10:58:06 +01:00