Nick Craig-Wood
2e21c58e6a
fs: deglobalise the config #4685
...
This is done by making fs.Config private and attaching it to the
context instead.
The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
2020-11-26 16:40:12 +00:00
Nick Craig-Wood
1fb6ad700f
accounting: add context.Context #3257 #4685
2020-11-09 18:05:54 +00:00
Nick Craig-Wood
d69b96a94c
test: Add context to mockfs.NewFs #3257 #4685
2020-11-09 18:05:54 +00:00
Nick Craig-Wood
d846210978
fs: Add context to NewFs #3257 #4685
...
This adds a context.Context parameter to NewFs and related calls.
This is necessary as part of reading config from the context -
backends need to be able to read the global config.
2020-11-09 18:05:54 +00:00
Josh Soref
e4a87f772f
docs: spelling: e.g.
...
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-28 18:16:23 +00:00
Chaitanya Bankanhal
c8a719ae0d
webui: Prompt user for updating webui if an update is available
2020-09-07 16:45:00 +01:00
Nick Craig-Wood
344bce7e2a
docs: fix formatting of rc docs page
...
See: https://forum.rclone.org/t/rclone-1-53-release/18880/24
2020-09-03 11:53:24 +01:00
Nick Craig-Wood
75d54d720c
version: replace internal code with github.com/coreos/go-semver
...
We were already importing go-semver so it makes sense to remove the
duplicated semver parsing code and just use go-semver
2020-08-31 13:55:04 +01:00
Nick Craig-Wood
cc0421cb9e
rc/webgui: skip AddPlugin and RemovePlugin tests if download fails
2020-08-31 13:45:06 +01:00
Nick Craig-Wood
9c01ac9894
rc/webgui: improve error handling on web fetches
2020-08-31 13:45:06 +01:00
Chaitanya Bankanhal
20300d1f61
plugins: Change failing plugin test to new repo rclone/rclone-test-plugin
2020-08-31 13:45:06 +01:00
Chaitanya Bankanhal
6231beefc5
webui: Fix broken webui because of plugins redirection
2020-08-31 13:45:06 +01:00
Aaron Gokaslan
bf685f600e
webgui: fixes previously unhandled error in JSON marshall in fs/rc/webgui/plugins.go:writeToFile
2020-08-30 17:15:03 +01:00
Chaitanya Bankanhal
d6996e3347
plugins: Add url query params to regex for referrer path
2020-08-24 10:56:04 +01:00
Chaitanya Bankanhal
dffcc99373
plugins: Create availablePlugins config file if it does not exist.
2020-08-24 10:56:04 +01:00
Chaitanya Bankanhal
09b79679cd
plugins: restructure and add tests for pluginsctl/* calls
2020-08-24 10:56:04 +01:00
Chaitanya Bankanhal
cf68e61f40
Add redirection for plugin urls
2020-08-24 10:56:04 +01:00
Chaitanya Bankanhal
22674d1146
plugins: Add reverse proxy pluginsHandler for serving plugins
2020-08-24 10:56:04 +01:00
Chaitanya Bankanhal
f9ee0dc3f2
plugins: allow installation and use of plugins and test plugins with rclone-webui
2020-08-24 10:56:04 +01:00
Chaitanya Bankanhal
65fa6a946a
webui: Expose webui downloader and other utility for use with plugins
2020-08-24 10:56:04 +01:00
Chaitanya
4cf82118d9
rc: add plugins support
2020-08-24 10:56:04 +01:00
Chaitanya
5f56611a76
webgui: Move to new package fs/rc/webgui.
2020-08-24 10:56:04 +01:00
Chaitanya Bankanhal
6308153ae7
rc: pass response writer when needsResponse is set instead of pointer
...
rc: Fix failing tests for *http.ResponseWriter
2020-08-17 17:09:31 +01:00
Chaitanya Bankanhal
a9713cd0ed
core/command: Add streaming output for long running commands.
2020-08-17 17:09:31 +01:00
Chaitanya Bankanhal
1cae4152f9
rc: add NeedsResponse for rc calls
2020-08-17 17:09:31 +01:00
Nick Craig-Wood
4884bee8ba
core/command: pretend to be "rclone version" to make tests pass
2020-08-17 17:09:31 +01:00
Chaitanya Bankanhal
54fc2821cd
core/command: Add version command instead of ls
2020-08-17 17:09:31 +01:00
Chaitanya Bankanhal
5549fd25fc
core/command: Allow rc to execute rclone terminal commands.
...
Allow command parameter to be skipped.
2020-08-17 17:09:31 +01:00
Nick Craig-Wood
aab9aa8a2e
js: add experimental interface for integrating rclone into browsers
...
This works by compiling rclone to wasm and exporting the RC api to
javascript.
2020-08-10 17:32:21 +01:00
Nick Craig-Wood
744828a4de
rc: allow JSON parameters to simplify command line usage
...
If the parameter being passed is an object then it can be passed as a
JSON string rather than using the `--json` flag which simplifies the
command line.
rclone rc operations/list fs=/tmp remote=test opt='{"showHash": true}'
Rather than
rclone rc operations/list --json '{"fs": "/tmp", "remote": "test", "opt": {"showHash": true}}'
2020-07-22 18:40:52 +01:00
Chaitanya Bankanhal
fa43d02874
rc: Add operations/uploadfile to upload a file through rc using encoding multipart/form-data
2020-06-25 11:38:24 +01:00
Chaitanya
d0de39ebcd
rc: add NeedsRequest to call.
2020-06-25 11:38:24 +01:00
Nick Craig-Wood
b58bb03e95
test: Don't run unreliable tests on CI #4171
2020-06-15 21:34:37 +01:00
Roman Kredentser
55ad1354b6
link: Add --expire and --unlink flags
...
This adds expire and unlink fields to the PublicLink interface.
This fixes up the affected backends and removes unlink parameters
where they are present.
2020-06-05 14:51:05 +01:00
Martin Michlmayr
041b201abd
doc: fix typos throughout docs and code
2020-05-25 11:23:58 +01:00
Martin Michlmayr
4aee962233
doc: fix typos throughout docs and code
2020-05-20 15:54:51 +01:00
calisro
c80b6d96dd
http: improved directory listing with new template from Caddy project
...
This includes a new directory listing template which was originally
from the Caddy project (used with permission and copyright attribution).
This is used whenever we serve directory listings so `rclone serve
http`, `rclone serve webdav` and `rclone rcd --rc-serve`
This also modifies the tests so they work with the original template which
is easier to debug.
2020-05-08 16:15:21 +01:00
ElonH
d119bfd934
rcd: disable duplicate log
...
if running `rclone rcd --rc-user=admin --rc-pass=admin
--rc-allow-origin="*"`, lots of duplicate warnings apperent in log
Warning: Allow origin set to *. This can cause serious security problems.
Warning: Allow origin set to *. This can cause serious security problems.
....
This is not conducive to analyzing debugging info.
Therefore, let's show it only once.
2020-05-05 13:47:25 +00:00
Nick Craig-Wood
195d152785
rc: add GetStructMissingOK
2020-04-29 09:42:31 +01:00
Xiaoxing Ye
c4572ebc91
rc: fix misplaced http server config - fixes #4130
2020-04-23 20:22:47 +01:00
Gary Kim
38a4d50e73
rcd: Add Prometheus metrics support - fixes #3858
...
Signed-off-by: Gary Kim <gary@garykim.dev>
2020-03-01 09:58:34 +00:00
Lars Lehtonen
fe6f4135b4
fs/rc: fix dropped error
2020-02-04 11:31:06 +00:00
Xiaoxing Ye
ccaca04a5d
rcd: move webgui apart; option to disable browser
...
Fix #3601 , #3785
2020-01-20 12:27:55 +00:00
Nick Craig-Wood
199ac61bde
rc: add methods to turn on blocking and mutex profiling
2019-10-28 22:11:38 +00:00
Nick Craig-Wood
1868c77e16
rc: fix formatting of docs
2019-10-27 10:43:40 +00:00
Saksham Khanna
0c265713fd
rc: added command core/quit
2019-10-17 15:04:22 +01:00
Nick Craig-Wood
b242b0a078
lib/cache,rc/jobs: make tests more reliable
2019-09-20 11:53:08 +01:00
Nick Craig-Wood
ffa1dac10b
build: apply gofmt from go1.13 to change case of number literals
2019-09-05 13:59:06 +01:00
Michał Matczuk
3247e69cf5
fs/rc/jobs: ExecuteJob propagate the error returned by function
...
Without this patch the resulting error is first converted to string and then recreated.
This makes it impossible to use the defined error types to figure out the cause of the error,
and may result in invalid HTTP status codes.
This patch adds a test TestExecuteJobErrorPropagation to validate that the errors are
properly propagated.
2019-08-22 16:10:48 +01:00
Chaitanya
159f2e29a8
rcd: prefix patch for rcd and web-gui
2019-08-22 08:36:10 +01:00
Chaitanya
efd826ad4b
rcd: auto-login for web-gui
...
rcd: auto use authentication if none is provided for web-gui
2019-08-22 08:36:10 +01:00
Michal Matczuk
5d6593de4f
* rc/jobs: Add SetInitialJobID function that allows for setting the jobID
2019-08-21 11:01:39 +01:00
Nick Craig-Wood
3ecbd603ab
rc: move job expire flags to rc to fix initalization problem
...
See: https://forum.rclone.org/t/rc-rc-job-expire-interval-bug/11188
rclone was ignoring the --rc-job-expire-duration and --rc-job-interval
flags. This turned out to be an initialization order problem and was
fixed by moving those flags out of global config into rc config.
2019-08-10 17:12:22 +01:00
Nick Craig-Wood
0693deea1c
rc: fix unmarshalable http.AuthFn in options and put in test for marshalability
2019-08-10 16:22:17 +01:00
Chaitanya
33677ff367
rc: Added command line parameter to control the cross origin resource sharing (CORS) in the rcd. (Security Improvement)
...
rc: Import statements
Fixing the problem with test
2019-08-09 15:14:52 +01:00
negative0
27a075e9fc
rcd: Removed the shorthand for webgui. Shorthand is reserved for rsync compatibility.
2019-08-06 12:50:31 +01:00
Nick Craig-Wood
5065c422b4
lib/random: unify random string generation into random.String
...
This was factored from fstest as we were including the testing
enviroment into the main binary because of it.
This was causing opening the browser to fail because of 8243ff8bc8
.
2019-08-06 12:44:08 +01:00
Chaitanya Bankanhal
d51a970932
rcd: Change URL after webgui move to rclone organization
2019-08-05 16:22:40 +01:00
Chaitanya Bankanhal
84e2806c4b
rc: Rclone-WebUI integration with rclone
...
This adds experimental support for web gui integration so that rclone can fetch and run a web based GUI using the --rc-web-ui and related flags.
It downloads and caches a webui zip file which it then unpacks and opens in the browser.
2019-08-04 12:32:37 +01:00
Aleksandar Jankovic
6a3e301303
accounting: add call to clear stats
...
- Make calls more consistent by changing path to kebab case.
- Add stacktrace information to job panics
2019-08-02 16:56:19 +01:00
Nick Craig-Wood
57d5de6fba
build: fix up package paths after repo move
...
git grep -l github.com/ncw/rclone | xargs -d'\n' perl -i~ -lpe 's|github.com/ncw/rclone|github.com/rclone/rclone|g'
goimports -w `find . -name \*.go`
2019-07-28 18:47:38 +01:00
Aleksandar Jankovic
8243ff8bc8
accounting: isolate stats to groups
...
Introduce stats groups that will isolate accounting for logically
different transferring operations. That way multiple accounting
operations can be done in parallel without interfering with each other
stats.
Using groups is optional. There is dedicated global stats that will be
used by default if no group is specified. This is operating mode for CLI
usage which is just fire and forget operation.
For running rclone as rc http server each request will create it's own
group. Also there is an option to specify your own group.
2019-07-28 14:48:19 +01:00
buengese
19ae053168
rcserver: remove _async key from input parameters after parsing so later operations won't get confused - fixes #3346
2019-07-20 19:35:10 +02:00
Aleksandar Jankovic
5935cb0a29
jobs: add ability to stop async jobs
...
Depends on #3257
2019-06-19 14:17:41 +01:00
Aleksandar Jankovic
f78cd1e043
Add context propagation to rclone
...
- Change rclone/fs interfaces to accept context.Context
- Update interface implementations to use context.Context
- Change top level usage to propagate context to lover level functions
Context propagation is needed for stopping transfers and passing other
request-scoped values.
2019-06-19 11:59:46 +01:00
Aleksandar Jankovic
93207ead9c
rc/jobs: make job expiry timeouts configurable
2019-06-15 09:55:32 +01:00
Nick Craig-Wood
f681d32996
rc: Fix serving bucket based objects with --rc-serve
...
Before this change serving bucket based objects
`[remote:bucket]/path/to/object` would fail with 404 not found.
This was because the leading `/` in `/path/to/object` was being passed
to NewObject.
2019-06-10 11:59:06 +01:00
Nick Craig-Wood
206e1caa99
fs/cache: factor Fs caching from fs/rc into its own package
2019-05-27 16:14:55 +01:00
Nick Craig-Wood
577fda059d
rc: fix race in tests
2019-05-01 16:09:50 +01:00
Nick Craig-Wood
9df322e889
tests: make test servers choose a random port to make more reliable
...
Tests have been randomly failing with messages like
listen tcp 127.0.0.1:51778: bind: address already in use
Rework all the test servers so they choose a random free port on
startup and use that for the tests to avoid.
2019-05-01 16:09:50 +01:00
Fabian Möller
48c09608ea
fix spelling
2019-04-30 14:12:18 +02:00
Nick Craig-Wood
16d8014cbb
build: drop support for go1.8
2019-04-15 21:49:58 +01:00
Nick Craig-Wood
0d475958c7
Fix errors discovered with go vet nilness tool
2019-03-18 11:23:00 +00:00
Nick Craig-Wood
cccadfa7ae
rc: add ability for options blocks to register reload functions
2019-03-10 13:09:44 +00:00
Nick Craig-Wood
6e153781a7
rc: add help to show how to set log level with options/set
2019-02-03 11:47:57 +00:00
Nick Craig-Wood
97752ca8fb
build: fix problems found with ineffasign linter
2019-02-02 11:45:07 +00:00
Jay
082a7065b1
Use vfsgen for static HTML templates
2018-12-26 15:07:21 +00:00
Nick Craig-Wood
57b85b8155
rc: fix job tests on Windows
2018-11-06 13:03:48 +00:00
Nick Craig-Wood
bc17ca7ed9
rc: implement core/obscure
2018-11-05 15:44:40 +00:00
Nick Craig-Wood
1916410316
rc: add core/version and put definitions next to implementations
2018-11-05 15:44:40 +00:00
Nick Craig-Wood
75a88de55c
rc/rcserver: with --rc-files if auth set, pass on to URL opened
...
If `--rc-user` or `--rc-pass` is set then the URL that is opened with
`--rc-files` will have the authorization in the URL in the
`http://user:pass@localhost/ ` style.
2018-11-05 15:44:40 +00:00
Nick Craig-Wood
fa0a9653d2
rc: methods marked as AuthRequired need auth unless --rc-no-auth
...
Methods which can read or mutate external storage will require
authorisation - enforce this. This can be overidden by `--rc-no-auth`.
2018-11-04 20:42:57 +00:00
Nick Craig-Wood
75e8ea383c
rc: implement rc.PutCachedFs for prefilling the remote cache
2018-11-04 20:42:57 +00:00
Nick Craig-Wood
8c8b58a7de
rc: expire remote cache and fix tests under race detector
2018-11-04 20:42:57 +00:00
Nick Craig-Wood
b961e07c57
rc: ensure rclone fails to start up if the --rc port is in use already
2018-11-04 15:11:51 +00:00
Nick Craig-Wood
89550e7121
rcserver: serve directories as well as files
2018-11-04 15:11:51 +00:00
Nick Craig-Wood
b972dcb0ae
rc: implement options/blocks,get,set and register options
2018-11-03 11:32:00 +00:00
Nick Craig-Wood
0bfa9811f7
rc: factor server code into rcserver and implement serving objects
...
If a GET or HEAD request is receivied with a URL parameter of fs then
it will be served from that remote.
2018-11-03 11:32:00 +00:00
Nick Craig-Wood
75252e4a89
rc: add --rc-files flag to serve files on the rc http server
...
This enables building a browser based UI for rclone
2018-11-03 11:32:00 +00:00
Nick Craig-Wood
2089405e1b
fs/rc: add more infrastructure to help writing rc functions
...
- Fs cache for rc commands
- Helper functions for parsing the input
- Reshape command for manipulating JSON blobs
- Background Job starting, control, query and expiry
2018-11-02 17:32:20 +00:00
Nick Craig-Wood
ade252f13b
build: fixup code formatting after goimports change
2018-10-13 22:47:12 +01:00
frenos
382a6863b5
rc: add support for OPTIONS and basic CORS - #2575
2018-09-24 17:04:47 +01:00
Nick Craig-Wood
8c0335a176
build: fix for goimports format change
...
See https://github.com/golang/go/issues/23709
2018-07-03 22:33:15 +01:00
Nick Craig-Wood
3a2248aa5f
rc: add core/gc to run a garbage collection on demand
2018-05-24 15:10:28 +01:00
Nick Craig-Wood
573ef4c8ee
rc: enable go profiling by default on the --rc port
...
This means you can use the pprof tool on a running rclone, eg
go tool pprof http://localhost:5572/debug/pprof/heap
2018-05-24 15:10:28 +01:00
Nick Craig-Wood
9ab2521ef2
rc: autogenerate and tidy the docs and commands
...
* Rename rc/pid -> core/pid
* Sort the output of `rc list`
* Make a script to autogenerate the docs
* Tidy docs
2018-04-23 20:57:17 +01:00
Nick Craig-Wood
21a10e58c9
rc: implement core/memstats to print internal memory usage info
2018-04-23 20:49:36 +01:00
Michael P. Dubner
92b266d361
rc: new call rc/pid - closes #2211
2018-04-09 13:33:04 +01:00
Nick Craig-Wood
be54fd8f70
Remove builds conditional on go1.7 since that is now guaranteed #2154
...
Old fallback code was deleted and the go1.7 style code inlined where
appropriate.
2018-04-07 11:42:55 +01:00
Matt Holt
e13f65b953
serve restic: Print actual listener address
2018-04-04 14:56:26 +01:00