extern/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2025-08-08 05:24:49 +02:00
Code Issues Packages Projects Releases Wiki Activity
7,939 Commits 193 Branches 122 Tags
04c69959b8e905dccad1c3a0d0c94480351bcf8c
Commit Graph

244 Commits

Author SHA1 Message Date
Nick Craig-Wood
96207f342c configmap: add consistent String() method to configmap.Simple #4996 2021-03-15 19:22:07 +00:00
Nick Craig-Wood
e25ac4dcf0 fs: Use connection string config as highest priority config #4996 2021-03-15 19:22:07 +00:00
Nick Craig-Wood
3b21857097 config: clear fs cache of stale entries when altering config - fixes #4811 
Before this change if a config was altered via the rc then when a new
backend was created from that config, if there was a backend already
running from the old config in the cache then it would be used instead
of creating a new backend with the new config, thus leading to
confusion.

This change flushes the fs cache of any backends based off a config
when that config is changed is over the rc.
 2021-03-15 19:22:07 +00:00
Nick Craig-Wood
294f090361 fs: make sure --low-level-retries, --checkers, --transfers are > 0 2021-03-15 17:05:35 +00:00
Nick Craig-Wood
3cf6ea848b config: remove log.Fatal and replace with error passing where possible 2021-03-14 16:03:35 +00:00
Nick Craig-Wood
bb0b6432ae config: --config "" or "/notfound" for in memory config only #4996 
If `--config` is set to empty string or the special value `/notfound`
then rclone will keep the config file in memory only.
 2021-03-14 16:03:35 +00:00
Nick Craig-Wood
46078d391f config: make config file reads reload the config file if needed #4996 
Before this change the config file needed to be explicitly reloaded.
This coupled the config file implementation with the backends
needlessly.

This change stats the config file to see if it needs to be reloaded on
every config file operation.

This allows us to remove calls to

- config.SaveConfig
- config.GetFresh

Which now makes the the only needed interface to the config file be
that provided by configmap.Map when rclone is not being configured.

This also adds tests for configfile
 2021-03-14 16:03:35 +00:00
Nick Craig-Wood
6a9ae32012 config: split up main file more and move tests into correct packages 
This splits config.go into ui.go for the user interface functions and
authorize.go for the implementation of `rclone authorize`.

It also moves the tests into the correct places (including one from
obscure which was in the wrong place).
 2021-03-11 17:29:26 +00:00
Nick Craig-Wood
a7fd65bf2d config: move account initialisation out into accounting 
Before this change the initialisation for the accounting package was
done in the config package for some strange historical reason.
 2021-03-11 17:29:26 +00:00
Nick Craig-Wood
1fed2d910c config: make config file system pluggable 
If you are using rclone a library you can decide to use the rclone
config file system or not by calling

    configfile.LoadConfig(ctx)

If you don't you will need to set `config.Data` to an implementation
of `config.Storage`.

Other changes
- change interface of config.FileGet to remove unused default
- remove MustValue from config.Storage interface
- change GetValue to return string or bool like elsewhere in rclone
- implement a default config file system which panics with helpful error
- implement getWithDefault to replace the removed MustValue
- don't embed goconfig.ConfigFile so we can change the methods
 2021-03-11 17:29:26 +00:00
Fionera
c95b580478 config: Wrap config library in an interface 2021-03-11 17:29:26 +00:00
K265
930bca2478 feat: add multiple paths support to --compare-dest and --copy-dest flag 2021-02-16 18:17:04 +00:00
Max Sum
edfe183ba2 fshttp: add DSCP support with --dscp for QoS with differentiated services 2021-02-10 18:29:18 +00:00
Nick Craig-Wood
3b6df71838 accounting: refactor bwlimit code to allow for multiple slots 2021-01-30 17:39:30 +00:00
albertony
7a496752f3 fs: add support for flag --no-console on windows to hide the console window 2021-01-27 18:44:35 +00:00
Nick Craig-Wood
50344e7792 accounting: factor --tpslimit code into accounting from fshttp 2021-01-08 10:29:57 +00:00
Nick Craig-Wood
f425950a52 fs: Always show stats when using --dry-run or --interactive #4624 2020-12-29 21:11:12 +00:00
Nick Craig-Wood
060642ad14 flags: improve error message when reading environment vars #4888 
The message now includes the flag name to help the user work out what
is happening.

    Invalid value for environment variable "RCLONE_VERSION" when setting default
    for --version: strconv.ParseBool: parsing "yes": invalid syntax
 2020-12-28 12:26:23 +00:00
Nick Craig-Wood
c98dd8755c log: fix enabling systemd logging when using --log-file 
This also moves all the systemd logging decisions to fs/log
 2020-12-17 11:55:27 +00:00
Nick Craig-Wood
2e21c58e6a fs: deglobalise the config #4685 
This is done by making fs.Config private and attaching it to the
context instead.

The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
 2020-11-26 16:40:12 +00:00
Nick Craig-Wood
f7efce594b config: add context.Context #3257 #4685 
This add config to the Config callback in the backends and the related
config functions.
 2020-11-09 18:05:54 +00:00
Nick Craig-Wood
1fb6ad700f accounting: add context.Context #3257 #4685 2020-11-09 18:05:54 +00:00
Josh Soref
e4a87f772f docs: spelling: e.g. 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref
bbe7eb35f1 docs: spelling: server-side 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref
d0888edc0a Spelling fixes 
Fix spelling of: above, already, anonymous, associated,
authentication, bandwidth, because, between, blocks, calculate,
candidates, cautious, changelog, cleaner, clipboard, command,
completely, concurrently, considered, constructs, corrupt, current,
daemon, dependencies, deprecated, directory, dispatcher, download,
eligible, ellipsis, encrypter, endpoint, entrieslist, essentially,
existing writers, existing, expires, filesystem, flushing, frequently,
hierarchy, however, implementation, implements, inaccurate,
individually, insensitive, longer, maximum, metadata, modified,
multipart, namedirfirst, nextcloud, obscured, opened, optional,
owncloud, pacific, passphrase, password, permanently, persimmon,
positive, potato, protocol, quota, receiving, recommends, referring,
requires, revisited, satisfied, satisfies, satisfy, semver,
serialized, session, storage, strategies, stringlist, successful,
supported, surprise, temporarily, temporary, transactions, unneeded,
update, uploads, wrapped

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-14 15:21:31 +01:00
Nick Craig-Wood
72083c65ad cmd: make backend env vars show in help as the defaults for backend flags 
Before this change

    RCLONE_DRIVE_CHUNK_SIZE=111M rclone help flags | grep drive-chunk-size

Would show the default value, not the setting of RCLONE_DRIVE_CHUNK_SIZE
as the non backend flags do.

This change makes it work as expected by setting the default of the
option to the environment variable.

Fixes #4659
 2020-10-13 15:43:58 +01:00
LaSombra
aac2406e19 cmd: add --progress-terminal-title to print ETA to terminal title 
Adds a flag, --progress-terminal-title, that when used with --progress,
will print the string `ETA: %s` to the terminal title.

This also adds WriteTerminalTitle to lib/terminal
 2020-10-06 16:34:26 +01:00
Hekmon
c58023a9ba enhance systemd integration 
* log level identification
* manual activation with flag
* automatic systemd launch detection
 2020-10-06 16:21:30 +01:00
Nick Craig-Wood
3a14b1d5a9 build: make rclone build with wasm 
Needed to drop
- azureblob backend
- cache backend
- qingstor backend
- cachestats command
- ncdu command
 2020-08-10 17:32:21 +01:00
Nick Craig-Wood
26001d520a fs: add --bwlimit-file flag to limit speeds of individual file transfers 2020-07-28 11:46:24 +01:00
Nick Craig-Wood
8f42532b6d sync: add --track-renames-strategy leaf 
See: https://forum.rclone.org/t/how-to-minimize-bandwith-w-r-t-renames-during-sync/16928/22
 2020-07-28 11:34:27 +01:00
Nick Craig-Wood
3efdf5e095 operations: implement --refresh-times flag to set modtimes on hashless backends 2020-06-17 10:48:13 +01:00
fishbullet
ba5eb230fb operations: interactive mode -i/--interactive for destructive operations #3886 2020-06-10 12:33:53 +01:00
Nick Craig-Wood
b62d08d136 config: set RCLONE_CONFIG_DIR for use in config files and subprocesses 
See: https://forum.rclone.org/t/relative-path-in-rclone-config-service-account-json/16693
 2020-06-03 17:39:08 +01:00
Nick Craig-Wood
518d39815c sync,copy,move: add --check-first to do all checking before starting transfers 
See: https://forum.rclone.org/t/rclone-sync-doing-transfer-and-checking-in-paralel/16352/
 2020-05-22 17:50:07 +01:00
Nick Craig-Wood
98c34e413d config: add --obscure and --no-obscure flags to config create/update 
Before this change there was some ambiguity about whether passwords
were obscured on not passing them into config create or config update.

This change adds the --obscure and --no-obscure flags to make the
intent clear.

It also updates the remote control and the tests.

Fixes #3728
 2020-05-15 16:41:37 +01:00
Ben Zenker
899c8e0697 march: added flag to allow Unicode filenames to remain unique 
If your filenames contain two near-identical Unicode characters,
rclone will normalize these, making them identical. This flag
gives you the ability to keep them unique. This might
create unintended side effects, such as duplicating files that
contain certain Unicode characters, when downloading them from
certain cloud providers to a macOS filesystem.

Fixes #4228
 2020-05-15 12:28:01 +01:00
Sébastien Gross
395f259978 cmd: when running --password-command allow use of stdin 
Bind rclone standard input to password command's standard input. This
allows to provide password from a pipe and collect it using cat.

The typical use case is when rclone is on a remote server with an
encrypted configuration. This solved the environment variable
issue (#3368) and the password storage on remote host.

Now the following chain is allowed:

    echo 'secret' | ssh host.example.com \
       sudo -u rclone \
       rclone --config /path/to/rclone.conf \
       --password-command 'cat' ls remote:

Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>

Co-authored-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
 2020-05-07 11:02:52 +01:00
Nick Craig-Wood
f6346a4d29 fs: add --header flag to add options to every HTTP transaction #59 2020-04-23 15:24:21 +01:00
Tim Gallant
93caa459e3 fs/config: add header-download and header-upload flags 2020-04-23 11:07:21 +01:00
Jon Fautley
53874bd8ee cmd: add --error-on-no-transfer option 
This allows rclone to exit with a non-zero return code if no files are
transferred. This is useful when calling rclone as part of a workflow/script
pipeline as it allows the end user to stop processing if no files have been
transferred.

NB: Enabling this option will return in rclone exiting non-zero if there are no
transfers. Depending on how your're currently using rclone in your scripts,
this may break existing setups!
 2020-04-15 17:06:40 +01:00
Bernd Schoolmann
158870bcdb fs: Add --track-renames-strategy for configurable matching criteria for --track-renames 
This commit adds the `--track-renames-strategy` flag which allows the
user to choose the strategy for tracking renames when using the
`--track-renames` flag.

This can be "hash" or "modtime" or both currently.

This, when used with `--track-renames-strategy modtime` enables
support for tracking renames in encrypted remotes.

Fixes #3696
Fixes #2721
 2020-03-20 13:04:56 +00:00
Nick Craig-Wood
47e2d5c415 config: fsync the config file after writing #3411 
This should help with data integrity
 2020-03-16 18:20:16 +00:00
Franklyn Tackitt
2b3d13a841 fs: Use --cutoff-mode hard,soft,catious instead of 3 --max-transfer-mode flags 
Fixes #2672
 2020-03-01 09:49:55 +00:00
Shing Kit Chan
6f1766dd9e fs: Add support for --max-transfer-cutoff modes #2672 
This also adds max transfer cut off check for server side copies too
 2020-03-01 09:49:55 +00:00
boosh
0d7573dd81 fs: Add --max-duration flag to control the maximum duration of a transfer session 
This gives you more control over how long rclone will run for, making
it easier to script backups, e.g. via cron. Once the `--max-duration`
time limit is reached, no new transfers will be initiated, but those
already in-flight will be allowed to complete.

Fixes #985
 2020-01-24 13:28:56 +00:00
Nick Craig-Wood
3afb2a4798 config: use SpaceSepList for argument to --password-command 
This is to enable arguments with spaces in.
 2020-01-23 15:39:15 +00:00
Nick Craig-Wood
62dbdcdbcc config: use the environment variable which goes with --password-command 2020-01-23 15:39:15 +00:00
Damon Permezel
06df133159 config: add --password-command to allow dynamic config password - fixes #3694 2020-01-23 15:39:15 +00:00
Nick Craig-Wood
277d94feac fshttp: add --expect-continue-timeout default 1s - fixes #3835 
Before this change the expect/continue timeout was set to
--conntimeout which was 60s by default which is too long to wait.

This was noticed when using s3 with a proxy which apparently didn't
support expect / continue properly.

Set --expect-continue-timeout 0 to disable expect/continue.
 2020-01-18 09:49:22 +00:00