178 Commits

Author	SHA1	Message	Date
Nick Craig-Wood	e43b5ce5e5	Remove github.com/pkg/errors and replace with std library version ... This is possible now that we no longer support go1.12 and brings rclone into line with standard practices in the Go world. This also removes errors.New and errors.Errorf from lib/errors and prefers the stdlib errors package over lib/errors.	2021-11-07 11:53:30 +00:00
Ivan Andreev	50df8cec9c	lib/kv: add key-value database api #5587 ... Add bolt-based key-value database support. Quick API description: https://github.com/rclone/rclone/pull/5587#issuecomment-942174768	2021-10-20 19:11:54 +03:00
Ivan Andreev	ffa1b1a258	config: enable verbose logging by the --verbose argument - #5594	2021-10-03 01:44:08 +03:00
albertony	59c74ea1b8	config: support hyphen in remote name from environment variable	2021-10-01 15:18:04 +02:00
albertony	721a9786a7	ls: introduce and global option to print human-readable sizes and consider it for ls commands ... Fixes #1890	2021-08-31 09:57:27 +02:00
Ole Frost	58c99427b3	config: fixed issues with flags/options set by environment vars. ... Some environment variables didn’t behave like their corresponding command line flags. The affected flags were --stats, --log-level, --separator, --multi-tread-streams, --rc-addr, --rc-user and --rc-pass. Example: RCLONE_STATS='10s' rclone check remote: remote: --progress # Expected: rclone check remote: remote: --progress –-stats=10s # Actual: rclone check remote: remote: --progress Remote specific options set by environment variables was overruled by less specific backend options set by environment variables. Example: RCLONE_DRIVE_USE_TRASH='false' RCLONE_CONFIG_MYDRIVE_USE_TRASH='true' rclone deletefile myDrive:my-test-file # Expected: my-test-file is recoverable in the trash folder # Actual: my-test-file is permanently deleted (not recoverable) Backend specific options set by environment variables was overruled by general backend options set by environment variables. Example: RCLONE_SKIP_LINKS='true' RCLONE_LOCAL_SKIP_LINKS='false' rclone lsd local: # Expected result: Warnings when symlinks are skipped # Actual result: No warnings when symlinks are skipped # That is RCLONE_SKIP_LINKS takes precedence The above issues have been fixed. The debug logging (-vv) has been enhanced to show when flags are set by environment variables. The documentation has been enhanced with details on the precedence of configuration options. See pull request #5341 for more information.	2021-07-05 16:38:20 +01:00
Nick Craig-Wood	296ceadda6	fs: add --all to rclone config create/update to ask all the config questions #3455 ... This also factors the config questions into a state based mechanism so a backend can be configured using the same dialog as rclone config but remotely.	2021-05-14 14:07:44 +01:00
Nick Craig-Wood	9e6c23d9af	fs: add --disable-http2 for global http2 disable #5253	2021-04-30 20:31:04 +01:00
Nick Craig-Wood	c0c74003f2	fs/cache: add --fs-cache-expire-duration to control the fs cache ... This commit makes the previously statically configured fs cache configurable. It introduces two parameters `--fs-cache-expire-duration` and `--fs-cache-expire-interval` to control the caching of the items. It also adds new interfaces to lib/cache to set these.	2021-03-30 12:46:47 +01:00
Nick Craig-Wood	f47893873d	fs: fix failed token refresh on mounts created via the rc ... Users have noticed that backends created via the rc have been failing to refresh their tokens with this error: Token refresh failed try 1/5: context canceled This is because the rc server cancels the context used to make the backend when the request has finished. This same context is used to refresh the token and the oauth library checks to see if the context has been cancelled. This patch creates a new context for the cached backends and copies the global and filter config into the new context. See: https://forum.rclone.org/t/google-drive-token-refresh-failed/22283	2021-03-16 16:29:22 +00:00
Nick Craig-Wood	0ad38dd6fa	dropbox,ftp,onedrive,yandex: make --timeout 0 work properly ... See: https://forum.rclone.org/t/an-issue-about-ftp-backend-in-2-different-systems/22551	2021-03-01 12:08:58 +00:00
K265	930bca2478	feat: add multiple paths support to --compare-dest and --copy-dest flag	2021-02-16 18:17:04 +00:00
Max Sum	edfe183ba2	fshttp: add DSCP support with --dscp for QoS with differentiated services	2021-02-10 18:29:18 +00:00
albertony	7a496752f3	fs: add support for flag --no-console on windows to hide the console window	2021-01-27 18:44:35 +00:00
Nick Craig-Wood	8053fc4e16	fs: correct default implementation of fs.CountError	2020-12-31 18:16:51 +00:00
Nick Craig-Wood	c98dd8755c	log: fix enabling systemd logging when using --log-file ... This also moves all the systemd logging decisions to fs/log	2020-12-17 11:55:27 +00:00
Nick Craig-Wood	2e21c58e6a	fs: deglobalise the config #4685 ... This is done by making fs.Config private and attaching it to the context instead. The Config should be obtained with fs.GetConfig and fs.AddConfig should be used to get a new mutable config that can be changed.	2020-11-26 16:40:12 +00:00
Josh Soref	e4a87f772f	docs: spelling: e.g. ... Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>	2020-10-28 18:16:23 +00:00
Josh Soref	d0888edc0a	Spelling fixes ... Fix spelling of: above, already, anonymous, associated, authentication, bandwidth, because, between, blocks, calculate, candidates, cautious, changelog, cleaner, clipboard, command, completely, concurrently, considered, constructs, corrupt, current, daemon, dependencies, deprecated, directory, dispatcher, download, eligible, ellipsis, encrypter, endpoint, entrieslist, essentially, existing writers, existing, expires, filesystem, flushing, frequently, hierarchy, however, implementation, implements, inaccurate, individually, insensitive, longer, maximum, metadata, modified, multipart, namedirfirst, nextcloud, obscured, opened, optional, owncloud, pacific, passphrase, password, permanently, persimmon, positive, potato, protocol, quota, receiving, recommends, referring, requires, revisited, satisfied, satisfies, satisfy, semver, serialized, session, storage, strategies, stringlist, successful, supported, surprise, temporarily, temporary, transactions, unneeded, update, uploads, wrapped Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>	2020-10-14 15:21:31 +01:00
LaSombra	aac2406e19	cmd: add --progress-terminal-title to print ETA to terminal title ... Adds a flag, --progress-terminal-title, that when used with --progress, will print the string `ETA: %s` to the terminal title. This also adds WriteTerminalTitle to lib/terminal	2020-10-06 16:34:26 +01:00
Hekmon	c58023a9ba	enhance systemd integration ... * log level identification * manual activation with flag * automatic systemd launch detection	2020-10-06 16:21:30 +01:00
Nick Craig-Wood	26001d520a	fs: add --bwlimit-file flag to limit speeds of individual file transfers	2020-07-28 11:46:24 +01:00
Nick Craig-Wood	3efdf5e095	operations: implement --refresh-times flag to set modtimes on hashless backends	2020-06-17 10:48:13 +01:00
fishbullet	ba5eb230fb	operations: interactive mode -i/--interactive for destructive operations #3886	2020-06-10 12:33:53 +01:00
Nick Craig-Wood	518d39815c	sync,copy,move: add --check-first to do all checking before starting transfers ... See: https://forum.rclone.org/t/rclone-sync-doing-transfer-and-checking-in-paralel/16352/	2020-05-22 17:50:07 +01:00
Martin Michlmayr	4aee962233	doc: fix typos throughout docs and code	2020-05-20 15:54:51 +01:00
Ben Zenker	899c8e0697	march: added flag to allow Unicode filenames to remain unique ... If your filenames contain two near-identical Unicode characters, rclone will normalize these, making them identical. This flag gives you the ability to keep them unique. This might create unintended side effects, such as duplicating files that contain certain Unicode characters, when downloading them from certain cloud providers to a macOS filesystem. Fixes #4228	2020-05-15 12:28:01 +01:00
Nick Craig-Wood	f6346a4d29	fs: add --header flag to add options to every HTTP transaction #59	2020-04-23 15:24:21 +01:00
Tim Gallant	93caa459e3	fs/config: add header-download and header-upload flags	2020-04-23 11:07:21 +01:00
Jon Fautley	53874bd8ee	cmd: add --error-on-no-transfer option ... This allows rclone to exit with a non-zero return code if no files are transferred. This is useful when calling rclone as part of a workflow/script pipeline as it allows the end user to stop processing if no files have been transferred. NB: Enabling this option will return in rclone exiting non-zero if there are no transfers. Depending on how your're currently using rclone in your scripts, this may break existing setups!	2020-04-15 17:06:40 +01:00
Bernd Schoolmann	158870bcdb	fs: Add --track-renames-strategy for configurable matching criteria for --track-renames ... This commit adds the `--track-renames-strategy` flag which allows the user to choose the strategy for tracking renames when using the `--track-renames` flag. This can be "hash" or "modtime" or both currently. This, when used with `--track-renames-strategy modtime` enables support for tracking renames in encrypted remotes. Fixes #3696 Fixes #2721	2020-03-20 13:04:56 +00:00
Franklyn Tackitt	2b3d13a841	fs: Use --cutoff-mode hard,soft,catious instead of 3 --max-transfer-mode flags ... Fixes #2672	2020-03-01 09:49:55 +00:00
Shing Kit Chan	6f1766dd9e	fs: Add support for --max-transfer-cutoff modes #2672 ... This also adds max transfer cut off check for server side copies too	2020-03-01 09:49:55 +00:00
boosh	0d7573dd81	fs: Add --max-duration flag to control the maximum duration of a transfer session ... This gives you more control over how long rclone will run for, making it easier to script backups, e.g. via cron. Once the `--max-duration` time limit is reached, no new transfers will be initiated, but those already in-flight will be allowed to complete. Fixes #985	2020-01-24 13:28:56 +00:00
Nick Craig-Wood	3afb2a4798	config: use SpaceSepList for argument to --password-command ... This is to enable arguments with spaces in.	2020-01-23 15:39:15 +00:00
Damon Permezel	06df133159	config: add --password-command to allow dynamic config password - fixes #3694	2020-01-23 15:39:15 +00:00
Nick Craig-Wood	277d94feac	fshttp: add --expect-continue-timeout default 1s - fixes #3835 ... Before this change the expect/continue timeout was set to --conntimeout which was 60s by default which is too long to wait. This was noticed when using s3 with a proxy which apparently didn't support expect / continue properly. Set --expect-continue-timeout 0 to disable expect/continue.	2020-01-18 09:49:22 +00:00
Nick Craig-Wood	375d25f158	sync: implement --order-by flag to order transfers - fixes #1205	2020-01-16 15:24:36 +00:00
Nick Craig-Wood	207474abab	sync: add --no-check-dest flag - fixes #3616	2019-12-29 16:47:57 +00:00
Ankur Gupta	75a6c49f87	Fix error counter - fixes #3650 ... For few commands, RClone counts a error multiple times. This was fixed by creating a new error type which keeps a flag to remember if the error has already been counted or not. The CountError function now wraps the original error eith the above new error type and returns it.	2019-11-18 14:13:02 +00:00
Nick Craig-Wood	24161d12ab	fs: make sure config is persisted to the config file when using config.Mapper	2019-08-14 20:54:08 +01:00
Nick Craig-Wood	eb087a3b04	operations: disable multi thread copy for local to local copies #3419 ... ...unless --multi-thread-streams has been set explicitly	2019-08-14 15:07:38 +01:00
Nick Craig-Wood	3ecbd603ab	rc: move job expire flags to rc to fix initalization problem ... See: https://forum.rclone.org/t/rc-rc-job-expire-interval-bug/11188 rclone was ignoring the --rc-job-expire-duration and --rc-job-interval flags. This turned out to be an initialization order problem and was fixed by moving those flags out of global config into rc config.	2019-08-10 17:12:22 +01:00
Aleksandar Jankovic	68e641f6cf	accounting: add limits and listing to stats groups	2019-07-28 14:48:19 +01:00
justinalin	520fb03bfd	log: add --use-json-log for JSON logging	2019-07-28 12:05:50 +01:00
yparitcher	8e8b78d7e5	Implement --compare-dest & --copy-dest Fixes #3278	2019-07-22 19:42:29 +01:00
Aleksandar Jankovic	93207ead9c	rc/jobs: make job expiry timeouts configurable	2019-06-15 09:55:32 +01:00
garry415	1124c423ee	fs: add --ignore-case-sync for forced case insensitivity - fixes #2773	2019-06-03 21:12:10 +01:00
Nick Craig-Wood	7d70e92664	operations: enable multi threaded downloads - Fixes #2252 ... This implements the --multi-thread-cutoff and --multi-thread-streams flags to control multi thread downloading to the local backend.	2019-05-11 23:35:19 +01:00
Peter Berbec	0d511b7878	cmd: implement --stats-one-line-date and --stats-one-line-date-format	2019-05-11 17:39:57 +01:00