Commit Graph

1817 Commits

Author SHA1 Message Date
Ivan Andreev
0d7426a2dd hasher: backend documentation #5587 2021-10-20 19:11:54 +03:00
Ivan Andreev
50df8cec9c lib/kv: add key-value database api #5587
Add bolt-based key-value database support.

Quick API description:
https://github.com/rclone/rclone/pull/5587#issuecomment-942174768
2021-10-20 19:11:54 +03:00
Ivan Andreev
6cda4c2c3c Add Ian Levesque to contributors 2021-10-19 23:02:51 +03:00
Ivan Andreev
023b666863 Add Filip Rysavy to contributors 2021-10-19 23:01:47 +03:00
Ivan Andreev
2a4c6ad0e7 Add Matthew Sevey to contributors 2021-10-19 23:00:16 +03:00
Ivan Andreev
6d02530f9d sia: finish documentation #4514
Also rename stuttering `--sia-sia-user-agent` to `--sia-user-agent`
2021-10-19 22:55:27 +03:00
Ian Levesque
3351b1e6ae sia: add backend for sia decentralized cloud #4514 2021-10-19 22:55:27 +03:00
Nick Craig-Wood
3cef84aabe Add r0kk3rz to contributors 2021-10-19 20:12:17 +01:00
Nick Craig-Wood
93afd5c346 Add Rajat Goel to contributors 2021-10-19 20:12:17 +01:00
Alex Chen
1c3c8babd3
docs: mention make for building and cmount tag for macos (#5487) 2021-10-19 12:18:06 +08:00
Ivan Andreev
bbcc9a45fe serve/docker: allow to customize proxy settings of docker plugin 2021-10-18 18:03:06 +03:00
r0kk3rz
8e4b87ae03
s3: Add AWS Snowball Edge to providers examples - fixes #5720 2021-10-18 12:52:59 +01:00
Nick Craig-Wood
af732c5431 Add Logeshwaran to contributors 2021-10-14 17:15:48 +01:00
Nick Craig-Wood
14de0cfb43 Add Joda Stößer to contributors 2021-10-14 17:14:53 +01:00
albertony
c2597a4fa3
docs: cleanup header levels in backend docs (#5698) 2021-10-14 15:40:18 +02:00
Joda Stößer
d41b9b46d4 docs: improve ordered list prefix for drive.md 2021-10-14 11:08:15 +01:00
Ivan Andreev
16fb608bee hashsum: treat hash values in sum file as case insensitive
Also warn duplicate file paths in sum files.

Fixes https://forum.rclone.org/t/rclone-check-sum/25566/45
2021-10-13 18:21:34 +03:00
albertony
7d66bfbb7c
docs: toc styling (#5695) 2021-10-13 15:04:11 +02:00
Ivan Andreev
1e7db7193e docs: note minimum supported docker engine 2021-10-12 13:27:20 +03:00
albertony
c7329d2ece docs: add section in install documentation about portable install
See #5591
2021-10-11 15:08:35 +02:00
albertony
0ffdca42d5 docs: document --cache-dir flag 2021-10-11 15:08:35 +02:00
albertony
dbb6f94d95 config: make temporary directory user-configurable
See #5591
2021-10-11 15:08:35 +02:00
Nick Craig-Wood
0dfffc0ed4 Add YenForYang to contributors 2021-10-11 10:43:16 +01:00
Alfonso Montero
19fc1b2a95
docs/compress: minor improvements 2021-10-09 18:22:38 +02:00
Ivan Andreev
bce395385d mount/docs: improve wording 2021-10-09 18:53:57 +03:00
albertony
a5b8fcc127 docs: align dropdown items when icons have different sizes 2021-10-09 18:49:05 +03:00
Ankur Gupta
c8d5606f2c Removed multiple emails for Ankur Gupta 2021-10-08 10:25:42 +01:00
Ivan Andreev
15a9816512 ftp: update encoding in integration tests with ProFtpd, PureFtpd, VsFtpd
PR #5589 established recommended encodings to use with major FTP servers.
This patch updates integration tests correspondingly.
2021-10-05 21:45:08 +03:00
Ivan Andreev
cace18d89a docs/ftp: state clearly that active mode is not supported 2021-10-05 15:52:50 +03:00
Ivan Andreev
a065fb23e5 mount: document the mount helper mode, make command docs - #5594 2021-10-03 01:44:08 +03:00
Ivan Andreev
68be24c88d log: optionally print pid in logs - #5593
This option is useful to troubleshoot `rclone mount --daemon`
2021-10-02 23:45:16 +03:00
albertony
26b6c83e49 docs: extend documentation on valid remote names 2021-10-01 15:18:04 +02:00
Ivan Andreev
dc589d3070 ftp: provide valid encoding for ProFtpd, PureFtpd, VsFtpd 2021-10-01 10:09:57 +03:00
Ivan Andreev
48e7246163 lib/encoder: add encoding of square brackets 2021-10-01 10:09:57 +03:00
albertony
b5a27b1c75 docs: cleanup header levels 2021-09-30 17:54:57 +02:00
albertony
db7db952c1 Add Jonta to contributors 2021-09-30 15:38:24 +02:00
Jonta
d8d621c175
docs: grammar/readability (#5633) 2021-09-30 15:34:00 +02:00
Nick Craig-Wood
5b6bcfc184 Add HNGamingUK to contributors 2021-09-28 11:51:41 +01:00
HNGamingUK
1409b89f6c
swift: document OVH Cloud Archive - fixes ##3041
Added documentation for OVH Cloud Archive, provides information on how to restore/unfreeze/download objects.
2021-09-20 17:32:13 +01:00
Nick Craig-Wood
3298493b0b Add wzl to contributors 2021-09-17 10:43:45 +01:00
wzl
b35db61a80 docs: add a step for drive.md 2021-09-11 23:00:59 +03:00
Abhinav Sharma
3c17762c4e update the email 2021-09-11 13:08:29 +03:00
albertony
f1f923a986 Change byte unit format from MiByte to MiB 2021-08-31 09:57:27 +02:00
Nick Craig-Wood
2cefae51a1 build: make go1.14 the minimum supported Go for building 2021-08-20 19:04:29 +01:00
Nick Craig-Wood
dd97fbc55f Add Parth Shukla to contributors 2021-08-20 19:04:23 +01:00
Nick Craig-Wood
b32d00ba37 Add Justin Hellings to contributors 2021-08-20 19:04:23 +01:00
Nick Craig-Wood
227119da16 Add Ken Enrique Morel to contributors 2021-08-14 12:53:36 +01:00
Ken Enrique Morel
3626f10f26 pcloud: add sha256 support - fixes #5496 2021-08-14 12:48:25 +01:00
Nick Craig-Wood
bec253fd39 Add vinibali to contributors 2021-08-11 11:55:09 +01:00
Nick Craig-Wood
815a6ac8aa Add hota to contributors 2021-08-11 11:55:09 +01:00
Ivan Andreev
8106f65e0b Add yedamo to contributors 2021-08-11 11:07:13 +03:00
Greg Sadetsky
36f0231082
docs/drive: Fix lsf example without drive-impersonate (#5504) 2021-08-10 21:59:36 +02:00
albertony
168cb65c61 Add Greg Sadetsky to contributors 2021-08-10 21:50:26 +02:00
Greg Sadetsky
e00db968aa
docs/s3: fix typo in s3 documentation (#5515) 2021-08-10 21:45:49 +02:00
partev
bb6b44d199 DOC: "OS X" -> "macOS" 2021-08-10 10:12:30 +03:00
vinibali
88b35bc32d Update yandex.md
add mail subscription exception
2021-08-09 23:28:41 +03:00
Nathan Collins
c32d5dd1f3 fs: move with --ignore-existing will not delete skipped files - #5463 2021-08-01 17:46:45 +01:00
Greg Sadetsky
3d9da896d2
drive: fix instructions for auto config #5499 2021-08-01 15:17:07 +01:00
hota
839c20bb35 s3: add Wasabi's AP-Northeast endpoint info
* Wasabi starts to provide AP Northeast (Tokyo) endpoint for all customers, so add it to the list

Signed-off-by: lindwurm <lindwurm.q@gmail.com>
2021-08-01 14:56:52 +01:00
Nick Craig-Wood
7c58148840 Start v1.57.0-DEV development 2021-08-01 13:43:36 +01:00
Nick Craig-Wood
1d280081d4 Add Mariano Absatz (git) to contributors 2021-07-30 19:31:02 +01:00
Nick Craig-Wood
f48cb5985f Add Justin Winokur (Jwink3101) to contributors 2021-07-30 19:31:02 +01:00
Alex Chen
63a24255f8
onedrive: handle HTTP 400 better in PublicLink() (#5419) 2021-07-27 17:55:57 +08:00
Mariano Absatz (git)
f39a08c9d7 clarification of the process for creating custom client_id 2021-07-24 09:19:48 +03:00
Nick Craig-Wood
37ff05a5fa Version v1.56.0 2021-07-20 19:45:41 +01:00
Nick Craig-Wood
11d86c74b2 docs: expand contents and make docs full screen 2021-07-20 16:53:21 +01:00
Nick Craig-Wood
feb6046a8a docs: add table of contents to every page 2021-07-20 16:53:21 +01:00
buengese
da36ce08e4 docs/jottacloud: add short note on how no versions option works 2021-07-15 17:29:30 +02:00
buengese
8652cfe575 jottacloud: add no versions option 2021-07-15 17:29:30 +02:00
Nick Craig-Wood
97c9e55ddb Add Antoine GIRARD to contributors 2021-07-14 11:40:57 +01:00
Ivan Andreev
c0b2832509 docs: serve docker: fix URL of systemd contrib files (#5415) 2021-07-11 13:23:00 +03:00
Ivan Andreev
7436768d62 docs for serve docker and docker plugin (#5415) 2021-07-10 23:56:09 +03:00
Nick Craig-Wood
aab29353d1 Update email address for Serge Pouliquen 2021-07-08 12:49:13 +01:00
Nick Craig-Wood
c24504b793 Add Chuan Zh to contributors 2021-07-08 12:47:35 +01:00
Nick Craig-Wood
6338d0026e Add Michael Hanselmann to contributors 2021-07-08 12:47:35 +01:00
Ole Frost
367cf984af docs: added tip to reduce SharePoint throttling - fixes #5404 2021-07-08 11:39:52 +01:00
Nick Craig-Wood
abb8fe8ba1 Add Haochen Tong to contributors 2021-07-07 16:59:51 +01:00
Nick Craig-Wood
765af387e6 Add Dmitry Sitnikov to contributors 2021-07-07 16:59:51 +01:00
Nick Craig-Wood
d05cf6aba8 Add partev to contributors 2021-07-07 16:59:51 +01:00
Nick Craig-Wood
76a3fef24d Add Xuanchen Wu to contributors 2021-07-07 16:59:51 +01:00
Ivan Andreev
b40d9bd4c4
cmd: add hashSUM file support (#5352)
Currently rclone check supports matching two file trees by sizes and hashes.
This change adds support for SUM files produced by GNU utilities like sha1sum.

Fixes #1005 

Note: checksum by default checks, hashsum by default prints sums.
New flag is named "--checkfile" but carries hash name.
Summary of introduced command forms:

```
rclone check sums.sha1 remote:path --checkfile sha1
rclone checksum sha1 sums.sha1 remote:path             
rclone hashsum sha1 remote:path --checkfile sums.sha1
rclone sha1sum remote:path --checkfile sums.sha1
rclone md5sum remote:path --checkfile sums.md5
```
2021-07-07 18:34:16 +03:00
Ole Frost
5e91b93e59 cmdtest: end-to-end test for commands, flags and environment variables
There was no easy way to automatically test the end-to-end functionality
of commands, flags, environment variables etc.

The need for end-to-end testing was highlighted by the issues fixed
in #5341. There was no automated test to continually verify current
behaviour, nor a framework to quickly test the correctness of the fixes.

This change adds an end-to-end testing framework in the cmdtest folder.
It has some simple examples in func TestCmdTest in cmdtest_test.go. The
tests should be readable by anybody familiar with rclone and look like
this:

    // Test the rclone version command with debug logging (-vv)
    out, err = rclone("version", "-vv")
    if assert.NoError(t, err) {
        assert.Contains(t, out, "rclone v")
        assert.Contains(t, out, "os/version:")
        assert.Contains(t, out, " DEBUG : ")
    }

The end-to-end tests are executed just like the Go unit tests, that is:

    go test ./cmdtest -v

The change also contains a thorough test of environment variables in
environment_test.go.

Thanks to @ncw for encouragement and introduction to the TestMain trick.
2021-07-05 16:38:20 +01:00
Ole Frost
58c99427b3 config: fixed issues with flags/options set by environment vars.
Some environment variables didn’t behave like their corresponding
command line flags. The affected flags were --stats, --log-level,
--separator, --multi-tread-streams, --rc-addr, --rc-user and --rc-pass.
Example:

    RCLONE_STATS='10s'
    rclone check remote: remote: --progress
    # Expected: rclone check remote: remote: --progress –-stats=10s
    # Actual: rclone check remote: remote: --progress

Remote specific options set by environment variables was overruled by
less specific backend options set by environment variables. Example:

    RCLONE_DRIVE_USE_TRASH='false'
    RCLONE_CONFIG_MYDRIVE_USE_TRASH='true'
    rclone deletefile myDrive:my-test-file
    # Expected: my-test-file is recoverable in the trash folder
    # Actual: my-test-file is permanently deleted (not recoverable)

Backend specific options set by environment variables was overruled by
general backend options set by environment variables. Example:

    RCLONE_SKIP_LINKS='true'
    RCLONE_LOCAL_SKIP_LINKS='false'
    rclone lsd local:
    # Expected result: Warnings when symlinks are skipped
    # Actual result: No warnings when symlinks are skipped
    # That is RCLONE_SKIP_LINKS takes precedence

The above issues have been fixed.

The debug logging (-vv) has been enhanced to show when flags are set by
environment variables.

The documentation has been enhanced with details on the precedence of
configuration options.

See pull request #5341 for more information.
2021-07-05 16:38:20 +01:00
albertony
fee0abf513 docs: add note about use of user and logname environment variables for current username 2021-07-05 16:31:16 +01:00
Ivan Andreev
1cc6dd349e Add google search widget to rclone.org 2021-07-05 16:21:36 +01:00
Dmitry Sitnikov
b439199578
azureblob: Fix typo in Azure Blob help
Change the command to create RBAC file to the correct one
`az ad sp create-for-rbac`
Add the link to the command documentation
https://docs.microsoft.com/en-us/cli/azure/ad/sp?view=azure-cli-latest#az_ad_sp_create_for_rbac
2021-07-05 15:58:41 +01:00
partev
0bfd6f793b
docs: replace OSX with macOS 2021-07-05 14:51:00 +01:00
Nick Craig-Wood
04608428bf Add Florian Penzkofer to contributors 2021-06-11 14:21:23 +01:00
Nick Craig-Wood
6aaa06d7be Add darrenrhs to contributors 2021-06-11 14:21:23 +01:00
Nick Craig-Wood
e53bad5353 Add Reid Buzby to contributors 2021-06-11 14:21:23 +01:00
Nick Craig-Wood
f5397246eb Add Chris Lu to contributors 2021-06-11 14:21:23 +01:00
Nick Craig-Wood
b8b73f2656 Add database64128 to contributors 2021-06-11 14:21:23 +01:00
Nick Craig-Wood
96b67ce0ec Add Tyson Moore to contributors 2021-06-11 14:21:23 +01:00
Nick Craig-Wood
e2beeffd76 Add Tom to contributors 2021-06-11 14:21:23 +01:00
Nick Craig-Wood
30b949642d Add acsfer to contributors 2021-06-11 14:21:23 +01:00
Ivan Andreev
062919e08c
deprecate cache backend (#5382) 2021-06-10 19:52:55 +03:00
darrenrhs
654f5309b0
docs: drive: include requirement to publish app in step-by-step - fixes #5393 2021-06-10 17:00:52 +01:00
Reid Buzby
5104e24153
docs: fix incorrect token type for yandex
https://forum.rclone.org/t/yandex-documentation/24445/2
2021-06-09 13:04:55 +02:00
albertony
9d87a5192d docs: fix code section formatting in filtering docs
Fixes #5387
2021-06-08 18:53:18 +02:00
Ivan Andreev
29f967dba3
make commanddocs for v1.56 (#5383) 2021-06-08 18:57:04 +03:00
Chris Lu
1f846c18d4
s3: Add SeaweedFS 2021-06-08 09:59:57 +01:00
database64128
58006a925a 📑 GCS: Update docs on mtime
- Mention the new modification time behavior and the modify window issue.
- Unify markdown format.
- ref rclone/rclone#5331
2021-05-28 14:44:50 +01:00
Tyson Moore
2188fe38e5 docs: add caveat about DSCP on Windows 2021-05-28 13:43:38 +01:00
acsfer
c9fce20249
tardigrade: add warning about too many open files - Fixes #5310 2021-05-21 20:04:57 +01:00
Ivan Andreev
5b6f637461
fs/hash: align hashsum names and update documentation (#5339)
- Unify all hash names as lowercase alphanumerics without punctuation.
- Legacy names continue to work but disappear from docs, they can be depreciated or dropped later.
- Make rclone hashsum print supported hash list in case of wrong spelling.
- Update documentation.

Fixes #5071
Fixes #4841
2021-05-21 17:32:33 +03:00
albertony
07f2f3a62e docs: fix link to paths on windows section 2021-05-19 22:11:17 +02:00
albertony
6dc190ec93 docs: mention that network/unc paths are supported in local filesystem on windows 2021-05-19 22:11:17 +02:00
Nick Craig-Wood
f572bf7829 Add sp31415t1 to contributors 2021-05-17 12:10:58 +01:00
sp31415t1
f593558dc2
docs: improve --disable help 2021-05-14 15:44:58 +01:00
Ivan Andreev
08040a57b0
dropbox: improve "own App IP" instructions (#5325)
Instructions in https://rclone.org/dropbox/#get-your-own-dropbox-app-id
are a little incomplete. I had to guess a few extra details to make things work.
This patch adds missing parts.

Fixes #5242
2021-05-14 17:42:30 +03:00
Nick Craig-Wood
798d1293df Add Alexey Ivanov to contributors 2021-05-14 14:07:44 +01:00
Nick Craig-Wood
2e4b65f888 dropbox: add --dropbox-batch-mode flag to speed up uploading #5156
This adds 3 upload modes for dropbox off, sync and async and makes
sync the default.

This should improve uploads (especially for small files) greatly.
2021-05-14 14:07:44 +01:00
Nick Craig-Wood
6f2e525821 Add Antoon Prins to contributors 2021-05-14 14:07:44 +01:00
Antoon Prins
326f3b35ff webdav: add headers option 2021-05-12 09:52:07 +01:00
Nick Craig-Wood
9ff6f48d74 Remove accidentally committed *.orig and *.rej files and ignore 2021-05-03 07:58:29 +01:00
Nick Craig-Wood
532af77fd1 Add Chris Macklin to contributors 2021-05-03 07:58:29 +01:00
Nick Craig-Wood
8bef972262 Add Gautam Kumar to contributors 2021-04-30 20:31:04 +01:00
Nick Craig-Wood
0a968818f6 Add Nolan Woods to contributors 2021-04-30 20:31:04 +01:00
Nick Craig-Wood
c2ac353183 Add lewisxy to contributors 2021-04-30 20:31:04 +01:00
Nick Craig-Wood
773da395fb Add Tatsuya Noyori to contributors 2021-04-30 20:31:04 +01:00
Romeo Kienzler
88aabd1f71
docs: corrected spelling
from "Check the integrity of an encrypted remote." to "Check the integrity of a crypted remote."
2021-04-28 22:50:55 +01:00
Nick Craig-Wood
742af80972 Add jtagcat to contributors 2021-04-27 18:18:08 +01:00
albertony
2925e1384c Use binary prefixes for size and rate units
Includes adding support for additional size input suffix Mi and MiB, treated equivalent to M.
Extends binary suffix output with letter i, e.g. Ki and Mi.
Centralizes creation of bit/byte unit strings.
2021-04-27 02:25:52 +03:00
albertony
98579608ec docs: cleanup spelling of size and rate units 2021-04-27 02:25:52 +03:00
albertony
f8d56bebaf
config: delay load config file (#5258)
Restructuring of config code in v1.55 resulted in config
file being loaded early at process startup. If configuration
file is encrypted this means user will need to supply the password,
even when running commands that does not use config.
This also lead to an issue where mount with --deamon failed to
decrypt the config file when it had to prompt user for passord.

Fixes #5236
Fixes #5228
2021-04-26 23:37:49 +02:00
Nick Craig-Wood
f4068d406b Add Jeffrey Tolar to contributors 2021-04-26 16:57:21 +01:00
Nick Craig-Wood
34dc257c55 Add Kenny Parsons to contributors 2021-04-26 16:55:50 +01:00
Kenny Parsons
4cacf5d30c
docs: clarify and add examples for sftp docs
- added clarification to default remote path if no path is specified 
- added examples for mounting a remote path (other than the default home directory) to a local folder.
2021-04-26 16:13:42 +01:00
Nick Craig-Wood
4b1d28550a Changelog updates from Version v1.55.1 2021-04-26 09:22:49 +01:00
Nick Craig-Wood
ffec0d4f03 Add OleFrost to contributors 2021-04-25 11:45:39 +01:00
Nick Craig-Wood
9799fdbae2 Add noabody to contributors 2021-04-20 16:36:38 +01:00
Nick Craig-Wood
492504a601 Add new email address for Caleb Case 2021-04-20 16:36:25 +01:00
Nick Craig-Wood
0c03a7fead Add Ansh Mittal to contributors 2021-04-20 16:31:40 +01:00
albertony
3544e09e95
config: treat any config file paths with filename notfound as memory-only config (#5235) 2021-04-18 00:09:03 +02:00
Nick Craig-Wood
7f5e347d94 Add Nazar Mishturak to contributors 2021-04-16 15:07:03 +01:00
albertony
60d376c323 docs: add guide to configuring autorun in install documentation 2021-04-13 22:33:19 +02:00
albertony
d8711cf7f9 config: create config file in windows appdata directory by default (#5226)
Use %AppData% as primary default for configuration file on Windows,
which is more in line with Windows standards, while existing default
of using home directory is more Unix standards - though that made rclone
more consistent accross different OS.

Fixes #4667
2021-04-13 19:25:09 +03:00
buengese
cd69f9e6e8 uptobox: add docs 2021-04-13 17:46:07 +02:00
Nick Craig-Wood
813bf029d4 Add Dominik Mydlil to contributors 2021-04-12 18:14:01 +01:00
albertony
b96ebfc40b docs: less confusing example with config path option 2021-04-12 18:17:19 +02:00
Dominik Mydlil
3fe2aaf96c crypt: support timestamped filenames from --b2-versions
With the file version format standardized in lib/version, `crypt` can
now treat the version strings separately from the encrypted/decrypted
file names. This allows --b2-versions to work with `crypt`.

Fixes #1627

Co-authored-by: Luc Ritchie <luc.ritchie@gmail.com>
2021-04-12 15:59:18 +01:00
Nick Craig-Wood
38a8071a58 Add Ashok Gelal to contributors 2021-04-12 15:52:31 +01:00
Ashok Gelal
8c68a76a4a
install.sh: silence the progress output with curl requests
This commit silences the progress output from the curl requests made by the install.sh script.

Having a progress seems to break some automated scripts and there isn't a way to pass some
flags to these curl requests to disable them.
2021-04-12 14:18:29 +01:00
Dan Dascalescu
e7b736f8ca
docs: fix minor typo in symlinks / junction points 2021-04-10 15:34:34 +02:00
albertony
ddb7eb6e0a
docs: fixed some typos 2021-04-08 10:19:03 +02:00
Nick Craig-Wood
fcba51557f dropbox: set visibility in link sharing when --expire is set
Note that due to a bug in the dropbox SDK you'll need to set --expire
to access this.

See: https://github.com/dropbox/dropbox-sdk-go-unofficial/issues/75
See: https://forum.rclone.org/t/rclone-link-dropbox-permissions/23211
2021-04-07 13:58:37 +01:00
Nick Craig-Wood
5fa8e7d957 Add Nick Gaya to contributors 2021-04-06 10:01:49 +01:00
Nick Gaya
1fd9b483c8
onedrive: add list_chunk option
Add --onedrive-list-chunk option similar to existing options for azureblob, drive, and s3.

Suggested as a workaround for a OneDrive pagination bug

See: https://forum.rclone.org/t/unexpected-duplicates-on-onedrive-with-0s-in-filename/23164/8
2021-04-04 09:08:16 +01:00
Nick Craig-Wood
9e2fbe0f1a install.sh: fix macOS arm64 download - fixes #5183 2021-03-31 21:48:31 +01:00
Nick Craig-Wood
6d65d116df Start v1.56.0-DEV development 2021-03-31 19:51:43 +01:00