Commit Graph

7358 Commits

Author SHA1 Message Date
Pat Patterson
71f2883562 operations: ensure concurrency is no greater than the number of chunks - fixes #7299 2023-09-13 12:12:27 +01:00
Nick Craig-Wood
d29d263329 docs: fix minimum Go version and update to 1.18
Fixes #7288
2023-09-12 11:24:17 +01:00
Nick Craig-Wood
20126de1aa Start v1.65.0-DEV development 2023-09-12 11:24:17 +01:00
Nick Craig-Wood
77f7bb08af Version v1.64.0 2023-09-11 15:59:44 +01:00
Nick Craig-Wood
a5a61f4874 protondrive: make cached keys rclone style and not show with rclone config redacted 2023-09-11 15:57:08 +01:00
Nick Craig-Wood
e8879f3e77 docs: document release signing and verification
Fixes #7257
2023-09-11 12:28:23 +01:00
Nick Craig-Wood
2a6675cffd docs: fix typo in rc docs - fixes #7287 2023-09-11 09:34:22 +01:00
Nick Craig-Wood
fa4d171f62 protondrive: complete docs with all references to Proton Drive 2023-09-10 11:57:39 +01:00
Nick Craig-Wood
d4d530bd8e drive: add --drive-fast-list-bug-fix to control ListR bug workaround
See: https://forum.rclone.org/t/how-to-list-empty-directories-recursively/40995/12
2023-09-09 17:46:03 +01:00
Nick Craig-Wood
f4b011e4e4 s3: add rclone backend restore-status command
This command shows the restore status of objects being retrieved from GLACIER.

See: https://forum.rclone.org/t/aws-s3-glacier-monitor-restore-status-command-for-glacier-restoring-process/41373/7
2023-09-09 17:44:36 +01:00
Nick Craig-Wood
d80890bf32 Add Drew Stinnett to contributors 2023-09-09 17:44:36 +01:00
Nick Craig-Wood
39392d70dd Add David Pedersen to contributors 2023-09-09 17:44:36 +01:00
Drew Stinnett
643386f026 rc: Add operations/settier to API 2023-09-09 17:41:02 +01:00
Chun-Hung Tseng
ed755bf04f
protondrive: implement two-password mode (#7279) 2023-09-08 22:54:46 +02:00
David Pedersen
071c3f28e5 vfs: Update parent directory modtimes on vfs actions
This isn't written back to the storage so might change on remount but
makes the VFS just a little more POSIX compatible.
2023-09-08 17:19:52 +01:00
Nick Craig-Wood
7453b7d5f3 hdfs: fix retry "replication in progress" errors when uploading
Before this change uploaded files could return the error "replication
in progress".

This error is harmless though and means the Close should be retried
which is what this patch does.
2023-09-08 15:35:50 +01:00
Nick Craig-Wood
c9350149d8 hdfs: fix uploading to the wrong object on Update with overriden remote name
In this commit we discovered a problem with objects being uploaded to
the incorrect object name. It added an integration test for the
problem.

65b2e378e0 drive: fix incorrect remote after Update on object

This test was tripped by the hdfs backend and this patch fixes the
problem.
2023-09-08 15:35:50 +01:00
Nick Craig-Wood
08789a5815 test_all: remove filefabric from integration tests
The filefabric test server doesn't seem to be working at all so remove
it from the integration tests pending revitalization.
2023-09-08 15:35:50 +01:00
Nick Craig-Wood
4037af9c1a Add Oksana and Volodymyr Kit to contributors 2023-09-08 15:35:50 +01:00
Oksana
628ff8e524
quatrix: add backend to support Quatrix
Co-authored-by: Volodymyr Kit <v.kit@maytech.net>
2023-09-08 14:31:29 +01:00
Chun-Hung Tseng
578c75cb1e
protondrive: fix signature verification logic by accounting for legacy signing scheme (#7278) 2023-09-08 16:00:34 +08:00
Nick Craig-Wood
63ab250817 vfs: add --vfs-cache-min-free-space to control minimum free space on the disk containing the cache
See: https://forum.rclone.org/t/rclone-fails-to-control-disk-usage-and-its-filling-the-disk-to-100/41494/
2023-09-07 15:57:45 +01:00
Nick Craig-Wood
39f910a65d rc: add core/du to measure local disk usage 2023-09-07 15:57:45 +01:00
Nick Craig-Wood
0fb36562dd Add lib/diskusage to measure used/free on disks 2023-09-07 15:57:45 +01:00
Nick Craig-Wood
8c25a15a40 Add zjx20 to contributors 2023-09-07 15:57:45 +01:00
zjx20
f5ee16e201 local: rmdir return an error if the path is not a dir 2023-09-07 14:30:08 +01:00
Nick Craig-Wood
2bcbed30bd s3: implement backend set command to update running config 2023-09-07 12:26:48 +01:00
Chun-Hung Tseng
5026a9171d
protondrive: improves 2fa and draft error messages (#7280) 2023-09-07 01:50:28 +08:00
Nick Craig-Wood
b750c50bfd zoho: remove Range requests workarounds to fix integration tests
Zoho are now responding to Range requests properly. The remnants of
our old workaround was breaking the integration tests so this removes
them.
2023-09-05 18:21:15 +01:00
Nick Craig-Wood
535acd0483 fstests: fix PublicLink failing on storj
Storj requires a minimum duration of 1 minute for the link expiry so
increase what we are asking for from 1 minute to 2 minutes.
2023-09-05 18:01:37 +01:00
Nick Craig-Wood
db37b3ef9e opendrive: fix List on a just deleted and remade directory
Sometimes opendrive reports "403 Folder is already deleted" on
directories which should exist.

This might be a bug in opendrive or in rclone however we work-around
here sufficient to get the tests passing.
2023-09-05 17:59:03 +01:00
Nick Craig-Wood
257607ab3d operations: fix TestCopyFileMaxTransfer test to not be quite so fussy 2023-09-05 17:42:17 +01:00
Nick Craig-Wood
3ea1c5c4d2 compress: fix ChangeNotify
ChangeNotify has been broken on the compress backend for a long time!

Before this change it was wrapping the file names received rather than
unwrapping them to discover the original names.

It is likely ChangeNotify was working adequately though for users as
the VFS just uses the directories rather than the file names.
2023-09-05 17:22:36 +01:00
Nick Craig-Wood
bd23ea028e azureblob: fix purging with directory markers 2023-09-05 17:07:44 +01:00
Nick Craig-Wood
c58d4fe939 test_all: ignore Rmdirs test failure on b2 as it fails because of versions 2023-09-05 15:47:14 +01:00
nielash
ddc7059a73 Add @nielash as bisync maintainer 2023-09-05 04:05:39 -04:00
dependabot[bot]
2677c43f26 build(deps): bump actions/checkout from 3 to 4
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-05 08:55:32 +01:00
nielash
48ab67f090 bisync: fix dryRun rc parameter being ignored
Before this change, bisync ignored the dryRun parameter (only when specified
via the rc.)

This change fixes the issue, so that the dryRun rc parameter is equivalent to
the --dry-run flag.
2023-09-05 08:53:58 +01:00
nielash
089df7d977 bisync: add rc parameters for new flags
Added rc support for the flags recently introduced in #6971.

createEmptySrcDirs
ignoreListingChecksum
resilient
2023-09-05 08:53:58 +01:00
Nick Craig-Wood
4fbe0652c9 compress: fix integration tests by adding missing OpenChunkWriter exclude 2023-09-04 19:26:14 +01:00
Nick Craig-Wood
47665dad07 cache: fix integration tests by adding missing OpenChunkWriter exclude 2023-09-04 19:26:14 +01:00
eNV25
ad724463a5
cmd: refactor and use sysdnotify in more commands
* cmd: refactor and use sysdnotify in more commands

Fixes #5117
2023-09-04 16:32:04 +01:00
Nick Craig-Wood
6afd7088d3 box: add --box-impersonate to impersonate a user ID - fixes #7267 2023-09-04 12:09:54 +01:00
Nick Craig-Wood
b33140ddeb union: add :writback to act as a simple cache
This adds a :writeback tag to upstreams. If set on a single upstream
then it writes back objects not found into that upstream.

Fixes #6934
2023-09-04 12:03:26 +01:00
Nick Craig-Wood
b1c0ae5e7d azureblob: fix creation of directory markers
This also fixes the integration tests which is why we didn't notice this before!
2023-09-03 18:09:31 +01:00
Nick Craig-Wood
40bcc7a90b fstest: fix sftp ssh integration tests
This adds a private and public key to the SFTP SSH test so that it
works when it doesn't have access to my ssh agent!
2023-09-03 15:23:12 +01:00
Nick Craig-Wood
be17f1523a b2: fix ChunkWriter size return 2023-09-03 13:53:11 +01:00
Nick Craig-Wood
bb58040d9c s3: fix multpart streaming uploads of 0 length files 2023-09-03 12:37:20 +01:00
Nick Craig-Wood
2db0e23584 backends: change OpenChunkWriter interface to allow backend concurrency override
Before this change the concurrency used for an upload was rather
inconsistent.

- if size below `--backend-upload-cutoff` (default 200M) do single part upload.

- if size below `--multi-thread-cutoff` (default 256M) or using streaming
  uploads (eg `rclone rcat) do multipart upload using
  `--backend-upload-concurrency` to set the concurrency used by the uploader.

- otherwise do multipart upload using `--multi-thread-streams` to set the
  concurrency.

This change makes the default for the concurrency used be the
`--backend-upload-concurrency`. If `--multi-thread-streams` is set and larger
than the `--backend-upload-concurrency` then that will be used instead.

This means that if the user sets `--backend-upload-concurrency` then it will be
obeyed for all multipart/multi-thread transfers and the user can override them
all with `--multi-thread-streams`.

See: #7056
2023-09-03 11:47:05 +01:00
Nick Craig-Wood
a7337b0a95 Add Alishan Ladhani to contributors 2023-09-03 11:47:05 +01:00