Nick Craig-Wood
d24e5bc7e4
build: upgrade golang.org/x/net to v0.17.0 to fix HTTP/2 rapid reset
...
Vulnerability1: GO-2023-2102
HTTP/2 rapid reset can cause excessive work in net/http
More info: https://pkg.go.dev/vuln/GO-2023-2102
2023-10-12 20:08:30 +01:00
Nick Craig-Wood
077c1a0f57
b2: fix server side copies greater than 4GB
...
After the multipart chunker refactor the multipart chunked server side
copy was accidentally sending one part too many. The last part was 0
length which was rejected by b2.
This was caused by a simple off by one error in the refactoring where
we went from 1 based part number counting to 0 based part number
counting.
Fixing this revealed that the metadata wasn't being re-read for the
copied object either.
This fixes both of those issues and adds an integration tests so it
won't happen again.
See: https://forum.rclone.org/t/large-server-side-copy-in-b2-fails-due-to-bad-byte-range/42294
2023-10-12 20:08:24 +01:00
Nick Craig-Wood
3bb82b4dd5
cmd: Make --progress output logs in the same format as without
...
See: https://forum.rclone.org/t/using-progress-change-dates-from-2023-10-05-to-2023-10-05/42173
2023-10-11 12:13:55 +01:00
Nick Craig-Wood
1591592936
operations: fix error message on delete to have file name - fixes #7355
2023-10-11 12:13:55 +01:00
Vitor Gomes
cc036884d4
operations: fix OpenOptions ignored in copy if operation was a multiThreadCopy
2023-10-11 12:13:55 +01:00
Nick Craig-Wood
f6b9fdf7c6
build: fix docker beta build running out of space
...
This removes some unused SDKs from the build machine to free some
space up before building. It also adds some lines to show the free
space.
2023-10-11 12:13:55 +01:00
Nick Craig-Wood
c9fe2f75a8
oracleobjectstorage: fix OpenOptions being ignored in uploadMultipart with chunkWriter
2023-10-11 12:13:55 +01:00
Vitor Gomes
340a67c012
s3: fix OpenOptions being ignored in uploadMultipart with chunkWriter
2023-10-11 12:13:55 +01:00
Saleh Dindar
264b3f0c90
vfs: [bugfix] Update dir modification time
...
A subtle bug where dir modification time is not updated when the dir already exists
in the cache. It is only noticeable when some clients use dir modification time to
invalidate cache.
2023-10-11 12:13:55 +01:00
Nick Craig-Wood
a7978cea56
operations: close file in TestUploadFile test so it can be deleted on Windows
2023-10-11 12:13:55 +01:00
Nick Craig-Wood
bebd82c586
b2: reduce default --b2-upload-concurrency to 4 to reduce memory usage
...
In v1.63 memory usage in the b2 backend was limited to `--transfers` *
`--b2-chunk-size`
However in v1.64 this was raised to `--transfers` * `--b2-chunk-size`
* `--b2-upload-concurrency`.
The default value for this was accidently set quite high at 16 which
means by default rclone could use up to 6.4GB of memory!
The new default sets a more reasonable (but still high) max memory of 1.6GB.
2023-10-11 12:13:55 +01:00
Nick Craig-Wood
af02c3b2a7
b2: fix locking window when getting mutipart upload URL
...
Before this change, the lock was held while the upload URL was being
fetched from the server.
This meant that any other threads were blocked from getting upload
URLs unecessarily.
It also increased the potential for deadlock.
2023-10-11 12:13:55 +01:00
Nick Craig-Wood
77dfe5f1fd
pacer: fix b2 deadlock by defaulting max connections to unlimited
...
Before this change, the maximum number of connections was set to 10.
This means that b2 could deadlock while uploading multipart uploads
due to a lock being held longer than it should have been.
2023-10-11 12:13:55 +01:00
Nick Craig-Wood
e9a95a78de
s3: fix slice bounds out of range error when listing
...
In this commit:
5f938fb9edhttps://forum.rclone.org/t/cryptcheck-panic-runtime-error-slice-bounds-out-of-range/41977
2023-10-11 12:13:55 +01:00
Nick Craig-Wood
82ca5295f4
docs: fix backend doc generator to not output duplicate config names
...
This was always the intention, it was just implemented wrong.
This shortens the s3 docs by 1369 bringing them down to half the size
just about.
Fixes #7325
2023-10-11 12:13:55 +01:00
Dimitri Papadopoulos Orfanos
9d8a40b813
docs: fix typos found by codespell in docs and code comments
2023-10-11 12:13:55 +01:00
Nick Craig-Wood
12d80c5219
onedrive: fix the configurator to allow /teams/ID in the config
...
See: https://forum.rclone.org/t/sharepoint-to-google/41548/
2023-10-11 12:08:40 +01:00
Nick Craig-Wood
038a87c569
lsjson: make sure we set the global metadata flag too
2023-10-11 12:08:40 +01:00
Nick Craig-Wood
3ef97993ad
b2: fix multipart upload: corrupted on transfer: sizes differ XXX vs 0
...
Before this change the b2 backend wasn't writing the metadata to the
object properly after a multipart upload.
The symptom of this was that sometimes it would give the error:
corrupted on transfer: sizes differ XXX vs 0
This was fixed by returning the metadata in the chunk writer and setting it in Update.
See: https://forum.rclone.org/t/multipart-upload-to-b2-sometimes-failing-with-corrupted-on-transfer-sizes-differ/41829
2023-10-11 12:08:40 +01:00
Nick Craig-Wood
04bba67cd5
azureblob: fix "fatal error: concurrent map writes"
...
Before this change, the metadata map could be accessed from multiple
goroutines at once, sometimes causing this error.
This fix adds a global mutex for adjusting the metadata map to make
all accesses safe.
See: https://forum.rclone.org/t/azure-blob-storage-with-vfs-cache-concurrent-map-writes-exception/41686
2023-10-11 12:08:40 +01:00
dependabot[bot]
29dd29b9f3
build(deps): bump docker/setup-qemu-action from 2 to 3
...
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action ) from 2 to 3.
- [Release notes](https://github.com/docker/setup-qemu-action/releases )
- [Commits](https://github.com/docker/setup-qemu-action/compare/v2...v3 )
---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-11 12:08:40 +01:00
dependabot[bot]
532248352b
build(deps): bump docker/setup-buildx-action from 2 to 3
...
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action ) from 2 to 3.
- [Release notes](https://github.com/docker/setup-buildx-action/releases )
- [Commits](https://github.com/docker/setup-buildx-action/compare/v2...v3 )
---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-11 12:08:40 +01:00
Kaloyan Raev
ab803942de
storj: update storj.io/uplink to v1.12.0
...
The improved upload logic is active by default in uplink v1.12.0, so the
`testuplink.WithConcurrentSegmentUploadsDefaultConfig(ctx)` is not
required anymore.
See https://github.com/rclone/rclone/pull/7198
2023-10-11 12:08:40 +01:00
Nick Craig-Wood
f933e80258
docs: add notes on how to update the website between releases
2023-10-11 12:08:40 +01:00
Nick Craig-Wood
1c6f0101a5
docs: remove minio sponsor box for the moment
2023-10-11 12:08:40 +01:00
Nick Craig-Wood
c6f161de90
docs: update Storj partner link
2023-10-11 12:08:40 +01:00
Herby Gillot
bdcf7fe28c
docs: add MacPorts install info
...
https://ports.macports.org/port/rclone/
2023-10-11 12:08:40 +01:00
dependabot[bot]
776dc47eb8
build(deps): bump docker/metadata-action from 4 to 5
...
Bumps [docker/metadata-action](https://github.com/docker/metadata-action ) from 4 to 5.
- [Release notes](https://github.com/docker/metadata-action/releases )
- [Upgrade guide](https://github.com/docker/metadata-action/blob/master/UPGRADE.md )
- [Commits](https://github.com/docker/metadata-action/compare/v4...v5 )
---
updated-dependencies:
- dependency-name: docker/metadata-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-11 12:08:40 +01:00
dependabot[bot]
167046e21a
build(deps): bump docker/login-action from 2 to 3
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from 2 to 3.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](https://github.com/docker/login-action/compare/v2...v3 )
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-11 12:08:40 +01:00
dependabot[bot]
98d50d545a
build(deps): bump docker/build-push-action from 4 to 5
...
Bumps [docker/build-push-action](https://github.com/docker/build-push-action ) from 4 to 5.
- [Release notes](https://github.com/docker/build-push-action/releases )
- [Commits](https://github.com/docker/build-push-action/compare/v4...v5 )
---
updated-dependencies:
- dependency-name: docker/build-push-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-11 12:08:40 +01:00
Manoj Ghosh
48242c5357
fix overview of oracle object storage as it supports multithreaded
2023-10-11 12:08:40 +01:00
Pat Patterson
e437e6c209
operations: ensure concurrency is no greater than the number of chunks - fixes #7299
2023-10-11 12:08:40 +01:00
Nick Craig-Wood
b813a01718
Start v1.64.1-DEV development
2023-10-11 11:55:24 +01:00
Nick Craig-Wood
77f7bb08af
Version v1.64.0
2023-09-11 15:59:44 +01:00
Nick Craig-Wood
a5a61f4874
protondrive: make cached keys rclone style and not show with rclone config redacted
2023-09-11 15:57:08 +01:00
Nick Craig-Wood
e8879f3e77
docs: document release signing and verification
...
Fixes #7257
2023-09-11 12:28:23 +01:00
Nick Craig-Wood
2a6675cffd
docs: fix typo in rc docs - fixes #7287
2023-09-11 09:34:22 +01:00
Nick Craig-Wood
fa4d171f62
protondrive: complete docs with all references to Proton Drive
2023-09-10 11:57:39 +01:00
Nick Craig-Wood
d4d530bd8e
drive: add --drive-fast-list-bug-fix to control ListR bug workaround
...
See: https://forum.rclone.org/t/how-to-list-empty-directories-recursively/40995/12
2023-09-09 17:46:03 +01:00
Nick Craig-Wood
f4b011e4e4
s3: add rclone backend restore-status command
...
This command shows the restore status of objects being retrieved from GLACIER.
See: https://forum.rclone.org/t/aws-s3-glacier-monitor-restore-status-command-for-glacier-restoring-process/41373/7
2023-09-09 17:44:36 +01:00
Nick Craig-Wood
d80890bf32
Add Drew Stinnett to contributors
2023-09-09 17:44:36 +01:00
Nick Craig-Wood
39392d70dd
Add David Pedersen to contributors
2023-09-09 17:44:36 +01:00
Drew Stinnett
643386f026
rc: Add operations/settier to API
2023-09-09 17:41:02 +01:00
Chun-Hung Tseng
ed755bf04f
protondrive: implement two-password mode ( #7279 )
2023-09-08 22:54:46 +02:00
David Pedersen
071c3f28e5
vfs: Update parent directory modtimes on vfs actions
...
This isn't written back to the storage so might change on remount but
makes the VFS just a little more POSIX compatible.
2023-09-08 17:19:52 +01:00
Nick Craig-Wood
7453b7d5f3
hdfs: fix retry "replication in progress" errors when uploading
...
Before this change uploaded files could return the error "replication
in progress".
This error is harmless though and means the Close should be retried
which is what this patch does.
2023-09-08 15:35:50 +01:00
Nick Craig-Wood
c9350149d8
hdfs: fix uploading to the wrong object on Update with overriden remote name
...
In this commit we discovered a problem with objects being uploaded to
the incorrect object name. It added an integration test for the
problem.
65b2e378e0
2023-09-08 15:35:50 +01:00
Nick Craig-Wood
08789a5815
test_all: remove filefabric from integration tests
...
The filefabric test server doesn't seem to be working at all so remove
it from the integration tests pending revitalization.
2023-09-08 15:35:50 +01:00
Nick Craig-Wood
4037af9c1a
Add Oksana and Volodymyr Kit to contributors
2023-09-08 15:35:50 +01:00
Oksana
628ff8e524
quatrix: add backend to support Quatrix
...
Co-authored-by: Volodymyr Kit <v.kit@maytech.net>
2023-09-08 14:31:29 +01:00
