This removes
- TestCompressSwift: - never finishes - too slow - we have TestCompressS3 instead
- TestCryptSwift: - never finishes - too slow - we have TestCryptS3 instead
- TestChunkerChunk50bBox: - often times out - covered by other tests
This adds a _connect_delay=5s which allows the server to startup
properly. It also makes sure it stores its config in /tmp rather than
the current working directory.
This ignores:
- cmd/bisync where it always fails
- cmd/gitannex where it always fails
- sharefile - citrix have refused to give us a testing account
- duplicated sia backend
- iclouddrive - token expiring every 30 days makes it too difficult
It would be nice to fix up these things at some point, but for the
integration test results to be useful they need less noise in them.
Pure Storage FlashBlade is an enterprise object storage platform that
provides S3-compatible APIs. This change adds FlashBlade as a new
provider option in the S3 backend.
Before this change, FlashBlade users had to use the "Other" provider
with manual configuration of various compatibility flags. This often
resulted in suboptimal performance due to conservative default settings.
After this change, users can select the "FlashBlade" S3 provider and
get an optimal configuration:
- ListObjectsV2 enabled for better performance
- AWS-compatible multipart ETags for reliable transfers
- Proper handling of "AlreadyOwnedByYou" bucket creation responses
- Path-style URLs by default (virtual-host style with DNS setup)
- Unsigned payloads to ensure compatibility with all rclone features
FlashBlade supports modern S3 features including trailer checksum
algorithms (SHA256, CRC32, CRC32C), object versioning, and lifecycle
management.
Provider settings were verified by testing against a FlashBlade//E
system running Purity//FB 4.5.7.
Documentation and test configurations are included.
Integration test results:
```
go test -v -fast-list -remote TestS3FlashBlade:
PASS
ok github.com/rclone/rclone/backend/s3 232.444s
```
This commit modernizes Go usage. This was done with:
go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...
Then files needed to be `go fmt`ed and a few comments needed to be
restored.
The modernizations include replacing
- if/else conditional assignment by a call to the built-in min or max functions added in go1.21
- sort.Slice(x, func(i, j int) bool) { return s[i] < s[j] } by a call to slices.Sort(s), added in go1.21
- interface{} by the 'any' type added in go1.18
- append([]T(nil), s...) by slices.Clone(s) or slices.Concat(s), added in go1.21
- loop around an m[k]=v map update by a call to one of the Collect, Copy, Clone, or Insert functions from the maps package, added in go1.21
- []byte(fmt.Sprintf...) by fmt.Appendf(nil, ...), added in go1.19
- append(s[:i], s[i+1]...) by slices.Delete(s, i, i+1), added in go1.21
- a 3-clause for i := 0; i < n; i++ {} loop by for i := range n {}, added in go1.22
This commit registers gitannex's unit tests with the integration tester
by updating the config.yaml file.
Since we have not yet updated the e2e tests to use the fstest framework,
this commit also adds a case to the e2e tests' skipE2eTestIfNecessary()
function.
Issue #7984
The Mailru backend integration tests have been failing due to new rate
limits on the backend.
This patch
- Removes Mailru from the chunker tests
- Adds the flag so we only run one Mailru test at once
This gives the error
> Update second step failed: Linkbox error 500: The file name needs to include a suffix, such as xxx.mp4
As linkbox can't have files starting with "." and we are trying to save a file called ".ignore".
This changes log statements from log to fs package, which is required for --use-json-log
to properly make log output in JSON format. The recently added custom linting rule,
handled by ruleguard via gocritic via golangci-lint, warns about these and suggests
the alternative. Fixing was therefore basically running "golangci-lint run --fix",
although some manual fixup of mainly imports are necessary following that.
Sometimes (particularly on macOS amd64) the serve s3 test fails with
TestIntegration/FsMkdir/FsPutError where it wasn't expecting to get an
object but it did.
This is likely caused by a race between the serve s3 goroutine
deleting the half uploaded file and the fstests code looking for it to
not exist.
This fix treats it like any other eventual consistency problem and
retries the check using the test framework.
After merging this commit
56caab2033 b2: Include custom upload headers in large file info
The compile failed as a change had been missed. Should have rebased
before merging!
QingStor support have disabled the integration test account with this message
尊敬的用户您好：依据监管部门相关内容安全合规要求，QingStor即日起限制对
个人客户提供对象存储服务，您的对象存储服务将被系统置于禁用状态，如需继
续使用QingsStor对象存储服务，您可以通过工单或者拨打400热线申请开通，未
解封期间您的数据将不受影响，感谢您的谅解和支持。
Which google translate renders as
> Dear user: In accordance with the relevant content security
> compliance requirements of the regulatory authorities, QingStor will
> limit the provision of object storage services to individual
> customers from now on. Your object storage service will be disabled
> by the system. If you need to continue to use the QingsStor object
> storage service, you can apply for activation through a work order
> or by calling the 400 hotline. Your data will not be affected during
> the period of unblocking. Thank you for your understanding and
> support.
This commit removes the test for PublicLink as it is not currently supported in the test environment.
This removes it from the integration tests to avoid meaningless retries.
