Nick Craig-Wood
8aef1de695
s3: fix Cloudflare R2 integration tests after SDKv2 update #4989
...
Cloudflare will normally automatically decompress files with
`Content-Encoding: gzip` when downloaded. This is not what AWS S3 does
and it breaks the integration tests.
This fudges the integration tests to upload the test file with
`Cache-Control: no-transform` on Cloudflare R2 and puts a note in the
docs about this problem.
2024-08-07 12:13:13 +01:00
Nick Craig-Wood
fd9049c83d
s3: update to using AWS SDK v2 - fixes #4989
...
SDK v2 conversion
Changes
- `--s3-sts-endpoint` is no longer supported
- `--s3-use-unsigned-payload` to control use of trailer checksums (needed for non AWS)
2024-08-07 12:13:13 +01:00
Nick Craig-Wood
9866d1c636
docs: s3: add section on using too much memory #7974
2024-08-01 16:33:09 +01:00
Thearas
1c9da8c96a
docs: recommend no_check_bucket = true for Alibaba - fixes #7889
...
Change-Id: Ib6246e416ce67dddc3cb69350de69129a8826ce3
2024-06-15 17:39:05 +01:00
Nick Craig-Wood
93e8a976ef
Version v1.67.0
2024-06-14 16:04:51 +01:00
Bruno Fernandes
4568feb5f9
s3: Add Magalu S3 Object Storage as provider
2024-06-06 11:25:45 +01:00
Pieter van Oostrum
1e3ab7acfd
docs: fix MANUAL formatting problems
...
1) Missing closing code backticks (```) in s3.md causes formatting problems
2) Pandoc requires blank lines before ATX headings
2024-04-05 10:41:47 +02:00
Erisa A
08bf5228a7
docs: Add R2 note about no_check_bucket
2024-04-02 12:08:56 +01:00
Alexandre Lavigne
f9429de807
s3: update Scaleway's configuration options - fixes #7507
...
In order to handle special character, the configuration must specify
rclone configuration to use `list_url_encode`.
2024-03-31 17:42:20 +01:00
Nick Craig-Wood
b6013a5e68
Version v1.66.0
2024-03-10 11:22:43 +00:00
Nick Craig-Wood
0e746f25a3
amazonclouddrive: remove Amazon Drive backend code and docs #7539
...
The Amazon Drive backend is closed from 2023-12-31.
See: https://www.amazon.com/b?ie=UTF8&node=23943055011
2024-01-04 17:05:54 +00:00
Eli Orzitzer
c3117d9efb
Doc change: Add the CreateBucket permission requirement for AWS S3
2023-12-07 16:46:04 +00:00
Nick Craig-Wood
82b963e372
Version v1.65.0
2023-11-26 16:07:39 +00:00
Nick Craig-Wood
ba11040d6b
s3: detect looping when using gcs and versions
...
Apparently gcs doesn't return an S3 compatible result when using
versions.
In particular it doesn't return a NextKeyMarker - this means rclone
loops and fetches the same page over and over again.
This patch detects the problem and stops the infinite retries but it
doesn't fix the underlying problem.
See: https://forum.rclone.org/t/list-s3-versions-files-looping-bug/42974
See: https://issuetracker.google.com/u/0/issues/312292516
2023-11-23 09:50:28 +00:00
albertony
a7faf05393
docs: cleanup backend hashes sections
2023-11-20 17:43:57 +00:00
Nick Craig-Wood
93f35c915a
serve s3: pre-merge tweaks
...
- Changes
- Rename `--s3-authkey` to `--auth-key` to get it out of the s3 backend namespace
- Enable `Content-MD5` integrity checks
- Remove locking after code audit
- Documentation
- Factor out documentation into seperate file
- Add Quickstart to docs
- Add Bugs section to docs
- Add experimental tag to docs
- Add rclone provider to s3 backend docs
- Fixes
- Correct quirks in s3 backend
- Change fmt.Printlns into fs.Logs
- Make metadata storage per backend not global
- Log on startup if anonymous access is enabled
- Coding style fixes
- rename fs to vfs to save confusion with the rest of rclone code
- rename db to b for *s3Backend
Fixes #7062
2023-11-16 16:59:56 +00:00
Nick Craig-Wood
eb3082a1eb
s3: add Linode provider
2023-09-23 14:34:00 +01:00
Nick Craig-Wood
77f7bb08af
Version v1.64.0
2023-09-11 15:59:44 +01:00
kapitainsky
363da9aa82
docs: s3 versions names caveat
2023-07-27 12:36:50 +09:00
Benjamin
33fec9c835
doc: Fix Leviia block
2023-07-18 19:58:19 +01:00
Benjamin
119ccb2b95
s3: add Leviia S3 Object Storage as provider
2023-07-16 18:08:47 +01:00
BakaWang
f1a8420814
s3: add synology to s3 provider list
2023-07-06 10:54:07 +01:00
Nick Craig-Wood
e90537b2e9
Version v1.63.0
2023-06-30 14:11:17 +01:00
danielkrajnik
8274712c2c
docs: s3: fix example for restoring single objects
...
See: https://forum.rclone.org/t/cant-restore-files-from-aws-glacier-deep-only-directories/39258/3
2023-06-26 11:41:15 +01:00
kapitainsky
666e34cf69
s3: docs: old broken link updated
2023-06-09 18:15:54 +01:00
Andrei Smirnov
f226f2dfb1
s3: add petabox.io to s3 providers
2023-05-05 09:44:25 +01:00
dlitster
0487e465ee
docs: s3: clarify that X-Amz-Meta-Md5chksum is really a base64-encoded hex
2023-04-25 11:39:36 +01:00
Anthony Pessy
54a9488e59
s3: add GCS to provider list
2023-03-16 14:24:21 +00:00
Nick Craig-Wood
77c7077458
Version v1.62.0
2023-03-14 12:42:23 +00:00
Nick Craig-Wood
ffd4ab222c
docs: add idrive e2 as a major sponsor
2023-03-14 12:37:34 +00:00
albertony
b9d9f9edb0
docs: use --interactive instead of -i in examples to avoid confusion
2023-01-24 20:43:51 +01:00
Nick Craig-Wood
2001cc0831
Version v1.61.0
2022-12-20 17:16:14 +00:00
Jack
c6ad15e3b8
s3: make DigitalOcean name canonical
2022-12-14 16:35:05 +00:00
MohammadReza
0a8b1fe5de
s3: add Liara LOS to provider list
2022-12-06 12:25:23 +00:00
Nick Craig-Wood
959cd938bc
docs: Add minimum versions to all the backend pages and some of the other pages
2022-11-18 14:41:24 +00:00
Nick Craig-Wood
01dbbff62e
Version v1.60.0
2022-10-21 15:06:08 +01:00
Bachue Zhou
66ed0ca726
s3: add Qiniu KODO to s3 provider list - fixes #6195
2022-10-13 15:49:22 +01:00
Dmitry Deniskin
c080b39e47
s3: add support for IONOS Cloud Storage
2022-09-15 16:04:34 +01:00
Josh Soref
ce3b65e6dc
all: fix spelling across the project
...
* abcdefghijklmnopqrstuvwxyz
* accounting
* additional
* allowed
* almost
* already
* appropriately
* arise
* bandwidth
* behave
* bidirectional
* brackets
* cached
* characters
* cloud
* committing
* concatenating
* configured
* constructs
* current
* cutoff
* deferred
* different
* directory
* disposition
* dropbox
* either way
* error
* excess
* experiments
* explicitly
* externally
* files
* github
* gzipped
* hierarchies
* huffman
* hyphen
* implicitly
* independent
* insensitive
* integrity
* libraries
* literally
* metadata
* mimics
* missing
* modification
* multipart
* multiple
* nightmare
* nonexistent
* number
* obscure
* ourselves
* overridden
* potatoes
* preexisting
* priority
* received
* remote
* replacement
* represents
* reproducibility
* response
* satisfies
* sensitive
* separately
* separator
* specifying
* string
* successful
* synchronization
* syncing
* šenfeld
* take
* temporarily
* testcontents
* that
* the
* themselves
* throttling
* timeout
* transaction
* transferred
* unnecessary
* using
* webbrowser
* which
* with
* workspace
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-08-30 11:16:26 +02:00
Nick Craig-Wood
4344a3e2ea
s3: implement --s3-version-at flag - Fixes #1776
2022-08-05 16:45:23 +01:00
Nick Craig-Wood
81d242473a
s3: implement Purge to purge versions and backend cleanup-hidden
2022-08-05 16:45:23 +01:00
Nick Craig-Wood
0ae171416f
s3: implement --s3-versions flag - See #1776
2022-08-05 16:45:23 +01:00
Nick Craig-Wood
7243918069
s3: implement backend versioning command to get/set bucket versioning
2022-08-05 16:42:30 +01:00
Nick Craig-Wood
00a684d877
Version v1.59.0
2022-07-09 18:09:25 +01:00
Nick Craig-Wood
22abd785eb
s3: implement reading and writing of metadata #111
2022-06-29 14:29:36 +01:00
vyloy
326c43ab3f
s3: add IDrive e2 to provider list
2022-06-28 09:12:36 +01:00
albertony
dbf1234edf
docs: skip "Connection" suffix from FTP, SSH/SFTP and HTTP backend names
2022-06-21 23:43:00 +02:00
m00594701
02b4638a22
backend: add Huawei OBS to s3 provider list
2022-06-14 09:21:01 +01:00
Nick Craig-Wood
813a5e0931
s3: Remove bucket ACL configuration for Cloudflare R2
...
Bucket ACLs are not supported by Cloudflare R2. All buckets are
private and must be shared using a Cloudflare Worker.
2022-05-17 15:57:09 +01:00
Kaspian
542c1616b8
docs: small style fixes
2022-05-13 23:06:45 +01:00
