2017-06-26 01:47:54 +02:00
|
|
|
---
|
|
|
|
title: "QingStor"
|
|
|
|
description: "Rclone docs for QingStor Object Storage"
|
|
|
|
---
|
|
|
|
|
2021-07-20 20:45:41 +02:00
|
|
|
# {{< icon "fas fa-hdd" >}} QingStor
|
2017-06-26 01:47:54 +02:00
|
|
|
|
|
|
|
Paths are specified as `remote:bucket` (or `remote:` for the `lsd`
|
2020-10-13 23:49:58 +02:00
|
|
|
command.) You may put subdirectories in too, e.g. `remote:bucket/path/to/dir`.
|
2017-06-26 01:47:54 +02:00
|
|
|
|
2021-10-14 15:40:18 +02:00
|
|
|
## Configuration
|
|
|
|
|
2017-06-26 01:47:54 +02:00
|
|
|
Here is an example of making an QingStor configuration. First run
|
|
|
|
|
|
|
|
rclone config
|
|
|
|
|
|
|
|
This will guide you through an interactive setup process.
|
|
|
|
|
|
|
|
```
|
2021-11-01 21:34:46 +01:00
|
|
|
No remotes found, make a new one?
|
2017-06-26 01:47:54 +02:00
|
|
|
n) New remote
|
|
|
|
r) Rename remote
|
|
|
|
c) Copy remote
|
|
|
|
s) Set configuration password
|
|
|
|
q) Quit config
|
|
|
|
n/r/c/s/q> n
|
|
|
|
name> remote
|
|
|
|
Type of storage to configure.
|
|
|
|
Choose a number from below, or type in your own value
|
2019-08-26 11:47:17 +02:00
|
|
|
[snip]
|
|
|
|
XX / QingStor Object Storage
|
2017-06-26 01:47:54 +02:00
|
|
|
\ "qingstor"
|
2019-08-26 11:47:17 +02:00
|
|
|
[snip]
|
|
|
|
Storage> qingstor
|
2017-06-26 01:47:54 +02:00
|
|
|
Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank.
|
|
|
|
Choose a number from below, or type in your own value
|
|
|
|
1 / Enter QingStor credentials in the next step
|
|
|
|
\ "false"
|
|
|
|
2 / Get QingStor credentials from the environment (env vars or IAM)
|
|
|
|
\ "true"
|
|
|
|
env_auth> 1
|
|
|
|
QingStor Access Key ID - leave blank for anonymous access or runtime credentials.
|
|
|
|
access_key_id> access_key
|
|
|
|
QingStor Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
|
|
|
|
secret_access_key> secret_key
|
2020-05-19 13:02:44 +02:00
|
|
|
Enter an endpoint URL to connection QingStor API.
|
2017-06-26 01:47:54 +02:00
|
|
|
Leave blank will use the default value "https://qingstor.com:443"
|
|
|
|
endpoint>
|
|
|
|
Zone connect to. Default is "pek3a".
|
|
|
|
Choose a number from below, or type in your own value
|
|
|
|
/ The Beijing (China) Three Zone
|
|
|
|
1 | Needs location constraint pek3a.
|
|
|
|
\ "pek3a"
|
|
|
|
/ The Shanghai (China) First Zone
|
|
|
|
2 | Needs location constraint sh1a.
|
|
|
|
\ "sh1a"
|
|
|
|
zone> 1
|
2020-02-10 16:14:35 +01:00
|
|
|
Number of connection retry.
|
2017-06-26 01:47:54 +02:00
|
|
|
Leave blank will use the default value "3".
|
|
|
|
connection_retries>
|
|
|
|
Remote config
|
|
|
|
--------------------
|
|
|
|
[remote]
|
|
|
|
env_auth = false
|
|
|
|
access_key_id = access_key
|
|
|
|
secret_access_key = secret_key
|
|
|
|
endpoint =
|
|
|
|
zone = pek3a
|
|
|
|
connection_retries =
|
|
|
|
--------------------
|
|
|
|
y) Yes this is OK
|
|
|
|
e) Edit this remote
|
|
|
|
d) Delete this remote
|
|
|
|
y/e/d> y
|
|
|
|
```
|
|
|
|
|
|
|
|
This remote is called `remote` and can now be used like this
|
|
|
|
|
|
|
|
See all buckets
|
|
|
|
|
|
|
|
rclone lsd remote:
|
|
|
|
|
|
|
|
Make a new bucket
|
|
|
|
|
|
|
|
rclone mkdir remote:bucket
|
|
|
|
|
|
|
|
List the contents of a bucket
|
|
|
|
|
|
|
|
rclone ls remote:bucket
|
|
|
|
|
|
|
|
Sync `/home/local/directory` to the remote bucket, deleting any excess
|
|
|
|
files in the bucket.
|
|
|
|
|
2020-06-05 18:04:23 +02:00
|
|
|
rclone sync -i /home/local/directory remote:bucket
|
2017-06-26 01:47:54 +02:00
|
|
|
|
2021-10-14 15:40:18 +02:00
|
|
|
### --fast-list
|
2017-08-03 15:31:55 +02:00
|
|
|
|
|
|
|
This remote supports `--fast-list` which allows you to use fewer
|
|
|
|
transactions in exchange for more memory. See the [rclone
|
|
|
|
docs](/docs/#fast-list) for more details.
|
|
|
|
|
2021-10-14 15:40:18 +02:00
|
|
|
### Multipart uploads
|
2017-06-26 01:47:54 +02:00
|
|
|
|
|
|
|
rclone supports multipart uploads with QingStor which means that it can
|
2021-03-02 20:11:57 +01:00
|
|
|
upload files bigger than 5 GiB. Note that files uploaded with multipart
|
2017-06-26 01:47:54 +02:00
|
|
|
upload don't have an MD5SUM.
|
|
|
|
|
2020-03-18 12:53:25 +01:00
|
|
|
Note that incomplete multipart uploads older than 24 hours can be
|
|
|
|
removed with `rclone cleanup remote:bucket` just for one bucket
|
|
|
|
`rclone cleanup remote:` for all buckets. QingStor does not ever
|
|
|
|
remove incomplete multipart uploads so it may be necessary to run this
|
|
|
|
from time to time.
|
|
|
|
|
2021-10-14 15:40:18 +02:00
|
|
|
### Buckets and Zone
|
2017-06-26 01:47:54 +02:00
|
|
|
|
|
|
|
With QingStor you can list buckets (`rclone lsd`) using any zone,
|
|
|
|
but you can only access the content of a bucket from the zone it was
|
|
|
|
created in. If you attempt to access a bucket from the wrong zone,
|
|
|
|
you will get an error, `incorrect zone, the bucket is not in 'XXX'
|
|
|
|
zone`.
|
|
|
|
|
2021-10-14 15:40:18 +02:00
|
|
|
### Authentication
|
2017-08-03 15:31:55 +02:00
|
|
|
|
2017-06-26 01:47:54 +02:00
|
|
|
There are two ways to supply `rclone` with a set of QingStor
|
|
|
|
credentials. In order of precedence:
|
|
|
|
|
|
|
|
- Directly in the rclone configuration file (as configured by `rclone config`)
|
|
|
|
- set `access_key_id` and `secret_access_key`
|
|
|
|
- Runtime configuration:
|
|
|
|
- set `env_auth` to `true` in the config file
|
|
|
|
- Exporting the following environment variables before running `rclone`
|
|
|
|
- Access Key ID: `QS_ACCESS_KEY_ID` or `QS_ACCESS_KEY`
|
|
|
|
- Secret Access Key: `QS_SECRET_ACCESS_KEY` or `QS_SECRET_KEY`
|
|
|
|
|
2019-09-21 14:24:30 +02:00
|
|
|
### Restricted filename characters
|
|
|
|
|
|
|
|
The control characters 0x00-0x1F and / are replaced as in the [default
|
|
|
|
restricted characters set](/overview/#restricted-characters). Note
|
|
|
|
that 0x7F is not replaced.
|
|
|
|
|
|
|
|
Invalid UTF-8 bytes will also be [replaced](/overview/#invalid-utf8),
|
|
|
|
as they can't be used in JSON strings.
|
|
|
|
|
2020-05-22 13:22:52 +02:00
|
|
|
{{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/qingstor/qingstor.go then run make backenddocs" >}}
|
2021-11-01 16:42:05 +01:00
|
|
|
### Standard options
|
2018-10-01 21:48:54 +02:00
|
|
|
|
|
|
|
Here are the standard options specific to qingstor (QingCloud Object Storage).
|
|
|
|
|
|
|
|
#### --qingstor-env-auth
|
|
|
|
|
2021-11-01 16:42:05 +01:00
|
|
|
Get QingStor credentials from runtime.
|
|
|
|
|
|
|
|
Only applies if access_key_id and secret_access_key is blank.
|
2018-10-01 21:48:54 +02:00
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
Properties:
|
|
|
|
|
2018-10-01 21:48:54 +02:00
|
|
|
- Config: env_auth
|
|
|
|
- Env Var: RCLONE_QINGSTOR_ENV_AUTH
|
|
|
|
- Type: bool
|
|
|
|
- Default: false
|
|
|
|
- Examples:
|
|
|
|
- "false"
|
2021-11-01 16:42:05 +01:00
|
|
|
- Enter QingStor credentials in the next step.
|
2018-10-01 21:48:54 +02:00
|
|
|
- "true"
|
2021-11-01 16:42:05 +01:00
|
|
|
- Get QingStor credentials from the environment (env vars or IAM).
|
2018-10-01 21:48:54 +02:00
|
|
|
|
|
|
|
#### --qingstor-access-key-id
|
|
|
|
|
2021-11-01 16:42:05 +01:00
|
|
|
QingStor Access Key ID.
|
|
|
|
|
2018-10-01 21:48:54 +02:00
|
|
|
Leave blank for anonymous access or runtime credentials.
|
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
Properties:
|
|
|
|
|
2018-10-01 21:48:54 +02:00
|
|
|
- Config: access_key_id
|
|
|
|
- Env Var: RCLONE_QINGSTOR_ACCESS_KEY_ID
|
|
|
|
- Type: string
|
2022-03-18 13:29:54 +01:00
|
|
|
- Required: false
|
2018-10-01 21:48:54 +02:00
|
|
|
|
|
|
|
#### --qingstor-secret-access-key
|
|
|
|
|
2021-11-01 16:42:05 +01:00
|
|
|
QingStor Secret Access Key (password).
|
|
|
|
|
2018-10-01 21:48:54 +02:00
|
|
|
Leave blank for anonymous access or runtime credentials.
|
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
Properties:
|
|
|
|
|
2018-10-01 21:48:54 +02:00
|
|
|
- Config: secret_access_key
|
|
|
|
- Env Var: RCLONE_QINGSTOR_SECRET_ACCESS_KEY
|
|
|
|
- Type: string
|
2022-03-18 13:29:54 +01:00
|
|
|
- Required: false
|
2018-10-01 21:48:54 +02:00
|
|
|
|
|
|
|
#### --qingstor-endpoint
|
|
|
|
|
2020-05-19 13:02:44 +02:00
|
|
|
Enter an endpoint URL to connection QingStor API.
|
2021-11-01 16:42:05 +01:00
|
|
|
|
|
|
|
Leave blank will use the default value "https://qingstor.com:443".
|
2018-10-01 21:48:54 +02:00
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
Properties:
|
|
|
|
|
2018-10-01 21:48:54 +02:00
|
|
|
- Config: endpoint
|
|
|
|
- Env Var: RCLONE_QINGSTOR_ENDPOINT
|
|
|
|
- Type: string
|
2022-03-18 13:29:54 +01:00
|
|
|
- Required: false
|
2018-10-01 21:48:54 +02:00
|
|
|
|
|
|
|
#### --qingstor-zone
|
|
|
|
|
|
|
|
Zone to connect to.
|
2021-11-01 16:42:05 +01:00
|
|
|
|
2018-10-01 21:48:54 +02:00
|
|
|
Default is "pek3a".
|
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
Properties:
|
|
|
|
|
2018-10-01 21:48:54 +02:00
|
|
|
- Config: zone
|
|
|
|
- Env Var: RCLONE_QINGSTOR_ZONE
|
|
|
|
- Type: string
|
2022-03-18 13:29:54 +01:00
|
|
|
- Required: false
|
2018-10-01 21:48:54 +02:00
|
|
|
- Examples:
|
|
|
|
- "pek3a"
|
2021-11-01 16:42:05 +01:00
|
|
|
- The Beijing (China) Three Zone.
|
2018-10-01 21:48:54 +02:00
|
|
|
- Needs location constraint pek3a.
|
|
|
|
- "sh1a"
|
2021-11-01 16:42:05 +01:00
|
|
|
- The Shanghai (China) First Zone.
|
2018-10-01 21:48:54 +02:00
|
|
|
- Needs location constraint sh1a.
|
|
|
|
- "gd2a"
|
2021-11-01 16:42:05 +01:00
|
|
|
- The Guangdong (China) Second Zone.
|
2018-10-01 21:48:54 +02:00
|
|
|
- Needs location constraint gd2a.
|
|
|
|
|
2021-11-01 16:42:05 +01:00
|
|
|
### Advanced options
|
2018-10-01 21:48:54 +02:00
|
|
|
|
|
|
|
Here are the advanced options specific to qingstor (QingCloud Object Storage).
|
|
|
|
|
|
|
|
#### --qingstor-connection-retries
|
|
|
|
|
2018-11-24 14:44:25 +01:00
|
|
|
Number of connection retries.
|
2018-10-01 21:48:54 +02:00
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
Properties:
|
|
|
|
|
2018-10-01 21:48:54 +02:00
|
|
|
- Config: connection_retries
|
|
|
|
- Env Var: RCLONE_QINGSTOR_CONNECTION_RETRIES
|
|
|
|
- Type: int
|
|
|
|
- Default: 3
|
|
|
|
|
2019-01-04 12:24:20 +01:00
|
|
|
#### --qingstor-upload-cutoff
|
|
|
|
|
2021-11-01 16:42:05 +01:00
|
|
|
Cutoff for switching to chunked upload.
|
2019-01-04 12:24:20 +01:00
|
|
|
|
|
|
|
Any files larger than this will be uploaded in chunks of chunk_size.
|
2021-03-02 20:11:57 +01:00
|
|
|
The minimum is 0 and the maximum is 5 GiB.
|
2019-01-04 12:24:20 +01:00
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
Properties:
|
|
|
|
|
2019-01-04 12:24:20 +01:00
|
|
|
- Config: upload_cutoff
|
|
|
|
- Env Var: RCLONE_QINGSTOR_UPLOAD_CUTOFF
|
|
|
|
- Type: SizeSuffix
|
2021-07-20 20:45:41 +02:00
|
|
|
- Default: 200Mi
|
2019-01-04 12:24:20 +01:00
|
|
|
|
|
|
|
#### --qingstor-chunk-size
|
|
|
|
|
|
|
|
Chunk size to use for uploading.
|
|
|
|
|
|
|
|
When uploading files larger than upload_cutoff they will be uploaded
|
|
|
|
as multipart uploads using this chunk size.
|
|
|
|
|
|
|
|
Note that "--qingstor-upload-concurrency" chunks of this size are buffered
|
|
|
|
in memory per transfer.
|
|
|
|
|
2020-10-13 23:50:53 +02:00
|
|
|
If you are transferring large files over high-speed links and you have
|
2019-01-04 12:24:20 +01:00
|
|
|
enough memory, then increasing this will speed up the transfers.
|
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
Properties:
|
|
|
|
|
2019-01-04 12:24:20 +01:00
|
|
|
- Config: chunk_size
|
|
|
|
- Env Var: RCLONE_QINGSTOR_CHUNK_SIZE
|
|
|
|
- Type: SizeSuffix
|
2021-07-20 20:45:41 +02:00
|
|
|
- Default: 4Mi
|
2019-01-04 12:24:20 +01:00
|
|
|
|
|
|
|
#### --qingstor-upload-concurrency
|
|
|
|
|
|
|
|
Concurrency for multipart uploads.
|
|
|
|
|
|
|
|
This is the number of chunks of the same file that are uploaded
|
|
|
|
concurrently.
|
|
|
|
|
Spelling fixes
Fix spelling of: above, already, anonymous, associated,
authentication, bandwidth, because, between, blocks, calculate,
candidates, cautious, changelog, cleaner, clipboard, command,
completely, concurrently, considered, constructs, corrupt, current,
daemon, dependencies, deprecated, directory, dispatcher, download,
eligible, ellipsis, encrypter, endpoint, entrieslist, essentially,
existing writers, existing, expires, filesystem, flushing, frequently,
hierarchy, however, implementation, implements, inaccurate,
individually, insensitive, longer, maximum, metadata, modified,
multipart, namedirfirst, nextcloud, obscured, opened, optional,
owncloud, pacific, passphrase, password, permanently, persimmon,
positive, potato, protocol, quota, receiving, recommends, referring,
requires, revisited, satisfied, satisfies, satisfy, semver,
serialized, session, storage, strategies, stringlist, successful,
supported, surprise, temporarily, temporary, transactions, unneeded,
update, uploads, wrapped
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-09 02:17:24 +02:00
|
|
|
NB if you set this to > 1 then the checksums of multipart uploads
|
2019-02-09 11:42:57 +01:00
|
|
|
become corrupted (the uploads themselves are not corrupted though).
|
|
|
|
|
2020-10-13 23:59:13 +02:00
|
|
|
If you are uploading small numbers of large files over high-speed links
|
2019-01-04 12:24:20 +01:00
|
|
|
and these uploads do not fully utilize your bandwidth, then increasing
|
|
|
|
this may help to speed up the transfers.
|
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
Properties:
|
|
|
|
|
2019-01-04 12:24:20 +01:00
|
|
|
- Config: upload_concurrency
|
|
|
|
- Env Var: RCLONE_QINGSTOR_UPLOAD_CONCURRENCY
|
|
|
|
- Type: int
|
2019-02-09 11:42:57 +01:00
|
|
|
- Default: 1
|
2019-01-04 12:24:20 +01:00
|
|
|
|
2020-02-01 11:31:42 +01:00
|
|
|
#### --qingstor-encoding
|
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
The encoding for the backend.
|
2020-02-01 11:31:42 +01:00
|
|
|
|
2021-11-01 16:42:05 +01:00
|
|
|
See the [encoding section in the overview](/overview/#encoding) for more info.
|
2020-02-01 11:31:42 +01:00
|
|
|
|
2022-03-18 13:29:54 +01:00
|
|
|
Properties:
|
|
|
|
|
2020-02-01 11:31:42 +01:00
|
|
|
- Config: encoding
|
|
|
|
- Env Var: RCLONE_QINGSTOR_ENCODING
|
|
|
|
- Type: MultiEncoder
|
|
|
|
- Default: Slash,Ctl,InvalidUtf8
|
|
|
|
|
2020-05-22 13:22:52 +02:00
|
|
|
{{< rem autogenerated options stop >}}
|
2021-10-14 15:40:18 +02:00
|
|
|
|
|
|
|
## Limitations
|
2020-11-27 15:08:52 +01:00
|
|
|
|
|
|
|
`rclone about` is not supported by the qingstor backend. Backends without
|
|
|
|
this capability cannot determine free space for an rclone mount or
|
|
|
|
use policy `mfs` (most free space) as a member of an rclone union
|
|
|
|
remote.
|
|
|
|
|
|
|
|
See [List of backends that do not support rclone about](https://rclone.org/overview/#optional-features)
|
|
|
|
See [rclone about](https://rclone.org/commands/rclone_about/)
|
|
|
|
|