rclone/docs/content/compress.md

168 lines
4.4 KiB
Markdown
Raw Normal View History

2020-11-11 19:56:16 +01:00
---
title: "Compress"
description: "Compression Remote"
versionIntroduced: "v1.54"
status: Experimental
2020-11-11 19:56:16 +01:00
---
# {{< icon "fas fa-compress" >}} Compress
2020-11-11 19:56:16 +01:00
## Warning
This remote is currently **experimental**. Things may break and data may be lost. Anything you do with this remote is
at your own risk. Please understand the risks associated with using experimental code and don't use this remote in
critical applications.
2020-11-11 19:56:16 +01:00
The `Compress` remote adds compression to another remote. It is best used with remotes containing
many large compressible files.
## Configuration
2020-11-11 19:56:16 +01:00
To use this remote, all you need to do is specify another remote and a compression mode to use:
```
Current remotes:
Name Type
==== ====
remote_to_press sometype
e) Edit existing remote
$ rclone config
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n
name> compress
...
8 / Compress a remote
\ "compress"
...
Storage> compress
** See help for compress backend at: https://rclone.org/compress/ **
Remote to compress.
Enter a string value. Press Enter for the default ("").
remote> remote_to_press:subdir
Compression mode.
Enter a string value. Press Enter for the default ("gzip").
Choose a number from below, or type in your own value
1 / Gzip compression balanced for speed and compression strength.
\ "gzip"
compression_mode> gzip
Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n> n
Remote config
--------------------
[compress]
type = compress
remote = remote_to_press:subdir
compression_mode = gzip
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y
```
### Compression Modes
2021-10-09 18:22:38 +02:00
Currently only gzip compression is supported. It provides a decent balance between speed and size and is well
supported by other applications. Compression strength can further be configured via an advanced setting where 0 is no
2020-11-11 19:56:16 +01:00
compression and 9 is strongest compression.
### File types
2021-10-09 18:22:38 +02:00
If you open a remote wrapped by compress, you will see that there are many files with an extension corresponding to
2020-11-11 19:56:16 +01:00
the compression algorithm you chose. These files are standard files that can be opened by various archive programs,
but they have some hidden metadata that allows them to be used by rclone.
While you may download and decompress these files at will, do **not** manually delete or rename files. Files without
correct metadata files will not be recognized by rclone.
### File names
The compressed files will be named `*.###########.gz` where `*` is the base file and the `#` part is base64 encoded
size of the uncompressed file. The file names should not be changed by anything other than the rclone compression backend.
{{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/compress/compress.go then run make backenddocs" >}}
2021-11-01 16:42:05 +01:00
### Standard options
2020-11-11 19:56:16 +01:00
2022-07-09 19:08:20 +02:00
Here are the Standard options specific to compress (Compress a remote).
2020-11-11 19:56:16 +01:00
#### --compress-remote
Remote to compress.
2022-03-18 13:29:54 +01:00
Properties:
2020-11-11 19:56:16 +01:00
- Config: remote
- Env Var: RCLONE_COMPRESS_REMOTE
- Type: string
2022-03-18 13:29:54 +01:00
- Required: true
2020-11-11 19:56:16 +01:00
#### --compress-mode
Compression mode.
2022-03-18 13:29:54 +01:00
Properties:
2020-11-11 19:56:16 +01:00
- Config: mode
- Env Var: RCLONE_COMPRESS_MODE
- Type: string
- Default: "gzip"
- Examples:
- "gzip"
- Standard gzip compression with fastest parameters.
2021-11-01 16:42:05 +01:00
### Advanced options
2020-11-11 19:56:16 +01:00
2022-07-09 19:08:20 +02:00
Here are the Advanced options specific to compress (Compress a remote).
2020-11-11 19:56:16 +01:00
#### --compress-level
GZIP compression level (-2 to 9).
2021-11-01 16:42:05 +01:00
Generally -1 (default, equivalent to 5) is recommended.
Levels 1 to 9 increase compression at the cost of speed. Going past 6
generally offers very little return.
Level -2 uses Huffman encoding only. Only use if you know what you
2021-11-01 16:42:05 +01:00
are doing.
Level 0 turns off compression.
2020-11-11 19:56:16 +01:00
2022-03-18 13:29:54 +01:00
Properties:
2020-11-11 19:56:16 +01:00
- Config: level
- Env Var: RCLONE_COMPRESS_LEVEL
- Type: int
- Default: -1
2021-02-02 14:42:35 +01:00
#### --compress-ram-cache-limit
Some remotes don't allow the upload of files with unknown size.
2021-11-01 16:42:05 +01:00
In this case the compressed file will need to be cached to determine
it's size.
Files smaller than this limit will be cached in RAM, files larger than
this limit will be cached on disk.
2021-02-02 14:42:35 +01:00
2022-03-18 13:29:54 +01:00
Properties:
2021-02-02 14:42:35 +01:00
- Config: ram_cache_limit
- Env Var: RCLONE_COMPRESS_RAM_CACHE_LIMIT
- Type: SizeSuffix
2021-07-20 20:45:41 +02:00
- Default: 20Mi
2021-02-02 14:42:35 +01:00
2022-07-09 19:08:20 +02:00
### Metadata
Any metadata supported by the underlying remote is read and written.
See the [metadata](/docs/#metadata) docs for more info.
2020-11-11 19:56:16 +01:00
{{< rem autogenerated options stop >}}