mirror of
https://github.com/rclone/rclone.git
synced 2025-12-11 05:43:15 +00:00
0f3aa17fb6
Replace custom rem hugo shortcode template with HTML comment. HTML comments are now allowed in Hugo without enabling unsafe HTML parsing. Improve the text in the comment: Remove unnecessary quoting, and avoid impression that make backenddocs has to be run and results committed, since we have a lint check which will then report error because we want to prevent manual changes in autogenerated sections. Disable the markdownlint rule line-length on the autogenerated marker line. Make the autogenerated marker detection a bit more robust. See #8942 for more details.
223 lines
6.3 KiB
Markdown
223 lines
6.3 KiB
Markdown
---
|
||
title: "Compress"
|
||
description: "Compression Remote"
|
||
versionIntroduced: "v1.54"
|
||
status: Experimental
|
||
---
|
||
|
||
# {{< icon "fas fa-compress" >}} Compress
|
||
|
||
## Warning
|
||
|
||
This remote is currently **experimental**. Things may break and data may be lost.
|
||
Anything you do with this remote is at your own risk. Please understand the risks
|
||
associated with using experimental code and don't use this remote in critical
|
||
applications.
|
||
|
||
The `Compress` remote adds compression to another remote. It is best used with
|
||
remotes containing many large compressible files.
|
||
|
||
## Configuration
|
||
|
||
To use this remote, all you need to do is specify another remote and a
|
||
compression mode to use:
|
||
|
||
```text
|
||
$ rclone config
|
||
Current remotes:
|
||
|
||
Name Type
|
||
==== ====
|
||
remote_to_press sometype
|
||
|
||
e) Edit existing remote
|
||
n) New remote
|
||
d) Delete remote
|
||
r) Rename remote
|
||
c) Copy remote
|
||
s) Set configuration password
|
||
q) Quit config
|
||
e/n/d/r/c/s/q> n
|
||
name> compress
|
||
|
||
Option Storage.
|
||
Type of storage to configure.
|
||
Choose a number from below, or type in your own value.
|
||
...
|
||
12 / Compress a remote
|
||
\ (compress)
|
||
...
|
||
Storage> compress
|
||
|
||
Option remote.
|
||
Remote to compress.
|
||
Enter a value.
|
||
remote> remote_to_press:subdir
|
||
|
||
Option mode.
|
||
Compression mode.
|
||
Choose a number from below, or type in your own value of type string.
|
||
Press Enter for the default (gzip).
|
||
1 / Standard gzip compression with fastest parameters.
|
||
\ (gzip)
|
||
2 / Zstandard compression — fast modern algorithm offering adjustable speed-to-compression tradeoffs.
|
||
\ (zstd)
|
||
mode> gzip
|
||
|
||
Option level.
|
||
GZIP (levels -2 to 9):
|
||
- -2 — Huffman encoding only. Only use if you know what you're doing.
|
||
- -1 (default) — recommended; equivalent to level 5.
|
||
- 0 — turns off compression.
|
||
- 1–9 — increase compression at the cost of speed. Going past 6 generally offers very little return.
|
||
|
||
ZSTD (levels 0 to 4):
|
||
- 0 — turns off compression entirely.
|
||
- 1 — fastest compression with the lowest ratio.
|
||
- 2 (default) — good balance of speed and compression.
|
||
- 3 — better compression, but uses about 2–3x more CPU than the default.
|
||
- 4 — best possible compression ratio (highest CPU cost).
|
||
|
||
Notes:
|
||
- Choose GZIP for wide compatibility; ZSTD for better speed/ratio tradeoffs.
|
||
- Negative gzip levels: -2 = Huffman-only, -1 = default (≈ level 5).
|
||
Enter a value.
|
||
level> -1
|
||
|
||
Edit advanced config?
|
||
y) Yes
|
||
n) No (default)
|
||
y/n> n
|
||
|
||
Configuration complete.
|
||
Options:
|
||
- type: compress
|
||
- remote: remote_to_press:subdir
|
||
- mode: gzip
|
||
- level: -1
|
||
Keep this "compress" remote?
|
||
y) Yes this is OK (default)
|
||
e) Edit this remote
|
||
d) Delete this remote
|
||
y/e/d> y
|
||
```
|
||
|
||
### Compression Algorithms
|
||
|
||
- **GZIP** – a well-established and widely adopted algorithm that strikes a solid balance between compression speed and ratio. It supports compression levels from -2 to 9, with the default -1 (roughly equivalent to level 5) offering an effective middle ground for most scenarios.
|
||
|
||
- **Zstandard (zstd)** – a modern, high-performance algorithm that offers precise control over the trade-off between speed and compression efficiency. Compression levels range from 0 (no compression) to 4 (maximum compression).
|
||
|
||
### File types
|
||
|
||
If you open a remote wrapped by compress, you will see that there are many
|
||
files with an extension corresponding to the compression algorithm you chose.
|
||
These files are standard files that can be opened by various archive programs,
|
||
but they have some hidden metadata that allows them to be used by rclone.
|
||
While you may download and decompress these files at will, do **not** manually
|
||
delete or rename files. Files without correct metadata files will not be
|
||
recognized by rclone.
|
||
|
||
### File names
|
||
|
||
The compressed files will be named `*.###########.gz` where `*` is the base
|
||
file and the `#` part is base64 encoded size of the uncompressed file. The file
|
||
names should not be changed by anything other than the rclone compression backend.
|
||
|
||
<!-- autogenerated options start - DO NOT EDIT - instead edit fs.RegInfo in backend/compress/compress.go and run make backenddocs to verify --> <!-- markdownlint-disable-line line-length -->
|
||
### Standard options
|
||
|
||
Here are the Standard options specific to compress (Compress a remote).
|
||
|
||
#### --compress-remote
|
||
|
||
Remote to compress.
|
||
|
||
Properties:
|
||
|
||
- Config: remote
|
||
- Env Var: RCLONE_COMPRESS_REMOTE
|
||
- Type: string
|
||
- Required: true
|
||
|
||
#### --compress-mode
|
||
|
||
Compression mode.
|
||
|
||
Properties:
|
||
|
||
- Config: mode
|
||
- Env Var: RCLONE_COMPRESS_MODE
|
||
- Type: string
|
||
- Default: "gzip"
|
||
- Examples:
|
||
- "gzip"
|
||
- Standard gzip compression with fastest parameters.
|
||
- "zstd"
|
||
- Zstandard compression — fast modern algorithm offering adjustable speed-to-compression tradeoffs.
|
||
|
||
#### --compress-level
|
||
|
||
GZIP (levels -2 to 9):
|
||
- -2 — Huffman encoding only. Only use if you know what you're doing.
|
||
- -1 (default) — recommended; equivalent to level 5.
|
||
- 0 — turns off compression.
|
||
- 1–9 — increase compression at the cost of speed. Going past 6 generally offers very little return.
|
||
|
||
ZSTD (levels 0 to 4):
|
||
- 0 — turns off compression entirely.
|
||
- 1 — fastest compression with the lowest ratio.
|
||
- 2 (default) — good balance of speed and compression.
|
||
- 3 — better compression, but uses about 2–3x more CPU than the default.
|
||
- 4 — best possible compression ratio (highest CPU cost).
|
||
|
||
Notes:
|
||
- Choose GZIP for wide compatibility; ZSTD for better speed/ratio tradeoffs.
|
||
- Negative gzip levels: -2 = Huffman-only, -1 = default (≈ level 5).
|
||
|
||
Properties:
|
||
|
||
- Config: level
|
||
- Env Var: RCLONE_COMPRESS_LEVEL
|
||
- Type: string
|
||
- Required: true
|
||
|
||
### Advanced options
|
||
|
||
Here are the Advanced options specific to compress (Compress a remote).
|
||
|
||
#### --compress-ram-cache-limit
|
||
|
||
Some remotes don't allow the upload of files with unknown size.
|
||
In this case the compressed file will need to be cached to determine
|
||
it's size.
|
||
|
||
Files smaller than this limit will be cached in RAM, files larger than
|
||
this limit will be cached on disk.
|
||
|
||
Properties:
|
||
|
||
- Config: ram_cache_limit
|
||
- Env Var: RCLONE_COMPRESS_RAM_CACHE_LIMIT
|
||
- Type: SizeSuffix
|
||
- Default: 20Mi
|
||
|
||
#### --compress-description
|
||
|
||
Description of the remote.
|
||
|
||
Properties:
|
||
|
||
- Config: description
|
||
- Env Var: RCLONE_COMPRESS_DESCRIPTION
|
||
- Type: string
|
||
- Required: false
|
||
|
||
### Metadata
|
||
|
||
Any metadata supported by the underlying remote is read and written.
|
||
|
||
See the [metadata](/docs/#metadata) docs for more info.
|
||
|
||
<!-- autogenerated options stop -->
|