backend/compress: add zstd compression

Added support for reading and writing zstd-compressed archives in seekable format using "github.com/klauspost/compress/zstd" and "github.com/SaveTheRbtz/zstd-seekable-format-go/pkg". Bumped Go version from 1.24.0 to 1.24.4 due to requirements of "github.com/SaveTheRbtz/zstd-seekable-format-go/pkg".
2025-12-06 00:03:32 +00:00 · 2025-11-04 21:50:56 +07:00
parent ee92673e1b
commit 8f74e7d331
11 changed files with 1093 additions and 228 deletions
--- a/docs/content/compress.md
+++ b/docs/content/compress.md
@@ -23,6 +23,7 @@ To use this remote, all you need to do is specify another remote and a
 compression mode to use:

 ```text
+$ rclone config
 Current remotes:

 Name                 Type
@@ -30,7 +31,6 @@ Name                 Type
 remote_to_press      sometype

 e) Edit existing remote
-$ rclone config
 n) New remote
 d) Delete remote
 r) Rename remote
@@ -39,45 +39,74 @@ s) Set configuration password
 q) Quit config
 e/n/d/r/c/s/q> n
 name> compress
+
+Option Storage.
+Type of storage to configure.
+Choose a number from below, or type in your own value.
 ...
- 8 / Compress a remote
-   \ "compress"
+12 / Compress a remote
+   \ (compress)
 ...
 Storage> compress
-** See help for compress backend at: https://rclone.org/compress/ **

+Option remote.
 Remote to compress.
-Enter a string value. Press Enter for the default ("").
+Enter a value.
 remote> remote_to_press:subdir 
+
+Option mode.
 Compression mode.
-Enter a string value. Press Enter for the default ("gzip").
-Choose a number from below, or type in your own value
- 1 / Gzip compression balanced for speed and compression strength.
-   \ "gzip"
-compression_mode> gzip
-Edit advanced config? (y/n)
+Choose a number from below, or type in your own value of type string.
+Press Enter for the default (gzip).
+ 1 / Standard gzip compression with fastest parameters.
+   \ (gzip)
+ 2 / Zstandard compression — fast modern algorithm offering adjustable speed-to-compression tradeoffs.
+   \ (zstd)
+mode> gzip
+
+Option level.
+GZIP (levels -2 to 9):
+- -2 — Huffman encoding only. Only use if you know what you're doing.
+- -1 (default) — recommended; equivalent to level 5.
+- 0 — turns off compression.
+- 1–9 — increase compression at the cost of speed. Going past 6 generally offers very little return.
+ 
+ZSTD (levels 0 to 4):
+- 0 — turns off compression entirely.
+- 1 — fastest compression with the lowest ratio.
+- 2 (default) — good balance of speed and compression.
+- 3 — better compression, but uses about 2–3x more CPU than the default.
+- 4 — best possible compression ratio (highest CPU cost).
+ 
+Notes:
+- Choose GZIP for wide compatibility; ZSTD for better speed/ratio tradeoffs.
+- Negative gzip levels: -2 = Huffman-only, -1 = default (≈ level 5).
+Enter a value.
+level> -1
+
+Edit advanced config?
 y) Yes
 n) No (default)
 y/n> n
-Remote config
--------------------
-[compress]
-type = compress
-remote = remote_to_press:subdir
-compression_mode = gzip
--------------------
+
+Configuration complete.
+Options:
+- type: compress
+- remote: remote_to_press:subdir
+- mode: gzip
+- level: -1
+Keep this "compress" remote?
 y) Yes this is OK (default)
 e) Edit this remote
 d) Delete this remote
 y/e/d> y
 ```

-### Compression Modes
+### Compression Algorithms

-Currently only gzip compression is supported. It provides a decent balance
-between speed and size and is well supported by other applications. Compression
-strength can further be configured via an advanced setting where 0 is no
-compression and 9 is strongest compression.
+- **GZIP** – a well-established and widely adopted algorithm that strikes a solid balance between compression speed and ratio. It supports compression levels from -2 to 9, with the default -1 (roughly equivalent to level 5) offering an effective middle ground for most scenarios.
+
+- **Zstandard (zstd)** – a modern, high-performance algorithm that offers precise control over the trade-off between speed and compression efficiency. Compression levels range from 0 (no compression) to 4 (maximum compression).

 ### File types

@@ -124,29 +153,38 @@ Properties:
 - Examples:
    - "gzip"
        - Standard gzip compression with fastest parameters.
-
-### Advanced options
-
-Here are the Advanced options specific to compress (Compress a remote).
+    - "zstd"
+        - Zstandard compression — fast modern algorithm offering adjustable speed-to-compression tradeoffs.

 #### --compress-level

-GZIP compression level (-2 to 9).
-
-Generally -1 (default, equivalent to 5) is recommended.
-Levels 1 to 9 increase compression at the cost of speed. Going past 6 
-generally offers very little return.
-
-Level -2 uses Huffman encoding only. Only use if you know what you
-are doing.
-Level 0 turns off compression.
+GZIP (levels -2 to 9):
+- -2 — Huffman encoding only. Only use if you know what you're doing.
+- -1 (default) — recommended; equivalent to level 5.
+- 0 — turns off compression.
+- 1–9 — increase compression at the cost of speed. Going past 6 generally offers very little return.
+ 
+ZSTD (levels 0 to 4):
+- 0 — turns off compression entirely.
+- 1 — fastest compression with the lowest ratio.
+- 2 (default) — good balance of speed and compression.
+- 3 — better compression, but uses about 2–3x more CPU than the default.
+- 4 — best possible compression ratio (highest CPU cost).
+ 
+Notes:
+- Choose GZIP for wide compatibility; ZSTD for better speed/ratio tradeoffs.
+- Negative gzip levels: -2 = Huffman-only, -1 = default (≈ level 5).

 Properties:

 - Config:      level
 - Env Var:     RCLONE_COMPRESS_LEVEL
- Type:        int
- Default:     -1
+- Type:        string
+- Required:    true
+
+### Advanced options
+
+Here are the Advanced options specific to compress (Compress a remote).

 #### --compress-ram-cache-limit