1
0
mirror of https://github.com/rclone/rclone.git synced 2026-01-05 18:13:17 +00:00

Compare commits

..

48 Commits

Author SHA1 Message Date
Nick Craig-Wood
80727496fc Version v1.69.3 2025-05-21 21:08:44 +01:00
Nick Craig-Wood
a5bd052fb4 build: update github.com/ebitengine/purego to work around bug in go1.24.3
Before this we get this error on darwin/amd64

link: duplicated definition of symbol dlopen, from github.com/ebitengine/purego and github.com/ebitengine/purego

See: https://github.com/golang/go/issues/73617
2025-05-21 16:02:19 +01:00
dependabot[bot]
538edfeee0 build: reapply update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 to fix CVE-2025-30204
This commit got accidentally reverted in

ef975129a8 build: bump golang.org/x/net from 0.36.0 to 0.38.0

Bumps [github.com/golang-jwt/jwt/v5](https://github.com/golang-jwt/jwt) from 5.2.1 to 5.2.2.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v5.2.1...v5.2.2)

See: https://github.com/golang-jwt/jwt/security/advisories/GHSA-mh63-6h87-95cp
See: https://www.cve.org/CVERecord?id=CVE-2025-30204

Signed-off-by: dependabot[bot] <support@github.com>
2025-05-13 11:53:42 +01:00
Nick Craig-Wood
20253fb4df Start v1.69.3-DEV development 2025-05-01 16:41:55 +01:00
Nick Craig-Wood
13875b4500 Version v1.69.2 2025-05-01 15:33:17 +01:00
Nick Craig-Wood
3d3ea9ee30 vfs: fix integration test failures
In this commit

ceef78ce44 vfs: fix directory cache serving stale data

We added a new test which caused lots of integration test failures.

This fixes the problem by disabling the test unless the feature flag
DirModTimeUpdatesOnWrite is present on the remote.
2025-05-01 11:08:58 +01:00
Nick Craig-Wood
84f11ae448 build: stop building with go < go1.23 as security updates forbade it 2025-04-30 16:13:34 +01:00
dependabot[bot]
ef975129a8 build: bump golang.org/x/net from 0.36.0 to 0.38.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.36.0 to 0.38.0.
- [Commits](https://github.com/golang/net/compare/v0.36.0...v0.38.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.38.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-30 16:09:30 +01:00
dependabot[bot]
c9f3456341 build: update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 to fix CVE-2025-30204
Bumps [github.com/golang-jwt/jwt/v5](https://github.com/golang-jwt/jwt) from 5.2.1 to 5.2.2.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v5.2.1...v5.2.2)

See: https://github.com/golang-jwt/jwt/security/advisories/GHSA-mh63-6h87-95cp
See: https://www.cve.org/CVERecord?id=CVE-2025-30204

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-30 16:09:30 +01:00
dependabot[bot]
0121dc11d3 build: bump github.com/golang-jwt/jwt/v4 from 4.5.1 to 4.5.2
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.5.1 to 4.5.2.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v4.5.1...v4.5.2)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-30 16:09:30 +01:00
dependabot[bot]
32ea8f6dc8 build: update golang.org/x/net to 0.36.0. to fix CVE-2025-22869
SSH servers which implement file transfer protocols are vulnerable to
a denial of service attack from clients which complete the key
exchange slowly, or not at all, causing pending content to be read
into memory, but never transmitted.

This updates golang.org/x/net to fix the problem.

See: https://pkg.go.dev/vuln/GO-2025-3487
See: https://www.cve.org/CVERecord?id=CVE-2025-22869
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-30 16:09:30 +01:00
Nick Craig-Wood
775f3e49f0 build: update golang.org/x/crypto to v0.35.0 to fix CVE-2025-22869
SSH servers which implement file transfer protocols are vulnerable to
a denial of service attack from clients which complete the key
exchange slowly, or not at all, causing pending content to be read
into memory, but never transmitted.

This affects users of `rclone serve sftp`.

See: https://pkg.go.dev/vuln/GO-2025-3487
2025-04-30 16:09:30 +01:00
Jugal Kishore
b0310c7273 docs: correct SSL docs anchor link from #ssl-tls to #tls-ssl
Fixed the anchor link in the documentation that points to the SSL/TLS section.
This change ensures the link directs correctly to the intended section (#tls-ssl) instead of the incorrect #ssl-tls.

No functional code changes, documentation only.
2025-04-30 16:09:30 +01:00
Nick Craig-Wood
07bb122d47 drive: metadata: fix error when setting copy-requires-writer-permission on a folder
This appears not to be allowed, so this fixes the problem by ignoring
that metadata for a folder.

Fixes #8517
2025-04-30 16:09:30 +01:00
Ed Craig-Wood
aedcd56531 Update README.md
removed warp as project sponsor
2025-04-30 16:09:30 +01:00
Andrew Kreimer
4a10491c2a docs: fix typos via codespell
There are some types in the changelog.

Fix them via codespell.
2025-04-30 16:09:30 +01:00
Nick Craig-Wood
bbceb84cad onedrive: fix metadata ordering in permissions
Before this change, due to a quirk in Graph, User permissions could be
lost when applying permissions.

Fixes #8465
2025-04-30 16:09:30 +01:00
Ben Alex
4c3737014a iclouddrive: fix so created files are writable
At present any created file (eg through the touch command, copy, mount
etc) is read-only in iCloud.

This has been reported by users at
https://forum.rclone.org/t/icloud-and-file-editing-permissions/50659.
2025-04-30 16:09:30 +01:00
simwai
a49ccddb81 cmd/authorize: show required arguments in help text 2025-04-30 16:09:30 +01:00
Markus Gerstel
7fd3d0d1ab s3: documentation regression - fixes #8438
We lost a previous documentation fix (#7077) detailing how to restore
single objects from AWS S3 Glacier.

Also make clearer that rclone provides restore functionality natively.

Co-authored-by: danielkrajnik <dan94kra@gmail.com>
2025-04-30 16:09:30 +01:00
huanghaojun
77c1031dcd vfs: fix inefficient directory caching when directory reads are slow
Before this change, when querying directories with large datasets, if
the query duration exceeded the directory cache expiration time, the
cache became invalid by the time results were retrieved. This means
every execution of `_readDir` triggers `_readDirFromEntries`,
resulting in prolonged processing times.

After this change we update the directory time with the time at the
end of the query.
2025-04-30 16:09:30 +01:00
simonmcnair
d8c7031dc3 docs: update fuse version in docker docs 2025-04-30 16:09:30 +01:00
Samantha Bowen
ab5844df77 fs/config: Read configuration passwords from stdin even when terminated with EOF - fixes #8480 2025-04-30 16:09:30 +01:00
Nick Craig-Wood
a3c4e2fd20 lib/http: fix race between Serve() and Shutdown()
This was discovered by the race detector.
2025-04-30 16:09:30 +01:00
Danny Garside
b12d2f32db docs: fix minor typo in box docs 2025-04-30 16:09:30 +01:00
Nick Craig-Wood
85c1563701 docs: Add rcloneui.com as Silver Sponsor 2025-04-30 16:09:30 +01:00
eccoisle
f52d753363 docs: replace option --auto-filename-header with --header-filename 2025-04-30 16:09:30 +01:00
alingse
f76c72b5cc operations: fix call fmt.Errorf with wrong err 2025-04-30 16:09:30 +01:00
Nick Craig-Wood
46dcc0eaf6 rc: fix debug/* commands not being available over unix sockets
This was caused by an incorrect handler URL which was passing the
debug/* commands to the debug/pprof handler by accident. This only
happened when using unix sockets.
2025-04-30 16:09:30 +01:00
nielash
f46787dcc3 http: correct root if definitely pointing to a file - fixes #8428
This was formalized in
c69eb84573
But it appears that we forgot to update `http`, and the `FsRoot` test didn't
catch it because we don't currently have an http integration test.
2025-04-30 16:09:30 +01:00
Nick Craig-Wood
e7c46912a3 fs: fix corruption of SizeSuffix with "B" suffix in config (eg --min-size)
Before this change, the config system round tripped fs.SizeSuffix
values through strings like this, corrupting them in the process.

    "2B" -> 2 -> "2" -> 2048

This caused `--min-size 2B` to be interpreted as `--min-size 2k`.

This fix makes sure SizeSuffix values have a "B" suffix when turned
into a string where necessary, so it becomes

    "2B" -> 2 -> "2B" -> 2

In rclone v2 we should probably declare unsuffixed SizeSuffix values
are in bytes not kBytes (done for rsync compatibility) but this would
be a backwards incompatible change which we don't want for v1.

Fixes #8437
Fixes #8212
Fixes #5169
2025-04-30 16:09:30 +01:00
jack
296c26b671 docs: fix incorrect mentions of vfs-cache-min-free-size 2025-04-30 16:09:30 +01:00
Nick Craig-Wood
ef99b9ad18 fs/object: fix memory object out of bounds Seek 2025-04-30 16:09:30 +01:00
Nick Craig-Wood
2e853b6e75 serve nfs: fix unlikely crash 2025-04-30 16:09:30 +01:00
Zachary Vorhies
d725f4f4de docs: add note for using rclone cat for slicing out a byte range from a file 2025-04-30 16:09:30 +01:00
Dave Vasilevsky
6a9c2350de dropbox: Retry link without expiry
Dropbox only allows public links with expiry for certain account types.
Rather than erroring for other accounts, retry without expiry.
2025-04-30 16:09:30 +01:00
emyarod
659aca8711 chore: update contributor email 2025-04-30 16:09:30 +01:00
Nick Craig-Wood
1ac7e32302 docs: correct stable release workflow 2025-04-30 16:09:30 +01:00
Lorenz Brun
286ae6a1b4 vfs: fix directory cache serving stale data
The VFS directory cache layer didn't update directory entry properties
if they are reused after cache invalidation.

Update them unconditionally as newDir sets them to the same value and
setting a pointer is cheaper in both LoC as well as CPU cycles than a
branch.

Also add a test exercising this behavior.

Fixes #6335
2025-04-30 16:09:30 +01:00
Anagh Kumar Baranwal
861456d970 build: fix docker plugin build - fixes #8394
Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
2025-04-30 16:09:30 +01:00
Michael Kebe
5fa9c0209e docs: improved sftp limitations
Added a link to `--sftp-path-override` for a better solution with working hash calculation.
2025-04-30 16:09:30 +01:00
Anagh Kumar Baranwal
b00dcc37bd accounting: fix percentDiff calculation -- fixes #8345
Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
2025-04-30 16:09:30 +01:00
Janne Hellsten
92efc5ff43 cmd: fix crash if rclone is invoked without any arguments - Fixes #8378 2025-04-30 16:09:30 +01:00
ll3006
57bbb4be9f sync: copy dir modtimes even when copyEmptySrcDirs is false - fixes #8317
Before, after a sync, only file modtimes were updated when not using
--copy-empty-src-dirs. This ensures modtimes are updated to match the source
folder, regardless of copyEmptySrcDir. The flag --no-update-dir-modtime
(which previously did nothing) will disable this.
2025-04-30 16:09:30 +01:00
Anagh Kumar Baranwal
c62fbc5269 stats: fix the speed not getting updated after a pause in the processing
This shifts the behavior of the average loop to be a persistent loop
that gets resumed/paused when transfers & checks are started/completed.

Previously, the averageLoop was stopped on completion of
transfers & checks but failed to start again due to the protection of
the sync.Once

Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
2025-04-30 16:09:30 +01:00
Nick Craig-Wood
6eb8919719 sync: fix cpu spinning when empty directory finding with leading slashes
Before this change the logic which makes sure we create all
directories could get confused with directories which started with
slashes and get into an infinite loop consuming 100% of the CPU.
2025-04-30 16:09:30 +01:00
hiddenmarten
ff4d7c8dd0 rc: disable the metrics server when running rclone rc
Fixes #8248
2025-04-30 16:09:29 +01:00
Nick Craig-Wood
4f8dfd14fc Start v1.69.2-DEV development 2025-02-14 17:14:31 +00:00
66 changed files with 1151 additions and 373 deletions

View File

@@ -26,7 +26,7 @@ jobs:
strategy:
fail-fast: false
matrix:
job_name: ['linux', 'linux_386', 'mac_amd64', 'mac_arm64', 'windows', 'other_os', 'go1.21', 'go1.22']
job_name: ['linux', 'linux_386', 'mac_amd64', 'mac_arm64', 'windows', 'other_os']
include:
- job_name: linux
@@ -80,18 +80,6 @@ jobs:
compile_all: true
deploy: true
- job_name: go1.21
os: ubuntu-latest
go: '1.21'
quicktest: true
racequicktest: true
- job_name: go1.22
os: ubuntu-latest
go: '1.22'
quicktest: true
racequicktest: true
name: ${{ matrix.job_name }}
runs-on: ${{ matrix.os }}

View File

@@ -7,39 +7,43 @@ name: Release Build for Docker Plugin
on:
release:
types: [published]
workflow_dispatch:
inputs:
manual:
description: Manual run (bypass default conditions)
type: boolean
default: true
jobs:
build_docker_volume_plugin:
if: github.repository == 'rclone/rclone'
needs: build
runs-on: ubuntu-latest
name: Build docker plugin job
steps:
- name: Free some space
shell: bash
run: |
df -h .
# Remove android SDK
sudo rm -rf /usr/local/lib/android || true
# Remove .net runtime
sudo rm -rf /usr/share/dotnet || true
df -h .
- name: Checkout master
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build and publish docker plugin
shell: bash
run: |
VER=${GITHUB_REF#refs/tags/}
PLUGIN_USER=rclone
docker login --username ${{ secrets.DOCKER_HUB_USER }} \
--password-stdin <<< "${{ secrets.DOCKER_HUB_PASSWORD }}"
for PLUGIN_ARCH in amd64 arm64 arm/v7 arm/v6 ;do
export PLUGIN_USER PLUGIN_ARCH
make docker-plugin PLUGIN_TAG=${PLUGIN_ARCH/\//-}
make docker-plugin PLUGIN_TAG=${PLUGIN_ARCH/\//-}-${VER#v}
done
make docker-plugin PLUGIN_ARCH=amd64 PLUGIN_TAG=latest
make docker-plugin PLUGIN_ARCH=amd64 PLUGIN_TAG=${VER#v}
build_docker_volume_plugin:
if: inputs.manual || github.repository == 'rclone/rclone'
name: Build docker plugin job
runs-on: ubuntu-latest
steps:
- name: Free some space
shell: bash
run: |
df -h .
# Remove android SDK
sudo rm -rf /usr/local/lib/android || true
# Remove .net runtime
sudo rm -rf /usr/share/dotnet || true
df -h .
- name: Checkout master
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build and publish docker plugin
shell: bash
run: |
VER=${GITHUB_REF#refs/tags/}
PLUGIN_USER=rclone
docker login --username ${{ secrets.DOCKER_HUB_USER }} \
--password-stdin <<< "${{ secrets.DOCKER_HUB_PASSWORD }}"
for PLUGIN_ARCH in amd64 arm64 arm/v7 arm/v6 ;do
export PLUGIN_USER PLUGIN_ARCH
make docker-plugin PLUGIN_TAG=${PLUGIN_ARCH/\//-}
make docker-plugin PLUGIN_TAG=${PLUGIN_ARCH/\//-}-${VER#v}
done
make docker-plugin PLUGIN_ARCH=amd64 PLUGIN_TAG=latest
make docker-plugin PLUGIN_ARCH=amd64 PLUGIN_TAG=${VER#v}

122
MANUAL.html generated
View File

@@ -5,6 +5,7 @@
<meta name="generator" content="pandoc" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
<meta name="author" content="Nick Craig-Wood" />
<meta name="dcterms.date" content="2025-05-21" />
<title>rclone(1) User Manual</title>
<style>
code{white-space: pre-wrap;}
@@ -81,7 +82,7 @@
<header id="title-block-header">
<h1 class="title">rclone(1) User Manual</h1>
<p class="author">Nick Craig-Wood</p>
<p class="date">Feb 14, 2025</p>
<p class="date">May 21, 2025</p>
</header>
<h1 id="name">NAME</h1>
<p>rclone - manage files on cloud storage</p>
@@ -1694,9 +1695,10 @@ Other: 8849156022</code></pre>
<p>Remote authorization.</p>
<h2 id="synopsis-19">Synopsis</h2>
<p>Remote authorization. Used to authorize a remote or headless rclone from a machine with a browser - use as instructed by rclone config.</p>
<p>The command requires 1-3 arguments: - fs name (e.g., "drive", "s3", etc.) - Either a base64 encoded JSON blob obtained from a previous rclone config session - Or a client_id and client_secret pair obtained from the remote service</p>
<p>Use --auth-no-open-browser to prevent rclone to open auth link in default browser automatically.</p>
<p>Use --template to generate HTML output via a custom Go template. If a blank string is provided as an argument to this flag, the default template is used.</p>
<pre><code>rclone authorize [flags]</code></pre>
<pre><code>rclone authorize &lt;fs name&gt; [base64_json_blob | client_id client_secret] [flags]</code></pre>
<h2 id="options-19">Options</h2>
<pre><code> --auth-no-open-browser Do not automatically open auth link in default browser
-h, --help help for authorize
@@ -2421,6 +2423,7 @@ if src is directory
copy it to dst, overwriting existing files if they exist
see copy command for full details</code></pre>
<p>This doesn't transfer files that are identical on src and dst, testing by size and modification time or MD5SUM. It doesn't delete files from the destination.</p>
<p><em>If you are looking to copy just a byte range of a file, please see 'rclone cat --offset X --count Y'</em></p>
<p><strong>Note</strong>: Use the <code>-P</code>/<code>--progress</code> flag to view real-time transfer statistics</p>
<pre><code>rclone copyto source:path dest:path [flags]</code></pre>
<h2 id="options-48">Options</h2>
@@ -2503,7 +2506,7 @@ if src is directory
<h2 id="synopsis-42">Synopsis</h2>
<p>Download a URL's content and copy it to the destination without saving it in temporary storage.</p>
<p>Setting <code>--auto-filename</code> will attempt to automatically determine the filename from the URL (after any redirections) and used in the destination path.</p>
<p>With <code>--auto-filename-header</code> in addition, if a specific filename is set in HTTP headers, it will be used instead of the name from the URL. With <code>--print-filename</code> in addition, the resulting file name will be printed.</p>
<p>With <code>--header-filename</code> in addition, if a specific filename is set in HTTP headers, it will be used instead of the name from the URL. With <code>--print-filename</code> in addition, the resulting file name will be printed.</p>
<p>Setting <code>--no-clobber</code> will prevent overwriting file on the destination if there is one with the same name.</p>
<p>Setting <code>--stdout</code> or making the output file name <code>-</code> will cause the output to be written to standard output.</p>
<h2 id="troubleshooting">Troubleshooting</h2>
@@ -3198,7 +3201,7 @@ WantedBy=multi-user.target</code></pre>
<p>If run with <code>-vv</code> rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with <code>--cache-dir</code> or setting the appropriate environment variable.</p>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for <code>--vfs-write-back</code> seconds. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>The <code>--vfs-cache-max-age</code> will evict files from the cache after the set time since last access has passed. The default value of 1 hour will start evicting files from cache that haven't been accessed for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 and will wait for 1 more hour before evicting. Specify the time with standard notation, s, m, h, d, w .</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off">--vfs-cache-mode off</h3>
@@ -3737,7 +3740,7 @@ WantedBy=multi-user.target</code></pre>
<p>If run with <code>-vv</code> rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with <code>--cache-dir</code> or setting the appropriate environment variable.</p>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for <code>--vfs-write-back</code> seconds. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>The <code>--vfs-cache-max-age</code> will evict files from the cache after the set time since last access has passed. The default value of 1 hour will start evicting files from cache that haven't been accessed for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 and will wait for 1 more hour before evicting. Specify the time with standard notation, s, m, h, d, w .</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-1">--vfs-cache-mode off</h3>
@@ -4321,7 +4324,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>If run with <code>-vv</code> rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with <code>--cache-dir</code> or setting the appropriate environment variable.</p>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for <code>--vfs-write-back</code> seconds. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>The <code>--vfs-cache-max-age</code> will evict files from the cache after the set time since last access has passed. The default value of 1 hour will start evicting files from cache that haven't been accessed for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 and will wait for 1 more hour before evicting. Specify the time with standard notation, s, m, h, d, w .</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-2">--vfs-cache-mode off</h3>
@@ -4544,7 +4547,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>If run with <code>-vv</code> rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with <code>--cache-dir</code> or setting the appropriate environment variable.</p>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for <code>--vfs-write-back</code> seconds. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>The <code>--vfs-cache-max-age</code> will evict files from the cache after the set time since last access has passed. The default value of 1 hour will start evicting files from cache that haven't been accessed for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 and will wait for 1 more hour before evicting. Specify the time with standard notation, s, m, h, d, w .</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-3">--vfs-cache-mode off</h3>
@@ -4788,7 +4791,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>If run with <code>-vv</code> rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with <code>--cache-dir</code> or setting the appropriate environment variable.</p>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for <code>--vfs-write-back</code> seconds. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>The <code>--vfs-cache-max-age</code> will evict files from the cache after the set time since last access has passed. The default value of 1 hour will start evicting files from cache that haven't been accessed for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 and will wait for 1 more hour before evicting. Specify the time with standard notation, s, m, h, d, w .</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-4">--vfs-cache-mode off</h3>
@@ -5181,7 +5184,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>If run with <code>-vv</code> rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with <code>--cache-dir</code> or setting the appropriate environment variable.</p>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for <code>--vfs-write-back</code> seconds. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>The <code>--vfs-cache-max-age</code> will evict files from the cache after the set time since last access has passed. The default value of 1 hour will start evicting files from cache that haven't been accessed for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 and will wait for 1 more hour before evicting. Specify the time with standard notation, s, m, h, d, w .</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-5">--vfs-cache-mode off</h3>
@@ -5455,7 +5458,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>If run with <code>-vv</code> rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with <code>--cache-dir</code> or setting the appropriate environment variable.</p>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for <code>--vfs-write-back</code> seconds. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>The <code>--vfs-cache-max-age</code> will evict files from the cache after the set time since last access has passed. The default value of 1 hour will start evicting files from cache that haven't been accessed for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 and will wait for 1 more hour before evicting. Specify the time with standard notation, s, m, h, d, w .</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-6">--vfs-cache-mode off</h3>
@@ -5739,7 +5742,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p><code>serve s3</code> is considered <strong>Experimental</strong> so use with care.</p>
<p>S3 server supports Signature Version 4 authentication. Just use <code>--auth-key accessKey,secretKey</code> and set the <code>Authorization</code> header correctly in the request. (See the <a href="https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html">AWS docs</a>).</p>
<p><code>--auth-key</code> can be repeated for multiple auth pairs. If <code>--auth-key</code> is not provided then <code>serve s3</code> will allow anonymous access.</p>
<p>Please note that some clients may require HTTPS endpoints. See <a href="#ssl-tls">the SSL docs</a> for more information.</p>
<p>Please note that some clients may require HTTPS endpoints. See <a href="#tls-ssl">the SSL docs</a> for more information.</p>
<p>This command uses the <a href="#vfs-virtual-file-system">VFS directory cache</a>. All the functionality will work with <code>--vfs-cache-mode off</code>. Using <code>--vfs-cache-mode full</code> (or <code>writes</code>) can be used to cache objects locally to improve performance.</p>
<p>Use <code>--force-path-style=false</code> if you want to use the bucket name as a part of the hostname (such as mybucket.local)</p>
<p>Use <code>--etag-hash</code> if you want to change the hash uses for the <code>ETag</code>. Note that using anything other than <code>MD5</code> (the default) is likely to cause problems for S3 clients which rely on the Etag being the MD5.</p>
@@ -5859,7 +5862,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>If run with <code>-vv</code> rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with <code>--cache-dir</code> or setting the appropriate environment variable.</p>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for <code>--vfs-write-back</code> seconds. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>The <code>--vfs-cache-max-age</code> will evict files from the cache after the set time since last access has passed. The default value of 1 hour will start evicting files from cache that haven't been accessed for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 and will wait for 1 more hour before evicting. Specify the time with standard notation, s, m, h, d, w .</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-7">--vfs-cache-mode off</h3>
@@ -6108,7 +6111,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>If run with <code>-vv</code> rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with <code>--cache-dir</code> or setting the appropriate environment variable.</p>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for <code>--vfs-write-back</code> seconds. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>The <code>--vfs-cache-max-age</code> will evict files from the cache after the set time since last access has passed. The default value of 1 hour will start evicting files from cache that haven't been accessed for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 and will wait for 1 more hour before evicting. Specify the time with standard notation, s, m, h, d, w .</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-8">--vfs-cache-mode off</h3>
@@ -6513,7 +6516,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>If run with <code>-vv</code> rclone will print the location of the file cache. The files are stored in the user cache file area which is OS dependent but can be controlled with <code>--cache-dir</code> or setting the appropriate environment variable.</p>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for <code>--vfs-write-back</code> seconds. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-size</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>If using <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> note that the cache may exceed these quotas for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache. When <code>--vfs-cache-max-size</code> or <code>--vfs-cache-min-free-space</code> is exceeded, rclone will attempt to evict the least accessed files from the cache first. rclone will start with files that haven't been accessed for the longest. This cache flushing strategy is efficient and more relevant files are likely to remain cached.</p>
<p>The <code>--vfs-cache-max-age</code> will evict files from the cache after the set time since last access has passed. The default value of 1 hour will start evicting files from cache that haven't been accessed for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 and will wait for 1 more hour before evicting. Specify the time with standard notation, s, m, h, d, w .</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-9">--vfs-cache-mode off</h3>
@@ -12289,7 +12292,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
--user-agent string Set the user-agent to a specified string (default &quot;rclone/v1.69.1&quot;)</code></pre>
--user-agent string Set the user-agent to a specified string (default &quot;rclone/v1.69.3&quot;)</code></pre>
<h2 id="performance">Performance</h2>
<p>Flags helpful for increasing performance.</p>
<pre><code> --buffer-size SizeSuffix In memory buffer size when reading files for each --transfer (default 16Mi)
@@ -13238,7 +13241,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
<p>In the first example we will use the <a href="https://rclone.org/sftp/">SFTP</a> rclone volume with Docker engine on a standalone Ubuntu machine.</p>
<p>Start from <a href="https://docs.docker.com/engine/install/">installing Docker</a> on the host.</p>
<p>The <em>FUSE</em> driver is a prerequisite for rclone mounting and should be installed on host:</p>
<pre><code>sudo apt-get -y install fuse</code></pre>
<pre><code>sudo apt-get -y install fuse3</code></pre>
<p>Create two directories required by rclone docker plugin:</p>
<pre><code>sudo mkdir -p /var/lib/docker-plugins/rclone/config
sudo mkdir -p /var/lib/docker-plugins/rclone/cache</code></pre>
@@ -15152,7 +15155,7 @@ $ rclone -q --s3-versions ls s3:cleanup-test
<h3 id="glacier-and-glacier-deep-archive">Glacier and Glacier Deep Archive</h3>
<p>You can upload objects using the glacier storage class or transition them to glacier using a <a href="http://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html">lifecycle policy</a>. The bucket can still be synced or copied into normally, but if rclone tries to access data from the glacier storage class you will see an error like below.</p>
<pre><code>2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file</code></pre>
<p>In this case you need to <a href="http://docs.aws.amazon.com/AmazonS3/latest/user-guide/restore-archived-objects.html">restore</a> the object(s) in question before using rclone.</p>
<p>In this case you need to <a href="http://docs.aws.amazon.com/AmazonS3/latest/user-guide/restore-archived-objects.html">restore</a> the object(s) in question before accessing object contents. The <a href="#restore">restore</a> section below shows how to do this with rclone.</p>
<p>Note that rclone only speaks the S3 API it does not speak the Glacier Vault API, so rclone cannot directly access Glacier Vaults.</p>
<h3 id="object-lock-enabled-s3-bucket">Object-lock enabled S3 bucket</h3>
<p>According to AWS's <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock-overview.html#object-lock-permission">documentation on S3 Object Lock</a>:</p>
@@ -16512,7 +16515,7 @@ Windows: &quot;%USERPROFILE%\.aws\credentials&quot;</code></pre>
<pre><code>rclone backend restore remote: [options] [&lt;arguments&gt;+]</code></pre>
<p>This command can be used to restore one or more objects from GLACIER to normal storage or from INTELLIGENT-TIERING Archive Access / Deep Archive Access tier to the Frequent Access tier.</p>
<p>Usage Examples:</p>
<pre><code>rclone backend restore s3:bucket/path/to/object -o priority=PRIORITY -o lifetime=DAYS
<pre><code>rclone backend restore s3:bucket/path/to/ --include /object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY</code></pre>
@@ -19889,7 +19892,7 @@ e) Edit this remote
d) Delete this remote
y/e/d&gt; y</code></pre>
<p>See the <a href="https://rclone.org/remote_setup/">remote setup docs</a> for how to set it up on a machine with no Internet browser available.</p>
<p>Note that rclone runs a webserver on your local machine to collect the token as returned from Box. This only runs from the moment it opens your browser to the moment you get back the verification code. This is on <code>http://127.0.0.1:53682/</code> and this it may require you to unblock it temporarily if you are running a host firewall.</p>
<p>Note that rclone runs a webserver on your local machine to collect the token as returned from Box. This only runs from the moment it opens your browser to the moment you get back the verification code. This is on <code>http://127.0.0.1:53682/</code> and this may require you to unblock it temporarily if you are running a host firewall.</p>
<p>Once configured you can then use <code>rclone</code> like this,</p>
<p>List directories in top level of your Box</p>
<pre><code>rclone lsd remote:</code></pre>
@@ -35553,7 +35556,7 @@ server_command = sudo /usr/libexec/openssh/sftp-server</code></pre>
<li>Required: false</li>
</ul>
<h2 id="limitations-36">Limitations</h2>
<p>On some SFTP servers (e.g. Synology) the paths are different for SSH and SFTP so the hashes can't be calculated properly. For them using <code>disable_hashcheck</code> is a good idea.</p>
<p>On some SFTP servers (e.g. Synology) the paths are different for SSH and SFTP so the hashes can't be calculated properly. You can either use <a href="#--sftp-path-override"><code>--sftp-path-override</code></a> or <a href="#--sftp-disable-hashcheck"><code>disable_hashcheck</code></a>.</p>
<p>The only ssh agent supported under Windows is Putty's pageant.</p>
<p>The Go SSH library disables the use of the aes128-cbc cipher by default, due to security concerns. This can be re-enabled on a per-connection basis by setting the <code>use_insecure_cipher</code> setting in the configuration file to <code>true</code>. Further details on the insecurity of this cipher can be found <a href="http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf">in this paper</a>.</p>
<p>SFTP isn't supported under plan9 until <a href="https://github.com/pkg/sftp/issues/156">this issue</a> is fixed.</p>
@@ -38311,6 +38314,78 @@ $ tree /tmp/c
<li>"error": return an error based on option value</li>
</ul>
<h1 id="changelog-1">Changelog</h1>
<h2 id="v1.69.3---2025-05-21">v1.69.3 - 2025-05-21</h2>
<p><a href="https://github.com/rclone/rclone/compare/v1.69.2...v1.69.3">See commits</a></p>
<ul>
<li>Bug Fixes
<ul>
<li>build: Reapply update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 to fix CVE-2025-30204 (dependabot[bot])</li>
<li>build: Update github.com/ebitengine/purego to work around bug in go1.24.3 (Nick Craig-Wood)</li>
</ul></li>
</ul>
<h2 id="v1.69.2---2025-05-01">v1.69.2 - 2025-05-01</h2>
<p><a href="https://github.com/rclone/rclone/compare/v1.69.1...v1.69.2">See commits</a></p>
<ul>
<li>Bug fixes
<ul>
<li>accounting: Fix percentDiff calculation -- (Anagh Kumar Baranwal)</li>
<li>build
<ul>
<li>Update github.com/golang-jwt/jwt/v4 from 4.5.1 to 4.5.2 to fix CVE-2025-30204 (dependabot[bot])</li>
<li>Update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 to fix CVE-2025-30204 (dependabot[bot])</li>
<li>Update golang.org/x/crypto to v0.35.0 to fix CVE-2025-22869 (Nick Craig-Wood)</li>
<li>Update golang.org/x/net from 0.36.0 to 0.38.0 to fix CVE-2025-22870 (dependabot[bot])</li>
<li>Update golang.org/x/net to 0.36.0. to fix CVE-2025-22869 (dependabot[bot])</li>
<li>Stop building with go &lt; go1.23 as security updates forbade it (Nick Craig-Wood)</li>
<li>Fix docker plugin build (Anagh Kumar Baranwal)</li>
</ul></li>
<li>cmd: Fix crash if rclone is invoked without any arguments (Janne Hellsten)</li>
<li>config: Read configuration passwords from stdin even when terminated with EOF (Samantha Bowen)</li>
<li>doc fixes (Andrew Kreimer, Danny Garside, eccoisle, Ed Craig-Wood, emyarod, jack, Jugal Kishore, Markus Gerstel, Michael Kebe, Nick Craig-Wood, simonmcnair, simwai, Zachary Vorhies)</li>
<li>fs: Fix corruption of SizeSuffix with "B" suffix in config (eg --min-size) (Nick Craig-Wood)</li>
<li>lib/http: Fix race between Serve() and Shutdown() (Nick Craig-Wood)</li>
<li>object: Fix memory object out of bounds Seek (Nick Craig-Wood)</li>
<li>operations: Fix call fmt.Errorf with wrong err (alingse)</li>
<li>rc
<ul>
<li>Disable the metrics server when running <code>rclone rc</code> (hiddenmarten)</li>
<li>Fix debug/* commands not being available over unix sockets (Nick Craig-Wood)</li>
</ul></li>
<li>serve nfs: Fix unlikely crash (Nick Craig-Wood)</li>
<li>stats: Fix the speed not getting updated after a pause in the processing (Anagh Kumar Baranwal)</li>
<li>sync
<ul>
<li>Fix cpu spinning when empty directory finding with leading slashes (Nick Craig-Wood)</li>
<li>Copy dir modtimes even when copyEmptySrcDirs is false (ll3006)</li>
</ul></li>
</ul></li>
<li>VFS
<ul>
<li>Fix directory cache serving stale data (Lorenz Brun)</li>
<li>Fix inefficient directory caching when directory reads are slow (huanghaojun)</li>
<li>Fix integration test failures (Nick Craig-Wood)</li>
</ul></li>
<li>Drive
<ul>
<li>Metadata: fix error when setting copy-requires-writer-permission on a folder (Nick Craig-Wood)</li>
</ul></li>
<li>Dropbox
<ul>
<li>Retry link without expiry (Dave Vasilevsky)</li>
</ul></li>
<li>HTTP
<ul>
<li>Correct root if definitely pointing to a file (nielash)</li>
</ul></li>
<li>Iclouddrive
<ul>
<li>Fix so created files are writable (Ben Alex)</li>
</ul></li>
<li>Onedrive
<ul>
<li>Fix metadata ordering in permissions (Nick Craig-Wood)</li>
</ul></li>
</ul>
<h2 id="v1.69.1---2025-02-14">v1.69.1 - 2025-02-14</h2>
<p><a href="https://github.com/rclone/rclone/compare/v1.69.0...v1.69.1">See commits</a></p>
<ul>
@@ -38390,7 +38465,7 @@ $ tree /tmp/c
<li>http servers: Disable automatic authentication skipping for unix sockets in http servers (Moises Lima)
<ul>
<li>This was making it impossible to use unix sockets with an proxy</li>
<li>This might now cause rclone to need authenticaton where it didn't before</li>
<li>This might now cause rclone to need authentication where it didn't before</li>
</ul></li>
<li>oauthutil: add support for OAuth client credential flow (Martin Hassack, Nick Craig-Wood)</li>
<li>operations: make log messages consistent for mkdir/rmdir at INFO level (Nick Craig-Wood)</li>
@@ -39430,7 +39505,7 @@ $ tree /tmp/c
<li>Refactor version info and icon resource handling on windows (albertony)</li>
</ul></li>
<li>doc updates (albertony, alfish2000, asdffdsazqqq, Dimitri Papadopoulos, Herby Gillot, Joda Stößer, Manoj Ghosh, Nick Craig-Wood)</li>
<li>Implement <code>--metadata-mapper</code> to transform metatadata with a user supplied program (Nick Craig-Wood)</li>
<li>Implement <code>--metadata-mapper</code> to transform metadata with a user supplied program (Nick Craig-Wood)</li>
<li>Add <code>ChunkWriterDoesntSeek</code> feature flag and set it for b2 (Nick Craig-Wood)</li>
<li>lib/http: Export basic go string functions for use in <code>--template</code> (Gabriel Espinoza)</li>
<li>makefile: Use POSIX compatible install arguments (Mina Galić)</li>
@@ -39605,7 +39680,7 @@ $ tree /tmp/c
<li>B2
<ul>
<li>Fix multipart upload: corrupted on transfer: sizes differ XXX vs 0 (Nick Craig-Wood)</li>
<li>Fix locking window when getting mutipart upload URL (Nick Craig-Wood)</li>
<li>Fix locking window when getting multipart upload URL (Nick Craig-Wood)</li>
<li>Fix server side copies greater than 4GB (Nick Craig-Wood)</li>
<li>Fix chunked streaming uploads (Nick Craig-Wood)</li>
<li>Reduce default <code>--b2-upload-concurrency</code> to 4 to reduce memory usage (Nick Craig-Wood)</li>
@@ -47606,7 +47681,6 @@ THE SOFTWARE.</code></pre>
<li>ben-ba <a href="mailto:benjamin.brauner@gmx.de" class="email">benjamin.brauner@gmx.de</a></li>
<li>Eli Orzitzer <a href="mailto:e_orz@yahoo.com" class="email">e_orz@yahoo.com</a></li>
<li>Anthony Metzidis <a href="mailto:anthony.metzidis@gmail.com" class="email">anthony.metzidis@gmail.com</a></li>
<li>emyarod <a href="mailto:afw5059@gmail.com" class="email">afw5059@gmail.com</a></li>
<li>keongalvin <a href="mailto:keongalvin@gmail.com" class="email">keongalvin@gmail.com</a></li>
<li>rarspace01 <a href="mailto:rarspace01@users.noreply.github.com" class="email">rarspace01@users.noreply.github.com</a></li>
<li>Paul Stern <a href="mailto:paulstern45@gmail.com" class="email">paulstern45@gmail.com</a></li>

128
MANUAL.md generated
View File

@@ -1,6 +1,6 @@
% rclone(1) User Manual
% Nick Craig-Wood
% Feb 14, 2025
% May 21, 2025
# NAME
@@ -2858,13 +2858,18 @@ Remote authorization. Used to authorize a remote or headless
rclone from a machine with a browser - use as instructed by
rclone config.
The command requires 1-3 arguments:
- fs name (e.g., "drive", "s3", etc.)
- Either a base64 encoded JSON blob obtained from a previous rclone config session
- Or a client_id and client_secret pair obtained from the remote service
Use --auth-no-open-browser to prevent rclone to open auth
link in default browser automatically.
Use --template to generate HTML output via a custom Go template. If a blank string is provided as an argument to this flag, the default template is used.
```
rclone authorize [flags]
rclone authorize <fs name> [base64_json_blob | client_id client_secret] [flags]
```
## Options
@@ -4232,6 +4237,8 @@ This doesn't transfer files that are identical on src and dst, testing
by size and modification time or MD5SUM. It doesn't delete files from
the destination.
*If you are looking to copy just a byte range of a file, please see 'rclone cat --offset X --count Y'*
**Note**: Use the `-P`/`--progress` flag to view real-time transfer statistics
@@ -4353,7 +4360,7 @@ Setting `--auto-filename` will attempt to automatically determine the
filename from the URL (after any redirections) and used in the
destination path.
With `--auto-filename-header` in addition, if a specific filename is
With `--header-filename` in addition, if a specific filename is
set in HTTP headers, it will be used instead of the name from the URL.
With `--print-filename` in addition, the resulting file name will be
printed.
@@ -5861,11 +5868,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant
@@ -7123,11 +7130,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant
@@ -8251,11 +8258,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant
@@ -8806,11 +8813,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant
@@ -9363,11 +9370,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant
@@ -10101,11 +10108,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant
@@ -10778,11 +10785,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant
@@ -11427,7 +11434,7 @@ docs](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html)).
access.
Please note that some clients may require HTTPS endpoints. See [the
SSL docs](#ssl-tls) for more information.
SSL docs](#tls-ssl) for more information.
This command uses the [VFS directory cache](#vfs-virtual-file-system).
All the functionality will work with `--vfs-cache-mode off`. Using
@@ -11734,11 +11741,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant
@@ -12328,11 +12335,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant
@@ -13109,11 +13116,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant
@@ -21097,7 +21104,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
--user-agent string Set the user-agent to a specified string (default "rclone/v1.69.1")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.69.3")
```
@@ -22141,7 +22148,7 @@ on the host.
The _FUSE_ driver is a prerequisite for rclone mounting and should be
installed on host:
```
sudo apt-get -y install fuse
sudo apt-get -y install fuse3
```
Create two directories required by rclone docker plugin:
@@ -25783,7 +25790,8 @@ tries to access data from the glacier storage class you will see an error like b
2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file
In this case you need to [restore](http://docs.aws.amazon.com/AmazonS3/latest/user-guide/restore-archived-objects.html)
the object(s) in question before using rclone.
the object(s) in question before accessing object contents.
The [restore](#restore) section below shows how to do this with rclone.
Note that rclone only speaks the S3 API it does not speak the Glacier
Vault API, so rclone cannot directly access Glacier Vaults.
@@ -27186,7 +27194,7 @@ or from INTELLIGENT-TIERING Archive Access / Deep Archive Access tier to the Fre
Usage Examples:
rclone backend restore s3:bucket/path/to/object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/ --include /object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY
@@ -31592,7 +31600,7 @@ machine with no Internet browser available.
Note that rclone runs a webserver on your local machine to collect the
token as returned from Box. This only runs from the moment it opens
your browser to the moment you get back the verification code. This
is on `http://127.0.0.1:53682/` and this it may require you to unblock
is on `http://127.0.0.1:53682/` and this may require you to unblock
it temporarily if you are running a host firewall.
Once configured you can then use `rclone` like this,
@@ -53113,7 +53121,8 @@ Properties:
On some SFTP servers (e.g. Synology) the paths are different
for SSH and SFTP so the hashes can't be calculated properly.
For them using `disable_hashcheck` is a good idea.
You can either use [`--sftp-path-override`](#--sftp-path-override)
or [`disable_hashcheck`](#--sftp-disable-hashcheck).
The only ssh agent supported under Windows is Putty's pageant.
@@ -56649,6 +56658,58 @@ Options:
# Changelog
## v1.69.3 - 2025-05-21
[See commits](https://github.com/rclone/rclone/compare/v1.69.2...v1.69.3)
* Bug Fixes
* build: Reapply update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 to fix CVE-2025-30204 (dependabot[bot])
* build: Update github.com/ebitengine/purego to work around bug in go1.24.3 (Nick Craig-Wood)
## v1.69.2 - 2025-05-01
[See commits](https://github.com/rclone/rclone/compare/v1.69.1...v1.69.2)
* Bug fixes
* accounting: Fix percentDiff calculation -- (Anagh Kumar Baranwal)
* build
* Update github.com/golang-jwt/jwt/v4 from 4.5.1 to 4.5.2 to fix CVE-2025-30204 (dependabot[bot])
* Update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 to fix CVE-2025-30204 (dependabot[bot])
* Update golang.org/x/crypto to v0.35.0 to fix CVE-2025-22869 (Nick Craig-Wood)
* Update golang.org/x/net from 0.36.0 to 0.38.0 to fix CVE-2025-22870 (dependabot[bot])
* Update golang.org/x/net to 0.36.0. to fix CVE-2025-22869 (dependabot[bot])
* Stop building with go < go1.23 as security updates forbade it (Nick Craig-Wood)
* Fix docker plugin build (Anagh Kumar Baranwal)
* cmd: Fix crash if rclone is invoked without any arguments (Janne Hellsten)
* config: Read configuration passwords from stdin even when terminated with EOF (Samantha Bowen)
* doc fixes (Andrew Kreimer, Danny Garside, eccoisle, Ed Craig-Wood, emyarod, jack, Jugal Kishore, Markus Gerstel, Michael Kebe, Nick Craig-Wood, simonmcnair, simwai, Zachary Vorhies)
* fs: Fix corruption of SizeSuffix with "B" suffix in config (eg --min-size) (Nick Craig-Wood)
* lib/http: Fix race between Serve() and Shutdown() (Nick Craig-Wood)
* object: Fix memory object out of bounds Seek (Nick Craig-Wood)
* operations: Fix call fmt.Errorf with wrong err (alingse)
* rc
* Disable the metrics server when running `rclone rc` (hiddenmarten)
* Fix debug/* commands not being available over unix sockets (Nick Craig-Wood)
* serve nfs: Fix unlikely crash (Nick Craig-Wood)
* stats: Fix the speed not getting updated after a pause in the processing (Anagh Kumar Baranwal)
* sync
* Fix cpu spinning when empty directory finding with leading slashes (Nick Craig-Wood)
* Copy dir modtimes even when copyEmptySrcDirs is false (ll3006)
* VFS
* Fix directory cache serving stale data (Lorenz Brun)
* Fix inefficient directory caching when directory reads are slow (huanghaojun)
* Fix integration test failures (Nick Craig-Wood)
* Drive
* Metadata: fix error when setting copy-requires-writer-permission on a folder (Nick Craig-Wood)
* Dropbox
* Retry link without expiry (Dave Vasilevsky)
* HTTP
* Correct root if definitely pointing to a file (nielash)
* Iclouddrive
* Fix so created files are writable (Ben Alex)
* Onedrive
* Fix metadata ordering in permissions (Nick Craig-Wood)
## v1.69.1 - 2025-02-14
[See commits](https://github.com/rclone/rclone/compare/v1.69.0...v1.69.1)
@@ -56704,7 +56765,7 @@ Options:
* fs: Make `--links` flag global and add new `--local-links` and `--vfs-links` flags (Nick Craig-Wood)
* http servers: Disable automatic authentication skipping for unix sockets in http servers (Moises Lima)
* This was making it impossible to use unix sockets with an proxy
* This might now cause rclone to need authenticaton where it didn't before
* This might now cause rclone to need authentication where it didn't before
* oauthutil: add support for OAuth client credential flow (Martin Hassack, Nick Craig-Wood)
* operations: make log messages consistent for mkdir/rmdir at INFO level (Nick Craig-Wood)
* rc: Add `relative` to [vfs/queue-set-expiry](https://rclone.org/rc/#vfs-queue-set-expiry) (Nick Craig-Wood)
@@ -57382,7 +57443,7 @@ instead of of `--size-only`, when `check` is not available.
* Update all dependencies (Nick Craig-Wood)
* Refactor version info and icon resource handling on windows (albertony)
* doc updates (albertony, alfish2000, asdffdsazqqq, Dimitri Papadopoulos, Herby Gillot, Joda Stößer, Manoj Ghosh, Nick Craig-Wood)
* Implement `--metadata-mapper` to transform metatadata with a user supplied program (Nick Craig-Wood)
* Implement `--metadata-mapper` to transform metadata with a user supplied program (Nick Craig-Wood)
* Add `ChunkWriterDoesntSeek` feature flag and set it for b2 (Nick Craig-Wood)
* lib/http: Export basic go string functions for use in `--template` (Gabriel Espinoza)
* makefile: Use POSIX compatible install arguments (Mina Galić)
@@ -57497,7 +57558,7 @@ instead of of `--size-only`, when `check` is not available.
* Fix "fatal error: concurrent map writes" (Nick Craig-Wood)
* B2
* Fix multipart upload: corrupted on transfer: sizes differ XXX vs 0 (Nick Craig-Wood)
* Fix locking window when getting mutipart upload URL (Nick Craig-Wood)
* Fix locking window when getting multipart upload URL (Nick Craig-Wood)
* Fix server side copies greater than 4GB (Nick Craig-Wood)
* Fix chunked streaming uploads (Nick Craig-Wood)
* Reduce default `--b2-upload-concurrency` to 4 to reduce memory usage (Nick Craig-Wood)
@@ -63404,7 +63465,6 @@ put them back in again.` >}}
* ben-ba <benjamin.brauner@gmx.de>
* Eli Orzitzer <e_orz@yahoo.com>
* Anthony Metzidis <anthony.metzidis@gmail.com>
* emyarod <afw5059@gmail.com>
* keongalvin <keongalvin@gmail.com>
* rarspace01 <rarspace01@users.noreply.github.com>
* Paul Stern <paulstern45@gmail.com>

175
MANUAL.txt generated
View File

@@ -1,6 +1,6 @@
rclone(1) User Manual
Nick Craig-Wood
Feb 14, 2025
May 21, 2025
NAME
@@ -2668,6 +2668,11 @@ Synopsis
Remote authorization. Used to authorize a remote or headless rclone from
a machine with a browser - use as instructed by rclone config.
The command requires 1-3 arguments: - fs name (e.g., "drive", "s3",
etc.) - Either a base64 encoded JSON blob obtained from a previous
rclone config session - Or a client_id and client_secret pair obtained
from the remote service
Use --auth-no-open-browser to prevent rclone to open auth link in
default browser automatically.
@@ -2675,7 +2680,7 @@ Use --template to generate HTML output via a custom Go template. If a
blank string is provided as an argument to this flag, the default
template is used.
rclone authorize [flags]
rclone authorize <fs name> [base64_json_blob | client_id client_secret] [flags]
Options
@@ -3904,6 +3909,9 @@ This doesn't transfer files that are identical on src and dst, testing
by size and modification time or MD5SUM. It doesn't delete files from
the destination.
If you are looking to copy just a byte range of a file, please see
'rclone cat --offset X --count Y'
Note: Use the -P/--progress flag to view real-time transfer statistics
rclone copyto source:path dest:path [flags]
@@ -4012,8 +4020,8 @@ Setting --auto-filename will attempt to automatically determine the
filename from the URL (after any redirections) and used in the
destination path.
With --auto-filename-header in addition, if a specific filename is set
in HTTP headers, it will be used instead of the name from the URL. With
With --header-filename in addition, if a specific filename is set in
HTTP headers, it will be used instead of the name from the URL. With
--print-filename in addition, the resulting file name will be printed.
Setting --no-clobber will prevent overwriting file on the destination if
@@ -5441,11 +5449,11 @@ and if they haven't been accessed for --vfs-write-back seconds. If
rclone is quit or dies with files that haven't been uploaded, these will
be uploaded next time rclone is run with the same flags.
If using --vfs-cache-max-size or --vfs-cache-min-free-size note that the
cache may exceed these quotas for two reasons. Firstly because it is
If using --vfs-cache-max-size or --vfs-cache-min-free-space note that
the cache may exceed these quotas for two reasons. Firstly because it is
only checked every --vfs-cache-poll-interval. Secondly because open
files cannot be evicted from the cache. When --vfs-cache-max-size or
--vfs-cache-min-free-size is exceeded, rclone will attempt to evict the
--vfs-cache-min-free-space is exceeded, rclone will attempt to evict the
least accessed files from the cache first. rclone will start with files
that haven't been accessed for the longest. This cache flushing strategy
is efficient and more relevant files are likely to remain cached.
@@ -6673,11 +6681,11 @@ and if they haven't been accessed for --vfs-write-back seconds. If
rclone is quit or dies with files that haven't been uploaded, these will
be uploaded next time rclone is run with the same flags.
If using --vfs-cache-max-size or --vfs-cache-min-free-size note that the
cache may exceed these quotas for two reasons. Firstly because it is
If using --vfs-cache-max-size or --vfs-cache-min-free-space note that
the cache may exceed these quotas for two reasons. Firstly because it is
only checked every --vfs-cache-poll-interval. Secondly because open
files cannot be evicted from the cache. When --vfs-cache-max-size or
--vfs-cache-min-free-size is exceeded, rclone will attempt to evict the
--vfs-cache-min-free-space is exceeded, rclone will attempt to evict the
least accessed files from the cache first. rclone will start with files
that haven't been accessed for the longest. This cache flushing strategy
is efficient and more relevant files are likely to remain cached.
@@ -7789,11 +7797,11 @@ and if they haven't been accessed for --vfs-write-back seconds. If
rclone is quit or dies with files that haven't been uploaded, these will
be uploaded next time rclone is run with the same flags.
If using --vfs-cache-max-size or --vfs-cache-min-free-size note that the
cache may exceed these quotas for two reasons. Firstly because it is
If using --vfs-cache-max-size or --vfs-cache-min-free-space note that
the cache may exceed these quotas for two reasons. Firstly because it is
only checked every --vfs-cache-poll-interval. Secondly because open
files cannot be evicted from the cache. When --vfs-cache-max-size or
--vfs-cache-min-free-size is exceeded, rclone will attempt to evict the
--vfs-cache-min-free-space is exceeded, rclone will attempt to evict the
least accessed files from the cache first. rclone will start with files
that haven't been accessed for the longest. This cache flushing strategy
is efficient and more relevant files are likely to remain cached.
@@ -8334,11 +8342,11 @@ and if they haven't been accessed for --vfs-write-back seconds. If
rclone is quit or dies with files that haven't been uploaded, these will
be uploaded next time rclone is run with the same flags.
If using --vfs-cache-max-size or --vfs-cache-min-free-size note that the
cache may exceed these quotas for two reasons. Firstly because it is
If using --vfs-cache-max-size or --vfs-cache-min-free-space note that
the cache may exceed these quotas for two reasons. Firstly because it is
only checked every --vfs-cache-poll-interval. Secondly because open
files cannot be evicted from the cache. When --vfs-cache-max-size or
--vfs-cache-min-free-size is exceeded, rclone will attempt to evict the
--vfs-cache-min-free-space is exceeded, rclone will attempt to evict the
least accessed files from the cache first. rclone will start with files
that haven't been accessed for the longest. This cache flushing strategy
is efficient and more relevant files are likely to remain cached.
@@ -8883,11 +8891,11 @@ and if they haven't been accessed for --vfs-write-back seconds. If
rclone is quit or dies with files that haven't been uploaded, these will
be uploaded next time rclone is run with the same flags.
If using --vfs-cache-max-size or --vfs-cache-min-free-size note that the
cache may exceed these quotas for two reasons. Firstly because it is
If using --vfs-cache-max-size or --vfs-cache-min-free-space note that
the cache may exceed these quotas for two reasons. Firstly because it is
only checked every --vfs-cache-poll-interval. Secondly because open
files cannot be evicted from the cache. When --vfs-cache-max-size or
--vfs-cache-min-free-size is exceeded, rclone will attempt to evict the
--vfs-cache-min-free-space is exceeded, rclone will attempt to evict the
least accessed files from the cache first. rclone will start with files
that haven't been accessed for the longest. This cache flushing strategy
is efficient and more relevant files are likely to remain cached.
@@ -9640,11 +9648,11 @@ and if they haven't been accessed for --vfs-write-back seconds. If
rclone is quit or dies with files that haven't been uploaded, these will
be uploaded next time rclone is run with the same flags.
If using --vfs-cache-max-size or --vfs-cache-min-free-size note that the
cache may exceed these quotas for two reasons. Firstly because it is
If using --vfs-cache-max-size or --vfs-cache-min-free-space note that
the cache may exceed these quotas for two reasons. Firstly because it is
only checked every --vfs-cache-poll-interval. Secondly because open
files cannot be evicted from the cache. When --vfs-cache-max-size or
--vfs-cache-min-free-size is exceeded, rclone will attempt to evict the
--vfs-cache-min-free-space is exceeded, rclone will attempt to evict the
least accessed files from the cache first. rclone will start with files
that haven't been accessed for the longest. This cache flushing strategy
is efficient and more relevant files are likely to remain cached.
@@ -10296,11 +10304,11 @@ and if they haven't been accessed for --vfs-write-back seconds. If
rclone is quit or dies with files that haven't been uploaded, these will
be uploaded next time rclone is run with the same flags.
If using --vfs-cache-max-size or --vfs-cache-min-free-size note that the
cache may exceed these quotas for two reasons. Firstly because it is
If using --vfs-cache-max-size or --vfs-cache-min-free-space note that
the cache may exceed these quotas for two reasons. Firstly because it is
only checked every --vfs-cache-poll-interval. Secondly because open
files cannot be evicted from the cache. When --vfs-cache-max-size or
--vfs-cache-min-free-size is exceeded, rclone will attempt to evict the
--vfs-cache-min-free-space is exceeded, rclone will attempt to evict the
least accessed files from the cache first. rclone will start with files
that haven't been accessed for the longest. This cache flushing strategy
is efficient and more relevant files are likely to remain cached.
@@ -11224,11 +11232,11 @@ and if they haven't been accessed for --vfs-write-back seconds. If
rclone is quit or dies with files that haven't been uploaded, these will
be uploaded next time rclone is run with the same flags.
If using --vfs-cache-max-size or --vfs-cache-min-free-size note that the
cache may exceed these quotas for two reasons. Firstly because it is
If using --vfs-cache-max-size or --vfs-cache-min-free-space note that
the cache may exceed these quotas for two reasons. Firstly because it is
only checked every --vfs-cache-poll-interval. Secondly because open
files cannot be evicted from the cache. When --vfs-cache-max-size or
--vfs-cache-min-free-size is exceeded, rclone will attempt to evict the
--vfs-cache-min-free-space is exceeded, rclone will attempt to evict the
least accessed files from the cache first. rclone will start with files
that haven't been accessed for the longest. This cache flushing strategy
is efficient and more relevant files are likely to remain cached.
@@ -11811,11 +11819,11 @@ and if they haven't been accessed for --vfs-write-back seconds. If
rclone is quit or dies with files that haven't been uploaded, these will
be uploaded next time rclone is run with the same flags.
If using --vfs-cache-max-size or --vfs-cache-min-free-size note that the
cache may exceed these quotas for two reasons. Firstly because it is
If using --vfs-cache-max-size or --vfs-cache-min-free-space note that
the cache may exceed these quotas for two reasons. Firstly because it is
only checked every --vfs-cache-poll-interval. Secondly because open
files cannot be evicted from the cache. When --vfs-cache-max-size or
--vfs-cache-min-free-size is exceeded, rclone will attempt to evict the
--vfs-cache-min-free-space is exceeded, rclone will attempt to evict the
least accessed files from the cache first. rclone will start with files
that haven't been accessed for the longest. This cache flushing strategy
is efficient and more relevant files are likely to remain cached.
@@ -12611,11 +12619,11 @@ and if they haven't been accessed for --vfs-write-back seconds. If
rclone is quit or dies with files that haven't been uploaded, these will
be uploaded next time rclone is run with the same flags.
If using --vfs-cache-max-size or --vfs-cache-min-free-size note that the
cache may exceed these quotas for two reasons. Firstly because it is
If using --vfs-cache-max-size or --vfs-cache-min-free-space note that
the cache may exceed these quotas for two reasons. Firstly because it is
only checked every --vfs-cache-poll-interval. Secondly because open
files cannot be evicted from the cache. When --vfs-cache-max-size or
--vfs-cache-min-free-size is exceeded, rclone will attempt to evict the
--vfs-cache-min-free-space is exceeded, rclone will attempt to evict the
least accessed files from the cache first. rclone will start with files
that haven't been accessed for the longest. This cache flushing strategy
is efficient and more relevant files are likely to remain cached.
@@ -20646,7 +20654,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
--user-agent string Set the user-agent to a specified string (default "rclone/v1.69.1")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.69.3")
Performance
@@ -21651,7 +21659,7 @@ Start from installing Docker on the host.
The FUSE driver is a prerequisite for rclone mounting and should be
installed on host:
sudo apt-get -y install fuse
sudo apt-get -y install fuse3
Create two directories required by rclone docker plugin:
@@ -25296,8 +25304,9 @@ glacier storage class you will see an error like below.
2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file
In this case you need to restore the object(s) in question before using
rclone.
In this case you need to restore the object(s) in question before
accessing object contents. The restore section below shows how to do
this with rclone.
Note that rclone only speaks the S3 API it does not speak the Glacier
Vault API, so rclone cannot directly access Glacier Vaults.
@@ -26728,7 +26737,7 @@ Access tier to the Frequent Access tier.
Usage Examples:
rclone backend restore s3:bucket/path/to/object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/ --include /object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY
@@ -30952,7 +30961,7 @@ Internet browser available.
Note that rclone runs a webserver on your local machine to collect the
token as returned from Box. This only runs from the moment it opens your
browser to the moment you get back the verification code. This is on
http://127.0.0.1:53682/ and this it may require you to unblock it
http://127.0.0.1:53682/ and this may require you to unblock it
temporarily if you are running a host firewall.
Once configured you can then use rclone like this,
@@ -52734,8 +52743,8 @@ Properties:
Limitations
On some SFTP servers (e.g. Synology) the paths are different for SSH and
SFTP so the hashes can't be calculated properly. For them using
disable_hashcheck is a good idea.
SFTP so the hashes can't be calculated properly. You can either use
--sftp-path-override or disable_hashcheck.
The only ssh agent supported under Windows is Putty's pageant.
@@ -56315,6 +56324,81 @@ Options:
Changelog
v1.69.3 - 2025-05-21
See commits
- Bug Fixes
- build: Reapply update github.com/golang-jwt/jwt/v5 from 5.2.1 to
5.2.2 to fix CVE-2025-30204 (dependabot[bot])
- build: Update github.com/ebitengine/purego to work around bug in
go1.24.3 (Nick Craig-Wood)
v1.69.2 - 2025-05-01
See commits
- Bug fixes
- accounting: Fix percentDiff calculation -- (Anagh Kumar
Baranwal)
- build
- Update github.com/golang-jwt/jwt/v4 from 4.5.1 to 4.5.2 to
fix CVE-2025-30204 (dependabot[bot])
- Update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 to
fix CVE-2025-30204 (dependabot[bot])
- Update golang.org/x/crypto to v0.35.0 to fix CVE-2025-22869
(Nick Craig-Wood)
- Update golang.org/x/net from 0.36.0 to 0.38.0 to fix
CVE-2025-22870 (dependabot[bot])
- Update golang.org/x/net to 0.36.0. to fix CVE-2025-22869
(dependabot[bot])
- Stop building with go < go1.23 as security updates forbade
it (Nick Craig-Wood)
- Fix docker plugin build (Anagh Kumar Baranwal)
- cmd: Fix crash if rclone is invoked without any arguments (Janne
Hellsten)
- config: Read configuration passwords from stdin even when
terminated with EOF (Samantha Bowen)
- doc fixes (Andrew Kreimer, Danny Garside, eccoisle, Ed
Craig-Wood, emyarod, jack, Jugal Kishore, Markus Gerstel,
Michael Kebe, Nick Craig-Wood, simonmcnair, simwai, Zachary
Vorhies)
- fs: Fix corruption of SizeSuffix with "B" suffix in config (eg
--min-size) (Nick Craig-Wood)
- lib/http: Fix race between Serve() and Shutdown() (Nick
Craig-Wood)
- object: Fix memory object out of bounds Seek (Nick Craig-Wood)
- operations: Fix call fmt.Errorf with wrong err (alingse)
- rc
- Disable the metrics server when running rclone rc
(hiddenmarten)
- Fix debug/* commands not being available over unix sockets
(Nick Craig-Wood)
- serve nfs: Fix unlikely crash (Nick Craig-Wood)
- stats: Fix the speed not getting updated after a pause in the
processing (Anagh Kumar Baranwal)
- sync
- Fix cpu spinning when empty directory finding with leading
slashes (Nick Craig-Wood)
- Copy dir modtimes even when copyEmptySrcDirs is false
(ll3006)
- VFS
- Fix directory cache serving stale data (Lorenz Brun)
- Fix inefficient directory caching when directory reads are slow
(huanghaojun)
- Fix integration test failures (Nick Craig-Wood)
- Drive
- Metadata: fix error when setting copy-requires-writer-permission
on a folder (Nick Craig-Wood)
- Dropbox
- Retry link without expiry (Dave Vasilevsky)
- HTTP
- Correct root if definitely pointing to a file (nielash)
- Iclouddrive
- Fix so created files are writable (Ben Alex)
- Onedrive
- Fix metadata ordering in permissions (Nick Craig-Wood)
v1.69.1 - 2025-02-14
See commits
@@ -56391,7 +56475,7 @@ See commits
sockets in http servers (Moises Lima)
- This was making it impossible to use unix sockets with an
proxy
- This might now cause rclone to need authenticaton where it
- This might now cause rclone to need authentication where it
didn't before
- oauthutil: add support for OAuth client credential flow (Martin
Hassack, Nick Craig-Wood)
@@ -57336,7 +57420,7 @@ See commits
- doc updates (albertony, alfish2000, asdffdsazqqq, Dimitri
Papadopoulos, Herby Gillot, Joda Stößer, Manoj Ghosh, Nick
Craig-Wood)
- Implement --metadata-mapper to transform metatadata with a user
- Implement --metadata-mapper to transform metadata with a user
supplied program (Nick Craig-Wood)
- Add ChunkWriterDoesntSeek feature flag and set it for b2 (Nick
Craig-Wood)
@@ -57498,7 +57582,7 @@ See commits
- B2
- Fix multipart upload: corrupted on transfer: sizes differ XXX vs
0 (Nick Craig-Wood)
- Fix locking window when getting mutipart upload URL (Nick
- Fix locking window when getting multipart upload URL (Nick
Craig-Wood)
- Fix server side copies greater than 4GB (Nick Craig-Wood)
- Fix chunked streaming uploads (Nick Craig-Wood)
@@ -64888,7 +64972,6 @@ email addresses removed from here need to be added to bin/.ignore-emails to make
- ben-ba benjamin.brauner@gmx.de
- Eli Orzitzer e_orz@yahoo.com
- Anthony Metzidis anthony.metzidis@gmail.com
- emyarod afw5059@gmail.com
- keongalvin keongalvin@gmail.com
- rarspace01 rarspace01@users.noreply.github.com
- Paul Stern paulstern45@gmail.com

View File

@@ -1,20 +1,4 @@
<div align="center">
<sup>Special thanks to our sponsor:</sup>
<br>
<br>
<a href="https://www.warp.dev/?utm_source=github&utm_medium=referral&utm_campaign=rclone_20231103">
<div>
<img src="https://rclone.org/img/logos/warp-github.svg" width="300" alt="Warp">
</div>
<b>Warp is a modern, Rust-based terminal with AI built in so you and your team can build great software, faster.</b>
<div>
<sup>Visit warp.dev to learn more.</sup>
</div>
</a>
<br>
<hr>
</div>
<br>
[<img src="https://rclone.org/img/logo_on_light__horizontal_color.svg" width="50%" alt="rclone logo">](https://rclone.org/#gh-light-mode-only)
[<img src="https://rclone.org/img/logo_on_dark__horizontal_color.svg" width="50%" alt="rclone logo">](https://rclone.org/#gh-dark-mode-only)

View File

@@ -124,8 +124,8 @@ Now
* git co ${BASE_TAG}-stable
* git cherry-pick any fixes
* Do the steps as above
* make startstable
* Do the steps as above
* git co master
* `#` cherry pick the changes to the changelog - check the diff to make sure it is correct
* git checkout ${BASE_TAG}-stable docs/content/changelog.md

View File

@@ -1 +1 @@
v1.69.1
v1.69.3

View File

@@ -1763,7 +1763,7 @@ func (f *Fs) createDir(ctx context.Context, pathID, leaf string, metadata fs.Met
}
var updateMetadata updateMetadataFn
if len(metadata) > 0 {
updateMetadata, err = f.updateMetadata(ctx, createInfo, metadata, true)
updateMetadata, err = f.updateMetadata(ctx, createInfo, metadata, true, true)
if err != nil {
return nil, fmt.Errorf("create dir: failed to update metadata: %w", err)
}
@@ -1794,7 +1794,7 @@ func (f *Fs) updateDir(ctx context.Context, dirID string, metadata fs.Metadata)
}
dirID = actualID(dirID)
updateInfo := &drive.File{}
updateMetadata, err := f.updateMetadata(ctx, updateInfo, metadata, true)
updateMetadata, err := f.updateMetadata(ctx, updateInfo, metadata, true, true)
if err != nil {
return nil, fmt.Errorf("update dir: failed to update metadata from source object: %w", err)
}

View File

@@ -508,7 +508,7 @@ type updateMetadataFn func(context.Context, *drive.File) error
//
// It returns a callback which should be called to finish the updates
// after the data is uploaded.
func (f *Fs) updateMetadata(ctx context.Context, updateInfo *drive.File, meta fs.Metadata, update bool) (callback updateMetadataFn, err error) {
func (f *Fs) updateMetadata(ctx context.Context, updateInfo *drive.File, meta fs.Metadata, update, isFolder bool) (callback updateMetadataFn, err error) {
callbackFns := []updateMetadataFn{}
callback = func(ctx context.Context, info *drive.File) error {
for _, fn := range callbackFns {
@@ -533,7 +533,9 @@ func (f *Fs) updateMetadata(ctx context.Context, updateInfo *drive.File, meta fs
}
switch k {
case "copy-requires-writer-permission":
if err := parseBool(&updateInfo.CopyRequiresWriterPermission); err != nil {
if isFolder {
fs.Debugf(f, "Ignoring %s=%s as can't set on folders", k, v)
} else if err := parseBool(&updateInfo.CopyRequiresWriterPermission); err != nil {
return nil, err
}
case "writers-can-share":
@@ -630,7 +632,7 @@ func (f *Fs) fetchAndUpdateMetadata(ctx context.Context, src fs.ObjectInfo, opti
if err != nil {
return nil, fmt.Errorf("failed to read metadata from source object: %w", err)
}
callback, err = f.updateMetadata(ctx, updateInfo, meta, update)
callback, err = f.updateMetadata(ctx, updateInfo, meta, update, false)
if err != nil {
return nil, fmt.Errorf("failed to update metadata from source object: %w", err)
}

View File

@@ -1174,6 +1174,16 @@ func (f *Fs) PublicLink(ctx context.Context, remote string, expire fs.Duration,
return shouldRetry(ctx, err)
})
if err != nil && createArg.Settings.Expires != nil && strings.Contains(err.Error(), sharing.SharedLinkSettingsErrorNotAuthorized) {
// Some plans can't create links with expiry
fs.Debugf(absPath, "can't create link with expiry, trying without")
createArg.Settings.Expires = nil
err = f.pacer.Call(func() (bool, error) {
linkRes, err = f.sharing.CreateSharedLinkWithSettings(&createArg)
return shouldRetry(ctx, err)
})
}
if err != nil && strings.Contains(err.Error(),
sharing.CreateSharedLinkWithSettingsErrorSharedLinkAlreadyExists) {
fs.Debugf(absPath, "has a public link already, attempting to retrieve it")

View File

@@ -180,7 +180,6 @@ func getFsEndpoint(ctx context.Context, client *http.Client, url string, opt *Op
}
addHeaders(req, opt)
res, err := noRedir.Do(req)
if err != nil {
fs.Debugf(nil, "Assuming path is a file as HEAD request could not be sent: %v", err)
return createFileResult()
@@ -249,6 +248,14 @@ func (f *Fs) httpConnection(ctx context.Context, opt *Options) (isFile bool, err
f.httpClient = client
f.endpoint = u
f.endpointURL = u.String()
if isFile {
// Correct root if definitely pointing to a file
f.root = path.Dir(f.root)
if f.root == "." || f.root == "/" {
f.root = ""
}
}
return isFile, nil
}

View File

@@ -631,7 +631,7 @@ func NewUpdateFileInfo() UpdateFileInfo {
FileFlags: FileFlags{
IsExecutable: true,
IsHidden: false,
IsWritable: false,
IsWritable: true,
},
}
}

View File

@@ -396,10 +396,57 @@ func (m *Metadata) WritePermissions(ctx context.Context) (err error) {
return nil
}
// Order the permissions so that any with users come first.
//
// This is to work around a quirk with Graph:
//
// 1. You are adding permissions for both a group and a user.
// 2. The user is a member of the group.
// 3. The permissions for the group and user are the same.
// 4. You are adding the group permission before the user permission.
//
// When all of the above are true, Graph indicates it has added the
// user permission, but it immediately drops it
//
// See: https://github.com/rclone/rclone/issues/8465
func (m *Metadata) orderPermissions(xs []*api.PermissionsType) {
// Return true if identity has any user permissions
hasUserIdentity := func(identity *api.IdentitySet) bool {
if identity == nil {
return false
}
return identity.User.ID != "" || identity.User.DisplayName != "" || identity.User.Email != "" || identity.User.LoginName != ""
}
// Return true if p has any user permissions
hasUser := func(p *api.PermissionsType) bool {
if hasUserIdentity(p.GetGrantedTo(m.fs.driveType)) {
return true
}
for _, identity := range p.GetGrantedToIdentities(m.fs.driveType) {
if hasUserIdentity(identity) {
return true
}
}
return false
}
// Put Permissions with a user first, leaving unsorted otherwise
slices.SortStableFunc(xs, func(a, b *api.PermissionsType) int {
aHasUser := hasUser(a)
bHasUser := hasUser(b)
if aHasUser && !bHasUser {
return -1
} else if !aHasUser && bHasUser {
return 1
}
return 0
})
}
// sortPermissions sorts the permissions (to be written) into add, update, and remove queues
func (m *Metadata) sortPermissions() (add, update, remove []*api.PermissionsType) {
new, old := m.queuedPermissions, m.permissions
if len(old) == 0 || m.permsAddOnly {
m.orderPermissions(new)
return new, nil, nil // they must all be "add"
}
@@ -447,6 +494,9 @@ func (m *Metadata) sortPermissions() (add, update, remove []*api.PermissionsType
remove = append(remove, o)
}
}
m.orderPermissions(add)
m.orderPermissions(update)
m.orderPermissions(remove)
return add, update, remove
}

View File

@@ -0,0 +1,125 @@
package onedrive
import (
"encoding/json"
"testing"
"github.com/rclone/rclone/backend/onedrive/api"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestOrderPermissions(t *testing.T) {
tests := []struct {
name string
input []*api.PermissionsType
expected []string
}{
{
name: "empty",
input: []*api.PermissionsType{},
expected: []string(nil),
},
{
name: "users first, then group, then none",
input: []*api.PermissionsType{
{ID: "1", GrantedTo: &api.IdentitySet{Group: api.Identity{DisplayName: "Group1"}}},
{ID: "2", GrantedToIdentities: []*api.IdentitySet{{User: api.Identity{DisplayName: "Alice"}}}},
{ID: "3", GrantedTo: &api.IdentitySet{User: api.Identity{DisplayName: "Alice"}}},
{ID: "4"},
},
expected: []string{"2", "3", "1", "4"},
},
{
name: "same type unsorted",
input: []*api.PermissionsType{
{ID: "b", GrantedTo: &api.IdentitySet{Group: api.Identity{DisplayName: "Group B"}}},
{ID: "a", GrantedTo: &api.IdentitySet{Group: api.Identity{DisplayName: "Group A"}}},
{ID: "c", GrantedToIdentities: []*api.IdentitySet{{Group: api.Identity{DisplayName: "Group A"}}, {User: api.Identity{DisplayName: "Alice"}}}},
},
expected: []string{"c", "b", "a"},
},
{
name: "all user identities",
input: []*api.PermissionsType{
{ID: "c", GrantedTo: &api.IdentitySet{User: api.Identity{DisplayName: "Bob"}}},
{ID: "a", GrantedTo: &api.IdentitySet{User: api.Identity{Email: "alice@example.com"}}},
{ID: "b", GrantedToIdentities: []*api.IdentitySet{{User: api.Identity{LoginName: "user3"}}}},
},
expected: []string{"c", "a", "b"},
},
{
name: "no user or group info",
input: []*api.PermissionsType{
{ID: "z"},
{ID: "x"},
{ID: "y"},
},
expected: []string{"z", "x", "y"},
},
}
for _, driveType := range []string{driveTypePersonal, driveTypeBusiness} {
t.Run(driveType, func(t *testing.T) {
for _, tt := range tests {
m := &Metadata{fs: &Fs{driveType: driveType}}
t.Run(tt.name, func(t *testing.T) {
if driveType == driveTypeBusiness {
for i := range tt.input {
tt.input[i].GrantedToV2 = tt.input[i].GrantedTo
tt.input[i].GrantedTo = nil
tt.input[i].GrantedToIdentitiesV2 = tt.input[i].GrantedToIdentities
tt.input[i].GrantedToIdentities = nil
}
}
m.orderPermissions(tt.input)
var gotIDs []string
for _, p := range tt.input {
gotIDs = append(gotIDs, p.ID)
}
assert.Equal(t, tt.expected, gotIDs)
})
}
})
}
}
func TestOrderPermissionsJSON(t *testing.T) {
testJSON := `[
{
"id": "1",
"grantedToV2": {
"group": {
"id": "group@example.com"
}
},
"roles": [
"write"
]
},
{
"id": "2",
"grantedToV2": {
"user": {
"id": "user@example.com"
}
},
"roles": [
"write"
]
}
]`
var testPerms []*api.PermissionsType
err := json.Unmarshal([]byte(testJSON), &testPerms)
require.NoError(t, err)
m := &Metadata{fs: &Fs{driveType: driveTypeBusiness}}
m.orderPermissions(testPerms)
var gotIDs []string
for _, p := range testPerms {
gotIDs = append(gotIDs, p.ID)
}
assert.Equal(t, []string{"2", "1"}, gotIDs)
}

View File

@@ -4976,7 +4976,7 @@ or from INTELLIGENT-TIERING Archive Access / Deep Archive Access tier to the Fre
Usage Examples:
rclone backend restore s3:bucket/path/to/object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/ --include /object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY

View File

@@ -11,4 +11,5 @@
<services+github@simjo.st>
<seb•ɑƬ•chezwam•ɖɵʈ•org>
<allllaboutyou@gmail.com>
<psycho@feltzv.fr>
<psycho@feltzv.fr>
<afw5059@gmail.com>

View File

@@ -23,19 +23,23 @@ func init() {
}
var commandDefinition = &cobra.Command{
Use: "authorize",
Use: "authorize <fs name> [base64_json_blob | client_id client_secret]",
Short: `Remote authorization.`,
Long: `Remote authorization. Used to authorize a remote or headless
rclone from a machine with a browser - use as instructed by
rclone config.
The command requires 1-3 arguments:
- fs name (e.g., "drive", "s3", etc.)
- Either a base64 encoded JSON blob obtained from a previous rclone config session
- Or a client_id and client_secret pair obtained from the remote service
Use --auth-no-open-browser to prevent rclone to open auth
link in default browser automatically.
Use --template to generate HTML output via a custom Go template. If a blank string is provided as an argument to this flag, the default template is used.`,
Annotations: map[string]string{
"versionIntroduced": "v1.27",
// "groups": "",
},
RunE: func(command *cobra.Command, args []string) error {
cmd.CheckArgs(1, 3, command, args)

View File

@@ -0,0 +1,32 @@
package authorize
import (
"bytes"
"strings"
"testing"
"github.com/spf13/cobra"
)
func TestAuthorizeCommand(t *testing.T) {
// Test that the Use string is correctly formatted
if commandDefinition.Use != "authorize <fs name> [base64_json_blob | client_id client_secret]" {
t.Errorf("Command Use string doesn't match expected format: %s", commandDefinition.Use)
}
// Test that help output contains the argument information
buf := &bytes.Buffer{}
cmd := &cobra.Command{}
cmd.AddCommand(commandDefinition)
cmd.SetOut(buf)
cmd.SetArgs([]string{"authorize", "--help"})
err := cmd.Execute()
if err != nil {
t.Fatalf("Failed to execute help command: %v", err)
}
helpOutput := buf.String()
if !strings.Contains(helpOutput, "authorize <fs name>") {
t.Errorf("Help output doesn't contain correct usage information")
}
}

View File

@@ -429,11 +429,12 @@ func initConfig() {
fs.Fatalf(nil, "Failed to start remote control: %v", err)
}
// Start the metrics server if configured
_, err = rcserver.MetricsStart(ctx, &rc.Opt)
if err != nil {
fs.Fatalf(nil, "Failed to start metrics server: %v", err)
// Start the metrics server if configured and not running the "rc" command
if len(os.Args) >= 2 && os.Args[1] != "rc" {
_, err = rcserver.MetricsStart(ctx, &rc.Opt)
if err != nil {
fs.Fatalf(nil, "Failed to start metrics server: %v", err)
}
}
// Setup CPU profiling if desired

View File

@@ -43,6 +43,8 @@ This doesn't transfer files that are identical on src and dst, testing
by size and modification time or MD5SUM. It doesn't delete files from
the destination.
*If you are looking to copy just a byte range of a file, please see 'rclone cat --offset X --count Y'*
**Note**: Use the ` + "`-P`" + `/` + "`--progress`" + ` flag to view real-time transfer statistics
`,
Annotations: map[string]string{

View File

@@ -43,7 +43,7 @@ Setting |--auto-filename| will attempt to automatically determine the
filename from the URL (after any redirections) and used in the
destination path.
With |--auto-filename-header| in addition, if a specific filename is
With |--header-filename| in addition, if a specific filename is
set in HTTP headers, it will be used instead of the name from the URL.
With |--print-filename| in addition, the resulting file name will be
printed.

View File

@@ -29,6 +29,7 @@ func setSys(fi os.FileInfo) {
node, ok := fi.(vfs.Node)
if !ok {
fs.Errorf(fi, "internal error: %T is not a vfs.Node", fi)
return
}
vfs := node.VFS()
// Set the UID and GID for the node passed in from the VFS defaults.

View File

@@ -14,7 +14,7 @@ docs](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html)).
access.
Please note that some clients may require HTTPS endpoints. See [the
SSL docs](#ssl-tls) for more information.
SSL docs](#tls-ssl) for more information.
This command uses the [VFS directory cache](#vfs-virtual-file-system).
All the functionality will work with `--vfs-cache-mode off`. Using

View File

@@ -809,7 +809,6 @@ put them back in again.` >}}
* ben-ba <benjamin.brauner@gmx.de>
* Eli Orzitzer <e_orz@yahoo.com>
* Anthony Metzidis <anthony.metzidis@gmail.com>
* emyarod <afw5059@gmail.com>
* keongalvin <keongalvin@gmail.com>
* rarspace01 <rarspace01@users.noreply.github.com>
* Paul Stern <paulstern45@gmail.com>

View File

@@ -87,7 +87,7 @@ machine with no Internet browser available.
Note that rclone runs a webserver on your local machine to collect the
token as returned from Box. This only runs from the moment it opens
your browser to the moment you get back the verification code. This
is on `http://127.0.0.1:53682/` and this it may require you to unblock
is on `http://127.0.0.1:53682/` and this may require you to unblock
it temporarily if you are running a host firewall.
Once configured you can then use `rclone` like this,

View File

@@ -5,6 +5,58 @@ description: "Rclone Changelog"
# Changelog
## v1.69.3 - 2025-05-21
[See commits](https://github.com/rclone/rclone/compare/v1.69.2...v1.69.3)
* Bug Fixes
* build: Reapply update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 to fix CVE-2025-30204 (dependabot[bot])
* build: Update github.com/ebitengine/purego to work around bug in go1.24.3 (Nick Craig-Wood)
## v1.69.2 - 2025-05-01
[See commits](https://github.com/rclone/rclone/compare/v1.69.1...v1.69.2)
* Bug fixes
* accounting: Fix percentDiff calculation -- (Anagh Kumar Baranwal)
* build
* Update github.com/golang-jwt/jwt/v4 from 4.5.1 to 4.5.2 to fix CVE-2025-30204 (dependabot[bot])
* Update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 to fix CVE-2025-30204 (dependabot[bot])
* Update golang.org/x/crypto to v0.35.0 to fix CVE-2025-22869 (Nick Craig-Wood)
* Update golang.org/x/net from 0.36.0 to 0.38.0 to fix CVE-2025-22870 (dependabot[bot])
* Update golang.org/x/net to 0.36.0. to fix CVE-2025-22869 (dependabot[bot])
* Stop building with go < go1.23 as security updates forbade it (Nick Craig-Wood)
* Fix docker plugin build (Anagh Kumar Baranwal)
* cmd: Fix crash if rclone is invoked without any arguments (Janne Hellsten)
* config: Read configuration passwords from stdin even when terminated with EOF (Samantha Bowen)
* doc fixes (Andrew Kreimer, Danny Garside, eccoisle, Ed Craig-Wood, emyarod, jack, Jugal Kishore, Markus Gerstel, Michael Kebe, Nick Craig-Wood, simonmcnair, simwai, Zachary Vorhies)
* fs: Fix corruption of SizeSuffix with "B" suffix in config (eg --min-size) (Nick Craig-Wood)
* lib/http: Fix race between Serve() and Shutdown() (Nick Craig-Wood)
* object: Fix memory object out of bounds Seek (Nick Craig-Wood)
* operations: Fix call fmt.Errorf with wrong err (alingse)
* rc
* Disable the metrics server when running `rclone rc` (hiddenmarten)
* Fix debug/* commands not being available over unix sockets (Nick Craig-Wood)
* serve nfs: Fix unlikely crash (Nick Craig-Wood)
* stats: Fix the speed not getting updated after a pause in the processing (Anagh Kumar Baranwal)
* sync
* Fix cpu spinning when empty directory finding with leading slashes (Nick Craig-Wood)
* Copy dir modtimes even when copyEmptySrcDirs is false (ll3006)
* VFS
* Fix directory cache serving stale data (Lorenz Brun)
* Fix inefficient directory caching when directory reads are slow (huanghaojun)
* Fix integration test failures (Nick Craig-Wood)
* Drive
* Metadata: fix error when setting copy-requires-writer-permission on a folder (Nick Craig-Wood)
* Dropbox
* Retry link without expiry (Dave Vasilevsky)
* HTTP
* Correct root if definitely pointing to a file (nielash)
* Iclouddrive
* Fix so created files are writable (Ben Alex)
* Onedrive
* Fix metadata ordering in permissions (Nick Craig-Wood)
## v1.69.1 - 2025-02-14
[See commits](https://github.com/rclone/rclone/compare/v1.69.0...v1.69.1)
@@ -60,7 +112,7 @@ description: "Rclone Changelog"
* fs: Make `--links` flag global and add new `--local-links` and `--vfs-links` flags (Nick Craig-Wood)
* http servers: Disable automatic authentication skipping for unix sockets in http servers (Moises Lima)
* This was making it impossible to use unix sockets with an proxy
* This might now cause rclone to need authenticaton where it didn't before
* This might now cause rclone to need authentication where it didn't before
* oauthutil: add support for OAuth client credential flow (Martin Hassack, Nick Craig-Wood)
* operations: make log messages consistent for mkdir/rmdir at INFO level (Nick Craig-Wood)
* rc: Add `relative` to [vfs/queue-set-expiry](/rc/#vfs-queue-set-expiry) (Nick Craig-Wood)
@@ -738,7 +790,7 @@ instead of of `--size-only`, when `check` is not available.
* Update all dependencies (Nick Craig-Wood)
* Refactor version info and icon resource handling on windows (albertony)
* doc updates (albertony, alfish2000, asdffdsazqqq, Dimitri Papadopoulos, Herby Gillot, Joda Stößer, Manoj Ghosh, Nick Craig-Wood)
* Implement `--metadata-mapper` to transform metatadata with a user supplied program (Nick Craig-Wood)
* Implement `--metadata-mapper` to transform metadata with a user supplied program (Nick Craig-Wood)
* Add `ChunkWriterDoesntSeek` feature flag and set it for b2 (Nick Craig-Wood)
* lib/http: Export basic go string functions for use in `--template` (Gabriel Espinoza)
* makefile: Use POSIX compatible install arguments (Mina Galić)
@@ -853,7 +905,7 @@ instead of of `--size-only`, when `check` is not available.
* Fix "fatal error: concurrent map writes" (Nick Craig-Wood)
* B2
* Fix multipart upload: corrupted on transfer: sizes differ XXX vs 0 (Nick Craig-Wood)
* Fix locking window when getting mutipart upload URL (Nick Craig-Wood)
* Fix locking window when getting multipart upload URL (Nick Craig-Wood)
* Fix server side copies greater than 4GB (Nick Craig-Wood)
* Fix chunked streaming uploads (Nick Craig-Wood)
* Reduce default `--b2-upload-concurrency` to 4 to reduce memory usage (Nick Craig-Wood)

View File

@@ -965,7 +965,7 @@ rclone [flags]
--use-json-log Use json log format
--use-mmap Use mmap allocator (see docs)
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string (default "rclone/v1.69.1")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.69.3")
-v, --verbose count Print lots more stuff (repeat for more)
-V, --version Print the version number
--webdav-auth-redirect Preserve authentication on redirect

View File

@@ -14,13 +14,18 @@ Remote authorization. Used to authorize a remote or headless
rclone from a machine with a browser - use as instructed by
rclone config.
The command requires 1-3 arguments:
- fs name (e.g., "drive", "s3", etc.)
- Either a base64 encoded JSON blob obtained from a previous rclone config session
- Or a client_id and client_secret pair obtained from the remote service
Use --auth-no-open-browser to prevent rclone to open auth
link in default browser automatically.
Use --template to generate HTML output via a custom Go template. If a blank string is provided as an argument to this flag, the default template is used.
```
rclone authorize [flags]
rclone authorize <fs name> [base64_json_blob | client_id client_secret] [flags]
```
## Options

View File

@@ -36,6 +36,8 @@ This doesn't transfer files that are identical on src and dst, testing
by size and modification time or MD5SUM. It doesn't delete files from
the destination.
*If you are looking to copy just a byte range of a file, please see 'rclone cat --offset X --count Y'*
**Note**: Use the `-P`/`--progress` flag to view real-time transfer statistics

View File

@@ -17,7 +17,7 @@ Setting `--auto-filename` will attempt to automatically determine the
filename from the URL (after any redirections) and used in the
destination path.
With `--auto-filename-header` in addition, if a specific filename is
With `--header-filename` in addition, if a specific filename is
set in HTTP headers, it will be used instead of the name from the URL.
With `--print-filename` in addition, the resulting file name will be
printed.

View File

@@ -571,11 +571,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant

View File

@@ -572,11 +572,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant

View File

@@ -134,11 +134,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant

View File

@@ -146,11 +146,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant

View File

@@ -127,11 +127,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant

View File

@@ -245,11 +245,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant

View File

@@ -176,11 +176,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant

View File

@@ -27,7 +27,7 @@ docs](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html)).
access.
Please note that some clients may require HTTPS endpoints. See [the
SSL docs](#ssl-tls) for more information.
SSL docs](#tls-ssl) for more information.
This command uses the [VFS directory cache](#vfs-virtual-file-system).
All the functionality will work with `--vfs-cache-mode off`. Using
@@ -334,11 +334,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant

View File

@@ -170,11 +170,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant

View File

@@ -288,11 +288,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant

View File

@@ -45,7 +45,7 @@ on the host.
The _FUSE_ driver is a prerequisite for rclone mounting and should be
installed on host:
```
sudo apt-get -y install fuse
sudo apt-get -y install fuse3
```
Create two directories required by rclone docker plugin:

View File

@@ -116,7 +116,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
--user-agent string Set the user-agent to a specified string (default "rclone/v1.69.1")
--user-agent string Set the user-agent to a specified string (default "rclone/v1.69.3")
```

View File

@@ -771,7 +771,8 @@ tries to access data from the glacier storage class you will see an error like b
2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file
In this case you need to [restore](http://docs.aws.amazon.com/AmazonS3/latest/user-guide/restore-archived-objects.html)
the object(s) in question before using rclone.
the object(s) in question before accessing object contents.
The [restore](#restore) section below shows how to do this with rclone.
Note that rclone only speaks the S3 API it does not speak the Glacier
Vault API, so rclone cannot directly access Glacier Vaults.
@@ -2174,7 +2175,7 @@ or from INTELLIGENT-TIERING Archive Access / Deep Archive Access tier to the Fre
Usage Examples:
rclone backend restore s3:bucket/path/to/object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/ --include /object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY

View File

@@ -1108,7 +1108,8 @@ Properties:
On some SFTP servers (e.g. Synology) the paths are different
for SSH and SFTP so the hashes can't be calculated properly.
For them using `disable_hashcheck` is a good idea.
You can either use [`--sftp-path-override`](#--sftp-path-override)
or [`disable_hashcheck`](#--sftp-disable-hashcheck).
The only ssh agent supported under Windows is Putty's pageant.

View File

@@ -28,6 +28,14 @@
<a href="https://rcloneview.com/?utm_source=rclone" target="_blank" rel="noopener" title="Visit rclone's sponsor RcloneView"><img src="/img/logos/rcloneview-banner.svg"></a><br />
</div>
</div>
<div class="card">
<div class="card-header" style="padding: 5px 15px;">
Silver Sponsor
</div>
<div class="card-body">
<a href="https://rcloneui.com" target="_blank" rel="noopener" title="Visit rclone's sponsor rclone UI"><img src="/img/logos/rcloneui.svg"></a><br />
</div>
</div>
{{end}}
<div class="card">

View File

@@ -1 +1 @@
v1.69.1
v1.69.3

View File

@@ -65,28 +65,29 @@ type StatsInfo struct {
}
type averageValues struct {
mu sync.Mutex
lpBytes int64
lpTime time.Time
speed float64
stop chan bool
stopped sync.WaitGroup
startOnce sync.Once
stopOnce sync.Once
mu sync.Mutex
lpBytes int64
lpTime time.Time
speed float64
stop chan bool
stopped sync.WaitGroup
started bool
}
// NewStats creates an initialised StatsInfo
func NewStats(ctx context.Context) *StatsInfo {
ci := fs.GetConfig(ctx)
return &StatsInfo{
s := &StatsInfo{
ctx: ctx,
ci: ci,
checking: newTransferMap(ci.Checkers, "checking"),
transferring: newTransferMap(ci.Transfers, "transferring"),
inProgress: newInProgress(ctx),
startTime: time.Now(),
average: averageValues{stop: make(chan bool)},
average: averageValues{},
}
s.startAverageLoop()
return s
}
// RemoteStats returns stats for rc
@@ -328,61 +329,96 @@ func (s *StatsInfo) averageLoop() {
ticker := time.NewTicker(averagePeriodLength)
defer ticker.Stop()
startTime := time.Now()
a := &s.average
defer a.stopped.Done()
shouldRun := false
for {
select {
case now := <-ticker.C:
a.mu.Lock()
var elapsed float64
if a.lpTime.IsZero() {
elapsed = now.Sub(startTime).Seconds()
} else {
elapsed = now.Sub(a.lpTime).Seconds()
if !shouldRun {
a.mu.Unlock()
continue
}
avg := 0.0
elapsed := now.Sub(a.lpTime).Seconds()
if elapsed > 0 {
avg = float64(a.lpBytes) / elapsed
}
if period < averagePeriod {
period++
}
a.speed = (avg + a.speed*(period-1)) / period
a.lpBytes = 0
a.lpTime = now
a.mu.Unlock()
case stop, ok := <-a.stop:
if !ok {
return // Channel closed, exit the loop
}
a.mu.Lock()
// If we are resuming, store the current time
if !shouldRun && !stop {
a.lpTime = time.Now()
}
shouldRun = !stop
a.mu.Unlock()
case <-a.stop:
return
}
}
}
// Resume the average loop
func (s *StatsInfo) resumeAverageLoop() {
s.mu.Lock()
defer s.mu.Unlock()
s.average.stop <- false
}
// Pause the average loop
func (s *StatsInfo) pauseAverageLoop() {
s.mu.Lock()
defer s.mu.Unlock()
s.average.stop <- true
}
// Start the average loop
//
// Call with the mutex held
func (s *StatsInfo) _startAverageLoop() {
if !s.average.started {
s.average.stop = make(chan bool)
s.average.started = true
s.average.stopped.Add(1)
go s.averageLoop()
}
}
// Start the average loop
func (s *StatsInfo) startAverageLoop() {
s.mu.Lock()
defer s.mu.Unlock()
s.average.startOnce.Do(func() {
s.average.stopped.Add(1)
go s.averageLoop()
})
s._startAverageLoop()
}
// Stop the average loop
//
// Call with the mutex held
func (s *StatsInfo) _stopAverageLoop() {
s.average.stopOnce.Do(func() {
if s.average.started {
close(s.average.stop)
s.average.stopped.Wait()
})
}
// Stop the average loop
func (s *StatsInfo) stopAverageLoop() {
s.mu.Lock()
defer s.mu.Unlock()
s._stopAverageLoop()
}
}
// String convert the StatsInfo to a string for printing
@@ -564,9 +600,9 @@ func (s *StatsInfo) GetBytesWithPending() int64 {
pending := int64(0)
for _, tr := range s.startedTransfers {
if tr.acc != nil {
bytes, size := tr.acc.progress()
if bytes < size {
pending += size - bytes
bytesRead, size := tr.acc.progress()
if bytesRead < size {
pending += size - bytesRead
}
}
}
@@ -699,7 +735,8 @@ func (s *StatsInfo) ResetCounters() {
s.oldDuration = 0
s._stopAverageLoop()
s.average = averageValues{stop: make(chan bool)}
s.average = averageValues{}
s._startAverageLoop()
}
// ResetErrors sets the errors count to 0 and resets lastError, fatalError and retryError
@@ -788,7 +825,7 @@ func (s *StatsInfo) NewTransfer(obj fs.DirEntry, dstFs fs.Fs) *Transfer {
}
tr := newTransfer(s, obj, srcFs, dstFs)
s.transferring.add(tr)
s.startAverageLoop()
s.resumeAverageLoop()
return tr
}
@@ -796,7 +833,7 @@ func (s *StatsInfo) NewTransfer(obj fs.DirEntry, dstFs fs.Fs) *Transfer {
func (s *StatsInfo) NewTransferRemoteSize(remote string, size int64, srcFs, dstFs fs.Fs) *Transfer {
tr := newTransferRemoteSize(s, remote, size, false, "", srcFs, dstFs)
s.transferring.add(tr)
s.startAverageLoop()
s.resumeAverageLoop()
return tr
}
@@ -811,7 +848,7 @@ func (s *StatsInfo) DoneTransferring(remote string, ok bool) {
s.mu.Unlock()
}
if s.transferring.empty() && s.checking.empty() {
time.AfterFunc(averageStopAfter, s.stopAverageLoop)
s.pauseAverageLoop()
}
}

View File

@@ -237,5 +237,14 @@ func TestCountError(t *testing.T) {
}
func percentDiff(start, end uint64) uint64 {
return (start - end) * 100 / start
if start == 0 {
return 0 // Handle zero start value to avoid division by zero
}
var diff uint64
if end > start {
diff = end - start // Handle case where end is larger than start
} else {
diff = start - end
}
return (diff * 100) / start
}

View File

@@ -7,6 +7,7 @@ import (
"context"
"errors"
"fmt"
"io"
"os"
"sort"
"strconv"
@@ -27,7 +28,7 @@ import (
var ReadLine = func() string {
buf := bufio.NewReader(os.Stdin)
line, err := buf.ReadString('\n')
if err != nil {
if err != nil && (line == "" || err != io.EOF) {
fs.Fatalf(nil, "Failed to read line: %v", err)
}
return strings.TrimSpace(line)

View File

@@ -261,6 +261,34 @@ func TestOptionString(t *testing.T) {
assert.Equal(t, "true", caseInsensitiveOption.String())
}
func TestOptionStringStringArray(t *testing.T) {
opt := Option{
Name: "string_array",
Default: []string(nil),
}
assert.Equal(t, "", opt.String())
opt.Default = []string{}
assert.Equal(t, "", opt.String())
opt.Default = []string{"a", "b"}
assert.Equal(t, "a,b", opt.String())
opt.Default = []string{"hello, world!", "goodbye, world!"}
assert.Equal(t, `"hello, world!","goodbye, world!"`, opt.String())
}
func TestOptionStringSizeSuffix(t *testing.T) {
opt := Option{
Name: "size_suffix",
Default: SizeSuffix(0),
}
assert.Equal(t, "0", opt.String())
opt.Default = SizeSuffix(-1)
assert.Equal(t, "off", opt.String())
opt.Default = SizeSuffix(100)
assert.Equal(t, "100B", opt.String())
opt.Default = SizeSuffix(1024)
assert.Equal(t, "1Ki", opt.String())
}
func TestOptionSet(t *testing.T) {
o := caseInsensitiveOption
assert.Equal(t, true, o.Value)

View File

@@ -279,19 +279,26 @@ func (o *MemoryObject) SetModTime(ctx context.Context, modTime time.Time) error
// Open opens the file for read. Call Close() on the returned io.ReadCloser
func (o *MemoryObject) Open(ctx context.Context, options ...fs.OpenOption) (io.ReadCloser, error) {
content := o.content
var offset, limit int64 = 0, -1
for _, option := range options {
switch x := option.(type) {
case *fs.RangeOption:
content = o.content[x.Start:x.End]
offset, limit = x.Decode(o.Size())
case *fs.SeekOption:
content = o.content[x.Offset:]
offset = x.Offset
default:
if option.Mandatory() {
fs.Logf(o, "Unsupported mandatory option: %v", option)
}
}
}
content := o.content
offset = max(offset, 0)
if limit < 0 {
content = content[offset:]
} else {
content = content[offset:min(offset+limit, int64(len(content)))]
}
return io.NopCloser(bytes.NewBuffer(content)), nil
}

View File

@@ -110,6 +110,7 @@ func TestMemoryObject(t *testing.T) {
assert.Equal(t, newNow, o.ModTime(context.Background()))
checkOpen := func(rc io.ReadCloser, expected string) {
t.Helper()
actual, err := io.ReadAll(rc)
assert.NoError(t, err)
err = rc.Close()
@@ -118,6 +119,7 @@ func TestMemoryObject(t *testing.T) {
}
checkContent := func(o fs.Object, expected string) {
t.Helper()
rc, err := o.Open(context.Background())
assert.NoError(t, err)
checkOpen(rc, expected)
@@ -127,12 +129,28 @@ func TestMemoryObject(t *testing.T) {
rc, err := o.Open(context.Background(), &fs.RangeOption{Start: 1, End: 3})
assert.NoError(t, err)
checkOpen(rc, "ot")
checkOpen(rc, "ota")
rc, err = o.Open(context.Background(), &fs.RangeOption{Start: 1, End: -1})
assert.NoError(t, err)
checkOpen(rc, "otato")
rc, err = o.Open(context.Background(), &fs.RangeOption{Start: 1, End: 4096})
assert.NoError(t, err)
checkOpen(rc, "otato")
rc, err = o.Open(context.Background(), &fs.RangeOption{Start: -1, End: 4})
assert.NoError(t, err)
checkOpen(rc, "tato")
rc, err = o.Open(context.Background(), &fs.SeekOption{Offset: 3})
assert.NoError(t, err)
checkOpen(rc, "ato")
rc, err = o.Open(context.Background(), &fs.SeekOption{Offset: -100})
assert.NoError(t, err)
checkOpen(rc, "potato")
// check it fits within the buffer
newNow = now.Add(2 * time.Minute)
newContent := bytes.NewBufferString("Rutabaga")

View File

@@ -315,8 +315,8 @@ func (w *writerAtChunkWriter) WriteChunk(ctx context.Context, chunkNumber int, r
// if we were buffering, flush to disk
switch w := writer.(type) {
case *bufio.Writer:
er2 := w.Flush()
if er2 != nil {
err = w.Flush()
if err != nil {
return -1, fmt.Errorf("multi-thread copy: flush failed: %w", err)
}
}

View File

@@ -123,7 +123,7 @@ func newServer(ctx context.Context, opt *rc.Options, mux *http.ServeMux) (*Serve
)
// Add the debug handler which is installed in the default mux
router.Handle("/debug/*", mux)
router.Handle("/debug/pprof/*", mux)
// FIXME split these up into individual functions
router.Get("/*", s.handler)

View File

@@ -258,15 +258,27 @@ func (o *Option) IsDefault() bool {
// String turns Option into a string
func (o *Option) String() string {
v := o.GetValue()
if stringArray, isStringArray := v.([]string); isStringArray {
switch x := v.(type) {
case []string:
// Treat empty string array as empty string
// This is to make the default value of the option help nice
if len(stringArray) == 0 {
if len(x) == 0 {
return ""
}
// Encode string arrays as CSV
// The default Go encoding can't be decoded uniquely
return CommaSepList(stringArray).String()
return CommaSepList(x).String()
case SizeSuffix:
str := x.String()
// Suffix bare numbers with "B" unless they are 0
//
// This makes sure that fs.SizeSuffix roundtrips through string
if len(str) > 0 && str != "0" {
if lastDigit := str[len(str)-1]; lastDigit >= '0' && lastDigit <= '9' {
str += "B"
}
}
return str
}
return fmt.Sprint(v)
}

View File

@@ -726,7 +726,7 @@ func (s *syncCopyMove) markParentNotEmpty(entry fs.DirEntry) {
parentDir = ""
}
delete(s.srcEmptyDirs, parentDir)
if parentDir == "" {
if parentDir == "" || parentDir == "/" {
break
}
parentDir = path.Dir(parentDir)
@@ -1109,23 +1109,17 @@ func (s *syncCopyMove) copyDirMetadata(ctx context.Context, f fs.Fs, dst fs.Dire
if !s.setDirModTimeAfter && equal {
return nil
}
if s.setDirModTimeAfter && equal {
newDst = dst
} else if s.copyEmptySrcDirs {
if s.setDirMetadata {
newDst = dst
if !equal {
if s.setDirMetadata && s.copyEmptySrcDirs {
newDst, err = operations.CopyDirMetadata(ctx, f, dst, dir, src)
} else if s.setDirModTime {
if dst == nil {
newDst, err = operations.MkdirModTime(ctx, f, dir, src.ModTime(ctx))
} else {
newDst, err = operations.SetDirModTime(ctx, f, dst, dir, src.ModTime(ctx))
}
} else if dst == nil {
// Create the directory if it doesn't exist
} else if dst == nil && s.setDirModTime && s.copyEmptySrcDirs {
newDst, err = operations.MkdirModTime(ctx, f, dir, src.ModTime(ctx))
} else if dst == nil && s.copyEmptySrcDirs {
err = operations.Mkdir(ctx, f, dir)
} else if dst != nil && s.setDirModTime {
newDst, err = operations.SetDirModTime(ctx, f, dst, dir, src.ModTime(ctx))
}
} else {
newDst = dst
}
// If we need to set modtime after and we created a dir, then save it for later
if s.setDirModTime && s.setDirModTimeAfter && err == nil {

View File

@@ -1,4 +1,4 @@
package fs
// VersionTag of rclone
var VersionTag = "v1.69.1"
var VersionTag = "v1.69.3"

View File

@@ -144,10 +144,6 @@ backends:
- backend: "dropbox"
remote: "TestDropbox:"
fastlist: false
ignore:
# This test doesn't work on a standard dropbox account because it
# tries to set the expiry of the link
- TestIntegration/FsMkdir/FsPutFiles/PublicLink
# - backend: "filefabric"
# remote: "TestFileFabric:"
# fastlist: false

20
go.mod
View File

@@ -1,6 +1,6 @@
module github.com/rclone/rclone
go 1.21
go 1.23.0
require (
bazil.org/fuse v0.0.0-20230120002735-62a210ff1fd5
@@ -80,12 +80,12 @@ require (
github.com/zeebo/blake3 v0.2.3
go.etcd.io/bbolt v1.3.10
goftp.io/server/v2 v2.0.1
golang.org/x/crypto v0.31.0
golang.org/x/net v0.33.0
golang.org/x/crypto v0.36.0
golang.org/x/net v0.38.0
golang.org/x/oauth2 v0.24.0
golang.org/x/sync v0.10.0
golang.org/x/sys v0.28.0
golang.org/x/text v0.21.0
golang.org/x/sync v0.12.0
golang.org/x/sys v0.31.0
golang.org/x/text v0.23.0
golang.org/x/time v0.8.0
google.golang.org/api v0.211.0
gopkg.in/validator.v2 v2.0.1
@@ -134,7 +134,7 @@ require (
github.com/cronokirby/saferith v0.33.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/ebitengine/purego v0.8.1 // indirect
github.com/ebitengine/purego v0.8.3 // indirect
github.com/emersion/go-message v0.18.0 // indirect
github.com/emersion/go-textwrapper v0.0.0-20200911093747-65d896831594 // indirect
github.com/emersion/go-vcard v0.0.0-20230815062825-8fda7d206ec9 // indirect
@@ -150,7 +150,7 @@ require (
github.com/goccy/go-json v0.10.3 // indirect
github.com/gofrs/flock v0.8.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v5 v5.2.1 // indirect
github.com/golang-jwt/jwt/v5 v5.2.2 // indirect
github.com/google/s2a-go v0.1.8 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect
github.com/googleapis/gax-go/v2 v2.14.0 // indirect
@@ -225,8 +225,8 @@ require (
require (
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/ProtonMail/go-crypto v1.1.3
github.com/golang-jwt/jwt/v4 v4.5.1
github.com/golang-jwt/jwt/v4 v4.5.2
github.com/pkg/xattr v0.4.10
golang.org/x/mobile v0.0.0-20240716161057-1ad2df20a8b6
golang.org/x/term v0.27.0
golang.org/x/term v0.30.0
)

36
go.sum
View File

@@ -213,8 +213,8 @@ github.com/dsnet/try v0.0.3/go.mod h1:WBM8tRpUmnXXhY1U6/S8dt6UWdHTQ7y8A5YSkRCkq4
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/ebitengine/purego v0.8.1 h1:sdRKd6plj7KYW33EH5As6YKfe8m9zbN9JMrOjNVF/BE=
github.com/ebitengine/purego v0.8.1/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/ebitengine/purego v0.8.3 h1:K+0AjQp63JEZTEMZiwsI9g0+hAMNohwUOtY0RPGexmc=
github.com/ebitengine/purego v0.8.3/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/emersion/go-message v0.18.0 h1:7LxAXHRpSeoO/Wom3ZApVZYG7c3d17yCScYce8WiXA8=
github.com/emersion/go-message v0.18.0/go.mod h1:Zi69ACvzaoV/MBnrxfVBPV3xWEuCmC2nEN39oJF4B8A=
github.com/emersion/go-textwrapper v0.0.0-20200911093747-65d896831594 h1:IbFBtwoTQyw0fIM5xv1HF+Y+3ZijDR839WMulgxCcUY=
@@ -287,10 +287,10 @@ github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-jwt/jwt/v4 v4.5.1 h1:JdqV9zKUdtaa9gdPlywC3aeoEsR681PlKC+4F5gQgeo=
github.com/golang-jwt/jwt/v4 v4.5.1/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
github.com/golang-jwt/jwt/v5 v5.2.1 h1:OuVbFODueb089Lh128TAcimifWaLhJwVflnrgM17wHk=
github.com/golang-jwt/jwt/v5 v5.2.1/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/golang-jwt/jwt/v4 v4.5.2 h1:YtQM7lnr8iZ+j5q71MGKkNw9Mn7AjHM68uc9g5fXeUI=
github.com/golang-jwt/jwt/v4 v4.5.2/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
github.com/golang-jwt/jwt/v5 v5.2.2 h1:Rl4B7itRWVtYIHFrSNd7vhTiz9UpLdi6gZhZ3wEeDy8=
github.com/golang-jwt/jwt/v5 v5.2.2/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
@@ -682,8 +682,8 @@ golang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58
golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4=
golang.org/x/crypto v0.17.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34=
golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -764,8 +764,8 @@ golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/net v0.9.0/go.mod h1:d48xBJpPfHeWQsugry2m+kC02ZBRGRgulfHnEXEuWns=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8=
golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -786,8 +786,8 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.12.0 h1:MHc5BpPuC30uJk597Ri8TV3CNZcTLu6B6z4lJy+g6Jw=
golang.org/x/sync v0.12.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -844,8 +844,8 @@ golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
@@ -856,8 +856,8 @@ golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U=
golang.org/x/term v0.15.0/go.mod h1:BDl952bC7+uMoWR75FIrCDx79TPU9oHkTZ9yRbYOrX0=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.27.0 h1:WP60Sv1nlK1T6SupCHbXzSaN0b9wUmsPoRS9b61A23Q=
golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
golang.org/x/term v0.30.0 h1:PQ39fJZ+mfadBm0y5WlL4vlM7Sx1Hgf13sMIY2+QS9Y=
golang.org/x/term v0.30.0/go.mod h1:NYYFdzHoI5wRh/h5tDMdMqCqPJZEuNqVR5xJLd/n67g=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -871,8 +871,8 @@ golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY=
golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=

View File

@@ -229,7 +229,8 @@ type Server struct {
cfg Config
template *TemplateConfig
htmlTemplate *template.Template
usingAuth bool // set if we are using auth middleware
usingAuth bool // set if we are using auth middleware
mu sync.Mutex // mutex protects RW variables below
atexitHandle atexit.FnHandle
}
@@ -517,7 +518,9 @@ func (s *Server) Serve() {
go ii.serve(&s.wg)
}
// Install an atexit handler to shutdown gracefully
s.mu.Lock()
s.atexitHandle = atexit.Register(func() { _ = s.Shutdown() })
s.mu.Unlock()
}
// Wait blocks while the server is serving requests
@@ -536,10 +539,12 @@ const gracefulShutdownTime = 10 * time.Second
// Shutdown gracefully shuts down the server
func (s *Server) Shutdown() error {
// Stop the atexit handler
s.mu.Lock()
if s.atexitHandle != nil {
atexit.Unregister(s.atexitHandle)
s.atexitHandle = nil
}
s.mu.Unlock()
for _, ii := range s.instances {
expiry := time.Now().Add(gracefulShutdownTime)
ctx, cancel := context.WithDeadline(context.Background(), expiry)

234
rclone.1 generated
View File

@@ -1,7 +1,7 @@
.\"t
.\" Automatically generated by Pandoc 2.9.2.1
.\"
.TH "rclone" "1" "Feb 14, 2025" "User Manual" ""
.TH "rclone" "1" "May 21, 2025" "User Manual" ""
.hy
.SH NAME
.PP
@@ -3583,6 +3583,11 @@ Remote authorization.
Used to authorize a remote or headless rclone from a machine with a
browser - use as instructed by rclone config.
.PP
The command requires 1-3 arguments: - fs name (e.g., \[dq]drive\[dq],
\[dq]s3\[dq], etc.) - Either a base64 encoded JSON blob obtained from a
previous rclone config session - Or a client_id and client_secret pair
obtained from the remote service
.PP
Use --auth-no-open-browser to prevent rclone to open auth link in
default browser automatically.
.PP
@@ -3592,7 +3597,7 @@ template is used.
.IP
.nf
\f[C]
rclone authorize [flags]
rclone authorize <fs name> [base64_json_blob | client_id client_secret] [flags]
\f[R]
.fi
.SS Options
@@ -5211,6 +5216,9 @@ This doesn\[aq]t transfer files that are identical on src and dst,
testing by size and modification time or MD5SUM.
It doesn\[aq]t delete files from the destination.
.PP
\f[I]If you are looking to copy just a byte range of a file, please see
\[aq]rclone cat --offset X --count Y\[aq]\f[R]
.PP
\f[B]Note\f[R]: Use the \f[C]-P\f[R]/\f[C]--progress\f[R] flag to view
real-time transfer statistics
.IP
@@ -5338,9 +5346,8 @@ Setting \f[C]--auto-filename\f[R] will attempt to automatically
determine the filename from the URL (after any redirections) and used in
the destination path.
.PP
With \f[C]--auto-filename-header\f[R] in addition, if a specific
filename is set in HTTP headers, it will be used instead of the name
from the URL.
With \f[C]--header-filename\f[R] in addition, if a specific filename is
set in HTTP headers, it will be used instead of the name from the URL.
With \f[C]--print-filename\f[R] in addition, the resulting file name
will be printed.
.PP
@@ -7178,13 +7185,13 @@ If rclone is quit or dies with files that haven\[aq]t been uploaded,
these will be uploaded next time rclone is run with the same flags.
.PP
If using \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these
quotas for two reasons.
\f[C]--vfs-cache-min-free-space\f[R] note that the cache may exceed
these quotas for two reasons.
Firstly because it is only checked every
\f[C]--vfs-cache-poll-interval\f[R].
Secondly because open files cannot be evicted from the cache.
When \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to
\f[C]--vfs-cache-min-free-space\f[R] is exceeded, rclone will attempt to
evict the least accessed files from the cache first.
rclone will start with files that haven\[aq]t been accessed for the
longest.
@@ -8698,13 +8705,13 @@ If rclone is quit or dies with files that haven\[aq]t been uploaded,
these will be uploaded next time rclone is run with the same flags.
.PP
If using \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these
quotas for two reasons.
\f[C]--vfs-cache-min-free-space\f[R] note that the cache may exceed
these quotas for two reasons.
Firstly because it is only checked every
\f[C]--vfs-cache-poll-interval\f[R].
Secondly because open files cannot be evicted from the cache.
When \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to
\f[C]--vfs-cache-min-free-space\f[R] is exceeded, rclone will attempt to
evict the least accessed files from the cache first.
rclone will start with files that haven\[aq]t been accessed for the
longest.
@@ -10147,13 +10154,13 @@ If rclone is quit or dies with files that haven\[aq]t been uploaded,
these will be uploaded next time rclone is run with the same flags.
.PP
If using \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these
quotas for two reasons.
\f[C]--vfs-cache-min-free-space\f[R] note that the cache may exceed
these quotas for two reasons.
Firstly because it is only checked every
\f[C]--vfs-cache-poll-interval\f[R].
Secondly because open files cannot be evicted from the cache.
When \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to
\f[C]--vfs-cache-min-free-space\f[R] is exceeded, rclone will attempt to
evict the least accessed files from the cache first.
rclone will start with files that haven\[aq]t been accessed for the
longest.
@@ -10808,13 +10815,13 @@ If rclone is quit or dies with files that haven\[aq]t been uploaded,
these will be uploaded next time rclone is run with the same flags.
.PP
If using \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these
quotas for two reasons.
\f[C]--vfs-cache-min-free-space\f[R] note that the cache may exceed
these quotas for two reasons.
Firstly because it is only checked every
\f[C]--vfs-cache-poll-interval\f[R].
Secondly because open files cannot be evicted from the cache.
When \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to
\f[C]--vfs-cache-min-free-space\f[R] is exceeded, rclone will attempt to
evict the least accessed files from the cache first.
rclone will start with files that haven\[aq]t been accessed for the
longest.
@@ -11460,13 +11467,13 @@ If rclone is quit or dies with files that haven\[aq]t been uploaded,
these will be uploaded next time rclone is run with the same flags.
.PP
If using \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these
quotas for two reasons.
\f[C]--vfs-cache-min-free-space\f[R] note that the cache may exceed
these quotas for two reasons.
Firstly because it is only checked every
\f[C]--vfs-cache-poll-interval\f[R].
Secondly because open files cannot be evicted from the cache.
When \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to
\f[C]--vfs-cache-min-free-space\f[R] is exceeded, rclone will attempt to
evict the least accessed files from the cache first.
rclone will start with files that haven\[aq]t been accessed for the
longest.
@@ -12425,13 +12432,13 @@ If rclone is quit or dies with files that haven\[aq]t been uploaded,
these will be uploaded next time rclone is run with the same flags.
.PP
If using \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these
quotas for two reasons.
\f[C]--vfs-cache-min-free-space\f[R] note that the cache may exceed
these quotas for two reasons.
Firstly because it is only checked every
\f[C]--vfs-cache-poll-interval\f[R].
Secondly because open files cannot be evicted from the cache.
When \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to
\f[C]--vfs-cache-min-free-space\f[R] is exceeded, rclone will attempt to
evict the least accessed files from the cache first.
rclone will start with files that haven\[aq]t been accessed for the
longest.
@@ -13227,13 +13234,13 @@ If rclone is quit or dies with files that haven\[aq]t been uploaded,
these will be uploaded next time rclone is run with the same flags.
.PP
If using \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these
quotas for two reasons.
\f[C]--vfs-cache-min-free-space\f[R] note that the cache may exceed
these quotas for two reasons.
Firstly because it is only checked every
\f[C]--vfs-cache-poll-interval\f[R].
Secondly because open files cannot be evicted from the cache.
When \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to
\f[C]--vfs-cache-min-free-space\f[R] is exceeded, rclone will attempt to
evict the least accessed files from the cache first.
rclone will start with files that haven\[aq]t been accessed for the
longest.
@@ -14366,13 +14373,13 @@ If rclone is quit or dies with files that haven\[aq]t been uploaded,
these will be uploaded next time rclone is run with the same flags.
.PP
If using \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these
quotas for two reasons.
\f[C]--vfs-cache-min-free-space\f[R] note that the cache may exceed
these quotas for two reasons.
Firstly because it is only checked every
\f[C]--vfs-cache-poll-interval\f[R].
Secondly because open files cannot be evicted from the cache.
When \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to
\f[C]--vfs-cache-min-free-space\f[R] is exceeded, rclone will attempt to
evict the least accessed files from the cache first.
rclone will start with files that haven\[aq]t been accessed for the
longest.
@@ -15072,13 +15079,13 @@ If rclone is quit or dies with files that haven\[aq]t been uploaded,
these will be uploaded next time rclone is run with the same flags.
.PP
If using \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these
quotas for two reasons.
\f[C]--vfs-cache-min-free-space\f[R] note that the cache may exceed
these quotas for two reasons.
Firstly because it is only checked every
\f[C]--vfs-cache-poll-interval\f[R].
Secondly because open files cannot be evicted from the cache.
When \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to
\f[C]--vfs-cache-min-free-space\f[R] is exceeded, rclone will attempt to
evict the least accessed files from the cache first.
rclone will start with files that haven\[aq]t been accessed for the
longest.
@@ -16090,13 +16097,13 @@ If rclone is quit or dies with files that haven\[aq]t been uploaded,
these will be uploaded next time rclone is run with the same flags.
.PP
If using \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these
quotas for two reasons.
\f[C]--vfs-cache-min-free-space\f[R] note that the cache may exceed
these quotas for two reasons.
Firstly because it is only checked every
\f[C]--vfs-cache-poll-interval\f[R].
Secondly because open files cannot be evicted from the cache.
When \f[C]--vfs-cache-max-size\f[R] or
\f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to
\f[C]--vfs-cache-min-free-space\f[R] is exceeded, rclone will attempt to
evict the least accessed files from the cache first.
rclone will start with files that haven\[aq]t been accessed for the
longest.
@@ -28524,7 +28531,7 @@ Flags for general networking and HTTP stuff.
--tpslimit float Limit HTTP transactions per second to this
--tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
--use-cookies Enable session cookiejar
--user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.69.1\[dq])
--user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.69.3\[dq])
\f[R]
.fi
.SS Performance
@@ -29571,7 +29578,7 @@ should be installed on host:
.IP
.nf
\f[C]
sudo apt-get -y install fuse
sudo apt-get -y install fuse3
\f[R]
.fi
.PP
@@ -34410,7 +34417,8 @@ error like below.
.PP
In this case you need to
restore (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/restore-archived-objects.html)
the object(s) in question before using rclone.
the object(s) in question before accessing object contents.
The restore section below shows how to do this with rclone.
.PP
Note that rclone only speaks the S3 API it does not speak the Glacier
Vault API, so rclone cannot directly access Glacier Vaults.
@@ -36691,7 +36699,7 @@ Usage Examples:
.IP
.nf
\f[C]
rclone backend restore s3:bucket/path/to/object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/ --include /object -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket -o priority=PRIORITY -o lifetime=DAYS
rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY
@@ -41752,8 +41760,8 @@ Note that rclone runs a webserver on your local machine to collect the
token as returned from Box.
This only runs from the moment it opens your browser to the moment you
get back the verification code.
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
to unblock it temporarily if you are running a host firewall.
This is on \f[C]http://127.0.0.1:53682/\f[R] and this may require you to
unblock it temporarily if you are running a host firewall.
.PP
Once configured you can then use \f[C]rclone\f[R] like this,
.PP
@@ -70151,7 +70159,8 @@ Required: false
On some SFTP servers (e.g.
Synology) the paths are different for SSH and SFTP so the hashes
can\[aq]t be calculated properly.
For them using \f[C]disable_hashcheck\f[R] is a good idea.
You can either use \f[C]--sftp-path-override\f[R] or
\f[C]disable_hashcheck\f[R].
.PP
The only ssh agent supported under Windows is Putty\[aq]s pageant.
.PP
@@ -75039,6 +75048,137 @@ Options:
.IP \[bu] 2
\[dq]error\[dq]: return an error based on option value
.SH Changelog
.SS v1.69.3 - 2025-05-21
.PP
See commits (https://github.com/rclone/rclone/compare/v1.69.2...v1.69.3)
.IP \[bu] 2
Bug Fixes
.RS 2
.IP \[bu] 2
build: Reapply update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2
to fix CVE-2025-30204 (dependabot[bot])
.IP \[bu] 2
build: Update github.com/ebitengine/purego to work around bug in
go1.24.3 (Nick Craig-Wood)
.RE
.SS v1.69.2 - 2025-05-01
.PP
See commits (https://github.com/rclone/rclone/compare/v1.69.1...v1.69.2)
.IP \[bu] 2
Bug fixes
.RS 2
.IP \[bu] 2
accounting: Fix percentDiff calculation -- (Anagh Kumar Baranwal)
.IP \[bu] 2
build
.RS 2
.IP \[bu] 2
Update github.com/golang-jwt/jwt/v4 from 4.5.1 to 4.5.2 to fix
CVE-2025-30204 (dependabot[bot])
.IP \[bu] 2
Update github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 to fix
CVE-2025-30204 (dependabot[bot])
.IP \[bu] 2
Update golang.org/x/crypto to v0.35.0 to fix CVE-2025-22869 (Nick
Craig-Wood)
.IP \[bu] 2
Update golang.org/x/net from 0.36.0 to 0.38.0 to fix CVE-2025-22870
(dependabot[bot])
.IP \[bu] 2
Update golang.org/x/net to 0.36.0.
to fix CVE-2025-22869 (dependabot[bot])
.IP \[bu] 2
Stop building with go < go1.23 as security updates forbade it (Nick
Craig-Wood)
.IP \[bu] 2
Fix docker plugin build (Anagh Kumar Baranwal)
.RE
.IP \[bu] 2
cmd: Fix crash if rclone is invoked without any arguments (Janne
Hellsten)
.IP \[bu] 2
config: Read configuration passwords from stdin even when terminated
with EOF (Samantha Bowen)
.IP \[bu] 2
doc fixes (Andrew Kreimer, Danny Garside, eccoisle, Ed Craig-Wood,
emyarod, jack, Jugal Kishore, Markus Gerstel, Michael Kebe, Nick
Craig-Wood, simonmcnair, simwai, Zachary Vorhies)
.IP \[bu] 2
fs: Fix corruption of SizeSuffix with \[dq]B\[dq] suffix in config (eg
--min-size) (Nick Craig-Wood)
.IP \[bu] 2
lib/http: Fix race between Serve() and Shutdown() (Nick Craig-Wood)
.IP \[bu] 2
object: Fix memory object out of bounds Seek (Nick Craig-Wood)
.IP \[bu] 2
operations: Fix call fmt.Errorf with wrong err (alingse)
.IP \[bu] 2
rc
.RS 2
.IP \[bu] 2
Disable the metrics server when running \f[C]rclone rc\f[R]
(hiddenmarten)
.IP \[bu] 2
Fix debug/* commands not being available over unix sockets (Nick
Craig-Wood)
.RE
.IP \[bu] 2
serve nfs: Fix unlikely crash (Nick Craig-Wood)
.IP \[bu] 2
stats: Fix the speed not getting updated after a pause in the processing
(Anagh Kumar Baranwal)
.IP \[bu] 2
sync
.RS 2
.IP \[bu] 2
Fix cpu spinning when empty directory finding with leading slashes (Nick
Craig-Wood)
.IP \[bu] 2
Copy dir modtimes even when copyEmptySrcDirs is false (ll3006)
.RE
.RE
.IP \[bu] 2
VFS
.RS 2
.IP \[bu] 2
Fix directory cache serving stale data (Lorenz Brun)
.IP \[bu] 2
Fix inefficient directory caching when directory reads are slow
(huanghaojun)
.IP \[bu] 2
Fix integration test failures (Nick Craig-Wood)
.RE
.IP \[bu] 2
Drive
.RS 2
.IP \[bu] 2
Metadata: fix error when setting copy-requires-writer-permission on a
folder (Nick Craig-Wood)
.RE
.IP \[bu] 2
Dropbox
.RS 2
.IP \[bu] 2
Retry link without expiry (Dave Vasilevsky)
.RE
.IP \[bu] 2
HTTP
.RS 2
.IP \[bu] 2
Correct root if definitely pointing to a file (nielash)
.RE
.IP \[bu] 2
Iclouddrive
.RS 2
.IP \[bu] 2
Fix so created files are writable (Ben Alex)
.RE
.IP \[bu] 2
Onedrive
.RS 2
.IP \[bu] 2
Fix metadata ordering in permissions (Nick Craig-Wood)
.RE
.SS v1.69.1 - 2025-02-14
.PP
See commits (https://github.com/rclone/rclone/compare/v1.69.0...v1.69.1)
@@ -75183,7 +75323,7 @@ in http servers (Moises Lima)
.IP \[bu] 2
This was making it impossible to use unix sockets with an proxy
.IP \[bu] 2
This might now cause rclone to need authenticaton where it didn\[aq]t
This might now cause rclone to need authentication where it didn\[aq]t
before
.RE
.IP \[bu] 2
@@ -77109,8 +77249,8 @@ Refactor version info and icon resource handling on windows (albertony)
doc updates (albertony, alfish2000, asdffdsazqqq, Dimitri Papadopoulos,
Herby Gillot, Joda St\[:o]\[ss]er, Manoj Ghosh, Nick Craig-Wood)
.IP \[bu] 2
Implement \f[C]--metadata-mapper\f[R] to transform metatadata with a
user supplied program (Nick Craig-Wood)
Implement \f[C]--metadata-mapper\f[R] to transform metadata with a user
supplied program (Nick Craig-Wood)
.IP \[bu] 2
Add \f[C]ChunkWriterDoesntSeek\f[R] feature flag and set it for b2 (Nick
Craig-Wood)
@@ -77430,7 +77570,7 @@ B2
Fix multipart upload: corrupted on transfer: sizes differ XXX vs 0 (Nick
Craig-Wood)
.IP \[bu] 2
Fix locking window when getting mutipart upload URL (Nick Craig-Wood)
Fix locking window when getting multipart upload URL (Nick Craig-Wood)
.IP \[bu] 2
Fix server side copies greater than 4GB (Nick Craig-Wood)
.IP \[bu] 2
@@ -91986,8 +92126,6 @@ Eli Orzitzer <e_orz@yahoo.com>
.IP \[bu] 2
Anthony Metzidis <anthony.metzidis@gmail.com>
.IP \[bu] 2
emyarod <afw5059@gmail.com>
.IP \[bu] 2
keongalvin <keongalvin@gmail.com>
.IP \[bu] 2
rarspace01 <rarspace01@users.noreply.github.com>

View File

@@ -579,7 +579,7 @@ func (d *Dir) _readDir() error {
return err
}
d.read = when
d.read = time.Now()
d.cleanupTimer.Reset(time.Duration(d.vfs.Opt.DirCacheTime * 2))
return nil
@@ -760,6 +760,7 @@ func (d *Dir) _readDirFromEntries(entries fs.DirEntries, dirTree dirtree.DirTree
dir := node.(*Dir)
dir.mu.Lock()
dir.modTime = item.ModTime(context.TODO())
dir.entry = item
if dirTree != nil {
err = dir._readDirFromDirTree(dirTree, when)
if err != nil {

View File

@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"os"
"runtime"
"sort"
"testing"
"time"
@@ -655,3 +656,38 @@ func TestDirFileOpen(t *testing.T) {
require.NoError(t, err)
assert.Equal(t, int64(12), fi.Size())
}
func TestDirEntryModTimeInvalidation(t *testing.T) {
r, vfs := newTestVFS(t)
features := r.Fremote.Features()
if !features.DirModTimeUpdatesOnWrite {
t.Skip("Need DirModTimeUpdatesOnWrite")
}
if features.IsLocal && runtime.GOOS == "windows" {
t.Skip("dirent modtime is unreliable on Windows filesystems")
}
// Needs to be less than 2x the wait time below, othewrwise the entry
// gets cleared out before it had a chance to be updated.
vfs.Opt.DirCacheTime = fs.Duration(50 * time.Millisecond)
r.WriteObject(context.Background(), "dir/file1", "file1 contents", t1)
node, err := vfs.Stat("dir")
require.NoError(t, err)
modTime1 := node.(*Dir).DirEntry().ModTime(context.Background())
// Wait some time, then write another file which must update ModTime of
// the directory.
time.Sleep(75 * time.Millisecond)
r.WriteObject(context.Background(), "dir/file2", "file2 contents", t2)
node2, err := vfs.Stat("dir")
require.NoError(t, err)
modTime2 := node2.(*Dir).DirEntry().ModTime(context.Background())
// ModTime of directory must be different after second file was written.
if modTime1.Equal(modTime2) {
t.Error("ModTime not invalidated")
}
}

View File

@@ -95,11 +95,11 @@ seconds. If rclone is quit or dies with files that haven't been
uploaded, these will be uploaded next time rclone is run with the same
flags.
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note
If using `--vfs-cache-max-size` or `--vfs-cache-min-free-space` note
that the cache may exceed these quotas for two reasons. Firstly
because it is only checked every `--vfs-cache-poll-interval`. Secondly
because open files cannot be evicted from the cache. When
`--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded,
`--vfs-cache-max-size` or `--vfs-cache-min-free-space` is exceeded,
rclone will attempt to evict the least accessed files from the cache
first. rclone will start with files that haven't been accessed for the
longest. This cache flushing strategy is efficient and more relevant