1
0
mirror of https://github.com/rclone/rclone.git synced 2026-01-12 21:43:13 +00:00

Compare commits

..

30 Commits

Author SHA1 Message Date
Nick Craig-Wood
e2531e08be Version v1.54.1 2021-03-08 10:04:23 +00:00
Nick Craig-Wood
86babc6393 build: fix nfpm install by using the released binary 2021-03-07 17:04:04 +00:00
Ivan Andreev
d45d48cbe5 chunker: fix integration tests after backport commit 6baa4e294
It was my stupid typo: "f.base.NewObject" instead of correct "f.NewObject"
I made it during backport. The commit on master was correct.
2021-03-07 13:28:43 +00:00
edwardxml
7ad2c22d5b docs: remove dead link from rc.md (#5038) 2021-03-06 11:51:44 +00:00
Dmitry Chepurovskiy
0cfcc08be1 s3: Fix shared_credentials_file auth
S3 backend shared_credentials_file option wasn't working neither from
config option nor from command line option. This was caused cause
shared_credentials_file_provider works as part of chain provider, but in
case user haven't specified access_token and access_key we had removed
(set nil) to credentials field, that may contain actual credentials got
from ChainProvider.

AWS_SHARED_CREDENTIALS_FILE env varible as far as i understood worked,
cause aws_sdk code handles it as one of default auth options, when
there's not configured credentials.
2021-03-06 11:51:19 +00:00
edwardxml
2c4a25de5b docs: convert bogus example link to code
Convert the bogus example plex url from a url that is auto created to code format that hopefully isn't.
2021-03-06 11:50:23 +00:00
edwardxml
f5a95b2ad0 docs: badly formed link
Fix for a badly formed link created in earlier rewrite
2021-03-06 11:50:08 +00:00
Nick Craig-Wood
f2caa0eabb vfs: document simultaneous usage with the same cache shouldn't be used
Fixes #2227
2021-03-06 11:49:48 +00:00
Miron Veryanskiy
4943a5028c docs: replace #file-caching with #vfs-file-caching
The documentation had dead links pointing to #file-caching. They've been
moved to point to #vfs-file-caching.
2021-03-06 11:49:23 +00:00
Romeo Kienzler
60bebe4b35 docs: fix typo in crypt.md (#5037) 2021-03-06 11:44:51 +00:00
edwardxml
61031cfdea docs: fix broken link in sftp page
Just a spare line break had crept in breaking the link form.
2021-03-06 11:44:38 +00:00
edwardxml
da7e4379fa docs: fix nesting of brackets and backticks in ftp docs 2021-03-06 11:43:35 +00:00
Nick Craig-Wood
7e7a91ce3d rc: sync,copy,move: document createEmptySrcDirs parameter - fixes #4489 2021-03-06 11:43:15 +00:00
Ivan Andreev
6baa4e2947 address stringent ineffectual assignment check in golangci-lint (#5093) 2021-03-05 20:52:38 +03:00
Nick Craig-Wood
3f53283ebf s3: fix Wasabi HEAD requests returning stale data by using only 1 transport
In this commit

fc5b14b620 s3: Added `--s3-disable-http2` to disable http/2

We created our own transport so we could disable http/2. However the
added function is called twice meaning that we create two HTTP
transports. This didn't happen with the original code because the
default transport is cached by fshttp.

Rclone normally does a PUT followed by a HEAD request to check an
upload has been successful.

With the two transports, the PUT and the HEAD were being done on
different HTTP transports. This means that it wasn't re-using the same
HTTP connection, so the HEAD request showed the previous object value.
This caused rclone to declare the upload was corrupted, delete the
object and try again.

This patch makes sure we only create one transport and use it for both
PUT and HEAD requests which fixes the problem with Wasabi.

See: https://forum.rclone.org/t/each-time-rclone-is-run-1-3-fails-2-3-succeeds/22545
2021-03-05 15:35:23 +00:00
Nick Craig-Wood
da9dd543e4 s3: fix failed to create file system with folder level permissions policy
Before this change, if folder level access permissions policy was in
use, with trailing `/` marking the folders then rclone would HEAD the
path without a trailing `/` to work out if it was a file or a folder.
This returned a permission denied error, which rclone returned to the
user.

    Failed to create file system for "s3:bucket/path/": Forbidden: Forbidden
        status code: 403, request id: XXXX, host id:

Previous to this change

53aa03cc44 s3: complete sse-c implementation

rclone would assume any errors when HEAD-ing the object implied it
didn't exist and this test would not fail.

This change reverts the functionality of the test to work as it did
before, meaning any errors on HEAD will make rclone assume the object
does not exist and the path is referring to a directory.

Fixes #4990
2021-02-24 20:36:09 +00:00
Ivan Andreev
e3cf4f82eb build: replace go 1.16-rc1 by 1.16.x (#5036) 2021-02-24 20:10:51 +00:00
buengese
406e26c7b7 zoho: fix custom client id's 2021-02-23 11:27:19 +00:00
Nick Craig-Wood
f4214882ab cmount: fix mount dropping on macOS by setting --daemon-timeout 10m
Previously rclone set --daemon-timeout to 15m by default. However
osxfuse seems to be ignoring that value since it is above the maximum
value of 10m. This is conjecture since the source of osxfuse is no
longer available.

Setting the value to 10m seems to resolve the problem.

See: https://forum.rclone.org/t/rclone-mount-frequently-drops-when-using-plex/22352
2021-02-21 13:00:47 +00:00
Nick Craig-Wood
231ab31d2a union: fix mkdir at root with remote:/
Before the this fix if you specified remote:/ then the union backend
would fail to notice the root directory existed.

This was fixed by stripping the trailing / from the root.

See: https://forum.rclone.org/t/upgraded-from-1-45-to-1-54-now-cant-create-new-directory-within-union-mount/22284/
2021-02-17 12:12:14 +00:00
Nick Craig-Wood
f76bc86cc8 accounting: fix --bwlimit when up or down is off - fixes #5019
Before this change the core bandwidth limit was limited to upload or
download value if the other value was off.

This fix only applies a core bandwidth limit when both values are set.
2021-02-13 12:45:45 +00:00
Nick Craig-Wood
2d11f5672d dropbox: add scopes to oauth request and optionally "members.read"
This change adds the scopes rclone wants during the oauth request.
Previously rclone left these blank to get a default set.

This allows rclone to add the "members.read" scope which is necessary
for "impersonate" to work, but only when it is in use as it require
authorisation from a Team Admin.

See: https://forum.rclone.org/t/dropbox-no-members-read/22223/3
2021-02-13 12:35:45 +00:00
Nick Craig-Wood
cf0563f99e b2: fix failed to create file system with application key limited to a prefix
Before this change, if an application key limited to a prefix was in
use, with trailing `/` marking the folders then rclone would HEAD the
path without a trailing `/` to work out if it was a file or a folder.
This returned a permission denied error, which rclone returned to the
user.

    Failed to create file system for "b2:bucket/path/":
        failed to HEAD for download: Unknown 401  (401 unknown)

This change assumes any errors on HEAD will make rclone assume the
object does not exist and the path is referring to a directory.

See: https://forum.rclone.org/t/b2-error-on-application-key-limited-to-a-prefix/22159/
2021-02-10 15:27:45 +00:00
Nick Craig-Wood
65f691f4de drive: refer to Shared Drives instead of Team Drives 2021-02-10 15:27:19 +00:00
Nick Craig-Wood
f627d42a51 lsjson: fix unterminated JSON in the presence of errors
See: https://forum.rclone.org/t/rclone-lsjson-invalid-json-produced-no-at-the-end/22046
2021-02-10 15:26:48 +00:00
Nick Craig-Wood
f08e43fb77 b2: automatically raise upload cutoff to avoid spurious error
Before this change, if --b2-chunk-size was raised above 200M then this
error would be produced:

    b2: upload cutoff: 200M is less than chunk size 1G

This change automatically reaises --b2-upload-cutoff to be the value
of --b2-chunk-size if it is below it, which stops this error being
generated.

Fixes #4475
2021-02-10 15:26:17 +00:00
Nick Craig-Wood
cd7611e7ce s3: add --s3-no-head to reducing costs docs - Fixes #2163 2021-02-10 15:24:46 +00:00
Nick Craig-Wood
42f28f9458 build: update GitHub release tool to use gh and put a link to changelog
Fixes #4994
2021-02-10 15:24:46 +00:00
Alex JOST
92046b457f docs: Changelog: Correct link to digitalis.io 2021-02-10 15:24:46 +00:00
Nick Craig-Wood
098de1cff5 Start v1.54.1-DEV development 2021-02-10 15:24:46 +00:00
75 changed files with 1607 additions and 872 deletions

View File

@@ -19,12 +19,12 @@ jobs:
strategy:
fail-fast: false
matrix:
job_name: ['linux', 'mac', 'windows_amd64', 'windows_386', 'other_os', 'go1.13', 'go1.14', 'go1.15']
job_name: ['linux', 'mac', 'windows_amd64', 'windows_386', 'other_os', 'go1.12', 'go1.13', 'go1.14', 'go1.16']
include:
- job_name: linux
os: ubuntu-latest
go: '1.16.0-rc1'
go: '1.15.x'
gotags: cmount
build_flags: '-include "^linux/"'
check: true
@@ -34,7 +34,7 @@ jobs:
- job_name: mac
os: macOS-latest
go: '1.16.0-rc1'
go: '1.15.x'
gotags: 'cmount'
build_flags: '-include "^darwin/amd64" -cgo'
quicktest: true
@@ -43,7 +43,7 @@ jobs:
- job_name: windows_amd64
os: windows-latest
go: '1.16.0-rc1'
go: '1.15.x'
gotags: cmount
build_flags: '-include "^windows/amd64" -cgo'
build_args: '-buildmode exe'
@@ -53,7 +53,7 @@ jobs:
- job_name: windows_386
os: windows-latest
go: '1.16.0-rc1'
go: '1.15.x'
gotags: cmount
goarch: '386'
cgo: '1'
@@ -64,11 +64,16 @@ jobs:
- job_name: other_os
os: ubuntu-latest
go: '1.16.0-rc1'
go: '1.15.x'
build_flags: '-exclude "^(windows/|darwin/amd64|linux/)"'
compile_all: true
deploy: true
- job_name: go1.12
os: ubuntu-latest
go: '1.12.x'
quicktest: true
- job_name: go1.13
os: ubuntu-latest
go: '1.13.x'
@@ -80,9 +85,9 @@ jobs:
quicktest: true
racequicktest: true
- job_name: go1.15
- job_name: go1.16
os: ubuntu-latest
go: '1.15.x'
go: '1.16.x'
quicktest: true
racequicktest: true

304
MANUAL.html generated
View File

@@ -17,7 +17,7 @@
<header id="title-block-header">
<h1 class="title">rclone(1) User Manual</h1>
<p class="author">Nick Craig-Wood</p>
<p class="date">Feb 02, 2021</p>
<p class="date">Mar 08, 2021</p>
</header>
<h1 id="rclone-syncs-your-files-to-cloud-storage">Rclone syncs your files to cloud storage</h1>
<p><img width="50%" src="https://rclone.org/img/logo_on_light__horizontal_color.svg" alt="rclone logo" style="float:right; padding: 5px;" ></p>
@@ -1471,11 +1471,11 @@ rclone mount remote:path/to/files * --volname \\cloud\remote</code></pre>
<p>Note that drives created as Administrator are not visible by other accounts (including the account that was elevated as Administrator). So if you start a Windows drive from an Administrative Command Prompt and then try to access the same drive from Explorer (which does not run as Administrator), you will not be able to see the new drive.</p>
<p>The easiest way around this is to start the drive from a normal command prompt. It is also possible to start a drive from the SYSTEM account (using <a href="https://github.com/billziss-gh/winfsp/wiki/WinFsp-Service-Architecture">the WinFsp.Launcher infrastructure</a>) which creates drives accessible for everyone on the system or alternatively using <a href="https://nssm.cc/usage">the nssm service manager</a>.</p>
<h2 id="limitations">Limitations</h2>
<p>Without the use of <code>--vfs-cache-mode</code> this can only write files sequentially, it can only seek when reading. This means that many applications won't work with their files on an rclone mount without <code>--vfs-cache-mode writes</code> or <code>--vfs-cache-mode full</code>. See the <a href="#file-caching">File Caching</a> section for more info.</p>
<p>Without the use of <code>--vfs-cache-mode</code> this can only write files sequentially, it can only seek when reading. This means that many applications won't work with their files on an rclone mount without <code>--vfs-cache-mode writes</code> or <code>--vfs-cache-mode full</code>. See the <a href="#vfs-file-caching">VFS File Caching</a> section for more info.</p>
<p>The bucket based remotes (e.g. Swift, S3, Google Compute Storage, B2, Hubic) do not support the concept of empty directories, so empty directories will have a tendency to disappear once they fall out of the directory cache.</p>
<p>Only supported on Linux, FreeBSD, OS X and Windows at the moment.</p>
<h2 id="rclone-mount-vs-rclone-synccopy">rclone mount vs rclone sync/copy</h2>
<p>File systems expect things to be 100% reliable, whereas cloud storage systems are a long way from 100% reliable. The rclone sync/copy commands cope with this with lots of retries. However rclone mount can't use retries in the same way without making local copies of the uploads. Look at the <a href="#file-caching">file caching</a> for solutions to make mount more reliable.</p>
<p>File systems expect things to be 100% reliable, whereas cloud storage systems are a long way from 100% reliable. The rclone sync/copy commands cope with this with lots of retries. However rclone mount can't use retries in the same way without making local copies of the uploads. Look at the <a href="#vfs-file-caching">VFS File Caching</a> for solutions to make mount more reliable.</p>
<h2 id="attribute-caching">Attribute caching</h2>
<p>You can use the flag <code>--attr-timeout</code> to set the time the kernel caches the attributes (size, modification time, etc.) for directory entries.</p>
<p>The default is <code>1s</code> which caches files just long enough to avoid too many callbacks to rclone from the kernel.</p>
@@ -1526,6 +1526,7 @@ rclone mount remote:path/to/files * --volname \\cloud\remote</code></pre>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for --vfs-write-back second. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> note that the cache may exceed this size for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache.</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off">--vfs-cache-mode off</h3>
<p>In this mode (the default) the cache will read directly from the remote and write directly to the remote without caching anything on disk.</p>
<p>This will mean some operations are not possible</p>
@@ -1848,6 +1849,7 @@ ffmpeg - | rclone rcat remote:path/to/file</code></pre>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for --vfs-write-back second. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> note that the cache may exceed this size for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache.</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-1">--vfs-cache-mode off</h3>
<p>In this mode (the default) the cache will read directly from the remote and write directly to the remote without caching anything on disk.</p>
<p>This will mean some operations are not possible</p>
@@ -1982,6 +1984,7 @@ ffmpeg - | rclone rcat remote:path/to/file</code></pre>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for --vfs-write-back second. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> note that the cache may exceed this size for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache.</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-2">--vfs-cache-mode off</h3>
<p>In this mode (the default) the cache will read directly from the remote and write directly to the remote without caching anything on disk.</p>
<p>This will mean some operations are not possible</p>
@@ -2246,6 +2249,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for --vfs-write-back second. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> note that the cache may exceed this size for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache.</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-3">--vfs-cache-mode off</h3>
<p>In this mode (the default) the cache will read directly from the remote and write directly to the remote without caching anything on disk.</p>
<p>This will mean some operations are not possible</p>
@@ -2558,6 +2562,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for --vfs-write-back second. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> note that the cache may exceed this size for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache.</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-4">--vfs-cache-mode off</h3>
<p>In this mode (the default) the cache will read directly from the remote and write directly to the remote without caching anything on disk.</p>
<p>This will mean some operations are not possible</p>
@@ -2823,6 +2828,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>The cache has 4 different modes selected by <code>--vfs-cache-mode</code>. The higher the cache mode the more compatible rclone becomes at the cost of using disk space.</p>
<p>Note that files are written back to the remote only when they are closed and if they haven't been accessed for --vfs-write-back second. If rclone is quit or dies with files that haven't been uploaded, these will be uploaded next time rclone is run with the same flags.</p>
<p>If using <code>--vfs-cache-max-size</code> note that the cache may exceed this size for two reasons. Firstly because it is only checked every <code>--vfs-cache-poll-interval</code>. Secondly because open files cannot be evicted from the cache.</p>
<p>You <strong>should not</strong> run two copies of rclone using the same VFS cache with the same or overlapping remotes if using <code>--vfs-cache-mode &gt; off</code>. This can potentially cause data corruption if you do. You can work around this by giving each rclone its own cache hierarchy with <code>--cache-dir</code>. You don't need to worry about this if the remotes in use don't overlap.</p>
<h3 id="vfs-cache-mode-off-5">--vfs-cache-mode off</h3>
<p>In this mode (the default) the cache will read directly from the remote and write directly to the remote without caching anything on disk.</p>
<p>This will mean some operations are not possible</p>
@@ -4085,7 +4091,7 @@ dir1/dir2/dir3/.ignore</code></pre>
<p>The command <code>rclone ls --exclude-if-present .ignore dir1</code> does not list <code>dir3</code>, <code>file3</code> or <code>.ignore</code>.</p>
<p><code>--exclude-if-present</code> can only be used once in an rclone command.</p>
<h2 id="common-pitfalls">Common pitfalls</h2>
<p>The most frequent filter support issues on the <a href="https://https://forum.rclone.org/">rclone forum</a> are:</p>
<p>The most frequent filter support issues on the <a href="https://forum.rclone.org/">rclone forum</a> are:</p>
<ul>
<li>Not using paths relative to the root of the remote</li>
<li>Not using <code>/</code> to match from the root of a remote</li>
@@ -4997,6 +5003,7 @@ rclone rc mount/mount fs=TestDrive: mountPoint=/mnt/tmp vfsOpt=&#39;{&quot;Cache
<ul>
<li>srcFs - a remote name string e.g. "drive:src" for the source</li>
<li>dstFs - a remote name string e.g. "drive:dst" for the destination</li>
<li>createEmptySrcDirs - create empty src directories on destination if set</li>
</ul>
<p>See the <a href="https://rclone.org/commands/rclone_copy/">copy command</a> command for more information on the above.</p>
<p><strong>Authentication is required for this call.</strong></p>
@@ -5005,6 +5012,7 @@ rclone rc mount/mount fs=TestDrive: mountPoint=/mnt/tmp vfsOpt=&#39;{&quot;Cache
<ul>
<li>srcFs - a remote name string e.g. "drive:src" for the source</li>
<li>dstFs - a remote name string e.g. "drive:dst" for the destination</li>
<li>createEmptySrcDirs - create empty src directories on destination if set</li>
<li>deleteEmptySrcDirs - delete empty src directories if set</li>
</ul>
<p>See the <a href="https://rclone.org/commands/rclone_move/">move command</a> command for more information on the above.</p>
@@ -5014,6 +5022,7 @@ rclone rc mount/mount fs=TestDrive: mountPoint=/mnt/tmp vfsOpt=&#39;{&quot;Cache
<ul>
<li>srcFs - a remote name string e.g. "drive:src" for the source</li>
<li>dstFs - a remote name string e.g. "drive:dst" for the destination</li>
<li>createEmptySrcDirs - create empty src directories on destination if set</li>
</ul>
<p>See the <a href="https://rclone.org/commands/rclone_sync/">sync command</a> command for more information on the above.</p>
<p><strong>Authentication is required for this call.</strong></p>
@@ -6491,7 +6500,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
--use-json-log Use json log format.
--use-mmap Use mmap allocator (see docs).
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string. The default is rclone/ version (default &quot;rclone/v1.54.0&quot;)
--user-agent string Set the user-agent to a specified string. The default is rclone/ version (default &quot;rclone/v1.54.1&quot;)
-v, --verbose count Print lots more stuff (repeat for more)</code></pre>
<h2 id="backend-flags">Backend Flags</h2>
<p>These flags are available for every command. They control the backends and may be set in the config file.</p>
@@ -6616,7 +6625,7 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
--drive-starred-only Only show files that are starred.
--drive-stop-on-download-limit Make download limit errors be fatal
--drive-stop-on-upload-limit Make upload limit errors be fatal
--drive-team-drive string ID of the Team Drive
--drive-team-drive string ID of the Shared Drive (Team Drive)
--drive-token string OAuth Access Token as a JSON blob.
--drive-token-url string Token server url.
--drive-trashed-only Only show files that are in the trash.
@@ -6897,8 +6906,13 @@ Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
--yandex-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,InvalidUtf8,Dot)
--yandex-token string OAuth Access Token as a JSON blob.
--yandex-token-url string Token server url.
--zoho-auth-url string Auth server URL.
--zoho-client-id string OAuth Client Id
--zoho-client-secret string OAuth Client Secret
--zoho-encoding MultiEncoder This sets the encoding for the backend. (default Del,Ctl,InvalidUtf8)
--zoho-region string Zoho region to connect to. You&#39;ll have to use the region you organization is registered in.</code></pre>
--zoho-region string Zoho region to connect to. You&#39;ll have to use the region you organization is registered in.
--zoho-token string OAuth Access Token as a JSON blob.
--zoho-token-url string Token server url.</code></pre>
<h2 id="fichier">1Fichier</h2>
<p>This is a backend for the <a href="https://1fichier.com">1fichier</a> cloud storage service. Note that a Premium subscription is required to use the API.</p>
<p>Paths are specified as <code>remote:path</code></p>
@@ -7575,6 +7589,10 @@ y/e/d&gt; </code></pre>
<pre><code>rclone copy --min-age 24h --no-traverse /path/to/source s3:bucket</code></pre>
<p>You'd then do a full <code>rclone sync</code> less often.</p>
<p>Note that <code>--fast-list</code> isn't required in the top-up sync.</p>
<h4 id="avoiding-head-requests-after-put">Avoiding HEAD requests after PUT</h4>
<p>By default rclone will HEAD every object it uploads. It does this to check the object got uploaded correctly.</p>
<p>You can disable this with the <a href="#s3-no-head">--s3-no-head</a> option - see there for more details.</p>
<p>Setting this flag increases the chance for undetected upload failures.</p>
<h3 id="hashes">Hashes</h3>
<p>For small objects which weren't uploaded as multipart uploads (objects sized below <code>--s3-upload-cutoff</code> if uploaded with rclone) rclone uses the <code>ETag:</code> header as an MD5 checksum.</p>
<p>However for objects which were uploaded as multipart uploads or with server side encryption (SSE-AWS or SSE-C) the <code>ETag</code> header is no longer the MD5 sum of the data, so rclone adds an additional piece of metadata <code>X-Amz-Meta-Md5chksum</code> which is a base64 encoded MD5 hash (in the same format as is required for <code>Content-MD5</code>).</p>
@@ -10735,7 +10753,7 @@ chunk_total_size = 10G</code></pre>
<h5 id="certificate-validation">Certificate Validation</h5>
<p>When the Plex server is configured to only accept secure connections, it is possible to use <code>.plex.direct</code> URLs to ensure certificate validation succeeds. These URLs are used by Plex internally to connect to the Plex server securely.</p>
<p>The format for these URLs is the following:</p>
<p>https://ip-with-dots-replaced.server-hash.plex.direct:32400/</p>
<p><code>https://ip-with-dots-replaced.server-hash.plex.direct:32400/</code></p>
<p>The <code>ip-with-dots-replaced</code> part can be any IPv4 address, where the dots have been replaced with dashes, e.g. <code>127.0.0.1</code> becomes <code>127-0-0-1</code>.</p>
<p>To get the <code>server-hash</code> part, the easiest way is to visit</p>
<p>https://plex.tv/api/resources?includeHttps=1&amp;X-Plex-Token=your-plex-token</p>
@@ -11491,7 +11509,7 @@ y/e/d&gt; y</code></pre>
<p>To use <code>crypt</code>, first set up the underlying remote. Follow the <code>rclone config</code> instructions for the specific backend.</p>
<p>Before configuring the crypt remote, check the underlying remote is working. In this example the underlying remote is called <code>remote</code>. We will configure a path <code>path</code> within this remote to contain the encrypted content. Anything inside <code>remote:path</code> will be encrypted and anything outside will not.</p>
<p>Configure <code>crypt</code> using <code>rclone config</code>. In this example the <code>crypt</code> remote is called <code>secret</code>, to differentiate it from the underlying <code>remote</code>.</p>
<p>When you are done you can use the crypt remote named <code>secret</code> just as you would with any other remote, e.g. <code>rclone copy D:\docs secret:\docs</code>, and rclone will encrypt and decrypt as needed on the fly. If you access the wrapped remote <code>remote:path</code> directly you will bypass the encryption, and anything you read will be in encrypted form, and anything you write will be undencrypted. To avoid issues it is best to configure a dedicated path for encrypted content, and access it exclusively through a crypt remote.</p>
<p>When you are done you can use the crypt remote named <code>secret</code> just as you would with any other remote, e.g. <code>rclone copy D:\docs secret:\docs</code>, and rclone will encrypt and decrypt as needed on the fly. If you access the wrapped remote <code>remote:path</code> directly you will bypass the encryption, and anything you read will be in encrypted form, and anything you write will be unencrypted. To avoid issues it is best to configure a dedicated path for encrypted content, and access it exclusively through a crypt remote.</p>
<pre><code>No remotes found - make a new one
n) New remote
s) Set configuration password
@@ -12125,6 +12143,9 @@ y/e/d&gt; y</code></pre>
</ul>
<h4 id="dropbox-impersonate">--dropbox-impersonate</h4>
<p>Impersonate this user when using a business account.</p>
<p>Note that if you want to use impersonate, you should make sure this flag is set when running "rclone config" as this will cause rclone to request the "members.read" scope which it won't normally. This is needed to lookup a members email address into the internal ID that dropbox uses in the API.</p>
<p>Using the "members.read" scope will require a Dropbox Team Admin to approve during the OAuth flow.</p>
<p>You will have to use your own App (setting your own client_id and client_secret) to use this option as currently rclone's default set of permissions doesn't include "members.read". This can be added once v1.55 or later is in use everywhere.</p>
<ul>
<li>Config: impersonate</li>
<li>Env Var: RCLONE_DROPBOX_IMPERSONATE</li>
@@ -12428,7 +12449,7 @@ y/e/d&gt; y</code></pre>
<h3 id="example-without-a-config-file">Example without a config file</h3>
<pre><code>rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=`rclone obscure dummy`</code></pre>
<h3 id="implicit-tls">Implicit TLS</h3>
<p>Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to be enabled in the FTP backend config for the remote, or with <code>[--ftp-tls]{#ftp-tls}</code>. The default FTPS port is <code>990</code>, not <code>21</code> and can be set with <code>[--ftp-port]{#ftp-port}</code>.</p>
<p>Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to be enabled in the FTP backend config for the remote, or with <a href="#ftp-tls"><code>--ftp-tls</code></a>. The default FTPS port is <code>990</code>, not <code>21</code> and can be set with <a href="#ftp-port"><code>--ftp-port</code></a>.</p>
<h3 id="standard-options-13">Standard Options</h3>
<p>Here are the standard options specific to ftp (FTP Connection).</p>
<h4 id="ftp-host">--ftp-host</h4>
@@ -13132,7 +13153,7 @@ If your browser doesn&#39;t open automatically go to the following link: http://
Log in and authorize rclone for access
Waiting for code...
Got code
Configure this as a team drive?
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No
y/n&gt; n
@@ -13233,15 +13254,15 @@ y/n&gt; # Auto config, y
</ul></li>
</ul>
<p>Note: in case you configured a specific root folder on gdrive and rclone is unable to access the contents of that folder when using <code>--drive-impersonate</code>, do this instead: - in the gdrive web interface, share your root folder with the user/email of the new Service Account you created/selected at step #1 - use rclone without specifying the <code>--drive-impersonate</code> option, like this: <code>rclone -v foo@example.com lsf gdrive:backup</code></p>
<h3 id="team-drives">Team drives</h3>
<p>If you want to configure the remote to point to a Google Team Drive then answer <code>y</code> to the question <code>Configure this as a team drive?</code>.</p>
<p>This will fetch the list of Team Drives from google and allow you to configure which one you want to use. You can also type in a team drive ID if you prefer.</p>
<h3 id="shared-drives-team-drives">Shared drives (team drives)</h3>
<p>If you want to configure the remote to point to a Google Shared Drive (previously known as Team Drives) then answer <code>y</code> to the question <code>Configure this as a Shared Drive (Team Drive)?</code>.</p>
<p>This will fetch the list of Shared Drives from google and allow you to configure which one you want to use. You can also type in a Shared Drive ID if you prefer.</p>
<p>For example:</p>
<pre><code>Configure this as a team drive?
<pre><code>Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No
y/n&gt; y
Fetching team drive list...
Fetching Shared Drive list...
Choose a number from below, or type in your own value
1 / Rclone Test
\ &quot;xxxxxxxxxxxxxxxxxxxx&quot;
@@ -13249,7 +13270,7 @@ Choose a number from below, or type in your own value
\ &quot;yyyyyyyyyyyyyyyyyyyy&quot;
3 / Rclone Test 3
\ &quot;zzzzzzzzzzzzzzzzzzzz&quot;
Enter a Team Drive ID&gt; 1
Enter a Shared Drive ID&gt; 1
--------------------
[remote]
client_id =
@@ -13659,7 +13680,7 @@ trashed=false and &#39;c&#39; in parents</code></pre>
<li>Default: ""</li>
</ul>
<h4 id="drive-team-drive">--drive-team-drive</h4>
<p>ID of the Team Drive</p>
<p>ID of the Shared Drive (Team Drive)</p>
<ul>
<li>Config: team_drive</li>
<li>Env Var: RCLONE_DRIVE_TEAM_DRIVE</li>
@@ -13972,9 +13993,9 @@ rclone backend shortcut drive: source_item -o target=drive2: destination_shortcu
<li>"target": optional target remote for the shortcut destination</li>
</ul>
<h4 id="drives">drives</h4>
<p>List the shared drives available to this account</p>
<p>List the Shared Drives available to this account</p>
<pre><code>rclone backend drives remote: [options] [&lt;arguments&gt;+]</code></pre>
<p>This command lists the shared drives (teamdrives) available to this account.</p>
<p>This command lists the Shared Drives (Team Drives) available to this account.</p>
<p>Usage:</p>
<pre><code>rclone backend drives drive:</code></pre>
<p>This will return a JSON list of objects like this</p>
@@ -18173,7 +18194,7 @@ known_hosts_file = ~/.ssh/known_hosts</code></pre>
<p>SFTP also supports <code>about</code> if the same login has shell access and <code>df</code> are in the remote's PATH. <code>about</code> will return the total space, free space, and used space on the remote for the disk of the specified path on the remote or, if not set, the disk of the root on the remote. <code>about</code> will fail if it does not have shell access or if <code>df</code> is not in the remote's PATH.</p>
<p>Note that some SFTP servers (e.g. Synology) the paths are different for SSH and SFTP so the hashes can't be calculated properly. For them using <code>disable_hashcheck</code> is a good idea.</p>
<p>The only ssh agent supported under Windows is Putty's pageant.</p>
<p>The Go SSH library disables the use of the aes128-cbc cipher by default, due to security concerns. This can be re-enabled on a per-connection basis by setting the <code>use_insecure_cipher</code> setting in the configuration file to <code>true</code>. Further details on the insecurity of this cipher can be found [in this paper] (http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).</p>
<p>The Go SSH library disables the use of the aes128-cbc cipher by default, due to security concerns. This can be re-enabled on a per-connection basis by setting the <code>use_insecure_cipher</code> setting in the configuration file to <code>true</code>. Further details on the insecurity of this cipher can be found <a href="http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf">in this paper</a>.</p>
<p>SFTP isn't supported under plan9 until <a href="https://github.com/pkg/sftp/issues/156">this issue</a> is fixed.</p>
<p>Note that since SFTP isn't HTTP based the following flags don't work with it: <code>--dump-headers</code>, <code>--dump-bodies</code>, <code>--dump-auth</code></p>
<p>Note that <code>--timeout</code> isn't supported (but <code>--contimeout</code> is).</p>
@@ -19267,6 +19288,22 @@ y/e/d&gt; </code></pre>
<p>Only control characters and invalid UTF-8 are replaced. In addition most Unicode full-width characters are not supported at all and will be removed from filenames during upload.</p>
<h3 id="standard-options-37">Standard Options</h3>
<p>Here are the standard options specific to zoho (Zoho).</p>
<h4 id="zoho-client-id">--zoho-client-id</h4>
<p>OAuth Client Id Leave blank normally.</p>
<ul>
<li>Config: client_id</li>
<li>Env Var: RCLONE_ZOHO_CLIENT_ID</li>
<li>Type: string</li>
<li>Default: ""</li>
</ul>
<h4 id="zoho-client-secret">--zoho-client-secret</h4>
<p>OAuth Client Secret Leave blank normally.</p>
<ul>
<li>Config: client_secret</li>
<li>Env Var: RCLONE_ZOHO_CLIENT_SECRET</li>
<li>Type: string</li>
<li>Default: ""</li>
</ul>
<h4 id="zoho-region">--zoho-region</h4>
<p>Zoho region to connect to. You'll have to use the region you organization is registered in.</p>
<ul>
@@ -19296,6 +19333,30 @@ y/e/d&gt; </code></pre>
</ul>
<h3 id="advanced-options-36">Advanced Options</h3>
<p>Here are the advanced options specific to zoho (Zoho).</p>
<h4 id="zoho-token">--zoho-token</h4>
<p>OAuth Access Token as a JSON blob.</p>
<ul>
<li>Config: token</li>
<li>Env Var: RCLONE_ZOHO_TOKEN</li>
<li>Type: string</li>
<li>Default: ""</li>
</ul>
<h4 id="zoho-auth-url">--zoho-auth-url</h4>
<p>Auth server URL. Leave blank to use the provider defaults.</p>
<ul>
<li>Config: auth_url</li>
<li>Env Var: RCLONE_ZOHO_AUTH_URL</li>
<li>Type: string</li>
<li>Default: ""</li>
</ul>
<h4 id="zoho-token-url">--zoho-token-url</h4>
<p>Token server url. Leave blank to use the provider defaults.</p>
<ul>
<li>Config: token_url</li>
<li>Env Var: RCLONE_ZOHO_TOKEN_URL</li>
<li>Type: string</li>
<li>Default: ""</li>
</ul>
<h4 id="zoho-encoding">--zoho-encoding</h4>
<p>This sets the encoding for the backend.</p>
<p>See: the <a href="https://rclone.org/overview/#encoding">encoding section in the overview</a> for more info.</p>
@@ -19824,12 +19885,68 @@ $ tree /tmp/b
<li>"error": return an error based on option value</li>
</ul>
<h1 id="changelog">Changelog</h1>
<h2 id="v1.54.1---2021-03-08">v1.54.1 - 2021-03-08</h2>
<p><a href="https://github.com/rclone/rclone/compare/v1.54.0...v1.54.1">See commits</a></p>
<ul>
<li>Bug Fixes
<ul>
<li>accounting: Fix --bwlimit when up or down is off (Nick Craig-Wood)</li>
<li>docs
<ul>
<li>Fix nesting of brackets and backticks in ftp docs (edwardxml)</li>
<li>Fix broken link in sftp page (edwardxml)</li>
<li>Fix typo in crypt.md (Romeo Kienzler)</li>
<li>Changelog: Correct link to digitalis.io (Alex JOST)</li>
<li>Replace #file-caching with #vfs-file-caching (Miron Veryanskiy)</li>
<li>Convert bogus example link to code (edwardxml)</li>
<li>Remove dead link from rc.md (edwardxml)</li>
</ul></li>
<li>rc: Sync,copy,move: document createEmptySrcDirs parameter (Nick Craig-Wood)</li>
<li>lsjson: Fix unterminated JSON in the presence of errors (Nick Craig-Wood)</li>
</ul></li>
<li>Mount
<ul>
<li>Fix mount dropping on macOS by setting --daemon-timeout 10m (Nick Craig-Wood)</li>
</ul></li>
<li>VFS
<ul>
<li>Document simultaneous usage with the same cache shouldn't be used (Nick Craig-Wood)</li>
</ul></li>
<li>B2
<ul>
<li>Automatically raise upload cutoff to avoid spurious error (Nick Craig-Wood)</li>
<li>Fix failed to create file system with application key limited to a prefix (Nick Craig-Wood)</li>
</ul></li>
<li>Drive
<ul>
<li>Refer to Shared Drives instead of Team Drives (Nick Craig-Wood)</li>
</ul></li>
<li>Dropbox
<ul>
<li>Add scopes to oauth request and optionally "members.read" (Nick Craig-Wood)</li>
</ul></li>
<li>S3
<ul>
<li>Fix failed to create file system with folder level permissions policy (Nick Craig-Wood)</li>
<li>Fix Wasabi HEAD requests returning stale data by using only 1 transport (Nick Craig-Wood)</li>
<li>Fix shared_credentials_file auth (Dmitry Chepurovskiy)</li>
<li>Add --s3-no-head to reducing costs docs (Nick Craig-Wood)</li>
</ul></li>
<li>Union
<ul>
<li>Fix mkdir at root with remote:/ (Nick Craig-Wood)</li>
</ul></li>
<li>Zoho
<ul>
<li>Fix custom client id's (buengese)</li>
</ul></li>
</ul>
<h2 id="v1.54.0---2021-02-02">v1.54.0 - 2021-02-02</h2>
<p><a href="https://github.com/rclone/rclone/compare/v1.53.0...v1.54.0">See commits</a></p>
<ul>
<li>New backends
<ul>
<li>Compression remote (experimental)(buengese)</li>
<li>Compression remote (experimental) (buengese)</li>
<li>Enterprise File Fabric (Nick Craig-Wood)
<ul>
<li>This work was sponsored by <a href="https://storagemadeeasy.com/">Storage Made Easy</a></li>
@@ -19842,8 +19959,8 @@ $ tree /tmp/b
<li>Deglobalise the config (Nick Craig-Wood)
<ul>
<li>Global config now read from the context</li>
<li>Global config can be passed into the rc</li>
<li>This work was sponsored by <a href="digitalis.io">Digitalis</a></li>
<li>This will enable passing of global config via the rc</li>
<li>This work was sponsored by <a href="https://digitalis.io/">Digitalis</a></li>
</ul></li>
<li>Add <code>--bwlimit</code> for upload and download (Nick Craig-Wood)
<ul>
@@ -19851,48 +19968,38 @@ $ tree /tmp/b
</ul></li>
<li>Enhance systemd integration (Hekmon)
<ul>
<li>log level identification</li>
<li>manual activation with flag</li>
<li>automatic systemd launch detection</li>
<li>log level identification, manual activation with flag, automatic systemd launch detection</li>
<li>Don't compile systemd log integration for non unix systems (Benjamin Gustin)</li>
</ul></li>
<li>Add a download flag to hashsum and related commands to force rclone to download and hash files locally (lostheli)</li>
<li>Add a <code>--download</code> flag to md5sum/sha1sum/hashsum to force rclone to download and hash files locally (lostheli)</li>
<li>Add <code>--progress-terminal-title</code> to print ETA to terminal title (LaSombra)</li>
<li>Make backend env vars show in help as the defaults for backend flags (Nick Craig-Wood)</li>
<li>build
<ul>
<li>Raise minimum go version to go1.12 (Nick Craig-Wood)</li>
</ul></li>
<li>check
<ul>
<li>Make the error count match up in the log message (Nick Craig-Wood)</li>
</ul></li>
<li>cmd
<ul>
<li>Add --progress-terminal-title to print ETA to terminal title (LaSombra)</li>
<li>Make backend env vars show in help as the defaults for backend flags (Nick Craig-Wood)</li>
</ul></li>
<li>dedupe
<ul>
<li>Add --by-hash to dedupe on hash not file name (Nick Craig-Wood)</li>
<li>Add --dedupe-mode list to just list dupes, changing nothing (Nick Craig-Wood)</li>
<li>Add <code>--by-hash</code> to dedupe on content hash not file name (Nick Craig-Wood)</li>
<li>Add <code>--dedupe-mode list</code> to just list dupes, changing nothing (Nick Craig-Wood)</li>
<li>Add warning if used on a remote which can't have duplicate names (Nick Craig-Wood)</li>
</ul></li>
<li>flags: Improve error message when reading environment vars (Nick Craig-Wood)</li>
<li>fs
<ul>
<li>Add Shutdown optional method for backends (Nick Craig-Wood)</li>
<li>When using --files-from check files concurrently (zhucan)</li>
<li>Accumulate stats when using --dry-run (Ingo Weiss)</li>
<li>Always show stats when using --dry-run or --interactive (Nick Craig-Wood)</li>
<li>Add support for flag --no-console on windows to hide the console window (albertony)</li>
<li>When using <code>--files-from</code> check files concurrently (zhucan)</li>
<li>Accumulate stats when using <code>--dry-run</code> (Ingo Weiss)</li>
<li>Always show stats when using <code>--dry-run</code> or <code>--interactive</code> (Nick Craig-Wood)</li>
<li>Add support for flag <code>--no-console</code> on windows to hide the console window (albertony)</li>
</ul></li>
<li>genautocomplete: Add support to output to stdout (Ingo)</li>
<li>ncdu
<ul>
<li>Highlight read errors instead of aborting (Claudio Bantaloukas)</li>
<li>Add sort by average size in directory (Adam Plánský)</li>
<li>Add toggle option for average size in directory - key 'a' (Adam Plánský)</li>
<li>Add toggle option for average s3ize in directory - key 'a' (Adam Plánský)</li>
<li>Add empty folder flag into ncdu browser (Adam Plánský)</li>
<li>Add ! (errror) and . (unreadable) file flags to go with e (empty) (Nick Craig-Wood)</li>
<li>Add <code>!</code> (errror) and <code>.</code> (unreadable) file flags to go with <code>e</code> (empty) (Nick Craig-Wood)</li>
</ul></li>
<li>obscure: Make <code>rclone osbcure -</code> ignore newline at end of line (Nick Craig-Wood)</li>
<li>operations
@@ -19919,36 +20026,32 @@ $ tree /tmp/b
</ul></li>
<li>Bug Fixes
<ul>
<li>build
<ul>
<li>Explicitly set ARM version to fix build (Nick Craig-Wood)</li>
<li>Don't explicitly set ARM version to fix ARMv5 build (Nick Craig-Wood)</li>
<li>Fix nfpm install (Nick Craig-Wood)</li>
<li>Fix docker build by upgrading ilteoood/docker_buildx (Nick Craig-Wood)</li>
<li>Temporary fix for Windows build errors (Ivan Andreev)</li>
</ul></li>
<li>fs
<ul>
<li>Fix nil pointer on copy &amp; move operations directly to remote (Anagh Kumar Baranwal)</li>
<li>Fix parsing of .. when joining remotes (Nick Craig-Wood)</li>
</ul></li>
<li>log: Fix enabling systemd logging when using --log-file (Nick Craig-Wood)</li>
<li>move: Fix data loss when moving the same object (Nick Craig-Wood)</li>
<li>log: Fix enabling systemd logging when using <code>--log-file</code> (Nick Craig-Wood)</li>
<li>check
<ul>
<li>Make the error count match up in the log message (Nick Craig-Wood)</li>
</ul></li>
<li>move: Fix data loss when source and destination are the same object (Nick Craig-Wood)</li>
<li>operations
<ul>
<li>Fix --cutof-mode hard not cutting off immediately (Nick Craig-Wood)</li>
<li>Fix --immutable error message (Nick Craig-Wood)</li>
<li>Fix <code>--cutof-mode</code> hard not cutting off immediately (Nick Craig-Wood)</li>
<li>Fix <code>--immutable</code> error message (Nick Craig-Wood)</li>
</ul></li>
<li>sync
<ul>
<li>Fix --cutoff-mode soft &amp; cautious so it doesn't end the transfer early (Nick Craig-Wood)</li>
<li>Fix --immutable errors retrying many times (Nick Craig-Wood)</li>
<li>Fix <code>--cutoff-mode</code> soft &amp; cautious so it doesn't end the transfer early (Nick Craig-Wood)</li>
<li>Fix <code>--immutable</code> errors retrying many times (Nick Craig-Wood)</li>
</ul></li>
</ul></li>
<li>Docs
<ul>
<li>Many fixes and a rewrite of the filtering docs (edwardxml)</li>
<li>Many spelling and grammar problems (Josh Soref)</li>
<li>Many spelling and grammar fixes (Josh Soref)</li>
<li>Doc fixes for commands delete, purge, rmdir, rmdirs and mount (albertony)</li>
<li>And thanks to these people for many doc fixes too numerous to list
<ul>
@@ -19963,13 +20066,11 @@ $ tree /tmp/b
<li>Update systemd status with cache stats (Hekmon)</li>
<li>Disable bazil/fuse based mount on macOS (Nick Craig-Wood)
<ul>
<li>Make mount be cmount under macOS (Nick Craig-Wood)</li>
<li>Make <code>rclone mount</code> actually run <code>rclone cmount</code> under macOS (Nick Craig-Wood)</li>
</ul></li>
<li>Implement mknod to make NFS file creation work (Nick Craig-Wood)</li>
<li>Make sure we don't call umount more than once (Nick Craig-Wood)</li>
<li>Don't call host.Umount if a signal has been received (Nick Craig-Wood)</li>
<li>More user friendly mounting as network drive on windows (albertony)</li>
<li>Cleanup OS specific option handling and documentation (albertony)</li>
<li>Detect if uid or gid are set in same option string: -o uid=123,gid=456 (albertony)</li>
<li>Don't attempt to unmount if fs has been destroyed already (Nick Craig-Wood)</li>
</ul></li>
@@ -19977,32 +20078,34 @@ $ tree /tmp/b
<ul>
<li>Fix virtual entries causing deleted files to still appear (Nick Craig-Wood)</li>
<li>Fix "file already exists" error for stale cache files (Nick Craig-Wood)</li>
<li>Fix file leaks with --vfs-cache-mode full and --buffer-size 0 (Nick Craig-Wood)</li>
<li>Fix file leaks with <code>--vfs-cache-mode</code> full and <code>--buffer-size 0</code> (Nick Craig-Wood)</li>
<li>Fix invalid cache path on windows when using :backend: as remote (albertony)</li>
</ul></li>
<li>Local
<ul>
<li>Continue listing files/folders when a circular symlink is detected (Manish Gupta)</li>
<li>New flag --local-zero-size-links to fix sync on some virtual filesystems (Riccardo Iaconelli)</li>
<li>New flag <code>--local-zero-size-links</code> to fix sync on some virtual filesystems (Riccardo Iaconelli)</li>
</ul></li>
<li>Azure Blob
<ul>
<li>Add support for service principals (James Lim)</li>
<li>Utilize streaming capabilities (Denis Neuling)</li>
<li>Update SDK to v0.13.0 and fix API breakage (Nick Craig-Wood, Mitsuo Heijo)</li>
<li>Add support for managed identities (Brad Ackerman)</li>
<li>Add examples for access tier (Bob Pusateri)</li>
<li>Utilize the streaming capabilities from the SDK for multipart uploads (Denis Neuling)</li>
<li>Fix setting of mime types (Nick Craig-Wood)</li>
<li>Fix crash when listing outside a SAS URL's root (Nick Craig-Wood)</li>
<li>Delete archive tier blobs before update if --azureblob-archive-tier-delete (Nick Craig-Wood)</li>
<li>Add support for managed identities (Brad Ackerman)</li>
<li>Delete archive tier blobs before update if <code>--azureblob-archive-tier-delete</code> (Nick Craig-Wood)</li>
<li>Fix crash on startup (Nick Craig-Wood)</li>
<li>Add examples for access tier (Bob Pusateri)</li>
<li>Fix memory usage by upgrading the SDK and implementing a TransferManager (Nick Craig-Wood)</li>
<li>Fix memory usage by upgrading the SDK to v0.13.0 and implementing a TransferManager (Nick Craig-Wood)</li>
<li>Require go1.14+ to compile due to SDK changes (Nick Craig-Wood)</li>
</ul></li>
<li>B2
<ul>
<li>Make NewObject use less expensive API calls (Nick Craig-Wood)</li>
<li>Fixed possible crash when accessing Backblaze b2 remote (lluuaapp)</li>
<li>Make NewObject use less expensive API calls (Nick Craig-Wood)
<ul>
<li>This will improve <code>--files-from</code> and <code>restic serve</code> in particular</li>
</ul></li>
<li>Fixed crash on an empty file name (lluuaapp)</li>
</ul></li>
<li>Box
<ul>
@@ -20012,12 +20115,12 @@ $ tree /tmp/b
<li>Chunker
<ul>
<li>Skip long local hashing, hash in-transit (fixes) (Ivan Andreev)</li>
<li>Set Features.ReadMimeType=false as Object.MimeType not supported (Nick Craig-Wood)</li>
<li>Set Features ReadMimeType to false as Object.MimeType not supported (Nick Craig-Wood)</li>
<li>Fix case-insensitive NewObject, test metadata detection (Ivan Andreev)</li>
</ul></li>
<li>Drive
<ul>
<li>Implement "rclone backend copyid" command for copying files by ID (Nick Craig-Wood)</li>
<li>Implement <code>rclone backend copyid</code> command for copying files by ID (Nick Craig-Wood)</li>
<li>Added flag <code>--drive-stop-on-download-limit</code> to stop transfers when the download limit is exceeded (Anagh Kumar Baranwal)</li>
<li>Implement CleanUp workaround for team drives (buengese)</li>
<li>Allow shortcut resolution and creation to be retried (Nick Craig-Wood)</li>
@@ -20027,45 +20130,44 @@ $ tree /tmp/b
<li>Dropbox
<ul>
<li>Add support for viewing shared files and folders (buengese)</li>
<li>Implement IDer (buengese)</li>
<li>Set Features.ReadMimeType=false as Object.MimeType not supported (Nick Craig-Wood)</li>
<li>Tidy repeated error message (Nick Craig-Wood)</li>
<li>Enable short lived access tokens (Nick Craig-Wood)</li>
<li>Implement IDer on Objects so <code>rclone lsf</code> etc can read the IDs (buengese)</li>
<li>Set Features ReadMimeType to false as Object.MimeType not supported (Nick Craig-Wood)</li>
<li>Make malformed_path errors from too long files not retriable (Nick Craig-Wood)</li>
<li>Test file name length before upload to fix upload loop (Nick Craig-Wood)</li>
<li>Enable short lived access tokens (Nick Craig-Wood)</li>
</ul></li>
<li>Fichier
<ul>
<li>Set Features.ReadMimeType=true as Object.MimeType is supported (Nick Craig-Wood)</li>
<li>Set Features ReadMimeType to true as Object.MimeType is supported (Nick Craig-Wood)</li>
</ul></li>
<li>FTP
<ul>
<li>Add --ftp-disable-msld option to ignore MLSD for really old servers (Nick Craig-Wood)</li>
<li>Make --tpslimit apply (Nick Craig-Wood)</li>
<li>Add <code>--ftp-disable-msld</code> option to ignore MLSD for really old servers (Nick Craig-Wood)</li>
<li>Make <code>--tpslimit apply</code> (Nick Craig-Wood)</li>
</ul></li>
<li>Google Cloud Storage
<ul>
<li>Storage class object header support (Laurens Janssen)</li>
<li>Fix anonymous client to use rclone's HTTP client (Nick Craig-Wood)</li>
<li>Fix Entry doesn't belong in directory "" (same as directory) - ignoring (Nick Craig-Wood)</li>
<li>Fix <code>Entry doesn't belong in directory "" (same as directory) - ignoring</code> (Nick Craig-Wood)</li>
</ul></li>
<li>Googlephotos
<ul>
<li>New flag --gphotos-include-archived (Nicolas Rueff)</li>
<li>New flag <code>--gphotos-include-archived</code> to show archived photos as well (Nicolas Rueff)</li>
</ul></li>
<li>Jottacloud
<ul>
<li>Don't erroniously report support for writing mime types (buengese)</li>
<li>Add support for Telia Cloud (#4930) (Patrik Nordlén)</li>
<li>Don't erroneously report support for writing mime types (buengese)</li>
<li>Add support for Telia Cloud (Patrik Nordlén)</li>
</ul></li>
<li>Mailru
<ul>
<li>Accept special folders eg camera-upload (Ivan Andreev)</li>
<li>Avoid prehashing of large local files (Ivan Andreev)</li>
<li>Fix uploads after recent changes on server (Ivan Andreev)</li>
<li>Fix range requests after June 2020 changes on server (Ivan Andreev)</li>
<li>Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev)</li>
<li>Remove deprecated protocol quirks (Ivan Andreev)</li>
<li>Accept special folders eg camera-upload (Ivan Andreev)</li>
<li>Avoid prehashing of large local files (Ivan Andreev)</li>
</ul></li>
<li>Memory
<ul>
@@ -20073,14 +20175,14 @@ $ tree /tmp/b
</ul></li>
<li>Onedrive
<ul>
<li>Add support for china region operated by 21vianet and other regional suppliers (#4963) (NyaMisty)</li>
<li>Add support for China region operated by 21vianet and other regional suppliers (NyaMisty)</li>
<li>Warn on gateway timeout errors (Nick Craig-Wood)</li>
<li>Fall back to normal copy if server-side copy unavailable (#4903) (Alex Chen)</li>
<li>Fall back to normal copy if server-side copy unavailable (Alex Chen)</li>
<li>Fix server-side copy completely disabled on OneDrive for Business (Cnly)</li>
<li>(business only) workaround to replace existing file on server-side copy (#4904) (Alex Chen)</li>
<li>(business only) workaround to replace existing file on server-side copy (Alex Chen)</li>
<li>Enhance link creation with expiry, scope, type and password (Nick Craig-Wood)</li>
<li>Remove % and # from the set of encoded characters (#4909) (Alex Chen)</li>
<li>Support addressing site by server-relative URL (#4761) (kice)</li>
<li>Remove % and # from the set of encoded characters (Alex Chen)</li>
<li>Support addressing site by server-relative URL (kice)</li>
</ul></li>
<li>Opendrive
<ul>
@@ -20102,14 +20204,16 @@ $ tree /tmp/b
<li>S3
<ul>
<li>Added <code>--s3-disable-http2</code> to disable http/2 (Anagh Kumar Baranwal)</li>
<li>Complete SSE-C implementation (Nick Craig-Wood)</li>
<li>Complete SSE-C implementation (Nick Craig-Wood)
<ul>
<li>Fix hashes on small files with AWS:KMS and SSE-C (Nick Craig-Wood)</li>
<li>Add MD5 metadata to objects uploaded with SSE-AWS/SSE-C (Nick Craig-Wood)</li>
</ul></li>
<li>Add <code>--s3-no-head parameter</code> to minimise transactions on upload (Nick Craig-Wood)</li>
<li>Update docs with a Reducing Costs section (Nick Craig-Wood)</li>
<li>Added error handling for error code 429 indicating too many requests (Anagh Kumar Baranwal)</li>
<li>Add requester pays option (kelv)</li>
<li>Fix copy multipart with v2 auth failing with 'SignatureDoesNotMatch' (Louis Koo)</li>
<li>Add --s3-no-head parameter to minimise transactions on upload (Nick Craig-Wood)</li>
</ul></li>
<li>SFTP
<ul>
@@ -20119,8 +20223,8 @@ $ tree /tmp/b
<li>Remember entered password in AskPass mode (Stephen Harris)</li>
<li>Implement Shutdown method (Nick Craig-Wood)</li>
<li>Implement keyboard interactive authentication (Nick Craig-Wood)</li>
<li>Make --tpslimit apply (Nick Craig-Wood)</li>
<li>Implement --sftp-use-fstat (Nick Craig-Wood)</li>
<li>Make <code>--tpslimit</code> apply (Nick Craig-Wood)</li>
<li>Implement <code>--sftp-use-fstat</code> for unusual SFTP servers (Nick Craig-Wood)</li>
</ul></li>
<li>Sugarsync
<ul>
@@ -20130,7 +20234,7 @@ $ tree /tmp/b
<li>Swift
<ul>
<li>Fix deletion of parts of Static Large Object (SLO) (Nguyễn Hữu Luân)</li>
<li>Ensure partially uploaded large files are uploaded unless --swift-leave-parts-on-error (Nguyễn Hữu Luân)</li>
<li>Ensure partially uploaded large files are uploaded unless <code>--swift-leave-parts-on-error</code> (Nguyễn Hữu Luân)</li>
</ul></li>
<li>Tardigrade
<ul>
@@ -20142,7 +20246,7 @@ $ tree /tmp/b
</ul></li>
<li>Yandex
<ul>
<li>Set Features.WriteMimeType=false as Yandex ignores mime types (Nick Craig-Wood)</li>
<li>Set Features WriteMimeType to false as Yandex ignores mime types (Nick Craig-Wood)</li>
</ul></li>
</ul>
<h2 id="v1.53.4---2021-01-20">v1.53.4 - 2021-01-20</h2>

345
MANUAL.md generated
View File

@@ -1,6 +1,6 @@
% rclone(1) User Manual
% Nick Craig-Wood
% Feb 02, 2021
% Mar 08, 2021
# Rclone syncs your files to cloud storage
@@ -2928,7 +2928,7 @@ Without the use of `--vfs-cache-mode` this can only write files
sequentially, it can only seek when reading. This means that many
applications won't work with their files on an rclone mount without
`--vfs-cache-mode writes` or `--vfs-cache-mode full`.
See the [File Caching](#file-caching) section for more info.
See the [VFS File Caching](#vfs-file-caching) section for more info.
The bucket based remotes (e.g. Swift, S3, Google Compute Storage, B2,
Hubic) do not support the concept of empty directories, so empty
@@ -2943,7 +2943,7 @@ File systems expect things to be 100% reliable, whereas cloud storage
systems are a long way from 100% reliable. The rclone sync/copy
commands cope with this with lots of retries. However rclone mount
can't use retries in the same way without making local copies of the
uploads. Look at the [file caching](#file-caching)
uploads. Look at the [VFS File Caching](#vfs-file-caching)
for solutions to make mount more reliable.
## Attribute caching
@@ -3108,6 +3108,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write
@@ -3832,6 +3839,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write
@@ -4145,6 +4159,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write
@@ -4616,6 +4637,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write
@@ -5153,6 +5181,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write
@@ -5631,6 +5666,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write
@@ -8636,7 +8678,7 @@ not list `dir3`, `file3` or `.ignore`.
## Common pitfalls
The most frequent filter support issues on
the [rclone forum](https://https://forum.rclone.org/) are:
the [rclone forum](https://forum.rclone.org/) are:
* Not using paths relative to the root of the remote
* Not using `/` to match from the root of a remote
@@ -10031,6 +10073,7 @@ This takes the following parameters
- srcFs - a remote name string e.g. "drive:src" for the source
- dstFs - a remote name string e.g. "drive:dst" for the destination
- createEmptySrcDirs - create empty src directories on destination if set
See the [copy command](https://rclone.org/commands/rclone_copy/) command for more information on the above.
@@ -10043,6 +10086,7 @@ This takes the following parameters
- srcFs - a remote name string e.g. "drive:src" for the source
- dstFs - a remote name string e.g. "drive:dst" for the destination
- createEmptySrcDirs - create empty src directories on destination if set
- deleteEmptySrcDirs - delete empty src directories if set
@@ -10056,6 +10100,7 @@ This takes the following parameters
- srcFs - a remote name string e.g. "drive:src" for the source
- dstFs - a remote name string e.g. "drive:dst" for the destination
- createEmptySrcDirs - create empty src directories on destination if set
See the [sync command](https://rclone.org/commands/rclone_sync/) command for more information on the above.
@@ -10954,7 +10999,7 @@ These flags are available for every command.
--use-json-log Use json log format.
--use-mmap Use mmap allocator (see docs).
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string. The default is rclone/ version (default "rclone/v1.54.0")
--user-agent string Set the user-agent to a specified string. The default is rclone/ version (default "rclone/v1.54.1")
-v, --verbose count Print lots more stuff (repeat for more)
```
@@ -11085,7 +11130,7 @@ and may be set in the config file.
--drive-starred-only Only show files that are starred.
--drive-stop-on-download-limit Make download limit errors be fatal
--drive-stop-on-upload-limit Make upload limit errors be fatal
--drive-team-drive string ID of the Team Drive
--drive-team-drive string ID of the Shared Drive (Team Drive)
--drive-token string OAuth Access Token as a JSON blob.
--drive-token-url string Token server url.
--drive-trashed-only Only show files that are in the trash.
@@ -11366,8 +11411,13 @@ and may be set in the config file.
--yandex-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,InvalidUtf8,Dot)
--yandex-token string OAuth Access Token as a JSON blob.
--yandex-token-url string Token server url.
--zoho-auth-url string Auth server URL.
--zoho-client-id string OAuth Client Id
--zoho-client-secret string OAuth Client Secret
--zoho-encoding MultiEncoder This sets the encoding for the backend. (default Del,Ctl,InvalidUtf8)
--zoho-region string Zoho region to connect to. You'll have to use the region you organization is registered in.
--zoho-token string OAuth Access Token as a JSON blob.
--zoho-token-url string Token server url.
```
1Fichier
@@ -12281,6 +12331,16 @@ You'd then do a full `rclone sync` less often.
Note that `--fast-list` isn't required in the top-up sync.
#### Avoiding HEAD requests after PUT
By default rclone will HEAD every object it uploads. It does this to
check the object got uploaded correctly.
You can disable this with the [--s3-no-head](#s3-no-head) option - see
there for more details.
Setting this flag increases the chance for undetected upload failures.
### Hashes ###
For small objects which weren't uploaded as multipart uploads (objects
@@ -15706,7 +15766,7 @@ These URLs are used by Plex internally to connect to the Plex server securely.
The format for these URLs is the following:
https://ip-with-dots-replaced.server-hash.plex.direct:32400/
`https://ip-with-dots-replaced.server-hash.plex.direct:32400/`
The `ip-with-dots-replaced` part can be any IPv4 address, where the dots
have been replaced with dashes, e.g. `127.0.0.1` becomes `127-0-0-1`.
@@ -16870,7 +16930,7 @@ as you would with any other remote, e.g. `rclone copy D:\docs secret:\docs`,
and rclone will encrypt and decrypt as needed on the fly.
If you access the wrapped remote `remote:path` directly you will bypass
the encryption, and anything you read will be in encrypted form, and
anything you write will be undencrypted. To avoid issues it is best to
anything you write will be unencrypted. To avoid issues it is best to
configure a dedicated path for encrypted content, and access it
exclusively through a crypt remote.
@@ -17821,6 +17881,21 @@ memory. It can be set smaller if you are tight on memory.
Impersonate this user when using a business account.
Note that if you want to use impersonate, you should make sure this
flag is set when running "rclone config" as this will cause rclone to
request the "members.read" scope which it won't normally. This is
needed to lookup a members email address into the internal ID that
dropbox uses in the API.
Using the "members.read" scope will require a Dropbox Team Admin
to approve during the OAuth flow.
You will have to use your own App (setting your own client_id and
client_secret) to use this option as currently rclone's default set of
permissions doesn't include "members.read". This can be added once
v1.55 or later is in use everywhere.
- Config: impersonate
- Env Var: RCLONE_DROPBOX_IMPERSONATE
- Type: string
@@ -18277,8 +18352,8 @@ excess files in the directory.
Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to
be enabled in the FTP backend config for the remote, or with
`[--ftp-tls]{#ftp-tls}`. The default FTPS port is `990`, not `21` and
can be set with `[--ftp-port]{#ftp-port}`.
[`--ftp-tls`](#ftp-tls). The default FTPS port is `990`, not `21` and
can be set with [`--ftp-port`](#ftp-port).
### Standard Options
@@ -19042,7 +19117,7 @@ If your browser doesn't open automatically go to the following link: http://127.
Log in and authorize rclone for access
Waiting for code...
Got code
Configure this as a team drive?
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No
y/n> n
@@ -19249,23 +19324,24 @@ Note: in case you configured a specific root folder on gdrive and rclone is unab
`rclone -v foo@example.com lsf gdrive:backup`
### Team drives ###
### Shared drives (team drives) ###
If you want to configure the remote to point to a Google Team Drive
then answer `y` to the question `Configure this as a team drive?`.
If you want to configure the remote to point to a Google Shared Drive
(previously known as Team Drives) then answer `y` to the question
`Configure this as a Shared Drive (Team Drive)?`.
This will fetch the list of Team Drives from google and allow you to
configure which one you want to use. You can also type in a team
drive ID if you prefer.
This will fetch the list of Shared Drives from google and allow you to
configure which one you want to use. You can also type in a Shared
Drive ID if you prefer.
For example:
```
Configure this as a team drive?
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No
y/n> y
Fetching team drive list...
Fetching Shared Drive list...
Choose a number from below, or type in your own value
1 / Rclone Test
\ "xxxxxxxxxxxxxxxxxxxx"
@@ -19273,7 +19349,7 @@ Choose a number from below, or type in your own value
\ "yyyyyyyyyyyyyyyyyyyy"
3 / Rclone Test 3
\ "zzzzzzzzzzzzzzzzzzzz"
Enter a Team Drive ID> 1
Enter a Shared Drive ID> 1
--------------------
[remote]
client_id =
@@ -19644,7 +19720,7 @@ Needed only if you want use SA instead of interactive login.
#### --drive-team-drive
ID of the Team Drive
ID of the Shared Drive (Team Drive)
- Config: team_drive
- Env Var: RCLONE_DRIVE_TEAM_DRIVE
@@ -20107,11 +20183,11 @@ Options:
#### drives
List the shared drives available to this account
List the Shared Drives available to this account
rclone backend drives remote: [options] [<arguments>+]
This command lists the shared drives (teamdrives) available to this
This command lists the Shared Drives (Team Drives) available to this
account.
Usage:
@@ -25747,8 +25823,8 @@ The Go SSH library disables the use of the aes128-cbc cipher by
default, due to security concerns. This can be re-enabled on a
per-connection basis by setting the `use_insecure_cipher` setting in
the configuration file to `true`. Further details on the insecurity of
this cipher can be found [in this paper]
(http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).
this cipher can be found
[in this paper](http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).
SFTP isn't supported under plan9 until [this
issue](https://github.com/pkg/sftp/issues/156) is fixed.
@@ -27199,6 +27275,26 @@ from filenames during upload.
Here are the standard options specific to zoho (Zoho).
#### --zoho-client-id
OAuth Client Id
Leave blank normally.
- Config: client_id
- Env Var: RCLONE_ZOHO_CLIENT_ID
- Type: string
- Default: ""
#### --zoho-client-secret
OAuth Client Secret
Leave blank normally.
- Config: client_secret
- Env Var: RCLONE_ZOHO_CLIENT_SECRET
- Type: string
- Default: ""
#### --zoho-region
Zoho region to connect to. You'll have to use the region you organization is registered in.
@@ -27221,6 +27317,35 @@ Zoho region to connect to. You'll have to use the region you organization is reg
Here are the advanced options specific to zoho (Zoho).
#### --zoho-token
OAuth Access Token as a JSON blob.
- Config: token
- Env Var: RCLONE_ZOHO_TOKEN
- Type: string
- Default: ""
#### --zoho-auth-url
Auth server URL.
Leave blank to use the provider defaults.
- Config: auth_url
- Env Var: RCLONE_ZOHO_AUTH_URL
- Type: string
- Default: ""
#### --zoho-token-url
Token server url.
Leave blank to use the provider defaults.
- Config: token_url
- Env Var: RCLONE_ZOHO_TOKEN_URL
- Type: string
- Default: ""
#### --zoho-encoding
This sets the encoding for the backend.
@@ -27752,12 +27877,49 @@ Options:
# Changelog
## v1.54.1 - 2021-03-08
[See commits](https://github.com/rclone/rclone/compare/v1.54.0...v1.54.1)
* Bug Fixes
* accounting: Fix --bwlimit when up or down is off (Nick Craig-Wood)
* docs
* Fix nesting of brackets and backticks in ftp docs (edwardxml)
* Fix broken link in sftp page (edwardxml)
* Fix typo in crypt.md (Romeo Kienzler)
* Changelog: Correct link to digitalis.io (Alex JOST)
* Replace #file-caching with #vfs-file-caching (Miron Veryanskiy)
* Convert bogus example link to code (edwardxml)
* Remove dead link from rc.md (edwardxml)
* rc: Sync,copy,move: document createEmptySrcDirs parameter (Nick Craig-Wood)
* lsjson: Fix unterminated JSON in the presence of errors (Nick Craig-Wood)
* Mount
* Fix mount dropping on macOS by setting --daemon-timeout 10m (Nick Craig-Wood)
* VFS
* Document simultaneous usage with the same cache shouldn't be used (Nick Craig-Wood)
* B2
* Automatically raise upload cutoff to avoid spurious error (Nick Craig-Wood)
* Fix failed to create file system with application key limited to a prefix (Nick Craig-Wood)
* Drive
* Refer to Shared Drives instead of Team Drives (Nick Craig-Wood)
* Dropbox
* Add scopes to oauth request and optionally "members.read" (Nick Craig-Wood)
* S3
* Fix failed to create file system with folder level permissions policy (Nick Craig-Wood)
* Fix Wasabi HEAD requests returning stale data by using only 1 transport (Nick Craig-Wood)
* Fix shared_credentials_file auth (Dmitry Chepurovskiy)
* Add --s3-no-head to reducing costs docs (Nick Craig-Wood)
* Union
* Fix mkdir at root with remote:/ (Nick Craig-Wood)
* Zoho
* Fix custom client id's (buengese)
## v1.54.0 - 2021-02-02
[See commits](https://github.com/rclone/rclone/compare/v1.53.0...v1.54.0)
* New backends
* Compression remote (experimental)(buengese)
* Compression remote (experimental) (buengese)
* Enterprise File Fabric (Nick Craig-Wood)
* This work was sponsored by [Storage Made Easy](https://storagemadeeasy.com/)
* HDFS (Hadoop Distributed File System) (Yury Stankevich)
@@ -27765,41 +27927,35 @@ Options:
* New Features
* Deglobalise the config (Nick Craig-Wood)
* Global config now read from the context
* Global config can be passed into the rc
* This work was sponsored by [Digitalis](digitalis.io)
* This will enable passing of global config via the rc
* This work was sponsored by [Digitalis](https://digitalis.io/)
* Add `--bwlimit` for upload and download (Nick Craig-Wood)
* Obey bwlimit in http Transport for better limiting
* Enhance systemd integration (Hekmon)
* log level identification
* manual activation with flag
* automatic systemd launch detection
* log level identification, manual activation with flag, automatic systemd launch detection
* Don't compile systemd log integration for non unix systems (Benjamin Gustin)
* Add a download flag to hashsum and related commands to force rclone to download and hash files locally (lostheli)
* Add a `--download` flag to md5sum/sha1sum/hashsum to force rclone to download and hash files locally (lostheli)
* Add `--progress-terminal-title` to print ETA to terminal title (LaSombra)
* Make backend env vars show in help as the defaults for backend flags (Nick Craig-Wood)
* build
* Raise minimum go version to go1.12 (Nick Craig-Wood)
* check
* Make the error count match up in the log message (Nick Craig-Wood)
* cmd
* Add --progress-terminal-title to print ETA to terminal title (LaSombra)
* Make backend env vars show in help as the defaults for backend flags (Nick Craig-Wood)
* dedupe
* Add --by-hash to dedupe on hash not file name (Nick Craig-Wood)
* Add --dedupe-mode list to just list dupes, changing nothing (Nick Craig-Wood)
* Add `--by-hash` to dedupe on content hash not file name (Nick Craig-Wood)
* Add `--dedupe-mode list` to just list dupes, changing nothing (Nick Craig-Wood)
* Add warning if used on a remote which can't have duplicate names (Nick Craig-Wood)
* flags: Improve error message when reading environment vars (Nick Craig-Wood)
* fs
* Add Shutdown optional method for backends (Nick Craig-Wood)
* When using --files-from check files concurrently (zhucan)
* Accumulate stats when using --dry-run (Ingo Weiss)
* Always show stats when using --dry-run or --interactive (Nick Craig-Wood)
* Add support for flag --no-console on windows to hide the console window (albertony)
* When using `--files-from` check files concurrently (zhucan)
* Accumulate stats when using `--dry-run` (Ingo Weiss)
* Always show stats when using `--dry-run` or `--interactive` (Nick Craig-Wood)
* Add support for flag `--no-console` on windows to hide the console window (albertony)
* genautocomplete: Add support to output to stdout (Ingo)
* ncdu
* Highlight read errors instead of aborting (Claudio Bantaloukas)
* Add sort by average size in directory (Adam Plánský)
* Add toggle option for average size in directory - key 'a' (Adam Plánský)
* Add toggle option for average s3ize in directory - key 'a' (Adam Plánský)
* Add empty folder flag into ncdu browser (Adam Plánský)
* Add ! (errror) and . (unreadable) file flags to go with e (empty) (Nick Craig-Wood)
* Add `!` (errror) and `.` (unreadable) file flags to go with `e` (empty) (Nick Craig-Wood)
* obscure: Make `rclone osbcure -` ignore newline at end of line (Nick Craig-Wood)
* operations
* Add logs when need to upload files to set mod times (Nick Craig-Wood)
@@ -27817,26 +27973,22 @@ Options:
* Prompt user for updating webui if an update is available (Chaitanya Bankanhal)
* Fix plugins initialization (negative0)
* Bug Fixes
* build
* Explicitly set ARM version to fix build (Nick Craig-Wood)
* Don't explicitly set ARM version to fix ARMv5 build (Nick Craig-Wood)
* Fix nfpm install (Nick Craig-Wood)
* Fix docker build by upgrading ilteoood/docker_buildx (Nick Craig-Wood)
* Temporary fix for Windows build errors (Ivan Andreev)
* fs
* Fix nil pointer on copy & move operations directly to remote (Anagh Kumar Baranwal)
* Fix parsing of .. when joining remotes (Nick Craig-Wood)
* log: Fix enabling systemd logging when using --log-file (Nick Craig-Wood)
* move: Fix data loss when moving the same object (Nick Craig-Wood)
* log: Fix enabling systemd logging when using `--log-file` (Nick Craig-Wood)
* check
* Make the error count match up in the log message (Nick Craig-Wood)
* move: Fix data loss when source and destination are the same object (Nick Craig-Wood)
* operations
* Fix --cutof-mode hard not cutting off immediately (Nick Craig-Wood)
* Fix --immutable error message (Nick Craig-Wood)
* Fix `--cutof-mode` hard not cutting off immediately (Nick Craig-Wood)
* Fix `--immutable` error message (Nick Craig-Wood)
* sync
* Fix --cutoff-mode soft & cautious so it doesn't end the transfer early (Nick Craig-Wood)
* Fix --immutable errors retrying many times (Nick Craig-Wood)
* Fix `--cutoff-mode` soft & cautious so it doesn't end the transfer early (Nick Craig-Wood)
* Fix `--immutable` errors retrying many times (Nick Craig-Wood)
* Docs
* Many fixes and a rewrite of the filtering docs (edwardxml)
* Many spelling and grammar problems (Josh Soref)
* Many spelling and grammar fixes (Josh Soref)
* Doc fixes for commands delete, purge, rmdir, rmdirs and mount (albertony)
* And thanks to these people for many doc fixes too numerous to list
* Ameer Dawood, Antoine GIRARD, Bob Bagwill, Christopher Stewart
@@ -27846,46 +27998,44 @@ Options:
* Mount
* Update systemd status with cache stats (Hekmon)
* Disable bazil/fuse based mount on macOS (Nick Craig-Wood)
* Make mount be cmount under macOS (Nick Craig-Wood)
* Make `rclone mount` actually run `rclone cmount` under macOS (Nick Craig-Wood)
* Implement mknod to make NFS file creation work (Nick Craig-Wood)
* Make sure we don't call umount more than once (Nick Craig-Wood)
* Don't call host.Umount if a signal has been received (Nick Craig-Wood)
* More user friendly mounting as network drive on windows (albertony)
* Cleanup OS specific option handling and documentation (albertony)
* Detect if uid or gid are set in same option string: -o uid=123,gid=456 (albertony)
* Don't attempt to unmount if fs has been destroyed already (Nick Craig-Wood)
* VFS
* Fix virtual entries causing deleted files to still appear (Nick Craig-Wood)
* Fix "file already exists" error for stale cache files (Nick Craig-Wood)
* Fix file leaks with --vfs-cache-mode full and --buffer-size 0 (Nick Craig-Wood)
* Fix file leaks with `--vfs-cache-mode` full and `--buffer-size 0` (Nick Craig-Wood)
* Fix invalid cache path on windows when using :backend: as remote (albertony)
* Local
* Continue listing files/folders when a circular symlink is detected (Manish Gupta)
* New flag --local-zero-size-links to fix sync on some virtual filesystems (Riccardo Iaconelli)
* New flag `--local-zero-size-links` to fix sync on some virtual filesystems (Riccardo Iaconelli)
* Azure Blob
* Add support for service principals (James Lim)
* Utilize streaming capabilities (Denis Neuling)
* Update SDK to v0.13.0 and fix API breakage (Nick Craig-Wood, Mitsuo Heijo)
* Add support for managed identities (Brad Ackerman)
* Add examples for access tier (Bob Pusateri)
* Utilize the streaming capabilities from the SDK for multipart uploads (Denis Neuling)
* Fix setting of mime types (Nick Craig-Wood)
* Fix crash when listing outside a SAS URL's root (Nick Craig-Wood)
* Delete archive tier blobs before update if --azureblob-archive-tier-delete (Nick Craig-Wood)
* Add support for managed identities (Brad Ackerman)
* Delete archive tier blobs before update if `--azureblob-archive-tier-delete` (Nick Craig-Wood)
* Fix crash on startup (Nick Craig-Wood)
* Add examples for access tier (Bob Pusateri)
* Fix memory usage by upgrading the SDK and implementing a TransferManager (Nick Craig-Wood)
* Fix memory usage by upgrading the SDK to v0.13.0 and implementing a TransferManager (Nick Craig-Wood)
* Require go1.14+ to compile due to SDK changes (Nick Craig-Wood)
* B2
* Make NewObject use less expensive API calls (Nick Craig-Wood)
* Fixed possible crash when accessing Backblaze b2 remote (lluuaapp)
* This will improve `--files-from` and `restic serve` in particular
* Fixed crash on an empty file name (lluuaapp)
* Box
* Fix NewObject for files that differ in case (Nick Craig-Wood)
* Fix finding directories in a case insentive way (Nick Craig-Wood)
* Chunker
* Skip long local hashing, hash in-transit (fixes) (Ivan Andreev)
* Set Features.ReadMimeType=false as Object.MimeType not supported (Nick Craig-Wood)
* Set Features ReadMimeType to false as Object.MimeType not supported (Nick Craig-Wood)
* Fix case-insensitive NewObject, test metadata detection (Ivan Andreev)
* Drive
* Implement "rclone backend copyid" command for copying files by ID (Nick Craig-Wood)
* Implement `rclone backend copyid` command for copying files by ID (Nick Craig-Wood)
* Added flag `--drive-stop-on-download-limit` to stop transfers when the download limit is exceeded (Anagh Kumar Baranwal)
* Implement CleanUp workaround for team drives (buengese)
* Allow shortcut resolution and creation to be retried (Nick Craig-Wood)
@@ -27893,44 +28043,43 @@ Options:
* Add xdg office icons to xdg desktop files (Pau Rodriguez-Estivill)
* Dropbox
* Add support for viewing shared files and folders (buengese)
* Implement IDer (buengese)
* Set Features.ReadMimeType=false as Object.MimeType not supported (Nick Craig-Wood)
* Tidy repeated error message (Nick Craig-Wood)
* Enable short lived access tokens (Nick Craig-Wood)
* Implement IDer on Objects so `rclone lsf` etc can read the IDs (buengese)
* Set Features ReadMimeType to false as Object.MimeType not supported (Nick Craig-Wood)
* Make malformed_path errors from too long files not retriable (Nick Craig-Wood)
* Test file name length before upload to fix upload loop (Nick Craig-Wood)
* Enable short lived access tokens (Nick Craig-Wood)
* Fichier
* Set Features.ReadMimeType=true as Object.MimeType is supported (Nick Craig-Wood)
* Set Features ReadMimeType to true as Object.MimeType is supported (Nick Craig-Wood)
* FTP
* Add --ftp-disable-msld option to ignore MLSD for really old servers (Nick Craig-Wood)
* Make --tpslimit apply (Nick Craig-Wood)
* Add `--ftp-disable-msld` option to ignore MLSD for really old servers (Nick Craig-Wood)
* Make `--tpslimit apply` (Nick Craig-Wood)
* Google Cloud Storage
* Storage class object header support (Laurens Janssen)
* Fix anonymous client to use rclone's HTTP client (Nick Craig-Wood)
* Fix Entry doesn't belong in directory "" (same as directory) - ignoring (Nick Craig-Wood)
* Fix `Entry doesn't belong in directory "" (same as directory) - ignoring` (Nick Craig-Wood)
* Googlephotos
* New flag --gphotos-include-archived (Nicolas Rueff)
* New flag `--gphotos-include-archived` to show archived photos as well (Nicolas Rueff)
* Jottacloud
* Don't erroniously report support for writing mime types (buengese)
* Add support for Telia Cloud (#4930) (Patrik Nordlén)
* Don't erroneously report support for writing mime types (buengese)
* Add support for Telia Cloud (Patrik Nordlén)
* Mailru
* Accept special folders eg camera-upload (Ivan Andreev)
* Avoid prehashing of large local files (Ivan Andreev)
* Fix uploads after recent changes on server (Ivan Andreev)
* Fix range requests after June 2020 changes on server (Ivan Andreev)
* Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev)
* Remove deprecated protocol quirks (Ivan Andreev)
* Accept special folders eg camera-upload (Ivan Andreev)
* Avoid prehashing of large local files (Ivan Andreev)
* Memory
* Fix setting of mime types (Nick Craig-Wood)
* Onedrive
* Add support for china region operated by 21vianet and other regional suppliers (#4963) (NyaMisty)
* Add support for China region operated by 21vianet and other regional suppliers (NyaMisty)
* Warn on gateway timeout errors (Nick Craig-Wood)
* Fall back to normal copy if server-side copy unavailable (#4903) (Alex Chen)
* Fall back to normal copy if server-side copy unavailable (Alex Chen)
* Fix server-side copy completely disabled on OneDrive for Business (Cnly)
* (business only) workaround to replace existing file on server-side copy (#4904) (Alex Chen)
* (business only) workaround to replace existing file on server-side copy (Alex Chen)
* Enhance link creation with expiry, scope, type and password (Nick Craig-Wood)
* Remove % and # from the set of encoded characters (#4909) (Alex Chen)
* Support addressing site by server-relative URL (#4761) (kice)
* Remove % and # from the set of encoded characters (Alex Chen)
* Support addressing site by server-relative URL (kice)
* Opendrive
* Fix finding directories in a case insensitive way (Nick Craig-Wood)
* Pcloud
@@ -27943,13 +28092,13 @@ Options:
* S3
* Added `--s3-disable-http2` to disable http/2 (Anagh Kumar Baranwal)
* Complete SSE-C implementation (Nick Craig-Wood)
* Fix hashes on small files with AWS:KMS and SSE-C (Nick Craig-Wood)
* Add MD5 metadata to objects uploaded with SSE-AWS/SSE-C (Nick Craig-Wood)
* Fix hashes on small files with AWS:KMS and SSE-C (Nick Craig-Wood)
* Add MD5 metadata to objects uploaded with SSE-AWS/SSE-C (Nick Craig-Wood)
* Add `--s3-no-head parameter` to minimise transactions on upload (Nick Craig-Wood)
* Update docs with a Reducing Costs section (Nick Craig-Wood)
* Added error handling for error code 429 indicating too many requests (Anagh Kumar Baranwal)
* Add requester pays option (kelv)
* Fix copy multipart with v2 auth failing with 'SignatureDoesNotMatch' (Louis Koo)
* Add --s3-no-head parameter to minimise transactions on upload (Nick Craig-Wood)
* SFTP
* Allow cert based auth via optional pubkey (Stephen Harris)
* Allow user to optionally check server hosts key to add security (Stephen Harris)
@@ -27957,20 +28106,20 @@ Options:
* Remember entered password in AskPass mode (Stephen Harris)
* Implement Shutdown method (Nick Craig-Wood)
* Implement keyboard interactive authentication (Nick Craig-Wood)
* Make --tpslimit apply (Nick Craig-Wood)
* Implement --sftp-use-fstat (Nick Craig-Wood)
* Make `--tpslimit` apply (Nick Craig-Wood)
* Implement `--sftp-use-fstat` for unusual SFTP servers (Nick Craig-Wood)
* Sugarsync
* Fix NewObject for files that differ in case (Nick Craig-Wood)
* Fix finding directories in a case insentive way (Nick Craig-Wood)
* Swift
* Fix deletion of parts of Static Large Object (SLO) (Nguyễn Hữu Luân)
* Ensure partially uploaded large files are uploaded unless --swift-leave-parts-on-error (Nguyễn Hữu Luân)
* Ensure partially uploaded large files are uploaded unless `--swift-leave-parts-on-error` (Nguyễn Hữu Luân)
* Tardigrade
* Upgrade to uplink v1.4.1 (Caleb Case)
* WebDAV
* Updated docs to show streaming to nextcloud is working (Durval Menezes)
* Yandex
* Set Features.WriteMimeType=false as Yandex ignores mime types (Nick Craig-Wood)
* Set Features WriteMimeType to false as Yandex ignores mime types (Nick Craig-Wood)
## v1.53.4 - 2021-01-20

358
MANUAL.txt generated
View File

@@ -1,6 +1,6 @@
rclone(1) User Manual
Nick Craig-Wood
Feb 02, 2021
Mar 08, 2021
@@ -2962,8 +2962,8 @@ Limitations
Without the use of --vfs-cache-mode this can only write files
sequentially, it can only seek when reading. This means that many
applications won't work with their files on an rclone mount without
--vfs-cache-mode writes or --vfs-cache-mode full. See the File Caching
section for more info.
--vfs-cache-mode writes or --vfs-cache-mode full. See the VFS File
Caching section for more info.
The bucket based remotes (e.g. Swift, S3, Google Compute Storage, B2,
Hubic) do not support the concept of empty directories, so empty
@@ -2979,7 +2979,7 @@ File systems expect things to be 100% reliable, whereas cloud storage
systems are a long way from 100% reliable. The rclone sync/copy commands
cope with this with lots of retries. However rclone mount can't use
retries in the same way without making local copies of the uploads. Look
at the file caching for solutions to make mount more reliable.
at the VFS File Caching for solutions to make mount more reliable.
Attribute caching
@@ -3150,6 +3150,12 @@ for two reasons. Firstly because it is only checked every
--vfs-cache-poll-interval. Secondly because open files cannot be evicted
from the cache.
You SHOULD NOT run two copies of rclone using the same VFS cache with
the same or overlapping remotes if using --vfs-cache-mode > off. This
can potentially cause data corruption if you do. You can work around
this by giving each rclone its own cache hierarchy with --cache-dir. You
don't need to worry about this if the remotes in use don't overlap.
--vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote
@@ -3878,6 +3884,12 @@ for two reasons. Firstly because it is only checked every
--vfs-cache-poll-interval. Secondly because open files cannot be evicted
from the cache.
You SHOULD NOT run two copies of rclone using the same VFS cache with
the same or overlapping remotes if using --vfs-cache-mode > off. This
can potentially cause data corruption if you do. You can work around
this by giving each rclone its own cache hierarchy with --cache-dir. You
don't need to worry about this if the remotes in use don't overlap.
--vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote
@@ -4200,6 +4212,12 @@ for two reasons. Firstly because it is only checked every
--vfs-cache-poll-interval. Secondly because open files cannot be evicted
from the cache.
You SHOULD NOT run two copies of rclone using the same VFS cache with
the same or overlapping remotes if using --vfs-cache-mode > off. This
can potentially cause data corruption if you do. You can work around
this by giving each rclone its own cache hierarchy with --cache-dir. You
don't need to worry about this if the remotes in use don't overlap.
--vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote
@@ -4697,6 +4715,12 @@ for two reasons. Firstly because it is only checked every
--vfs-cache-poll-interval. Secondly because open files cannot be evicted
from the cache.
You SHOULD NOT run two copies of rclone using the same VFS cache with
the same or overlapping remotes if using --vfs-cache-mode > off. This
can potentially cause data corruption if you do. You can work around
this by giving each rclone its own cache hierarchy with --cache-dir. You
don't need to worry about this if the remotes in use don't overlap.
--vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote
@@ -5268,6 +5292,12 @@ for two reasons. Firstly because it is only checked every
--vfs-cache-poll-interval. Secondly because open files cannot be evicted
from the cache.
You SHOULD NOT run two copies of rclone using the same VFS cache with
the same or overlapping remotes if using --vfs-cache-mode > off. This
can potentially cause data corruption if you do. You can work around
this by giving each rclone its own cache hierarchy with --cache-dir. You
don't need to worry about this if the remotes in use don't overlap.
--vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote
@@ -5771,6 +5801,12 @@ for two reasons. Firstly because it is only checked every
--vfs-cache-poll-interval. Secondly because open files cannot be evicted
from the cache.
You SHOULD NOT run two copies of rclone using the same VFS cache with
the same or overlapping remotes if using --vfs-cache-mode > off. This
can potentially cause data corruption if you do. You can work around
this by giving each rclone its own cache hierarchy with --cache-dir. You
don't need to worry about this if the remotes in use don't overlap.
--vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote
@@ -10156,6 +10192,8 @@ This takes the following parameters
- srcFs - a remote name string e.g. "drive:src" for the source
- dstFs - a remote name string e.g. "drive:dst" for the destination
- createEmptySrcDirs - create empty src directories on destination if
set
See the copy command command for more information on the above.
@@ -10167,6 +10205,8 @@ This takes the following parameters
- srcFs - a remote name string e.g. "drive:src" for the source
- dstFs - a remote name string e.g. "drive:dst" for the destination
- createEmptySrcDirs - create empty src directories on destination if
set
- deleteEmptySrcDirs - delete empty src directories if set
See the move command command for more information on the above.
@@ -10179,6 +10219,8 @@ This takes the following parameters
- srcFs - a remote name string e.g. "drive:src" for the source
- dstFs - a remote name string e.g. "drive:dst" for the destination
- createEmptySrcDirs - create empty src directories on destination if
set
See the sync command command for more information on the above.
@@ -11048,7 +11090,7 @@ These flags are available for every command.
--use-json-log Use json log format.
--use-mmap Use mmap allocator (see docs).
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string. The default is rclone/ version (default "rclone/v1.54.0")
--user-agent string Set the user-agent to a specified string. The default is rclone/ version (default "rclone/v1.54.1")
-v, --verbose count Print lots more stuff (repeat for more)
@@ -11178,7 +11220,7 @@ and may be set in the config file.
--drive-starred-only Only show files that are starred.
--drive-stop-on-download-limit Make download limit errors be fatal
--drive-stop-on-upload-limit Make upload limit errors be fatal
--drive-team-drive string ID of the Team Drive
--drive-team-drive string ID of the Shared Drive (Team Drive)
--drive-token string OAuth Access Token as a JSON blob.
--drive-token-url string Token server url.
--drive-trashed-only Only show files that are in the trash.
@@ -11459,8 +11501,13 @@ and may be set in the config file.
--yandex-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,InvalidUtf8,Dot)
--yandex-token string OAuth Access Token as a JSON blob.
--yandex-token-url string Token server url.
--zoho-auth-url string Auth server URL.
--zoho-client-id string OAuth Client Id
--zoho-client-secret string OAuth Client Secret
--zoho-encoding MultiEncoder This sets the encoding for the backend. (default Del,Ctl,InvalidUtf8)
--zoho-region string Zoho region to connect to. You'll have to use the region you organization is registered in.
--zoho-token string OAuth Access Token as a JSON blob.
--zoho-token-url string Token server url.
1Fichier
@@ -12350,6 +12397,16 @@ You'd then do a full rclone sync less often.
Note that --fast-list isn't required in the top-up sync.
Avoiding HEAD requests after PUT
By default rclone will HEAD every object it uploads. It does this to
check the object got uploaded correctly.
You can disable this with the --s3-no-head option - see there for more
details.
Setting this flag increases the chance for undetected upload failures.
Hashes
For small objects which weren't uploaded as multipart uploads (objects
@@ -16882,9 +16939,8 @@ would with any other remote, e.g. rclone copy D:\docs secret:\docs, and
rclone will encrypt and decrypt as needed on the fly. If you access the
wrapped remote remote:path directly you will bypass the encryption, and
anything you read will be in encrypted form, and anything you write will
be undencrypted. To avoid issues it is best to configure a dedicated
path for encrypted content, and access it exclusively through a crypt
remote.
be unencrypted. To avoid issues it is best to configure a dedicated path
for encrypted content, and access it exclusively through a crypt remote.
No remotes found - make a new one
n) New remote
@@ -17814,6 +17870,20 @@ can be set smaller if you are tight on memory.
Impersonate this user when using a business account.
Note that if you want to use impersonate, you should make sure this flag
is set when running "rclone config" as this will cause rclone to request
the "members.read" scope which it won't normally. This is needed to
lookup a members email address into the internal ID that dropbox uses in
the API.
Using the "members.read" scope will require a Dropbox Team Admin to
approve during the OAuth flow.
You will have to use your own App (setting your own client_id and
client_secret) to use this option as currently rclone's default set of
permissions doesn't include "members.read". This can be added once v1.55
or later is in use everywhere.
- Config: impersonate
- Env Var: RCLONE_DROPBOX_IMPERSONATE
- Type: string
@@ -18257,9 +18327,8 @@ Example without a config file
Implicit TLS
Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to be
enabled in the FTP backend config for the remote, or with
[--ftp-tls]{#ftp-tls}. The default FTPS port is 990, not 21 and can be
set with [--ftp-port]{#ftp-port}.
enabled in the FTP backend config for the remote, or with --ftp-tls. The
default FTPS port is 990, not 21 and can be set with --ftp-port.
Standard Options
@@ -19009,7 +19078,7 @@ This will guide you through an interactive setup process:
Log in and authorize rclone for access
Waiting for code...
Got code
Configure this as a team drive?
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No
y/n> n
@@ -19212,22 +19281,23 @@ you created/selected at step #1 - use rclone without specifying the
--drive-impersonate option, like this:
rclone -v foo@example.com lsf gdrive:backup
Team drives
Shared drives (team drives)
If you want to configure the remote to point to a Google Team Drive then
answer y to the question Configure this as a team drive?.
If you want to configure the remote to point to a Google Shared Drive
(previously known as Team Drives) then answer y to the question
Configure this as a Shared Drive (Team Drive)?.
This will fetch the list of Team Drives from google and allow you to
configure which one you want to use. You can also type in a team drive
This will fetch the list of Shared Drives from google and allow you to
configure which one you want to use. You can also type in a Shared Drive
ID if you prefer.
For example:
Configure this as a team drive?
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No
y/n> y
Fetching team drive list...
Fetching Shared Drive list...
Choose a number from below, or type in your own value
1 / Rclone Test
\ "xxxxxxxxxxxxxxxxxxxx"
@@ -19235,7 +19305,7 @@ For example:
\ "yyyyyyyyyyyyyyyyyyyy"
3 / Rclone Test 3
\ "zzzzzzzzzzzzzzzzzzzz"
Enter a Team Drive ID> 1
Enter a Shared Drive ID> 1
--------------------
[remote]
client_id =
@@ -19635,7 +19705,7 @@ if you want use SA instead of interactive login.
--drive-team-drive
ID of the Team Drive
ID of the Shared Drive (Team Drive)
- Config: team_drive
- Env Var: RCLONE_DRIVE_TEAM_DRIVE
@@ -20093,11 +20163,11 @@ Options:
drives
List the shared drives available to this account
List the Shared Drives available to this account
rclone backend drives remote: [options] [<arguments>+]
This command lists the shared drives (teamdrives) available to this
This command lists the Shared Drives (Team Drives) available to this
account.
Usage:
@@ -25693,7 +25763,7 @@ The Go SSH library disables the use of the aes128-cbc cipher by default,
due to security concerns. This can be re-enabled on a per-connection
basis by setting the use_insecure_cipher setting in the configuration
file to true. Further details on the insecurity of this cipher can be
found [in this paper] (http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).
found in this paper.
SFTP isn't supported under plan9 until this issue is fixed.
@@ -27183,6 +27253,24 @@ Standard Options
Here are the standard options specific to zoho (Zoho).
--zoho-client-id
OAuth Client Id Leave blank normally.
- Config: client_id
- Env Var: RCLONE_ZOHO_CLIENT_ID
- Type: string
- Default: ""
--zoho-client-secret
OAuth Client Secret Leave blank normally.
- Config: client_secret
- Env Var: RCLONE_ZOHO_CLIENT_SECRET
- Type: string
- Default: ""
--zoho-region
Zoho region to connect to. You'll have to use the region you
@@ -27206,6 +27294,33 @@ Advanced Options
Here are the advanced options specific to zoho (Zoho).
--zoho-token
OAuth Access Token as a JSON blob.
- Config: token
- Env Var: RCLONE_ZOHO_TOKEN
- Type: string
- Default: ""
--zoho-auth-url
Auth server URL. Leave blank to use the provider defaults.
- Config: auth_url
- Env Var: RCLONE_ZOHO_AUTH_URL
- Type: string
- Default: ""
--zoho-token-url
Token server url. Leave blank to use the provider defaults.
- Config: token_url
- Env Var: RCLONE_ZOHO_TOKEN_URL
- Type: string
- Default: ""
--zoho-encoding
This sets the encoding for the backend.
@@ -27707,12 +27822,62 @@ Options:
CHANGELOG
v1.54.1 - 2021-03-08
See commits
- Bug Fixes
- accounting: Fix --bwlimit when up or down is off (Nick
Craig-Wood)
- docs
- Fix nesting of brackets and backticks in ftp docs
(edwardxml)
- Fix broken link in sftp page (edwardxml)
- Fix typo in crypt.md (Romeo Kienzler)
- Changelog: Correct link to digitalis.io (Alex JOST)
- Replace #file-caching with #vfs-file-caching (Miron
Veryanskiy)
- Convert bogus example link to code (edwardxml)
- Remove dead link from rc.md (edwardxml)
- rc: Sync,copy,move: document createEmptySrcDirs parameter (Nick
Craig-Wood)
- lsjson: Fix unterminated JSON in the presence of errors (Nick
Craig-Wood)
- Mount
- Fix mount dropping on macOS by setting --daemon-timeout 10m
(Nick Craig-Wood)
- VFS
- Document simultaneous usage with the same cache shouldn't be
used (Nick Craig-Wood)
- B2
- Automatically raise upload cutoff to avoid spurious error (Nick
Craig-Wood)
- Fix failed to create file system with application key limited to
a prefix (Nick Craig-Wood)
- Drive
- Refer to Shared Drives instead of Team Drives (Nick Craig-Wood)
- Dropbox
- Add scopes to oauth request and optionally "members.read" (Nick
Craig-Wood)
- S3
- Fix failed to create file system with folder level permissions
policy (Nick Craig-Wood)
- Fix Wasabi HEAD requests returning stale data by using only 1
transport (Nick Craig-Wood)
- Fix shared_credentials_file auth (Dmitry Chepurovskiy)
- Add --s3-no-head to reducing costs docs (Nick Craig-Wood)
- Union
- Fix mkdir at root with remote:/ (Nick Craig-Wood)
- Zoho
- Fix custom client id's (buengese)
v1.54.0 - 2021-02-02
See commits
- New backends
- Compression remote (experimental)(buengese)
- Compression remote (experimental) (buengese)
- Enterprise File Fabric (Nick Craig-Wood)
- This work was sponsored by Storage Made Easy
- HDFS (Hadoop Distributed File System) (Yury Stankevich)
@@ -27720,37 +27885,30 @@ See commits
- New Features
- Deglobalise the config (Nick Craig-Wood)
- Global config now read from the context
- Global config can be passed into the rc
- This will enable passing of global config via the rc
- This work was sponsored by Digitalis
- Add --bwlimit for upload and download (Nick Craig-Wood)
- Obey bwlimit in http Transport for better limiting
- Enhance systemd integration (Hekmon)
- log level identification
- manual activation with flag
- automatic systemd launch detection
- log level identification, manual activation with flag,
automatic systemd launch detection
- Don't compile systemd log integration for non unix systems
(Benjamin Gustin)
- Add a download flag to hashsum and related commands to force
rclone to download and hash files locally (lostheli)
- Add a --download flag to md5sum/sha1sum/hashsum to force rclone
to download and hash files locally (lostheli)
- Add --progress-terminal-title to print ETA to terminal title
(LaSombra)
- Make backend env vars show in help as the defaults for backend
flags (Nick Craig-Wood)
- build
- Raise minimum go version to go1.12 (Nick Craig-Wood)
- check
- Make the error count match up in the log message (Nick
Craig-Wood)
- cmd
- Add --progress-terminal-title to print ETA to terminal title
(LaSombra)
- Make backend env vars show in help as the defaults for
backend flags (Nick Craig-Wood)
- dedupe
- Add --by-hash to dedupe on hash not file name (Nick
- Add --by-hash to dedupe on content hash not file name (Nick
Craig-Wood)
- Add --dedupe-mode list to just list dupes, changing nothing
(Nick Craig-Wood)
- Add warning if used on a remote which can't have duplicate
names (Nick Craig-Wood)
- flags: Improve error message when reading environment vars (Nick
Craig-Wood)
- fs
- Add Shutdown optional method for backends (Nick Craig-Wood)
- When using --files-from check files concurrently (zhucan)
@@ -27764,7 +27922,7 @@ See commits
- Highlight read errors instead of aborting (Claudio
Bantaloukas)
- Add sort by average size in directory (Adam Plánský)
- Add toggle option for average size in directory - key 'a'
- Add toggle option for average s3ize in directory - key 'a'
(Adam Plánský)
- Add empty folder flag into ncdu browser (Adam Plánský)
- Add ! (errror) and . (unreadable) file flags to go with e
@@ -27794,22 +27952,17 @@ See commits
(Chaitanya Bankanhal)
- Fix plugins initialization (negative0)
- Bug Fixes
- build
- Explicitly set ARM version to fix build (Nick Craig-Wood)
- Don't explicitly set ARM version to fix ARMv5 build (Nick
Craig-Wood)
- Fix nfpm install (Nick Craig-Wood)
- Fix docker build by upgrading ilteoood/docker_buildx (Nick
Craig-Wood)
- Temporary fix for Windows build errors (Ivan Andreev)
- fs
- Fix nil pointer on copy & move operations directly to remote
(Anagh Kumar Baranwal)
- Fix parsing of .. when joining remotes (Nick Craig-Wood)
- log: Fix enabling systemd logging when using --log-file (Nick
Craig-Wood)
- move: Fix data loss when moving the same object (Nick
Craig-Wood)
- check
- Make the error count match up in the log message (Nick
Craig-Wood)
- move: Fix data loss when source and destination are the same
object (Nick Craig-Wood)
- operations
- Fix --cutof-mode hard not cutting off immediately (Nick
Craig-Wood)
@@ -27820,7 +27973,7 @@ See commits
- Fix --immutable errors retrying many times (Nick Craig-Wood)
- Docs
- Many fixes and a rewrite of the filtering docs (edwardxml)
- Many spelling and grammar problems (Josh Soref)
- Many spelling and grammar fixes (Josh Soref)
- Doc fixes for commands delete, purge, rmdir, rmdirs and mount
(albertony)
- And thanks to these people for many doc fixes too numerous to
@@ -27834,15 +27987,12 @@ See commits
- Mount
- Update systemd status with cache stats (Hekmon)
- Disable bazil/fuse based mount on macOS (Nick Craig-Wood)
- Make mount be cmount under macOS (Nick Craig-Wood)
- Make rclone mount actually run rclone cmount under macOS
(Nick Craig-Wood)
- Implement mknod to make NFS file creation work (Nick Craig-Wood)
- Make sure we don't call umount more than once (Nick Craig-Wood)
- Don't call host.Umount if a signal has been received (Nick
Craig-Wood)
- More user friendly mounting as network drive on windows
(albertony)
- Cleanup OS specific option handling and documentation
(albertony)
- Detect if uid or gid are set in same option string: -o
uid=123,gid=456 (albertony)
- Don't attempt to unmount if fs has been destroyed already (Nick
@@ -27863,37 +28013,37 @@ See commits
filesystems (Riccardo Iaconelli)
- Azure Blob
- Add support for service principals (James Lim)
- Utilize streaming capabilities (Denis Neuling)
- Update SDK to v0.13.0 and fix API breakage (Nick Craig-Wood,
Mitsuo Heijo)
- Add support for managed identities (Brad Ackerman)
- Add examples for access tier (Bob Pusateri)
- Utilize the streaming capabilities from the SDK for multipart
uploads (Denis Neuling)
- Fix setting of mime types (Nick Craig-Wood)
- Fix crash when listing outside a SAS URL's root (Nick
Craig-Wood)
- Delete archive tier blobs before update if
--azureblob-archive-tier-delete (Nick Craig-Wood)
- Add support for managed identities (Brad Ackerman)
- Fix crash on startup (Nick Craig-Wood)
- Add examples for access tier (Bob Pusateri)
- Fix memory usage by upgrading the SDK and implementing a
TransferManager (Nick Craig-Wood)
- Fix memory usage by upgrading the SDK to v0.13.0 and
implementing a TransferManager (Nick Craig-Wood)
- Require go1.14+ to compile due to SDK changes (Nick Craig-Wood)
- B2
- Make NewObject use less expensive API calls (Nick Craig-Wood)
- Fixed possible crash when accessing Backblaze b2 remote
(lluuaapp)
- This will improve --files-from and restic serve in
particular
- Fixed crash on an empty file name (lluuaapp)
- Box
- Fix NewObject for files that differ in case (Nick Craig-Wood)
- Fix finding directories in a case insentive way (Nick
Craig-Wood)
- Chunker
- Skip long local hashing, hash in-transit (fixes) (Ivan Andreev)
- Set Features.ReadMimeType=false as Object.MimeType not supported
(Nick Craig-Wood)
- Set Features ReadMimeType to false as Object.MimeType not
supported (Nick Craig-Wood)
- Fix case-insensitive NewObject, test metadata detection (Ivan
Andreev)
- Drive
- Implement "rclone backend copyid" command for copying files by
ID (Nick Craig-Wood)
- Implement rclone backend copyid command for copying files by ID
(Nick Craig-Wood)
- Added flag --drive-stop-on-download-limit to stop transfers when
the download limit is exceeded (Anagh Kumar Baranwal)
- Implement CleanUp workaround for team drives (buengese)
@@ -27904,18 +28054,18 @@ See commits
Rodriguez-Estivill)
- Dropbox
- Add support for viewing shared files and folders (buengese)
- Implement IDer (buengese)
- Set Features.ReadMimeType=false as Object.MimeType not supported
(Nick Craig-Wood)
- Tidy repeated error message (Nick Craig-Wood)
- Enable short lived access tokens (Nick Craig-Wood)
- Implement IDer on Objects so rclone lsf etc can read the IDs
(buengese)
- Set Features ReadMimeType to false as Object.MimeType not
supported (Nick Craig-Wood)
- Make malformed_path errors from too long files not retriable
(Nick Craig-Wood)
- Test file name length before upload to fix upload loop (Nick
Craig-Wood)
- Enable short lived access tokens (Nick Craig-Wood)
- Fichier
- Set Features.ReadMimeType=true as Object.MimeType is supported
(Nick Craig-Wood)
- Set Features ReadMimeType to true as Object.MimeType is
supported (Nick Craig-Wood)
- FTP
- Add --ftp-disable-msld option to ignore MLSD for really old
servers (Nick Craig-Wood)
@@ -27924,39 +28074,40 @@ See commits
- Storage class object header support (Laurens Janssen)
- Fix anonymous client to use rclone's HTTP client (Nick
Craig-Wood)
- Fix Entry doesn't belong in directory "" (same as directory) -
ignoring (Nick Craig-Wood)
- Fix
Entry doesn't belong in directory "" (same as directory) - ignoring
(Nick Craig-Wood)
- Googlephotos
- New flag --gphotos-include-archived (Nicolas Rueff)
- New flag --gphotos-include-archived to show archived photos as
well (Nicolas Rueff)
- Jottacloud
- Don't erroniously report support for writing mime types
- Don't erroneously report support for writing mime types
(buengese)
- Add support for Telia Cloud (#4930) (Patrik Nordlén)
- Add support for Telia Cloud (Patrik Nordlén)
- Mailru
- Accept special folders eg camera-upload (Ivan Andreev)
- Avoid prehashing of large local files (Ivan Andreev)
- Fix uploads after recent changes on server (Ivan Andreev)
- Fix range requests after June 2020 changes on server (Ivan
Andreev)
- Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev)
- Remove deprecated protocol quirks (Ivan Andreev)
- Accept special folders eg camera-upload (Ivan Andreev)
- Avoid prehashing of large local files (Ivan Andreev)
- Memory
- Fix setting of mime types (Nick Craig-Wood)
- Onedrive
- Add support for china region operated by 21vianet and other
regional suppliers (#4963) (NyaMisty)
- Add support for China region operated by 21vianet and other
regional suppliers (NyaMisty)
- Warn on gateway timeout errors (Nick Craig-Wood)
- Fall back to normal copy if server-side copy unavailable (#4903)
(Alex Chen)
- Fall back to normal copy if server-side copy unavailable (Alex
Chen)
- Fix server-side copy completely disabled on OneDrive for
Business (Cnly)
- (business only) workaround to replace existing file on
server-side copy (#4904) (Alex Chen)
server-side copy (Alex Chen)
- Enhance link creation with expiry, scope, type and password
(Nick Craig-Wood)
- Remove % and # from the set of encoded characters (#4909) (Alex
Chen)
- Support addressing site by server-relative URL (#4761) (kice)
- Remove % and # from the set of encoded characters (Alex Chen)
- Support addressing site by server-relative URL (kice)
- Opendrive
- Fix finding directories in a case insensitive way (Nick
Craig-Wood)
@@ -27972,18 +28123,18 @@ See commits
- Added --s3-disable-http2 to disable http/2 (Anagh Kumar
Baranwal)
- Complete SSE-C implementation (Nick Craig-Wood)
- Fix hashes on small files with AWS:KMS and SSE-C (Nick
Craig-Wood)
- Add MD5 metadata to objects uploaded with SSE-AWS/SSE-C (Nick
Craig-Wood)
- Fix hashes on small files with AWS:KMS and SSE-C (Nick
Craig-Wood)
- Add MD5 metadata to objects uploaded with SSE-AWS/SSE-C
(Nick Craig-Wood)
- Add --s3-no-head parameter to minimise transactions on upload
(Nick Craig-Wood)
- Update docs with a Reducing Costs section (Nick Craig-Wood)
- Added error handling for error code 429 indicating too many
requests (Anagh Kumar Baranwal)
- Add requester pays option (kelv)
- Fix copy multipart with v2 auth failing with
'SignatureDoesNotMatch' (Louis Koo)
- Add --s3-no-head parameter to minimise transactions on upload
(Nick Craig-Wood)
- SFTP
- Allow cert based auth via optional pubkey (Stephen Harris)
- Allow user to optionally check server hosts key to add security
@@ -27994,7 +28145,8 @@ See commits
- Implement Shutdown method (Nick Craig-Wood)
- Implement keyboard interactive authentication (Nick Craig-Wood)
- Make --tpslimit apply (Nick Craig-Wood)
- Implement --sftp-use-fstat (Nick Craig-Wood)
- Implement --sftp-use-fstat for unusual SFTP servers (Nick
Craig-Wood)
- Sugarsync
- Fix NewObject for files that differ in case (Nick Craig-Wood)
- Fix finding directories in a case insentive way (Nick
@@ -28010,7 +28162,7 @@ See commits
- Updated docs to show streaming to nextcloud is working (Durval
Menezes)
- Yandex
- Set Features.WriteMimeType=false as Yandex ignores mime types
- Set Features WriteMimeType to false as Yandex ignores mime types
(Nick Craig-Wood)

View File

@@ -93,7 +93,7 @@ build_dep:
# Get the release dependencies we only install on linux
release_dep_linux:
cd /tmp && go get github.com/goreleaser/nfpm/v2/...
go run bin/get-github-release.go -extract nfpm goreleaser/nfpm 'nfpm_.*_Linux_x86_64\.tar\.gz'
# Get the release dependencies we only install on Windows
release_dep_windows:

View File

@@ -4,7 +4,7 @@ This file describes how to make the various kinds of releases
## Extra required software for making a release
* [gh the github cli](https://github.com/cli/cli) for uploading packages
* [github-release](https://github.com/aktau/github-release) for uploading packages
* pandoc for making the html and man pages
## Making a release

View File

@@ -1 +1 @@
v1.55.0
v1.54.1

View File

@@ -479,12 +479,9 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
f.setRoot(newRoot)
_, err := f.NewObject(ctx, leaf)
if err != nil {
if err == fs.ErrorObjectNotFound {
// File doesn't exist so return old f
f.setRoot(oldRoot)
return f, nil
}
return nil, err
// File doesn't exist so return old f
f.setRoot(oldRoot)
return f, nil
}
// return an error with an fs which points to the parent
return f, fs.ErrorIsFile

View File

@@ -1034,7 +1034,7 @@ func (r *run) updateObjectRemote(t *testing.T, f fs.Fs, remote string, data1 []b
objInfo1 := object.NewStaticObjectInfo(remote, time.Now(), int64(len(data1)), true, nil, f)
objInfo2 := object.NewStaticObjectInfo(remote, time.Now(), int64(len(data2)), true, nil, f)
obj, err = f.Put(context.Background(), in1, objInfo1)
_, err = f.Put(context.Background(), in1, objInfo1)
require.NoError(t, err)
obj, err = f.NewObject(context.Background(), remote)
require.NoError(t, err)

View File

@@ -487,11 +487,11 @@ func testPreventCorruption(t *testing.T, f *Fs) {
// accessing chunks in strict mode is prohibited
f.opt.FailHard = true
billyChunk4Name := billyChunkName(4)
billyChunk4, err := f.NewObject(ctx, billyChunk4Name)
_, err = f.NewObject(ctx, billyChunk4Name)
assertOverlapError(err)
f.opt.FailHard = false
billyChunk4, err = f.NewObject(ctx, billyChunk4Name)
billyChunk4, err := f.NewObject(ctx, billyChunk4Name)
assert.NoError(t, err)
require.NotNil(t, billyChunk4)

View File

@@ -207,7 +207,7 @@ func init() {
}
err = configTeamDrive(ctx, opt, m, name)
if err != nil {
log.Fatalf("Failed to configure team drive: %v", err)
log.Fatalf("Failed to configure Shared Drive: %v", err)
}
},
Options: append(driveOAuthOptions(), []fs.Option{{
@@ -247,7 +247,7 @@ a non root folder as its starting point.
Advanced: true,
}, {
Name: "team_drive",
Help: "ID of the Team Drive",
Help: "ID of the Shared Drive (Team Drive)",
Hide: fs.OptionHideConfigurator,
Advanced: true,
}, {
@@ -666,7 +666,7 @@ func (f *Fs) shouldRetry(err error) (bool, error) {
fs.Errorf(f, "Received download limit error: %v", err)
return false, fserrors.FatalError(err)
} else if f.opt.StopOnUploadLimit && reason == "teamDriveFileLimitExceeded" {
fs.Errorf(f, "Received team drive file limit error: %v", err)
fs.Errorf(f, "Received Shared Drive file limit error: %v", err)
return false, fserrors.FatalError(err)
}
}
@@ -955,24 +955,24 @@ func configTeamDrive(ctx context.Context, opt *Options, m configmap.Mapper, name
return nil
}
if opt.TeamDriveID == "" {
fmt.Printf("Configure this as a team drive?\n")
fmt.Printf("Configure this as a Shared Drive (Team Drive)?\n")
} else {
fmt.Printf("Change current team drive ID %q?\n", opt.TeamDriveID)
fmt.Printf("Change current Shared Drive (Team Drive) ID %q?\n", opt.TeamDriveID)
}
if !config.Confirm(false) {
return nil
}
f, err := newFs(ctx, name, "", m)
if err != nil {
return errors.Wrap(err, "failed to make Fs to list teamdrives")
return errors.Wrap(err, "failed to make Fs to list Shared Drives")
}
fmt.Printf("Fetching team drive list...\n")
fmt.Printf("Fetching Shared Drive list...\n")
teamDrives, err := f.listTeamDrives(ctx)
if err != nil {
return err
}
if len(teamDrives) == 0 {
fmt.Printf("No team drives found in your account")
fmt.Printf("No Shared Drives found in your account")
return nil
}
var driveIDs, driveNames []string
@@ -980,7 +980,7 @@ func configTeamDrive(ctx context.Context, opt *Options, m configmap.Mapper, name
driveIDs = append(driveIDs, teamDrive.Id)
driveNames = append(driveNames, teamDrive.Name)
}
driveID := config.Choose("Enter a Team Drive ID", driveIDs, driveNames, true)
driveID := config.Choose("Enter a Shared Drive ID", driveIDs, driveNames, true)
m.Set("team_drive", driveID)
m.Set("root_folder_id", "")
opt.TeamDriveID = driveID
@@ -2475,9 +2475,9 @@ func (f *Fs) teamDriveOK(ctx context.Context) (err error) {
return f.shouldRetry(err)
})
if err != nil {
return errors.Wrap(err, "failed to get Team/Shared Drive info")
return errors.Wrap(err, "failed to get Shared Drive info")
}
fs.Debugf(f, "read info from team drive %q", td.Name)
fs.Debugf(f, "read info from Shared Drive %q", td.Name)
return err
}
@@ -2963,7 +2963,7 @@ func (f *Fs) listTeamDrives(ctx context.Context) (drives []*drive.TeamDrive, err
return defaultFs.shouldRetry(err)
})
if err != nil {
return drives, errors.Wrap(err, "listing team drives failed")
return drives, errors.Wrap(err, "listing Team Drives failed")
}
drives = append(drives, teamDrives.TeamDrives...)
if teamDrives.NextPageToken == "" {
@@ -3131,8 +3131,8 @@ authenticated with "drive2:" can't read files from "drive:".
},
}, {
Name: "drives",
Short: "List the shared drives available to this account",
Long: `This command lists the shared drives (teamdrives) available to this
Short: "List the Shared Drives available to this account",
Long: `This command lists the Shared Drives (Team Drives) available to this
account.
Usage:
@@ -3427,10 +3427,11 @@ func (o *baseObject) httpResponse(ctx context.Context, url, method string, optio
if url == "" {
return nil, nil, errors.New("forbidden to download - check sharing permission")
}
req, err = http.NewRequestWithContext(ctx, method, url, nil)
req, err = http.NewRequest(method, url, nil)
if err != nil {
return req, nil, err
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
fs.OpenOptionAddHTTPHeaders(req.Header, options)
if o.bytes == 0 {
// Don't supply range requests for 0 length objects as they always fail

View File

@@ -77,10 +77,11 @@ func (f *Fs) Upload(ctx context.Context, in io.Reader, size int64, contentType,
return false, err
}
var req *http.Request
req, err = http.NewRequestWithContext(ctx, method, urls, body)
req, err = http.NewRequest(method, urls, body)
if err != nil {
return false, err
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
googleapi.Expand(req.URL, map[string]string{
"fileId": fileID,
})
@@ -113,7 +114,8 @@ func (f *Fs) Upload(ctx context.Context, in io.Reader, size int64, contentType,
// Make an http.Request for the range passed in
func (rx *resumableUpload) makeRequest(ctx context.Context, start int64, body io.ReadSeeker, reqSize int64) *http.Request {
req, _ := http.NewRequestWithContext(ctx, "POST", rx.URI, body)
req, _ := http.NewRequest("POST", rx.URI, body)
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
req.ContentLength = reqSize
totalSize := "*"
if rx.ContentLength >= 0 {

View File

@@ -94,7 +94,14 @@ const (
var (
// Description of how to auth for this app
dropboxConfig = &oauth2.Config{
Scopes: []string{},
Scopes: []string{
"files.metadata.write",
"files.content.write",
"files.content.read",
"sharing.write",
// "file_requests.write",
// "members.read", // needed for impersonate - but causes app to need to be approved by Dropbox Team Admin during the flow
},
// Endpoint: oauth2.Endpoint{
// AuthURL: "https://www.dropbox.com/1/oauth2/authorize",
// TokenURL: "https://api.dropboxapi.com/1/oauth2/token",
@@ -115,6 +122,19 @@ var (
errNotSupportedInSharedMode = fserrors.NoRetryError(errors.New("not supported in shared files mode"))
)
// Gets an oauth config with the right scopes
func getOauthConfig(m configmap.Mapper) *oauth2.Config {
// If not impersonating, use standard scopes
if impersonate, _ := m.Get("impersonate"); impersonate == "" {
return dropboxConfig
}
// Make a copy of the config
config := *dropboxConfig
// Make a copy of the scopes with "members.read" appended
config.Scopes = append(config.Scopes, "members.read")
return &config
}
// Register with Fs
func init() {
DbHashType = hash.RegisterHash("DropboxHash", 64, dbhash.New)
@@ -129,7 +149,7 @@ func init() {
oauth2.SetAuthURLParam("token_access_type", "offline"),
},
}
err := oauthutil.Config(ctx, "dropbox", name, m, dropboxConfig, &opt)
err := oauthutil.Config(ctx, "dropbox", name, m, getOauthConfig(m), &opt)
if err != nil {
log.Fatalf("Failed to configure token: %v", err)
}
@@ -147,8 +167,23 @@ memory. It can be set smaller if you are tight on memory.`, maxChunkSize),
Default: defaultChunkSize,
Advanced: true,
}, {
Name: "impersonate",
Help: "Impersonate this user when using a business account.",
Name: "impersonate",
Help: `Impersonate this user when using a business account.
Note that if you want to use impersonate, you should make sure this
flag is set when running "rclone config" as this will cause rclone to
request the "members.read" scope which it won't normally. This is
needed to lookup a members email address into the internal ID that
dropbox uses in the API.
Using the "members.read" scope will require a Dropbox Team Admin
to approve during the OAuth flow.
You will have to use your own App (setting your own client_id and
client_secret) to use this option as currently rclone's default set of
permissions doesn't include "members.read". This can be added once
v1.55 or later is in use everywhere.
`,
Default: "",
Advanced: true,
}, {
@@ -327,7 +362,7 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
}
}
oAuthClient, _, err := oauthutil.NewClient(ctx, name, m, dropboxConfig)
oAuthClient, _, err := oauthutil.NewClient(ctx, name, m, getOauthConfig(m))
if err != nil {
return nil, errors.Wrap(err, "failed to configure dropbox")
}

View File

@@ -1028,10 +1028,11 @@ func (o *Object) Storable() bool {
// Open an object for read
func (o *Object) Open(ctx context.Context, options ...fs.OpenOption) (in io.ReadCloser, err error) {
req, err := http.NewRequestWithContext(ctx, "GET", o.url, nil)
req, err := http.NewRequest("GET", o.url, nil)
if err != nil {
return nil, err
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
fs.FixRangeOption(options, o.bytes)
fs.OpenOptionAddHTTPHeaders(req.Header, options)
var res *http.Response

View File

@@ -183,8 +183,9 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
return http.ErrUseLastResponse
}
// check to see if points to a file
req, err := http.NewRequestWithContext(ctx, "HEAD", u.String(), nil)
req, err := http.NewRequest("HEAD", u.String(), nil)
if err == nil {
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
addHeaders(req, opt)
res, err := noRedir.Do(req)
err = statusError(res, err)
@@ -390,10 +391,11 @@ func (f *Fs) readDir(ctx context.Context, dir string) (names []string, err error
return nil, errors.Errorf("internal error: readDir URL %q didn't end in /", URL)
}
// Do the request
req, err := http.NewRequestWithContext(ctx, "GET", URL, nil)
req, err := http.NewRequest("GET", URL, nil)
if err != nil {
return nil, errors.Wrap(err, "readDir failed")
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
f.addHeaders(req)
res, err := f.httpClient.Do(req)
if err == nil {
@@ -545,10 +547,11 @@ func (o *Object) stat(ctx context.Context) error {
return nil
}
url := o.url()
req, err := http.NewRequestWithContext(ctx, "HEAD", url, nil)
req, err := http.NewRequest("HEAD", url, nil)
if err != nil {
return errors.Wrap(err, "stat failed")
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
o.fs.addHeaders(req)
res, err := o.fs.httpClient.Do(req)
if err == nil && res.StatusCode == http.StatusNotFound {
@@ -593,10 +596,11 @@ func (o *Object) Storable() bool {
// Open a remote http file object for reading. Seek is supported
func (o *Object) Open(ctx context.Context, options ...fs.OpenOption) (in io.ReadCloser, err error) {
url := o.url()
req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, errors.Wrap(err, "Open failed")
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
// Add optional headers
for k, v := range fs.OpenOptionHeaders(options) {

View File

@@ -5,7 +5,7 @@ import (
"net/http"
"time"
"github.com/ncw/swift/v2"
"github.com/ncw/swift"
"github.com/rclone/rclone/fs"
)
@@ -24,7 +24,7 @@ func newAuth(f *Fs) *auth {
// Request constructs an http.Request for authentication
//
// returns nil for not needed
func (a *auth) Request(ctx context.Context, c *swift.Connection) (r *http.Request, err error) {
func (a *auth) Request(*swift.Connection) (r *http.Request, err error) {
const retries = 10
for try := 1; try <= retries; try++ {
err = a.f.getCredentials(context.TODO())
@@ -38,7 +38,7 @@ func (a *auth) Request(ctx context.Context, c *swift.Connection) (r *http.Reques
}
// Response parses the result of an http request
func (a *auth) Response(ctx context.Context, resp *http.Response) error {
func (a *auth) Response(resp *http.Response) error {
return nil
}

View File

@@ -16,7 +16,7 @@ import (
"strings"
"time"
swiftLib "github.com/ncw/swift/v2"
swiftLib "github.com/ncw/swift"
"github.com/pkg/errors"
"github.com/rclone/rclone/backend/swift"
"github.com/rclone/rclone/fs"
@@ -110,10 +110,11 @@ func (f *Fs) String() string {
//
// The credentials are read into the Fs
func (f *Fs) getCredentials(ctx context.Context) (err error) {
req, err := http.NewRequestWithContext(ctx, "GET", "https://api.hubic.com/1.0/account/credentials", nil)
req, err := http.NewRequest("GET", "https://api.hubic.com/1.0/account/credentials", nil)
if err != nil {
return err
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
resp, err := f.client.Do(req)
if err != nil {
return err
@@ -163,7 +164,7 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
Timeout: 10 * ci.Timeout, // Use the timeouts in the transport
Transport: fshttp.NewTransport(ctx),
}
err = c.Authenticate(ctx)
err = c.Authenticate()
if err != nil {
return nil, errors.Wrap(err, "error authenticating swift connection")
}

View File

@@ -282,10 +282,11 @@ func (f *Fs) PutUnchecked(ctx context.Context, in io.Reader, src fs.ObjectInfo,
func (f *Fs) createUpload(ctx context.Context, name string, size int64, parentID string, modTime time.Time, options []fs.OpenOption) (location string, err error) {
// defer log.Trace(f, "name=%v, size=%v, parentID=%v, modTime=%v", name, size, parentID, modTime.String())("location=%v, err=%v", location, &err)
err = f.pacer.Call(func() (bool, error) {
req, err := http.NewRequestWithContext(ctx, "POST", "https://upload.put.io/files/", nil)
req, err := http.NewRequest("POST", "https://upload.put.io/files/", nil)
if err != nil {
return false, err
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
req.Header.Set("tus-resumable", "1.0.0")
req.Header.Set("upload-length", strconv.FormatInt(size, 10))
b64name := base64.StdEncoding.EncodeToString([]byte(f.opt.Enc.FromStandardName(name)))
@@ -427,19 +428,21 @@ func (f *Fs) transferChunk(ctx context.Context, location string, start int64, ch
}
func (f *Fs) makeUploadHeadRequest(ctx context.Context, location string) (*http.Request, error) {
req, err := http.NewRequestWithContext(ctx, "HEAD", location, nil)
req, err := http.NewRequest("HEAD", location, nil)
if err != nil {
return nil, err
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
req.Header.Set("tus-resumable", "1.0.0")
return req, nil
}
func (f *Fs) makeUploadPatchRequest(ctx context.Context, location string, in io.Reader, offset, length int64) (*http.Request, error) {
req, err := http.NewRequestWithContext(ctx, "PATCH", location, in)
req, err := http.NewRequest("PATCH", location, in)
if err != nil {
return nil, err
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
req.Header.Set("tus-resumable", "1.0.0")
req.Header.Set("upload-offset", strconv.FormatInt(offset, 10))
req.Header.Set("content-length", strconv.FormatInt(length, 10))

View File

@@ -229,10 +229,11 @@ func (o *Object) Open(ctx context.Context, options ...fs.OpenOption) (in io.Read
var resp *http.Response
headers := fs.OpenOptionHeaders(options)
err = o.fs.pacer.Call(func() (bool, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, storageURL, nil)
req, err := http.NewRequest(http.MethodGet, storageURL, nil)
if err != nil {
return shouldRetry(err)
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
req.Header.Set("User-Agent", o.fs.client.UserAgent)
// merge headers with extra headers

View File

@@ -33,7 +33,7 @@ import (
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/ncw/swift/v2"
"github.com/ncw/swift"
"github.com/pkg/errors"
"github.com/rclone/rclone/fs"
"github.com/rclone/rclone/fs/config"
@@ -1462,7 +1462,7 @@ func getClient(ctx context.Context, opt *Options) *http.Client {
}
// s3Connection makes a connection to s3
func s3Connection(ctx context.Context, opt *Options) (*s3.S3, *session.Session, error) {
func s3Connection(ctx context.Context, opt *Options, client *http.Client) (*s3.S3, *session.Session, error) {
// Make the auth
v := credentials.Value{
AccessKeyID: opt.AccessKeyID,
@@ -1540,7 +1540,7 @@ func s3Connection(ctx context.Context, opt *Options) (*s3.S3, *session.Session,
awsConfig := aws.NewConfig().
WithMaxRetries(0). // Rely on rclone's retry logic
WithCredentials(cred).
WithHTTPClient(getClient(ctx, opt)).
WithHTTPClient(client).
WithS3ForcePathStyle(opt.ForcePathStyle).
WithS3UseAccelerate(opt.UseAccelerateEndpoint).
WithS3UsEast1RegionalEndpoint(endpoints.RegionalS3UsEast1Endpoint)
@@ -1559,9 +1559,6 @@ func s3Connection(ctx context.Context, opt *Options) (*s3.S3, *session.Session,
if opt.EnvAuth && opt.AccessKeyID == "" && opt.SecretAccessKey == "" {
// Enable loading config options from ~/.aws/config (selected by AWS_PROFILE env)
awsSessionOpts.SharedConfigState = session.SharedConfigEnable
// The session constructor (aws/session/mergeConfigSrcs) will only use the user's preferred credential source
// (from the shared config file) if the passed-in Options.Config.Credentials is nil.
awsSessionOpts.Config.Credentials = nil
}
ses, err := session.NewSessionWithOptions(awsSessionOpts)
if err != nil {
@@ -1647,7 +1644,8 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
md5sumBinary := md5.Sum([]byte(opt.SSECustomerKey))
opt.SSECustomerKeyMD5 = base64.StdEncoding.EncodeToString(md5sumBinary[:])
}
c, ses, err := s3Connection(ctx, opt)
srv := getClient(ctx, opt)
c, ses, err := s3Connection(ctx, opt, srv)
if err != nil {
return nil, err
}
@@ -1662,7 +1660,7 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
ses: ses,
pacer: fs.NewPacer(ctx, pacer.NewS3(pacer.MinSleep(minSleep))),
cache: bucket.NewCache(),
srv: getClient(ctx, opt),
srv: srv,
pool: pool.New(
time.Duration(opt.MemoryPoolFlushTime),
int(opt.ChunkSize),
@@ -1697,12 +1695,9 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
f.setRoot(newRoot)
_, err := f.NewObject(ctx, leaf)
if err != nil {
if err == fs.ErrorObjectNotFound || err == fs.ErrorNotAFile {
// File doesn't exist or is a directory so return old f
f.setRoot(oldRoot)
return f, nil
}
return nil, err
// File doesn't exist or is a directory so return old f
f.setRoot(oldRoot)
return f, nil
}
// return an error with an fs which points to the parent
return f, fs.ErrorIsFile
@@ -1779,7 +1774,7 @@ func (f *Fs) updateRegionForBucket(bucket string) error {
// Make a new session with the new region
oldRegion := f.opt.Region
f.opt.Region = region
c, ses, err := s3Connection(f.ctx, &f.opt)
c, ses, err := s3Connection(f.ctx, &f.opt, f.srv)
if err != nil {
return errors.Wrap(err, "creating new session failed")
}
@@ -3298,10 +3293,11 @@ func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, op
}
// create the vanilla http request
httpReq, err := http.NewRequestWithContext(ctx, "PUT", url, in)
httpReq, err := http.NewRequest("PUT", url, in)
if err != nil {
return errors.Wrap(err, "s3 upload: new request")
}
httpReq = httpReq.WithContext(ctx) // go1.13 can use NewRequestWithContext
// set the headers we signed and the length
httpReq.Header = headers

View File

@@ -1,11 +1,10 @@
package swift
import (
"context"
"net/http"
"time"
"github.com/ncw/swift/v2"
"github.com/ncw/swift"
)
// auth is an authenticator for swift. It overrides the StorageUrl
@@ -29,19 +28,19 @@ func newAuth(parentAuth swift.Authenticator, storageURL string, authToken string
}
// Request creates an http.Request for the auth - return nil if not needed
func (a *auth) Request(ctx context.Context, c *swift.Connection) (*http.Request, error) {
func (a *auth) Request(c *swift.Connection) (*http.Request, error) {
if a.parentAuth == nil {
return nil, nil
}
return a.parentAuth.Request(ctx, c)
return a.parentAuth.Request(c)
}
// Response parses the http.Response
func (a *auth) Response(ctx context.Context, resp *http.Response) error {
func (a *auth) Response(resp *http.Response) error {
if a.parentAuth == nil {
return nil
}
return a.parentAuth.Response(ctx, resp)
return a.parentAuth.Response(resp)
}
// The public storage URL - set Internal to true to read

View File

@@ -13,7 +13,7 @@ import (
"strings"
"time"
"github.com/ncw/swift/v2"
"github.com/ncw/swift"
"github.com/pkg/errors"
"github.com/rclone/rclone/fs"
"github.com/rclone/rclone/fs/config"
@@ -391,7 +391,7 @@ func swiftConnection(ctx context.Context, opt *Options, name string) (*swift.Con
if c.AuthUrl == "" {
return nil, errors.New("auth not found")
}
err := c.Authenticate(ctx) // fills in c.StorageUrl and c.AuthToken
err := c.Authenticate() // fills in c.StorageUrl and c.AuthToken
if err != nil {
return nil, err
}
@@ -467,7 +467,7 @@ func NewFsWithConnection(ctx context.Context, opt *Options, name, root string, c
encodedDirectory := f.opt.Enc.FromStandardPath(f.rootDirectory)
err = f.pacer.Call(func() (bool, error) {
var rxHeaders swift.Headers
info, rxHeaders, err = f.c.Object(ctx, f.rootContainer, encodedDirectory)
info, rxHeaders, err = f.c.Object(f.rootContainer, encodedDirectory)
return shouldRetryHeaders(rxHeaders, err)
})
if err == nil && info.ContentType != directoryMarkerContentType {
@@ -506,7 +506,7 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
// Return an Object from a path
//
// If it can't be found it returns the error fs.ErrorObjectNotFound.
func (f *Fs) newObjectWithInfo(ctx context.Context, remote string, info *swift.Object) (fs.Object, error) {
func (f *Fs) newObjectWithInfo(remote string, info *swift.Object) (fs.Object, error) {
o := &Object{
fs: f,
remote: remote,
@@ -516,7 +516,7 @@ func (f *Fs) newObjectWithInfo(ctx context.Context, remote string, info *swift.O
// making sure we read the full metadata for all 0 byte files.
// We don't read the metadata for directory marker objects.
if info != nil && info.Bytes == 0 && info.ContentType != "application/directory" {
err := o.readMetaData(ctx) // reads info and headers, returning an error
err := o.readMetaData() // reads info and headers, returning an error
if err == fs.ErrorObjectNotFound {
// We have a dangling large object here so just return the original metadata
fs.Errorf(o, "dangling large object with no contents")
@@ -533,7 +533,7 @@ func (f *Fs) newObjectWithInfo(ctx context.Context, remote string, info *swift.O
return nil, err
}
} else {
err := o.readMetaData(ctx) // reads info and headers, returning an error
err := o.readMetaData() // reads info and headers, returning an error
if err != nil {
return nil, err
}
@@ -544,7 +544,7 @@ func (f *Fs) newObjectWithInfo(ctx context.Context, remote string, info *swift.O
// NewObject finds the Object at remote. If it can't be found it
// returns the error fs.ErrorObjectNotFound.
func (f *Fs) NewObject(ctx context.Context, remote string) (fs.Object, error) {
return f.newObjectWithInfo(ctx, remote, nil)
return f.newObjectWithInfo(remote, nil)
}
// listFn is called from list and listContainerRoot to handle an object.
@@ -556,7 +556,7 @@ type listFn func(remote string, object *swift.Object, isDirectory bool) error
// container to the start.
//
// Set recurse to read sub directories
func (f *Fs) listContainerRoot(ctx context.Context, container, directory, prefix string, addContainer bool, recurse bool, includeDirMarkers bool, fn listFn) error {
func (f *Fs) listContainerRoot(container, directory, prefix string, addContainer bool, recurse bool, includeDirMarkers bool, fn listFn) error {
if prefix != "" && !strings.HasSuffix(prefix, "/") {
prefix += "/"
}
@@ -571,11 +571,11 @@ func (f *Fs) listContainerRoot(ctx context.Context, container, directory, prefix
if !recurse {
opts.Delimiter = '/'
}
return f.c.ObjectsWalk(ctx, container, &opts, func(ctx context.Context, opts *swift.ObjectsOpts) (interface{}, error) {
return f.c.ObjectsWalk(container, &opts, func(opts *swift.ObjectsOpts) (interface{}, error) {
var objects []swift.Object
var err error
err = f.pacer.Call(func() (bool, error) {
objects, err = f.c.Objects(ctx, container, opts)
objects, err = f.c.Objects(container, opts)
return shouldRetry(err)
})
if err == nil {
@@ -613,8 +613,8 @@ func (f *Fs) listContainerRoot(ctx context.Context, container, directory, prefix
type addEntryFn func(fs.DirEntry) error
// list the objects into the function supplied
func (f *Fs) list(ctx context.Context, container, directory, prefix string, addContainer bool, recurse bool, includeDirMarkers bool, fn addEntryFn) error {
err := f.listContainerRoot(ctx, container, directory, prefix, addContainer, recurse, includeDirMarkers, func(remote string, object *swift.Object, isDirectory bool) (err error) {
func (f *Fs) list(container, directory, prefix string, addContainer bool, recurse bool, includeDirMarkers bool, fn addEntryFn) error {
err := f.listContainerRoot(container, directory, prefix, addContainer, recurse, includeDirMarkers, func(remote string, object *swift.Object, isDirectory bool) (err error) {
if isDirectory {
remote = strings.TrimRight(remote, "/")
d := fs.NewDir(remote, time.Time{}).SetSize(object.Bytes)
@@ -622,7 +622,7 @@ func (f *Fs) list(ctx context.Context, container, directory, prefix string, addC
} else {
// newObjectWithInfo does a full metadata read on 0 size objects which might be dynamic large objects
var o fs.Object
o, err = f.newObjectWithInfo(ctx, remote, object)
o, err = f.newObjectWithInfo(remote, object)
if err != nil {
return err
}
@@ -639,12 +639,12 @@ func (f *Fs) list(ctx context.Context, container, directory, prefix string, addC
}
// listDir lists a single directory
func (f *Fs) listDir(ctx context.Context, container, directory, prefix string, addContainer bool) (entries fs.DirEntries, err error) {
func (f *Fs) listDir(container, directory, prefix string, addContainer bool) (entries fs.DirEntries, err error) {
if container == "" {
return nil, fs.ErrorListBucketRequired
}
// List the objects
err = f.list(ctx, container, directory, prefix, addContainer, false, false, func(entry fs.DirEntry) error {
err = f.list(container, directory, prefix, addContainer, false, false, func(entry fs.DirEntry) error {
entries = append(entries, entry)
return nil
})
@@ -660,7 +660,7 @@ func (f *Fs) listDir(ctx context.Context, container, directory, prefix string, a
func (f *Fs) listContainers(ctx context.Context) (entries fs.DirEntries, err error) {
var containers []swift.Container
err = f.pacer.Call(func() (bool, error) {
containers, err = f.c.ContainersAll(ctx, nil)
containers, err = f.c.ContainersAll(nil)
return shouldRetry(err)
})
if err != nil {
@@ -691,7 +691,7 @@ func (f *Fs) List(ctx context.Context, dir string) (entries fs.DirEntries, err e
}
return f.listContainers(ctx)
}
return f.listDir(ctx, container, directory, f.rootDirectory, f.rootContainer == "")
return f.listDir(container, directory, f.rootDirectory, f.rootContainer == "")
}
// ListR lists the objects and directories of the Fs starting
@@ -714,7 +714,7 @@ func (f *Fs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) (
container, directory := f.split(dir)
list := walk.NewListRHelper(callback)
listR := func(container, directory, prefix string, addContainer bool) error {
return f.list(ctx, container, directory, prefix, addContainer, true, false, func(entry fs.DirEntry) error {
return f.list(container, directory, prefix, addContainer, true, false, func(entry fs.DirEntry) error {
return list.Add(entry)
})
}
@@ -752,7 +752,7 @@ func (f *Fs) About(ctx context.Context) (*fs.Usage, error) {
var containers []swift.Container
var err error
err = f.pacer.Call(func() (bool, error) {
containers, err = f.c.ContainersAll(ctx, nil)
containers, err = f.c.ContainersAll(nil)
return shouldRetry(err)
})
if err != nil {
@@ -804,7 +804,7 @@ func (f *Fs) makeContainer(ctx context.Context, container string) error {
if !f.noCheckContainer {
err = f.pacer.Call(func() (bool, error) {
var rxHeaders swift.Headers
_, rxHeaders, err = f.c.Container(ctx, container)
_, rxHeaders, err = f.c.Container(container)
return shouldRetryHeaders(rxHeaders, err)
})
}
@@ -814,7 +814,7 @@ func (f *Fs) makeContainer(ctx context.Context, container string) error {
headers["X-Storage-Policy"] = f.opt.StoragePolicy
}
err = f.pacer.Call(func() (bool, error) {
err = f.c.ContainerCreate(ctx, container, headers)
err = f.c.ContainerCreate(container, headers)
return shouldRetry(err)
})
if err == nil {
@@ -835,7 +835,7 @@ func (f *Fs) Rmdir(ctx context.Context, dir string) error {
}
err := f.cache.Remove(container, func() error {
err := f.pacer.Call(func() (bool, error) {
err := f.c.ContainerDelete(ctx, container)
err := f.c.ContainerDelete(container)
return shouldRetry(err)
})
if err == nil {
@@ -865,7 +865,7 @@ func (f *Fs) Purge(ctx context.Context, dir string) error {
go func() {
delErr <- operations.DeleteFiles(ctx, toBeDeleted)
}()
err := f.list(ctx, container, directory, f.rootDirectory, false, true, true, func(entry fs.DirEntry) error {
err := f.list(container, directory, f.rootDirectory, false, true, true, func(entry fs.DirEntry) error {
if o, ok := entry.(*Object); ok {
toBeDeleted <- o
}
@@ -905,7 +905,7 @@ func (f *Fs) Copy(ctx context.Context, src fs.Object, remote string) (fs.Object,
srcContainer, srcPath := srcObj.split()
err = f.pacer.Call(func() (bool, error) {
var rxHeaders swift.Headers
rxHeaders, err = f.c.ObjectCopy(ctx, srcContainer, srcPath, dstContainer, dstPath, nil)
rxHeaders, err = f.c.ObjectCopy(srcContainer, srcPath, dstContainer, dstPath, nil)
return shouldRetryHeaders(rxHeaders, err)
})
if err != nil {
@@ -944,11 +944,11 @@ func (o *Object) Hash(ctx context.Context, t hash.Type) (string, error) {
if t != hash.MD5 {
return "", hash.ErrUnsupported
}
isDynamicLargeObject, err := o.isDynamicLargeObject(ctx)
isDynamicLargeObject, err := o.isDynamicLargeObject()
if err != nil {
return "", err
}
isStaticLargeObject, err := o.isStaticLargeObject(ctx)
isStaticLargeObject, err := o.isStaticLargeObject()
if err != nil {
return "", err
}
@@ -961,8 +961,8 @@ func (o *Object) Hash(ctx context.Context, t hash.Type) (string, error) {
// hasHeader checks for the header passed in returning false if the
// object isn't found.
func (o *Object) hasHeader(ctx context.Context, header string) (bool, error) {
err := o.readMetaData(ctx)
func (o *Object) hasHeader(header string) (bool, error) {
err := o.readMetaData()
if err != nil {
if err == fs.ErrorObjectNotFound {
return false, nil
@@ -974,29 +974,29 @@ func (o *Object) hasHeader(ctx context.Context, header string) (bool, error) {
}
// isDynamicLargeObject checks for X-Object-Manifest header
func (o *Object) isDynamicLargeObject(ctx context.Context) (bool, error) {
return o.hasHeader(ctx, "X-Object-Manifest")
func (o *Object) isDynamicLargeObject() (bool, error) {
return o.hasHeader("X-Object-Manifest")
}
// isStaticLargeObjectFile checks for the X-Static-Large-Object header
func (o *Object) isStaticLargeObject(ctx context.Context) (bool, error) {
return o.hasHeader(ctx, "X-Static-Large-Object")
func (o *Object) isStaticLargeObject() (bool, error) {
return o.hasHeader("X-Static-Large-Object")
}
func (o *Object) isLargeObject(ctx context.Context) (result bool, err error) {
result, err = o.hasHeader(ctx, "X-Static-Large-Object")
func (o *Object) isLargeObject() (result bool, err error) {
result, err = o.hasHeader("X-Static-Large-Object")
if result {
return
}
result, err = o.hasHeader(ctx, "X-Object-Manifest")
result, err = o.hasHeader("X-Object-Manifest")
if result {
return
}
return false, nil
}
func (o *Object) isInContainerVersioning(ctx context.Context, container string) (bool, error) {
_, headers, err := o.fs.c.Container(ctx, container)
func (o *Object) isInContainerVersioning(container string) (bool, error) {
_, headers, err := o.fs.c.Container(container)
if err != nil {
return false, err
}
@@ -1032,7 +1032,7 @@ func (o *Object) decodeMetaData(info *swift.Object) (err error) {
// it also sets the info
//
// it returns fs.ErrorObjectNotFound if the object isn't found
func (o *Object) readMetaData(ctx context.Context) (err error) {
func (o *Object) readMetaData() (err error) {
if o.headers != nil {
return nil
}
@@ -1040,7 +1040,7 @@ func (o *Object) readMetaData(ctx context.Context) (err error) {
var h swift.Headers
container, containerPath := o.split()
err = o.fs.pacer.Call(func() (bool, error) {
info, h, err = o.fs.c.Object(ctx, container, containerPath)
info, h, err = o.fs.c.Object(container, containerPath)
return shouldRetryHeaders(h, err)
})
if err != nil {
@@ -1066,7 +1066,7 @@ func (o *Object) ModTime(ctx context.Context) time.Time {
if o.fs.ci.UseServerModTime {
return o.lastModified
}
err := o.readMetaData(ctx)
err := o.readMetaData()
if err != nil {
fs.Debugf(o, "Failed to read metadata: %s", err)
return o.lastModified
@@ -1081,7 +1081,7 @@ func (o *Object) ModTime(ctx context.Context) time.Time {
// SetModTime sets the modification time of the local fs object
func (o *Object) SetModTime(ctx context.Context, modTime time.Time) error {
err := o.readMetaData(ctx)
err := o.readMetaData()
if err != nil {
return err
}
@@ -1099,7 +1099,7 @@ func (o *Object) SetModTime(ctx context.Context, modTime time.Time) error {
}
container, containerPath := o.split()
return o.fs.pacer.Call(func() (bool, error) {
err = o.fs.c.ObjectUpdate(ctx, container, containerPath, newHeaders)
err = o.fs.c.ObjectUpdate(container, containerPath, newHeaders)
return shouldRetry(err)
})
}
@@ -1120,7 +1120,7 @@ func (o *Object) Open(ctx context.Context, options ...fs.OpenOption) (in io.Read
container, containerPath := o.split()
err = o.fs.pacer.Call(func() (bool, error) {
var rxHeaders swift.Headers
in, rxHeaders, err = o.fs.c.ObjectOpen(ctx, container, containerPath, !isRanging, headers)
in, rxHeaders, err = o.fs.c.ObjectOpen(container, containerPath, !isRanging, headers)
return shouldRetryHeaders(rxHeaders, err)
})
return
@@ -1134,9 +1134,9 @@ func min(x, y int64) int64 {
return y
}
func (o *Object) getSegmentsLargeObject(ctx context.Context) (map[string][]string, error) {
func (o *Object) getSegmentsLargeObject() (map[string][]string, error) {
container, objectName := o.split()
segmentContainer, segmentObjects, err := o.fs.c.LargeObjectGetSegments(ctx, container, objectName)
segmentContainer, segmentObjects, err := o.fs.c.LargeObjectGetSegments(container, objectName)
if err != nil {
fs.Debugf(o, "Failed to get list segments of object: %v", err)
return nil, err
@@ -1153,12 +1153,12 @@ func (o *Object) getSegmentsLargeObject(ctx context.Context) (map[string][]strin
return containerSegments, nil
}
func (o *Object) removeSegmentsLargeObject(ctx context.Context, containerSegments map[string][]string) error {
func (o *Object) removeSegmentsLargeObject(containerSegments map[string][]string) error {
if containerSegments == nil || len(containerSegments) <= 0 {
return nil
}
for container, segments := range containerSegments {
_, err := o.fs.c.BulkDelete(ctx, container, segments)
_, err := o.fs.c.BulkDelete(container, segments)
if err != nil {
fs.Debugf(o, "Failed to delete bulk segments %v", err)
return err
@@ -1167,8 +1167,8 @@ func (o *Object) removeSegmentsLargeObject(ctx context.Context, containerSegment
return nil
}
func (o *Object) getSegmentsDlo(ctx context.Context) (segmentsContainer string, prefix string, err error) {
if err = o.readMetaData(ctx); err != nil {
func (o *Object) getSegmentsDlo() (segmentsContainer string, prefix string, err error) {
if err = o.readMetaData(); err != nil {
return
}
dirManifest := o.headers["X-Object-Manifest"]
@@ -1203,14 +1203,14 @@ func urlEncode(str string) string {
// updateChunks updates the existing object using chunks to a separate
// container. It returns a string which prefixes current segments.
func (o *Object) updateChunks(ctx context.Context, in0 io.Reader, headers swift.Headers, size int64, contentType string) (string, error) {
func (o *Object) updateChunks(in0 io.Reader, headers swift.Headers, size int64, contentType string) (string, error) {
container, containerPath := o.split()
segmentsContainer := container + "_segments"
// Create the segmentsContainer if it doesn't exist
var err error
err = o.fs.pacer.Call(func() (bool, error) {
var rxHeaders swift.Headers
_, rxHeaders, err = o.fs.c.Container(ctx, segmentsContainer)
_, rxHeaders, err = o.fs.c.Container(segmentsContainer)
return shouldRetryHeaders(rxHeaders, err)
})
if err == swift.ContainerNotFound {
@@ -1219,7 +1219,7 @@ func (o *Object) updateChunks(ctx context.Context, in0 io.Reader, headers swift.
headers["X-Storage-Policy"] = o.fs.opt.StoragePolicy
}
err = o.fs.pacer.Call(func() (bool, error) {
err = o.fs.c.ContainerCreate(ctx, segmentsContainer, headers)
err = o.fs.c.ContainerCreate(segmentsContainer, headers)
return shouldRetry(err)
})
}
@@ -1241,7 +1241,7 @@ func (o *Object) updateChunks(ctx context.Context, in0 io.Reader, headers swift.
if segmentInfos == nil || len(segmentInfos) == 0 {
return
}
deleteChunks(ctx, o, segmentsContainer, segmentInfos)
deleteChunks(o, segmentsContainer, segmentInfos)
})()
for {
// can we read at least one byte?
@@ -1263,7 +1263,7 @@ func (o *Object) updateChunks(ctx context.Context, in0 io.Reader, headers swift.
fs.Debugf(o, "Uploading segment file %q into %q", segmentPath, segmentsContainer)
err = o.fs.pacer.CallNoRetry(func() (bool, error) {
var rxHeaders swift.Headers
rxHeaders, err = o.fs.c.ObjectPut(ctx, segmentsContainer, segmentPath, segmentReader, true, "", "", headers)
rxHeaders, err = o.fs.c.ObjectPut(segmentsContainer, segmentPath, segmentReader, true, "", "", headers)
if err == nil {
segmentInfos = append(segmentInfos, segmentPath)
}
@@ -1280,7 +1280,7 @@ func (o *Object) updateChunks(ctx context.Context, in0 io.Reader, headers swift.
emptyReader := bytes.NewReader(nil)
err = o.fs.pacer.Call(func() (bool, error) {
var rxHeaders swift.Headers
rxHeaders, err = o.fs.c.ObjectPut(ctx, container, containerPath, emptyReader, true, "", contentType, headers)
rxHeaders, err = o.fs.c.ObjectPut(container, containerPath, emptyReader, true, "", contentType, headers)
return shouldRetryHeaders(rxHeaders, err)
})
@@ -1291,13 +1291,13 @@ func (o *Object) updateChunks(ctx context.Context, in0 io.Reader, headers swift.
return uniquePrefix + "/", err
}
func deleteChunks(ctx context.Context, o *Object, segmentsContainer string, segmentInfos []string) {
func deleteChunks(o *Object, segmentsContainer string, segmentInfos []string) {
if segmentInfos == nil || len(segmentInfos) == 0 {
return
}
for _, v := range segmentInfos {
fs.Debugf(o, "Delete segment file %q on %q", v, segmentsContainer)
e := o.fs.c.ObjectDelete(ctx, segmentsContainer, v)
e := o.fs.c.ObjectDelete(segmentsContainer, v)
if e != nil {
fs.Errorf(o, "Error occurred in delete segment file %q on %q, error: %q", v, segmentsContainer, e)
}
@@ -1320,7 +1320,7 @@ func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, op
modTime := src.ModTime(ctx)
// Note whether this is a dynamic large object before starting
isLargeObject, err := o.isLargeObject(ctx)
isLargeObject, err := o.isLargeObject()
if err != nil {
return err
}
@@ -1328,7 +1328,7 @@ func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, op
//capture segments before upload
var segmentsContainer map[string][]string
if isLargeObject {
segmentsContainer, _ = o.getSegmentsLargeObject(ctx)
segmentsContainer, _ = o.getSegmentsLargeObject()
}
// Set the mtime
@@ -1339,7 +1339,7 @@ func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, op
fs.OpenOptionAddHeaders(options, headers)
if size > int64(o.fs.opt.ChunkSize) || (size == -1 && !o.fs.opt.NoChunk) {
_, err = o.updateChunks(ctx, in, headers, size, contentType)
_, err = o.updateChunks(in, headers, size, contentType)
if err != nil {
return err
}
@@ -1355,7 +1355,7 @@ func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, op
}
var rxHeaders swift.Headers
err = o.fs.pacer.CallNoRetry(func() (bool, error) {
rxHeaders, err = o.fs.c.ObjectPut(ctx, container, containerPath, in, true, "", contentType, headers)
rxHeaders, err = o.fs.c.ObjectPut(container, containerPath, in, true, "", contentType, headers)
return shouldRetryHeaders(rxHeaders, err)
})
if err != nil {
@@ -1373,17 +1373,17 @@ func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, op
o.size = int64(inCount.BytesRead())
}
}
isInContainerVersioning, _ := o.isInContainerVersioning(ctx, container)
isInContainerVersioning, _ := o.isInContainerVersioning(container)
// If file was a large object and the container is not enable versioning then remove old/all segments
if isLargeObject && len(segmentsContainer) > 0 && !isInContainerVersioning {
err := o.removeSegmentsLargeObject(ctx, segmentsContainer)
err := o.removeSegmentsLargeObject(segmentsContainer)
if err != nil {
fs.Logf(o, "Failed to remove old segments - carrying on with upload: %v", err)
}
}
// Read the metadata from the newly created object if necessary
return o.readMetaData(ctx)
return o.readMetaData()
}
// Remove an object
@@ -1391,14 +1391,14 @@ func (o *Object) Remove(ctx context.Context) (err error) {
container, containerPath := o.split()
//check object is large object
isLargeObject, err := o.isLargeObject(ctx)
isLargeObject, err := o.isLargeObject()
if err != nil {
return err
}
//check container has enabled version to reserve segment when delete
isInContainerVersioning := false
if isLargeObject {
isInContainerVersioning, err = o.isInContainerVersioning(ctx, container)
isInContainerVersioning, err = o.isInContainerVersioning(container)
if err != nil {
return err
}
@@ -1406,14 +1406,14 @@ func (o *Object) Remove(ctx context.Context) (err error) {
//capture segments object if this object is large object
var containerSegments map[string][]string
if isLargeObject {
containerSegments, err = o.getSegmentsLargeObject(ctx)
containerSegments, err = o.getSegmentsLargeObject()
if err != nil {
return err
}
}
// Remove file/manifest first
err = o.fs.pacer.Call(func() (bool, error) {
err = o.fs.c.ObjectDelete(ctx, container, containerPath)
err = o.fs.c.ObjectDelete(container, containerPath)
return shouldRetry(err)
})
if err != nil {
@@ -1425,7 +1425,7 @@ func (o *Object) Remove(ctx context.Context) (err error) {
}
if isLargeObject {
return o.removeSegmentsLargeObject(ctx, containerSegments)
return o.removeSegmentsLargeObject(containerSegments)
}
return nil
}

View File

@@ -4,7 +4,7 @@ import (
"testing"
"time"
"github.com/ncw/swift/v2"
"github.com/ncw/swift"
"github.com/rclone/rclone/fs/fserrors"
"github.com/stretchr/testify/assert"
)

View File

@@ -9,7 +9,7 @@ import (
"io/ioutil"
"testing"
"github.com/ncw/swift/v2"
"github.com/ncw/swift"
"github.com/rclone/rclone/fs"
"github.com/rclone/rclone/fs/hash"
"github.com/rclone/rclone/fs/object"
@@ -144,10 +144,10 @@ func (f *Fs) testWithChunkFail(t *testing.T) {
// error is potato
require.NotNil(t, err)
require.Equal(t, errMessage, err.Error())
_, _, err = f.c.Object(ctx, f.rootContainer, path)
_, _, err = f.c.Object(f.rootContainer, path)
assert.Equal(t, swift.ObjectNotFound, err)
prefix := path
objs, err := f.c.Objects(ctx, segmentContainer, &swift.ObjectsOpts{
objs, err := f.c.Objects(segmentContainer, &swift.ObjectsOpts{
Prefix: prefix,
})
require.NoError(t, err)

View File

@@ -1,4 +1,4 @@
// +build !plan9
// +build go1.13,!plan9
// Package tardigrade provides an interface to Tardigrade decentralized object storage.
package tardigrade

View File

@@ -1,4 +1,4 @@
// +build !plan9
// +build go1.13,!plan9
package tardigrade

View File

@@ -1,4 +1,4 @@
// +build !plan9
// +build go1.13,!plan9
// Test Tardigrade filesystem interface
package tardigrade_test

View File

@@ -1,3 +1,3 @@
// +build plan9
// +build !go1.13 plan9
package tardigrade

View File

@@ -67,7 +67,7 @@ func New(ctx context.Context, remote, root string, cacheTime time.Duration) (*Fs
return nil, err
}
f := &Fs{
RootPath: root,
RootPath: strings.TrimRight(root, "/"),
writable: true,
creatable: true,
cacheExpiry: time.Now().Unix(),

View File

@@ -176,10 +176,11 @@ func (ca *CookieAuth) getSPToken(ctx context.Context) (conf *SharepointSuccessRe
// Create and execute the first request which returns an auth token for the sharepoint service
// With this token we can authenticate on the login page and save the returned cookies
req, err := http.NewRequestWithContext(ctx, "POST", "https://login.microsoftonline.com/extSTS.srf", buf)
req, err := http.NewRequest("POST", "https://login.microsoftonline.com/extSTS.srf", buf)
if err != nil {
return nil, err
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
client := fshttp.NewClient(ctx)
resp, err := client.Do(req)

View File

@@ -100,7 +100,7 @@ func init() {
log.Fatalf("Failed to configure root directory: %v", err)
}
},
Options: []fs.Option{{
Options: append(oauthutil.SharedOptions, []fs.Option{{
Name: "region",
Help: "Zoho region to connect to. You'll have to use the region you organization is registered in.",
Examples: []fs.OptionExample{{
@@ -123,7 +123,7 @@ func init() {
encoder.EncodeCtl |
encoder.EncodeDel |
encoder.EncodeInvalidUtf8),
}},
}}...),
})
}

View File

@@ -72,7 +72,7 @@ var osarches = []string{
// Special environment flags for a given arch
var archFlags = map[string][]string{
"386": {"GO386=softfloat"},
"386": {"GO386=387"},
"mips": {"GOMIPS=softfloat"},
"mipsle": {"GOMIPS=softfloat"},
"arm-v7": {"GOARM=7"},

View File

@@ -1,4 +1,4 @@
// +build linux freebsd
// +build linux,go1.13 freebsd,go1.13
package mount

View File

@@ -1,4 +1,4 @@
// +build linux freebsd
// +build linux,go1.13 freebsd,go1.13
package mount

View File

@@ -1,6 +1,6 @@
// FUSE main Fs
// +build linux freebsd
// +build linux,go1.13 freebsd,go1.13
package mount

View File

@@ -1,4 +1,4 @@
// +build linux freebsd
// +build linux,go1.13 freebsd,go1.13
package mount

View File

@@ -1,6 +1,6 @@
// Package mount implements a FUSE mounting system for rclone remotes.
// +build linux freebsd
// +build linux,go1.13 freebsd,go1.13
package mount

View File

@@ -1,4 +1,4 @@
// +build linux freebsd
// +build linux,go1.13 freebsd,go1.13
package mount

View File

@@ -1,9 +1,13 @@
// Build for mount for unsupported platforms to stop go complaining
// about "no buildable Go source files "
// Invert the build constraint: linux freebsd
// Invert the build constraint: linux,go1.13 freebsd,go1.13
//
// !((linux&&go1.13) || (freebsd&&go1.13))
// == !(linux&&go1.13) && !(freebsd&&go1.13))
// == (!linux || !go1.13) && (!freebsd || !go1.13))
// +build !linux
// +build !freebsd
// +build !linux !go1.13
// +build !freebsd !go1.13
package mount

View File

@@ -71,7 +71,7 @@ const (
func init() {
// DaemonTimeout defaults to non zero for macOS
if runtime.GOOS == "darwin" {
DefaultOpt.DaemonTimeout = 15 * time.Minute
DefaultOpt.DaemonTimeout = 10 * time.Minute
}
}
@@ -348,7 +348,7 @@ Without the use of |--vfs-cache-mode| this can only write files
sequentially, it can only seek when reading. This means that many
applications won't work with their files on an rclone mount without
|--vfs-cache-mode writes| or |--vfs-cache-mode full|.
See the [File Caching](#file-caching) section for more info.
See the [VFS File Caching](#vfs-file-caching) section for more info.
The bucket based remotes (e.g. Swift, S3, Google Compute Storage, B2,
Hubic) do not support the concept of empty directories, so empty
@@ -363,7 +363,7 @@ File systems expect things to be 100% reliable, whereas cloud storage
systems are a long way from 100% reliable. The rclone sync/copy
commands cope with this with lots of retries. However rclone @
can't use retries in the same way without making local copies of the
uploads. Look at the [file caching](#file-caching)
uploads. Look at the [VFS File Caching](#vfs-file-caching)
for solutions to make @ more reliable.
### Attribute caching

View File

@@ -184,10 +184,11 @@ func doCall(ctx context.Context, path string, in rc.Params) (out rc.Params, err
return nil, errors.Wrap(err, "failed to encode JSON")
}
req, err := http.NewRequestWithContext(ctx, "POST", url, bytes.NewBuffer(data))
req, err := http.NewRequest("POST", url, bytes.NewBuffer(data))
if err != nil {
return nil, errors.Wrap(err, "failed to make request")
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
req.Header.Set("Content-Type", "application/json")
if authUser != "" || authPass != "" {

View File

@@ -1,6 +1,6 @@
// Package ftp implements an FTP server for rclone
//+build !plan9
//+build !plan9,go1.13
package ftp

View File

@@ -3,7 +3,7 @@
//
// We skip tests on platforms with troublesome character mappings
//+build !windows,!darwin,!plan9
//+build !windows,!darwin,!plan9,go1.13
package ftp

View File

@@ -1,7 +1,7 @@
// Build for unsupported platforms to stop go complaining
// about "no buildable Go source files "
// +build plan9
// +build plan9 !go1.13
package ftp

View File

@@ -205,7 +205,7 @@ These URLs are used by Plex internally to connect to the Plex server securely.
The format for these URLs is the following:
https://ip-with-dots-replaced.server-hash.plex.direct:32400/
`https://ip-with-dots-replaced.server-hash.plex.direct:32400/`
The `ip-with-dots-replaced` part can be any IPv4 address, where the dots
have been replaced with dashes, e.g. `127.0.0.1` becomes `127-0-0-1`.

View File

@@ -5,6 +5,43 @@ description: "Rclone Changelog"
# Changelog
## v1.54.1 - 2021-03-08
[See commits](https://github.com/rclone/rclone/compare/v1.54.0...v1.54.1)
* Bug Fixes
* accounting: Fix --bwlimit when up or down is off (Nick Craig-Wood)
* docs
* Fix nesting of brackets and backticks in ftp docs (edwardxml)
* Fix broken link in sftp page (edwardxml)
* Fix typo in crypt.md (Romeo Kienzler)
* Changelog: Correct link to digitalis.io (Alex JOST)
* Replace #file-caching with #vfs-file-caching (Miron Veryanskiy)
* Convert bogus example link to code (edwardxml)
* Remove dead link from rc.md (edwardxml)
* rc: Sync,copy,move: document createEmptySrcDirs parameter (Nick Craig-Wood)
* lsjson: Fix unterminated JSON in the presence of errors (Nick Craig-Wood)
* Mount
* Fix mount dropping on macOS by setting --daemon-timeout 10m (Nick Craig-Wood)
* VFS
* Document simultaneous usage with the same cache shouldn't be used (Nick Craig-Wood)
* B2
* Automatically raise upload cutoff to avoid spurious error (Nick Craig-Wood)
* Fix failed to create file system with application key limited to a prefix (Nick Craig-Wood)
* Drive
* Refer to Shared Drives instead of Team Drives (Nick Craig-Wood)
* Dropbox
* Add scopes to oauth request and optionally "members.read" (Nick Craig-Wood)
* S3
* Fix failed to create file system with folder level permissions policy (Nick Craig-Wood)
* Fix Wasabi HEAD requests returning stale data by using only 1 transport (Nick Craig-Wood)
* Fix shared_credentials_file auth (Dmitry Chepurovskiy)
* Add --s3-no-head to reducing costs docs (Nick Craig-Wood)
* Union
* Fix mkdir at root with remote:/ (Nick Craig-Wood)
* Zoho
* Fix custom client id's (buengese)
## v1.54.0 - 2021-02-02
[See commits](https://github.com/rclone/rclone/compare/v1.53.0...v1.54.0)

View File

@@ -198,7 +198,7 @@ Without the use of `--vfs-cache-mode` this can only write files
sequentially, it can only seek when reading. This means that many
applications won't work with their files on an rclone mount without
`--vfs-cache-mode writes` or `--vfs-cache-mode full`.
See the [File Caching](#file-caching) section for more info.
See the [VFS File Caching](#vfs-file-caching) section for more info.
The bucket based remotes (e.g. Swift, S3, Google Compute Storage, B2,
Hubic) do not support the concept of empty directories, so empty
@@ -213,7 +213,7 @@ File systems expect things to be 100% reliable, whereas cloud storage
systems are a long way from 100% reliable. The rclone sync/copy
commands cope with this with lots of retries. However rclone mount
can't use retries in the same way without making local copies of the
uploads. Look at the [file caching](#file-caching)
uploads. Look at the [VFS File Caching](#vfs-file-caching)
for solutions to make mount more reliable.
## Attribute caching
@@ -378,6 +378,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write

View File

@@ -134,6 +134,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write

View File

@@ -133,6 +133,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write

View File

@@ -205,6 +205,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write

View File

@@ -144,6 +144,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write

View File

@@ -213,6 +213,13 @@ for two reasons. Firstly because it is only checked every
`--vfs-cache-poll-interval`. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using `--vfs-cache-mode > off`.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
`--cache-dir`. You don't need to worry about this if the remotes in
use don't overlap.
### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write

View File

@@ -82,7 +82,7 @@ as you would with any other remote, e.g. `rclone copy D:\docs secret:\docs`,
and rclone will encrypt and decrypt as needed on the fly.
If you access the wrapped remote `remote:path` directly you will bypass
the encryption, and anything you read will be in encrypted form, and
anything you write will be undencrypted. To avoid issues it is best to
anything you write will be unencrypted. To avoid issues it is best to
configure a dedicated path for encrypted content, and access it
exclusively through a crypt remote.

View File

@@ -72,7 +72,7 @@ If your browser doesn't open automatically go to the following link: http://127.
Log in and authorize rclone for access
Waiting for code...
Got code
Configure this as a team drive?
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No
y/n> n
@@ -279,23 +279,24 @@ Note: in case you configured a specific root folder on gdrive and rclone is unab
`rclone -v foo@example.com lsf gdrive:backup`
### Team drives ###
### Shared drives (team drives) ###
If you want to configure the remote to point to a Google Team Drive
then answer `y` to the question `Configure this as a team drive?`.
If you want to configure the remote to point to a Google Shared Drive
(previously known as Team Drives) then answer `y` to the question
`Configure this as a Shared Drive (Team Drive)?`.
This will fetch the list of Team Drives from google and allow you to
configure which one you want to use. You can also type in a team
drive ID if you prefer.
This will fetch the list of Shared Drives from google and allow you to
configure which one you want to use. You can also type in a Shared
Drive ID if you prefer.
For example:
```
Configure this as a team drive?
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No
y/n> y
Fetching team drive list...
Fetching Shared Drive list...
Choose a number from below, or type in your own value
1 / Rclone Test
\ "xxxxxxxxxxxxxxxxxxxx"
@@ -303,7 +304,7 @@ Choose a number from below, or type in your own value
\ "yyyyyyyyyyyyyyyyyyyy"
3 / Rclone Test 3
\ "zzzzzzzzzzzzzzzzzzzz"
Enter a Team Drive ID> 1
Enter a Shared Drive ID> 1
--------------------
[remote]
client_id =
@@ -674,7 +675,7 @@ Needed only if you want use SA instead of interactive login.
#### --drive-team-drive
ID of the Team Drive
ID of the Shared Drive (Team Drive)
- Config: team_drive
- Env Var: RCLONE_DRIVE_TEAM_DRIVE
@@ -1137,11 +1138,11 @@ Options:
#### drives
List the shared drives available to this account
List the Shared Drives available to this account
rclone backend drives remote: [options] [<arguments>+]
This command lists the shared drives (teamdrives) available to this
This command lists the Shared Drives (Team Drives) available to this
account.
Usage:

View File

@@ -197,6 +197,21 @@ memory. It can be set smaller if you are tight on memory.
Impersonate this user when using a business account.
Note that if you want to use impersonate, you should make sure this
flag is set when running "rclone config" as this will cause rclone to
request the "members.read" scope which it won't normally. This is
needed to lookup a members email address into the internal ID that
dropbox uses in the API.
Using the "members.read" scope will require a Dropbox Team Admin
to approve during the OAuth flow.
You will have to use your own App (setting your own client_id and
client_secret) to use this option as currently rclone's default set of
permissions doesn't include "members.read". This can be added once
v1.55 or later is in use everywhere.
- Config: impersonate
- Env Var: RCLONE_DROPBOX_IMPERSONATE
- Type: string

View File

@@ -643,7 +643,7 @@ not list `dir3`, `file3` or `.ignore`.
## Common pitfalls
The most frequent filter support issues on
the [rclone forum](https://https://forum.rclone.org/) are:
the [rclone forum](https://forum.rclone.org/) are:
* Not using paths relative to the root of the remote
* Not using `/` to match from the root of a remote

View File

@@ -150,7 +150,7 @@ These flags are available for every command.
--use-json-log Use json log format.
--use-mmap Use mmap allocator (see docs).
--use-server-modtime Use server modified time instead of object metadata
--user-agent string Set the user-agent to a specified string. The default is rclone/ version (default "rclone/v1.54.0")
--user-agent string Set the user-agent to a specified string. The default is rclone/ version (default "rclone/v1.54.1")
-v, --verbose count Print lots more stuff (repeat for more)
```
@@ -281,7 +281,7 @@ and may be set in the config file.
--drive-starred-only Only show files that are starred.
--drive-stop-on-download-limit Make download limit errors be fatal
--drive-stop-on-upload-limit Make upload limit errors be fatal
--drive-team-drive string ID of the Team Drive
--drive-team-drive string ID of the Shared Drive (Team Drive)
--drive-token string OAuth Access Token as a JSON blob.
--drive-token-url string Token server url.
--drive-trashed-only Only show files that are in the trash.
@@ -562,6 +562,11 @@ and may be set in the config file.
--yandex-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,InvalidUtf8,Dot)
--yandex-token string OAuth Access Token as a JSON blob.
--yandex-token-url string Token server url.
--zoho-auth-url string Auth server URL.
--zoho-client-id string OAuth Client Id
--zoho-client-secret string OAuth Client Secret
--zoho-encoding MultiEncoder This sets the encoding for the backend. (default Del,Ctl,InvalidUtf8)
--zoho-region string Zoho region to connect to. You'll have to use the region you organization is registered in.
--zoho-token string OAuth Access Token as a JSON blob.
--zoho-token-url string Token server url.
```

View File

@@ -109,8 +109,8 @@ excess files in the directory.
Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to
be enabled in the FTP backend config for the remote, or with
`[--ftp-tls]{#ftp-tls}`. The default FTPS port is `990`, not `21` and
can be set with `[--ftp-port]{#ftp-port}`.
[`--ftp-tls`](#ftp-tls). The default FTPS port is `990`, not `21` and
can be set with [`--ftp-port`](#ftp-port).
{{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/ftp/ftp.go then run make backenddocs" >}}
### Standard Options

View File

@@ -181,7 +181,7 @@ kill %1
## Install from source ##
Make sure you have at least [Go](https://golang.org/) go1.13
Make sure you have at least [Go](https://golang.org/) 1.12
installed. [Download go](https://golang.org/dl/) if necessary. The
latest release is recommended. Then

View File

@@ -1288,6 +1288,7 @@ This takes the following parameters
- srcFs - a remote name string e.g. "drive:src" for the source
- dstFs - a remote name string e.g. "drive:dst" for the destination
- createEmptySrcDirs - create empty src directories on destination if set
See the [copy command](/commands/rclone_copy/) command for more information on the above.
@@ -1300,6 +1301,7 @@ This takes the following parameters
- srcFs - a remote name string e.g. "drive:src" for the source
- dstFs - a remote name string e.g. "drive:dst" for the destination
- createEmptySrcDirs - create empty src directories on destination if set
- deleteEmptySrcDirs - delete empty src directories if set
@@ -1313,6 +1315,7 @@ This takes the following parameters
- srcFs - a remote name string e.g. "drive:src" for the source
- dstFs - a remote name string e.g. "drive:dst" for the destination
- createEmptySrcDirs - create empty src directories on destination if set
See the [sync command](/commands/rclone_sync/) command for more information on the above.

View File

@@ -526,8 +526,8 @@ The Go SSH library disables the use of the aes128-cbc cipher by
default, due to security concerns. This can be re-enabled on a
per-connection basis by setting the `use_insecure_cipher` setting in
the configuration file to `true`. Further details on the insecurity of
this cipher can be found [in this paper]
(http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).
this cipher can be found
[in this paper](http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).
SFTP isn't supported under plan9 until [this
issue](https://github.com/pkg/sftp/issues/156) is fixed.

View File

@@ -127,6 +127,26 @@ from filenames during upload.
Here are the standard options specific to zoho (Zoho).
#### --zoho-client-id
OAuth Client Id
Leave blank normally.
- Config: client_id
- Env Var: RCLONE_ZOHO_CLIENT_ID
- Type: string
- Default: ""
#### --zoho-client-secret
OAuth Client Secret
Leave blank normally.
- Config: client_secret
- Env Var: RCLONE_ZOHO_CLIENT_SECRET
- Type: string
- Default: ""
#### --zoho-region
Zoho region to connect to. You'll have to use the region you organization is registered in.
@@ -149,6 +169,35 @@ Zoho region to connect to. You'll have to use the region you organization is reg
Here are the advanced options specific to zoho (Zoho).
#### --zoho-token
OAuth Access Token as a JSON blob.
- Config: token
- Env Var: RCLONE_ZOHO_TOKEN
- Type: string
- Default: ""
#### --zoho-auth-url
Auth server URL.
Leave blank to use the provider defaults.
- Config: auth_url
- Env Var: RCLONE_ZOHO_AUTH_URL
- Type: string
- Default: ""
#### --zoho-token-url
Token server url.
Leave blank to use the provider defaults.
- Config: token_url
- Env Var: RCLONE_ZOHO_TOKEN_URL
- Type: string
- Default: ""
#### --zoho-encoding
This sets the encoding for the backend.

View File

@@ -1 +1 @@
v1.55.0
v1.54.1

View File

@@ -68,7 +68,8 @@ func newTokenBucket(bandwidth fs.BwPair) (tbs buckets) {
bandwidthAccounting = bandwidth.Rx
}
}
if bandwidthAccounting > 0 {
// Limit core bandwidth to max of Rx and Tx if both are limited
if bandwidth.Tx > 0 && bandwidth.Rx > 0 {
tbs[TokenBucketSlotAccounting] = rate.NewLimiter(rate.Limit(bandwidthAccounting), maxBurstSize)
}
for _, tb := range tbs {

View File

@@ -24,6 +24,7 @@ func init() {
- srcFs - a remote name string e.g. "drive:src" for the source
- dstFs - a remote name string e.g. "drive:dst" for the destination
- createEmptySrcDirs - create empty src directories on destination if set
` + moveHelp + `
See the [` + name + ` command](/commands/rclone_` + name + `/) command for more information on the above.`,

View File

@@ -1,4 +1,4 @@
package fs
// Version of rclone
var Version = "v1.55.0-DEV"
var Version = "v1.54.1-DEV"

View File

@@ -1,7 +1,7 @@
//+build !go1.13
//+build !go1.12
package fs
// Upgrade to Go version 1.13 to compile rclone - latest stable go
// Upgrade to Go version 1.12 to compile rclone - latest stable go
// compiler recommended.
func init() { Go_version_1_13_required_for_compilation() }
func init() { Go_version_1_12_required_for_compilation() }

62
go.mod
View File

@@ -4,78 +4,72 @@ go 1.14
require (
bazil.org/fuse v0.0.0-20200524192727-fb710f7dfd05
cloud.google.com/go v0.76.0 // indirect
cloud.google.com/go v0.70.0 // indirect
github.com/Azure/azure-pipeline-go v0.2.3
github.com/Azure/azure-storage-blob-go v0.13.0
github.com/Azure/go-autorest/autorest/adal v0.9.10
github.com/Microsoft/go-winio v0.4.16 // indirect
github.com/Azure/go-autorest/autorest/adal v0.9.8
github.com/Unknwon/goconfig v0.0.0-20200908083735-df7de6a44db8
github.com/a8m/tree v0.0.0-20210115125333-10a5fd5b637d
github.com/a8m/tree v0.0.0-20201026183218-fce18e2a750e
github.com/aalpar/deheap v0.0.0-20200318053559-9a0c2883bd56
github.com/abbot/go-http-auth v0.4.0
github.com/anacrolix/dms v1.2.0
github.com/anacrolix/dms v1.1.0
github.com/atotto/clipboard v0.1.2
github.com/aws/aws-sdk-go v1.37.3
github.com/aws/aws-sdk-go v1.35.17
github.com/billziss-gh/cgofuse v1.4.0
github.com/buengese/sgzip v0.1.1
github.com/calebcase/tmpfile v1.0.2 // indirect
github.com/colinmarc/hdfs/v2 v2.2.0
github.com/coreos/go-semver v0.3.0
github.com/dropbox/dropbox-sdk-go-unofficial v5.6.0+incompatible
github.com/gabriel-vasile/mimetype v1.1.2
github.com/gogo/protobuf v1.3.2 // indirect
github.com/gabriel-vasile/mimetype v1.1.1
github.com/gogo/protobuf v1.3.1 // indirect
github.com/google/go-querystring v1.0.0 // indirect
github.com/google/uuid v1.2.0 // indirect
github.com/hanwen/go-fuse/v2 v2.0.3
github.com/iguanesolutions/go-systemd/v5 v5.0.0
github.com/jcmturner/gokrb5/v8 v8.4.2
github.com/jlaffaye/ftp v0.0.0-20201112195030-9aae4d151126
github.com/jzelinskie/whirlpool v0.0.0-20201016144138-0675e54bb004
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0 // indirect
github.com/klauspost/compress v1.11.7
github.com/klauspost/compress v1.11.2
github.com/koofr/go-httpclient v0.0.0-20200420163713-93aa7c75b348
github.com/koofr/go-koofrclient v0.0.0-20190724113126-8e5366da203a
github.com/mattn/go-colorable v0.1.8
github.com/mattn/go-runewidth v0.0.10
github.com/mattn/go-runewidth v0.0.9
github.com/mitchellh/go-homedir v1.1.0
github.com/ncw/go-acd v0.0.0-20201019170801-fe55f33415b1
github.com/ncw/swift/v2 v2.0.0
github.com/nsf/termbox-go v0.0.0-20210114135735-d04385b850e8
github.com/ncw/swift v1.0.52
github.com/nsf/termbox-go v0.0.0-20200418040025-38ba6e5628f1
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pkg/errors v0.9.1
github.com/pkg/sftp v1.12.0
github.com/prometheus/client_golang v1.9.0
github.com/prometheus/procfs v0.3.0 // indirect
github.com/prometheus/client_golang v1.8.0
github.com/putdotio/go-putio/putio v0.0.0-20200123120452-16d982cac2b8
github.com/rfjakob/eme v1.1.1
github.com/rivo/uniseg v0.2.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/sevlyar/go-daemon v0.1.5
github.com/sirupsen/logrus v1.7.0
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966
github.com/spf13/cobra v1.1.1
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.7.0
github.com/stretchr/testify v1.6.1
github.com/t3rm1n4l/go-mega v0.0.0-20200416171014-ffad7fcb44b8
github.com/xanzy/ssh-agent v0.3.0
github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a
github.com/yunify/qingstor-sdk-go/v3 v3.2.0
go.etcd.io/bbolt v1.3.5
go.opencensus.io v0.22.6 // indirect
go.uber.org/zap v1.16.0 // indirect
goftp.io/server v0.4.1
golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad
golang.org/x/net v0.0.0-20210119194325-5f4716e94777
golang.org/x/oauth2 v0.0.0-20210201163806-010130855d6c
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c
golang.org/x/term v0.0.0-20201210144234-2321bbc49cbf // indirect
golang.org/x/text v0.3.5
golang.org/x/time v0.0.0-20201208040808-7e3f01d25324
google.golang.org/api v0.38.0
google.golang.org/genproto v0.0.0-20210203152818-3206188e46ba // indirect
gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
storj.io/common v0.0.0-20210203145648-3768017a858e // indirect
storj.io/uplink v1.4.5
goftp.io/server v0.4.0
golang.org/x/crypto v0.0.0-20201112155050-0c6587e931a9
golang.org/x/net v0.0.0-20201029055024-942e2f445f3c
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9
golang.org/x/sys v0.0.0-20201029080932-201ba4db2418
golang.org/x/text v0.3.4
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e
google.golang.org/api v0.34.0
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20201028140639-c77dae4b0522 // indirect
google.golang.org/grpc v1.33.1 // indirect
gopkg.in/yaml.v2 v2.3.0
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776 // indirect
storj.io/uplink v1.4.1
)

190
go.sum
View File

@@ -16,10 +16,8 @@ cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKV
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=
cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=
cloud.google.com/go v0.76.0 h1:Ckw+E/QYZgd/5bpI4wz4h6f+jmpvh9S9uSrKNnbicJI=
cloud.google.com/go v0.76.0/go.mod h1:r9EvIAvLrunusnetGdQ50M/gKui1x3zdGW/VELGkdpw=
cloud.google.com/go v0.70.0 h1:ujhG1RejZYi+HYfJNlgBh3j/bVKD8DewM7AkJ5UPyBc=
cloud.google.com/go v0.70.0/go.mod h1:/UTKYRQTWjVnSe7nGvoSzxEFUELzSI/yAYd0JQT6cRo=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
@@ -47,8 +45,8 @@ github.com/Azure/go-autorest v14.2.0+incompatible h1:V5VMDjClD3GiElqLWO7mz2MxNAK
github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=
github.com/Azure/go-autorest/autorest/adal v0.9.2 h1:Aze/GQeAN1RRbGmnUJvUj+tFGBzFdIg3293/A9rbxC4=
github.com/Azure/go-autorest/autorest/adal v0.9.2/go.mod h1:/3SMAM86bP6wC9Ev35peQDUeqFZBMH07vvUOmg4z/fE=
github.com/Azure/go-autorest/autorest/adal v0.9.10 h1:r6fZHMaHD8B6LDCn0o5vyBFHIHrM6Ywwx7mb49lPItI=
github.com/Azure/go-autorest/autorest/adal v0.9.10/go.mod h1:B7KF7jKIeC9Mct5spmyCB/A8CG/sEz1vwIRGv/bbw7A=
github.com/Azure/go-autorest/autorest/adal v0.9.8 h1:bW6ZdxqMYWsxGikpM62SSE3jnvOXVu9SXzJTuj1WM3Y=
github.com/Azure/go-autorest/autorest/adal v0.9.8/go.mod h1:B7KF7jKIeC9Mct5spmyCB/A8CG/sEz1vwIRGv/bbw7A=
github.com/Azure/go-autorest/autorest/date v0.3.0 h1:7gUk1U5M/CQbp9WoqinNzJar+8KY+LPI6wiWrP/myHw=
github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74=
github.com/Azure/go-autorest/autorest/mocks v0.4.1 h1:K0laFcLE6VLTOwNgSxaGbUcLPuGXlNkbVvq4cW4nIHk=
@@ -62,8 +60,6 @@ github.com/Julusian/godocdown v0.0.0-20170816220326-6d19f8ff2df8/go.mod h1:INZr5
github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible/go.mod h1:r7JcOSlj0wfOMncg0iLm8Leh48TZaKVeNIfJntJ2wa0=
github.com/Microsoft/go-winio v0.4.14 h1:+hMXMk01us9KgxGb7ftKQt2Xpf5hH/yky+TDA+qxleU=
github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA=
github.com/Microsoft/go-winio v0.4.16 h1:FtSW/jqD+l4ba5iPBj9CODVtgfYAD8w2wS923g/cFDk=
github.com/Microsoft/go-winio v0.4.16/go.mod h1:XB6nPKklQyQ7GC9LdcBEcBl8PF76WugXOPRXwdLnMv0=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/RoaringBitmap/roaring v0.4.7/go.mod h1:8khRDP4HmeXns4xIj9oGrKSz7XTQiJx2zgh7AcNke4w=
github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWXgklEdEo=
@@ -71,8 +67,8 @@ github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMx
github.com/Unknwon/goconfig v0.0.0-20200908083735-df7de6a44db8 h1:1TrMV1HmBApBbM+Hy7RCKZD6UlYWYIPPfoeXomG7+zE=
github.com/Unknwon/goconfig v0.0.0-20200908083735-df7de6a44db8/go.mod h1:wngxua9XCNjvHjDiTiV26DaKDT+0c63QR6H5hjVUUxw=
github.com/VividCortex/gohistogram v1.0.0/go.mod h1:Pf5mBqqDxYaXu3hDrrU+w6nw50o/4+TcAqDqk/vUH7g=
github.com/a8m/tree v0.0.0-20210115125333-10a5fd5b637d h1:4E8RufAN3UQ/weB6AnQ4y5miZCO0Yco8ZdGId41WuQs=
github.com/a8m/tree v0.0.0-20210115125333-10a5fd5b637d/go.mod h1:FSdwKX97koS5efgm8WevNf7XS3PqtyFkKDDXrz778cg=
github.com/a8m/tree v0.0.0-20201026183218-fce18e2a750e h1:8YO27VG7yrHhATTepO2FYLbGUB1wfYbkqoKiVwaAQ+Q=
github.com/a8m/tree v0.0.0-20201026183218-fce18e2a750e/go.mod h1:FSdwKX97koS5efgm8WevNf7XS3PqtyFkKDDXrz778cg=
github.com/aalpar/deheap v0.0.0-20200318053559-9a0c2883bd56 h1:hJO00l0f92EcQn8Ygc9Y0oP++eESKvcyp+KedtfT5SQ=
github.com/aalpar/deheap v0.0.0-20200318053559-9a0c2883bd56/go.mod h1:EJFoWbcEEVK22GYKONJjtMNamGYe6p+3x1Pr6zV5gFs=
github.com/abbot/go-http-auth v0.4.0 h1:QjmvZ5gSC7jm3Zg54DqWE/T5m1t2AfDu6QlXJT0EVT0=
@@ -84,8 +80,8 @@ github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuy
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
github.com/anacrolix/dms v1.2.0 h1:e5/oAtynS31rCC3yBDoun40SdDPc7uDQryOietVYfck=
github.com/anacrolix/dms v1.2.0/go.mod h1:msPKAoppoNRfrYplJqx63FZ+VipDZ4Xsj3KzIQxyU7k=
github.com/anacrolix/dms v1.1.0 h1:vbBXZS7T5FaZm+9p1pdmVVo9tN3qdc27bKSETdeT3xo=
github.com/anacrolix/dms v1.1.0/go.mod h1:msPKAoppoNRfrYplJqx63FZ+VipDZ4Xsj3KzIQxyU7k=
github.com/anacrolix/envpprof v0.0.0-20180404065416-323002cec2fa/go.mod h1:KgHhUaQMc8cC0+cEflSgCFNFbKwi5h54gqtVn8yhP7c=
github.com/anacrolix/envpprof v1.0.0/go.mod h1:KgHhUaQMc8cC0+cEflSgCFNFbKwi5h54gqtVn8yhP7c=
github.com/anacrolix/ffprobe v1.0.0/go.mod h1:BIw+Bjol6CWjm/CRWrVLk2Vy+UYlkgmBZ05vpSYqZPw=
@@ -101,8 +97,8 @@ github.com/atotto/clipboard v0.1.2 h1:YZCtFu5Ie8qX2VmVTBnrqLSiU9XOWwqNRmdT3gIQzb
github.com/atotto/clipboard v0.1.2/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI=
github.com/aws/aws-lambda-go v1.13.3/go.mod h1:4UKl9IzQMoD+QF79YdCuzCwp8VbmG4VAQwij/eHl5CU=
github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-sdk-go v1.37.3 h1:1f0groABc4AuapskpHf6EBRaG2tqw0Sx3ebCMwfp1Ys=
github.com/aws/aws-sdk-go v1.37.3/go.mod h1:hcU610XS61/+aQV88ixoOzUoG7v3b31pl2zKMmprdro=
github.com/aws/aws-sdk-go v1.35.17 h1:zhahppAMdPvJ9GP302SMOPW5SNoAbnjdOyaTmxA9WJU=
github.com/aws/aws-sdk-go v1.35.17/go.mod h1:tlPOdRjfxPBpNIwqDj61rmsnA85v9jc0Ps9+muhnW+k=
github.com/aws/aws-sdk-go-v2 v0.18.0/go.mod h1:JWVYvqSMppoMJC0x5wdwiImzgXTI9FuZwxzkQq9wy+g=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
@@ -119,8 +115,6 @@ github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f/go.mod h1:TdznJufo
github.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d/go.mod h1:+5NJ2+qvTyV9exUAL/rxXi3DcLg2Ts+ymUAY5y4NvMg=
github.com/btcsuite/btcutil v1.0.3-0.20201124182144-4031bdc69ded h1:WcPFZzCIqGt/TdFJHsOiX5dIlB/MUzrftltMhpjzfA8=
github.com/btcsuite/btcutil v1.0.3-0.20201124182144-4031bdc69ded/go.mod h1:0DVlHczLPewLcPGEIeUEzfOJhqGPQ0mJJRDBtD307+o=
github.com/btcsuite/btcutil v1.0.3-0.20201208143702-a53e38424cce h1:YtWJF7RHm2pYCvA5t0RPmAaLUhREsKuKd+SLhxFbFeQ=
github.com/btcsuite/btcutil v1.0.3-0.20201208143702-a53e38424cce/go.mod h1:0DVlHczLPewLcPGEIeUEzfOJhqGPQ0mJJRDBtD307+o=
github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd/go.mod h1:HHNXQzUsZCxOoE+CPiyCTO6x34Zs86zZUiwtpXoGdtg=
github.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd/go.mod h1:F+uVaaLLH7j4eDXPRvw78tMflu7Ie2bzYOH4Y8rRKBY=
github.com/btcsuite/snappy-go v0.0.0-20151229074030-0bdef8d06723/go.mod h1:8woku9dyThutzjeg+3xrA5iCpBRH8XEEg3lh6TiUghc=
@@ -144,8 +138,6 @@ github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMn
github.com/clbanning/x2j v0.0.0-20191024224557-825249438eec/go.mod h1:jMjuTZXRI4dUb/I5gc9Hdhagfvm9+RyrPryS/auMzxE=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8=
github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd/go.mod h1:sE/e/2PUdi/liOCUjSTXgM1o87ZssimdTWN964YiIeI=
github.com/colinmarc/hdfs/v2 v2.2.0 h1:4AaIlTq+/sWmeqYhI0dX8bD4YrMQM990tRjm636FkGM=
@@ -186,8 +178,6 @@ github.com/envoyproxy/go-control-plane v0.6.9/go.mod h1:SBwIajubJHhxtWwsL9s8ss4s
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=
github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/form3tech-oss/jwt-go v3.2.2+incompatible h1:TcekIExNqud5crz4xD2pavyTgWiPvpYe4Xau31I0PRk=
@@ -196,8 +186,8 @@ github.com/franela/goblin v0.0.0-20200105215937-c9ffbefa60db/go.mod h1:7dvUGVsVB
github.com/franela/goreq v0.0.0-20171204163338-bcd34c9993f8/go.mod h1:ZhphrRTfi2rbfLwlschooIH4+wKKDR4Pdxhh+TRoA20=
github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/gabriel-vasile/mimetype v1.1.2 h1:gaPnPcNor5aZSVCJVSGipcpbgMWiAAj9z182ocSGbHU=
github.com/gabriel-vasile/mimetype v1.1.2/go.mod h1:6CDPel/o/3/s4+bp6kIbsWATq8pmgOisOPG40CJa6To=
github.com/gabriel-vasile/mimetype v1.1.1 h1:qbN9MPuRf3bstHu9zkI9jDWNfH//9+9kHxr9oRBBBOA=
github.com/gabriel-vasile/mimetype v1.1.1/go.mod h1:6CDPel/o/3/s4+bp6kIbsWATq8pmgOisOPG40CJa6To=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/glycerine/go-unsnap-stream v0.0.0-20180323001048-9f0cb55181dd/go.mod h1:/20jfyN9Y5QPEAprSgKAUr+glWDY39ZiUEAYOEv5dsE=
github.com/glycerine/goconvey v0.0.0-20180728074245-46e3a41ad493/go.mod h1:Ogl1Tioa0aV7gstGFO7KhffUsb9M4ydbEbbxpcEDc24=
@@ -217,8 +207,8 @@ github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7a
github.com/gogo/protobuf v1.2.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1 h1:/s5zKNz0uPFCZ5hddgPdo2TK2TVrUNMn0OOX8/aZMTE=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/gogo/protobuf v1.3.1 h1:DqDEcV5aeaTmdFBePNpYsp3FlcVH/2ISVVM9Qf8PSls=
github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
@@ -265,15 +255,11 @@ github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2 h1:X2ev0eStA3AbceY54o37/0PQ/UWqKEiiO2dKL5OPaFM=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.4 h1:L8R9j+yAqZuZjsqh/z+F1NCffTKKLShY6zXTItVIZ8M=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
@@ -281,17 +267,13 @@ github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hf
github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20201009210932-67992a1a5a35/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.2.0 h1:qJYtXnJRWmpe7m/3XlyhrsLrEURqHRM2kxzoxXqyUDs=
github.com/google/uuid v1.2.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5 h1:sjZBwGj9Jlw33ImPtvFviGYvseOtDM7hkSKB7+Tv3SM=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
@@ -397,11 +379,11 @@ github.com/jzelinskie/whirlpool v0.0.0-20201016144138-0675e54bb004/go.mod h1:KmH
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0 h1:iQTw/8FWTuc7uiaSepXwyf3o52HaUYcV+Tu66S3F5GA=
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0/go.mod h1:1NbS8ALrpOvjt0rHPNLyCIeMtbizbir8U//inJ+zuB8=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23/go.mod h1:J+Gs4SYgM6CZQHDETBtE9HaSEkGmuNXF86RwHhHUvq4=
github.com/klauspost/compress v1.11.7 h1:0hzRabrMN4tSTvMfnL3SCv1ZGeAP23ynzodBgaHeMeg=
github.com/klauspost/compress v1.11.7/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.11.2 h1:MiK62aErc3gIiVEtyzKfeOHgW7atJb5g/KNX5m3c2nQ=
github.com/klauspost/compress v1.11.2/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.3 h1:CE8S1cTafDpPvMhIxNJKvHsGVBgn1xWYf1NbHQhywc8=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
@@ -435,8 +417,6 @@ github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Ky
github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0=
github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/mattn/go-runewidth v0.0.10 h1:CoZ3S2P7pvtP45xOtBw+/mDL2z0RKI576gSkzRRpdGg=
github.com/mattn/go-runewidth v0.0.10/go.mod h1:RAqKPSqVFrSLVXbA8x7dzmKdmGzieGRCM46jaSJTDAk=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
@@ -471,13 +451,13 @@ github.com/nats-io/nkeys v0.1.3/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxzi
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/ncw/go-acd v0.0.0-20201019170801-fe55f33415b1 h1:nAjWYc03awJAjsozNehdGZsm5LP7AhLOvjgbS8zN1tk=
github.com/ncw/go-acd v0.0.0-20201019170801-fe55f33415b1/go.mod h1:MLIrzg7gp/kzVBxRE1olT7CWYMCklcUWU+ekoxOD9x0=
github.com/ncw/swift/v2 v2.0.0 h1:Q1jkMe/yhCkx7yAKq4bBZ/Th3NR+ejRcwbVK8Pi1i/0=
github.com/ncw/swift/v2 v2.0.0/go.mod h1:z0A9RVdYPjNjXVo2pDOPxZ4eu3oarO1P91fTItcb+Kg=
github.com/ncw/swift v1.0.52 h1:ACF3JufDGgeKp/9mrDgQlEgS8kRYC4XKcuzj/8EJjQU=
github.com/ncw/swift v1.0.52/go.mod h1:23YIA4yWVnGwv2dQlN4bB7egfYX6YLn0Yo/S6zZO/ZM=
github.com/nfnt/resize v0.0.0-20180221191011-83c6a9932646/go.mod h1:jpp1/29i3P1S/RLdc7JQKbRpFeM1dOBd8T9ki5s+AY8=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/nsf/termbox-go v0.0.0-20210114135735-d04385b850e8 h1:3vzIuru1svOK2sXlg4XcrO3KkGRneIejmfQfR+ptSW8=
github.com/nsf/termbox-go v0.0.0-20210114135735-d04385b850e8/go.mod h1:T0cTdVuOwf7pHQNtfhnEbzHbcNyCEcVU4YPpouCbVxo=
github.com/nsf/termbox-go v0.0.0-20200418040025-38ba6e5628f1 h1:lh3PyZvY+B9nFliSGTn5uFuqQQJGuNrD0MLCokv09ag=
github.com/nsf/termbox-go v0.0.0-20200418040025-38ba6e5628f1/go.mod h1:IuKpRQcYE1Tfu+oAQqaLisqDeXgjyyltCfsaoYN18NQ=
github.com/oklog/oklog v0.3.2/go.mod h1:FCV+B7mhrz4o+ueLpx+KqkyXRGMWOYEvfiXtdGtbWGs=
github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
@@ -529,8 +509,8 @@ github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5Fsn
github.com/prometheus/client_golang v1.3.0/go.mod h1:hJaj2vgQTGQmVCsAACORcieXFeDPbaTKGT+JTgUa3og=
github.com/prometheus/client_golang v1.7.1 h1:NTGy1Ja9pByO+xAeH/qiWnLrKtr3hJPNjaVUwnjpdpA=
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
github.com/prometheus/client_golang v1.9.0 h1:Rrch9mh17XcxvEu9D9DEpb4isxjGBtcevQjKvxPRQIU=
github.com/prometheus/client_golang v1.9.0/go.mod h1:FqZLKOZnGdFAhOK4nqGHa7D66IdsO+O441Eve7ptJDU=
github.com/prometheus/client_golang v1.8.0 h1:zvJNkoCFAnYFNC24FV8nW4JdRJ3GIFcLbg65lL/JDcw=
github.com/prometheus/client_golang v1.8.0/go.mod h1:O9VU6huf47PktckDQfMTX0Y8tY0/7TSWwj+ITvv0TnM=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
@@ -545,8 +525,8 @@ github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y8
github.com/prometheus/common v0.7.0/go.mod h1:DjGbpBbp5NYNiECxcL/VnbXCCaQpKd3tt26CguLLsqA=
github.com/prometheus/common v0.10.0 h1:RyRA7RzGXQZiW+tGMr7sxa85G1z0yOpM1qq5c8lNawc=
github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo=
github.com/prometheus/common v0.15.0 h1:4fgOnadei3EZvgRwxJ7RMpG1k1pOZth5Pc13tyspaKM=
github.com/prometheus/common v0.15.0/go.mod h1:U+gB1OBLb1lF3O42bTCL+FK18tX9Oar16Clt/msog/s=
github.com/prometheus/common v0.14.0 h1:RHRyE8UocrbjU+6UvRzwi6HjiDfxrrBU91TtbKzkGp4=
github.com/prometheus/common v0.14.0/go.mod h1:U+gB1OBLb1lF3O42bTCL+FK18tX9Oar16Clt/msog/s=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
@@ -556,25 +536,17 @@ github.com/prometheus/procfs v0.1.3 h1:F0+tqvhOksq22sc6iCHF5WGlWjdwj92p0udFh1VFB
github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
github.com/prometheus/procfs v0.2.0 h1:wH4vA7pcjKuZzjF7lM8awk4fnuJO6idemZXoKnULUx4=
github.com/prometheus/procfs v0.2.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
github.com/prometheus/procfs v0.3.0 h1:Uehi/mxLK0eiUc0H0++5tpMGTexB8wZ598MIgU8VpDM=
github.com/prometheus/procfs v0.3.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
github.com/putdotio/go-putio/putio v0.0.0-20200123120452-16d982cac2b8 h1:Y258uzXU/potCYnQd1r6wlAnoMB68BiCkCcCnKx1SH8=
github.com/putdotio/go-putio/putio v0.0.0-20200123120452-16d982cac2b8/go.mod h1:bSJjRokAHHOhA+XFxplld8w2R/dXLH7Z3BZ532vhFwU=
github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
github.com/rfjakob/eme v1.1.1 h1:t+CgvcOn+eDvj2xdglxsSnkgg8LM8jwdxnV7OnsrTn0=
github.com/rfjakob/eme v1.1.1/go.mod h1:U2bmx0hDj8EyDdcxmD5t3XHDnBFnyNNc22n1R4008eM=
github.com/rivo/uniseg v0.1.0 h1:+2KBaVoUmb9XzDsrx/Ct0W/EYOSFf/nWTauy++DprtY=
github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/robertkrimen/godocdown v0.0.0-20130622164427-0bfa04905481/go.mod h1:C9WhFzY47SzYBIvzFqSvHIR6ROgDo4TtdTuRaOMjF/s=
github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/russross/blackfriday/v2 v2.0.1 h1:lPqVAte+HuHNfhJ/0LC98ESWRz8afy9tM/0RK8m9o+Q=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
github.com/ryszard/goskiplist v0.0.0-20150312221310-2dfbae5fcf46/go.mod h1:uAQ5PCi+MFsC7HjREoAz1BU+Mq60+05gifQSsHSDG/8=
github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da/go.mod h1:gi+0XIa01GRL2eRQVjQkKGqKF3SF9vZR/HnPullcV2E=
@@ -629,11 +601,8 @@ github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.3.1-0.20190311161405-34c6fa2dc709/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/t3rm1n4l/go-mega v0.0.0-20200416171014-ffad7fcb44b8 h1:IGJQmLBLYBdAknj21W3JsVof0yjEXfy1Q0K3YZebDOg=
github.com/t3rm1n4l/go-mega v0.0.0-20200416171014-ffad7fcb44b8/go.mod h1:XWL4vDyd3JKmJx+hZWUVgCNmmhZ2dTBcaNDcxH465s0=
@@ -681,8 +650,6 @@ go.opencensus.io v0.22.4 h1:LYy1Hy3MJdrCdMwwzxA/dRok4ejH+RwNGbuoD9fCjto=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5 h1:dntmOdLpSpHlVqbW5Eay97DelsZHe+55D+xC6i0dDS0=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
go.opencensus.io v0.22.6 h1:BdkrbWrzDlV9dnbzoP7sfN+dHheJ4J9JOaYxcUDL+ok=
go.opencensus.io v0.22.6/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=
go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/atomic v1.4.0 h1:cxzIVoETapQEqDhQu3QfnvXAV4AlzcvUCxkVUFw3+EU=
go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
@@ -705,8 +672,8 @@ go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
go.uber.org/zap v1.13.0/go.mod h1:zwrFLgMcdUuIBviXEYEH1YKNaOBnKXsx2IPda5bBwHM=
go.uber.org/zap v1.16.0 h1:uFRZXykJGK9lLY4HtgSw44DnIcAM+kRBP7x5m+NpAOM=
go.uber.org/zap v1.16.0/go.mod h1:MA8QOfq0BHJwdXa996Y4dYkAqRKB8/1K1QMMZVaNZjQ=
goftp.io/server v0.4.1 h1:x7KG4HIxSMdK/rpYhExMinRN/aO/T9icvaG/B5e/XfY=
goftp.io/server v0.4.1/go.mod h1:hFZeR656ErRt3ojMKt7H10vQ5nuWV1e0YeUTeorlR6k=
goftp.io/server v0.4.0 h1:hqsVdwd1/l6QtYxD9pxca9mEAJYZ7+FPCnmeXKXHQNw=
goftp.io/server v0.4.0/go.mod h1:hFZeR656ErRt3ojMKt7H10vQ5nuWV1e0YeUTeorlR6k=
golang.org/x/crypto v0.0.0-20170930174604-9419663f5a44/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
@@ -729,8 +696,6 @@ golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0 h1:hb9wdF1z5waM+dSIICn1l0
golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201112155050-0c6587e931a9 h1:umElSU9WZirRdgu2yFHY0ayQkEnKiOC1TtM3fWXFnoU=
golang.org/x/crypto v0.0.0-20201112155050-0c6587e931a9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad h1:DN0cp81fZ3njFcrLCytUHRSUkqBjfTo4Tx9RJTWs0EY=
golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -754,8 +719,6 @@ golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRu
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b h1:Wh+f8QHJXR411sJR8/vRBTZ7YapZaRvUcLFFJhusH0k=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5 h1:2M3HP5CCK1Si9FQhwnzYhXdG6DXeebvUHFpre8QvbyI=
golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
@@ -765,9 +728,6 @@ golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzB
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0 h1:RM4zey1++hCTbCVQfnWeKs9/IEsaBLA8vTkd0WVtmH4=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1 h1:Kvvh58BN8Y9/lBi7hTekvtMpm07eUZ0ck5pRHpsMWrY=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -807,12 +767,10 @@ golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81R
golang.org/x/net v0.0.0-20200707034311-ab3426394381 h1:VXak5I6aEWmAXeQjA+QSZzlgNrpq9mjcfDemuexIKsU=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210119194325-5f4716e94777 h1:003p0dJM77cxMSyCPFphvZf/Y5/NXf5fzg6ufd1/Oew=
golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20201010224723-4f7140c49acb h1:mUVeFHoDKis5nxCAzoAi7E8Ghb86EXh/RK6wtvJIqRY=
golang.org/x/net v0.0.0-20201010224723-4f7140c49acb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201029055024-942e2f445f3c h1:rpcgRPA7OvNEOdprt2Wx8/Re2cBTd8NPo/lvo3AyMqk=
golang.org/x/net v0.0.0-20201029055024-942e2f445f3c/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -821,11 +779,6 @@ golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BG
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43 h1:ld7aEMNHoBnnDAX15v1T6z31v8HwR2A9FYOuAhWqkwc=
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210113205817-d3ed898aa8a3/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20210201163806-010130855d6c h1:HiAZXo96zOhVhtFHchj/ojzoxCFiPrp9/j0GtS38V3g=
golang.org/x/oauth2 v0.0.0-20210201163806-010130855d6c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -837,8 +790,6 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208 h1:qwRHBd0NqMbJxfbotnDhm2By
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9 h1:SQFwaSi55rU7vdNs9Yr0Z324VNlrF+0wMqRXT4St8ck=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a h1:DcqTD9SDLc+1P/r1EmRBwnVsrOwW+kk2vWf9n+1sGhs=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -858,7 +809,6 @@ golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191112214154-59a1497f0cea/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -888,19 +838,10 @@ golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200828194041-157a740278f4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201214210602-f9fddec55a1e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210112091331-59c308dcf3cc/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c h1:VwygUrnw9jn88c4u8GD3rZQbqrP/tgas88tPUbBxQrk=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20201210144234-2321bbc49cbf h1:MZ2shdL+ZM/XzY3ZGOnh4Nlpnxz5GSOhOmtHo3iPU6M=
golang.org/x/term v0.0.0-20201210144234-2321bbc49cbf/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/sys v0.0.0-20201015000850-e3ed0017c211 h1:9UQO31fZ+0aKQOFldThf7BKPMJTiBfWycGh/u3UoO88=
golang.org/x/sys v0.0.0-20201015000850-e3ed0017c211/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201029080932-201ba4db2418 h1:HlFl4V6pEMziuLXyRkm5BIYq1y1GAbb02pRlWvI54OM=
golang.org/x/sys v0.0.0-20201029080932-201ba4db2418/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -909,17 +850,16 @@ golang.org/x/text v0.3.3 h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4 h1:0YWbFKbhXG/wIiuHDSKpS0Iy7FSA+u45VtBMfQcFTTc=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.5 h1:i6eZZ+zk0SOf0xgBpEpPD18qWcJda6q1sxt3S0kzyUQ=
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20201208040808-7e3f01d25324 h1:Hir2P/De0WpUhtrKGGjvSb2YxUgyZ7EFOSLIcSSpiwE=
golang.org/x/time v0.0.0-20201208040808-7e3f01d25324/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e h1:EHBhcS0mlXEAVwNyO2dLfjToGsyY4j24pTs2ScHnX7s=
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
@@ -963,18 +903,12 @@ golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roY
golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.0 h1:po9/4sTYwZU9lPhi1tOrb4hCv3qrhiQ77LZfGa2OjwY=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/tools v0.0.0-20201017001424-6003fad69a88 h1:ZB1XYzdDo7c/O48jzjMkvIjnC120Z9/CwgDWhePjQdQ=
golang.org/x/tools v0.0.0-20201017001424-6003fad69a88/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
@@ -999,10 +933,10 @@ google.golang.org/api v0.28.0 h1:jMF5hhVfMkTZwHW1SDpKq5CkgWLXOb31Foaca9Zr3oM=
google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=
google.golang.org/api v0.38.0 h1:vDyWk6eup8eQAidaZ31sNWIn8tZEL8qpbtGkBD4ytQo=
google.golang.org/api v0.38.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=
google.golang.org/api v0.33.0 h1:+gL0XvACeMIvpwLZ5rQZzLn5cwOsgg8dIcfJ2SYfBVw=
google.golang.org/api v0.33.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/api v0.34.0 h1:k40adF3uR+6x/+hO5Dh4ZFUqFp67vxvbpafFiJxl10A=
google.golang.org/api v0.34.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
@@ -1046,13 +980,10 @@ google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6D
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210202153253-cf70463f6119/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20210203152818-3206188e46ba h1:np3A9jnmE/eMtrOwwvUycmQ1XoLyj5nqZ41bAyYLqJ0=
google.golang.org/genproto v0.0.0-20210203152818-3206188e46ba/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201019141844-1ed22bb0c154 h1:bFFRpT+e8JJVY7lMMfvezL1ZIwqiwmPl2bsE2yx4HqM=
google.golang.org/genproto v0.0.0-20201019141844-1ed22bb0c154/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201028140639-c77dae4b0522 h1:7RoRaOmOAXwqnurgQ5g5/d0yCi9ha2UxuTZULXudK7A=
google.golang.org/genproto v0.0.0-20201028140639-c77dae4b0522/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.0/go.mod h1:chYK+tFQF0nDUGJgXMSgLCQk3phJEuONr2DCgLDdAQM=
@@ -1073,10 +1004,10 @@ google.golang.org/grpc v1.30.0 h1:M5a8xTlYTxwMn5ZFkwhRabsygDY5G8TYLyQDBxJNAxE=
google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
google.golang.org/grpc v1.35.0 h1:TwIQcH3es+MojMVojxxfQ3l3OF2KzlRxML2xZq0kRo8=
google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
google.golang.org/grpc v1.32.0 h1:zWTV+LMdc3kaiJMSTOFz2UgSBgx8RNQoTGiZu3fR9S0=
google.golang.org/grpc v1.32.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.33.1 h1:DGeFlSan2f+WEtCERJ4J9GJWk15TxUi8QGagfI87Xyc=
google.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
@@ -1118,12 +1049,10 @@ gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776 h1:tQIYjPdBoyREyB9XMu+nnTclpTYkz2zFM+lzLJFO4gQ=
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20180728063816-88497007e858/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
@@ -1138,10 +1067,9 @@ rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o=
sourcegraph.com/sourcegraph/appdash v0.0.0-20190731080439-ebfcffb1b5c0/go.mod h1:hI742Nqp5OhwiqlzhgfbWU4mW4yO10fP+LoT9WOswdU=
storj.io/common v0.0.0-20201207172416-78f4e59925c3/go.mod h1:6sepaQTRLuygvA+GNPzdgRPOB1+wFfjde76KBWofbMY=
storj.io/common v0.0.0-20210203145648-3768017a858e h1:lMLJoRJ8jmvHd/gtyy73wVry6D5ampTWSX1vh6y4fI4=
storj.io/common v0.0.0-20210203145648-3768017a858e/go.mod h1:KhVByBTvjV2rsaUQsft0pKgBRRMvCcY1JsDqt6BWr3I=
storj.io/common v0.0.0-20201204143755-a03c37168cb1 h1:SwSIESeyaX3kOhZN1jeNPbegSraFTdxtWD+Dn0dT7y4=
storj.io/common v0.0.0-20201204143755-a03c37168cb1/go.mod h1:6sepaQTRLuygvA+GNPzdgRPOB1+wFfjde76KBWofbMY=
storj.io/drpc v0.0.16 h1:9sxypc5lKi/0D69cR21BR0S21+IvXfON8L5nXMVNTwQ=
storj.io/drpc v0.0.16/go.mod h1:zdmQ93nx4Z35u11pQ+GAnBy4DGOK3HJCSOfeh2RryTo=
storj.io/uplink v1.4.5 h1:aeJgbob2YtnVPgzrzw16XwmYr241ibuZBhPqVwvyR3E=
storj.io/uplink v1.4.5/go.mod h1:VoxYTP5AzJ+gnzsqptuB5Ra8Old+fVVbwLCmi4jr5y4=
storj.io/uplink v1.4.1 h1:+vqMsmDArZuFC3TGW+fwJ1FaukjjSmj7dPcOhfYuHBY=
storj.io/uplink v1.4.1/go.mod h1:cN4UhPehwrdrbrtJPyvMp0sW5JbjaeDEFajQP3RX1ug=

39
lib/cache/cache.go vendored
View File

@@ -22,17 +22,15 @@ func New() *Cache {
return &Cache{
cache: map[string]*cacheEntry{},
expireRunning: false,
expireDuration: 24 * time.Hour,
expireDuration: 300 * time.Second,
expireInterval: 60 * time.Second,
}
}
// cacheEntry is stored in the cache
type cacheEntry struct {
createMu sync.Mutex // held while creating the item
value interface{} // cached item
err error // creation error
ok bool // true if entry is valid
key string // key
lastUsed time.Time // time used for expiry
pinCount int // non zero if the entry should not be removed
@@ -57,27 +55,23 @@ func (c *Cache) used(entry *cacheEntry) {
// afresh with the create function.
func (c *Cache) Get(key string, create CreateFunc) (value interface{}, err error) {
c.mu.Lock()
entry, found := c.cache[key]
if !found {
entry = &cacheEntry{
key: key,
entry, ok := c.cache[key]
if !ok {
c.mu.Unlock() // Unlock in case Get is called recursively
value, ok, err = create(key)
if err != nil && !ok {
return value, err
}
entry = &cacheEntry{
value: value,
key: key,
err: err,
}
c.mu.Lock()
c.cache[key] = entry
}
c.mu.Unlock()
// Only one racing Get will have found=false here
entry.createMu.Lock()
if !found {
entry.value, entry.ok, entry.err = create(key)
}
entry.createMu.Unlock()
c.mu.Lock()
if !found && !entry.ok {
delete(c.cache, key)
} else {
c.used(entry)
}
c.mu.Unlock()
defer c.mu.Unlock()
c.used(entry)
return entry.value, entry.err
}
@@ -108,7 +102,6 @@ func (c *Cache) Put(key string, value interface{}) {
entry := &cacheEntry{
value: value,
key: key,
ok: true,
}
c.used(entry)
c.cache[key] = entry
@@ -119,7 +112,7 @@ func (c *Cache) GetMaybe(key string) (value interface{}, found bool) {
c.mu.Lock()
defer c.mu.Unlock()
entry, found := c.cache[key]
if !found || !entry.ok {
if !found {
return nil, found
}
c.used(entry)

View File

@@ -3,8 +3,6 @@ package cache
import (
"errors"
"fmt"
"sync"
"sync/atomic"
"testing"
"time"
@@ -13,7 +11,7 @@ import (
)
var (
called = int32(0)
called = 0
errSentinel = errors.New("an error")
errCached = errors.New("a cached error")
)
@@ -21,19 +19,17 @@ var (
func setup(t *testing.T) (*Cache, CreateFunc) {
called = 0
create := func(path string) (interface{}, bool, error) {
newCalled := atomic.AddInt32(&called, 1)
assert.Equal(t, int32(1), newCalled)
assert.Equal(t, 0, called)
called++
switch path {
case "/":
time.Sleep(100 * time.Millisecond)
return "/", true, nil
case "/file.txt":
return "/file.txt", true, errCached
case "/error":
return nil, false, errSentinel
}
assert.Fail(t, fmt.Sprintf("Unknown path %q", path))
return nil, false, nil
panic(fmt.Sprintf("Unknown path %q", path))
}
c := New()
return c, create
@@ -55,24 +51,6 @@ func TestGet(t *testing.T) {
assert.Equal(t, f, f2)
}
func TestGetConcurrent(t *testing.T) {
c, create := setup(t)
assert.Equal(t, 0, len(c.cache))
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func() {
defer wg.Done()
_, err := c.Get("/", create)
require.NoError(t, err)
}()
}
wg.Wait()
assert.Equal(t, 1, len(c.cache))
}
func TestGetFile(t *testing.T) {
c, create := setup(t)

View File

@@ -639,6 +639,7 @@ func (s *authServer) Init() error {
http.Error(w, "State did not match - please try again", http.StatusForbidden)
return
}
fs.Debugf(nil, "Redirecting browser to: %s", s.authURL)
http.Redirect(w, req, s.authURL, http.StatusTemporaryRedirect)
return
})

View File

@@ -213,10 +213,11 @@ func (api *Client) Call(ctx context.Context, opts *Opts) (resp *http.Response, e
if opts.ContentLength != nil && *opts.ContentLength == 0 {
body = nil
}
req, err := http.NewRequestWithContext(ctx, opts.Method, url, body)
req, err := http.NewRequest(opts.Method, url, body)
if err != nil {
return
}
req = req.WithContext(ctx) // go1.13 can use NewRequestWithContext
headers := make(map[string]string)
// Set default headers
for k, v := range api.headers {

519
rclone.1 generated
View File

@@ -1,7 +1,7 @@
.\"t
.\" Automatically generated by Pandoc 2.5
.\"
.TH "rclone" "1" "Feb 02, 2021" "User Manual" ""
.TH "rclone" "1" "Mar 08, 2021" "User Manual" ""
.hy
.SH Rclone syncs your files to cloud storage
.PP
@@ -3663,7 +3663,7 @@ files sequentially, it can only seek when reading.
This means that many applications won\[aq]t work with their files on an
rclone mount without \f[C]\-\-vfs\-cache\-mode writes\f[R] or
\f[C]\-\-vfs\-cache\-mode full\f[R].
See the File Caching section for more info.
See the VFS File Caching section for more info.
.PP
The bucket based remotes (e.g.
Swift, S3, Google Compute Storage, B2, Hubic) do not support the concept
@@ -3678,7 +3678,7 @@ systems are a long way from 100% reliable.
The rclone sync/copy commands cope with this with lots of retries.
However rclone mount can\[aq]t use retries in the same way without
making local copies of the uploads.
Look at the file caching for solutions to make mount more reliable.
Look at the VFS File Caching for solutions to make mount more reliable.
.SS Attribute caching
.PP
You can use the flag \f[C]\-\-attr\-timeout\f[R] to set the time the
@@ -3875,6 +3875,15 @@ exceed this size for two reasons.
Firstly because it is only checked every
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
Secondly because open files cannot be evicted from the cache.
.PP
You \f[B]should not\f[R] run two copies of rclone using the same VFS
cache with the same or overlapping remotes if using
\f[C]\-\-vfs\-cache\-mode > off\f[R].
This can potentially cause data corruption if you do.
You can work around this by giving each rclone its own cache hierarchy
with \f[C]\-\-cache\-dir\f[R].
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
overlap.
.SS \-\-vfs\-cache\-mode off
.PP
In this mode (the default) the cache will read directly from the remote
@@ -4730,6 +4739,15 @@ exceed this size for two reasons.
Firstly because it is only checked every
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
Secondly because open files cannot be evicted from the cache.
.PP
You \f[B]should not\f[R] run two copies of rclone using the same VFS
cache with the same or overlapping remotes if using
\f[C]\-\-vfs\-cache\-mode > off\f[R].
This can potentially cause data corruption if you do.
You can work around this by giving each rclone its own cache hierarchy
with \f[C]\-\-cache\-dir\f[R].
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
overlap.
.SS \-\-vfs\-cache\-mode off
.PP
In this mode (the default) the cache will read directly from the remote
@@ -5102,6 +5120,15 @@ exceed this size for two reasons.
Firstly because it is only checked every
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
Secondly because open files cannot be evicted from the cache.
.PP
You \f[B]should not\f[R] run two copies of rclone using the same VFS
cache with the same or overlapping remotes if using
\f[C]\-\-vfs\-cache\-mode > off\f[R].
This can potentially cause data corruption if you do.
You can work around this by giving each rclone its own cache hierarchy
with \f[C]\-\-cache\-dir\f[R].
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
overlap.
.SS \-\-vfs\-cache\-mode off
.PP
In this mode (the default) the cache will read directly from the remote
@@ -5726,6 +5753,15 @@ exceed this size for two reasons.
Firstly because it is only checked every
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
Secondly because open files cannot be evicted from the cache.
.PP
You \f[B]should not\f[R] run two copies of rclone using the same VFS
cache with the same or overlapping remotes if using
\f[C]\-\-vfs\-cache\-mode > off\f[R].
This can potentially cause data corruption if you do.
You can work around this by giving each rclone its own cache hierarchy
with \f[C]\-\-cache\-dir\f[R].
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
overlap.
.SS \-\-vfs\-cache\-mode off
.PP
In this mode (the default) the cache will read directly from the remote
@@ -6418,6 +6454,15 @@ exceed this size for two reasons.
Firstly because it is only checked every
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
Secondly because open files cannot be evicted from the cache.
.PP
You \f[B]should not\f[R] run two copies of rclone using the same VFS
cache with the same or overlapping remotes if using
\f[C]\-\-vfs\-cache\-mode > off\f[R].
This can potentially cause data corruption if you do.
You can work around this by giving each rclone its own cache hierarchy
with \f[C]\-\-cache\-dir\f[R].
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
overlap.
.SS \-\-vfs\-cache\-mode off
.PP
In this mode (the default) the cache will read directly from the remote
@@ -7044,6 +7089,15 @@ exceed this size for two reasons.
Firstly because it is only checked every
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
Secondly because open files cannot be evicted from the cache.
.PP
You \f[B]should not\f[R] run two copies of rclone using the same VFS
cache with the same or overlapping remotes if using
\f[C]\-\-vfs\-cache\-mode > off\f[R].
This can potentially cause data corruption if you do.
You can work around this by giving each rclone its own cache hierarchy
with \f[C]\-\-cache\-dir\f[R].
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
overlap.
.SS \-\-vfs\-cache\-mode off
.PP
In this mode (the default) the cache will read directly from the remote
@@ -10623,7 +10677,7 @@ command.
.SS Common pitfalls
.PP
The most frequent filter support issues on the rclone
forum (https://https://forum.rclone.org/) are:
forum (https://forum.rclone.org/) are:
.IP \[bu] 2
Not using paths relative to the root of the remote
.IP \[bu] 2
@@ -12274,6 +12328,8 @@ srcFs \- a remote name string e.g.
.IP \[bu] 2
dstFs \- a remote name string e.g.
\[dq]drive:dst\[dq] for the destination
.IP \[bu] 2
createEmptySrcDirs \- create empty src directories on destination if set
.PP
See the copy command (https://rclone.org/commands/rclone_copy/) command
for more information on the above.
@@ -12289,6 +12345,8 @@ srcFs \- a remote name string e.g.
dstFs \- a remote name string e.g.
\[dq]drive:dst\[dq] for the destination
.IP \[bu] 2
createEmptySrcDirs \- create empty src directories on destination if set
.IP \[bu] 2
deleteEmptySrcDirs \- delete empty src directories if set
.PP
See the move command (https://rclone.org/commands/rclone_move/) command
@@ -12304,6 +12362,8 @@ srcFs \- a remote name string e.g.
.IP \[bu] 2
dstFs \- a remote name string e.g.
\[dq]drive:dst\[dq] for the destination
.IP \[bu] 2
createEmptySrcDirs \- create empty src directories on destination if set
.PP
See the sync command (https://rclone.org/commands/rclone_sync/) command
for more information on the above.
@@ -14886,7 +14946,7 @@ These flags are available for every command.
\-\-use\-json\-log Use json log format.
\-\-use\-mmap Use mmap allocator (see docs).
\-\-use\-server\-modtime Use server modified time instead of object metadata
\-\-user\-agent string Set the user\-agent to a specified string. The default is rclone/ version (default \[dq]rclone/v1.54.0\[dq])
\-\-user\-agent string Set the user\-agent to a specified string. The default is rclone/ version (default \[dq]rclone/v1.54.1\[dq])
\-v, \-\-verbose count Print lots more stuff (repeat for more)
\f[R]
.fi
@@ -15018,7 +15078,7 @@ They control the backends and may be set in the config file.
\-\-drive\-starred\-only Only show files that are starred.
\-\-drive\-stop\-on\-download\-limit Make download limit errors be fatal
\-\-drive\-stop\-on\-upload\-limit Make upload limit errors be fatal
\-\-drive\-team\-drive string ID of the Team Drive
\-\-drive\-team\-drive string ID of the Shared Drive (Team Drive)
\-\-drive\-token string OAuth Access Token as a JSON blob.
\-\-drive\-token\-url string Token server url.
\-\-drive\-trashed\-only Only show files that are in the trash.
@@ -15299,8 +15359,13 @@ They control the backends and may be set in the config file.
\-\-yandex\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,InvalidUtf8,Dot)
\-\-yandex\-token string OAuth Access Token as a JSON blob.
\-\-yandex\-token\-url string Token server url.
\-\-zoho\-auth\-url string Auth server URL.
\-\-zoho\-client\-id string OAuth Client Id
\-\-zoho\-client\-secret string OAuth Client Secret
\-\-zoho\-encoding MultiEncoder This sets the encoding for the backend. (default Del,Ctl,InvalidUtf8)
\-\-zoho\-region string Zoho region to connect to. You\[aq]ll have to use the region you organization is registered in.
\-\-zoho\-token string OAuth Access Token as a JSON blob.
\-\-zoho\-token\-url string Token server url.
\f[R]
.fi
.SS 1Fichier
@@ -16452,6 +16517,15 @@ You\[aq]d then do a full \f[C]rclone sync\f[R] less often.
.PP
Note that \f[C]\-\-fast\-list\f[R] isn\[aq]t required in the top\-up
sync.
.SS Avoiding HEAD requests after PUT
.PP
By default rclone will HEAD every object it uploads.
It does this to check the object got uploaded correctly.
.PP
You can disable this with the \-\-s3\-no\-head option \- see there for
more details.
.PP
Setting this flag increases the chance for undetected upload failures.
.SS Hashes
.PP
For small objects which weren\[aq]t uploaded as multipart uploads
@@ -21620,7 +21694,7 @@ securely.
.PP
The format for these URLs is the following:
.PP
https://ip\-with\-dots\-replaced.server\-hash.plex.direct:32400/
\f[C]https://ip\-with\-dots\-replaced.server\-hash.plex.direct:32400/\f[R]
.PP
The \f[C]ip\-with\-dots\-replaced\f[R] part can be any IPv4 address,
where the dots have been replaced with dashes, e.g.
@@ -23160,7 +23234,7 @@ just as you would with any other remote, e.g.
encrypt and decrypt as needed on the fly.
If you access the wrapped remote \f[C]remote:path\f[R] directly you will
bypass the encryption, and anything you read will be in encrypted form,
and anything you write will be undencrypted.
and anything you write will be unencrypted.
To avoid issues it is best to configure a dedicated path for encrypted
content, and access it exclusively through a crypt remote.
.IP
@@ -24321,6 +24395,20 @@ Default: 48M
.SS \-\-dropbox\-impersonate
.PP
Impersonate this user when using a business account.
.PP
Note that if you want to use impersonate, you should make sure this flag
is set when running \[dq]rclone config\[dq] as this will cause rclone to
request the \[dq]members.read\[dq] scope which it won\[aq]t normally.
This is needed to lookup a members email address into the internal ID
that dropbox uses in the API.
.PP
Using the \[dq]members.read\[dq] scope will require a Dropbox Team Admin
to approve during the OAuth flow.
.PP
You will have to use your own App (setting your own client_id and
client_secret) to use this option as currently rclone\[aq]s default set
of permissions doesn\[aq]t include \[dq]members.read\[dq].
This can be added once v1.55 or later is in use everywhere.
.IP \[bu] 2
Config: impersonate
.IP \[bu] 2
@@ -24866,9 +24954,9 @@ rclone lsf :ftp: \-\-ftp\-host=speedtest.tele2.net \-\-ftp\-user=anonymous \-\-f
.PP
Rlone FTP supports implicit FTP over TLS servers (FTPS).
This has to be enabled in the FTP backend config for the remote, or with
\f[C][\-\-ftp\-tls]{#ftp\-tls}\f[R].
\f[C]\-\-ftp\-tls\f[R].
The default FTPS port is \f[C]990\f[R], not \f[C]21\f[R] and can be set
with \f[C][\-\-ftp\-port]{#ftp\-port}\f[R].
with \f[C]\-\-ftp\-port\f[R].
.SS Standard Options
.PP
Here are the standard options specific to ftp (FTP Connection).
@@ -25961,7 +26049,7 @@ If your browser doesn\[aq]t open automatically go to the following link: http://
Log in and authorize rclone for access
Waiting for code...
Got code
Configure this as a team drive?
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No
y/n> n
@@ -26200,25 +26288,25 @@ interface, share your root folder with the user/email of the new Service
Account you created/selected at step #1 \- use rclone without specifying
the \f[C]\-\-drive\-impersonate\f[R] option, like this:
\f[C]rclone \-v foo\[at]example.com lsf gdrive:backup\f[R]
.SS Team drives
.SS Shared drives (team drives)
.PP
If you want to configure the remote to point to a Google Team Drive then
answer \f[C]y\f[R] to the question
\f[C]Configure this as a team drive?\f[R].
If you want to configure the remote to point to a Google Shared Drive
(previously known as Team Drives) then answer \f[C]y\f[R] to the
question \f[C]Configure this as a Shared Drive (Team Drive)?\f[R].
.PP
This will fetch the list of Team Drives from google and allow you to
This will fetch the list of Shared Drives from google and allow you to
configure which one you want to use.
You can also type in a team drive ID if you prefer.
You can also type in a Shared Drive ID if you prefer.
.PP
For example:
.IP
.nf
\f[C]
Configure this as a team drive?
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No
y/n> y
Fetching team drive list...
Fetching Shared Drive list...
Choose a number from below, or type in your own value
1 / Rclone Test
\[rs] \[dq]xxxxxxxxxxxxxxxxxxxx\[dq]
@@ -26226,7 +26314,7 @@ Choose a number from below, or type in your own value
\[rs] \[dq]yyyyyyyyyyyyyyyyyyyy\[dq]
3 / Rclone Test 3
\[rs] \[dq]zzzzzzzzzzzzzzzzzzzz\[dq]
Enter a Team Drive ID> 1
Enter a Shared Drive ID> 1
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
[remote]
client_id =
@@ -26927,7 +27015,7 @@ Type: string
Default: \[dq]\[dq]
.SS \-\-drive\-team\-drive
.PP
ID of the Team Drive
ID of the Shared Drive (Team Drive)
.IP \[bu] 2
Config: team_drive
.IP \[bu] 2
@@ -27487,7 +27575,7 @@ Options:
\[dq]target\[dq]: optional target remote for the shortcut destination
.SS drives
.PP
List the shared drives available to this account
List the Shared Drives available to this account
.IP
.nf
\f[C]
@@ -27495,7 +27583,7 @@ rclone backend drives remote: [options] [<arguments>+]
\f[R]
.fi
.PP
This command lists the shared drives (teamdrives) available to this
This command lists the Shared Drives (Team Drives) available to this
account.
.PP
Usage:
@@ -35000,8 +35088,8 @@ default, due to security concerns.
This can be re\-enabled on a per\-connection basis by setting the
\f[C]use_insecure_cipher\f[R] setting in the configuration file to
\f[C]true\f[R].
Further details on the insecurity of this cipher can be found [in this
paper] (http://www.isg.rhul.ac.uk/\[ti]kp/SandPfinal.pdf).
Further details on the insecurity of this cipher can be found in this
paper (http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).
.PP
SFTP isn\[aq]t supported under plan9 until this
issue (https://github.com/pkg/sftp/issues/156) is fixed.
@@ -36941,6 +37029,28 @@ and will be removed from filenames during upload.
.SS Standard Options
.PP
Here are the standard options specific to zoho (Zoho).
.SS \-\-zoho\-client\-id
.PP
OAuth Client Id Leave blank normally.
.IP \[bu] 2
Config: client_id
.IP \[bu] 2
Env Var: RCLONE_ZOHO_CLIENT_ID
.IP \[bu] 2
Type: string
.IP \[bu] 2
Default: \[dq]\[dq]
.SS \-\-zoho\-client\-secret
.PP
OAuth Client Secret Leave blank normally.
.IP \[bu] 2
Config: client_secret
.IP \[bu] 2
Env Var: RCLONE_ZOHO_CLIENT_SECRET
.IP \[bu] 2
Type: string
.IP \[bu] 2
Default: \[dq]\[dq]
.SS \-\-zoho\-region
.PP
Zoho region to connect to.
@@ -36984,6 +37094,41 @@ Australia
.SS Advanced Options
.PP
Here are the advanced options specific to zoho (Zoho).
.SS \-\-zoho\-token
.PP
OAuth Access Token as a JSON blob.
.IP \[bu] 2
Config: token
.IP \[bu] 2
Env Var: RCLONE_ZOHO_TOKEN
.IP \[bu] 2
Type: string
.IP \[bu] 2
Default: \[dq]\[dq]
.SS \-\-zoho\-auth\-url
.PP
Auth server URL.
Leave blank to use the provider defaults.
.IP \[bu] 2
Config: auth_url
.IP \[bu] 2
Env Var: RCLONE_ZOHO_AUTH_URL
.IP \[bu] 2
Type: string
.IP \[bu] 2
Default: \[dq]\[dq]
.SS \-\-zoho\-token\-url
.PP
Token server url.
Leave blank to use the provider defaults.
.IP \[bu] 2
Config: token_url
.IP \[bu] 2
Env Var: RCLONE_ZOHO_TOKEN_URL
.IP \[bu] 2
Type: string
.IP \[bu] 2
Default: \[dq]\[dq]
.SS \-\-zoho\-encoding
.PP
This sets the encoding for the backend.
@@ -37910,6 +38055,102 @@ Options:
.IP \[bu] 2
\[dq]error\[dq]: return an error based on option value
.SH Changelog
.SS v1.54.1 \- 2021\-03\-08
.PP
See commits (https://github.com/rclone/rclone/compare/v1.54.0...v1.54.1)
.IP \[bu] 2
Bug Fixes
.RS 2
.IP \[bu] 2
accounting: Fix \-\-bwlimit when up or down is off (Nick Craig\-Wood)
.IP \[bu] 2
docs
.RS 2
.IP \[bu] 2
Fix nesting of brackets and backticks in ftp docs (edwardxml)
.IP \[bu] 2
Fix broken link in sftp page (edwardxml)
.IP \[bu] 2
Fix typo in crypt.md (Romeo Kienzler)
.IP \[bu] 2
Changelog: Correct link to digitalis.io (Alex JOST)
.IP \[bu] 2
Replace #file\-caching with #vfs\-file\-caching (Miron Veryanskiy)
.IP \[bu] 2
Convert bogus example link to code (edwardxml)
.IP \[bu] 2
Remove dead link from rc.md (edwardxml)
.RE
.IP \[bu] 2
rc: Sync,copy,move: document createEmptySrcDirs parameter (Nick
Craig\-Wood)
.IP \[bu] 2
lsjson: Fix unterminated JSON in the presence of errors (Nick
Craig\-Wood)
.RE
.IP \[bu] 2
Mount
.RS 2
.IP \[bu] 2
Fix mount dropping on macOS by setting \-\-daemon\-timeout 10m (Nick
Craig\-Wood)
.RE
.IP \[bu] 2
VFS
.RS 2
.IP \[bu] 2
Document simultaneous usage with the same cache shouldn\[aq]t be used
(Nick Craig\-Wood)
.RE
.IP \[bu] 2
B2
.RS 2
.IP \[bu] 2
Automatically raise upload cutoff to avoid spurious error (Nick
Craig\-Wood)
.IP \[bu] 2
Fix failed to create file system with application key limited to a
prefix (Nick Craig\-Wood)
.RE
.IP \[bu] 2
Drive
.RS 2
.IP \[bu] 2
Refer to Shared Drives instead of Team Drives (Nick Craig\-Wood)
.RE
.IP \[bu] 2
Dropbox
.RS 2
.IP \[bu] 2
Add scopes to oauth request and optionally \[dq]members.read\[dq] (Nick
Craig\-Wood)
.RE
.IP \[bu] 2
S3
.RS 2
.IP \[bu] 2
Fix failed to create file system with folder level permissions policy
(Nick Craig\-Wood)
.IP \[bu] 2
Fix Wasabi HEAD requests returning stale data by using only 1 transport
(Nick Craig\-Wood)
.IP \[bu] 2
Fix shared_credentials_file auth (Dmitry Chepurovskiy)
.IP \[bu] 2
Add \-\-s3\-no\-head to reducing costs docs (Nick Craig\-Wood)
.RE
.IP \[bu] 2
Union
.RS 2
.IP \[bu] 2
Fix mkdir at root with remote:/ (Nick Craig\-Wood)
.RE
.IP \[bu] 2
Zoho
.RS 2
.IP \[bu] 2
Fix custom client id\[aq]s (buengese)
.RE
.SS v1.54.0 \- 2021\-02\-02
.PP
See commits (https://github.com/rclone/rclone/compare/v1.53.0...v1.54.0)
@@ -37917,7 +38158,7 @@ See commits (https://github.com/rclone/rclone/compare/v1.53.0...v1.54.0)
New backends
.RS 2
.IP \[bu] 2
Compression remote (experimental)(buengese)
Compression remote (experimental) (buengese)
.IP \[bu] 2
Enterprise File Fabric (Nick Craig\-Wood)
.RS 2
@@ -37939,9 +38180,9 @@ Deglobalise the config (Nick Craig\-Wood)
.IP \[bu] 2
Global config now read from the context
.IP \[bu] 2
Global config can be passed into the rc
This will enable passing of global config via the rc
.IP \[bu] 2
This work was sponsored by Digitalis (digitalis.io)
This work was sponsored by Digitalis (https://digitalis.io/)
.RE
.IP \[bu] 2
Add \f[C]\-\-bwlimit\f[R] for upload and download (Nick Craig\-Wood)
@@ -37953,18 +38194,21 @@ Obey bwlimit in http Transport for better limiting
Enhance systemd integration (Hekmon)
.RS 2
.IP \[bu] 2
log level identification
.IP \[bu] 2
manual activation with flag
.IP \[bu] 2
automatic systemd launch detection
log level identification, manual activation with flag, automatic systemd
launch detection
.IP \[bu] 2
Don\[aq]t compile systemd log integration for non unix systems (Benjamin
Gustin)
.RE
.IP \[bu] 2
Add a download flag to hashsum and related commands to force rclone to
download and hash files locally (lostheli)
Add a \f[C]\-\-download\f[R] flag to md5sum/sha1sum/hashsum to force
rclone to download and hash files locally (lostheli)
.IP \[bu] 2
Add \f[C]\-\-progress\-terminal\-title\f[R] to print ETA to terminal
title (LaSombra)
.IP \[bu] 2
Make backend env vars show in help as the defaults for backend flags
(Nick Craig\-Wood)
.IP \[bu] 2
build
.RS 2
@@ -37972,51 +38216,33 @@ build
Raise minimum go version to go1.12 (Nick Craig\-Wood)
.RE
.IP \[bu] 2
check
.RS 2
.IP \[bu] 2
Make the error count match up in the log message (Nick Craig\-Wood)
.RE
.IP \[bu] 2
cmd
.RS 2
.IP \[bu] 2
Add \-\-progress\-terminal\-title to print ETA to terminal title
(LaSombra)
.IP \[bu] 2
Make backend env vars show in help as the defaults for backend flags
(Nick Craig\-Wood)
.RE
.IP \[bu] 2
dedupe
.RS 2
.IP \[bu] 2
Add \-\-by\-hash to dedupe on hash not file name (Nick Craig\-Wood)
.IP \[bu] 2
Add \-\-dedupe\-mode list to just list dupes, changing nothing (Nick
Add \f[C]\-\-by\-hash\f[R] to dedupe on content hash not file name (Nick
Craig\-Wood)
.IP \[bu] 2
Add \f[C]\-\-dedupe\-mode list\f[R] to just list dupes, changing nothing
(Nick Craig\-Wood)
.IP \[bu] 2
Add warning if used on a remote which can\[aq]t have duplicate names
(Nick Craig\-Wood)
.RE
.IP \[bu] 2
flags: Improve error message when reading environment vars (Nick
Craig\-Wood)
.IP \[bu] 2
fs
.RS 2
.IP \[bu] 2
Add Shutdown optional method for backends (Nick Craig\-Wood)
.IP \[bu] 2
When using \-\-files\-from check files concurrently (zhucan)
When using \f[C]\-\-files\-from\f[R] check files concurrently (zhucan)
.IP \[bu] 2
Accumulate stats when using \-\-dry\-run (Ingo Weiss)
Accumulate stats when using \f[C]\-\-dry\-run\f[R] (Ingo Weiss)
.IP \[bu] 2
Always show stats when using \-\-dry\-run or \-\-interactive (Nick
Craig\-Wood)
Always show stats when using \f[C]\-\-dry\-run\f[R] or
\f[C]\-\-interactive\f[R] (Nick Craig\-Wood)
.IP \[bu] 2
Add support for flag \-\-no\-console on windows to hide the console
window (albertony)
Add support for flag \f[C]\-\-no\-console\f[R] on windows to hide the
console window (albertony)
.RE
.IP \[bu] 2
genautocomplete: Add support to output to stdout (Ingo)
@@ -38028,13 +38254,13 @@ Highlight read errors instead of aborting (Claudio Bantaloukas)
.IP \[bu] 2
Add sort by average size in directory (Adam Pl\['a]nsk\['y])
.IP \[bu] 2
Add toggle option for average size in directory \- key \[aq]a\[aq] (Adam
Pl\['a]nsk\['y])
Add toggle option for average s3ize in directory \- key \[aq]a\[aq]
(Adam Pl\['a]nsk\['y])
.IP \[bu] 2
Add empty folder flag into ncdu browser (Adam Pl\['a]nsk\['y])
.IP \[bu] 2
Add ! (errror) and .
(unreadable) file flags to go with e (empty) (Nick Craig\-Wood)
Add \f[C]!\f[R] (errror) and \f[C].\f[R] (unreadable) file flags to go
with \f[C]e\f[R] (empty) (Nick Craig\-Wood)
.RE
.IP \[bu] 2
obscure: Make \f[C]rclone osbcure \-\f[R] ignore newline at end of line
@@ -38085,21 +38311,6 @@ Fix plugins initialization (negative0)
Bug Fixes
.RS 2
.IP \[bu] 2
build
.RS 2
.IP \[bu] 2
Explicitly set ARM version to fix build (Nick Craig\-Wood)
.IP \[bu] 2
Don\[aq]t explicitly set ARM version to fix ARMv5 build (Nick
Craig\-Wood)
.IP \[bu] 2
Fix nfpm install (Nick Craig\-Wood)
.IP \[bu] 2
Fix docker build by upgrading ilteoood/docker_buildx (Nick Craig\-Wood)
.IP \[bu] 2
Temporary fix for Windows build errors (Ivan Andreev)
.RE
.IP \[bu] 2
fs
.RS 2
.IP \[bu] 2
@@ -38110,26 +38321,35 @@ Fix parsing of ..
when joining remotes (Nick Craig\-Wood)
.RE
.IP \[bu] 2
log: Fix enabling systemd logging when using \-\-log\-file (Nick
Craig\-Wood)
log: Fix enabling systemd logging when using \f[C]\-\-log\-file\f[R]
(Nick Craig\-Wood)
.IP \[bu] 2
move: Fix data loss when moving the same object (Nick Craig\-Wood)
check
.RS 2
.IP \[bu] 2
Make the error count match up in the log message (Nick Craig\-Wood)
.RE
.IP \[bu] 2
move: Fix data loss when source and destination are the same object
(Nick Craig\-Wood)
.IP \[bu] 2
operations
.RS 2
.IP \[bu] 2
Fix \-\-cutof\-mode hard not cutting off immediately (Nick Craig\-Wood)
Fix \f[C]\-\-cutof\-mode\f[R] hard not cutting off immediately (Nick
Craig\-Wood)
.IP \[bu] 2
Fix \-\-immutable error message (Nick Craig\-Wood)
Fix \f[C]\-\-immutable\f[R] error message (Nick Craig\-Wood)
.RE
.IP \[bu] 2
sync
.RS 2
.IP \[bu] 2
Fix \-\-cutoff\-mode soft & cautious so it doesn\[aq]t end the transfer
early (Nick Craig\-Wood)
Fix \f[C]\-\-cutoff\-mode\f[R] soft & cautious so it doesn\[aq]t end the
transfer early (Nick Craig\-Wood)
.IP \[bu] 2
Fix \-\-immutable errors retrying many times (Nick Craig\-Wood)
Fix \f[C]\-\-immutable\f[R] errors retrying many times (Nick
Craig\-Wood)
.RE
.RE
.IP \[bu] 2
@@ -38138,7 +38358,7 @@ Docs
.IP \[bu] 2
Many fixes and a rewrite of the filtering docs (edwardxml)
.IP \[bu] 2
Many spelling and grammar problems (Josh Soref)
Many spelling and grammar fixes (Josh Soref)
.IP \[bu] 2
Doc fixes for commands delete, purge, rmdir, rmdirs and mount
(albertony)
@@ -38164,20 +38384,16 @@ Update systemd status with cache stats (Hekmon)
Disable bazil/fuse based mount on macOS (Nick Craig\-Wood)
.RS 2
.IP \[bu] 2
Make mount be cmount under macOS (Nick Craig\-Wood)
Make \f[C]rclone mount\f[R] actually run \f[C]rclone cmount\f[R] under
macOS (Nick Craig\-Wood)
.RE
.IP \[bu] 2
Implement mknod to make NFS file creation work (Nick Craig\-Wood)
.IP \[bu] 2
Make sure we don\[aq]t call umount more than once (Nick Craig\-Wood)
.IP \[bu] 2
Don\[aq]t call host.Umount if a signal has been received (Nick
Craig\-Wood)
.IP \[bu] 2
More user friendly mounting as network drive on windows (albertony)
.IP \[bu] 2
Cleanup OS specific option handling and documentation (albertony)
.IP \[bu] 2
Detect if uid or gid are set in same option string: \-o uid=123,gid=456
(albertony)
.IP \[bu] 2
@@ -38194,8 +38410,8 @@ Craig\-Wood)
Fix \[dq]file already exists\[dq] error for stale cache files (Nick
Craig\-Wood)
.IP \[bu] 2
Fix file leaks with \-\-vfs\-cache\-mode full and \-\-buffer\-size 0
(Nick Craig\-Wood)
Fix file leaks with \f[C]\-\-vfs\-cache\-mode\f[R] full and
\f[C]\-\-buffer\-size 0\f[R] (Nick Craig\-Wood)
.IP \[bu] 2
Fix invalid cache path on windows when using :backend: as remote
(albertony)
@@ -38207,8 +38423,8 @@ Local
Continue listing files/folders when a circular symlink is detected
(Manish Gupta)
.IP \[bu] 2
New flag \-\-local\-zero\-size\-links to fix sync on some virtual
filesystems (Riccardo Iaconelli)
New flag \f[C]\-\-local\-zero\-size\-links\f[R] to fix sync on some
virtual filesystems (Riccardo Iaconelli)
.RE
.IP \[bu] 2
Azure Blob
@@ -38216,26 +38432,24 @@ Azure Blob
.IP \[bu] 2
Add support for service principals (James Lim)
.IP \[bu] 2
Utilize streaming capabilities (Denis Neuling)
Add support for managed identities (Brad Ackerman)
.IP \[bu] 2
Update SDK to v0.13.0 and fix API breakage (Nick Craig\-Wood, Mitsuo
Heijo)
Add examples for access tier (Bob Pusateri)
.IP \[bu] 2
Utilize the streaming capabilities from the SDK for multipart uploads
(Denis Neuling)
.IP \[bu] 2
Fix setting of mime types (Nick Craig\-Wood)
.IP \[bu] 2
Fix crash when listing outside a SAS URL\[aq]s root (Nick Craig\-Wood)
.IP \[bu] 2
Delete archive tier blobs before update if
\-\-azureblob\-archive\-tier\-delete (Nick Craig\-Wood)
.IP \[bu] 2
Add support for managed identities (Brad Ackerman)
\f[C]\-\-azureblob\-archive\-tier\-delete\f[R] (Nick Craig\-Wood)
.IP \[bu] 2
Fix crash on startup (Nick Craig\-Wood)
.IP \[bu] 2
Add examples for access tier (Bob Pusateri)
.IP \[bu] 2
Fix memory usage by upgrading the SDK and implementing a TransferManager
(Nick Craig\-Wood)
Fix memory usage by upgrading the SDK to v0.13.0 and implementing a
TransferManager (Nick Craig\-Wood)
.IP \[bu] 2
Require go1.14+ to compile due to SDK changes (Nick Craig\-Wood)
.RE
@@ -38244,8 +38458,13 @@ B2
.RS 2
.IP \[bu] 2
Make NewObject use less expensive API calls (Nick Craig\-Wood)
.RS 2
.IP \[bu] 2
Fixed possible crash when accessing Backblaze b2 remote (lluuaapp)
This will improve \f[C]\-\-files\-from\f[R] and \f[C]restic serve\f[R]
in particular
.RE
.IP \[bu] 2
Fixed crash on an empty file name (lluuaapp)
.RE
.IP \[bu] 2
Box
@@ -38261,8 +38480,8 @@ Chunker
.IP \[bu] 2
Skip long local hashing, hash in\-transit (fixes) (Ivan Andreev)
.IP \[bu] 2
Set Features.ReadMimeType=false as Object.MimeType not supported (Nick
Craig\-Wood)
Set Features ReadMimeType to false as Object.MimeType not supported
(Nick Craig\-Wood)
.IP \[bu] 2
Fix case\-insensitive NewObject, test metadata detection (Ivan Andreev)
.RE
@@ -38270,7 +38489,7 @@ Fix case\-insensitive NewObject, test metadata detection (Ivan Andreev)
Drive
.RS 2
.IP \[bu] 2
Implement \[dq]rclone backend copyid\[dq] command for copying files by
Implement \f[C]rclone backend copyid\f[R] command for copying files by
ID (Nick Craig\-Wood)
.IP \[bu] 2
Added flag \f[C]\-\-drive\-stop\-on\-download\-limit\f[R] to stop
@@ -38290,36 +38509,35 @@ Dropbox
.IP \[bu] 2
Add support for viewing shared files and folders (buengese)
.IP \[bu] 2
Implement IDer (buengese)
Enable short lived access tokens (Nick Craig\-Wood)
.IP \[bu] 2
Set Features.ReadMimeType=false as Object.MimeType not supported (Nick
Craig\-Wood)
Implement IDer on Objects so \f[C]rclone lsf\f[R] etc can read the IDs
(buengese)
.IP \[bu] 2
Tidy repeated error message (Nick Craig\-Wood)
Set Features ReadMimeType to false as Object.MimeType not supported
(Nick Craig\-Wood)
.IP \[bu] 2
Make malformed_path errors from too long files not retriable (Nick
Craig\-Wood)
.IP \[bu] 2
Test file name length before upload to fix upload loop (Nick
Craig\-Wood)
.IP \[bu] 2
Enable short lived access tokens (Nick Craig\-Wood)
.RE
.IP \[bu] 2
Fichier
.RS 2
.IP \[bu] 2
Set Features.ReadMimeType=true as Object.MimeType is supported (Nick
Set Features ReadMimeType to true as Object.MimeType is supported (Nick
Craig\-Wood)
.RE
.IP \[bu] 2
FTP
.RS 2
.IP \[bu] 2
Add \-\-ftp\-disable\-msld option to ignore MLSD for really old servers
(Nick Craig\-Wood)
Add \f[C]\-\-ftp\-disable\-msld\f[R] option to ignore MLSD for really
old servers (Nick Craig\-Wood)
.IP \[bu] 2
Make \-\-tpslimit apply (Nick Craig\-Wood)
Make \f[C]\-\-tpslimit apply\f[R] (Nick Craig\-Wood)
.RE
.IP \[bu] 2
Google Cloud Storage
@@ -38329,27 +38547,33 @@ Storage class object header support (Laurens Janssen)
.IP \[bu] 2
Fix anonymous client to use rclone\[aq]s HTTP client (Nick Craig\-Wood)
.IP \[bu] 2
Fix Entry doesn\[aq]t belong in directory \[dq]\[dq] (same as directory)
\- ignoring (Nick Craig\-Wood)
Fix
\f[C]Entry doesn\[aq]t belong in directory \[dq]\[dq] (same as directory) \- ignoring\f[R]
(Nick Craig\-Wood)
.RE
.IP \[bu] 2
Googlephotos
.RS 2
.IP \[bu] 2
New flag \-\-gphotos\-include\-archived (Nicolas Rueff)
New flag \f[C]\-\-gphotos\-include\-archived\f[R] to show archived
photos as well (Nicolas Rueff)
.RE
.IP \[bu] 2
Jottacloud
.RS 2
.IP \[bu] 2
Don\[aq]t erroniously report support for writing mime types (buengese)
Don\[aq]t erroneously report support for writing mime types (buengese)
.IP \[bu] 2
Add support for Telia Cloud (#4930) (Patrik Nordl\['e]n)
Add support for Telia Cloud (Patrik Nordl\['e]n)
.RE
.IP \[bu] 2
Mailru
.RS 2
.IP \[bu] 2
Accept special folders eg camera\-upload (Ivan Andreev)
.IP \[bu] 2
Avoid prehashing of large local files (Ivan Andreev)
.IP \[bu] 2
Fix uploads after recent changes on server (Ivan Andreev)
.IP \[bu] 2
Fix range requests after June 2020 changes on server (Ivan Andreev)
@@ -38357,10 +38581,6 @@ Fix range requests after June 2020 changes on server (Ivan Andreev)
Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev)
.IP \[bu] 2
Remove deprecated protocol quirks (Ivan Andreev)
.IP \[bu] 2
Accept special folders eg camera\-upload (Ivan Andreev)
.IP \[bu] 2
Avoid prehashing of large local files (Ivan Andreev)
.RE
.IP \[bu] 2
Memory
@@ -38372,26 +38592,25 @@ Fix setting of mime types (Nick Craig\-Wood)
Onedrive
.RS 2
.IP \[bu] 2
Add support for china region operated by 21vianet and other regional
suppliers (#4963) (NyaMisty)
Add support for China region operated by 21vianet and other regional
suppliers (NyaMisty)
.IP \[bu] 2
Warn on gateway timeout errors (Nick Craig\-Wood)
.IP \[bu] 2
Fall back to normal copy if server\-side copy unavailable (#4903) (Alex
Chen)
Fall back to normal copy if server\-side copy unavailable (Alex Chen)
.IP \[bu] 2
Fix server\-side copy completely disabled on OneDrive for Business
(Cnly)
.IP \[bu] 2
(business only) workaround to replace existing file on server\-side copy
(#4904) (Alex Chen)
(Alex Chen)
.IP \[bu] 2
Enhance link creation with expiry, scope, type and password (Nick
Craig\-Wood)
.IP \[bu] 2
Remove % and # from the set of encoded characters (#4909) (Alex Chen)
Remove % and # from the set of encoded characters (Alex Chen)
.IP \[bu] 2
Support addressing site by server\-relative URL (#4761) (kice)
Support addressing site by server\-relative URL (kice)
.RE
.IP \[bu] 2
Opendrive
@@ -38427,11 +38646,16 @@ Added \f[C]\-\-s3\-disable\-http2\f[R] to disable http/2 (Anagh Kumar
Baranwal)
.IP \[bu] 2
Complete SSE\-C implementation (Nick Craig\-Wood)
.RS 2
.IP \[bu] 2
Fix hashes on small files with AWS:KMS and SSE\-C (Nick Craig\-Wood)
.IP \[bu] 2
Add MD5 metadata to objects uploaded with SSE\-AWS/SSE\-C (Nick
Craig\-Wood)
.RE
.IP \[bu] 2
Add \f[C]\-\-s3\-no\-head parameter\f[R] to minimise transactions on
upload (Nick Craig\-Wood)
.IP \[bu] 2
Update docs with a Reducing Costs section (Nick Craig\-Wood)
.IP \[bu] 2
@@ -38442,9 +38666,6 @@ Add requester pays option (kelv)
.IP \[bu] 2
Fix copy multipart with v2 auth failing with
\[aq]SignatureDoesNotMatch\[aq] (Louis Koo)
.IP \[bu] 2
Add \-\-s3\-no\-head parameter to minimise transactions on upload (Nick
Craig\-Wood)
.RE
.IP \[bu] 2
SFTP
@@ -38464,9 +38685,10 @@ Implement Shutdown method (Nick Craig\-Wood)
.IP \[bu] 2
Implement keyboard interactive authentication (Nick Craig\-Wood)
.IP \[bu] 2
Make \-\-tpslimit apply (Nick Craig\-Wood)
Make \f[C]\-\-tpslimit\f[R] apply (Nick Craig\-Wood)
.IP \[bu] 2
Implement \-\-sftp\-use\-fstat (Nick Craig\-Wood)
Implement \f[C]\-\-sftp\-use\-fstat\f[R] for unusual SFTP servers (Nick
Craig\-Wood)
.RE
.IP \[bu] 2
Sugarsync
@@ -38484,7 +38706,8 @@ Fix deletion of parts of Static Large Object (SLO) (Nguy\[u1EC5]n
H\[u1EEF]u Lu\[^a]n)
.IP \[bu] 2
Ensure partially uploaded large files are uploaded unless
\-\-swift\-leave\-parts\-on\-error (Nguy\[u1EC5]n H\[u1EEF]u Lu\[^a]n)
\f[C]\-\-swift\-leave\-parts\-on\-error\f[R] (Nguy\[u1EC5]n H\[u1EEF]u
Lu\[^a]n)
.RE
.IP \[bu] 2
Tardigrade
@@ -38502,7 +38725,7 @@ Updated docs to show streaming to nextcloud is working (Durval Menezes)
Yandex
.RS 2
.IP \[bu] 2
Set Features.WriteMimeType=false as Yandex ignores mime types (Nick
Set Features WriteMimeType to false as Yandex ignores mime types (Nick
Craig\-Wood)
.RE
.SS v1.53.4 \- 2021\-01\-20

View File

@@ -110,6 +110,13 @@ for two reasons. Firstly because it is only checked every
!--vfs-cache-poll-interval!. Secondly because open files cannot be
evicted from the cache.
You **should not** run two copies of rclone using the same VFS cache
with the same or overlapping remotes if using !--vfs-cache-mode > off!.
This can potentially cause data corruption if you do. You can work
around this by giving each rclone its own cache hierarchy with
!--cache-dir!. You don't need to worry about this if the remotes in
use don't overlap.
#### --vfs-cache-mode off
In this mode (the default) the cache will read directly from the remote and write