1
0
mirror of https://github.com/rclone/rclone.git synced 2025-12-15 15:53:41 +00:00

docs: fix markdownlint issue md046/code-block-style

This commit is contained in:
albertony
2025-07-10 14:09:07 +02:00
parent 2c369aedf5
commit 667ad093eb
9 changed files with 623 additions and 298 deletions

View File

@@ -32,6 +32,7 @@ Then [install Git](https://git-scm.com/downloads) and set your public contributi
Next open your terminal, change directory to your preferred folder and initialise your local rclone project: Next open your terminal, change directory to your preferred folder and initialise your local rclone project:
```
git clone https://github.com/rclone/rclone.git git clone https://github.com/rclone/rclone.git
cd rclone cd rclone
git remote rename origin upstream git remote rename origin upstream
@@ -39,23 +40,30 @@ Next open your terminal, change directory to your preferred folder and initialis
git remote add origin git@github.com:YOURUSER/rclone.git git remote add origin git@github.com:YOURUSER/rclone.git
# otherwise: # otherwise:
git remote add origin https://github.com/YOURUSER/rclone.git git remote add origin https://github.com/YOURUSER/rclone.git
```
Note that most of the terminal commands in the rest of this guide must be executed from the rclone folder created above. Note that most of the terminal commands in the rest of this guide must be executed from the rclone folder created above.
Now [install Go](https://golang.org/doc/install) and verify your installation: Now [install Go](https://golang.org/doc/install) and verify your installation:
```
go version go version
```
Great, you can now compile and execute your own version of rclone: Great, you can now compile and execute your own version of rclone:
```
go build go build
./rclone version ./rclone version
```
(Note that you can also replace `go build` with `make`, which will include a (Note that you can also replace `go build` with `make`, which will include a
more accurate version number in the executable as well as enable you to specify more accurate version number in the executable as well as enable you to specify
more build options.) Finally make a branch to add your new feature more build options.) Finally make a branch to add your new feature
```
git checkout -b my-new-feature git checkout -b my-new-feature
```
And get hacking. And get hacking.
@@ -63,8 +71,10 @@ You may like one of the [popular editors/IDE's for Go](https://github.com/golang
When ready - test the affected functionality and run the unit tests for the code you changed When ready - test the affected functionality and run the unit tests for the code you changed
```
cd folder/with/changed/files cd folder/with/changed/files
go test -v go test -v
```
Note that you may need to make a test remote, e.g. `TestSwift` for some Note that you may need to make a test remote, e.g. `TestSwift` for some
of the unit tests. of the unit tests.
@@ -79,7 +89,9 @@ Make sure you
When you are done with that push your changes to GitHub: When you are done with that push your changes to GitHub:
```
git push -u origin my-new-feature git push -u origin my-new-feature
```
and open the GitHub website to [create your pull and open the GitHub website to [create your pull
request](https://help.github.com/articles/creating-a-pull-request/). request](https://help.github.com/articles/creating-a-pull-request/).
@@ -94,16 +106,20 @@ You may sometimes be asked to [base your changes on the latest master](#basing-y
Follow the guideline for [commit messages](#commit-messages) and then: Follow the guideline for [commit messages](#commit-messages) and then:
```
git checkout my-new-feature # To switch to your branch git checkout my-new-feature # To switch to your branch
git status # To see the new and changed files git status # To see the new and changed files
git add FILENAME # To select FILENAME for the commit git add FILENAME # To select FILENAME for the commit
git status # To verify the changes to be committed git status # To verify the changes to be committed
git commit # To do the commit git commit # To do the commit
git log # To verify the commit. Use q to quit the log git log # To verify the commit. Use q to quit the log
```
You can modify the message or changes in the latest commit using: You can modify the message or changes in the latest commit using:
```
git commit --amend git commit --amend
```
If you amend to commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits). If you amend to commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits).
@@ -113,18 +129,22 @@ Note that you are about to rewrite the GitHub history of your branch. It is good
Your previously pushed commits are replaced by: Your previously pushed commits are replaced by:
```
git push --force origin my-new-feature git push --force origin my-new-feature
```
### Basing your changes on the latest master ### Basing your changes on the latest master
To base your changes on the latest version of the [rclone master](https://github.com/rclone/rclone/tree/master) (upstream): To base your changes on the latest version of the [rclone master](https://github.com/rclone/rclone/tree/master) (upstream):
```
git checkout master git checkout master
git fetch upstream git fetch upstream
git merge --ff-only git merge --ff-only
git push origin --follow-tags # optional update of your fork in GitHub git push origin --follow-tags # optional update of your fork in GitHub
git checkout my-new-feature git checkout my-new-feature
git rebase master git rebase master
```
If you rebase commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits). If you rebase commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits).
@@ -132,18 +152,24 @@ If you rebase commits that have been pushed to GitHub, then you will have to [re
To combine your commits into one commit: To combine your commits into one commit:
```
git log # To count the commits to squash, e.g. the last 2 git log # To count the commits to squash, e.g. the last 2
git reset --soft HEAD~2 # To undo the 2 latest commits git reset --soft HEAD~2 # To undo the 2 latest commits
git status # To check everything is as expected git status # To check everything is as expected
```
If everything is fine, then make the new combined commit: If everything is fine, then make the new combined commit:
```
git commit # To commit the undone commits as one git commit # To commit the undone commits as one
```
otherwise, you may roll back using: otherwise, you may roll back using:
```
git reflog # To check that HEAD{1} is your previous state git reflog # To check that HEAD{1} is your previous state
git reset --soft 'HEAD@{1}' # To roll back to your previous state git reset --soft 'HEAD@{1}' # To roll back to your previous state
```
If you squash commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits). If you squash commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits).
@@ -168,11 +194,15 @@ Using these tests ensures that the rclone codebase all uses the same coding stan
rclone's tests are run from the go testing framework, so at the top rclone's tests are run from the go testing framework, so at the top
level you can run this to run all the tests. level you can run this to run all the tests.
```
go test -v ./... go test -v ./...
```
You can also use `make`, if supported by your platform You can also use `make`, if supported by your platform
```
make quicktest make quicktest
```
The quicktest is [automatically run by GitHub](#github-continuous-integration) when you push your branch to GitHub. The quicktest is [automatically run by GitHub](#github-continuous-integration) when you push your branch to GitHub.
@@ -190,38 +220,48 @@ need to make a remote called `TestDrive`.
You can then run the unit tests in the drive directory. These tests You can then run the unit tests in the drive directory. These tests
are skipped if `TestDrive:` isn't defined. are skipped if `TestDrive:` isn't defined.
```
cd backend/drive cd backend/drive
go test -v go test -v
```
You can then run the integration tests which test all of rclone's You can then run the integration tests which test all of rclone's
operations. Normally these get run against the local file system, operations. Normally these get run against the local file system,
but they can be run against any of the remotes. but they can be run against any of the remotes.
```
cd fs/sync cd fs/sync
go test -v -remote TestDrive: go test -v -remote TestDrive:
go test -v -remote TestDrive: -fast-list go test -v -remote TestDrive: -fast-list
cd fs/operations cd fs/operations
go test -v -remote TestDrive: go test -v -remote TestDrive:
```
If you want to use the integration test framework to run these tests If you want to use the integration test framework to run these tests
altogether with an HTML report and test retries then from the altogether with an HTML report and test retries then from the
project root: project root:
```
go install github.com/rclone/rclone/fstest/test_all go install github.com/rclone/rclone/fstest/test_all
test_all -backends drive test_all -backends drive
```
### Full integration testing ### Full integration testing
If you want to run all the integration tests against all the remotes, If you want to run all the integration tests against all the remotes,
then change into the project root and run then change into the project root and run
```
make check make check
make test make test
```
The commands may require some extra go packages which you can install with The commands may require some extra go packages which you can install with
```
make build_dep make build_dep
```
The full integration tests are run daily on the integration test server. You can The full integration tests are run daily on the integration test server. You can
find the results at https://pub.rclone.org/integration-tests/ find the results at https://pub.rclone.org/integration-tests/
@@ -379,7 +419,9 @@ To add a dependency `github.com/ncw/new_dependency` see the
instructions below. These will fetch the dependency and add it to instructions below. These will fetch the dependency and add it to
`go.mod` and `go.sum`. `go.mod` and `go.sum`.
```
go get github.com/ncw/new_dependency go get github.com/ncw/new_dependency
```
You can add constraints on that package when doing `go get` (see the You can add constraints on that package when doing `go get` (see the
go docs linked above), but don't unless you really need to. go docs linked above), but don't unless you really need to.
@@ -391,7 +433,9 @@ and `go.sum` in the same commit as your other changes.
If you need to update a dependency then run If you need to update a dependency then run
```
go get golang.org/x/crypto go get golang.org/x/crypto
```
Check in a single commit as above. Check in a single commit as above.

View File

@@ -48,7 +48,9 @@ Early in the next release cycle update the dependencies.
If the `make updatedirect` upgrades the version of go in the `go.mod` If the `make updatedirect` upgrades the version of go in the `go.mod`
```
go 1.22.0 go 1.22.0
```
then go to manual mode. `go1.22` here is the lowest supported version then go to manual mode. `go1.22` here is the lowest supported version
in the `go.mod`. in the `go.mod`.
@@ -99,7 +101,9 @@ The above procedure will not upgrade major versions, so v2 to v3.
However this tool can show which major versions might need to be However this tool can show which major versions might need to be
upgraded: upgraded:
```
go run github.com/icholy/gomajor@latest list -major go run github.com/icholy/gomajor@latest list -major
```
Expect API breakage when updating major versions. Expect API breakage when updating major versions.
@@ -107,7 +111,9 @@ Expect API breakage when updating major versions.
At some point after the release run At some point after the release run
```
bin/tidy-beta v1.55 bin/tidy-beta v1.55
```
where the version number is that of a couple ago to remove old beta binaries. where the version number is that of a couple ago to remove old beta binaries.
@@ -150,21 +156,29 @@ which is a private repo containing artwork from sponsors.
Create an update website branch based off the last release Create an update website branch based off the last release
```
git co -b update-website git co -b update-website
```
If the branch already exists, double check there are no commits that need saving. If the branch already exists, double check there are no commits that need saving.
Now reset the branch to the last release Now reset the branch to the last release
```
git reset --hard v1.64.0 git reset --hard v1.64.0
```
Create the changes, check them in, test with `make serve` then Create the changes, check them in, test with `make serve` then
```
make upload_test_website make upload_test_website
```
Check out https://test.rclone.org and when happy Check out https://test.rclone.org and when happy
```
make upload_website make upload_website
```
Cherry pick any changes back to master and the stable branch if it is active. Cherry pick any changes back to master and the stable branch if it is active.

View File

@@ -22,7 +22,9 @@ file and choose its location.)
The easiest way to make the config is to run rclone with the config The easiest way to make the config is to run rclone with the config
option: option:
```
rclone config rclone config
```
See the following for detailed instructions for See the following for detailed instructions for
@@ -97,7 +99,9 @@ Rclone syncs a directory tree from one storage system to another.
Its syntax is like this Its syntax is like this
```
rclone subcommand [options] <parameters> <parameters...> rclone subcommand [options] <parameters> <parameters...>
```
A `subcommand` is a the rclone operation required, (e.g. `sync`, A `subcommand` is a the rclone operation required, (e.g. `sync`,
`copy`, `ls`). `copy`, `ls`).
@@ -110,7 +114,9 @@ used before the `subcommand`. Anything after a `--` option will not be
interpreted as an option so if you need to add a parameter which interpreted as an option so if you need to add a parameter which
starts with a `-` then put a `--` on its own first, eg starts with a `-` then put a `--` on its own first, eg
```
rclone lsf -- -directory-starting-with-dash rclone lsf -- -directory-starting-with-dash
```
A `parameter` is usually a file path or [rclone remote](#syntax-of-remote-paths), eg A `parameter` is usually a file path or [rclone remote](#syntax-of-remote-paths), eg
`/path/to/file` or `remote:path/to/file` but it can be other things - `/path/to/file` or `remote:path/to/file` but it can be other things -
@@ -129,9 +135,11 @@ learning rclone to avoid accidental data loss.
rclone uses a system of subcommands. For example rclone uses a system of subcommands. For example
```
rclone ls remote:path # lists a remote rclone ls remote:path # lists a remote
rclone copy /local/path remote:path # copies /local/path to the remote rclone copy /local/path remote:path # copies /local/path to the remote
rclone sync --interactive /local/path remote:path # syncs /local/path to the remote rclone sync --interactive /local/path remote:path # syncs /local/path to the remote
```
The main rclone commands with most used first The main rclone commands with most used first
@@ -180,17 +188,23 @@ directory` if it isn't.
For example, suppose you have a remote with a file in called For example, suppose you have a remote with a file in called
`test.jpg`, then you could copy just that file like this `test.jpg`, then you could copy just that file like this
```
rclone copy remote:test.jpg /tmp/download rclone copy remote:test.jpg /tmp/download
```
The file `test.jpg` will be placed inside `/tmp/download`. The file `test.jpg` will be placed inside `/tmp/download`.
This is equivalent to specifying This is equivalent to specifying
```
rclone copy --files-from /tmp/files remote: /tmp/download rclone copy --files-from /tmp/files remote: /tmp/download
```
Where `/tmp/files` contains the single line Where `/tmp/files` contains the single line
```
test.jpg test.jpg
```
It is recommended to use `copy` when copying individual files, not `sync`. It is recommended to use `copy` when copying individual files, not `sync`.
They have pretty much the same effect but `copy` will use a lot less They have pretty much the same effect but `copy` will use a lot less
@@ -234,19 +248,27 @@ the command line (or in environment variables).
Here are some examples: Here are some examples:
```
rclone lsd --http-url https://pub.rclone.org :http: rclone lsd --http-url https://pub.rclone.org :http:
```
To list all the directories in the root of `https://pub.rclone.org/`. To list all the directories in the root of `https://pub.rclone.org/`.
```
rclone lsf --http-url https://example.com :http:path/to/dir rclone lsf --http-url https://example.com :http:path/to/dir
```
To list files and directories in `https://example.com/path/to/dir/` To list files and directories in `https://example.com/path/to/dir/`
```
rclone copy --http-url https://example.com :http:path/to/dir /tmp/dir rclone copy --http-url https://example.com :http:path/to/dir /tmp/dir
```
To copy files and directories in `https://example.com/path/to/dir` to `/tmp/dir`. To copy files and directories in `https://example.com/path/to/dir` to `/tmp/dir`.
```
rclone copy --sftp-host example.com :sftp:path/to/dir /tmp/dir rclone copy --sftp-host example.com :sftp:path/to/dir /tmp/dir
```
To copy files and directories from `example.com` in the relative To copy files and directories from `example.com` in the relative
directory `path/to/dir` to `/tmp/dir` using sftp. directory `path/to/dir` to `/tmp/dir` using sftp.
@@ -258,16 +280,20 @@ syntax, so instead of providing the arguments as command line
parameters `--http-url https://pub.rclone.org` they are provided as parameters `--http-url https://pub.rclone.org` they are provided as
part of the remote specification as a kind of connection string. part of the remote specification as a kind of connection string.
```
rclone lsd ":http,url='https://pub.rclone.org':" rclone lsd ":http,url='https://pub.rclone.org':"
rclone lsf ":http,url='https://example.com':path/to/dir" rclone lsf ":http,url='https://example.com':path/to/dir"
rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
rclone copy :sftp,host=example.com:path/to/dir /tmp/dir rclone copy :sftp,host=example.com:path/to/dir /tmp/dir
```
These can apply to modify existing remotes as well as create new These can apply to modify existing remotes as well as create new
remotes with the on the fly syntax. This example is equivalent to remotes with the on the fly syntax. This example is equivalent to
adding the `--drive-shared-with-me` parameter to the remote `gdrive:`. adding the `--drive-shared-with-me` parameter to the remote `gdrive:`.
```
rclone lsf "gdrive,shared_with_me:path/to/dir" rclone lsf "gdrive,shared_with_me:path/to/dir"
```
The major advantage to using the connection string style syntax is The major advantage to using the connection string style syntax is
that it only applies to the remote, not to all the remotes of that that it only applies to the remote, not to all the remotes of that
@@ -276,34 +302,46 @@ file shared on google drive to the normal drive which **does not
work** because the `--drive-shared-with-me` flag applies to both the work** because the `--drive-shared-with-me` flag applies to both the
source and the destination. source and the destination.
```
rclone copy --drive-shared-with-me gdrive:shared-file.txt gdrive: rclone copy --drive-shared-with-me gdrive:shared-file.txt gdrive:
```
However using the connection string syntax, this does work. However using the connection string syntax, this does work.
```
rclone copy "gdrive,shared_with_me:shared-file.txt" gdrive: rclone copy "gdrive,shared_with_me:shared-file.txt" gdrive:
```
Note that the connection string only affects the options of the immediate Note that the connection string only affects the options of the immediate
backend. If for example gdriveCrypt is a crypt based on gdrive, then the backend. If for example gdriveCrypt is a crypt based on gdrive, then the
following command **will not work** as intended, because following command **will not work** as intended, because
`shared_with_me` is ignored by the crypt backend: `shared_with_me` is ignored by the crypt backend:
```
rclone copy "gdriveCrypt,shared_with_me:shared-file.txt" gdriveCrypt: rclone copy "gdriveCrypt,shared_with_me:shared-file.txt" gdriveCrypt:
```
The connection strings have the following syntax The connection strings have the following syntax
```
remote,parameter=value,parameter2=value2:path/to/dir remote,parameter=value,parameter2=value2:path/to/dir
:backend,parameter=value,parameter2=value2:path/to/dir :backend,parameter=value,parameter2=value2:path/to/dir
```
If the `parameter` has a `:` or `,` then it must be placed in quotes `"` or If the `parameter` has a `:` or `,` then it must be placed in quotes `"` or
`'`, so `'`, so
```
remote,parameter="colon:value",parameter2="comma,value":path/to/dir remote,parameter="colon:value",parameter2="comma,value":path/to/dir
:backend,parameter='colon:value',parameter2='comma,value':path/to/dir :backend,parameter='colon:value',parameter2='comma,value':path/to/dir
```
If a quoted value needs to include that quote, then it should be If a quoted value needs to include that quote, then it should be
doubled, so doubled, so
```
remote,parameter="with""quote",parameter2='with''quote':path/to/dir remote,parameter="with""quote",parameter2='with''quote':path/to/dir
```
This will make `parameter` be `with"quote` and `parameter2` be This will make `parameter` be `with"quote` and `parameter2` be
`with'quote`. `with'quote`.
@@ -312,11 +350,15 @@ If you leave off the `=parameter` then rclone will substitute `=true`
which works very well with flags. For example, to use s3 configured in which works very well with flags. For example, to use s3 configured in
the environment you could use: the environment you could use:
```
rclone lsd :s3,env_auth: rclone lsd :s3,env_auth:
```
Which is equivalent to Which is equivalent to
```
rclone lsd :s3,env_auth=true: rclone lsd :s3,env_auth=true:
```
Note that on the command line you might need to surround these Note that on the command line you might need to surround these
connection strings with `"` or `'` to stop the shell interpreting any connection strings with `"` or `'` to stop the shell interpreting any
@@ -326,14 +368,18 @@ If you are a shell master then you'll know which strings are OK and
which aren't, but if you aren't sure then enclose them in `"` and use which aren't, but if you aren't sure then enclose them in `"` and use
`'` as the inside quote. This syntax works on all OSes. `'` as the inside quote. This syntax works on all OSes.
```
rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
```
On Linux/macOS some characters are still interpreted inside `"` On Linux/macOS some characters are still interpreted inside `"`
strings in the shell (notably `\` and `$` and `"`) so if your strings strings in the shell (notably `\` and `$` and `"`) so if your strings
contain those you can swap the roles of `"` and `'` thus. (This syntax contain those you can swap the roles of `"` and `'` thus. (This syntax
does not work on Windows.) does not work on Windows.)
```
rclone copy ':http,url="https://example.com":path/to/dir' /tmp/dir rclone copy ':http,url="https://example.com":path/to/dir' /tmp/dir
```
#### Connection strings, config and logging #### Connection strings, config and logging
@@ -341,11 +387,15 @@ If you supply extra configuration to a backend by command line flag,
environment variable or connection string then rclone will add a environment variable or connection string then rclone will add a
suffix based on the hash of the config to the name of the remote, eg suffix based on the hash of the config to the name of the remote, eg
```
rclone -vv lsf --s3-chunk-size 20M s3: rclone -vv lsf --s3-chunk-size 20M s3:
```
Has the log message Has the log message
```
DEBUG : s3: detected overridden config - adding "{Srj1p}" suffix to name DEBUG : s3: detected overridden config - adding "{Srj1p}" suffix to name
```
This is so rclone can tell the modified remote apart from the This is so rclone can tell the modified remote apart from the
unmodified remote when caching the backends. unmodified remote when caching the backends.
@@ -354,11 +404,15 @@ This should only be noticeable in the logs.
This means that on the fly backends such as This means that on the fly backends such as
```
rclone -vv lsf :s3,env_auth: rclone -vv lsf :s3,env_auth:
```
Will get their own names Will get their own names
```
DEBUG : :s3: detected overridden config - adding "{YTu53}" suffix to name DEBUG : :s3: detected overridden config - adding "{YTu53}" suffix to name
```
### Valid remote names ### Valid remote names
@@ -393,11 +447,15 @@ Here are some gotchas which may help users unfamiliar with the shell rules
If your names have spaces or shell metacharacters (e.g. `*`, `?`, `$`, If your names have spaces or shell metacharacters (e.g. `*`, `?`, `$`,
`'`, `"`, etc.) then you must quote them. Use single quotes `'` by default. `'`, `"`, etc.) then you must quote them. Use single quotes `'` by default.
```
rclone copy 'Important files?' remote:backup rclone copy 'Important files?' remote:backup
```
If you want to send a `'` you will need to use `"`, e.g. If you want to send a `'` you will need to use `"`, e.g.
```
rclone copy "O'Reilly Reviews" remote:backup rclone copy "O'Reilly Reviews" remote:backup
```
The rules for quoting metacharacters are complicated and if you want The rules for quoting metacharacters are complicated and if you want
the full details you'll have to consult the manual page for your the full details you'll have to consult the manual page for your
@@ -407,12 +465,16 @@ shell.
If your names have spaces in you need to put them in `"`, e.g. If your names have spaces in you need to put them in `"`, e.g.
```
rclone copy "E:\folder name\folder name\folder name" remote:backup rclone copy "E:\folder name\folder name\folder name" remote:backup
```
If you are using the root directory on its own then don't quote it If you are using the root directory on its own then don't quote it
(see [#464](https://github.com/rclone/rclone/issues/464) for why), e.g. (see [#464](https://github.com/rclone/rclone/issues/464) for why), e.g.
```
rclone copy E:\ remote:backup rclone copy E:\ remote:backup
```
## Copying files or directories with `:` in the names ## Copying files or directories with `:` in the names
@@ -424,11 +486,15 @@ file or directory like this then use the full path starting with a
So to sync a directory called `sync:me` to a remote called `remote:` use So to sync a directory called `sync:me` to a remote called `remote:` use
```
rclone sync --interactive ./sync:me remote:path rclone sync --interactive ./sync:me remote:path
```
or or
```
rclone sync --interactive /full/path/to/sync:me remote:path rclone sync --interactive /full/path/to/sync:me remote:path
```
## Server-side copy ## Server-side copy
@@ -441,7 +507,9 @@ to copy them in place.
Eg Eg
```
rclone copy s3:oldbucket s3:newbucket rclone copy s3:oldbucket s3:newbucket
```
Will copy the contents of `oldbucket` to `newbucket` without Will copy the contents of `oldbucket` to `newbucket` without
downloading and re-uploading. downloading and re-uploading.
@@ -460,8 +528,10 @@ same.
This can be used when scripting to make aged backups efficiently, e.g. This can be used when scripting to make aged backups efficiently, e.g.
```
rclone sync --interactive remote:current-backup remote:previous-backup rclone sync --interactive remote:current-backup remote:previous-backup
rclone sync --interactive /path/to/files remote:current-backup rclone sync --interactive /path/to/files remote:current-backup
```
## Metadata support {#metadata} ## Metadata support {#metadata}
@@ -698,7 +768,9 @@ excluded by a filter rule.
For example For example
```
rclone sync --interactive /path/to/local remote:current --backup-dir remote:old rclone sync --interactive /path/to/local remote:current --backup-dir remote:old
```
will sync `/path/to/local` to `remote:current`, but for any files will sync `/path/to/local` to `remote:current`, but for any files
which would have been updated or deleted will be stored in which would have been updated or deleted will be stored in
@@ -724,7 +796,9 @@ You can use `--bind 0.0.0.0` to force rclone to use IPv4 addresses and
This option controls the bandwidth limit. For example This option controls the bandwidth limit. For example
```
--bwlimit 10M --bwlimit 10M
```
would mean limit the upload and download bandwidth to 10 MiB/s. would mean limit the upload and download bandwidth to 10 MiB/s.
**NB** this is **bytes** per second not **bits** per second. To use a **NB** this is **bytes** per second not **bits** per second. To use a
@@ -734,13 +808,17 @@ suffix B|K|M|G|T|P. The default is `0` which means to not limit bandwidth.
The upload and download bandwidth can be specified separately, as The upload and download bandwidth can be specified separately, as
`--bwlimit UP:DOWN`, so `--bwlimit UP:DOWN`, so
```
--bwlimit 10M:100k --bwlimit 10M:100k
```
would mean limit the upload bandwidth to 10 MiB/s and the download would mean limit the upload bandwidth to 10 MiB/s and the download
bandwidth to 100 KiB/s. Either limit can be "off" meaning no limit, so bandwidth to 100 KiB/s. Either limit can be "off" meaning no limit, so
to just limit the upload bandwidth you would use to just limit the upload bandwidth you would use
```
--bwlimit 10M:off --bwlimit 10M:off
```
this would limit the upload bandwidth to 10 MiB/s but the download this would limit the upload bandwidth to 10 MiB/s but the download
bandwidth would be unlimited. bandwidth would be unlimited.
@@ -816,12 +894,16 @@ of a long running rclone transfer and to restore it back to the value specified
with `--bwlimit` quickly when needed. Assuming there is only one rclone instance with `--bwlimit` quickly when needed. Assuming there is only one rclone instance
running, you can toggle the limiter like this: running, you can toggle the limiter like this:
```
kill -SIGUSR2 $(pidof rclone) kill -SIGUSR2 $(pidof rclone)
```
If you configure rclone with a [remote control](/rc) then you can use If you configure rclone with a [remote control](/rc) then you can use
change the bwlimit dynamically: change the bwlimit dynamically:
```
rclone rc core/bwlimit rate=1M rclone rc core/bwlimit rate=1M
```
### --bwlimit-file BwTimetable ### --bwlimit-file BwTimetable
@@ -830,7 +912,9 @@ This option controls per file bandwidth limit. For the options see the
For example use this to allow no transfers to be faster than 1 MiB/s For example use this to allow no transfers to be faster than 1 MiB/s
```
--bwlimit-file 1M --bwlimit-file 1M
```
This can be used in conjunction with `--bwlimit`. This can be used in conjunction with `--bwlimit`.
@@ -1030,10 +1114,12 @@ beginning of a line.
Example: Example:
```
[megaremote] [megaremote]
type = mega type = mega
user = you@example.com user = you@example.com
pass = PDPcQVVjVtzFY-GTdDFozqBhTdsPg3qH pass = PDPcQVVjVtzFY-GTdDFozqBhTdsPg3qH
```
Note that passwords are in [obscured](/commands/rclone_obscure/) Note that passwords are in [obscured](/commands/rclone_obscure/)
form. Also, many storage systems uses token-based authentication instead form. Also, many storage systems uses token-based authentication instead
@@ -1113,17 +1199,23 @@ time rclone started up.
This disables a comma separated list of optional features. For example This disables a comma separated list of optional features. For example
to disable server-side move and server-side copy use: to disable server-side move and server-side copy use:
```
--disable move,copy --disable move,copy
```
The features can be put in any case. The features can be put in any case.
To see a list of which features can be disabled use: To see a list of which features can be disabled use:
```
--disable help --disable help
```
The features a remote has can be seen in JSON format with: The features a remote has can be seen in JSON format with:
```
rclone backend features remote: rclone backend features remote:
```
See the overview [features](/overview/#features) and See the overview [features](/overview/#features) and
[optional features](/overview/#optional-features) to get an idea of [optional features](/overview/#optional-features) to get an idea of
@@ -1401,7 +1493,9 @@ temporary file with an extension like this, where `XXXXXX` represents a
hash of the source file's fingerprint and `.partial` is hash of the source file's fingerprint and `.partial` is
[--partial-suffix](#partial-suffix) value (`.partial` by default). [--partial-suffix](#partial-suffix) value (`.partial` by default).
```
original-file-name.XXXXXX.partial original-file-name.XXXXXX.partial
```
(rclone will make sure the final name is no longer than 100 characters (rclone will make sure the final name is no longer than 100 characters
by truncating the `original-file-name` part if necessary). by truncating the `original-file-name` part if necessary).
@@ -1573,7 +1667,9 @@ once as administrator to create the registry key in advance.
severe) than or equal to the `--log-level`. For example to log DEBUG severe) than or equal to the `--log-level`. For example to log DEBUG
to a log file but ERRORs to the event log you would use to a log file but ERRORs to the event log you would use
```
--log-file rclone.log --log-level DEBUG --windows-event-log ERROR --log-file rclone.log --log-level DEBUG --windows-event-log ERROR
```
This option is only supported Windows platforms. This option is only supported Windows platforms.
@@ -1802,9 +1898,11 @@ it in `"`, if you want a literal `"` in an argument then enclose the
argument in `"` and double the `"`. See [CSV encoding](https://godoc.org/encoding/csv) argument in `"` and double the `"`. See [CSV encoding](https://godoc.org/encoding/csv)
for more info. for more info.
```
--metadata-mapper "python bin/test_metadata_mapper.py" --metadata-mapper "python bin/test_metadata_mapper.py"
--metadata-mapper 'python bin/test_metadata_mapper.py "argument with a space"' --metadata-mapper 'python bin/test_metadata_mapper.py "argument with a space"'
--metadata-mapper 'python bin/test_metadata_mapper.py "argument with ""two"" quotes"' --metadata-mapper 'python bin/test_metadata_mapper.py "argument with ""two"" quotes"'
```
This uses a simple JSON based protocol with input on STDIN and output This uses a simple JSON based protocol with input on STDIN and output
on STDOUT. This will be called for every file and directory copied and on STDOUT. This will be called for every file and directory copied and
@@ -2170,9 +2268,11 @@ for more info.
Eg Eg
```
--password-command "echo hello" --password-command "echo hello"
--password-command 'echo "hello with space"' --password-command 'echo "hello with space"'
--password-command 'echo "hello with ""quotes"" and space"' --password-command 'echo "hello with ""quotes"" and space"'
```
Note that when changing the configuration password the environment Note that when changing the configuration password the environment
variable `RCLONE_PASSWORD_CHANGE=1` will be set. This can be used to variable `RCLONE_PASSWORD_CHANGE=1` will be set. This can be used to
@@ -2372,7 +2472,9 @@ or with `--backup-dir`. See `--backup-dir` for more info.
For example For example
```
rclone copy --interactive /path/to/local/file remote:current --suffix .bak rclone copy --interactive /path/to/local/file remote:current --suffix .bak
```
will copy `/path/to/local` to `remote:current`, but for any files will copy `/path/to/local` to `remote:current`, but for any files
which would have been updated or deleted have .bak added. which would have been updated or deleted have .bak added.
@@ -2381,7 +2483,9 @@ If using `rclone sync` with `--suffix` and without `--backup-dir` then
it is recommended to put a filter rule in excluding the suffix it is recommended to put a filter rule in excluding the suffix
otherwise the `sync` will delete the backup files. otherwise the `sync` will delete the backup files.
```
rclone sync --interactive /path/to/local/file remote:current --suffix .bak --exclude "*.bak" rclone sync --interactive /path/to/local/file remote:current --suffix .bak --exclude "*.bak"
```
### --suffix-keep-extension ### --suffix-keep-extension
@@ -3194,7 +3298,9 @@ set the access key of all remotes using S3, including myS3Crypt.
Note also that now rclone has [connection strings](#connection-strings), Note also that now rclone has [connection strings](#connection-strings),
it is probably easier to use those instead which makes the above example it is probably easier to use those instead which makes the above example
```
rclone lsd :s3,access_key_id=XXX,secret_access_key=XXX: rclone lsd :s3,access_key_id=XXX,secret_access_key=XXX:
```
### Precedence ### Precedence

View File

@@ -51,11 +51,15 @@ signatures on the release.
To install rclone on Linux/macOS/BSD systems, run: To install rclone on Linux/macOS/BSD systems, run:
```
sudo -v ; curl https://rclone.org/install.sh | sudo bash sudo -v ; curl https://rclone.org/install.sh | sudo bash
```
For beta installation, run: For beta installation, run:
```
sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta
```
Note that this script checks the version of rclone installed first and Note that this script checks the version of rclone installed first and
won't re-download if not needed. won't re-download if not needed.
@@ -65,11 +69,15 @@ won't re-download if not needed.
[Beta releases](https://beta.rclone.org) are generated from each commit [Beta releases](https://beta.rclone.org) are generated from each commit
to master. Note these are named like to master. Note these are named like
```
{Version Tag}.beta.{Commit Number}.{Git Commit Hash} {Version Tag}.beta.{Commit Number}.{Git Commit Hash}
```
e.g. e.g.
```
v1.53.0-beta.4677.b657a2204 v1.53.0-beta.4677.b657a2204
```
The `Version Tag` is the version that the beta release will become The `Version Tag` is the version that the beta release will become
when it is released. You can match the `Git Commit Hash` up with the when it is released. You can match the `Git Commit Hash` up with the
@@ -79,11 +87,15 @@ and will normally be at the end of the list.
Some beta releases may have a branch name also: Some beta releases may have a branch name also:
```
{Version Tag}-beta.{Commit Number}.{Git Commit Hash}.{Branch Name} {Version Tag}-beta.{Commit Number}.{Git Commit Hash}.{Branch Name}
```
e.g. e.g.
```
v1.53.0-beta.4677.b657a2204.semver v1.53.0-beta.4677.b657a2204.semver
```
The presence of `Branch Name` indicates that this is a feature under The presence of `Branch Name` indicates that this is a feature under
development which will at some point be merged into the normal betas development which will at some point be merged into the normal betas

View File

@@ -47,7 +47,9 @@ The syncs would be incremental (on a file by file basis).
e.g. e.g.
```
rclone sync --interactive drive:Folder s3:bucket rclone sync --interactive drive:Folder s3:bucket
```
### Using rclone from multiple locations at the same time ### ### Using rclone from multiple locations at the same time ###
@@ -116,17 +118,21 @@ may use `http_proxy` but another one `HTTP_PROXY`. The `Go` libraries
used by `rclone` will try both variations, but you may wish to set all used by `rclone` will try both variations, but you may wish to set all
possibilities. So, on Linux, you may end up with code similar to possibilities. So, on Linux, you may end up with code similar to
```
export http_proxy=http://proxyserver:12345 export http_proxy=http://proxyserver:12345
export https_proxy=$http_proxy export https_proxy=$http_proxy
export HTTP_PROXY=$http_proxy export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy export HTTPS_PROXY=$http_proxy
```
Note: If the proxy server requires a username and password, then use Note: If the proxy server requires a username and password, then use
```
export http_proxy=http://username:password@proxyserver:12345 export http_proxy=http://username:password@proxyserver:12345
export https_proxy=$http_proxy export https_proxy=$http_proxy
export HTTP_PROXY=$http_proxy export HTTP_PROXY=$http_proxy
export HTTPS_PROXY=$http_proxy export HTTPS_PROXY=$http_proxy
```
The `NO_PROXY` allows you to disable the proxy for specific hosts. The `NO_PROXY` allows you to disable the proxy for specific hosts.
Hosts must be comma separated, and can contain domains or parts. Hosts must be comma separated, and can contain domains or parts.
@@ -134,8 +140,10 @@ For instance "foo.com" also matches "bar.foo.com".
e.g. e.g.
```
export no_proxy=localhost,127.0.0.0/8,my.host.name export no_proxy=localhost,127.0.0.0/8,my.host.name
export NO_PROXY=$no_proxy export NO_PROXY=$no_proxy
```
Note that the FTP backend does not support `ftp_proxy` yet. Note that the FTP backend does not support `ftp_proxy` yet.
@@ -148,10 +156,12 @@ possibly on Solaris.
Rclone (via the Go runtime) tries to load the root certificates from Rclone (via the Go runtime) tries to load the root certificates from
these places on Linux. these places on Linux.
```
"/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc. "/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc.
"/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL "/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL
"/etc/ssl/ca-bundle.pem", // OpenSUSE "/etc/ssl/ca-bundle.pem", // OpenSUSE
"/etc/pki/tls/cacert.pem", // OpenELEC "/etc/pki/tls/cacert.pem", // OpenELEC
```
So doing something like this should fix the problem. It also sets the So doing something like this should fix the problem. It also sets the
time which is important for SSL to work properly. time which is important for SSL to work properly.

View File

@@ -39,6 +39,7 @@ Here is a formal definition of the pattern syntax,
Rclone matching rules follow a glob style: Rclone matching rules follow a glob style:
```
* matches any sequence of non-separator (/) characters * matches any sequence of non-separator (/) characters
** matches any sequence of characters including / separators ** matches any sequence of characters including / separators
? matches any single non-separator (/) character ? matches any single non-separator (/) character
@@ -50,27 +51,37 @@ Rclone matching rules follow a glob style:
regular expression to match regular expression to match
c matches character c (c != *, **, ?, \, [, {, }) c matches character c (c != *, **, ?, \, [, {, })
\c matches reserved character c (c = *, **, ?, \, [, {, }) or character class \c matches reserved character c (c = *, **, ?, \, [, {, }) or character class
```
character-range: character-range:
```
c matches character c (c != \, -, ]) c matches character c (c != \, -, ])
\c matches reserved character c (c = \, -, ]) \c matches reserved character c (c = \, -, ])
lo - hi matches character c for lo <= c <= hi lo - hi matches character c for lo <= c <= hi
```
pattern-list: pattern-list:
```
pattern { , pattern } pattern { , pattern }
comma-separated (without spaces) patterns comma-separated (without spaces) patterns
```
character classes (see [Go regular expression reference](https://golang.org/pkg/regexp/syntax/)) include: character classes (see [Go regular expression reference](https://golang.org/pkg/regexp/syntax/)) include:
```
Named character classes (e.g. [\d], [^\d], [\D], [^\D]) Named character classes (e.g. [\d], [^\d], [\D], [^\D])
Perl character classes (e.g. \s, \S, \w, \W) Perl character classes (e.g. \s, \S, \w, \W)
ASCII character classes (e.g. [[:alnum:]], [[:alpha:]], [[:punct:]], [[:xdigit:]]) ASCII character classes (e.g. [[:alnum:]], [[:alpha:]], [[:punct:]], [[:xdigit:]])
```
regexp for advanced users to insert a regular expression - see [below](#regexp) for more info: regexp for advanced users to insert a regular expression - see [below](#regexp) for more info:
```
Any re2 regular expression not containing `}}` Any re2 regular expression not containing `}}`
```
If the filter pattern starts with a `/` then it only matches If the filter pattern starts with a `/` then it only matches
at the top level of the directory tree, at the top level of the directory tree,
@@ -80,6 +91,7 @@ starting at the **end of the path/file name** but it only matches
a complete path element - it must match from a `/` a complete path element - it must match from a `/`
separator or the beginning of the path/file. separator or the beginning of the path/file.
```
file.jpg - matches "file.jpg" file.jpg - matches "file.jpg"
- matches "directory/file.jpg" - matches "directory/file.jpg"
- doesn't match "afile.jpg" - doesn't match "afile.jpg"
@@ -87,11 +99,13 @@ separator or the beginning of the path/file.
/file.jpg - matches "file.jpg" in the root directory of the remote /file.jpg - matches "file.jpg" in the root directory of the remote
- doesn't match "afile.jpg" - doesn't match "afile.jpg"
- doesn't match "directory/file.jpg" - doesn't match "directory/file.jpg"
```
The top level of the remote might not be the top level of the drive. The top level of the remote might not be the top level of the drive.
E.g. for a Microsoft Windows local directory structure E.g. for a Microsoft Windows local directory structure
```
F: F:
├── bkp ├── bkp
├── data ├── data
@@ -100,6 +114,7 @@ E.g. for a Microsoft Windows local directory structure
│ │ └── 456.jpg │ │ └── 456.jpg
│ ├── incl │ ├── incl
│ │ └── document.pdf │ │ └── document.pdf
```
To copy the contents of folder `data` into folder `bkp` excluding the contents of subfolder To copy the contents of folder `data` into folder `bkp` excluding the contents of subfolder
`excl`the following command treats `F:\data` and `F:\bkp` as top level for filtering. `excl`the following command treats `F:\data` and `F:\bkp` as top level for filtering.
@@ -113,13 +128,17 @@ Simple patterns are case sensitive unless the `--ignore-case` flag is used.
Without `--ignore-case` (default) Without `--ignore-case` (default)
```
potato - matches "potato" potato - matches "potato"
- doesn't match "POTATO" - doesn't match "POTATO"
```
With `--ignore-case` With `--ignore-case`
```
potato - matches "potato" potato - matches "potato"
- matches "POTATO" - matches "POTATO"
```
## Using regular expressions in filter patterns {#regexp} ## Using regular expressions in filter patterns {#regexp}
@@ -141,26 +160,36 @@ the supplied regular expression(s).
Here is how the `{{regexp}}` is transformed into an full regular Here is how the `{{regexp}}` is transformed into an full regular
expression to match the entire path: expression to match the entire path:
```
{{regexp}} becomes (^|/)(regexp)$ {{regexp}} becomes (^|/)(regexp)$
/{{regexp}} becomes ^(regexp)$ /{{regexp}} becomes ^(regexp)$
```
Regexp syntax can be mixed with glob syntax, for example Regexp syntax can be mixed with glob syntax, for example
```
*.{{jpe?g}} to match file.jpg, file.jpeg but not file.png *.{{jpe?g}} to match file.jpg, file.jpeg but not file.png
```
You can also use regexp flags - to set case insensitive, for example You can also use regexp flags - to set case insensitive, for example
```
*.{{(?i)jpg}} to match file.jpg, file.JPG but not file.png *.{{(?i)jpg}} to match file.jpg, file.JPG but not file.png
```
Be careful with wildcards in regular expressions - you don't want them Be careful with wildcards in regular expressions - you don't want them
to match path separators normally. To match any file name starting to match path separators normally. To match any file name starting
with `start` and ending with `end` write with `start` and ending with `end` write
```
{{start[^/]*end\.jpg}} {{start[^/]*end\.jpg}}
```
Not Not
```
{{start.*end\.jpg}} {{start.*end\.jpg}}
```
Which will match a directory called `start` with a file called Which will match a directory called `start` with a file called
`end.jpg` in it as the `.*` will match `/` characters. `end.jpg` in it as the `.*` will match `/` characters.
@@ -290,11 +319,15 @@ command specify the `--dump filters` flag.
E.g. for an include rule E.g. for an include rule
```
/a/*.jpg /a/*.jpg
```
Rclone implies the directory include rule Rclone implies the directory include rule
```
/a/ /a/
```
Directory filter rules specified in an rclone command can limit Directory filter rules specified in an rclone command can limit
the scope of an rclone command but path/file filters still have the scope of an rclone command but path/file filters still have
@@ -308,10 +341,12 @@ access to the remote by ignoring everything outside of that directory.
E.g. `rclone ls remote: --filter-from filter-list.txt` with a file E.g. `rclone ls remote: --filter-from filter-list.txt` with a file
`filter-list.txt`: `filter-list.txt`:
```
- /dir1/ - /dir1/
- /dir2/ - /dir2/
+ *.pdf + *.pdf
- ** - **
```
All files in directories `dir1` or `dir2` or their subdirectories All files in directories `dir1` or `dir2` or their subdirectories
are completely excluded from the listing. Only files of suffix are completely excluded from the listing. Only files of suffix
@@ -329,7 +364,9 @@ from this pattern list.
E.g. for an include rule E.g. for an include rule
```
{dir1/**,dir2/**} {dir1/**,dir2/**}
```
Rclone will match files below directories `dir1` or `dir2` only, Rclone will match files below directories `dir1` or `dir2` only,
but will not be able to use this filter to exclude a directory `dir3` but will not be able to use this filter to exclude a directory `dir3`
@@ -381,9 +418,11 @@ named file. The file contains a list of remarks and pattern rules.
For an example `exclude-file.txt`: For an example `exclude-file.txt`:
```
# a sample exclude rule file # a sample exclude rule file
*.bak *.bak
file2.jpg file2.jpg
```
`rclone ls remote: --exclude-from exclude-file.txt` lists the files on `rclone ls remote: --exclude-from exclude-file.txt` lists the files on
`remote:` except those named `file2.jpg` or with a suffix `.bak`. That is `remote:` except those named `file2.jpg` or with a suffix `.bak`. That is
@@ -426,12 +465,16 @@ E.g. `rclone ls remote: --include "*.{png,jpg}"` lists the files on
E.g. multiple rclone copy commands can be combined with `--include` and a E.g. multiple rclone copy commands can be combined with `--include` and a
pattern-list. pattern-list.
```
rclone copy /vol1/A remote:A rclone copy /vol1/A remote:A
rclone copy /vol1/B remote:B rclone copy /vol1/B remote:B
```
is equivalent to: is equivalent to:
```
rclone copy /vol1 remote: --include "{A,B}/**" rclone copy /vol1 remote: --include "{A,B}/**"
```
E.g. `rclone ls remote:/wheat --include "??[^[:punct:]]*"` lists the E.g. `rclone ls remote:/wheat --include "??[^[:punct:]]*"` lists the
files `remote:` directory `wheat` (and subdirectories) whose third files `remote:` directory `wheat` (and subdirectories) whose third
@@ -445,9 +488,11 @@ named file. The file contains a list of remarks and pattern rules.
For an example `include-file.txt`: For an example `include-file.txt`:
```
# a sample include rule file # a sample include rule file
*.jpg *.jpg
file2.avi file2.avi
```
`rclone ls remote: --include-from include-file.txt` lists the files on `rclone ls remote: --include-from include-file.txt` lists the files on
`remote:` with name `file2.avi` or suffix `.jpg`. That is equivalent to `remote:` with name `file2.avi` or suffix `.jpg`. That is equivalent to
@@ -509,6 +554,7 @@ Lines starting with # or ; are ignored, and can be used to write comments. Inlin
E.g. for `filter-file.txt`: E.g. for `filter-file.txt`:
```
# a sample filter rule file # a sample filter rule file
- secret*.jpg - secret*.jpg
+ *.jpg + *.jpg
@@ -519,6 +565,7 @@ E.g. for `filter-file.txt`:
+ /dir/** + /dir/**
# exclude everything else # exclude everything else
- * - *
```
`rclone ls remote: --filter-from filter-file.txt` lists the path/files on `rclone ls remote: --filter-from filter-file.txt` lists the path/files on
`remote:` including all `jpg` and `png` files, excluding any `remote:` including all `jpg` and `png` files, excluding any
@@ -528,22 +575,26 @@ everything in the directory `dir` at the root of `remote`, except
E.g. for an alternative `filter-file.txt`: E.g. for an alternative `filter-file.txt`:
```
- secret*.jpg - secret*.jpg
+ *.jpg + *.jpg
+ *.png + *.png
+ file2.avi + file2.avi
- * - *
```
Files `file1.jpg`, `file3.png` and `file2.avi` are listed whilst Files `file1.jpg`, `file3.png` and `file2.avi` are listed whilst
`secret17.jpg` and files without the suffix `.jpg` or `.png` are excluded. `secret17.jpg` and files without the suffix `.jpg` or `.png` are excluded.
E.g. for an alternative `filter-file.txt`: E.g. for an alternative `filter-file.txt`:
```
+ *.jpg + *.jpg
+ *.gif + *.gif
! !
+ 42.doc + 42.doc
- * - *
```
Only file 42.doc is listed. Prior rules are cleared by the `!`. Only file 42.doc is listed. Prior rules are cleared by the `!`.
@@ -586,55 +637,73 @@ you need the input to be processed in a raw manner.
E.g. for a file `files-from.txt`: E.g. for a file `files-from.txt`:
```
# comment # comment
file1.jpg file1.jpg
subdir/file2.jpg subdir/file2.jpg
```
`rclone copy --files-from files-from.txt /home/me/pics remote:pics` `rclone copy --files-from files-from.txt /home/me/pics remote:pics`
copies the following, if they exist, and only those files. copies the following, if they exist, and only those files.
```
/home/me/pics/file1.jpg → remote:pics/file1.jpg /home/me/pics/file1.jpg → remote:pics/file1.jpg
/home/me/pics/subdir/file2.jpg → remote:pics/subdir/file2.jpg /home/me/pics/subdir/file2.jpg → remote:pics/subdir/file2.jpg
```
E.g. to copy the following files referenced by their absolute paths: E.g. to copy the following files referenced by their absolute paths:
```
/home/user1/42 /home/user1/42
/home/user1/dir/ford /home/user1/dir/ford
/home/user2/prefect /home/user2/prefect
```
First find a common subdirectory - in this case `/home` First find a common subdirectory - in this case `/home`
and put the remaining files in `files-from.txt` with or without and put the remaining files in `files-from.txt` with or without
leading `/`, e.g. leading `/`, e.g.
```
user1/42 user1/42
user1/dir/ford user1/dir/ford
user2/prefect user2/prefect
```
Then copy these to a remote: Then copy these to a remote:
```
rclone copy --files-from files-from.txt /home remote:backup rclone copy --files-from files-from.txt /home remote:backup
```
The three files are transferred as follows: The three files are transferred as follows:
```
/home/user1/42 → remote:backup/user1/important /home/user1/42 → remote:backup/user1/important
/home/user1/dir/ford → remote:backup/user1/dir/file /home/user1/dir/ford → remote:backup/user1/dir/file
/home/user2/prefect → remote:backup/user2/stuff /home/user2/prefect → remote:backup/user2/stuff
```
Alternatively if `/` is chosen as root `files-from.txt` will be: Alternatively if `/` is chosen as root `files-from.txt` will be:
```
/home/user1/42 /home/user1/42
/home/user1/dir/ford /home/user1/dir/ford
/home/user2/prefect /home/user2/prefect
```
The copy command will be: The copy command will be:
```
rclone copy --files-from files-from.txt / remote:backup rclone copy --files-from files-from.txt / remote:backup
```
Then there will be an extra `home` directory on the remote: Then there will be an extra `home` directory on the remote:
```
/home/user1/42 → remote:backup/home/user1/42 /home/user1/42 → remote:backup/home/user1/42
/home/user1/dir/ford → remote:backup/home/user1/dir/ford /home/user1/dir/ford → remote:backup/home/user1/dir/ford
/home/user2/prefect → remote:backup/home/user2/prefect /home/user2/prefect → remote:backup/home/user2/prefect
```
### `--files-from-raw` - Read list of source-file names without any processing ### `--files-from-raw` - Read list of source-file names without any processing
@@ -822,7 +891,9 @@ on the destination which are excluded from the command.
E.g. the scope of `rclone sync --interactive A: B:` can be restricted: E.g. the scope of `rclone sync --interactive A: B:` can be restricted:
```
rclone --min-size 50k --delete-excluded sync A: B: rclone --min-size 50k --delete-excluded sync A: B:
```
All files on `B:` which are less than 50 KiB are deleted All files on `B:` which are less than 50 KiB are deleted
because they are excluded from the rclone sync command. because they are excluded from the rclone sync command.
@@ -846,10 +917,12 @@ This flag has a priority over other filter flags.
E.g. for the following directory structure: E.g. for the following directory structure:
```
dir1/file1 dir1/file1
dir1/dir2/file2 dir1/dir2/file2
dir1/dir2/dir3/file3 dir1/dir2/dir3/file3
dir1/dir2/dir3/.ignore dir1/dir2/dir3/.ignore
```
The command `rclone ls --exclude-if-present .ignore dir1` does The command `rclone ls --exclude-if-present .ignore dir1` does
not list `dir3`, `file3` or `.ignore`. not list `dir3`, `file3` or `.ignore`.
@@ -867,11 +940,15 @@ expressions](#regexp).
For example if you wished to list only local files with a mode of For example if you wished to list only local files with a mode of
`100664` you could do that with: `100664` you could do that with:
```
rclone lsf -M --files-only --metadata-include "mode=100664" . rclone lsf -M --files-only --metadata-include "mode=100664" .
```
Or if you wished to show files with an `atime`, `mtime` or `btime` at a given date: Or if you wished to show files with an `atime`, `mtime` or `btime` at a given date:
```
rclone lsf -M --files-only --metadata-include "[abm]time=2022-12-16*" . rclone lsf -M --files-only --metadata-include "[abm]time=2022-12-16*" .
```
Like file filtering, metadata filtering only applies to files not to Like file filtering, metadata filtering only applies to files not to
directories. directories.

View File

@@ -29,11 +29,15 @@ signatures on the release.
To install rclone on Linux/macOS/BSD systems, run: To install rclone on Linux/macOS/BSD systems, run:
```
sudo -v ; curl https://rclone.org/install.sh | sudo bash sudo -v ; curl https://rclone.org/install.sh | sudo bash
```
For beta installation, run: For beta installation, run:
```
sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta
```
Note that this script checks the version of rclone installed first and Note that this script checks the version of rclone installed first and
won't re-download if not needed. won't re-download if not needed.
@@ -44,31 +48,41 @@ won't re-download if not needed.
Fetch and unpack Fetch and unpack
```
curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
unzip rclone-current-linux-amd64.zip unzip rclone-current-linux-amd64.zip
cd rclone-*-linux-amd64 cd rclone-*-linux-amd64
```
Copy binary file Copy binary file
```
sudo cp rclone /usr/bin/ sudo cp rclone /usr/bin/
sudo chown root:root /usr/bin/rclone sudo chown root:root /usr/bin/rclone
sudo chmod 755 /usr/bin/rclone sudo chmod 755 /usr/bin/rclone
```
Install manpage Install manpage
```
sudo mkdir -p /usr/local/share/man/man1 sudo mkdir -p /usr/local/share/man/man1
sudo cp rclone.1 /usr/local/share/man/man1/ sudo cp rclone.1 /usr/local/share/man/man1/
sudo mandb sudo mandb
```
Run `rclone config` to setup. See [rclone config docs](/docs/) for more details. Run `rclone config` to setup. See [rclone config docs](/docs/) for more details.
```
rclone config rclone config
```
## macOS installation {#macos} ## macOS installation {#macos}
### Installation with brew {#macos-brew} ### Installation with brew {#macos-brew}
```
brew install rclone brew install rclone
```
NOTE: This version of rclone will not support `mount` any more (see NOTE: This version of rclone will not support `mount` any more (see
[#5373](https://github.com/rclone/rclone/issues/5373)). If mounting is wanted [#5373](https://github.com/rclone/rclone/issues/5373)). If mounting is wanted
@@ -84,7 +98,9 @@ developers so it may be out of date. Its current version is as below.
On macOS, rclone can also be installed via [MacPorts](https://www.macports.org): On macOS, rclone can also be installed via [MacPorts](https://www.macports.org):
```
sudo port install rclone sudo port install rclone
```
Note that this is a third party installer not controlled by the rclone Note that this is a third party installer not controlled by the rclone
developers so it may be out of date. Its current version is as below. developers so it may be out of date. Its current version is as below.
@@ -100,26 +116,36 @@ notarized it is enough to download with `curl`.
Download the latest version of rclone. Download the latest version of rclone.
```
cd && curl -O https://downloads.rclone.org/rclone-current-osx-amd64.zip cd && curl -O https://downloads.rclone.org/rclone-current-osx-amd64.zip
```
Unzip the download and cd to the extracted folder. Unzip the download and cd to the extracted folder.
```
unzip -a rclone-current-osx-amd64.zip && cd rclone-*-osx-amd64 unzip -a rclone-current-osx-amd64.zip && cd rclone-*-osx-amd64
```
Move rclone to your $PATH. You will be prompted for your password. Move rclone to your $PATH. You will be prompted for your password.
```
sudo mkdir -p /usr/local/bin sudo mkdir -p /usr/local/bin
sudo mv rclone /usr/local/bin/ sudo mv rclone /usr/local/bin/
```
(the `mkdir` command is safe to run, even if the directory already exists). (the `mkdir` command is safe to run, even if the directory already exists).
Remove the leftover files. Remove the leftover files.
```
cd .. && rm -rf rclone-*-osx-amd64 rclone-current-osx-amd64.zip cd .. && rm -rf rclone-*-osx-amd64 rclone-current-osx-amd64.zip
```
Run `rclone config` to setup. See [rclone config docs](/docs/) for more details. Run `rclone config` to setup. See [rclone config docs](/docs/) for more details.
```
rclone config rclone config
```
### Precompiled binary, using a web browser {#macos-precompiled-web} ### Precompiled binary, using a web browser {#macos-precompiled-web}
@@ -127,12 +153,16 @@ When downloading a binary with a web browser, the browser will set the macOS
gatekeeper quarantine attribute. Starting from Catalina, when attempting to run gatekeeper quarantine attribute. Starting from Catalina, when attempting to run
`rclone`, a pop-up will appear saying: `rclone`, a pop-up will appear saying:
```
"rclone" cannot be opened because the developer cannot be verified. "rclone" cannot be opened because the developer cannot be verified.
macOS cannot verify that this app is free from malware. macOS cannot verify that this app is free from malware.
```
The simplest fix is to run The simplest fix is to run
```
xattr -d com.apple.quarantine rclone xattr -d com.apple.quarantine rclone
```
## Windows installation {#windows} ## Windows installation {#windows}

View File

@@ -402,18 +402,24 @@ and to maintain backward compatibility, its behavior has not been changed.
To take a specific example, the FTP backend's default encoding is To take a specific example, the FTP backend's default encoding is
```
--ftp-encoding "Slash,Del,Ctl,RightSpace,Dot" --ftp-encoding "Slash,Del,Ctl,RightSpace,Dot"
```
However, let's say the FTP server is running on Windows and can't have However, let's say the FTP server is running on Windows and can't have
any of the invalid Windows characters in file names. You are backing any of the invalid Windows characters in file names. You are backing
up Linux servers to this FTP server which do have those characters in up Linux servers to this FTP server which do have those characters in
file names. So you would add the Windows set which are file names. So you would add the Windows set which are
```
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
```
to the existing ones, giving: to the existing ones, giving:
```
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot,Del,RightSpace Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot,Del,RightSpace
```
This can be specified using the `--ftp-encoding` flag or using an `encoding` parameter in the config file. This can be specified using the `--ftp-encoding` flag or using an `encoding` parameter in the config file.

View File

@@ -321,21 +321,29 @@ duration of an rc call only then pass in the `_config` parameter.
This should be in the same format as the `main` key returned by This should be in the same format as the `main` key returned by
[options/get](#options-get). [options/get](#options-get).
```
rclone rc --loopback options/get blocks=main rclone rc --loopback options/get blocks=main
```
You can see more help on these options with this command (see [the You can see more help on these options with this command (see [the
options blocks section](#option-blocks) for more info). options blocks section](#option-blocks) for more info).
```
rclone rc --loopback options/info blocks=main rclone rc --loopback options/info blocks=main
```
For example, if you wished to run a sync with the `--checksum` For example, if you wished to run a sync with the `--checksum`
parameter, you would pass this parameter in your JSON blob. parameter, you would pass this parameter in your JSON blob.
```
"_config":{"CheckSum": true} "_config":{"CheckSum": true}
```
If using `rclone rc` this could be passed as If using `rclone rc` this could be passed as
```
rclone rc sync/sync ... _config='{"CheckSum": true}' rclone rc sync/sync ... _config='{"CheckSum": true}'
```
Any config parameters you don't set will inherit the global defaults Any config parameters you don't set will inherit the global defaults
which were set with command line flags or environment variables. which were set with command line flags or environment variables.
@@ -344,8 +352,10 @@ Note that it is possible to set some values as strings or integers -
see [data types](#data-types) for more info. Here is an example see [data types](#data-types) for more info. Here is an example
setting the equivalent of `--buffer-size` in string or integer format. setting the equivalent of `--buffer-size` in string or integer format.
```
"_config":{"BufferSize": "42M"} "_config":{"BufferSize": "42M"}
"_config":{"BufferSize": 44040192} "_config":{"BufferSize": 44040192}
```
If you wish to check the `_config` assignment has worked properly then If you wish to check the `_config` assignment has worked properly then
calling `options/local` will show what the value got set to. calling `options/local` will show what the value got set to.
@@ -358,24 +368,34 @@ pass in the `_filter` parameter.
This should be in the same format as the `filter` key returned by This should be in the same format as the `filter` key returned by
[options/get](#options-get). [options/get](#options-get).
```
rclone rc --loopback options/get blocks=filter rclone rc --loopback options/get blocks=filter
```
You can see more help on these options with this command (see [the You can see more help on these options with this command (see [the
options blocks section](#option-blocks) for more info). options blocks section](#option-blocks) for more info).
```
rclone rc --loopback options/info blocks=filter rclone rc --loopback options/info blocks=filter
```
For example, if you wished to run a sync with these flags For example, if you wished to run a sync with these flags
```
--max-size 1M --max-age 42s --include "a" --include "b" --max-size 1M --max-age 42s --include "a" --include "b"
```
you would pass this parameter in your JSON blob. you would pass this parameter in your JSON blob.
```
"_filter":{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"} "_filter":{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}
```
If using `rclone rc` this could be passed as If using `rclone rc` this could be passed as
```
rclone rc ... _filter='{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}' rclone rc ... _filter='{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}'
```
Any filter parameters you don't set will inherit the global defaults Any filter parameters you don't set will inherit the global defaults
which were set with command line flags or environment variables. which were set with command line flags or environment variables.
@@ -384,8 +404,10 @@ Note that it is possible to set some values as strings or integers -
see [data types](#data-types) for more info. Here is an example see [data types](#data-types) for more info. Here is an example
setting the equivalent of `--buffer-size` in string or integer format. setting the equivalent of `--buffer-size` in string or integer format.
```
"_filter":{"MinSize": "42M"} "_filter":{"MinSize": "42M"}
"_filter":{"MinSize": 44040192} "_filter":{"MinSize": 44040192}
```
If you wish to check the `_filter` assignment has worked properly then If you wish to check the `_filter` assignment has worked properly then
calling `options/local` will show what the value got set to. calling `options/local` will show what the value got set to.
@@ -2490,7 +2512,9 @@ To use these, first [install go](https://golang.org/doc/install).
To profile rclone's memory use you can run: To profile rclone's memory use you can run:
```
go tool pprof -web http://localhost:5572/debug/pprof/heap go tool pprof -web http://localhost:5572/debug/pprof/heap
```
This should open a page in your browser showing what is using what This should open a page in your browser showing what is using what
memory. memory.
@@ -2522,7 +2546,9 @@ alive which should have been garbage collected.
See all active go routines using See all active go routines using
```
curl http://localhost:5572/debug/pprof/goroutine?debug=1 curl http://localhost:5572/debug/pprof/goroutine?debug=1
```
Or go to http://localhost:5572/debug/pprof/goroutine?debug=1 in your browser. Or go to http://localhost:5572/debug/pprof/goroutine?debug=1 in your browser.