mirror of
https://github.com/rclone/rclone.git
synced 2025-12-15 15:53:41 +00:00
docs: fix markdownlint issue md046/code-block-style
This commit is contained in:
@@ -32,6 +32,7 @@ Then [install Git](https://git-scm.com/downloads) and set your public contributi
|
|||||||
|
|
||||||
Next open your terminal, change directory to your preferred folder and initialise your local rclone project:
|
Next open your terminal, change directory to your preferred folder and initialise your local rclone project:
|
||||||
|
|
||||||
|
```
|
||||||
git clone https://github.com/rclone/rclone.git
|
git clone https://github.com/rclone/rclone.git
|
||||||
cd rclone
|
cd rclone
|
||||||
git remote rename origin upstream
|
git remote rename origin upstream
|
||||||
@@ -39,23 +40,30 @@ Next open your terminal, change directory to your preferred folder and initialis
|
|||||||
git remote add origin git@github.com:YOURUSER/rclone.git
|
git remote add origin git@github.com:YOURUSER/rclone.git
|
||||||
# otherwise:
|
# otherwise:
|
||||||
git remote add origin https://github.com/YOURUSER/rclone.git
|
git remote add origin https://github.com/YOURUSER/rclone.git
|
||||||
|
```
|
||||||
|
|
||||||
Note that most of the terminal commands in the rest of this guide must be executed from the rclone folder created above.
|
Note that most of the terminal commands in the rest of this guide must be executed from the rclone folder created above.
|
||||||
|
|
||||||
Now [install Go](https://golang.org/doc/install) and verify your installation:
|
Now [install Go](https://golang.org/doc/install) and verify your installation:
|
||||||
|
|
||||||
|
```
|
||||||
go version
|
go version
|
||||||
|
```
|
||||||
|
|
||||||
Great, you can now compile and execute your own version of rclone:
|
Great, you can now compile and execute your own version of rclone:
|
||||||
|
|
||||||
|
```
|
||||||
go build
|
go build
|
||||||
./rclone version
|
./rclone version
|
||||||
|
```
|
||||||
|
|
||||||
(Note that you can also replace `go build` with `make`, which will include a
|
(Note that you can also replace `go build` with `make`, which will include a
|
||||||
more accurate version number in the executable as well as enable you to specify
|
more accurate version number in the executable as well as enable you to specify
|
||||||
more build options.) Finally make a branch to add your new feature
|
more build options.) Finally make a branch to add your new feature
|
||||||
|
|
||||||
|
```
|
||||||
git checkout -b my-new-feature
|
git checkout -b my-new-feature
|
||||||
|
```
|
||||||
|
|
||||||
And get hacking.
|
And get hacking.
|
||||||
|
|
||||||
@@ -63,8 +71,10 @@ You may like one of the [popular editors/IDE's for Go](https://github.com/golang
|
|||||||
|
|
||||||
When ready - test the affected functionality and run the unit tests for the code you changed
|
When ready - test the affected functionality and run the unit tests for the code you changed
|
||||||
|
|
||||||
|
```
|
||||||
cd folder/with/changed/files
|
cd folder/with/changed/files
|
||||||
go test -v
|
go test -v
|
||||||
|
```
|
||||||
|
|
||||||
Note that you may need to make a test remote, e.g. `TestSwift` for some
|
Note that you may need to make a test remote, e.g. `TestSwift` for some
|
||||||
of the unit tests.
|
of the unit tests.
|
||||||
@@ -79,7 +89,9 @@ Make sure you
|
|||||||
|
|
||||||
When you are done with that push your changes to GitHub:
|
When you are done with that push your changes to GitHub:
|
||||||
|
|
||||||
|
```
|
||||||
git push -u origin my-new-feature
|
git push -u origin my-new-feature
|
||||||
|
```
|
||||||
|
|
||||||
and open the GitHub website to [create your pull
|
and open the GitHub website to [create your pull
|
||||||
request](https://help.github.com/articles/creating-a-pull-request/).
|
request](https://help.github.com/articles/creating-a-pull-request/).
|
||||||
@@ -94,16 +106,20 @@ You may sometimes be asked to [base your changes on the latest master](#basing-y
|
|||||||
|
|
||||||
Follow the guideline for [commit messages](#commit-messages) and then:
|
Follow the guideline for [commit messages](#commit-messages) and then:
|
||||||
|
|
||||||
|
```
|
||||||
git checkout my-new-feature # To switch to your branch
|
git checkout my-new-feature # To switch to your branch
|
||||||
git status # To see the new and changed files
|
git status # To see the new and changed files
|
||||||
git add FILENAME # To select FILENAME for the commit
|
git add FILENAME # To select FILENAME for the commit
|
||||||
git status # To verify the changes to be committed
|
git status # To verify the changes to be committed
|
||||||
git commit # To do the commit
|
git commit # To do the commit
|
||||||
git log # To verify the commit. Use q to quit the log
|
git log # To verify the commit. Use q to quit the log
|
||||||
|
```
|
||||||
|
|
||||||
You can modify the message or changes in the latest commit using:
|
You can modify the message or changes in the latest commit using:
|
||||||
|
|
||||||
|
```
|
||||||
git commit --amend
|
git commit --amend
|
||||||
|
```
|
||||||
|
|
||||||
If you amend to commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits).
|
If you amend to commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits).
|
||||||
|
|
||||||
@@ -113,18 +129,22 @@ Note that you are about to rewrite the GitHub history of your branch. It is good
|
|||||||
|
|
||||||
Your previously pushed commits are replaced by:
|
Your previously pushed commits are replaced by:
|
||||||
|
|
||||||
|
```
|
||||||
git push --force origin my-new-feature
|
git push --force origin my-new-feature
|
||||||
|
```
|
||||||
|
|
||||||
### Basing your changes on the latest master
|
### Basing your changes on the latest master
|
||||||
|
|
||||||
To base your changes on the latest version of the [rclone master](https://github.com/rclone/rclone/tree/master) (upstream):
|
To base your changes on the latest version of the [rclone master](https://github.com/rclone/rclone/tree/master) (upstream):
|
||||||
|
|
||||||
|
```
|
||||||
git checkout master
|
git checkout master
|
||||||
git fetch upstream
|
git fetch upstream
|
||||||
git merge --ff-only
|
git merge --ff-only
|
||||||
git push origin --follow-tags # optional update of your fork in GitHub
|
git push origin --follow-tags # optional update of your fork in GitHub
|
||||||
git checkout my-new-feature
|
git checkout my-new-feature
|
||||||
git rebase master
|
git rebase master
|
||||||
|
```
|
||||||
|
|
||||||
If you rebase commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits).
|
If you rebase commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits).
|
||||||
|
|
||||||
@@ -132,18 +152,24 @@ If you rebase commits that have been pushed to GitHub, then you will have to [re
|
|||||||
|
|
||||||
To combine your commits into one commit:
|
To combine your commits into one commit:
|
||||||
|
|
||||||
|
```
|
||||||
git log # To count the commits to squash, e.g. the last 2
|
git log # To count the commits to squash, e.g. the last 2
|
||||||
git reset --soft HEAD~2 # To undo the 2 latest commits
|
git reset --soft HEAD~2 # To undo the 2 latest commits
|
||||||
git status # To check everything is as expected
|
git status # To check everything is as expected
|
||||||
|
```
|
||||||
|
|
||||||
If everything is fine, then make the new combined commit:
|
If everything is fine, then make the new combined commit:
|
||||||
|
|
||||||
|
```
|
||||||
git commit # To commit the undone commits as one
|
git commit # To commit the undone commits as one
|
||||||
|
```
|
||||||
|
|
||||||
otherwise, you may roll back using:
|
otherwise, you may roll back using:
|
||||||
|
|
||||||
|
```
|
||||||
git reflog # To check that HEAD{1} is your previous state
|
git reflog # To check that HEAD{1} is your previous state
|
||||||
git reset --soft 'HEAD@{1}' # To roll back to your previous state
|
git reset --soft 'HEAD@{1}' # To roll back to your previous state
|
||||||
|
```
|
||||||
|
|
||||||
If you squash commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits).
|
If you squash commits that have been pushed to GitHub, then you will have to [replace your previously pushed commits](#replacing-your-previously-pushed-commits).
|
||||||
|
|
||||||
@@ -168,11 +194,15 @@ Using these tests ensures that the rclone codebase all uses the same coding stan
|
|||||||
rclone's tests are run from the go testing framework, so at the top
|
rclone's tests are run from the go testing framework, so at the top
|
||||||
level you can run this to run all the tests.
|
level you can run this to run all the tests.
|
||||||
|
|
||||||
|
```
|
||||||
go test -v ./...
|
go test -v ./...
|
||||||
|
```
|
||||||
|
|
||||||
You can also use `make`, if supported by your platform
|
You can also use `make`, if supported by your platform
|
||||||
|
|
||||||
|
```
|
||||||
make quicktest
|
make quicktest
|
||||||
|
```
|
||||||
|
|
||||||
The quicktest is [automatically run by GitHub](#github-continuous-integration) when you push your branch to GitHub.
|
The quicktest is [automatically run by GitHub](#github-continuous-integration) when you push your branch to GitHub.
|
||||||
|
|
||||||
@@ -190,38 +220,48 @@ need to make a remote called `TestDrive`.
|
|||||||
You can then run the unit tests in the drive directory. These tests
|
You can then run the unit tests in the drive directory. These tests
|
||||||
are skipped if `TestDrive:` isn't defined.
|
are skipped if `TestDrive:` isn't defined.
|
||||||
|
|
||||||
|
```
|
||||||
cd backend/drive
|
cd backend/drive
|
||||||
go test -v
|
go test -v
|
||||||
|
```
|
||||||
|
|
||||||
You can then run the integration tests which test all of rclone's
|
You can then run the integration tests which test all of rclone's
|
||||||
operations. Normally these get run against the local file system,
|
operations. Normally these get run against the local file system,
|
||||||
but they can be run against any of the remotes.
|
but they can be run against any of the remotes.
|
||||||
|
|
||||||
|
```
|
||||||
cd fs/sync
|
cd fs/sync
|
||||||
go test -v -remote TestDrive:
|
go test -v -remote TestDrive:
|
||||||
go test -v -remote TestDrive: -fast-list
|
go test -v -remote TestDrive: -fast-list
|
||||||
|
|
||||||
cd fs/operations
|
cd fs/operations
|
||||||
go test -v -remote TestDrive:
|
go test -v -remote TestDrive:
|
||||||
|
```
|
||||||
|
|
||||||
If you want to use the integration test framework to run these tests
|
If you want to use the integration test framework to run these tests
|
||||||
altogether with an HTML report and test retries then from the
|
altogether with an HTML report and test retries then from the
|
||||||
project root:
|
project root:
|
||||||
|
|
||||||
|
```
|
||||||
go install github.com/rclone/rclone/fstest/test_all
|
go install github.com/rclone/rclone/fstest/test_all
|
||||||
test_all -backends drive
|
test_all -backends drive
|
||||||
|
```
|
||||||
|
|
||||||
### Full integration testing
|
### Full integration testing
|
||||||
|
|
||||||
If you want to run all the integration tests against all the remotes,
|
If you want to run all the integration tests against all the remotes,
|
||||||
then change into the project root and run
|
then change into the project root and run
|
||||||
|
|
||||||
|
```
|
||||||
make check
|
make check
|
||||||
make test
|
make test
|
||||||
|
```
|
||||||
|
|
||||||
The commands may require some extra go packages which you can install with
|
The commands may require some extra go packages which you can install with
|
||||||
|
|
||||||
|
```
|
||||||
make build_dep
|
make build_dep
|
||||||
|
```
|
||||||
|
|
||||||
The full integration tests are run daily on the integration test server. You can
|
The full integration tests are run daily on the integration test server. You can
|
||||||
find the results at https://pub.rclone.org/integration-tests/
|
find the results at https://pub.rclone.org/integration-tests/
|
||||||
@@ -379,7 +419,9 @@ To add a dependency `github.com/ncw/new_dependency` see the
|
|||||||
instructions below. These will fetch the dependency and add it to
|
instructions below. These will fetch the dependency and add it to
|
||||||
`go.mod` and `go.sum`.
|
`go.mod` and `go.sum`.
|
||||||
|
|
||||||
|
```
|
||||||
go get github.com/ncw/new_dependency
|
go get github.com/ncw/new_dependency
|
||||||
|
```
|
||||||
|
|
||||||
You can add constraints on that package when doing `go get` (see the
|
You can add constraints on that package when doing `go get` (see the
|
||||||
go docs linked above), but don't unless you really need to.
|
go docs linked above), but don't unless you really need to.
|
||||||
@@ -391,7 +433,9 @@ and `go.sum` in the same commit as your other changes.
|
|||||||
|
|
||||||
If you need to update a dependency then run
|
If you need to update a dependency then run
|
||||||
|
|
||||||
|
```
|
||||||
go get golang.org/x/crypto
|
go get golang.org/x/crypto
|
||||||
|
```
|
||||||
|
|
||||||
Check in a single commit as above.
|
Check in a single commit as above.
|
||||||
|
|
||||||
|
|||||||
14
RELEASE.md
14
RELEASE.md
@@ -48,7 +48,9 @@ Early in the next release cycle update the dependencies.
|
|||||||
|
|
||||||
If the `make updatedirect` upgrades the version of go in the `go.mod`
|
If the `make updatedirect` upgrades the version of go in the `go.mod`
|
||||||
|
|
||||||
|
```
|
||||||
go 1.22.0
|
go 1.22.0
|
||||||
|
```
|
||||||
|
|
||||||
then go to manual mode. `go1.22` here is the lowest supported version
|
then go to manual mode. `go1.22` here is the lowest supported version
|
||||||
in the `go.mod`.
|
in the `go.mod`.
|
||||||
@@ -99,7 +101,9 @@ The above procedure will not upgrade major versions, so v2 to v3.
|
|||||||
However this tool can show which major versions might need to be
|
However this tool can show which major versions might need to be
|
||||||
upgraded:
|
upgraded:
|
||||||
|
|
||||||
|
```
|
||||||
go run github.com/icholy/gomajor@latest list -major
|
go run github.com/icholy/gomajor@latest list -major
|
||||||
|
```
|
||||||
|
|
||||||
Expect API breakage when updating major versions.
|
Expect API breakage when updating major versions.
|
||||||
|
|
||||||
@@ -107,7 +111,9 @@ Expect API breakage when updating major versions.
|
|||||||
|
|
||||||
At some point after the release run
|
At some point after the release run
|
||||||
|
|
||||||
|
```
|
||||||
bin/tidy-beta v1.55
|
bin/tidy-beta v1.55
|
||||||
|
```
|
||||||
|
|
||||||
where the version number is that of a couple ago to remove old beta binaries.
|
where the version number is that of a couple ago to remove old beta binaries.
|
||||||
|
|
||||||
@@ -150,21 +156,29 @@ which is a private repo containing artwork from sponsors.
|
|||||||
|
|
||||||
Create an update website branch based off the last release
|
Create an update website branch based off the last release
|
||||||
|
|
||||||
|
```
|
||||||
git co -b update-website
|
git co -b update-website
|
||||||
|
```
|
||||||
|
|
||||||
If the branch already exists, double check there are no commits that need saving.
|
If the branch already exists, double check there are no commits that need saving.
|
||||||
|
|
||||||
Now reset the branch to the last release
|
Now reset the branch to the last release
|
||||||
|
|
||||||
|
```
|
||||||
git reset --hard v1.64.0
|
git reset --hard v1.64.0
|
||||||
|
```
|
||||||
|
|
||||||
Create the changes, check them in, test with `make serve` then
|
Create the changes, check them in, test with `make serve` then
|
||||||
|
|
||||||
|
```
|
||||||
make upload_test_website
|
make upload_test_website
|
||||||
|
```
|
||||||
|
|
||||||
Check out https://test.rclone.org and when happy
|
Check out https://test.rclone.org and when happy
|
||||||
|
|
||||||
|
```
|
||||||
make upload_website
|
make upload_website
|
||||||
|
```
|
||||||
|
|
||||||
Cherry pick any changes back to master and the stable branch if it is active.
|
Cherry pick any changes back to master and the stable branch if it is active.
|
||||||
|
|
||||||
|
|||||||
@@ -22,7 +22,9 @@ file and choose its location.)
|
|||||||
The easiest way to make the config is to run rclone with the config
|
The easiest way to make the config is to run rclone with the config
|
||||||
option:
|
option:
|
||||||
|
|
||||||
|
```
|
||||||
rclone config
|
rclone config
|
||||||
|
```
|
||||||
|
|
||||||
See the following for detailed instructions for
|
See the following for detailed instructions for
|
||||||
|
|
||||||
@@ -97,7 +99,9 @@ Rclone syncs a directory tree from one storage system to another.
|
|||||||
|
|
||||||
Its syntax is like this
|
Its syntax is like this
|
||||||
|
|
||||||
|
```
|
||||||
rclone subcommand [options] <parameters> <parameters...>
|
rclone subcommand [options] <parameters> <parameters...>
|
||||||
|
```
|
||||||
|
|
||||||
A `subcommand` is a the rclone operation required, (e.g. `sync`,
|
A `subcommand` is a the rclone operation required, (e.g. `sync`,
|
||||||
`copy`, `ls`).
|
`copy`, `ls`).
|
||||||
@@ -110,7 +114,9 @@ used before the `subcommand`. Anything after a `--` option will not be
|
|||||||
interpreted as an option so if you need to add a parameter which
|
interpreted as an option so if you need to add a parameter which
|
||||||
starts with a `-` then put a `--` on its own first, eg
|
starts with a `-` then put a `--` on its own first, eg
|
||||||
|
|
||||||
|
```
|
||||||
rclone lsf -- -directory-starting-with-dash
|
rclone lsf -- -directory-starting-with-dash
|
||||||
|
```
|
||||||
|
|
||||||
A `parameter` is usually a file path or [rclone remote](#syntax-of-remote-paths), eg
|
A `parameter` is usually a file path or [rclone remote](#syntax-of-remote-paths), eg
|
||||||
`/path/to/file` or `remote:path/to/file` but it can be other things -
|
`/path/to/file` or `remote:path/to/file` but it can be other things -
|
||||||
@@ -129,9 +135,11 @@ learning rclone to avoid accidental data loss.
|
|||||||
|
|
||||||
rclone uses a system of subcommands. For example
|
rclone uses a system of subcommands. For example
|
||||||
|
|
||||||
|
```
|
||||||
rclone ls remote:path # lists a remote
|
rclone ls remote:path # lists a remote
|
||||||
rclone copy /local/path remote:path # copies /local/path to the remote
|
rclone copy /local/path remote:path # copies /local/path to the remote
|
||||||
rclone sync --interactive /local/path remote:path # syncs /local/path to the remote
|
rclone sync --interactive /local/path remote:path # syncs /local/path to the remote
|
||||||
|
```
|
||||||
|
|
||||||
The main rclone commands with most used first
|
The main rclone commands with most used first
|
||||||
|
|
||||||
@@ -180,17 +188,23 @@ directory` if it isn't.
|
|||||||
For example, suppose you have a remote with a file in called
|
For example, suppose you have a remote with a file in called
|
||||||
`test.jpg`, then you could copy just that file like this
|
`test.jpg`, then you could copy just that file like this
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy remote:test.jpg /tmp/download
|
rclone copy remote:test.jpg /tmp/download
|
||||||
|
```
|
||||||
|
|
||||||
The file `test.jpg` will be placed inside `/tmp/download`.
|
The file `test.jpg` will be placed inside `/tmp/download`.
|
||||||
|
|
||||||
This is equivalent to specifying
|
This is equivalent to specifying
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy --files-from /tmp/files remote: /tmp/download
|
rclone copy --files-from /tmp/files remote: /tmp/download
|
||||||
|
```
|
||||||
|
|
||||||
Where `/tmp/files` contains the single line
|
Where `/tmp/files` contains the single line
|
||||||
|
|
||||||
|
```
|
||||||
test.jpg
|
test.jpg
|
||||||
|
```
|
||||||
|
|
||||||
It is recommended to use `copy` when copying individual files, not `sync`.
|
It is recommended to use `copy` when copying individual files, not `sync`.
|
||||||
They have pretty much the same effect but `copy` will use a lot less
|
They have pretty much the same effect but `copy` will use a lot less
|
||||||
@@ -234,19 +248,27 @@ the command line (or in environment variables).
|
|||||||
|
|
||||||
Here are some examples:
|
Here are some examples:
|
||||||
|
|
||||||
|
```
|
||||||
rclone lsd --http-url https://pub.rclone.org :http:
|
rclone lsd --http-url https://pub.rclone.org :http:
|
||||||
|
```
|
||||||
|
|
||||||
To list all the directories in the root of `https://pub.rclone.org/`.
|
To list all the directories in the root of `https://pub.rclone.org/`.
|
||||||
|
|
||||||
|
```
|
||||||
rclone lsf --http-url https://example.com :http:path/to/dir
|
rclone lsf --http-url https://example.com :http:path/to/dir
|
||||||
|
```
|
||||||
|
|
||||||
To list files and directories in `https://example.com/path/to/dir/`
|
To list files and directories in `https://example.com/path/to/dir/`
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy --http-url https://example.com :http:path/to/dir /tmp/dir
|
rclone copy --http-url https://example.com :http:path/to/dir /tmp/dir
|
||||||
|
```
|
||||||
|
|
||||||
To copy files and directories in `https://example.com/path/to/dir` to `/tmp/dir`.
|
To copy files and directories in `https://example.com/path/to/dir` to `/tmp/dir`.
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy --sftp-host example.com :sftp:path/to/dir /tmp/dir
|
rclone copy --sftp-host example.com :sftp:path/to/dir /tmp/dir
|
||||||
|
```
|
||||||
|
|
||||||
To copy files and directories from `example.com` in the relative
|
To copy files and directories from `example.com` in the relative
|
||||||
directory `path/to/dir` to `/tmp/dir` using sftp.
|
directory `path/to/dir` to `/tmp/dir` using sftp.
|
||||||
@@ -258,16 +280,20 @@ syntax, so instead of providing the arguments as command line
|
|||||||
parameters `--http-url https://pub.rclone.org` they are provided as
|
parameters `--http-url https://pub.rclone.org` they are provided as
|
||||||
part of the remote specification as a kind of connection string.
|
part of the remote specification as a kind of connection string.
|
||||||
|
|
||||||
|
```
|
||||||
rclone lsd ":http,url='https://pub.rclone.org':"
|
rclone lsd ":http,url='https://pub.rclone.org':"
|
||||||
rclone lsf ":http,url='https://example.com':path/to/dir"
|
rclone lsf ":http,url='https://example.com':path/to/dir"
|
||||||
rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
|
rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
|
||||||
rclone copy :sftp,host=example.com:path/to/dir /tmp/dir
|
rclone copy :sftp,host=example.com:path/to/dir /tmp/dir
|
||||||
|
```
|
||||||
|
|
||||||
These can apply to modify existing remotes as well as create new
|
These can apply to modify existing remotes as well as create new
|
||||||
remotes with the on the fly syntax. This example is equivalent to
|
remotes with the on the fly syntax. This example is equivalent to
|
||||||
adding the `--drive-shared-with-me` parameter to the remote `gdrive:`.
|
adding the `--drive-shared-with-me` parameter to the remote `gdrive:`.
|
||||||
|
|
||||||
|
```
|
||||||
rclone lsf "gdrive,shared_with_me:path/to/dir"
|
rclone lsf "gdrive,shared_with_me:path/to/dir"
|
||||||
|
```
|
||||||
|
|
||||||
The major advantage to using the connection string style syntax is
|
The major advantage to using the connection string style syntax is
|
||||||
that it only applies to the remote, not to all the remotes of that
|
that it only applies to the remote, not to all the remotes of that
|
||||||
@@ -276,34 +302,46 @@ file shared on google drive to the normal drive which **does not
|
|||||||
work** because the `--drive-shared-with-me` flag applies to both the
|
work** because the `--drive-shared-with-me` flag applies to both the
|
||||||
source and the destination.
|
source and the destination.
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy --drive-shared-with-me gdrive:shared-file.txt gdrive:
|
rclone copy --drive-shared-with-me gdrive:shared-file.txt gdrive:
|
||||||
|
```
|
||||||
|
|
||||||
However using the connection string syntax, this does work.
|
However using the connection string syntax, this does work.
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy "gdrive,shared_with_me:shared-file.txt" gdrive:
|
rclone copy "gdrive,shared_with_me:shared-file.txt" gdrive:
|
||||||
|
```
|
||||||
|
|
||||||
Note that the connection string only affects the options of the immediate
|
Note that the connection string only affects the options of the immediate
|
||||||
backend. If for example gdriveCrypt is a crypt based on gdrive, then the
|
backend. If for example gdriveCrypt is a crypt based on gdrive, then the
|
||||||
following command **will not work** as intended, because
|
following command **will not work** as intended, because
|
||||||
`shared_with_me` is ignored by the crypt backend:
|
`shared_with_me` is ignored by the crypt backend:
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy "gdriveCrypt,shared_with_me:shared-file.txt" gdriveCrypt:
|
rclone copy "gdriveCrypt,shared_with_me:shared-file.txt" gdriveCrypt:
|
||||||
|
```
|
||||||
|
|
||||||
The connection strings have the following syntax
|
The connection strings have the following syntax
|
||||||
|
|
||||||
|
```
|
||||||
remote,parameter=value,parameter2=value2:path/to/dir
|
remote,parameter=value,parameter2=value2:path/to/dir
|
||||||
:backend,parameter=value,parameter2=value2:path/to/dir
|
:backend,parameter=value,parameter2=value2:path/to/dir
|
||||||
|
```
|
||||||
|
|
||||||
If the `parameter` has a `:` or `,` then it must be placed in quotes `"` or
|
If the `parameter` has a `:` or `,` then it must be placed in quotes `"` or
|
||||||
`'`, so
|
`'`, so
|
||||||
|
|
||||||
|
```
|
||||||
remote,parameter="colon:value",parameter2="comma,value":path/to/dir
|
remote,parameter="colon:value",parameter2="comma,value":path/to/dir
|
||||||
:backend,parameter='colon:value',parameter2='comma,value':path/to/dir
|
:backend,parameter='colon:value',parameter2='comma,value':path/to/dir
|
||||||
|
```
|
||||||
|
|
||||||
If a quoted value needs to include that quote, then it should be
|
If a quoted value needs to include that quote, then it should be
|
||||||
doubled, so
|
doubled, so
|
||||||
|
|
||||||
|
```
|
||||||
remote,parameter="with""quote",parameter2='with''quote':path/to/dir
|
remote,parameter="with""quote",parameter2='with''quote':path/to/dir
|
||||||
|
```
|
||||||
|
|
||||||
This will make `parameter` be `with"quote` and `parameter2` be
|
This will make `parameter` be `with"quote` and `parameter2` be
|
||||||
`with'quote`.
|
`with'quote`.
|
||||||
@@ -312,11 +350,15 @@ If you leave off the `=parameter` then rclone will substitute `=true`
|
|||||||
which works very well with flags. For example, to use s3 configured in
|
which works very well with flags. For example, to use s3 configured in
|
||||||
the environment you could use:
|
the environment you could use:
|
||||||
|
|
||||||
|
```
|
||||||
rclone lsd :s3,env_auth:
|
rclone lsd :s3,env_auth:
|
||||||
|
```
|
||||||
|
|
||||||
Which is equivalent to
|
Which is equivalent to
|
||||||
|
|
||||||
|
```
|
||||||
rclone lsd :s3,env_auth=true:
|
rclone lsd :s3,env_auth=true:
|
||||||
|
```
|
||||||
|
|
||||||
Note that on the command line you might need to surround these
|
Note that on the command line you might need to surround these
|
||||||
connection strings with `"` or `'` to stop the shell interpreting any
|
connection strings with `"` or `'` to stop the shell interpreting any
|
||||||
@@ -326,14 +368,18 @@ If you are a shell master then you'll know which strings are OK and
|
|||||||
which aren't, but if you aren't sure then enclose them in `"` and use
|
which aren't, but if you aren't sure then enclose them in `"` and use
|
||||||
`'` as the inside quote. This syntax works on all OSes.
|
`'` as the inside quote. This syntax works on all OSes.
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
|
rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
|
||||||
|
```
|
||||||
|
|
||||||
On Linux/macOS some characters are still interpreted inside `"`
|
On Linux/macOS some characters are still interpreted inside `"`
|
||||||
strings in the shell (notably `\` and `$` and `"`) so if your strings
|
strings in the shell (notably `\` and `$` and `"`) so if your strings
|
||||||
contain those you can swap the roles of `"` and `'` thus. (This syntax
|
contain those you can swap the roles of `"` and `'` thus. (This syntax
|
||||||
does not work on Windows.)
|
does not work on Windows.)
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy ':http,url="https://example.com":path/to/dir' /tmp/dir
|
rclone copy ':http,url="https://example.com":path/to/dir' /tmp/dir
|
||||||
|
```
|
||||||
|
|
||||||
#### Connection strings, config and logging
|
#### Connection strings, config and logging
|
||||||
|
|
||||||
@@ -341,11 +387,15 @@ If you supply extra configuration to a backend by command line flag,
|
|||||||
environment variable or connection string then rclone will add a
|
environment variable or connection string then rclone will add a
|
||||||
suffix based on the hash of the config to the name of the remote, eg
|
suffix based on the hash of the config to the name of the remote, eg
|
||||||
|
|
||||||
|
```
|
||||||
rclone -vv lsf --s3-chunk-size 20M s3:
|
rclone -vv lsf --s3-chunk-size 20M s3:
|
||||||
|
```
|
||||||
|
|
||||||
Has the log message
|
Has the log message
|
||||||
|
|
||||||
|
```
|
||||||
DEBUG : s3: detected overridden config - adding "{Srj1p}" suffix to name
|
DEBUG : s3: detected overridden config - adding "{Srj1p}" suffix to name
|
||||||
|
```
|
||||||
|
|
||||||
This is so rclone can tell the modified remote apart from the
|
This is so rclone can tell the modified remote apart from the
|
||||||
unmodified remote when caching the backends.
|
unmodified remote when caching the backends.
|
||||||
@@ -354,11 +404,15 @@ This should only be noticeable in the logs.
|
|||||||
|
|
||||||
This means that on the fly backends such as
|
This means that on the fly backends such as
|
||||||
|
|
||||||
|
```
|
||||||
rclone -vv lsf :s3,env_auth:
|
rclone -vv lsf :s3,env_auth:
|
||||||
|
```
|
||||||
|
|
||||||
Will get their own names
|
Will get their own names
|
||||||
|
|
||||||
|
```
|
||||||
DEBUG : :s3: detected overridden config - adding "{YTu53}" suffix to name
|
DEBUG : :s3: detected overridden config - adding "{YTu53}" suffix to name
|
||||||
|
```
|
||||||
|
|
||||||
### Valid remote names
|
### Valid remote names
|
||||||
|
|
||||||
@@ -393,11 +447,15 @@ Here are some gotchas which may help users unfamiliar with the shell rules
|
|||||||
If your names have spaces or shell metacharacters (e.g. `*`, `?`, `$`,
|
If your names have spaces or shell metacharacters (e.g. `*`, `?`, `$`,
|
||||||
`'`, `"`, etc.) then you must quote them. Use single quotes `'` by default.
|
`'`, `"`, etc.) then you must quote them. Use single quotes `'` by default.
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy 'Important files?' remote:backup
|
rclone copy 'Important files?' remote:backup
|
||||||
|
```
|
||||||
|
|
||||||
If you want to send a `'` you will need to use `"`, e.g.
|
If you want to send a `'` you will need to use `"`, e.g.
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy "O'Reilly Reviews" remote:backup
|
rclone copy "O'Reilly Reviews" remote:backup
|
||||||
|
```
|
||||||
|
|
||||||
The rules for quoting metacharacters are complicated and if you want
|
The rules for quoting metacharacters are complicated and if you want
|
||||||
the full details you'll have to consult the manual page for your
|
the full details you'll have to consult the manual page for your
|
||||||
@@ -407,12 +465,16 @@ shell.
|
|||||||
|
|
||||||
If your names have spaces in you need to put them in `"`, e.g.
|
If your names have spaces in you need to put them in `"`, e.g.
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy "E:\folder name\folder name\folder name" remote:backup
|
rclone copy "E:\folder name\folder name\folder name" remote:backup
|
||||||
|
```
|
||||||
|
|
||||||
If you are using the root directory on its own then don't quote it
|
If you are using the root directory on its own then don't quote it
|
||||||
(see [#464](https://github.com/rclone/rclone/issues/464) for why), e.g.
|
(see [#464](https://github.com/rclone/rclone/issues/464) for why), e.g.
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy E:\ remote:backup
|
rclone copy E:\ remote:backup
|
||||||
|
```
|
||||||
|
|
||||||
## Copying files or directories with `:` in the names
|
## Copying files or directories with `:` in the names
|
||||||
|
|
||||||
@@ -424,11 +486,15 @@ file or directory like this then use the full path starting with a
|
|||||||
|
|
||||||
So to sync a directory called `sync:me` to a remote called `remote:` use
|
So to sync a directory called `sync:me` to a remote called `remote:` use
|
||||||
|
|
||||||
|
```
|
||||||
rclone sync --interactive ./sync:me remote:path
|
rclone sync --interactive ./sync:me remote:path
|
||||||
|
```
|
||||||
|
|
||||||
or
|
or
|
||||||
|
|
||||||
|
```
|
||||||
rclone sync --interactive /full/path/to/sync:me remote:path
|
rclone sync --interactive /full/path/to/sync:me remote:path
|
||||||
|
```
|
||||||
|
|
||||||
## Server-side copy
|
## Server-side copy
|
||||||
|
|
||||||
@@ -441,7 +507,9 @@ to copy them in place.
|
|||||||
|
|
||||||
Eg
|
Eg
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy s3:oldbucket s3:newbucket
|
rclone copy s3:oldbucket s3:newbucket
|
||||||
|
```
|
||||||
|
|
||||||
Will copy the contents of `oldbucket` to `newbucket` without
|
Will copy the contents of `oldbucket` to `newbucket` without
|
||||||
downloading and re-uploading.
|
downloading and re-uploading.
|
||||||
@@ -460,8 +528,10 @@ same.
|
|||||||
|
|
||||||
This can be used when scripting to make aged backups efficiently, e.g.
|
This can be used when scripting to make aged backups efficiently, e.g.
|
||||||
|
|
||||||
|
```
|
||||||
rclone sync --interactive remote:current-backup remote:previous-backup
|
rclone sync --interactive remote:current-backup remote:previous-backup
|
||||||
rclone sync --interactive /path/to/files remote:current-backup
|
rclone sync --interactive /path/to/files remote:current-backup
|
||||||
|
```
|
||||||
|
|
||||||
## Metadata support {#metadata}
|
## Metadata support {#metadata}
|
||||||
|
|
||||||
@@ -698,7 +768,9 @@ excluded by a filter rule.
|
|||||||
|
|
||||||
For example
|
For example
|
||||||
|
|
||||||
|
```
|
||||||
rclone sync --interactive /path/to/local remote:current --backup-dir remote:old
|
rclone sync --interactive /path/to/local remote:current --backup-dir remote:old
|
||||||
|
```
|
||||||
|
|
||||||
will sync `/path/to/local` to `remote:current`, but for any files
|
will sync `/path/to/local` to `remote:current`, but for any files
|
||||||
which would have been updated or deleted will be stored in
|
which would have been updated or deleted will be stored in
|
||||||
@@ -724,7 +796,9 @@ You can use `--bind 0.0.0.0` to force rclone to use IPv4 addresses and
|
|||||||
|
|
||||||
This option controls the bandwidth limit. For example
|
This option controls the bandwidth limit. For example
|
||||||
|
|
||||||
|
```
|
||||||
--bwlimit 10M
|
--bwlimit 10M
|
||||||
|
```
|
||||||
|
|
||||||
would mean limit the upload and download bandwidth to 10 MiB/s.
|
would mean limit the upload and download bandwidth to 10 MiB/s.
|
||||||
**NB** this is **bytes** per second not **bits** per second. To use a
|
**NB** this is **bytes** per second not **bits** per second. To use a
|
||||||
@@ -734,13 +808,17 @@ suffix B|K|M|G|T|P. The default is `0` which means to not limit bandwidth.
|
|||||||
The upload and download bandwidth can be specified separately, as
|
The upload and download bandwidth can be specified separately, as
|
||||||
`--bwlimit UP:DOWN`, so
|
`--bwlimit UP:DOWN`, so
|
||||||
|
|
||||||
|
```
|
||||||
--bwlimit 10M:100k
|
--bwlimit 10M:100k
|
||||||
|
```
|
||||||
|
|
||||||
would mean limit the upload bandwidth to 10 MiB/s and the download
|
would mean limit the upload bandwidth to 10 MiB/s and the download
|
||||||
bandwidth to 100 KiB/s. Either limit can be "off" meaning no limit, so
|
bandwidth to 100 KiB/s. Either limit can be "off" meaning no limit, so
|
||||||
to just limit the upload bandwidth you would use
|
to just limit the upload bandwidth you would use
|
||||||
|
|
||||||
|
```
|
||||||
--bwlimit 10M:off
|
--bwlimit 10M:off
|
||||||
|
```
|
||||||
|
|
||||||
this would limit the upload bandwidth to 10 MiB/s but the download
|
this would limit the upload bandwidth to 10 MiB/s but the download
|
||||||
bandwidth would be unlimited.
|
bandwidth would be unlimited.
|
||||||
@@ -816,12 +894,16 @@ of a long running rclone transfer and to restore it back to the value specified
|
|||||||
with `--bwlimit` quickly when needed. Assuming there is only one rclone instance
|
with `--bwlimit` quickly when needed. Assuming there is only one rclone instance
|
||||||
running, you can toggle the limiter like this:
|
running, you can toggle the limiter like this:
|
||||||
|
|
||||||
|
```
|
||||||
kill -SIGUSR2 $(pidof rclone)
|
kill -SIGUSR2 $(pidof rclone)
|
||||||
|
```
|
||||||
|
|
||||||
If you configure rclone with a [remote control](/rc) then you can use
|
If you configure rclone with a [remote control](/rc) then you can use
|
||||||
change the bwlimit dynamically:
|
change the bwlimit dynamically:
|
||||||
|
|
||||||
|
```
|
||||||
rclone rc core/bwlimit rate=1M
|
rclone rc core/bwlimit rate=1M
|
||||||
|
```
|
||||||
|
|
||||||
### --bwlimit-file BwTimetable
|
### --bwlimit-file BwTimetable
|
||||||
|
|
||||||
@@ -830,7 +912,9 @@ This option controls per file bandwidth limit. For the options see the
|
|||||||
|
|
||||||
For example use this to allow no transfers to be faster than 1 MiB/s
|
For example use this to allow no transfers to be faster than 1 MiB/s
|
||||||
|
|
||||||
|
```
|
||||||
--bwlimit-file 1M
|
--bwlimit-file 1M
|
||||||
|
```
|
||||||
|
|
||||||
This can be used in conjunction with `--bwlimit`.
|
This can be used in conjunction with `--bwlimit`.
|
||||||
|
|
||||||
@@ -1030,10 +1114,12 @@ beginning of a line.
|
|||||||
|
|
||||||
Example:
|
Example:
|
||||||
|
|
||||||
|
```
|
||||||
[megaremote]
|
[megaremote]
|
||||||
type = mega
|
type = mega
|
||||||
user = you@example.com
|
user = you@example.com
|
||||||
pass = PDPcQVVjVtzFY-GTdDFozqBhTdsPg3qH
|
pass = PDPcQVVjVtzFY-GTdDFozqBhTdsPg3qH
|
||||||
|
```
|
||||||
|
|
||||||
Note that passwords are in [obscured](/commands/rclone_obscure/)
|
Note that passwords are in [obscured](/commands/rclone_obscure/)
|
||||||
form. Also, many storage systems uses token-based authentication instead
|
form. Also, many storage systems uses token-based authentication instead
|
||||||
@@ -1113,17 +1199,23 @@ time rclone started up.
|
|||||||
This disables a comma separated list of optional features. For example
|
This disables a comma separated list of optional features. For example
|
||||||
to disable server-side move and server-side copy use:
|
to disable server-side move and server-side copy use:
|
||||||
|
|
||||||
|
```
|
||||||
--disable move,copy
|
--disable move,copy
|
||||||
|
```
|
||||||
|
|
||||||
The features can be put in any case.
|
The features can be put in any case.
|
||||||
|
|
||||||
To see a list of which features can be disabled use:
|
To see a list of which features can be disabled use:
|
||||||
|
|
||||||
|
```
|
||||||
--disable help
|
--disable help
|
||||||
|
```
|
||||||
|
|
||||||
The features a remote has can be seen in JSON format with:
|
The features a remote has can be seen in JSON format with:
|
||||||
|
|
||||||
|
```
|
||||||
rclone backend features remote:
|
rclone backend features remote:
|
||||||
|
```
|
||||||
|
|
||||||
See the overview [features](/overview/#features) and
|
See the overview [features](/overview/#features) and
|
||||||
[optional features](/overview/#optional-features) to get an idea of
|
[optional features](/overview/#optional-features) to get an idea of
|
||||||
@@ -1401,7 +1493,9 @@ temporary file with an extension like this, where `XXXXXX` represents a
|
|||||||
hash of the source file's fingerprint and `.partial` is
|
hash of the source file's fingerprint and `.partial` is
|
||||||
[--partial-suffix](#partial-suffix) value (`.partial` by default).
|
[--partial-suffix](#partial-suffix) value (`.partial` by default).
|
||||||
|
|
||||||
|
```
|
||||||
original-file-name.XXXXXX.partial
|
original-file-name.XXXXXX.partial
|
||||||
|
```
|
||||||
|
|
||||||
(rclone will make sure the final name is no longer than 100 characters
|
(rclone will make sure the final name is no longer than 100 characters
|
||||||
by truncating the `original-file-name` part if necessary).
|
by truncating the `original-file-name` part if necessary).
|
||||||
@@ -1573,7 +1667,9 @@ once as administrator to create the registry key in advance.
|
|||||||
severe) than or equal to the `--log-level`. For example to log DEBUG
|
severe) than or equal to the `--log-level`. For example to log DEBUG
|
||||||
to a log file but ERRORs to the event log you would use
|
to a log file but ERRORs to the event log you would use
|
||||||
|
|
||||||
|
```
|
||||||
--log-file rclone.log --log-level DEBUG --windows-event-log ERROR
|
--log-file rclone.log --log-level DEBUG --windows-event-log ERROR
|
||||||
|
```
|
||||||
|
|
||||||
This option is only supported Windows platforms.
|
This option is only supported Windows platforms.
|
||||||
|
|
||||||
@@ -1802,9 +1898,11 @@ it in `"`, if you want a literal `"` in an argument then enclose the
|
|||||||
argument in `"` and double the `"`. See [CSV encoding](https://godoc.org/encoding/csv)
|
argument in `"` and double the `"`. See [CSV encoding](https://godoc.org/encoding/csv)
|
||||||
for more info.
|
for more info.
|
||||||
|
|
||||||
|
```
|
||||||
--metadata-mapper "python bin/test_metadata_mapper.py"
|
--metadata-mapper "python bin/test_metadata_mapper.py"
|
||||||
--metadata-mapper 'python bin/test_metadata_mapper.py "argument with a space"'
|
--metadata-mapper 'python bin/test_metadata_mapper.py "argument with a space"'
|
||||||
--metadata-mapper 'python bin/test_metadata_mapper.py "argument with ""two"" quotes"'
|
--metadata-mapper 'python bin/test_metadata_mapper.py "argument with ""two"" quotes"'
|
||||||
|
```
|
||||||
|
|
||||||
This uses a simple JSON based protocol with input on STDIN and output
|
This uses a simple JSON based protocol with input on STDIN and output
|
||||||
on STDOUT. This will be called for every file and directory copied and
|
on STDOUT. This will be called for every file and directory copied and
|
||||||
@@ -2170,9 +2268,11 @@ for more info.
|
|||||||
|
|
||||||
Eg
|
Eg
|
||||||
|
|
||||||
|
```
|
||||||
--password-command "echo hello"
|
--password-command "echo hello"
|
||||||
--password-command 'echo "hello with space"'
|
--password-command 'echo "hello with space"'
|
||||||
--password-command 'echo "hello with ""quotes"" and space"'
|
--password-command 'echo "hello with ""quotes"" and space"'
|
||||||
|
```
|
||||||
|
|
||||||
Note that when changing the configuration password the environment
|
Note that when changing the configuration password the environment
|
||||||
variable `RCLONE_PASSWORD_CHANGE=1` will be set. This can be used to
|
variable `RCLONE_PASSWORD_CHANGE=1` will be set. This can be used to
|
||||||
@@ -2372,7 +2472,9 @@ or with `--backup-dir`. See `--backup-dir` for more info.
|
|||||||
|
|
||||||
For example
|
For example
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy --interactive /path/to/local/file remote:current --suffix .bak
|
rclone copy --interactive /path/to/local/file remote:current --suffix .bak
|
||||||
|
```
|
||||||
|
|
||||||
will copy `/path/to/local` to `remote:current`, but for any files
|
will copy `/path/to/local` to `remote:current`, but for any files
|
||||||
which would have been updated or deleted have .bak added.
|
which would have been updated or deleted have .bak added.
|
||||||
@@ -2381,7 +2483,9 @@ If using `rclone sync` with `--suffix` and without `--backup-dir` then
|
|||||||
it is recommended to put a filter rule in excluding the suffix
|
it is recommended to put a filter rule in excluding the suffix
|
||||||
otherwise the `sync` will delete the backup files.
|
otherwise the `sync` will delete the backup files.
|
||||||
|
|
||||||
|
```
|
||||||
rclone sync --interactive /path/to/local/file remote:current --suffix .bak --exclude "*.bak"
|
rclone sync --interactive /path/to/local/file remote:current --suffix .bak --exclude "*.bak"
|
||||||
|
```
|
||||||
|
|
||||||
### --suffix-keep-extension
|
### --suffix-keep-extension
|
||||||
|
|
||||||
@@ -3194,7 +3298,9 @@ set the access key of all remotes using S3, including myS3Crypt.
|
|||||||
Note also that now rclone has [connection strings](#connection-strings),
|
Note also that now rclone has [connection strings](#connection-strings),
|
||||||
it is probably easier to use those instead which makes the above example
|
it is probably easier to use those instead which makes the above example
|
||||||
|
|
||||||
|
```
|
||||||
rclone lsd :s3,access_key_id=XXX,secret_access_key=XXX:
|
rclone lsd :s3,access_key_id=XXX,secret_access_key=XXX:
|
||||||
|
```
|
||||||
|
|
||||||
### Precedence
|
### Precedence
|
||||||
|
|
||||||
|
|||||||
@@ -51,11 +51,15 @@ signatures on the release.
|
|||||||
|
|
||||||
To install rclone on Linux/macOS/BSD systems, run:
|
To install rclone on Linux/macOS/BSD systems, run:
|
||||||
|
|
||||||
|
```
|
||||||
sudo -v ; curl https://rclone.org/install.sh | sudo bash
|
sudo -v ; curl https://rclone.org/install.sh | sudo bash
|
||||||
|
```
|
||||||
|
|
||||||
For beta installation, run:
|
For beta installation, run:
|
||||||
|
|
||||||
|
```
|
||||||
sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta
|
sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta
|
||||||
|
```
|
||||||
|
|
||||||
Note that this script checks the version of rclone installed first and
|
Note that this script checks the version of rclone installed first and
|
||||||
won't re-download if not needed.
|
won't re-download if not needed.
|
||||||
@@ -65,11 +69,15 @@ won't re-download if not needed.
|
|||||||
[Beta releases](https://beta.rclone.org) are generated from each commit
|
[Beta releases](https://beta.rclone.org) are generated from each commit
|
||||||
to master. Note these are named like
|
to master. Note these are named like
|
||||||
|
|
||||||
|
```
|
||||||
{Version Tag}.beta.{Commit Number}.{Git Commit Hash}
|
{Version Tag}.beta.{Commit Number}.{Git Commit Hash}
|
||||||
|
```
|
||||||
|
|
||||||
e.g.
|
e.g.
|
||||||
|
|
||||||
|
```
|
||||||
v1.53.0-beta.4677.b657a2204
|
v1.53.0-beta.4677.b657a2204
|
||||||
|
```
|
||||||
|
|
||||||
The `Version Tag` is the version that the beta release will become
|
The `Version Tag` is the version that the beta release will become
|
||||||
when it is released. You can match the `Git Commit Hash` up with the
|
when it is released. You can match the `Git Commit Hash` up with the
|
||||||
@@ -79,11 +87,15 @@ and will normally be at the end of the list.
|
|||||||
|
|
||||||
Some beta releases may have a branch name also:
|
Some beta releases may have a branch name also:
|
||||||
|
|
||||||
|
```
|
||||||
{Version Tag}-beta.{Commit Number}.{Git Commit Hash}.{Branch Name}
|
{Version Tag}-beta.{Commit Number}.{Git Commit Hash}.{Branch Name}
|
||||||
|
```
|
||||||
|
|
||||||
e.g.
|
e.g.
|
||||||
|
|
||||||
|
```
|
||||||
v1.53.0-beta.4677.b657a2204.semver
|
v1.53.0-beta.4677.b657a2204.semver
|
||||||
|
```
|
||||||
|
|
||||||
The presence of `Branch Name` indicates that this is a feature under
|
The presence of `Branch Name` indicates that this is a feature under
|
||||||
development which will at some point be merged into the normal betas
|
development which will at some point be merged into the normal betas
|
||||||
|
|||||||
@@ -47,7 +47,9 @@ The syncs would be incremental (on a file by file basis).
|
|||||||
|
|
||||||
e.g.
|
e.g.
|
||||||
|
|
||||||
|
```
|
||||||
rclone sync --interactive drive:Folder s3:bucket
|
rclone sync --interactive drive:Folder s3:bucket
|
||||||
|
```
|
||||||
|
|
||||||
### Using rclone from multiple locations at the same time ###
|
### Using rclone from multiple locations at the same time ###
|
||||||
|
|
||||||
@@ -116,17 +118,21 @@ may use `http_proxy` but another one `HTTP_PROXY`. The `Go` libraries
|
|||||||
used by `rclone` will try both variations, but you may wish to set all
|
used by `rclone` will try both variations, but you may wish to set all
|
||||||
possibilities. So, on Linux, you may end up with code similar to
|
possibilities. So, on Linux, you may end up with code similar to
|
||||||
|
|
||||||
|
```
|
||||||
export http_proxy=http://proxyserver:12345
|
export http_proxy=http://proxyserver:12345
|
||||||
export https_proxy=$http_proxy
|
export https_proxy=$http_proxy
|
||||||
export HTTP_PROXY=$http_proxy
|
export HTTP_PROXY=$http_proxy
|
||||||
export HTTPS_PROXY=$http_proxy
|
export HTTPS_PROXY=$http_proxy
|
||||||
|
```
|
||||||
|
|
||||||
Note: If the proxy server requires a username and password, then use
|
Note: If the proxy server requires a username and password, then use
|
||||||
|
|
||||||
|
```
|
||||||
export http_proxy=http://username:password@proxyserver:12345
|
export http_proxy=http://username:password@proxyserver:12345
|
||||||
export https_proxy=$http_proxy
|
export https_proxy=$http_proxy
|
||||||
export HTTP_PROXY=$http_proxy
|
export HTTP_PROXY=$http_proxy
|
||||||
export HTTPS_PROXY=$http_proxy
|
export HTTPS_PROXY=$http_proxy
|
||||||
|
```
|
||||||
|
|
||||||
The `NO_PROXY` allows you to disable the proxy for specific hosts.
|
The `NO_PROXY` allows you to disable the proxy for specific hosts.
|
||||||
Hosts must be comma separated, and can contain domains or parts.
|
Hosts must be comma separated, and can contain domains or parts.
|
||||||
@@ -134,8 +140,10 @@ For instance "foo.com" also matches "bar.foo.com".
|
|||||||
|
|
||||||
e.g.
|
e.g.
|
||||||
|
|
||||||
|
```
|
||||||
export no_proxy=localhost,127.0.0.0/8,my.host.name
|
export no_proxy=localhost,127.0.0.0/8,my.host.name
|
||||||
export NO_PROXY=$no_proxy
|
export NO_PROXY=$no_proxy
|
||||||
|
```
|
||||||
|
|
||||||
Note that the FTP backend does not support `ftp_proxy` yet.
|
Note that the FTP backend does not support `ftp_proxy` yet.
|
||||||
|
|
||||||
@@ -148,10 +156,12 @@ possibly on Solaris.
|
|||||||
Rclone (via the Go runtime) tries to load the root certificates from
|
Rclone (via the Go runtime) tries to load the root certificates from
|
||||||
these places on Linux.
|
these places on Linux.
|
||||||
|
|
||||||
|
```
|
||||||
"/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc.
|
"/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc.
|
||||||
"/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL
|
"/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL
|
||||||
"/etc/ssl/ca-bundle.pem", // OpenSUSE
|
"/etc/ssl/ca-bundle.pem", // OpenSUSE
|
||||||
"/etc/pki/tls/cacert.pem", // OpenELEC
|
"/etc/pki/tls/cacert.pem", // OpenELEC
|
||||||
|
```
|
||||||
|
|
||||||
So doing something like this should fix the problem. It also sets the
|
So doing something like this should fix the problem. It also sets the
|
||||||
time which is important for SSL to work properly.
|
time which is important for SSL to work properly.
|
||||||
|
|||||||
@@ -39,6 +39,7 @@ Here is a formal definition of the pattern syntax,
|
|||||||
|
|
||||||
Rclone matching rules follow a glob style:
|
Rclone matching rules follow a glob style:
|
||||||
|
|
||||||
|
```
|
||||||
* matches any sequence of non-separator (/) characters
|
* matches any sequence of non-separator (/) characters
|
||||||
** matches any sequence of characters including / separators
|
** matches any sequence of characters including / separators
|
||||||
? matches any single non-separator (/) character
|
? matches any single non-separator (/) character
|
||||||
@@ -50,27 +51,37 @@ Rclone matching rules follow a glob style:
|
|||||||
regular expression to match
|
regular expression to match
|
||||||
c matches character c (c != *, **, ?, \, [, {, })
|
c matches character c (c != *, **, ?, \, [, {, })
|
||||||
\c matches reserved character c (c = *, **, ?, \, [, {, }) or character class
|
\c matches reserved character c (c = *, **, ?, \, [, {, }) or character class
|
||||||
|
```
|
||||||
|
|
||||||
character-range:
|
character-range:
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
c matches character c (c != \, -, ])
|
c matches character c (c != \, -, ])
|
||||||
\c matches reserved character c (c = \, -, ])
|
\c matches reserved character c (c = \, -, ])
|
||||||
lo - hi matches character c for lo <= c <= hi
|
lo - hi matches character c for lo <= c <= hi
|
||||||
|
```
|
||||||
|
|
||||||
pattern-list:
|
pattern-list:
|
||||||
|
|
||||||
|
```
|
||||||
pattern { , pattern }
|
pattern { , pattern }
|
||||||
comma-separated (without spaces) patterns
|
comma-separated (without spaces) patterns
|
||||||
|
```
|
||||||
|
|
||||||
character classes (see [Go regular expression reference](https://golang.org/pkg/regexp/syntax/)) include:
|
character classes (see [Go regular expression reference](https://golang.org/pkg/regexp/syntax/)) include:
|
||||||
|
|
||||||
|
```
|
||||||
Named character classes (e.g. [\d], [^\d], [\D], [^\D])
|
Named character classes (e.g. [\d], [^\d], [\D], [^\D])
|
||||||
Perl character classes (e.g. \s, \S, \w, \W)
|
Perl character classes (e.g. \s, \S, \w, \W)
|
||||||
ASCII character classes (e.g. [[:alnum:]], [[:alpha:]], [[:punct:]], [[:xdigit:]])
|
ASCII character classes (e.g. [[:alnum:]], [[:alpha:]], [[:punct:]], [[:xdigit:]])
|
||||||
|
```
|
||||||
|
|
||||||
regexp for advanced users to insert a regular expression - see [below](#regexp) for more info:
|
regexp for advanced users to insert a regular expression - see [below](#regexp) for more info:
|
||||||
|
|
||||||
|
```
|
||||||
Any re2 regular expression not containing `}}`
|
Any re2 regular expression not containing `}}`
|
||||||
|
```
|
||||||
|
|
||||||
If the filter pattern starts with a `/` then it only matches
|
If the filter pattern starts with a `/` then it only matches
|
||||||
at the top level of the directory tree,
|
at the top level of the directory tree,
|
||||||
@@ -80,6 +91,7 @@ starting at the **end of the path/file name** but it only matches
|
|||||||
a complete path element - it must match from a `/`
|
a complete path element - it must match from a `/`
|
||||||
separator or the beginning of the path/file.
|
separator or the beginning of the path/file.
|
||||||
|
|
||||||
|
```
|
||||||
file.jpg - matches "file.jpg"
|
file.jpg - matches "file.jpg"
|
||||||
- matches "directory/file.jpg"
|
- matches "directory/file.jpg"
|
||||||
- doesn't match "afile.jpg"
|
- doesn't match "afile.jpg"
|
||||||
@@ -87,11 +99,13 @@ separator or the beginning of the path/file.
|
|||||||
/file.jpg - matches "file.jpg" in the root directory of the remote
|
/file.jpg - matches "file.jpg" in the root directory of the remote
|
||||||
- doesn't match "afile.jpg"
|
- doesn't match "afile.jpg"
|
||||||
- doesn't match "directory/file.jpg"
|
- doesn't match "directory/file.jpg"
|
||||||
|
```
|
||||||
|
|
||||||
The top level of the remote might not be the top level of the drive.
|
The top level of the remote might not be the top level of the drive.
|
||||||
|
|
||||||
E.g. for a Microsoft Windows local directory structure
|
E.g. for a Microsoft Windows local directory structure
|
||||||
|
|
||||||
|
```
|
||||||
F:
|
F:
|
||||||
├── bkp
|
├── bkp
|
||||||
├── data
|
├── data
|
||||||
@@ -100,6 +114,7 @@ E.g. for a Microsoft Windows local directory structure
|
|||||||
│ │ └── 456.jpg
|
│ │ └── 456.jpg
|
||||||
│ ├── incl
|
│ ├── incl
|
||||||
│ │ └── document.pdf
|
│ │ └── document.pdf
|
||||||
|
```
|
||||||
|
|
||||||
To copy the contents of folder `data` into folder `bkp` excluding the contents of subfolder
|
To copy the contents of folder `data` into folder `bkp` excluding the contents of subfolder
|
||||||
`excl`the following command treats `F:\data` and `F:\bkp` as top level for filtering.
|
`excl`the following command treats `F:\data` and `F:\bkp` as top level for filtering.
|
||||||
@@ -113,13 +128,17 @@ Simple patterns are case sensitive unless the `--ignore-case` flag is used.
|
|||||||
|
|
||||||
Without `--ignore-case` (default)
|
Without `--ignore-case` (default)
|
||||||
|
|
||||||
|
```
|
||||||
potato - matches "potato"
|
potato - matches "potato"
|
||||||
- doesn't match "POTATO"
|
- doesn't match "POTATO"
|
||||||
|
```
|
||||||
|
|
||||||
With `--ignore-case`
|
With `--ignore-case`
|
||||||
|
|
||||||
|
```
|
||||||
potato - matches "potato"
|
potato - matches "potato"
|
||||||
- matches "POTATO"
|
- matches "POTATO"
|
||||||
|
```
|
||||||
|
|
||||||
## Using regular expressions in filter patterns {#regexp}
|
## Using regular expressions in filter patterns {#regexp}
|
||||||
|
|
||||||
@@ -141,26 +160,36 @@ the supplied regular expression(s).
|
|||||||
Here is how the `{{regexp}}` is transformed into an full regular
|
Here is how the `{{regexp}}` is transformed into an full regular
|
||||||
expression to match the entire path:
|
expression to match the entire path:
|
||||||
|
|
||||||
|
```
|
||||||
{{regexp}} becomes (^|/)(regexp)$
|
{{regexp}} becomes (^|/)(regexp)$
|
||||||
/{{regexp}} becomes ^(regexp)$
|
/{{regexp}} becomes ^(regexp)$
|
||||||
|
```
|
||||||
|
|
||||||
Regexp syntax can be mixed with glob syntax, for example
|
Regexp syntax can be mixed with glob syntax, for example
|
||||||
|
|
||||||
|
```
|
||||||
*.{{jpe?g}} to match file.jpg, file.jpeg but not file.png
|
*.{{jpe?g}} to match file.jpg, file.jpeg but not file.png
|
||||||
|
```
|
||||||
|
|
||||||
You can also use regexp flags - to set case insensitive, for example
|
You can also use regexp flags - to set case insensitive, for example
|
||||||
|
|
||||||
|
```
|
||||||
*.{{(?i)jpg}} to match file.jpg, file.JPG but not file.png
|
*.{{(?i)jpg}} to match file.jpg, file.JPG but not file.png
|
||||||
|
```
|
||||||
|
|
||||||
Be careful with wildcards in regular expressions - you don't want them
|
Be careful with wildcards in regular expressions - you don't want them
|
||||||
to match path separators normally. To match any file name starting
|
to match path separators normally. To match any file name starting
|
||||||
with `start` and ending with `end` write
|
with `start` and ending with `end` write
|
||||||
|
|
||||||
|
```
|
||||||
{{start[^/]*end\.jpg}}
|
{{start[^/]*end\.jpg}}
|
||||||
|
```
|
||||||
|
|
||||||
Not
|
Not
|
||||||
|
|
||||||
|
```
|
||||||
{{start.*end\.jpg}}
|
{{start.*end\.jpg}}
|
||||||
|
```
|
||||||
|
|
||||||
Which will match a directory called `start` with a file called
|
Which will match a directory called `start` with a file called
|
||||||
`end.jpg` in it as the `.*` will match `/` characters.
|
`end.jpg` in it as the `.*` will match `/` characters.
|
||||||
@@ -290,11 +319,15 @@ command specify the `--dump filters` flag.
|
|||||||
|
|
||||||
E.g. for an include rule
|
E.g. for an include rule
|
||||||
|
|
||||||
|
```
|
||||||
/a/*.jpg
|
/a/*.jpg
|
||||||
|
```
|
||||||
|
|
||||||
Rclone implies the directory include rule
|
Rclone implies the directory include rule
|
||||||
|
|
||||||
|
```
|
||||||
/a/
|
/a/
|
||||||
|
```
|
||||||
|
|
||||||
Directory filter rules specified in an rclone command can limit
|
Directory filter rules specified in an rclone command can limit
|
||||||
the scope of an rclone command but path/file filters still have
|
the scope of an rclone command but path/file filters still have
|
||||||
@@ -308,10 +341,12 @@ access to the remote by ignoring everything outside of that directory.
|
|||||||
E.g. `rclone ls remote: --filter-from filter-list.txt` with a file
|
E.g. `rclone ls remote: --filter-from filter-list.txt` with a file
|
||||||
`filter-list.txt`:
|
`filter-list.txt`:
|
||||||
|
|
||||||
|
```
|
||||||
- /dir1/
|
- /dir1/
|
||||||
- /dir2/
|
- /dir2/
|
||||||
+ *.pdf
|
+ *.pdf
|
||||||
- **
|
- **
|
||||||
|
```
|
||||||
|
|
||||||
All files in directories `dir1` or `dir2` or their subdirectories
|
All files in directories `dir1` or `dir2` or their subdirectories
|
||||||
are completely excluded from the listing. Only files of suffix
|
are completely excluded from the listing. Only files of suffix
|
||||||
@@ -329,7 +364,9 @@ from this pattern list.
|
|||||||
|
|
||||||
E.g. for an include rule
|
E.g. for an include rule
|
||||||
|
|
||||||
|
```
|
||||||
{dir1/**,dir2/**}
|
{dir1/**,dir2/**}
|
||||||
|
```
|
||||||
|
|
||||||
Rclone will match files below directories `dir1` or `dir2` only,
|
Rclone will match files below directories `dir1` or `dir2` only,
|
||||||
but will not be able to use this filter to exclude a directory `dir3`
|
but will not be able to use this filter to exclude a directory `dir3`
|
||||||
@@ -381,9 +418,11 @@ named file. The file contains a list of remarks and pattern rules.
|
|||||||
|
|
||||||
For an example `exclude-file.txt`:
|
For an example `exclude-file.txt`:
|
||||||
|
|
||||||
|
```
|
||||||
# a sample exclude rule file
|
# a sample exclude rule file
|
||||||
*.bak
|
*.bak
|
||||||
file2.jpg
|
file2.jpg
|
||||||
|
```
|
||||||
|
|
||||||
`rclone ls remote: --exclude-from exclude-file.txt` lists the files on
|
`rclone ls remote: --exclude-from exclude-file.txt` lists the files on
|
||||||
`remote:` except those named `file2.jpg` or with a suffix `.bak`. That is
|
`remote:` except those named `file2.jpg` or with a suffix `.bak`. That is
|
||||||
@@ -426,12 +465,16 @@ E.g. `rclone ls remote: --include "*.{png,jpg}"` lists the files on
|
|||||||
E.g. multiple rclone copy commands can be combined with `--include` and a
|
E.g. multiple rclone copy commands can be combined with `--include` and a
|
||||||
pattern-list.
|
pattern-list.
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy /vol1/A remote:A
|
rclone copy /vol1/A remote:A
|
||||||
rclone copy /vol1/B remote:B
|
rclone copy /vol1/B remote:B
|
||||||
|
```
|
||||||
|
|
||||||
is equivalent to:
|
is equivalent to:
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy /vol1 remote: --include "{A,B}/**"
|
rclone copy /vol1 remote: --include "{A,B}/**"
|
||||||
|
```
|
||||||
|
|
||||||
E.g. `rclone ls remote:/wheat --include "??[^[:punct:]]*"` lists the
|
E.g. `rclone ls remote:/wheat --include "??[^[:punct:]]*"` lists the
|
||||||
files `remote:` directory `wheat` (and subdirectories) whose third
|
files `remote:` directory `wheat` (and subdirectories) whose third
|
||||||
@@ -445,9 +488,11 @@ named file. The file contains a list of remarks and pattern rules.
|
|||||||
|
|
||||||
For an example `include-file.txt`:
|
For an example `include-file.txt`:
|
||||||
|
|
||||||
|
```
|
||||||
# a sample include rule file
|
# a sample include rule file
|
||||||
*.jpg
|
*.jpg
|
||||||
file2.avi
|
file2.avi
|
||||||
|
```
|
||||||
|
|
||||||
`rclone ls remote: --include-from include-file.txt` lists the files on
|
`rclone ls remote: --include-from include-file.txt` lists the files on
|
||||||
`remote:` with name `file2.avi` or suffix `.jpg`. That is equivalent to
|
`remote:` with name `file2.avi` or suffix `.jpg`. That is equivalent to
|
||||||
@@ -509,6 +554,7 @@ Lines starting with # or ; are ignored, and can be used to write comments. Inlin
|
|||||||
|
|
||||||
E.g. for `filter-file.txt`:
|
E.g. for `filter-file.txt`:
|
||||||
|
|
||||||
|
```
|
||||||
# a sample filter rule file
|
# a sample filter rule file
|
||||||
- secret*.jpg
|
- secret*.jpg
|
||||||
+ *.jpg
|
+ *.jpg
|
||||||
@@ -519,6 +565,7 @@ E.g. for `filter-file.txt`:
|
|||||||
+ /dir/**
|
+ /dir/**
|
||||||
# exclude everything else
|
# exclude everything else
|
||||||
- *
|
- *
|
||||||
|
```
|
||||||
|
|
||||||
`rclone ls remote: --filter-from filter-file.txt` lists the path/files on
|
`rclone ls remote: --filter-from filter-file.txt` lists the path/files on
|
||||||
`remote:` including all `jpg` and `png` files, excluding any
|
`remote:` including all `jpg` and `png` files, excluding any
|
||||||
@@ -528,22 +575,26 @@ everything in the directory `dir` at the root of `remote`, except
|
|||||||
|
|
||||||
E.g. for an alternative `filter-file.txt`:
|
E.g. for an alternative `filter-file.txt`:
|
||||||
|
|
||||||
|
```
|
||||||
- secret*.jpg
|
- secret*.jpg
|
||||||
+ *.jpg
|
+ *.jpg
|
||||||
+ *.png
|
+ *.png
|
||||||
+ file2.avi
|
+ file2.avi
|
||||||
- *
|
- *
|
||||||
|
```
|
||||||
|
|
||||||
Files `file1.jpg`, `file3.png` and `file2.avi` are listed whilst
|
Files `file1.jpg`, `file3.png` and `file2.avi` are listed whilst
|
||||||
`secret17.jpg` and files without the suffix `.jpg` or `.png` are excluded.
|
`secret17.jpg` and files without the suffix `.jpg` or `.png` are excluded.
|
||||||
|
|
||||||
E.g. for an alternative `filter-file.txt`:
|
E.g. for an alternative `filter-file.txt`:
|
||||||
|
|
||||||
|
```
|
||||||
+ *.jpg
|
+ *.jpg
|
||||||
+ *.gif
|
+ *.gif
|
||||||
!
|
!
|
||||||
+ 42.doc
|
+ 42.doc
|
||||||
- *
|
- *
|
||||||
|
```
|
||||||
|
|
||||||
Only file 42.doc is listed. Prior rules are cleared by the `!`.
|
Only file 42.doc is listed. Prior rules are cleared by the `!`.
|
||||||
|
|
||||||
@@ -586,55 +637,73 @@ you need the input to be processed in a raw manner.
|
|||||||
|
|
||||||
E.g. for a file `files-from.txt`:
|
E.g. for a file `files-from.txt`:
|
||||||
|
|
||||||
|
```
|
||||||
# comment
|
# comment
|
||||||
file1.jpg
|
file1.jpg
|
||||||
subdir/file2.jpg
|
subdir/file2.jpg
|
||||||
|
```
|
||||||
|
|
||||||
`rclone copy --files-from files-from.txt /home/me/pics remote:pics`
|
`rclone copy --files-from files-from.txt /home/me/pics remote:pics`
|
||||||
copies the following, if they exist, and only those files.
|
copies the following, if they exist, and only those files.
|
||||||
|
|
||||||
|
```
|
||||||
/home/me/pics/file1.jpg → remote:pics/file1.jpg
|
/home/me/pics/file1.jpg → remote:pics/file1.jpg
|
||||||
/home/me/pics/subdir/file2.jpg → remote:pics/subdir/file2.jpg
|
/home/me/pics/subdir/file2.jpg → remote:pics/subdir/file2.jpg
|
||||||
|
```
|
||||||
|
|
||||||
E.g. to copy the following files referenced by their absolute paths:
|
E.g. to copy the following files referenced by their absolute paths:
|
||||||
|
|
||||||
|
```
|
||||||
/home/user1/42
|
/home/user1/42
|
||||||
/home/user1/dir/ford
|
/home/user1/dir/ford
|
||||||
/home/user2/prefect
|
/home/user2/prefect
|
||||||
|
```
|
||||||
|
|
||||||
First find a common subdirectory - in this case `/home`
|
First find a common subdirectory - in this case `/home`
|
||||||
and put the remaining files in `files-from.txt` with or without
|
and put the remaining files in `files-from.txt` with or without
|
||||||
leading `/`, e.g.
|
leading `/`, e.g.
|
||||||
|
|
||||||
|
```
|
||||||
user1/42
|
user1/42
|
||||||
user1/dir/ford
|
user1/dir/ford
|
||||||
user2/prefect
|
user2/prefect
|
||||||
|
```
|
||||||
|
|
||||||
Then copy these to a remote:
|
Then copy these to a remote:
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy --files-from files-from.txt /home remote:backup
|
rclone copy --files-from files-from.txt /home remote:backup
|
||||||
|
```
|
||||||
|
|
||||||
The three files are transferred as follows:
|
The three files are transferred as follows:
|
||||||
|
|
||||||
|
```
|
||||||
/home/user1/42 → remote:backup/user1/important
|
/home/user1/42 → remote:backup/user1/important
|
||||||
/home/user1/dir/ford → remote:backup/user1/dir/file
|
/home/user1/dir/ford → remote:backup/user1/dir/file
|
||||||
/home/user2/prefect → remote:backup/user2/stuff
|
/home/user2/prefect → remote:backup/user2/stuff
|
||||||
|
```
|
||||||
|
|
||||||
Alternatively if `/` is chosen as root `files-from.txt` will be:
|
Alternatively if `/` is chosen as root `files-from.txt` will be:
|
||||||
|
|
||||||
|
```
|
||||||
/home/user1/42
|
/home/user1/42
|
||||||
/home/user1/dir/ford
|
/home/user1/dir/ford
|
||||||
/home/user2/prefect
|
/home/user2/prefect
|
||||||
|
```
|
||||||
|
|
||||||
The copy command will be:
|
The copy command will be:
|
||||||
|
|
||||||
|
```
|
||||||
rclone copy --files-from files-from.txt / remote:backup
|
rclone copy --files-from files-from.txt / remote:backup
|
||||||
|
```
|
||||||
|
|
||||||
Then there will be an extra `home` directory on the remote:
|
Then there will be an extra `home` directory on the remote:
|
||||||
|
|
||||||
|
```
|
||||||
/home/user1/42 → remote:backup/home/user1/42
|
/home/user1/42 → remote:backup/home/user1/42
|
||||||
/home/user1/dir/ford → remote:backup/home/user1/dir/ford
|
/home/user1/dir/ford → remote:backup/home/user1/dir/ford
|
||||||
/home/user2/prefect → remote:backup/home/user2/prefect
|
/home/user2/prefect → remote:backup/home/user2/prefect
|
||||||
|
```
|
||||||
|
|
||||||
### `--files-from-raw` - Read list of source-file names without any processing
|
### `--files-from-raw` - Read list of source-file names without any processing
|
||||||
|
|
||||||
@@ -822,7 +891,9 @@ on the destination which are excluded from the command.
|
|||||||
|
|
||||||
E.g. the scope of `rclone sync --interactive A: B:` can be restricted:
|
E.g. the scope of `rclone sync --interactive A: B:` can be restricted:
|
||||||
|
|
||||||
|
```
|
||||||
rclone --min-size 50k --delete-excluded sync A: B:
|
rclone --min-size 50k --delete-excluded sync A: B:
|
||||||
|
```
|
||||||
|
|
||||||
All files on `B:` which are less than 50 KiB are deleted
|
All files on `B:` which are less than 50 KiB are deleted
|
||||||
because they are excluded from the rclone sync command.
|
because they are excluded from the rclone sync command.
|
||||||
@@ -846,10 +917,12 @@ This flag has a priority over other filter flags.
|
|||||||
|
|
||||||
E.g. for the following directory structure:
|
E.g. for the following directory structure:
|
||||||
|
|
||||||
|
```
|
||||||
dir1/file1
|
dir1/file1
|
||||||
dir1/dir2/file2
|
dir1/dir2/file2
|
||||||
dir1/dir2/dir3/file3
|
dir1/dir2/dir3/file3
|
||||||
dir1/dir2/dir3/.ignore
|
dir1/dir2/dir3/.ignore
|
||||||
|
```
|
||||||
|
|
||||||
The command `rclone ls --exclude-if-present .ignore dir1` does
|
The command `rclone ls --exclude-if-present .ignore dir1` does
|
||||||
not list `dir3`, `file3` or `.ignore`.
|
not list `dir3`, `file3` or `.ignore`.
|
||||||
@@ -867,11 +940,15 @@ expressions](#regexp).
|
|||||||
For example if you wished to list only local files with a mode of
|
For example if you wished to list only local files with a mode of
|
||||||
`100664` you could do that with:
|
`100664` you could do that with:
|
||||||
|
|
||||||
|
```
|
||||||
rclone lsf -M --files-only --metadata-include "mode=100664" .
|
rclone lsf -M --files-only --metadata-include "mode=100664" .
|
||||||
|
```
|
||||||
|
|
||||||
Or if you wished to show files with an `atime`, `mtime` or `btime` at a given date:
|
Or if you wished to show files with an `atime`, `mtime` or `btime` at a given date:
|
||||||
|
|
||||||
|
```
|
||||||
rclone lsf -M --files-only --metadata-include "[abm]time=2022-12-16*" .
|
rclone lsf -M --files-only --metadata-include "[abm]time=2022-12-16*" .
|
||||||
|
```
|
||||||
|
|
||||||
Like file filtering, metadata filtering only applies to files not to
|
Like file filtering, metadata filtering only applies to files not to
|
||||||
directories.
|
directories.
|
||||||
|
|||||||
@@ -29,11 +29,15 @@ signatures on the release.
|
|||||||
|
|
||||||
To install rclone on Linux/macOS/BSD systems, run:
|
To install rclone on Linux/macOS/BSD systems, run:
|
||||||
|
|
||||||
|
```
|
||||||
sudo -v ; curl https://rclone.org/install.sh | sudo bash
|
sudo -v ; curl https://rclone.org/install.sh | sudo bash
|
||||||
|
```
|
||||||
|
|
||||||
For beta installation, run:
|
For beta installation, run:
|
||||||
|
|
||||||
|
```
|
||||||
sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta
|
sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta
|
||||||
|
```
|
||||||
|
|
||||||
Note that this script checks the version of rclone installed first and
|
Note that this script checks the version of rclone installed first and
|
||||||
won't re-download if not needed.
|
won't re-download if not needed.
|
||||||
@@ -44,31 +48,41 @@ won't re-download if not needed.
|
|||||||
|
|
||||||
Fetch and unpack
|
Fetch and unpack
|
||||||
|
|
||||||
|
```
|
||||||
curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
|
curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
|
||||||
unzip rclone-current-linux-amd64.zip
|
unzip rclone-current-linux-amd64.zip
|
||||||
cd rclone-*-linux-amd64
|
cd rclone-*-linux-amd64
|
||||||
|
```
|
||||||
|
|
||||||
Copy binary file
|
Copy binary file
|
||||||
|
|
||||||
|
```
|
||||||
sudo cp rclone /usr/bin/
|
sudo cp rclone /usr/bin/
|
||||||
sudo chown root:root /usr/bin/rclone
|
sudo chown root:root /usr/bin/rclone
|
||||||
sudo chmod 755 /usr/bin/rclone
|
sudo chmod 755 /usr/bin/rclone
|
||||||
|
```
|
||||||
|
|
||||||
Install manpage
|
Install manpage
|
||||||
|
|
||||||
|
```
|
||||||
sudo mkdir -p /usr/local/share/man/man1
|
sudo mkdir -p /usr/local/share/man/man1
|
||||||
sudo cp rclone.1 /usr/local/share/man/man1/
|
sudo cp rclone.1 /usr/local/share/man/man1/
|
||||||
sudo mandb
|
sudo mandb
|
||||||
|
```
|
||||||
|
|
||||||
Run `rclone config` to setup. See [rclone config docs](/docs/) for more details.
|
Run `rclone config` to setup. See [rclone config docs](/docs/) for more details.
|
||||||
|
|
||||||
|
```
|
||||||
rclone config
|
rclone config
|
||||||
|
```
|
||||||
|
|
||||||
## macOS installation {#macos}
|
## macOS installation {#macos}
|
||||||
|
|
||||||
### Installation with brew {#macos-brew}
|
### Installation with brew {#macos-brew}
|
||||||
|
|
||||||
|
```
|
||||||
brew install rclone
|
brew install rclone
|
||||||
|
```
|
||||||
|
|
||||||
NOTE: This version of rclone will not support `mount` any more (see
|
NOTE: This version of rclone will not support `mount` any more (see
|
||||||
[#5373](https://github.com/rclone/rclone/issues/5373)). If mounting is wanted
|
[#5373](https://github.com/rclone/rclone/issues/5373)). If mounting is wanted
|
||||||
@@ -84,7 +98,9 @@ developers so it may be out of date. Its current version is as below.
|
|||||||
|
|
||||||
On macOS, rclone can also be installed via [MacPorts](https://www.macports.org):
|
On macOS, rclone can also be installed via [MacPorts](https://www.macports.org):
|
||||||
|
|
||||||
|
```
|
||||||
sudo port install rclone
|
sudo port install rclone
|
||||||
|
```
|
||||||
|
|
||||||
Note that this is a third party installer not controlled by the rclone
|
Note that this is a third party installer not controlled by the rclone
|
||||||
developers so it may be out of date. Its current version is as below.
|
developers so it may be out of date. Its current version is as below.
|
||||||
@@ -100,26 +116,36 @@ notarized it is enough to download with `curl`.
|
|||||||
|
|
||||||
Download the latest version of rclone.
|
Download the latest version of rclone.
|
||||||
|
|
||||||
|
```
|
||||||
cd && curl -O https://downloads.rclone.org/rclone-current-osx-amd64.zip
|
cd && curl -O https://downloads.rclone.org/rclone-current-osx-amd64.zip
|
||||||
|
```
|
||||||
|
|
||||||
Unzip the download and cd to the extracted folder.
|
Unzip the download and cd to the extracted folder.
|
||||||
|
|
||||||
|
```
|
||||||
unzip -a rclone-current-osx-amd64.zip && cd rclone-*-osx-amd64
|
unzip -a rclone-current-osx-amd64.zip && cd rclone-*-osx-amd64
|
||||||
|
```
|
||||||
|
|
||||||
Move rclone to your $PATH. You will be prompted for your password.
|
Move rclone to your $PATH. You will be prompted for your password.
|
||||||
|
|
||||||
|
```
|
||||||
sudo mkdir -p /usr/local/bin
|
sudo mkdir -p /usr/local/bin
|
||||||
sudo mv rclone /usr/local/bin/
|
sudo mv rclone /usr/local/bin/
|
||||||
|
```
|
||||||
|
|
||||||
(the `mkdir` command is safe to run, even if the directory already exists).
|
(the `mkdir` command is safe to run, even if the directory already exists).
|
||||||
|
|
||||||
Remove the leftover files.
|
Remove the leftover files.
|
||||||
|
|
||||||
|
```
|
||||||
cd .. && rm -rf rclone-*-osx-amd64 rclone-current-osx-amd64.zip
|
cd .. && rm -rf rclone-*-osx-amd64 rclone-current-osx-amd64.zip
|
||||||
|
```
|
||||||
|
|
||||||
Run `rclone config` to setup. See [rclone config docs](/docs/) for more details.
|
Run `rclone config` to setup. See [rclone config docs](/docs/) for more details.
|
||||||
|
|
||||||
|
```
|
||||||
rclone config
|
rclone config
|
||||||
|
```
|
||||||
|
|
||||||
### Precompiled binary, using a web browser {#macos-precompiled-web}
|
### Precompiled binary, using a web browser {#macos-precompiled-web}
|
||||||
|
|
||||||
@@ -127,12 +153,16 @@ When downloading a binary with a web browser, the browser will set the macOS
|
|||||||
gatekeeper quarantine attribute. Starting from Catalina, when attempting to run
|
gatekeeper quarantine attribute. Starting from Catalina, when attempting to run
|
||||||
`rclone`, a pop-up will appear saying:
|
`rclone`, a pop-up will appear saying:
|
||||||
|
|
||||||
|
```
|
||||||
"rclone" cannot be opened because the developer cannot be verified.
|
"rclone" cannot be opened because the developer cannot be verified.
|
||||||
macOS cannot verify that this app is free from malware.
|
macOS cannot verify that this app is free from malware.
|
||||||
|
```
|
||||||
|
|
||||||
The simplest fix is to run
|
The simplest fix is to run
|
||||||
|
|
||||||
|
```
|
||||||
xattr -d com.apple.quarantine rclone
|
xattr -d com.apple.quarantine rclone
|
||||||
|
```
|
||||||
|
|
||||||
## Windows installation {#windows}
|
## Windows installation {#windows}
|
||||||
|
|
||||||
|
|||||||
@@ -402,18 +402,24 @@ and to maintain backward compatibility, its behavior has not been changed.
|
|||||||
|
|
||||||
To take a specific example, the FTP backend's default encoding is
|
To take a specific example, the FTP backend's default encoding is
|
||||||
|
|
||||||
|
```
|
||||||
--ftp-encoding "Slash,Del,Ctl,RightSpace,Dot"
|
--ftp-encoding "Slash,Del,Ctl,RightSpace,Dot"
|
||||||
|
```
|
||||||
|
|
||||||
However, let's say the FTP server is running on Windows and can't have
|
However, let's say the FTP server is running on Windows and can't have
|
||||||
any of the invalid Windows characters in file names. You are backing
|
any of the invalid Windows characters in file names. You are backing
|
||||||
up Linux servers to this FTP server which do have those characters in
|
up Linux servers to this FTP server which do have those characters in
|
||||||
file names. So you would add the Windows set which are
|
file names. So you would add the Windows set which are
|
||||||
|
|
||||||
|
```
|
||||||
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
|
||||||
|
```
|
||||||
|
|
||||||
to the existing ones, giving:
|
to the existing ones, giving:
|
||||||
|
|
||||||
|
```
|
||||||
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot,Del,RightSpace
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot,Del,RightSpace
|
||||||
|
```
|
||||||
|
|
||||||
This can be specified using the `--ftp-encoding` flag or using an `encoding` parameter in the config file.
|
This can be specified using the `--ftp-encoding` flag or using an `encoding` parameter in the config file.
|
||||||
|
|
||||||
|
|||||||
@@ -321,21 +321,29 @@ duration of an rc call only then pass in the `_config` parameter.
|
|||||||
This should be in the same format as the `main` key returned by
|
This should be in the same format as the `main` key returned by
|
||||||
[options/get](#options-get).
|
[options/get](#options-get).
|
||||||
|
|
||||||
|
```
|
||||||
rclone rc --loopback options/get blocks=main
|
rclone rc --loopback options/get blocks=main
|
||||||
|
```
|
||||||
|
|
||||||
You can see more help on these options with this command (see [the
|
You can see more help on these options with this command (see [the
|
||||||
options blocks section](#option-blocks) for more info).
|
options blocks section](#option-blocks) for more info).
|
||||||
|
|
||||||
|
```
|
||||||
rclone rc --loopback options/info blocks=main
|
rclone rc --loopback options/info blocks=main
|
||||||
|
```
|
||||||
|
|
||||||
For example, if you wished to run a sync with the `--checksum`
|
For example, if you wished to run a sync with the `--checksum`
|
||||||
parameter, you would pass this parameter in your JSON blob.
|
parameter, you would pass this parameter in your JSON blob.
|
||||||
|
|
||||||
|
```
|
||||||
"_config":{"CheckSum": true}
|
"_config":{"CheckSum": true}
|
||||||
|
```
|
||||||
|
|
||||||
If using `rclone rc` this could be passed as
|
If using `rclone rc` this could be passed as
|
||||||
|
|
||||||
|
```
|
||||||
rclone rc sync/sync ... _config='{"CheckSum": true}'
|
rclone rc sync/sync ... _config='{"CheckSum": true}'
|
||||||
|
```
|
||||||
|
|
||||||
Any config parameters you don't set will inherit the global defaults
|
Any config parameters you don't set will inherit the global defaults
|
||||||
which were set with command line flags or environment variables.
|
which were set with command line flags or environment variables.
|
||||||
@@ -344,8 +352,10 @@ Note that it is possible to set some values as strings or integers -
|
|||||||
see [data types](#data-types) for more info. Here is an example
|
see [data types](#data-types) for more info. Here is an example
|
||||||
setting the equivalent of `--buffer-size` in string or integer format.
|
setting the equivalent of `--buffer-size` in string or integer format.
|
||||||
|
|
||||||
|
```
|
||||||
"_config":{"BufferSize": "42M"}
|
"_config":{"BufferSize": "42M"}
|
||||||
"_config":{"BufferSize": 44040192}
|
"_config":{"BufferSize": 44040192}
|
||||||
|
```
|
||||||
|
|
||||||
If you wish to check the `_config` assignment has worked properly then
|
If you wish to check the `_config` assignment has worked properly then
|
||||||
calling `options/local` will show what the value got set to.
|
calling `options/local` will show what the value got set to.
|
||||||
@@ -358,24 +368,34 @@ pass in the `_filter` parameter.
|
|||||||
This should be in the same format as the `filter` key returned by
|
This should be in the same format as the `filter` key returned by
|
||||||
[options/get](#options-get).
|
[options/get](#options-get).
|
||||||
|
|
||||||
|
```
|
||||||
rclone rc --loopback options/get blocks=filter
|
rclone rc --loopback options/get blocks=filter
|
||||||
|
```
|
||||||
|
|
||||||
You can see more help on these options with this command (see [the
|
You can see more help on these options with this command (see [the
|
||||||
options blocks section](#option-blocks) for more info).
|
options blocks section](#option-blocks) for more info).
|
||||||
|
|
||||||
|
```
|
||||||
rclone rc --loopback options/info blocks=filter
|
rclone rc --loopback options/info blocks=filter
|
||||||
|
```
|
||||||
|
|
||||||
For example, if you wished to run a sync with these flags
|
For example, if you wished to run a sync with these flags
|
||||||
|
|
||||||
|
```
|
||||||
--max-size 1M --max-age 42s --include "a" --include "b"
|
--max-size 1M --max-age 42s --include "a" --include "b"
|
||||||
|
```
|
||||||
|
|
||||||
you would pass this parameter in your JSON blob.
|
you would pass this parameter in your JSON blob.
|
||||||
|
|
||||||
|
```
|
||||||
"_filter":{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}
|
"_filter":{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}
|
||||||
|
```
|
||||||
|
|
||||||
If using `rclone rc` this could be passed as
|
If using `rclone rc` this could be passed as
|
||||||
|
|
||||||
|
```
|
||||||
rclone rc ... _filter='{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}'
|
rclone rc ... _filter='{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}'
|
||||||
|
```
|
||||||
|
|
||||||
Any filter parameters you don't set will inherit the global defaults
|
Any filter parameters you don't set will inherit the global defaults
|
||||||
which were set with command line flags or environment variables.
|
which were set with command line flags or environment variables.
|
||||||
@@ -384,8 +404,10 @@ Note that it is possible to set some values as strings or integers -
|
|||||||
see [data types](#data-types) for more info. Here is an example
|
see [data types](#data-types) for more info. Here is an example
|
||||||
setting the equivalent of `--buffer-size` in string or integer format.
|
setting the equivalent of `--buffer-size` in string or integer format.
|
||||||
|
|
||||||
|
```
|
||||||
"_filter":{"MinSize": "42M"}
|
"_filter":{"MinSize": "42M"}
|
||||||
"_filter":{"MinSize": 44040192}
|
"_filter":{"MinSize": 44040192}
|
||||||
|
```
|
||||||
|
|
||||||
If you wish to check the `_filter` assignment has worked properly then
|
If you wish to check the `_filter` assignment has worked properly then
|
||||||
calling `options/local` will show what the value got set to.
|
calling `options/local` will show what the value got set to.
|
||||||
@@ -2490,7 +2512,9 @@ To use these, first [install go](https://golang.org/doc/install).
|
|||||||
|
|
||||||
To profile rclone's memory use you can run:
|
To profile rclone's memory use you can run:
|
||||||
|
|
||||||
|
```
|
||||||
go tool pprof -web http://localhost:5572/debug/pprof/heap
|
go tool pprof -web http://localhost:5572/debug/pprof/heap
|
||||||
|
```
|
||||||
|
|
||||||
This should open a page in your browser showing what is using what
|
This should open a page in your browser showing what is using what
|
||||||
memory.
|
memory.
|
||||||
@@ -2522,7 +2546,9 @@ alive which should have been garbage collected.
|
|||||||
|
|
||||||
See all active go routines using
|
See all active go routines using
|
||||||
|
|
||||||
|
```
|
||||||
curl http://localhost:5572/debug/pprof/goroutine?debug=1
|
curl http://localhost:5572/debug/pprof/goroutine?debug=1
|
||||||
|
```
|
||||||
|
|
||||||
Or go to http://localhost:5572/debug/pprof/goroutine?debug=1 in your browser.
|
Or go to http://localhost:5572/debug/pprof/goroutine?debug=1 in your browser.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user