mirror of
https://github.com/rclone/rclone.git
synced 2025-12-06 00:03:32 +00:00
docs: fix various markdownlint issues
This commit is contained in:
2
.github/workflows/build.yml
vendored
2
.github/workflows/build.yml
vendored
@@ -291,6 +291,8 @@ jobs:
|
||||
README.md
|
||||
RELEASE.md
|
||||
CODE_OF_CONDUCT.md
|
||||
librclone\README.md
|
||||
backend\s3\README.md
|
||||
docs/content/{authors,bugs,changelog,docs,downloads,faq,filtering,gui,install,licence,overview,privacy}.md
|
||||
|
||||
- name: Scan edits of autogenerated files
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
## Adding a new s3 provider
|
||||
# Adding a new s3 provider
|
||||
|
||||
It is quite easy to add a new S3 provider to rclone.
|
||||
|
||||
@@ -12,179 +12,202 @@ All tags can be found in `backend/s3/providers.go` Provider Struct.
|
||||
Looking through a few of the yaml files as examples should make things
|
||||
clear. `AWS.yaml` as the most config. pasting.
|
||||
|
||||
### YAML
|
||||
## YAML
|
||||
|
||||
In `backend/s3/provider/YourProvider.yaml`
|
||||
|
||||
- name
|
||||
- description
|
||||
- More like the full name often "YourProvider + Object Storage"
|
||||
- More like the full name often "YourProvider + Object Storage"
|
||||
- [Region]
|
||||
- Any regions your provider supports or the defaults (use `region: {}` for this)
|
||||
- Example from AWS.yaml:
|
||||
```yaml
|
||||
region:
|
||||
us-east-1: |-
|
||||
The default endpoint - a good choice if you are unsure.
|
||||
US Region, Northern Virginia, or Pacific Northwest.
|
||||
Leave location constraint empty.
|
||||
```
|
||||
- The defaults (as seen in Rclone.yaml):
|
||||
```yaml
|
||||
region:
|
||||
"": |-
|
||||
Use this if unsure.
|
||||
Will use v4 signatures and an empty region.
|
||||
other-v2-signature: |-
|
||||
Use this only if v4 signatures don't work.
|
||||
E.g. pre Jewel/v10 CEPH.
|
||||
```
|
||||
- Any regions your provider supports or the defaults (use `region: {}` for this)
|
||||
- Example from AWS.yaml:
|
||||
|
||||
```yaml
|
||||
region:
|
||||
us-east-1: |-
|
||||
The default endpoint - a good choice if you are unsure.
|
||||
US Region, Northern Virginia, or Pacific Northwest.
|
||||
Leave location constraint empty.
|
||||
```
|
||||
|
||||
- The defaults (as seen in Rclone.yaml):
|
||||
|
||||
```yaml
|
||||
region:
|
||||
"": |-
|
||||
Use this if unsure.
|
||||
Will use v4 signatures and an empty region.
|
||||
other-v2-signature: |-
|
||||
Use this only if v4 signatures don't work.
|
||||
E.g. pre Jewel/v10 CEPH.
|
||||
```
|
||||
|
||||
- [Endpoint]
|
||||
- Any endpoints your provider supports
|
||||
- Example from Mega.yaml
|
||||
```yaml
|
||||
endpoint:
|
||||
s3.eu-central-1.s4.mega.io: Mega S4 eu-central-1 (Amsterdam)
|
||||
```
|
||||
- Any endpoints your provider supports
|
||||
|
||||
- Example from Mega.yaml
|
||||
|
||||
```yaml
|
||||
endpoint:
|
||||
s3.eu-central-1.s4.mega.io: Mega S4 eu-central-1 (Amsterdam)
|
||||
```
|
||||
|
||||
- [Location Constraint]
|
||||
- The Location Constraint of your remote, often same as region.
|
||||
- Example from AWS.yaml
|
||||
```yaml
|
||||
location_constraint:
|
||||
"": Empty for US Region, Northern Virginia, or Pacific Northwest
|
||||
us-east-2: US East (Ohio) Region
|
||||
```
|
||||
- The Location Constraint of your remote, often same as region.
|
||||
- Example from AWS.yaml
|
||||
|
||||
```yaml
|
||||
location_constraint:
|
||||
"": Empty for US Region, Northern Virginia, or Pacific Northwest
|
||||
us-east-2: US East (Ohio) Region
|
||||
```
|
||||
|
||||
- [ACL]
|
||||
- Identical across *most* providers. Select the default with `acl: {}`
|
||||
- Example from AWS.yaml
|
||||
```yaml
|
||||
acl:
|
||||
private: |-
|
||||
Owner gets FULL_CONTROL.
|
||||
No one else has access rights (default).
|
||||
public-read: |-
|
||||
Owner gets FULL_CONTROL.
|
||||
The AllUsers group gets READ access.
|
||||
public-read-write: |-
|
||||
Owner gets FULL_CONTROL.
|
||||
The AllUsers group gets READ and WRITE access.
|
||||
Granting this on a bucket is generally not recommended.
|
||||
authenticated-read: |-
|
||||
Owner gets FULL_CONTROL.
|
||||
The AuthenticatedUsers group gets READ access.
|
||||
bucket-owner-read: |-
|
||||
Object owner gets FULL_CONTROL.
|
||||
Bucket owner gets READ access.
|
||||
If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
|
||||
bucket-owner-full-control: |-
|
||||
Both the object owner and the bucket owner get FULL_CONTROL over the object.
|
||||
If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
|
||||
```
|
||||
- Identical across *most* providers. Select the default with `acl: {}`
|
||||
- Example from AWS.yaml
|
||||
|
||||
```yaml
|
||||
acl:
|
||||
private: |-
|
||||
Owner gets FULL_CONTROL.
|
||||
No one else has access rights (default).
|
||||
public-read: |-
|
||||
Owner gets FULL_CONTROL.
|
||||
The AllUsers group gets READ access.
|
||||
public-read-write: |-
|
||||
Owner gets FULL_CONTROL.
|
||||
The AllUsers group gets READ and WRITE access.
|
||||
Granting this on a bucket is generally not recommended.
|
||||
authenticated-read: |-
|
||||
Owner gets FULL_CONTROL.
|
||||
The AuthenticatedUsers group gets READ access.
|
||||
bucket-owner-read: |-
|
||||
Object owner gets FULL_CONTROL.
|
||||
Bucket owner gets READ access.
|
||||
If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
|
||||
bucket-owner-full-control: |-
|
||||
Both the object owner and the bucket owner get FULL_CONTROL over the object.
|
||||
If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
|
||||
```
|
||||
|
||||
- [Storage Class]
|
||||
- Identical across *most* providers.
|
||||
- Defaults from AWS.yaml
|
||||
```yaml
|
||||
storage_class:
|
||||
"": Default
|
||||
STANDARD: Standard storage class
|
||||
REDUCED_REDUNDANCY: Reduced redundancy storage class
|
||||
STANDARD_IA: Standard Infrequent Access storage class
|
||||
ONEZONE_IA: One Zone Infrequent Access storage class
|
||||
GLACIER: Glacier Flexible Retrieval storage class
|
||||
DEEP_ARCHIVE: Glacier Deep Archive storage class
|
||||
INTELLIGENT_TIERING: Intelligent-Tiering storage class
|
||||
GLACIER_IR: Glacier Instant Retrieval storage class
|
||||
```
|
||||
- Identical across *most* providers.
|
||||
- Defaults from AWS.yaml
|
||||
|
||||
```yaml
|
||||
storage_class:
|
||||
"": Default
|
||||
STANDARD: Standard storage class
|
||||
REDUCED_REDUNDANCY: Reduced redundancy storage class
|
||||
STANDARD_IA: Standard Infrequent Access storage class
|
||||
ONEZONE_IA: One Zone Infrequent Access storage class
|
||||
GLACIER: Glacier Flexible Retrieval storage class
|
||||
DEEP_ARCHIVE: Glacier Deep Archive storage class
|
||||
INTELLIGENT_TIERING: Intelligent-Tiering storage class
|
||||
GLACIER_IR: Glacier Instant Retrieval storage class
|
||||
```
|
||||
|
||||
- [Server Side Encryption]
|
||||
- Not common, identical across *most* providers.
|
||||
- Defaults from AWS.yaml
|
||||
```yaml
|
||||
server_side_encryption:
|
||||
"": None
|
||||
AES256: AES256
|
||||
aws:kms: aws:kms
|
||||
```
|
||||
- Not common, identical across *most* providers.
|
||||
- Defaults from AWS.yaml
|
||||
|
||||
```yaml
|
||||
server_side_encryption:
|
||||
"": None
|
||||
AES256: AES256
|
||||
aws:kms: aws:kms
|
||||
```
|
||||
|
||||
- [Advanced Options]
|
||||
- All advanced options are Boolean - if true the configurator asks about that value, if not it doesn't:
|
||||
```go
|
||||
BucketACL bool `yaml:"bucket_acl,omitempty"`
|
||||
DirectoryBucket bool `yaml:"directory_bucket,omitempty"`
|
||||
LeavePartsOnError bool `yaml:"leave_parts_on_error,omitempty"`
|
||||
RequesterPays bool `yaml:"requester_pays,omitempty"`
|
||||
SSECustomerAlgorithm bool `yaml:"sse_customer_algorithm,omitempty"`
|
||||
SSECustomerKey bool `yaml:"sse_customer_key,omitempty"`
|
||||
SSECustomerKeyBase64 bool `yaml:"sse_customer_key_base64,omitempty"`
|
||||
SSECustomerKeyMd5 bool `yaml:"sse_customer_key_md5,omitempty"`
|
||||
SSEKmsKeyID bool `yaml:"sse_kms_key_id,omitempty"`
|
||||
STSEndpoint bool `yaml:"sts_endpoint,omitempty"`
|
||||
UseAccelerateEndpoint bool `yaml:"use_accelerate_endpoint,omitempty"`
|
||||
```
|
||||
- Example from AWS.yaml:
|
||||
```yaml
|
||||
bucket_acl: true
|
||||
directory_bucket: true
|
||||
leave_parts_on_error: true
|
||||
requester_pays: true
|
||||
sse_customer_algorithm: true
|
||||
sse_customer_key: true
|
||||
sse_customer_key_base64: true
|
||||
sse_customer_key_md5: true
|
||||
sse_kms_key_id: true
|
||||
sts_endpoint: true
|
||||
use_accelerate_endpoint: true
|
||||
```
|
||||
- All advanced options are Boolean - if true the configurator asks about that
|
||||
value, if not it doesn't:
|
||||
|
||||
```go
|
||||
BucketACL bool `yaml:"bucket_acl,omitempty"`
|
||||
DirectoryBucket bool `yaml:"directory_bucket,omitempty"`
|
||||
LeavePartsOnError bool `yaml:"leave_parts_on_error,omitempty"`
|
||||
RequesterPays bool `yaml:"requester_pays,omitempty"`
|
||||
SSECustomerAlgorithm bool `yaml:"sse_customer_algorithm,omitempty"`
|
||||
SSECustomerKey bool `yaml:"sse_customer_key,omitempty"`
|
||||
SSECustomerKeyBase64 bool `yaml:"sse_customer_key_base64,omitempty"`
|
||||
SSECustomerKeyMd5 bool `yaml:"sse_customer_key_md5,omitempty"`
|
||||
SSEKmsKeyID bool `yaml:"sse_kms_key_id,omitempty"`
|
||||
STSEndpoint bool `yaml:"sts_endpoint,omitempty"`
|
||||
UseAccelerateEndpoint bool `yaml:"use_accelerate_endpoint,omitempty"`
|
||||
```
|
||||
|
||||
- Example from AWS.yaml:
|
||||
|
||||
```yaml
|
||||
bucket_acl: true
|
||||
directory_bucket: true
|
||||
leave_parts_on_error: true
|
||||
requester_pays: true
|
||||
sse_customer_algorithm: true
|
||||
sse_customer_key: true
|
||||
sse_customer_key_base64: true
|
||||
sse_customer_key_md5: true
|
||||
sse_kms_key_id: true
|
||||
sts_endpoint: true
|
||||
use_accelerate_endpoint: true
|
||||
```
|
||||
|
||||
- Quirks
|
||||
- Quirks are discovered through documentation and running the tests as seen below.
|
||||
- Most quirks are *bool as to have 3 values, `true`, `false` and `dont care`.
|
||||
```go
|
||||
type Quirks struct {
|
||||
ListVersion *int `yaml:"list_version,omitempty"` // 1 or 2
|
||||
ForcePathStyle *bool `yaml:"force_path_style,omitempty"` // true = path-style
|
||||
ListURLEncode *bool `yaml:"list_url_encode,omitempty"`
|
||||
UseMultipartEtag *bool `yaml:"use_multipart_etag,omitempty"`
|
||||
UseAlreadyExists *bool `yaml:"use_already_exists,omitempty"`
|
||||
UseAcceptEncodingGzip *bool `yaml:"use_accept_encoding_gzip,omitempty"`
|
||||
MightGzip *bool `yaml:"might_gzip,omitempty"`
|
||||
UseMultipartUploads *bool `yaml:"use_multipart_uploads,omitempty"`
|
||||
UseUnsignedPayload *bool `yaml:"use_unsigned_payload,omitempty"`
|
||||
UseXID *bool `yaml:"use_x_id,omitempty"`
|
||||
SignAcceptEncoding *bool `yaml:"sign_accept_encoding,omitempty"`
|
||||
CopyCutoff *int64 `yaml:"copy_cutoff,omitempty"`
|
||||
MaxUploadParts *int `yaml:"max_upload_parts,omitempty"`
|
||||
MinChunkSize *int64 `yaml:"min_chunk_size,omitempty"`
|
||||
}
|
||||
- Quirks are discovered through documentation and running the tests as seen below.
|
||||
- Most quirks are *bool as to have 3 values, `true`, `false` and `dont care`.
|
||||
|
||||
```go
|
||||
type Quirks struct {
|
||||
ListVersion *int `yaml:"list_version,omitempty"` // 1 or 2
|
||||
ForcePathStyle *bool `yaml:"force_path_style,omitempty"` // true = path-style
|
||||
ListURLEncode *bool `yaml:"list_url_encode,omitempty"`
|
||||
UseMultipartEtag *bool `yaml:"use_multipart_etag,omitempty"`
|
||||
UseAlreadyExists *bool `yaml:"use_already_exists,omitempty"`
|
||||
UseAcceptEncodingGzip *bool `yaml:"use_accept_encoding_gzip,omitempty"`
|
||||
MightGzip *bool `yaml:"might_gzip,omitempty"`
|
||||
UseMultipartUploads *bool `yaml:"use_multipart_uploads,omitempty"`
|
||||
UseUnsignedPayload *bool `yaml:"use_unsigned_payload,omitempty"`
|
||||
UseXID *bool `yaml:"use_x_id,omitempty"`
|
||||
SignAcceptEncoding *bool `yaml:"sign_accept_encoding,omitempty"`
|
||||
CopyCutoff *int64 `yaml:"copy_cutoff,omitempty"`
|
||||
MaxUploadParts *int `yaml:"max_upload_parts,omitempty"`
|
||||
MinChunkSize *int64 `yaml:"min_chunk_size,omitempty"`
|
||||
}
|
||||
```
|
||||
- Example from AWS.yaml
|
||||
|
||||
- Example from AWS.yaml
|
||||
|
||||
```yaml
|
||||
quirks:
|
||||
might_gzip: false # Never auto gzips objects
|
||||
use_unsigned_payload: false # AWS has trailer support
|
||||
quirks:
|
||||
might_gzip: false # Never auto gzips objects
|
||||
use_unsigned_payload: false # AWS has trailer support
|
||||
```
|
||||
|
||||
Note that if you omit a section, eg `region` then the user won't be
|
||||
asked that question, and if you add an empty section e.g. `region: {}`
|
||||
then the defaults from the `Other.yaml` will be used.
|
||||
|
||||
### DOCS
|
||||
## DOCS
|
||||
|
||||
- `docs/content/s3.md`
|
||||
- Add the provider at the top of the page.
|
||||
- Add a section about the provider linked from there.
|
||||
- Make sure this is in alphabetical order in the `Providers` section.
|
||||
- Add a transcript of a trial `rclone config` session
|
||||
- Edit the transcript to remove things which might change in subsequent versions
|
||||
- **Do not** alter or add to the autogenerated parts of `s3.md`
|
||||
- Rule of thumb: don't edit anything not mentioned above.
|
||||
- **Do not** run `make backenddocs` or `bin/make_backend_docs.py s3`
|
||||
- This will make autogenerated changes!
|
||||
- Add the provider at the top of the page.
|
||||
- Add a section about the provider linked from there.
|
||||
- Make sure this is in alphabetical order in the `Providers` section.
|
||||
- Add a transcript of a trial `rclone config` session
|
||||
- Edit the transcript to remove things which might change in subsequent versions
|
||||
- **Do not** alter or add to the autogenerated parts of `s3.md`
|
||||
- Rule of thumb: don't edit anything not mentioned above.
|
||||
- **Do not** run `make backenddocs` or `bin/make_backend_docs.py s3`
|
||||
- This will make autogenerated changes!
|
||||
- `README.md` - this is the home page in github
|
||||
- Add the provider and a link to the section you wrote in `docs/contents/s3.md`
|
||||
- Add the provider and a link to the section you wrote in `docs/contents/s3.md`
|
||||
- `docs/content/_index.md` - this is the home page of rclone.org
|
||||
- Add the provider and a link to the section you wrote in `docs/contents/s3.md`
|
||||
- Add the provider and a link to the section you wrote in `docs/contents/s3.md`
|
||||
- Once you've written the docs, run `make serve` and check they look OK
|
||||
in the web browser and the links (internal and external) all work.
|
||||
in the web browser and the links (internal and external) all work.
|
||||
|
||||
### TESTS
|
||||
## TESTS
|
||||
|
||||
Once you've written the code, test `rclone config` works to your
|
||||
satisfaction and looks correct, and check the integration tests work
|
||||
|
||||
@@ -78,7 +78,6 @@ List the contents of a bucket
|
||||
rclone ls remote:bucket
|
||||
```
|
||||
|
||||
|
||||
Sync `/home/local/directory` to the remote bucket, deleting any
|
||||
excess files in the bucket.
|
||||
|
||||
@@ -98,7 +97,7 @@ Follow Backblaze's docs to create an Application Key with the required
|
||||
permission and add the `applicationKeyId` as the `account` and the
|
||||
`Application Key` itself as the `key`.
|
||||
|
||||
Note that you must put the _applicationKeyId_ as the `account` – you
|
||||
Note that you must put the *applicationKeyId* as the `account` – you
|
||||
can't use the master Account ID. If you try then B2 will return 401
|
||||
errors.
|
||||
|
||||
@@ -192,7 +191,7 @@ You may opt in to a "hard delete" of files with the `--b2-hard-delete`
|
||||
flag which permanently removes files on deletion instead of hiding
|
||||
them.
|
||||
|
||||
Old versions of files, where available, are visible using the
|
||||
Old versions of files, where available, are visible using the
|
||||
`--b2-versions` flag.
|
||||
|
||||
It is also possible to view a bucket as it was at a certain point in time,
|
||||
|
||||
@@ -109,7 +109,9 @@ rclone ls remote:
|
||||
|
||||
To copy a local directory to an Box directory called backup
|
||||
|
||||
rclone copy /home/source remote:backup
|
||||
```console
|
||||
rclone copy /home/source remote:backup
|
||||
```
|
||||
|
||||
### Using rclone with an Enterprise account with SSO
|
||||
|
||||
@@ -249,8 +251,8 @@ either be actually deleted from Box or moved to the trash.
|
||||
|
||||
Emptying the trash is supported via the rclone however cleanup command
|
||||
however this deletes every trashed file and folder individually so it
|
||||
may take a very long time.
|
||||
Emptying the trash via the WebUI does not have this limitation
|
||||
may take a very long time.
|
||||
Emptying the trash via the WebUI does not have this limitation
|
||||
so it is advised to empty the trash via the WebUI.
|
||||
|
||||
### Root folder ID
|
||||
|
||||
@@ -707,9 +707,9 @@ the same in the new encrypted remote.
|
||||
|
||||
This will have the following advantages
|
||||
|
||||
* `rclone sync` will check the checksums while copying
|
||||
* you can use `rclone check` between the encrypted remotes
|
||||
* you don't decrypt and encrypt unnecessarily
|
||||
- `rclone sync` will check the checksums while copying
|
||||
- you can use `rclone check` between the encrypted remotes
|
||||
- you don't decrypt and encrypt unnecessarily
|
||||
|
||||
For example, let's say you have your original remote at `remote:` with
|
||||
the encrypted version at `eremote:` with path `remote:crypt`. You
|
||||
@@ -719,11 +719,15 @@ as `eremote:`.
|
||||
|
||||
To sync the two remotes you would do
|
||||
|
||||
rclone sync --interactive remote:crypt remote2:crypt
|
||||
```console
|
||||
rclone sync --interactive remote:crypt remote2:crypt
|
||||
```
|
||||
|
||||
And to check the integrity you would do
|
||||
|
||||
rclone check remote:crypt remote2:crypt
|
||||
```console
|
||||
rclone check remote:crypt remote2:crypt
|
||||
```
|
||||
|
||||
## File formats
|
||||
|
||||
@@ -734,8 +738,8 @@ has a header and is divided into chunks.
|
||||
|
||||
#### Header
|
||||
|
||||
* 8 bytes magic string `RCLONE\x00\x00`
|
||||
* 24 bytes Nonce (IV)
|
||||
- 8 bytes magic string `RCLONE\x00\x00`
|
||||
- 24 bytes Nonce (IV)
|
||||
|
||||
The initial nonce is generated from the operating systems crypto
|
||||
strong random number generator. The nonce is incremented for each
|
||||
@@ -753,8 +757,8 @@ authenticate messages.
|
||||
|
||||
Each chunk contains:
|
||||
|
||||
* 16 Bytes of Poly1305 authenticator
|
||||
* 1 - 65536 bytes XSalsa20 encrypted data
|
||||
- 16 Bytes of Poly1305 authenticator
|
||||
- 1 - 65536 bytes XSalsa20 encrypted data
|
||||
|
||||
64k chunk size was chosen as the best performing chunk size (the
|
||||
authenticator takes too much time below this and the performance drops
|
||||
@@ -767,15 +771,15 @@ This uses a 32 byte (256 bit key) key derived from the user password.
|
||||
|
||||
1 byte file will encrypt to
|
||||
|
||||
* 32 bytes header
|
||||
* 17 bytes data chunk
|
||||
- 32 bytes header
|
||||
- 17 bytes data chunk
|
||||
|
||||
49 bytes total
|
||||
|
||||
1 MiB (1048576 bytes) file will encrypt to
|
||||
|
||||
* 32 bytes header
|
||||
* 16 chunks of 65568 bytes
|
||||
- 32 bytes header
|
||||
- 16 chunks of 65568 bytes
|
||||
|
||||
1049120 bytes total (a 0.05% overhead). This is the overhead for big
|
||||
files.
|
||||
@@ -798,8 +802,8 @@ it on the cloud storage system.
|
||||
|
||||
This means that
|
||||
|
||||
* filenames with the same name will encrypt the same
|
||||
* filenames which start the same won't have a common prefix
|
||||
- filenames with the same name will encrypt the same
|
||||
- filenames which start the same won't have a common prefix
|
||||
|
||||
This uses a 32 byte key (256 bits) and a 16 byte (128 bits) IV both of
|
||||
which are derived from the user password.
|
||||
@@ -808,8 +812,8 @@ After encryption they are written out using a modified version of
|
||||
standard `base32` encoding as described in RFC4648. The standard
|
||||
encoding is modified in two ways:
|
||||
|
||||
* it becomes lower case (no-one likes upper case filenames!)
|
||||
* we strip the padding character `=`
|
||||
- it becomes lower case (no-one likes upper case filenames!)
|
||||
- we strip the padding character `=`
|
||||
|
||||
`base32` is used rather than the more efficient `base64` so rclone can be
|
||||
used on case insensitive remotes (e.g. Windows, Box, Dropbox, Onedrive etc).
|
||||
@@ -825,6 +829,7 @@ then rclone uses an internal one.
|
||||
encrypted data. For full protection against this you should always use
|
||||
a salt.
|
||||
|
||||
## SEE ALSO
|
||||
## See Also
|
||||
|
||||
* [rclone cryptdecode](/commands/rclone_cryptdecode/) - Show forward/reverse mapping of encrypted filenames
|
||||
- [rclone cryptdecode](/commands/rclone_cryptdecode/) - Show forward/reverse
|
||||
mapping of encrypted filenames.
|
||||
|
||||
@@ -604,10 +604,9 @@ issue an error message `File name disallowed - not uploading` if it
|
||||
attempts to upload one of those file names, but the sync won't fail.
|
||||
|
||||
Some errors may occur if you try to sync copyright-protected files
|
||||
because Dropbox has its own [copyright detector](https://techcrunch.com/2014/03/30/how-dropbox-knows-when-youre-sharing-copyrighted-stuff-without-actually-looking-at-your-stuff/) that
|
||||
prevents this sort of file being downloaded. This will return the error `ERROR :
|
||||
/path/to/your/file: Failed to copy: failed to open source object:
|
||||
path/restricted_content/.`
|
||||
because Dropbox has its own [copyright detector](https://techcrunch.com/2014/03/30/how-dropbox-knows-when-youre-sharing-copyrighted-stuff-without-actually-looking-at-your-stuff/)
|
||||
that prevents this sort of file being downloaded. This will return the error
|
||||
`ERROR : /path/to/your/file: Failed to copy: failed to open source object: path/restricted_content/.`
|
||||
|
||||
If you have more than 10,000 files in a directory then `rclone purge
|
||||
dropbox:dir` will return the error `Failed to purge: There are too
|
||||
@@ -617,7 +616,8 @@ many files involved in this operation`. As a work-around do an
|
||||
When using `rclone link` you'll need to set `--expire` if using a
|
||||
non-personal account otherwise the visibility may not be correct.
|
||||
(Note that `--expire` isn't supported on personal accounts). See the
|
||||
[forum discussion](https://forum.rclone.org/t/rclone-link-dropbox-permissions/23211) and the
|
||||
[forum discussion](https://forum.rclone.org/t/rclone-link-dropbox-permissions/23211)
|
||||
and the
|
||||
[dropbox SDK issue](https://github.com/dropbox/dropbox-sdk-go-unofficial/issues/75).
|
||||
|
||||
Modification times for Dropbox Paper documents are not exact, and
|
||||
@@ -627,23 +627,34 @@ or so, or use `--ignore-times` to force a full sync.
|
||||
|
||||
## Get your own Dropbox App ID
|
||||
|
||||
When you use rclone with Dropbox in its default configuration you are using rclone's App ID. This is shared between all the rclone users.
|
||||
When you use rclone with Dropbox in its default configuration you are using
|
||||
rclone's App ID. This is shared between all the rclone users.
|
||||
|
||||
Here is how to create your own Dropbox App ID for rclone:
|
||||
|
||||
1. Log into the [Dropbox App console](https://www.dropbox.com/developers/apps/create) with your Dropbox Account (It need not
|
||||
to be the same account as the Dropbox you want to access)
|
||||
1. Log into the [Dropbox App console](https://www.dropbox.com/developers/apps/create)
|
||||
with your Dropbox Account (It need not to be the same account as the Dropbox you
|
||||
want to access)
|
||||
|
||||
2. Choose an API => Usually this should be `Dropbox API`
|
||||
|
||||
3. Choose the type of access you want to use => `Full Dropbox` or `App Folder`. If you want to use Team Folders, `Full Dropbox` is required ([see here](https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/How-to-create-team-folder-inside-my-app-s-folder/m-p/601005/highlight/true#M27911)).
|
||||
3. Choose the type of access you want to use => `Full Dropbox` or `App Folder`.
|
||||
If you want to use Team Folders, `Full Dropbox` is required
|
||||
([see here](https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/How-to-create-team-folder-inside-my-app-s-folder/m-p/601005/highlight/true#M27911)).
|
||||
|
||||
4. Name your App. The app name is global, so you can't use `rclone` for example
|
||||
|
||||
5. Click the button `Create App`
|
||||
|
||||
6. Switch to the `Permissions` tab. Enable at least the following permissions: `account_info.read`, `files.metadata.write`, `files.content.write`, `files.content.read`, `sharing.write`. The `files.metadata.read` and `sharing.read` checkboxes will be marked too. Click `Submit`
|
||||
6. Switch to the `Permissions` tab. Enable at least the following permissions:
|
||||
`account_info.read`, `files.metadata.write`, `files.content.write`, `files.content.read`,
|
||||
`sharing.write`. The `files.metadata.read` and `sharing.read` checkboxes will be
|
||||
marked too. Click `Submit`
|
||||
|
||||
7. Switch to the `Settings` tab. Fill `OAuth2 - Redirect URIs` as `http://localhost:53682/` and click on `Add`
|
||||
7. Switch to the `Settings` tab. Fill `OAuth2 - Redirect URIs` as `http://localhost:53682/`
|
||||
and click on `Add`
|
||||
|
||||
8. Find the `App key` and `App secret` values on the `Settings` tab. Use these values in rclone config to add a new remote or edit an existing remote. The `App key` setting corresponds to `client_id` in rclone config, the `App secret` corresponds to `client_secret`
|
||||
8. Find the `App key` and `App secret` values on the `Settings` tab. Use these
|
||||
values in rclone config to add a new remote or edit an existing remote.
|
||||
The `App key` setting corresponds to `client_id` in rclone config, the
|
||||
`App secret` corresponds to `client_secret`
|
||||
|
||||
@@ -147,7 +147,9 @@ To create your own Client ID, please follow these steps:
|
||||
- If you have not created an Azure account, you will be prompted to. This is free,
|
||||
but you need to provide a phone number, address, and credit card for identity
|
||||
verification.
|
||||
2. Enter a name for your app, choose account type `Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)`,
|
||||
2. Enter a name for your app, choose account type `Accounts in any organizational
|
||||
directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts
|
||||
(e.g. Skype, Xbox)`,
|
||||
select `Web` in `Redirect URI`, then type (do not copy and paste)
|
||||
`http://localhost:53682/` and click Register. Copy and keep the
|
||||
`Application (client) ID` under the app name for later use.
|
||||
@@ -170,7 +172,8 @@ The access_scopes option allows you to configure the permissions requested by rc
|
||||
See [Microsoft Docs](https://docs.microsoft.com/en-us/graph/permissions-reference#files-permissions)
|
||||
for more information about the different scopes.
|
||||
|
||||
The `Sites.Read.All` permission is required if you need to [search SharePoint sites when configuring the remote](https://github.com/rclone/rclone/pull/5883).
|
||||
The `Sites.Read.All` permission is required if you need to
|
||||
[search SharePoint sites when configuring the remote](https://github.com/rclone/rclone/pull/5883).
|
||||
However, if that permission is not assigned, you need to exclude `Sites.Read.All`
|
||||
from your access scopes or set `disable_site_permission` option to true in the
|
||||
advanced options.
|
||||
@@ -1109,7 +1112,8 @@ setting:
|
||||
1. `Install-Module -Name Microsoft.Online.SharePoint.PowerShell` (in case you
|
||||
haven't installed this already)
|
||||
2. `Import-Module Microsoft.Online.SharePoint.PowerShell -DisableNameChecking`
|
||||
3. `Connect-SPOService -Url https://YOURSITE-admin.sharepoint.com -Credential YOU@YOURSITE.COM` (replacing `YOURSITE`, `YOU`, `YOURSITE.COM` with the actual values; this will
|
||||
3. `Connect-SPOService -Url https://YOURSITE-admin.sharepoint.com -Credential YOU@YOURSITE.COM`
|
||||
(replacing `YOURSITE`, `YOU`, `YOURSITE.COM` with the actual values; this will
|
||||
prompt for your credentials)
|
||||
4. `Set-SPOTenant -EnableMinimumVersionRequirement $False`
|
||||
5. `Disconnect-SPOService` (to disconnect from the server)
|
||||
|
||||
@@ -156,10 +156,10 @@ rclone ls remote:bucket
|
||||
rclone ls remote:bucket --max-depth 1
|
||||
```
|
||||
|
||||
## Authentication Providers
|
||||
## Authentication Providers
|
||||
|
||||
OCI has various authentication methods. To learn more about authentication methods please refer [oci authentication
|
||||
methods](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdk_authentication_methods.htm)
|
||||
OCI has various authentication methods. To learn more about authentication methods
|
||||
please refer [oci authentication methods](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdk_authentication_methods.htm)
|
||||
These choices can be specified in the rclone config file.
|
||||
|
||||
Rclone supports the following OCI authentication provider.
|
||||
@@ -900,4 +900,5 @@ Options:
|
||||
{{< rem autogenerated options stop >}}
|
||||
|
||||
## Tutorials
|
||||
|
||||
### [Mounting Buckets](/oracleobjectstorage/tutorial_mount/)
|
||||
|
||||
@@ -41,7 +41,7 @@ From repo : ol8_developer
|
||||
Summary : rsync for cloud storage
|
||||
URL : http://rclone.org/
|
||||
License : MIT
|
||||
Description : Rclone is a command line program to sync files and directories to and from various cloud services.
|
||||
Description : Rclone is a command line program to sync files and directories to and from various cloud services.
|
||||
```
|
||||
|
||||
To run it as a mount helper you should symlink rclone binary to /sbin/mount.rclone
|
||||
|
||||
@@ -161,15 +161,17 @@ The command will enumerate available directories, allowing you to locate the
|
||||
appropriate Folder ID for subsequent use.
|
||||
|
||||
Example:
|
||||
```
|
||||
|
||||
```console
|
||||
$ rclone lsf --dirs-only -Fip --csv TestPcloud:
|
||||
dxxxxxxxx2,My Music/
|
||||
dxxxxxxxx3,My Pictures/
|
||||
dxxxxxxxx4,My Videos/
|
||||
```
|
||||
|
||||
So if the folder you want rclone to use your is "My Music/", then use the returned id from ```rclone lsf``` command (ex. `dxxxxxxxx2`) as
|
||||
the `root_folder_id` variable value in the config file.
|
||||
So if the folder you want rclone to use your is "My Music/", then use the returned
|
||||
id from ```rclone lsf``` command (ex. `dxxxxxxxx2`) as the `root_folder_id` variable
|
||||
value in the config file.
|
||||
|
||||
{{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/pcloud/pcloud.go then run make backenddocs" >}}
|
||||
### Standard options
|
||||
|
||||
@@ -16,7 +16,8 @@ three ways of doing it, described below.
|
||||
## Configuring using rclone authorize
|
||||
|
||||
On the headless machine run [rclone config](/commands/rclone_config), but
|
||||
answer `N` to the question `Use web browser to automatically authenticate rclone with remote?`.
|
||||
answer `N` to the question `Use web browser to automatically authenticate
|
||||
rclone with remote?`.
|
||||
|
||||
```text
|
||||
Use web browser to automatically authenticate rclone with remote?
|
||||
@@ -108,7 +109,8 @@ ssh -L localhost:53682:localhost:53682 username@remote_server
|
||||
```
|
||||
|
||||
Then on the headless machine run [rclone config](/commands/rclone_config) and
|
||||
answer `Y` to the question `Use web browser to automatically authenticate rclone with remote?`.
|
||||
answer `Y` to the question `Use web browser to automatically authenticate rclone
|
||||
with remote?`.
|
||||
|
||||
```text
|
||||
Use web browser to automatically authenticate rclone with remote?
|
||||
|
||||
@@ -531,7 +531,7 @@ installed and configured, an access token is obtained by running the
|
||||
`oidc-token` command. The following example shows a (shortened)
|
||||
access token obtained from the *XDC* OIDC Provider.
|
||||
|
||||
```
|
||||
```text
|
||||
paul@celebrimbor:~$ oidc-token XDC
|
||||
eyJraWQ[...]QFXDt0
|
||||
paul@celebrimbor:~$
|
||||
@@ -555,7 +555,7 @@ edit the advanced config and enter the command to get a bearer token
|
||||
The following example config shows a WebDAV endpoint that uses
|
||||
oidc-agent to supply an access token from the *XDC* OIDC Provider.
|
||||
|
||||
```
|
||||
```ini
|
||||
[dcache]
|
||||
type = webdav
|
||||
url = https://dcache.example.org/
|
||||
|
||||
@@ -16,10 +16,11 @@ This contains the following files
|
||||
|
||||
## Compiling
|
||||
|
||||
This can be compiled by using `make` or alternatively `GOARCH=wasm GOOS=js go build -o rclone.wasm`
|
||||
This can be compiled by using `make` or alternatively
|
||||
`GOARCH=wasm GOOS=js go build -o rclone.wasm`.
|
||||
|
||||
## Running
|
||||
|
||||
Run the test server with `make serve` and examine the page at
|
||||
http://localhost:3000/ - look at the javascript console and look at
|
||||
<http://localhost:3000/> - look at the javascript console and look at
|
||||
the end of `loader.js` for how that works.
|
||||
|
||||
@@ -13,18 +13,22 @@ The shims are a thin wrapper over the rclone RPC.
|
||||
|
||||
The implementation is based on cgo; to build it you need Go and a GCC compatible
|
||||
C compiler (GCC or Clang). On Windows you can use the MinGW ports, e.g. by installing
|
||||
in a [MSYS2](https://www.msys2.org) distribution (you may now install GCC in the newer
|
||||
and recommended UCRT64 subsystem, however there were compatibility issues with previous
|
||||
versions of cgo where, if not force rebuild with go build option `-a` helped, you had
|
||||
to resort to the classic MINGW64 subsystem).
|
||||
in a [MSYS2](https://www.msys2.org) distribution (you may now install GCC in the
|
||||
newer and recommended UCRT64 subsystem, however there were compatibility issues
|
||||
with previous versions of cgo where, if not force rebuild with go build option `-a`
|
||||
helped, you had to resort to the classic MINGW64 subsystem).
|
||||
|
||||
Build a shared library like this (change from .so to .dll on Windows):
|
||||
|
||||
go build --buildmode=c-shared -o librclone.so github.com/rclone/rclone/librclone
|
||||
```console
|
||||
go build --buildmode=c-shared -o librclone.so github.com/rclone/rclone/librclone
|
||||
```
|
||||
|
||||
Build a static library like this (change from .a to .lib on Windows):
|
||||
|
||||
go build --buildmode=c-archive -o librclone.a github.com/rclone/rclone/librclone
|
||||
```console
|
||||
go build --buildmode=c-archive -o librclone.a github.com/rclone/rclone/librclone
|
||||
```
|
||||
|
||||
Both the above commands will also generate `librclone.h` which should
|
||||
be `#include`d in `C` programs wishing to use the library (with some
|
||||
@@ -50,7 +54,7 @@ On Windows, when you build a shared library, you can embed version information
|
||||
as binary resource. To do that you need to run the following command **before**
|
||||
the build command.
|
||||
|
||||
```
|
||||
```console
|
||||
go run bin/resource_windows.go -binary librclone.dll -dir librclone
|
||||
```
|
||||
|
||||
@@ -75,12 +79,19 @@ The official [C example](#linux-c-example) is targeting Linux/macOS, and will
|
||||
not work on Windows. It is very possible to use `librclone` from a C/C++
|
||||
application on Windows, but there are some pitfalls that you can avoid by
|
||||
following these guidelines:
|
||||
- Build `librclone` as shared library, and use run-time dynamic linking (see [linking](#linking)).
|
||||
- Do not try to unload the library with `FreeLibrary` (see [unloading](#unloading)).
|
||||
- Deallocate returned strings with API function `RcloneFreeString` (see [memory management](#memory-management)).
|
||||
- Define struct `RcloneRPCResult`, instead of including `librclone.h` (see [include file](#include-file)).
|
||||
- Use UTF-8 encoded strings (see [encoding](#encoding)).
|
||||
- Properly escape JSON strings, beware of the native path separator (see [escaping](#escaping)).
|
||||
|
||||
- Build `librclone` as shared library, and use run-time dynamic linking
|
||||
(see [linking](#linking)).
|
||||
- Do not try to unload the library with `FreeLibrary`
|
||||
(see [unloading](#unloading)).
|
||||
- Deallocate returned strings with API function `RcloneFreeString`
|
||||
(see [memory management](#memory-management)).
|
||||
- Define struct `RcloneRPCResult`, instead of including `librclone.h`
|
||||
(see [include file](#include-file)).
|
||||
- Use UTF-8 encoded strings
|
||||
(see [encoding](#encoding)).
|
||||
- Properly escape JSON strings, beware of the native path separator
|
||||
(see [escaping](#escaping)).
|
||||
|
||||
#### Linking
|
||||
|
||||
@@ -133,7 +144,7 @@ small struct `RcloneRPCResult`, from [librclone.go](librclone.go):
|
||||
```c++
|
||||
struct RcloneRPCResult {
|
||||
char* Output;
|
||||
int Status;
|
||||
int Status;
|
||||
};
|
||||
```
|
||||
|
||||
@@ -174,7 +185,7 @@ constants, leaving escaping only necessary for the contained JSON.
|
||||
|
||||
## Example in golang
|
||||
|
||||
Here is a go example to help you move files :
|
||||
Here is a go example to help you move files:
|
||||
|
||||
```go
|
||||
func main() {
|
||||
@@ -188,11 +199,10 @@ func main() {
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
}
|
||||
|
||||
|
||||
out, status: = librclone.RPC("sync/copy", string(syncRequestJSON))
|
||||
fmt.Println("Got status : %d and output %q", status, out)
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
## gomobile
|
||||
@@ -201,7 +211,9 @@ The `gomobile` subdirectory contains the equivalent of the C binding but
|
||||
suitable for using with [gomobile](https://pkg.go.dev/golang.org/x/mobile/cmd/gomobile)
|
||||
using something like this.
|
||||
|
||||
gomobile bind -v -target=android -javapkg=org.rclone github.com/rclone/rclone/librclone/gomobile
|
||||
```console
|
||||
gomobile bind -v -target=android -javapkg=org.rclone github.com/rclone/rclone/librclone/gomobile
|
||||
```
|
||||
|
||||
The command generates an Android library (`aar`) that can be imported
|
||||
into an Android application project. Librclone will be contained
|
||||
@@ -233,8 +245,10 @@ Further docs:
|
||||
|
||||
- [gomobile main website](https://pkg.go.dev/golang.org/x/mobile/cmd/gomobile)
|
||||
- [gomobile wiki](https://github.com/golang/go/wiki/Mobile)
|
||||
- [go issue #16876](https://github.com/golang/go/issues/16876) where the feature was added
|
||||
- [gomobile design doc](https://docs.google.com/document/d/1y9hStonl9wpj-5VM-xWrSTuEJFUAxGOXOhxvAs7GZHE/edit) for extra details not in the docs.
|
||||
- [go issue #16876](https://github.com/golang/go/issues/16876) where the feature
|
||||
was added
|
||||
- [gomobile design doc](https://docs.google.com/document/d/1y9hStonl9wpj-5VM-xWrSTuEJFUAxGOXOhxvAs7GZHE/edit)
|
||||
for extra details not in the docs.
|
||||
|
||||
## python
|
||||
|
||||
@@ -247,19 +261,20 @@ This needs expanding and submitting to pypi...
|
||||
|
||||
## Rust
|
||||
|
||||
Rust bindings are available in the `librclone` crate: https://crates.io/crates/librclone
|
||||
Rust bindings are available in the `librclone` crate: <https://crates.io/crates/librclone>
|
||||
|
||||
## PHP
|
||||
|
||||
The `php` subdirectory contains how to use the C library librclone in php through foreign
|
||||
function interface (FFI).
|
||||
The `php` subdirectory contains how to use the C library librclone in php through
|
||||
foreign function interface (FFI).
|
||||
|
||||
Useful docs:
|
||||
|
||||
- [PHP / FFI](https://www.php.net/manual/en/book.ffi.php)
|
||||
|
||||
## TODO
|
||||
|
||||
- Async jobs must currently be cancelled manually at the moment - RcloneFinalize doesn't do it.
|
||||
- Async jobs must currently be cancelled manually at the moment - RcloneFinalize
|
||||
doesn't do it.
|
||||
- This will use the rclone config system and rclone logging system.
|
||||
- Need examples showing how to configure things,
|
||||
|
||||
|
||||
Reference in New Issue
Block a user