- use rclone's http Transport
- fix handling of 0 length files
- combine into one file and remove uneeded abstraction
- make `chunk_size` and `upload_concurrency` settable
- make auth the same as azureblob
- set the Features correctly
- implement `--azurefiles-max-stream-size`
- remove arbitrary sleep on Mkdir
- implement `--header-upload`
- implement read and write MimeType for objects
- implement optional methods
- About
- Copy
- DirMove
- Move
- OpenWriterAt
- PutStream
- finish documentation
- disable build on plan9 and js
Fixes#365Fixes#7378
Stored as azure standard `LastModified` time stored on files
The modified time is stored as Azure standard `LastModified` time on
files
### Performance
When uploading large files, increasing the value of
`--azurefiles-upload-concurrency` will increase performance at the cost
of using more memory. The default of 16 is set quite conservatively to
use less memory. It maybe be necessary raise it to 64 or higher to
fully utilize a 1 GBit/s link with a single file transfer.
### Restricted filename characters
In addition to the [default restricted characters set](/overview/#restricted-characters)
the following characters are also replaced:
| Character | Value | Replacement |
| --------- |:-----:|:-----------:|
| " | 0x22 | ＂ |
| * | 0x2A | ＊ |
| : | 0x3A | ： |
| < | 0x3C | ＜ |
| > | 0x3E | ＞ |
| ? | 0x3F | ？ |
| \ | 0x5C | ＼ |
| \| | 0x7C | ｜ |
File names can also not end with the following characters.
These only get replaced if they are the last character in the name:
| Character | Value | Replacement |
| --------- |:-----:|:-----------:|
| . | 0x2E | ． |
Invalid UTF-8 bytes will also be [replaced](/overview/#invalid-utf8),
as they can't be used in JSON strings.
### Hashes
MD5 hashes are stored with files.
MD5 hashes are stored with files. Not all files will have MD5 hashes
as these have to be uploaded with the file.
### Authentication {#authentication}
1. ConnectionString
2. Accout and Key
3. SAS URL
There are a number of ways of supplying credentials for Azure Files
Storage. Rclone tries them in the order of the sections below.
#### Env Auth
If the `env_auth` config parameter is `true` then rclone will pull
credentials from the environment or runtime.
It tries these authentication methods in this order:
1. Environment Variables
2. Managed Service Identity Credentials
3. Azure CLI credentials (as used by the az tool)
These are described in the following sections
##### Env Auth: 1. Environment Variables
If `env_auth` is set and environment variables are present rclone
authenticates a service principal with a secret or certificate, or a
user with a password, depending on which environment variable are set.
It reads configuration from these variables, in the following order:
1. Service principal with client secret
-`AZURE_TENANT_ID`: ID of the service principal's tenant. Also called its "directory" ID.
-`AZURE_CLIENT_ID`: the service principal's client ID
-`AZURE_CLIENT_SECRET`: one of the service principal's client secrets
2. Service principal with certificate
-`AZURE_TENANT_ID`: ID of the service principal's tenant. Also called its "directory" ID.
-`AZURE_CLIENT_ID`: the service principal's client ID
-`AZURE_CLIENT_CERTIFICATE_PATH`: path to a PEM or PKCS12 certificate file including the private key.
-`AZURE_CLIENT_CERTIFICATE_PASSWORD`: (optional) password for the certificate file.
-`AZURE_CLIENT_SEND_CERTIFICATE_CHAIN`: (optional) Specifies whether an authentication request will include an x5c header to support subject name / issuer based authentication. When set to "true" or "1", authentication requests include the x5c header.
3. User with username and password
-`AZURE_TENANT_ID`: (optional) tenant to authenticate in. Defaults to "organizations".
-`AZURE_CLIENT_ID`: client ID of the application the user will authenticate to
-`AZURE_USERNAME`: a username (usually an email address)
-`AZURE_PASSWORD`: the user's password
4. Workload Identity
-`AZURE_TENANT_ID`: Tenant to authenticate in.
-`AZURE_CLIENT_ID`: Client ID of the application the user will authenticate to.
-`AZURE_FEDERATED_TOKEN_FILE`: Path to projected service account token file.
-`AZURE_AUTHORITY_HOST`: Authority of an Azure Active Directory endpoint (default: login.microsoftonline.com).
##### Env Auth: 2. Managed Service Identity Credentials
When using Managed Service Identity if the VM(SS) on which this
program is running has a system-assigned identity, it will be used by
default. If the resource has no system-assigned but exactly one
user-assigned identity, the user-assigned identity will be used by
default.
If the resource has multiple user-assigned identities you will need to
unset `env_auth` and set `use_msi` instead. See the [`use_msi`
section](#use_msi).
##### Env Auth: 3. Azure CLI credentials (as used by the az tool)
Credentials created with the `az` tool can be picked up using `env_auth`.
For example if you were to login with a service principal like this:
az login --service-principal -u XXX -p XXX --tenant XXX
This is the most straight forward and least flexible way. Just fill
in the `account` and `key` lines and leave the rest blank.
#### SAS URL
To use it leave `account`, `key` and `connection_string` blank and fill in `sas_url`.
#### Connection String
To use it leave `account`, `key` and "sas_url" blank and fill in `connection_string`.
#### Service principal with client secret
If these variables are set, rclone will authenticate with a service principal with a client secret.
-`tenant`: ID of the service principal's tenant. Also called its "directory" ID.
-`client_id`: the service principal's client ID
-`client_secret`: one of the service principal's client secrets
The credentials can also be placed in a file using the
`service_principal_file` configuration option.
#### Service principal with certificate
If these variables are set, rclone will authenticate with a service principal with certificate.
-`tenant`: ID of the service principal's tenant. Also called its "directory" ID.
-`client_id`: the service principal's client ID
-`client_certificate_path`: path to a PEM or PKCS12 certificate file including the private key.
-`client_certificate_password`: (optional) password for the certificate file.
-`client_send_certificate_chain`: (optional) Specifies whether an authentication request will include an x5c header to support subject name / issuer based authentication. When set to "true" or "1", authentication requests include the x5c header.
**NB**`client_certificate_password` must be obscured - see [rclone obscure](/commands/rclone_obscure/).
#### User with username and password
If these variables are set, rclone will authenticate with username and password.
-`tenant`: (optional) tenant to authenticate in. Defaults to "organizations".
-`client_id`: client ID of the application the user will authenticate to
-`username`: a username (usually an email address)
-`password`: the user's password
Microsoft doesn't recommend this kind of authentication, because it's
less secure than other authentication flows. This method is not
interactive, so it isn't compatible with any form of multi-factor
authentication, and the application must already have user or admin
consent. This credential can only authenticate work and school
accounts; it can't authenticate Microsoft accounts.
**NB**`password` must be obscured - see [rclone obscure](/commands/rclone_obscure/).
#### Managed Service Identity Credentials {#use_msi}
If `use_msi` is set then managed service identity credentials are
used. This authentication only works when running in an Azure service.
`env_auth` needs to be unset to use this.
However if you have multiple user identities to choose from these must
be explicitly specified using exactly one of the `msi_object_id`,
`msi_client_id`, or `msi_mi_res_id` parameters.
If none of `msi_object_id`, `msi_client_id`, or `msi_mi_res_id` is
See ["Create an Azure service principal"](https://docs.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli) and ["Assign an Azure role for access to files data"](https://docs.microsoft.com/en-us/azure/storage/common/storage-auth-aad-rbac-cli) pages for more details.
**NB** this section needs updating for Azure Files - pull requests appreciated!
It may be more convenient to put the credentials directly into the
rclone config file under the `client_id`, `tenant` and `client_secret`
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Nick Craig-Wood