jgaunt/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2025-12-06 00:03:32 +00:00
Code Issues Projects Releases Wiki Activity

onedrive: add metadata support

Browse Source 
This change adds support for metadata on OneDrive. Metadata (including
permissions) is supported for both files and directories.

OneDrive supports System Metadata (not User Metadata, as of this writing.) Much
of the metadata is read-only, and there are some differences between OneDrive
Personal and Business (see table in OneDrive backend docs for details).

Permissions are also supported, if --onedrive-metadata-permissions is set. The
accepted values for --onedrive-metadata-permissions are read, write, read,write, and
off (the default). write supports adding new permissions, updating the "role" of
existing permissions, and removing permissions. Updating and removing require
the Permission ID to be known, so it is recommended to use read,write instead of
write if you wish to update/remove permissions.

Permissions are read/written in JSON format using the same schema as the
OneDrive API, which differs slightly between OneDrive Personal and Business.
(See OneDrive backend docs for examples.)

To write permissions, pass in a "permissions" metadata key using this same
format. The --metadata-mapper tool can be very helpful for this.

When adding permissions, an email address can be provided in the User.ID or
DisplayName properties of grantedTo or grantedToIdentities. Alternatively, an
ObjectID can be provided in User.ID. At least one valid recipient must be
provided in order to add a permission for a user. Creating a Public Link is also
supported, if Link.Scope is set to "anonymous".

Note that adding a permission can fail if a conflicting permission already
exists for the file/folder.

To update an existing permission, include both the Permission ID and the new
roles to be assigned. roles is the only property that can be changed.

To remove permissions, pass in a blob containing only the permissions you wish
to keep (which can be empty, to remove all.)

Note that both reading and writing permissions requires extra API calls, so if
you don't need to read or write permissions it is recommended to omit --onedrive-
metadata-permissions.

Metadata and permissions are supported for Folders (directories) as well as
Files. Note that setting the mtime or btime on a Folder requires one extra API
call on OneDrive Business only.

OneDrive does not currently support User Metadata. When writing metadata, only
writeable system properties will be written -- any read-only or unrecognized keys
passed in will be ignored.

TIP: to see the metadata and permissions for any file or folder, run:

rclone lsjson remote:path --stat -M --onedrive-metadata-permissions read

See the OneDrive backend docs for a table of all the supported metadata
properties.
This commit is contained in:
nielash
2024-02-22 09:17:14 -05:00
committed by Nick Craig-Wood
parent 4e07a72dc7
commit 1473de3f04
8 changed files with 2093 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

209
docs/content/onedrive.md
View File

@@ -657,6 +657,30 @@ Properties:
- Type: bool
- Default: false
#### --onedrive-metadata-permissions
Control whether permissions should be read or written in metadata.
Reading permissions metadata from files can be done quickly, but it
isn't always desirable to set the permissions from the metadata.
Properties:
- Config: metadata_permissions
- Env Var: RCLONE_ONEDRIVE_METADATA_PERMISSIONS
- Type: Bits
- Default: off
- Examples:
- "off"
- Do not read or write the value
- "read"
- Read the value only
- "write"
- Write the value only
- "read,write"
- Read and Write the value.
#### --onedrive-encoding
The encoding for the backend.
@@ -670,6 +694,191 @@ Properties:
- Type: Encoding
- Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot
#### --onedrive-description
Description of the remote
Properties:
- Config: description
- Env Var: RCLONE_ONEDRIVE_DESCRIPTION
- Type: string
- Required: false
### Metadata
OneDrive supports System Metadata (not User Metadata, as of this writing) for
both files and directories. Much of the metadata is read-only, and there are some
differences between OneDrive Personal and Business (see table below for
details).
Permissions are also supported, if `--onedrive-metadata-permissions` is set. The
accepted values for `--onedrive-metadata-permissions` are `read`, `write`,
`read,write`, and `off` (the default). `write` supports adding new permissions,
updating the "role" of existing permissions, and removing permissions. Updating
and removing require the Permission ID to be known, so it is recommended to use
`read,write` instead of `write` if you wish to update/remove permissions.
Permissions are read/written in JSON format using the same schema as the
[OneDrive API](https://learn.microsoft.com/en-us/onedrive/developer/rest-api/resources/permission?view=odsp-graph-online),
which differs slightly between OneDrive Personal and Business.
Example for OneDrive Personal:
```json
[
{
"id": "1234567890ABC!123",
"grantedTo": {
"user": {
"id": "ryan@contoso.com"
},
"application": {},
"device": {}
},
"invitation": {
"email": "ryan@contoso.com"
},
"link": {
"webUrl": "https://1drv.ms/t/s!1234567890ABC"
},
"roles": [
"read"
],
"shareId": "s!1234567890ABC"
}
]
```
Example for OneDrive Business:
```json
[
{
"id": "48d31887-5fad-4d73-a9f5-3c356e68a038",
"grantedToIdentities": [
{
"user": {
"displayName": "ryan@contoso.com"
},
"application": {},
"device": {}
}
],
"link": {
"type": "view",
"scope": "users",
"webUrl": "https://contoso.sharepoint.com/:w:/t/design/a577ghg9hgh737613bmbjf839026561fmzhsr85ng9f3hjck2t5s"
},
"roles": [
"read"
],
"shareId": "u!LKj1lkdlals90j1nlkascl"
},
{
"id": "5D33DD65C6932946",
"grantedTo": {
"user": {
"displayName": "John Doe",
"id": "efee1b77-fb3b-4f65-99d6-274c11914d12"
},
"application": {},
"device": {}
},
"roles": [
"owner"
],
"shareId": "FWxc1lasfdbEAGM5fI7B67aB5ZMPDMmQ11U"
}
]
```
To write permissions, pass in a "permissions" metadata key using this same
format. The [`--metadata-mapper`](https://rclone.org/docs/#metadata-mapper) tool can
be very helpful for this.
When adding permissions, an email address can be provided in the `User.ID` or
`DisplayName` properties of `grantedTo` or `grantedToIdentities`. Alternatively,
an ObjectID can be provided in `User.ID`. At least one valid recipient must be
provided in order to add a permission for a user. Creating a Public Link is also
supported, if `Link.Scope` is set to `"anonymous"`.
Example request to add a "read" permission:
```json
[
{
"id": "",
"grantedTo": {
"user": {},
"application": {},
"device": {}
},
"grantedToIdentities": [
{
"user": {
"id": "ryan@contoso.com"
},
"application": {},
"device": {}
}
],
"roles": [
"read"
]
}
]
```
Note that adding a permission can fail if a conflicting permission already
exists for the file/folder.
To update an existing permission, include both the Permission ID and the new
`roles` to be assigned. `roles` is the only property that can be changed.
To remove permissions, pass in a blob containing only the permissions you wish
to keep (which can be empty, to remove all.)
Note that both reading and writing permissions requires extra API calls, so if
you don't need to read or write permissions it is recommended to omit
`--onedrive-metadata-permissions`.
Metadata and permissions are supported for Folders (directories) as well as
Files. Note that setting the `mtime` or `btime` on a Folder requires one extra
API call on OneDrive Business only.
OneDrive does not currently support User Metadata. When writing metadata, only
writeable system properties will be written -- any read-only or unrecognized keys
passed in will be ignored.
TIP: to see the metadata and permissions for any file or folder, run:
```
rclone lsjson remote:path --stat -M --onedrive-metadata-permissions read
```
Here are the possible system metadata items for the onedrive backend.
| Name | Help | Type | Example | Read Only |
|------|------|------|---------|-----------|
| btime | Time of file birth (creation) with S accuracy (mS for OneDrive Personal). | RFC 3339 | 2006-01-02T15:04:05Z | N |
| content-type | The MIME type of the file. | string | text/plain | **Y** |
| created-by-display-name | Display name of the user that created the item. | string | John Doe | **Y** |
| created-by-id | ID of the user that created the item. | string | 48d31887-5fad-4d73-a9f5-3c356e68a038 | **Y** |
| description | A short description of the file. Max 1024 characters. Only supported for OneDrive Personal. | string | Contract for signing | N |
| id | The unique identifier of the item within OneDrive. | string | 01BYE5RZ6QN3ZWBTUFOFD3GSPGOHDJD36K | **Y** |
| last-modified-by-display-name | Display name of the user that last modified the item. | string | John Doe | **Y** |
| last-modified-by-id | ID of the user that last modified the item. | string | 48d31887-5fad-4d73-a9f5-3c356e68a038 | **Y** |
| malware-detected | Whether OneDrive has detected that the item contains malware. | boolean | true | **Y** |
| mtime | Time of last modification with S accuracy (mS for OneDrive Personal). | RFC 3339 | 2006-01-02T15:04:05Z | N |
| package-type | If present, indicates that this item is a package instead of a folder or file. Packages are treated like files in some contexts and folders in others. | string | oneNote | **Y** |
| permissions | Permissions in a JSON dump of OneDrive format. Enable with --onedrive-metadata-permissions. Properties: id, grantedTo, grantedToIdentities, invitation, inheritedFrom, link, roles, shareId | JSON | {} | N |
| shared-by-id | ID of the user that shared the item (if shared). | string | 48d31887-5fad-4d73-a9f5-3c356e68a038 | **Y** |
| shared-owner-id | ID of the owner of the shared item (if shared). | string | 48d31887-5fad-4d73-a9f5-3c356e68a038 | **Y** |
| shared-scope | If shared, indicates the scope of how the item is shared: anonymous, organization, or users. | string | users | **Y** |
| shared-time | Time when the item was shared, with S accuracy (mS for OneDrive Personal). | RFC 3339 | 2006-01-02T15:04:05Z | **Y** |
| utime | Time of upload with S accuracy (mS for OneDrive Personal). | RFC 3339 | 2006-01-02T15:04:05Z | **Y** |
See the [metadata](/docs/#metadata) docs for more info.
{{< rem autogenerated options stop >}}
## Limitations

2
docs/content/overview.md
View File

@@ -40,7 +40,7 @@ Here is an overview of the major features of each cloud storage system.
| Memory | MD5 | R/W | No | No | - | - |
| Microsoft Azure Blob Storage | MD5 | R/W | No | No | R/W | - |
| Microsoft Azure Files Storage | MD5 | R/W | Yes | No | R/W | - |
| Microsoft OneDrive | QuickXorHash ⁵ | R/W | Yes | No | R | - |
| Microsoft OneDrive | QuickXorHash ⁵ | DR/W | Yes | No | R | DRW |
| OpenDrive | MD5 | R/W | Yes | Partial ⁸ | - | - |
| OpenStack Swift | MD5 | R/W | No | No | R/W | - |
| Oracle Object Storage | MD5 | R/W | No | No | R/W | - |