jgaunt/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2025-12-15 15:53:41 +00:00
Code Issues Projects Releases Wiki Activity

march: Implement --assume-listings-sorted to speed up sync starting

Browse Source 
This can be used when the source and destination backends are
guaranteed to return the items in the same sorted order.

Fixes #5859
This commit is contained in:
Nick Craig-Wood
2025-06-30 16:19:25 +01:00
parent e7f11af1ca
commit d376838f77
5 changed files with 86 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
docs/content/docs.md
View File

@@ -852,6 +852,32 @@ the binary units, e.g. 1, 2\*\*10, 2\*\*20, 2\*\*30 respectively.
See also [--human-readable](#human-readable).
### --assume-listings-sorted
This flag can be used when the source and destination backends are
guaranteed to return the items in the same sorted order and in that
case it will speed up the sync.
Not all backends are guaranteed to return sorted entries (eg local)
but s3 should, so an s3 to s3 sync could benefit from this flag.
If rclone finds an out of order directory entry then it will cancel
the sync with the error:
```console
out of order listing in source (remote:dir)
```
In this case you should remove the `--assume-listings-sorted` flag.
If you are using `--assume-listings-sorted` then rclone will assume
`--no-unicode-normalization` and it will compare file names in a case
sensitive way.
Normally sorting directory entries is not a bottleneck, but it can
become so with syncs of millions of items in a single directory as the
sync will not start until the directory listing is complete.
## Main options
### --backup-dir string