mirror of
https://github.com/rclone/rclone.git
synced 2025-12-06 00:03:32 +00:00
fs: allow setting a write buffer for multithread
when multi-thread downloading is enabled, rclone used to send a write to disk after every read, resulting in a lot of small writes to different locations of the file. depending on the underlying filesystem or device, it can be more efficient to send bigger writes.
This commit is contained in:
committed by
Nick Craig-Wood
parent
5f938fb9ed
commit
fcb912a664
@@ -1511,6 +1511,25 @@ if you are reading and writing to an OS X filing system this will be
|
||||
|
||||
This command line flag allows you to override that computed default.
|
||||
|
||||
### --multi-thread-write-buffer-size=SIZE ###
|
||||
|
||||
When downloading with multiple threads, rclone will buffer SIZE bytes in
|
||||
memory before writing to disk for each thread.
|
||||
|
||||
This can improve performance if the underlying filesystem does not deal
|
||||
well with a lot of small writes in different positions of the file, so
|
||||
if you see downloads being limited by disk write speed, you might want
|
||||
to experiment with different values. Specially for magnetic drives and
|
||||
remote file systems a higher value can be useful.
|
||||
|
||||
Nevertheless, the default of `128k` should be fine for almost all use
|
||||
cases, so before changing it ensure that network is not really your
|
||||
bottleneck.
|
||||
|
||||
As a final hint, size is not the only factor: block size (or similar
|
||||
concept) can have an impact. In one case, we observed that exact
|
||||
multiples of 16k performed much better than other values.
|
||||
|
||||
### --multi-thread-cutoff=SIZE ###
|
||||
|
||||
When downloading files to the local backend above this size, rclone
|
||||
|
||||
Reference in New Issue
Block a user