1
0
mirror of https://github.com/rclone/rclone.git synced 2025-12-29 22:53:30 +00:00
Commit Graph

7 Commits

Author SHA1 Message Date
Nick Craig-Wood
0c0fb93111 pool: fix deadlock with --max-buffer-memory
Before this change we used an overcomplicated method of memory
reservations in the pool.RW which caused deadlocks.

This changes it to use a much simpler reservation system where we
actually reserve the memory and store it in the pool.RW. This allows
us to use the semaphore.Weighted to count the actually memory in use
(rather than the memory in use and in the cache). This in turn allows
accurate use of the semaphore by users wanting memory.
2025-08-14 16:14:59 +01:00
Nick Craig-Wood
117d8d9fdb pool: fix deadlock with --max-memory and multipart transfers
Because multipart transfers can need more than one buffer to complete,
if transfers was set very high, it was possible for lots of multipart
transfers to start, grab fewer buffers than chunk size, then deadlock
because no more memory was available.

This fixes the problem by introducing a reservation system which the
multipart transfer uses to ensure it can reserve all the memory for
one chunk before starting.
2025-08-07 12:45:44 +01:00
Nick Craig-Wood
d08b49d723 pool: Add ability to wait for a write to RW 2024-08-14 21:13:09 +01:00
Nick Craig-Wood
cb2d2d72a0 pool: Make RW thread safe so can read and write at the same time 2024-08-14 21:13:09 +01:00
Nick Craig-Wood
bc986b44b2 lib/pool: add DelayAccounting() to fix accounting when reading hashes 2023-08-24 16:42:09 +01:00
Nick Craig-Wood
f4b1a51af6 lib/pool: add SetAccounting to RW 2023-08-24 15:28:40 +01:00
Nick Craig-Wood
0dea83a4aa pool: add page backed reader/writer for multi thread uploads 2023-08-24 12:39:27 +01:00