Nick Craig-Wood 8bcb1bde1e fs/march: fix runtime: program exceeds 10000-thread limit ...

Before this change when doing a sync with `--no-traverse` and
`--files-from` we could call `NewObject` a total of `--checkers` *
`--checkers` times simultaneously.

With `--checkers 128` this can exceed the 10,000 thread limit and
fails when run on a local to local transfer because `NewObject` calls
`lstat` which is a syscall which needs an OS thread of its own.

This patch uses a weighted semaphore to limit the number of
simultaneous calls to `NewObject` to `--checkers` instead which won't
blow the 10,000 thread limit and is far more sensible use of OS
resources.

Fixes #9073

2025-12-31 18:04:10 +00:00

..

accounting

accounting: add AccountReadN for use in cluster

2025-11-03 14:44:39 +00:00

asyncreader

refactor: use strings.Builder to improve performance

2025-10-29 16:48:34 +00:00

cache

config: make parsing of duration options consistent

2025-07-08 12:08:14 +02:00

chunkedreader

build: modernize Go usage

2025-02-28 11:31:14 +00:00

chunksize

build: modernize Go usage

2025-02-28 11:31:14 +00:00

config

build: fix lint warning after linter upgrade

2025-12-09 16:15:17 +00:00

dirtree

build: modernize benchmark usage

2025-09-15 12:45:57 +01:00

driveletter

build: remove build constraint syntax for go 1.16 and older

2024-04-18 16:53:55 +02:00

filter

filter: Add --hash-filter to deterministically select a subset of files

2025-03-17 17:25:59 +00:00

fserrors

build: remove obsolete build tag

2025-10-21 18:56:06 +01:00

fshttp

fs: tls: add --client-pass support for encrypted --client-key files

2025-08-22 12:19:29 +01:00

fspath

fs: allow global variables to be overriden or set on backend creation

2025-07-23 15:09:51 +01:00

hash

hashsum: changed output format when listing algorithms

2025-08-27 23:36:28 +02:00

list

build: use "for i := range n", added in go1.22

2025-09-15 12:45:57 +01:00

log

log: fix backtrace not going to the --log-file #9014

2025-12-09 16:35:07 +00:00

logger

build: remove obsolete build tag

2025-10-21 18:56:06 +01:00

march

fs/march: fix runtime: program exceeds 10000-thread limit

2025-12-31 18:04:10 +00:00

object

http: add basic metadata and provide it via serve

2025-11-17 16:52:30 +00:00

operations

check: improved reporting of differences in sizes and contents

2025-11-01 19:23:01 +00:00

rc

rc: fix formatting in job/batch

2025-11-21 17:06:18 +00:00

sync

sync: fix error propagation in tests (#9025)

2025-12-10 15:43:22 +00:00

walk

walk: move NewListRHelper into list.Helper to avoid circular dependency

2025-04-08 15:14:00 +01:00

backend_config_test.go

fs: add --all to rclone config create/update to ask all the config questions #3455

2021-05-14 14:07:44 +01:00

backend_config.go

build: modernize Go usage

2025-02-28 11:31:14 +00:00

bits_test.go

fs: make Flagger and FlaggerNP interfaces public so we can test flags elsewhere

2024-07-15 11:09:54 +01:00

bits.go

fs: fix docs for Bits

2023-10-23 15:43:55 +01:00

bwtimetable_test.go

fs: Allow semicolons as well as spaces in --bwlimit timetable parsing - fixes #7595

2024-07-17 11:04:01 +01:00

bwtimetable.go

refactor: use strings.Cut to simplify code

2025-11-27 14:42:11 +00:00

config_list_test.go

fs: fix tests for *SepList

2018-08-21 10:58:59 +01:00

config_list.go

Make flags show up with their proper names, eg SizeSuffix rather than int

2019-02-07 11:57:26 +00:00

config_test.go

build: add lint ignore comment required for golangci-staticcheck in addition to stand-alone staticcheck

2023-03-26 14:28:15 +02:00

config.go

fs: tls: add --client-pass support for encrypted --client-key files

2025-08-22 12:19:29 +01:00

configmap.go

fs: fix the defaults overriding the actual config

2024-07-15 11:09:54 +01:00

countsuffix_test.go

fs: make Flagger and FlaggerNP interfaces public so we can test flags elsewhere

2024-07-15 11:09:54 +01:00

countsuffix.go

build: fix gocritic lint issue dupbranchbody

2024-08-15 22:08:34 +01:00

cutoffmode_test.go

fs: make Flagger and FlaggerNP interfaces public so we can test flags elsewhere

2024-07-15 11:09:54 +01:00

cutoffmode.go

fs: re-implement CutoffMode, LogLevel, TerminalColorMode with Enum

2023-10-03 15:14:24 +01:00

daemon_other.go

build: remove build constraint syntax for go 1.16 and older

2024-04-18 16:53:55 +02:00

daemon_unix.go

build: remove build constraint syntax for go 1.16 and older

2024-04-18 16:53:55 +02:00

deletemode.go

Break the fs package up into smaller parts.

2018-01-15 17:51:14 +00:00

dir_wrapper.go

combine: fix CopyDirMetadata error on upstream root

2024-03-07 11:09:07 +00:00

dir.go

fs: allow Metadata calls to be called with Directory or Object

2024-02-27 10:56:19 +00:00

direntries_test.go

build: fix up package paths after repo move

2019-07-28 18:47:38 +01:00

direntries.go

fs: Allow sync of a file and a directory with the same name

2019-06-09 15:57:05 +01:00

dump_test.go

fs: make Flagger and FlaggerNP interfaces public so we can test flags elsewhere

2024-07-15 11:09:54 +01:00

dump.go

fs: implement --metadata-mapper to transform metatadata with a user supplied program

2023-11-18 17:49:35 +00:00

enum_test.go

fs: make Flagger and FlaggerNP interfaces public so we can test flags elsewhere

2024-07-15 11:09:54 +01:00

enum.go

fs: re-implement CutoffMode, LogLevel, TerminalColorMode with Enum

2023-10-03 15:14:24 +01:00

features.go

fs: define ListP interface for paged listing #4788

2025-04-08 15:12:53 +01:00

fingerprint_test.go

mockfs: make it so it can be registered as an Fs

2023-04-28 14:31:05 +01:00

fingerprint.go

fs: add Fingerprint to detect changes in an object

2020-06-30 12:01:36 +01:00

fs_test.go

fs: move tests into correct files

2025-11-03 14:44:39 +00:00

fs.go

config: make parsing of duration options consistent

2025-07-08 12:08:14 +02:00

log_test.go

fs: make Flagger and FlaggerNP interfaces public so we can test flags elsewhere

2024-07-15 11:09:54 +01:00

log.go

rc: make sure fatal errors don't crash rclone - fixes #8955

2025-11-12 12:22:04 +00:00

metadata_mapper_code.go

build: remove build constraint syntax for go 1.16 and older

2024-04-18 16:53:55 +02:00

metadata_test.go

fs: implement --metadata-mapper to transform metatadata with a user supplied program

2023-11-18 17:49:35 +00:00

metadata.go

fs: allow Metadata calls to be called with Directory or Object

2024-02-27 10:56:19 +00:00

mimetype.go

build: use sequence Split introduced in go1.24

2025-09-15 12:45:57 +01:00

mount_helper_test.go

rclone.mount: ignore _netdev mount argument - FIxes #5808

2022-05-12 20:27:13 +01:00

mount_helper.go

build: update logging statements to make json log work - fixes #6038

2024-09-06 17:04:18 +01:00

newfs_internal_test.go

fs: allow global variables to be overriden or set on backend creation

2025-07-23 15:09:51 +01:00

newfs_test.go

fs: allow global variables to be overriden or set on backend creation

2025-07-23 15:09:51 +01:00

newfs.go

fs: allow global variables to be overriden or set on backend creation

2025-07-23 15:09:51 +01:00

open_options_test.go

fs: add --metadata-set flag to specify metadata for uploads

2022-06-29 14:29:36 +01:00

open_options.go

fs: fix Don't know how to set key "chunkSize" on upload errors in tests

2024-10-22 19:54:33 +01:00

override_dir_test.go

fs: add OverrideDirectory for overriding path of directory

2023-05-14 11:22:57 +01:00

override_dir.go

fs: add OverrideDirectory for overriding path of directory

2023-05-14 11:22:57 +01:00

override_test.go

fs: move operations.NewOverrideRemote to fs.NewOverrideRemote

2022-11-21 08:02:09 +00:00

override.go

fs: when creating new fs.OverrideRemotes don't layer overrides if not needed

2023-05-09 16:28:10 +01:00

pacer_test.go

fs: move tests into correct files

2025-11-03 14:44:39 +00:00

pacer.go

Add --max-connections to control maximum backend concurrency

2025-03-25 15:49:27 +00:00

parseduration_test.go

fs: make Flagger and FlaggerNP interfaces public so we can test flags elsewhere

2024-07-15 11:09:54 +01:00

parseduration.go

refactor: use strings.Builder to improve performance

2025-10-29 16:48:34 +00:00

parsetime_test.go

fs: make Flagger and FlaggerNP interfaces public so we can test flags elsewhere

2024-07-15 11:09:54 +01:00

parsetime.go

rc: fix output of Time values in options/get

2023-05-04 15:04:11 +01:00

registry_test.go

fs: add NonDefaultRC for discovering options in use

2025-11-03 14:44:39 +00:00

registry.go

fs: add NonDefaultRC for discovering options in use

2025-11-03 14:44:39 +00:00

sizesuffix_test.go

fs: make Flagger and FlaggerNP interfaces public so we can test flags elsewhere

2024-07-15 11:09:54 +01:00

sizesuffix.go

build: fix gocritic lint issue dupbranchbody

2024-08-15 22:08:34 +01:00

terminalcolormode_test.go

fs: re-implement CutoffMode, LogLevel, TerminalColorMode with Enum

2023-10-03 15:14:24 +01:00

terminalcolormode.go

fs: re-implement CutoffMode, LogLevel, TerminalColorMode with Enum

2023-10-03 15:14:24 +01:00

tristate_test.go

fs: make Flagger and FlaggerNP interfaces public so we can test flags elsewhere

2024-07-15 11:09:54 +01:00

tristate.go

fs: fix tristate conversion to JSON

2023-03-27 16:17:49 +01:00

types.go

box: fix about: cannot unmarshal number 1.0e+18 into Go struct field

2025-07-31 11:38:51 +01:00

version.go

Configurable version suffix independent of version number

2022-05-17 12:10:01 +01:00

versioncheck.go

build: update to go1.25 and make go1.24 the minimum required version

2025-08-13 17:54:45 +01:00

versionsuffix.go

Configurable version suffix independent of version number

2022-05-17 12:10:01 +01:00

versiontag.go

Start v1.73.0-DEV development

2025-11-21 18:23:07 +00:00