jgaunt/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2025-12-06 00:03:32 +00:00
Code Issues Projects Releases Wiki Activity

sync,operations: optimise --copy-dest and --compare-dest

Browse Source 
Before this change --compare-dest and --copy-dest would check to see
if the compare/copy object existed first, before seeing if the
destination object was present.

This is inefficient, because in most --copy-dest syncs the destination
will be present and the compare/copy object need never be tested.
--compare-dest syncs may also be speeded up if they are done to the
same directory repeatedly.

This fixes the problem by re-arranging the logic so if the transfer is
not needed then the compare/copy object is never tested.

See: https://forum.rclone.org/t/union-with-copy-dest-enabled-is-slower-than-expected/32172
This commit is contained in:
Nick Craig-Wood
2022-08-01 17:51:46 +01:00
parent 639624184d
commit a875320e37
2 changed files with 20 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
fs/operations/operations.go
View File

@@ -1952,11 +1952,17 @@ func moveOrCopyFile(ctx context.Context, fdst fs.Fs, fsrc fs.Fs, dstFileName str
return err
}
}
NoNeedTransfer, err := CompareOrCopyDest(ctx, fdst, dstObj, srcObj, copyDestDir, backupDir)
if err != nil {
return err
needTransfer := NeedTransfer(ctx, dstObj, srcObj)
if needTransfer {
NoNeedTransfer, err := CompareOrCopyDest(ctx, fdst, dstObj, srcObj, copyDestDir, backupDir)
if err != nil {
return err
}
if NoNeedTransfer {
needTransfer = false
}
}
if !NoNeedTransfer && NeedTransfer(ctx, dstObj, srcObj) {
if needTransfer {
// If destination already exists, then we must move it into --backup-dir if required
if dstObj != nil && backupDir != nil {
err = MoveBackupDir(ctx, backupDir, dstObj)