1
0
mirror of https://github.com/gilbertchen/duplicacy synced 2025-12-06 00:03:38 +00:00

830 Commits

Author SHA1 Message Date
Gilbert Chen
2def0161b3 Bump version to 3.2.5 v3.2.5 2025-05-02 15:39:59 -04:00
Gilbert Chen
df76bd0a34 OneDrive: use correct parent reference when moving files
switch the parent reference from '/root:/' to '/drive/root:/'.  OneDrive seems
to stop accepting the former.

The other change in this commit is to convert the chunk file path correctly
if the chunk is a fossil.
2025-05-02 15:30:48 -04:00
Gilbert Chen
065ae50868 Improve parsing logic for swift storage URLs that contain multiple '@'
Parse the storage URL first, then use a regex to ensure the server name is
correctly identified starting after the final '@'.
2024-12-19 08:54:22 -05:00
Gilbert Chen
bb214b6e04 Bump version to 3.2.4 v3.2.4 2024-10-30 12:05:18 -04:00
Gilbert Chen
6bca9fccdd maxCollectionNumber must be increased even in collect-only mode
This fixed a bug that caused collect-only mode to keep overriding
collect 1 during every prune
2024-10-29 22:46:46 -04:00
Gilbert Chen
a06d925e53 Remove 'incomplete_files' in deleteIncompleteSnapshot()
This file is used for storing the on-disk entry list.  When an error occurs,
this file is renamed to 'incomplete_snapshot' for fast resuming on next run.
But if there is no error this file should be removed.
2024-10-29 21:04:51 -04:00
Gilbert Chen
69f5d2f7bf Don't save the incomplete snapshot for a dry run
Saving the incomple snapshot would trick the next backup into thinking that
all chunks have been uploaded.
2024-07-25 22:54:19 -04:00
Gilbert Chen
d182708eb5 Fix zstd level name (fast -> fastest) 2024-07-11 14:47:40 -04:00
Gilbert Chen
f8a0964ac0 Save the list of verified chunks every 5 minutes.
This can be useful if the list isn't saved at the end of the run for some
reason, such as when the program is terminated abruptly.
2024-07-11 14:12:12 -04:00
Gilbert Chen
b659456f12 Don't add corrupt chunks to verified_chunks
When -persist is specified, the chunk downloader may return the chunk even if
the chunk is corrupt.  We use the chunk.isBroken flag to detect this situation.
2024-07-10 10:32:40 -04:00
Gilbert Chen
0d3ee4186c Use a different api to find the id of a GCD drive by name
Previous a listing api was used to find the id.  That api only returns 10
drives for each call.  If there are more than 10 shared drives some may
not be returned.
2024-01-16 12:50:59 -05:00
Gilbert Chen
2549532676 Bump version to 3.2.3 v3.2.3 2023-10-06 22:13:33 -04:00
Gilbert Chen
e99dfea048 Upgrade storj.io/uplink to v1.12.1 to fix an 64 bit alignment issue 2023-10-06 22:12:12 -04:00
Gilbert Chen
50120146df Bump version to 3.2.2 v3.2.2 2023-10-03 22:35:18 -04:00
Gilbert Chen
7bfc0e7d51 Save passwords after a storj storage has been created 2023-10-03 22:33:41 -04:00
Gilbert Chen
fd3bceae19 Bump version to 3.2.1 v3.2.1 2023-10-02 12:30:23 -04:00
Gilbert Chen
7cc1b4222c Update go version to 1.19.13 2023-09-28 23:17:24 -04:00
Gilbert Chen
d92b1734f4 Skip identical entries when listing chunks
The prune command can remove redundant chunks (chunks with the same chunk id
but at different subdirectory level).  However, if the same chunk appears
mutliple times in the listing returned by the storage, it will be treated as
a redundant chunk and thus removed.
2023-09-27 15:31:08 -04:00
Gilbert Chen
4e9d2c4cca Allow two copy-compatible storages to have different compression levels
This is useful for upgrading an existing storage to zstd compression or others.
Chunks need to be decompressed and re-compressed during copy anyway.  Only
the bit-identical option requires the same compression level

Also fix a typo: compatiable -> compatible
2023-09-18 14:44:41 -04:00
gilbertchen
cc482beb95 Merge pull request #653 from gorbak25/fix-compression
Fix compression level check
2023-09-18 10:50:55 -04:00
Grzegorz Uriasz
bf3ea8a83c Fix compression level check 2023-09-11 23:03:34 +02:00
Gilbert Chen
981efc13e6 Bump version to 3.2.0 v3.2.0 2023-09-08 15:25:22 -04:00
Gilbert Chen
6445ecbcde Add dependencies required by github.com/hirochachacha/go-smb2 2023-09-08 13:54:03 -04:00
Gilbert Chen
ff207ba5bf Convert the file path to the real one when downloading a chunk
This is mainly to handle the case when a fossil needs to be downloaded.
This happens when a metadata chunk has been marked as a fossil while the
correpsonding snapshot must be reconstructed for determine referenced
chunks.
2023-09-08 13:30:28 -04:00
Gilbert Chen
3a81c1065a Add a new Samba backend
The storage url is smb://user@server[:port]/share/path.  The password can be
set in the environment variable DUPLICACY_SMB_PASSWORD for default storage or
DUPLICACY_<STORAGE_NAME>_SMB_PASSWORD.

This backend is based on https://github.com/hirochachacha/go-smb2.  The
previous samba:// backend is just an alias for the disk-based backend with
caching enabled.
2023-07-05 22:51:24 -04:00
Gilbert Chen
cdf8f5a857 Check the length of 'file' before checking if it ends with '/' 2023-04-09 22:11:08 -04:00
Gilbert Chen
1f9ad0e35c B2 backend should be able to download .fsl files
This is needed when a metadata chunk has been turned into a fossil.  In B2,
the fossil file is the last version of the file with an 'upload' action, so
to download the file, the 'b2_list_file_versions' api is called to find the
file id and then download it using 'b2_download_file_by_id'.

This is needed when a metadata chunk has been turned into a fossil
2023-03-30 13:27:29 -04:00
Gilbert Chen
53b0f3f7b6 Implement zstd compression
Zstd compression can be enabled by providing `-zstd` or `-zstd-level <level>`
to `init`, `add`, or `backup`. With `-zstd` the compression level will be
`default`, and with `-zstd-level` the level can be any of `fastest`, `default`,
`better`, or `best`.
2023-03-26 21:31:51 -04:00
Gilbert Chen
9f276047db Fixed a typo in log id
SFT_RECONNECT -> SFTP_RECONNECT
2023-03-26 21:26:51 -04:00
gilbertchen
c237269589 Merge pull request #649 from northnose/saveVerifiedChunksLock
Acquire verifiedChunksLock in saveVerifiedChunks
2023-03-23 22:11:53 -04:00
gilbertchen
493ef603e3 Merge pull request #648 from northnose/dropbox-pointer
Upgrade go-dropbox to the latest
2023-03-23 22:10:24 -04:00
David Zhang
889191a814 Upgrade go-dropbox to the latest 2023-03-20 20:19:43 -07:00
David Zhang
df80096cdf Acquire verifiedChunksLock in saveVerifiedChunks 2023-03-12 21:00:51 -07:00
gilbertchen
24c2ea76b9 Merge pull request #633 from sevimo123/sharepoint_support
Sharepoint support
2023-01-19 14:06:42 -05:00
gilbertchen
15b6ef9d76 Merge pull request #632 from sevimo123/custom_odb_creds
CLI support for custom credentials for OneDrive (client_id/client_secret)
2023-01-19 13:01:01 -05:00
gilbertchen
75b310b98e Merge pull request #641 from A-wels/patch-1
Fixed typo incomlete -> incomplete
2023-01-17 11:51:20 -05:00
Alexander Welsing
039b749a3e Fixed typo incomlete -> incomplete
"Previous incomlete backup contains %d files and %d chunks -> "Previous incomplete backup contains %d files and %d chunks
2023-01-12 12:41:45 +01:00
Gilbert Chen
9be475f876 Fix another chunk leak in listing files in a revision.
This bug leaks a chunk every time files in a revision are listed.  Not a big
deal for backup and restore, but it becomes problematic when listing files in
many revisions for commands such check and history.
2023-01-06 23:02:48 -05:00
Gilbert Chen
27ff3e216b Bump version to 3.1.0 v3.1.0 2022-12-06 23:28:44 -05:00
Gilbert Chen
1ba204a21b Upgrade go-dropbox to the latest
This is to incorporate the fix:
https://github.com/gilbertchen/go-dropbox/commit/60ebcd

Otherwise the access token won't get updated after token refresh
2022-12-06 23:24:11 -05:00
Gilbert Chen
b8c7594dbf Release the chunk used to download files when finished
Without this fix, a chunk is leaked for each snapshot checked
with `-files`.
2022-12-06 22:46:25 -05:00
Gilbert Chen
58f0d2be5a Fixed a bug that didn't preserve the version bit when copying old snapshots
The version bit should not be set to 1 when encoding a snapshot.  Instead,
it must be set to 1 on snapshot creation.

To correctly process old snapshots encoded incorrectly with version bit set
to 1, the first byte of the encoded file list is also checked.  If the first
byte is `[`, then it must be an old snapshot, since the file list in the new
snapshot format always starts with a string encoded in msgpack, the first
byte of which can't be `[`.
2022-11-22 21:31:24 -05:00
Gilbert Chen
0a794e6fea Fixed test errors and remove obsolete tests 2022-11-15 11:53:46 -05:00
Gilbert Chen
bc2d762e41 Add -rewrite to the check command to fix corrupted chunks
This option is useful only when erasure coding is enabled.  It will
download and re-upload chunks that contain corruption but are
generally recoverable.  It can also be used to fix chunks that
are created by 3.0.1 on arm64 machines with wrong hashes.
2022-11-15 11:47:02 -05:00
Gilbert Chen
6a7a2c8048 Upgrade github.com/minio/highwayhash to 1.0.2
highwayhash 1.0.1 contains a bug leading to incorrect hashes on arm64 machines.
The 1.0.1 version is retained in github.com/gilbertchen/highwayhash so the hash
can be checked again if a mismatch is detected by 1.0.2.
2022-11-09 14:44:24 -05:00
Gilbert Chen
3472206bcf Handle zero-byte files correctly
This commit fixed 2 bugs.  The first bug occurs when an incomplete backup
contains a zero-byte file and no chunks.  The second bug occurs when the
repository contains only zero-byte files.
2022-11-08 22:54:35 -05:00
Gilbert Chen
72eb339837 Bump version to 3.0.1 v3.0.1 2022-10-06 20:30:05 -04:00
Gilbert Chen
901044b348 Update dependencies 2022-10-06 20:29:51 -04:00
Gilbert Chen
d6f5336784 Bump version to 3.0.0 v3.0.0 2022-10-05 22:21:17 -04:00
Gilbert Chen
4b47ea55e4 Bump version to 2.8.0 2022-10-04 12:49:50 -04:00