1
0
mirror of https://github.com/gilbertchen/duplicacy synced 2025-12-06 00:03:38 +00:00
Commit Graph

807 Commits

Author SHA1 Message Date
Gilbert Chen
ff207ba5bf Convert the file path to the real one when downloading a chunk
This is mainly to handle the case when a fossil needs to be downloaded.
This happens when a metadata chunk has been marked as a fossil while the
correpsonding snapshot must be reconstructed for determine referenced
chunks.
2023-09-08 13:30:28 -04:00
Gilbert Chen
3a81c1065a Add a new Samba backend
The storage url is smb://user@server[:port]/share/path.  The password can be
set in the environment variable DUPLICACY_SMB_PASSWORD for default storage or
DUPLICACY_<STORAGE_NAME>_SMB_PASSWORD.

This backend is based on https://github.com/hirochachacha/go-smb2.  The
previous samba:// backend is just an alias for the disk-based backend with
caching enabled.
2023-07-05 22:51:24 -04:00
Gilbert Chen
cdf8f5a857 Check the length of 'file' before checking if it ends with '/' 2023-04-09 22:11:08 -04:00
Gilbert Chen
1f9ad0e35c B2 backend should be able to download .fsl files
This is needed when a metadata chunk has been turned into a fossil.  In B2,
the fossil file is the last version of the file with an 'upload' action, so
to download the file, the 'b2_list_file_versions' api is called to find the
file id and then download it using 'b2_download_file_by_id'.

This is needed when a metadata chunk has been turned into a fossil
2023-03-30 13:27:29 -04:00
Gilbert Chen
53b0f3f7b6 Implement zstd compression
Zstd compression can be enabled by providing `-zstd` or `-zstd-level <level>`
to `init`, `add`, or `backup`. With `-zstd` the compression level will be
`default`, and with `-zstd-level` the level can be any of `fastest`, `default`,
`better`, or `best`.
2023-03-26 21:31:51 -04:00
Gilbert Chen
9f276047db Fixed a typo in log id
SFT_RECONNECT -> SFTP_RECONNECT
2023-03-26 21:26:51 -04:00
gilbertchen
c237269589 Merge pull request #649 from northnose/saveVerifiedChunksLock
Acquire verifiedChunksLock in saveVerifiedChunks
2023-03-23 22:11:53 -04:00
gilbertchen
493ef603e3 Merge pull request #648 from northnose/dropbox-pointer
Upgrade go-dropbox to the latest
2023-03-23 22:10:24 -04:00
David Zhang
889191a814 Upgrade go-dropbox to the latest 2023-03-20 20:19:43 -07:00
David Zhang
df80096cdf Acquire verifiedChunksLock in saveVerifiedChunks 2023-03-12 21:00:51 -07:00
gilbertchen
24c2ea76b9 Merge pull request #633 from sevimo123/sharepoint_support
Sharepoint support
2023-01-19 14:06:42 -05:00
gilbertchen
15b6ef9d76 Merge pull request #632 from sevimo123/custom_odb_creds
CLI support for custom credentials for OneDrive (client_id/client_secret)
2023-01-19 13:01:01 -05:00
gilbertchen
75b310b98e Merge pull request #641 from A-wels/patch-1
Fixed typo incomlete -> incomplete
2023-01-17 11:51:20 -05:00
Alexander Welsing
039b749a3e Fixed typo incomlete -> incomplete
"Previous incomlete backup contains %d files and %d chunks -> "Previous incomplete backup contains %d files and %d chunks
2023-01-12 12:41:45 +01:00
Gilbert Chen
9be475f876 Fix another chunk leak in listing files in a revision.
This bug leaks a chunk every time files in a revision are listed.  Not a big
deal for backup and restore, but it becomes problematic when listing files in
many revisions for commands such check and history.
2023-01-06 23:02:48 -05:00
Gilbert Chen
27ff3e216b Bump version to 3.1.0 v3.1.0 2022-12-06 23:28:44 -05:00
Gilbert Chen
1ba204a21b Upgrade go-dropbox to the latest
This is to incorporate the fix:
https://github.com/gilbertchen/go-dropbox/commit/60ebcd

Otherwise the access token won't get updated after token refresh
2022-12-06 23:24:11 -05:00
Gilbert Chen
b8c7594dbf Release the chunk used to download files when finished
Without this fix, a chunk is leaked for each snapshot checked
with `-files`.
2022-12-06 22:46:25 -05:00
Gilbert Chen
58f0d2be5a Fixed a bug that didn't preserve the version bit when copying old snapshots
The version bit should not be set to 1 when encoding a snapshot.  Instead,
it must be set to 1 on snapshot creation.

To correctly process old snapshots encoded incorrectly with version bit set
to 1, the first byte of the encoded file list is also checked.  If the first
byte is `[`, then it must be an old snapshot, since the file list in the new
snapshot format always starts with a string encoded in msgpack, the first
byte of which can't be `[`.
2022-11-22 21:31:24 -05:00
Gilbert Chen
0a794e6fea Fixed test errors and remove obsolete tests 2022-11-15 11:53:46 -05:00
Gilbert Chen
bc2d762e41 Add -rewrite to the check command to fix corrupted chunks
This option is useful only when erasure coding is enabled.  It will
download and re-upload chunks that contain corruption but are
generally recoverable.  It can also be used to fix chunks that
are created by 3.0.1 on arm64 machines with wrong hashes.
2022-11-15 11:47:02 -05:00
Gilbert Chen
6a7a2c8048 Upgrade github.com/minio/highwayhash to 1.0.2
highwayhash 1.0.1 contains a bug leading to incorrect hashes on arm64 machines.
The 1.0.1 version is retained in github.com/gilbertchen/highwayhash so the hash
can be checked again if a mismatch is detected by 1.0.2.
2022-11-09 14:44:24 -05:00
Gilbert Chen
3472206bcf Handle zero-byte files correctly
This commit fixed 2 bugs.  The first bug occurs when an incomplete backup
contains a zero-byte file and no chunks.  The second bug occurs when the
repository contains only zero-byte files.
2022-11-08 22:54:35 -05:00
Gilbert Chen
72eb339837 Bump version to 3.0.1 v3.0.1 2022-10-06 20:30:05 -04:00
Gilbert Chen
901044b348 Update dependencies 2022-10-06 20:29:51 -04:00
Gilbert Chen
d6f5336784 Bump version to 3.0.0 v3.0.0 2022-10-05 22:21:17 -04:00
Gilbert Chen
4b47ea55e4 Bump version to 2.8.0 2022-10-04 12:49:50 -04:00
Gilbert Chen
5c35ef799a Switch to go modules 2022-10-04 12:48:58 -04:00
Gilbert Chen
2c63d32142 Use swift V2 2022-10-04 12:47:38 -04:00
Gilbert Chen
6009f64b66 Add a storage backend for Storj
The url format is storj://satellite/bucket/path.  You can get the
satellite along with the api access key when requesting an Access
Grant of type API Access.
2022-09-30 10:30:06 -04:00
Gilbert Chen
cde660ee9f Use long-lived refresh token for the Dropbox backend
The refresh token can be downloaded from https://duplicacy.com/dropbox_start
2022-08-12 22:17:06 -04:00
Victor Mozgin
d7593a828c Added support for SharePoint document libraries via odb://DRIVEID@path/to/storage. 2022-07-22 23:31:27 -04:00
Victor Mozgin
238ef63e16 CLI support for custom credentials for OneDrive (client_id/client_secret) 2022-07-22 21:18:52 -04:00
Gilbert Chen
54952cef26 Fixed a bug that referenced uninitialized operator.snapshotCache 2022-07-10 12:23:11 -04:00
Gilbert Chen
fc2386f9cc Initialize startTime correctly in CreateChunkOperator 2022-06-09 23:28:29 -04:00
Gilbert Chen
0d8a37f9f3 Dependency change: update github.com/ncw/swift to v2.0.1 2022-04-08 23:16:51 -04:00
gilbertchen
345fc5ed87 Merge pull request #626 from markfeit/swift-v2
Swift v2
2022-04-08 22:29:30 -04:00
Gilbert Chen
8df529dffe Fixed the type of a test parameter 2022-04-07 23:34:48 -04:00
gilbertchen
f2d6de3fff Merge pull request #625 from gilbertchen/memory_optimization
Rewrite the backup procedure to reduce memory usage
2022-04-07 23:26:46 -04:00
Gilbert Chen
fede9c74b5 Merge branch 'master' into memory_optimization 2022-04-07 23:26:14 -04:00
Gilbert Chen
a953c4ec28 Don't parse test parameters in init()
This is to make test parameter parsing work with newer versions
of Go
2022-04-07 22:31:18 -04:00
gilbertchen
f52dcf761b Merge branch 'master' into memory_optimization 2022-03-31 15:05:50 -04:00
Gilbert Chen
ade669d14e Update totalChunkSize in the chunk operator to show stats during restore 2022-03-04 16:53:40 -05:00
Mark Feit
4743c7ba0d DOn't cancel the context and use a sane deadline. 2021-12-17 10:09:36 -05:00
Mark Feit
590d3b1b5b More development 2021-12-04 10:56:47 -05:00
Mark Feit
0590daff85 More dev 2021-12-04 10:53:34 -05:00
Mark Feit
95b1227d93 Use empty context 2021-12-04 10:48:01 -05:00
Mark Feit
1661caeb92 More fixups 2021-12-04 10:37:11 -05:00
Mark Feit
041ba944c4 Typo 2021-12-04 10:24:31 -05:00
Mark Feit
934c2515cc Import context 2021-12-04 10:23:42 -05:00