Gilbert Chen
5c35ef799a
Switch to go modules
2022-10-04 12:48:58 -04:00
Gilbert Chen
2c63d32142
Use swift V2
2022-10-04 12:47:38 -04:00
Gilbert Chen
6009f64b66
Add a storage backend for Storj
...
The url format is storj://satellite/bucket/path. You can get the
satellite along with the api access key when requesting an Access
Grant of type API Access.
2022-09-30 10:30:06 -04:00
Gilbert Chen
cde660ee9f
Use long-lived refresh token for the Dropbox backend
...
The refresh token can be downloaded from https://duplicacy.com/dropbox_start
2022-08-12 22:17:06 -04:00
Gilbert Chen
54952cef26
Fixed a bug that referenced uninitialized operator.snapshotCache
2022-07-10 12:23:11 -04:00
Gilbert Chen
fc2386f9cc
Initialize startTime correctly in CreateChunkOperator
2022-06-09 23:28:29 -04:00
Gilbert Chen
0d8a37f9f3
Dependency change: update github.com/ncw/swift to v2.0.1
2022-04-08 23:16:51 -04:00
gilbertchen
345fc5ed87
Merge pull request #626 from markfeit/swift-v2
...
Swift v2
2022-04-08 22:29:30 -04:00
Gilbert Chen
8df529dffe
Fixed the type of a test parameter
2022-04-07 23:34:48 -04:00
gilbertchen
f2d6de3fff
Merge pull request #625 from gilbertchen/memory_optimization
...
Rewrite the backup procedure to reduce memory usage
2022-04-07 23:26:46 -04:00
Gilbert Chen
fede9c74b5
Merge branch 'master' into memory_optimization
2022-04-07 23:26:14 -04:00
Gilbert Chen
a953c4ec28
Don't parse test parameters in init()
...
This is to make test parameter parsing work with newer versions
of Go
2022-04-07 22:31:18 -04:00
gilbertchen
f52dcf761b
Merge branch 'master' into memory_optimization
2022-03-31 15:05:50 -04:00
Gilbert Chen
ade669d14e
Update totalChunkSize in the chunk operator to show stats during restore
2022-03-04 16:53:40 -05:00
Mark Feit
4743c7ba0d
DOn't cancel the context and use a sane deadline.
2021-12-17 10:09:36 -05:00
Mark Feit
590d3b1b5b
More development
2021-12-04 10:56:47 -05:00
Mark Feit
0590daff85
More dev
2021-12-04 10:53:34 -05:00
Mark Feit
95b1227d93
Use empty context
2021-12-04 10:48:01 -05:00
Mark Feit
1661caeb92
More fixups
2021-12-04 10:37:11 -05:00
Mark Feit
041ba944c4
Typo
2021-12-04 10:24:31 -05:00
Mark Feit
934c2515cc
Import context
2021-12-04 10:23:42 -05:00
Mark Feit
c363d21954
First cut of Swift v2
2021-12-04 10:11:19 -05:00
Gilbert Chen
d9f6545d63
Rewrite the backup procedure to reduce memory usage
...
Main changes:
* Change the listing order of files/directories so that the local and remote
snapshots can be compared on-the-fly.
* Introduce a new struct called EntryList that maintains a list of
files/directories, which are kept in memory when the number is lower, and
serialized into a file when there are too many.
* EntryList can also be turned into an on-disk incomplete snapshot quickly,
to support fast-resume on next run.
* ChunkOperator can now download and upload chunks, thus replacing original
ChunkDownloader and ChunkUploader. The new ChunkDownloader is only used
to prefetch chunks during the restore operation.
2021-10-24 23:34:49 -04:00
Gilbert Chen
68b60499d7
Add a global option to print memory usage
...
This option, -print-memory-usage, will print memory usage every second while
the program is running.
2021-10-15 20:45:53 -04:00
Gilbert Chen
cacf6618d2
Download a fossil directly instead of turning it back to a chunk first
...
This is to avoid the read-after-rename consistency issue where the effect
of renaming may not be observed by the subsequent attempt to download the
just renamed chunk.
2021-10-08 14:04:56 -04:00
Gilbert Chen
e43e848d47
Find the storage path in shared folders first when connecting to Google Drive
...
When connecting to Google Drive with a service account key, only files in the
service account's own hidden drive space are listable. This change finds
the given storage path among shared folders first so that folders from the user
space can be made accessible via service account.
2021-03-09 22:46:23 -05:00
gilbertchen
fd1b7e1d20
Merge pull request #612 from gilbertchen/gcd_impersonate
...
Support GCD impersonation via modified service account file
2021-03-09 10:24:08 -05:00
Gilbert Chen
f83e4f3c44
Fix SNAPSHOT_INACTIVE log message
2021-01-28 00:06:49 -05:00
gilbertchen
ecf5191400
Update README.md
2021-01-04 14:19:51 -05:00
gilbertchen
ba091fbe42
Add a link to the paper
2021-01-04 14:17:48 -05:00
Gilbert Chen
ee9355b974
Check in the paper accepted to IEEE Transactions on Cloud Computing
2021-01-04 10:18:04 -05:00
Gilbert Chen
4cfecf12f8
Show the path in the error when a subdirectory can't be listed
2021-01-04 10:17:11 -05:00
Gilbert Chen
4104c2f934
Exit with code 2 when an invalid command is provided
2021-01-04 10:16:10 -05:00
Gilbert Chen
41a8f657c4
Add test storage for StorageMadeEasy's File Fabric
2020-11-23 14:38:15 -05:00
Gilbert Chen
474f07e5cc
Support GCD impersonation via modified service account file
2020-11-23 09:44:10 -05:00
Gilbert Chen
175adb14cb
Bump version to 2.7.2
2020-11-15 23:20:11 -05:00
Gilbert Chen
ae706e3dcf
Update dependency (for gilbertchen/go-dropbox and github.com/pkg/xattr)
2020-11-15 23:19:20 -05:00
Gilbert Chen
5eed6c65f6
Validate the repository id for the init and add command
...
Only letter, numbers, dashes, and underscores are allowed.
2020-11-04 21:32:07 -05:00
Gilbert Chen
bec3a0edcd
Fixed a bug that caused a fresh restore to fail without the -overwrite option
...
When restoring a file that doesn't exit locally, if the file is large (>100M)
Duplicacy will create an empty sparse file. But this newly created file will
be mistaken for a local copy and hence the restore will fail with a message
suggesting the -overwrite option.
2020-11-03 10:57:47 -05:00
Gilbert Chen
b392302c06
Use github.com/pkg/xattr for reading/writing extended attributes.
...
The one previously used, github.com/redsift/xattr, is old and can only process
user-defined extended attributes, not system ones.
2020-10-16 21:16:05 -04:00
Gilbert Chen
7c36311aa9
Change snapshot source path from / to /System/Volumes/Data
...
Also use a regex to extract the snapshot date from tmutil output.
2020-10-11 15:23:09 -04:00
Gilbert Chen
7f834e84f6
Don't attemp to load verified_chunks when it doesn't exist.
2020-10-09 14:22:45 -04:00
Gilbert Chen
d7c1903d5a
Skip chunks already verified in previous runs for check -chunks.
...
This is done by storing the list of verified chunks in a file
`.duplicacy/cache/<storage>/verified_chunks`.
2020-10-08 19:59:39 -04:00
Gilbert Chen
7da58c6d49
Bump version to 2.7.1
2020-10-01 21:59:48 -04:00
gilbertchen
4402be6763
Update README.md
2020-10-01 14:38:10 -04:00
gilbertchen
3abec4e37a
Update README.md
2020-10-01 14:36:04 -04:00
gilbertchen
dd40d4cd2f
Update README.md
2020-10-01 13:26:50 -04:00
gilbertchen
923e906b7e
Merge pull request #609 from lokeshsammeta/master
...
Update README.md
2020-10-01 13:13:20 -04:00
Gilbert Chen
0da55f95ab
Fixed a 64-bit integer alighment problem on 32-bit OS
...
A new variable was added previosuly which caused a 64-bit variable to be not
aligned on a 8-byte boundary. Go still can't handle such variables on 32-bit
OS.
2020-10-01 10:35:59 -04:00
Gilbert Chen
2f407d6af9
Another change needed for symlinked chunk subdirectories
2020-10-01 09:27:20 -04:00
