Gilbert Chen
045be3905b
Better handling of B2 authorization failures
...
This commit fixed 2 issues wrt Backblaze B2 authorization:
* every thread may call b2_authorize_account at the same time when there
are 401 errors
* if B2 has a login outage, then all threads will call b2_authorize_account
repeatedly without delay
A simple solution is to limit one b2_authorize_account call to once every
30 second regardless of how many threads there are. If the call to
b2_authorize_account is not allowed, the random exponential backoff will
be performed.
2019-06-13 22:43:07 -04:00
Gilbert Chen
4da7f7b6f9
Check -files may download a chunk multple times
...
This commit fixed a bug that caused 'check -files' to download the same chunk
multiple times if shared by multiple small files.
2019-06-13 14:47:21 -04:00
Gilbert Chen
41668d4bbd
Update dependency github.com/gilbertchen/go.dbus
2019-06-07 15:17:46 -04:00
Gilbert Chen
9d4ac34f4b
Don't compare hashes of empty files in the diff command
...
Empty files may or may not have a hash depending if the -hash option is used
during backup.
2019-06-06 12:35:34 -04:00
Gilbert Chen
eba5aa6eea
Bump version to 2.2.1
2019-06-04 22:28:04 -04:00
Gilbert Chen
47c4c25d8b
Fixed a bug that restoring files doesn't work due to missing parent directory
...
The root cause was path.Dir can't handle Windows paths that use \ as the
separator.
2019-06-04 21:57:10 -04:00
Gilbert Chen
37781f9540
Swtich CLI licensing to per-computer
2019-05-30 13:35:09 -04:00
Gilbert Chen
6e7d45caac
Add a TRACE message when skipping a file to be restored
2019-05-22 12:03:21 -04:00
Gilbert Chen
18ba415f56
Bump version to 2.2.0
2019-05-06 12:26:40 -04:00
Gilbert Chen
458687d543
The cat command doesn't need to load the entire file into memory
...
It can print out the chunk as soon as a chunk is retrieved. This avoids
reconstructing the file in the memory which can be an issue with large files.
2019-05-03 11:33:16 -04:00
Gilbert Chen
57a408a577
Rework the Backblaze B2 backend
...
* All APIs include UploadFile are done via the call() function
* New retry mechanism limiting the maximum backoff each time to 1 minute
* Add an env var DUPLICACY_B2_RETRIES to specify the number of retries
* Handle special/unicode characters in repositor ids
* Allow a directory in a bucket to be used as the storage destination
2019-04-30 23:31:57 -04:00
Gilbert Chen
a73ed462b6
Roll back the import path change 'import duplicacy/src'
...
Import paths are relative to $GOPATH and $GOROOT, so 'import duplicacy/src'
unfortunately doesn't work.
2019-04-27 22:01:17 -04:00
gilbertchen
e56efc1d3a
Merge pull request #554 from arikorn/ask_b2_application_key
...
Request B2 "Backblaze Account or Application ID"
2019-04-27 10:55:12 -04:00
gilbertchen
bb58f42a37
Merge pull request #529 from turtleleo/patch-1
...
Retry on 408 error from Google Drive (Update to duplicacy_gcdstorage.go)
2019-04-27 10:53:26 -04:00
Thomas Tempelmann
22e8d9e60a
Change the import from "github.com/gilbertchen/duplicacy/src" to "duplicacy/src" so that a forked project uses the forked "src" dir and not the original one.
2019-04-27 00:10:50 -04:00
Gilbert Chen
4eb174cec5
Remove a few util functions that aren't necessary
2019-04-26 23:47:25 -04:00
gilbertchen
6fd3fbd568
Merge pull request #514 from a-s-z-home/filter_extension
...
Filter extension: @ to include another file
2019-04-26 21:56:42 -04:00
Gilbert Chen
a6fe3d785e
Fixed a MoveFile bug in Wasabi when the storage is at the root of a bucket
...
When the storage dir is empty, the destination path passed to the MOVE api starts
with a / which causes Wasabi to fail silently.
2019-04-24 16:48:25 -04:00
Gilbert Chen
1da151f9d9
Add an additional lookup for a chunk that isn't in the chunk list
...
A chunk not in the chunk list may actually exists in two scenarios:
* the chunk may be a special snapshot chunk that contains the chunk sequence,
so it may be resurrected by the chunk downloader if it had been turned into
a fossil before
* if the API to list all chunks doesn't return the complete list due to some
bug
This additional lookup avoid reporting the missing chunk prematurely.
2019-04-21 20:32:21 -04:00
Gilbert Chen
4b69c1162e
Fix a memory issue that check -tabular uses too much memory with many revisions
...
The call to GetSnapshotChunks in ShowStatisticsTabular sets keepChunkHashes to
true -- this can cause too much memory consumption with hundreds of revisions.
2019-04-20 22:47:03 -04:00
Gilbert Chen
abcb4d75c1
Fixed a bug where filenames starting with i or e are mistakenly interpreted as regex
2019-04-07 22:43:36 -04:00
Ari Kornfeld
10d2058738
Request B2 "Backblaze Account or Application ID" (rather than "Account ID")
...
fixes #539 (Duplicacy init for B2 storage still ask for account ID)
2019-04-02 22:20:29 -07:00
Gilbert Chen
43a5ffe011
Fixed a bug where a wrong variable is used as the number of threads
2019-03-13 15:38:26 -04:00
Gilbert Chen
d16273fe2b
Set the content length for upload
2019-03-04 15:34:32 -05:00
Gilbert Chen
2b56d576c7
Fixed a webdav compatibility issue with rclone and other bugs
2019-02-26 14:00:02 -05:00
turtleleo
82c6c15f1c
Update duplicacy_gcdstorage.go
...
Add automatic retry on receiving error 408 (request timeout) from Google Drive.
2019-01-16 13:12:46 -05:00
gilbertchen
bebd7c4b77
Merge pull request #495 from plasticrake/environment-variables
...
Replace special characters in environment variable name with underscores
2019-01-04 17:04:29 -05:00
gilbertchen
46376d82ed
Merge pull request #489 from gilbertchen/sftp_retry
...
Retry on EOF errors in the SFTP backend
2019-01-04 13:53:44 -05:00
gilbertchen
c4a3dd1eeb
Merge pull request #454 from mikecook/master
...
spelling fix, go fmt, go vet
2019-01-04 13:50:17 -05:00
gilbertchen
31c25e98f7
Merge branch 'master' into master
2019-01-04 13:48:44 -05:00
gilbertchen
242db8377e
Merge pull request #447 from s4y/patch-1
...
Acknowledge malware/spam warnings from GCD
2019-01-04 13:33:11 -05:00
Gilbert Chen
e6d8b7d070
Use 1024*1024 as 1M as opposed to 10^6
2019-01-04 13:29:30 -05:00
Gilbert Chen
bb652d0a8c
Add a Sync call before close when uploading a file to local storage
2019-01-03 12:44:50 -05:00
Gilbert Chen
a354d03bc9
Remove a binary file accidentally checked in
2019-01-02 21:36:04 -05:00
Michael Cook
4b9524bd43
go vet: unreachable code
2018-12-29 13:20:11 +01:00
Michael Cook
a782d42ad6
go vet: result of fmt.Errorf call not used
2018-12-29 13:20:10 +01:00
Michael Cook
0762c448c4
gofmt -s
2018-12-29 13:20:10 +01:00
Michael Cook
741644b575
spelling
2018-12-29 13:04:40 +01:00
a-s-z-home
df7487cc0b
Merge remote-tracking branch 'origin/master' into filter_extension
2018-11-15 01:40:39 +01:00
Gilbert Chen
8aa67c8162
Support ssh private key files encrypted by passphrases
2018-11-09 14:17:56 -05:00
Gilbert Chen
53548a895f
Add the \?\ prefix to all paths on Windows
2018-11-08 21:29:02 -05:00
a-s-z-home
5e8baab4ec
- Reverted changes to exclude mechanism of .duplicacy directory.
2018-11-05 22:39:11 +01:00
a-s-z-home
e1fa39008d
Use new filter processing function for restore command.
...
- You can now include a filter file by using "@<filename>".
2018-11-05 00:59:39 +01:00
a-s-z-home
aaebf4510c
- Replaced static check for .duplicacy directory with usage of predefined filters.
...
Do not "misuse" property nobackupFile to trigger this feature.
- Restructured ProcessFilterFile function and splitted it in smaller parts.
- Prepare usage of new filter syntax for arguments of restore command.
2018-11-05 00:32:12 +01:00
a-s-z-home
96dd28995b
Added an include mechanism for filter file.
...
- Using @<filename>, you can now include other files. Relative paths are supported.
This is useful, if you have several repositories with some different filters and a common filter base set.
2018-11-03 20:39:03 +01:00
a-s-z-home
166f6e6266
Added string array helper functions Contains and Find.
2018-11-03 20:20:00 +01:00
a-s-z-home
86c89f43a0
Automatically exclude .duplicacy directory only, if nobackup_file is not
...
set.
2018-11-03 20:13:43 +01:00
Gilbert Chen
2e5cbc73b9
Bump version to 2.1.2
2018-11-03 11:45:50 -04:00
Gilbert Chen
21b3d9e57f
Padding size was incorrect -- didn't pad to multiples of 256
2018-11-03 11:42:03 -04:00
Gilbert Chen
244b797a1c
Print the number of files if available in the snapshot file
2018-11-03 10:38:35 -04:00
