gilbertchen
1fedfd1b1a
Merge branch 'master' into mac-exclude
2020-09-25 20:13:43 -04:00
gilbertchen
3fd3f6b267
Merge pull request #606 from gilbertchen/erasure_coding
...
Implement Erasure Coding
2020-09-25 14:30:39 -04:00
Gilbert Chen
e3e3e97046
Improvements for the WebDAV backend
...
* CreateDirectory() looks up the directory in the cache first and don't create
it if found in cache
* ListFiles() puts subdirectories in the cache
* If CreateDirectory() encounters EOF, assume the directory already exists
2020-09-25 13:56:44 -04:00
Gilbert Chen
3f29ec2ffb
File metedata must be restored even if the content is unchanged.
2020-09-24 22:44:43 -04:00
Gilbert Chen
947006411b
Improvements for the copy command
...
* Added a `-download-threads` option for specifying the number of downloading
threads
* Show progress log messages during copy
2020-09-24 14:56:19 -04:00
Gilbert Chen
6841c989c6
Fixed a bug that caused check -chunks -persist to succeed with broken chunks
...
The bug was not setting the `isBroken` flag in WaitForChunk()
2020-09-24 14:53:42 -04:00
Gilbert Chen
d0b3b5dc2e
Print progress logs when verifying chunks (check -chunks)
2020-09-23 09:02:53 -04:00
Gilbert Chen
73ae3f809e
Revert "Add a -max-list-rate option to backup to slow down the listing"
...
This reverts commit 67a3103467
2020-09-22 22:08:43 -04:00
Gilbert Chen
67a3103467
Add a -max-list-rate option to backup to slow down the listing
...
This option sets the maximum number of files that can be listed in one
second.
2020-09-22 08:27:09 -04:00
Gilbert Chen
6ee01a2e74
Allow RSA keys to be passed directly via CML instead of a file
...
This change is intended to be used by the web GUI to create an RSA encrypted
storage.
2020-09-20 20:03:52 -04:00
Gilbert Chen
b7d820195a
Remove a debug log message accidentally checked in
2020-09-18 14:59:44 -04:00
Gilbert Chen
16d2c14c5a
Follow-up changes for the -persist PR
...
* Restore/check should report an error instead of a success at the end if there
were any errors and -persist is specified
* Don't compute the file hash before passing the file to the chunk maker; this is
redundant as the chunk maker will produce the file hash
* Add a LOG_WERROR function to switch between LOG_WARN and LOG_ERROR dynamically
2020-09-18 11:23:35 -04:00
Gilbert Chen
eecbb8fa99
Fix OneDrive Business and improve retry mechanism.
...
* Switch to the upload-by-session api for OneDrive Bussiness as their servers
may keep incomplete files when an upload is aborted when the simple upload API
is used
* Use the delay value in the http Retry-After header if there is one
* Decorate the https traffic in the hope of less rate limiting.
2020-09-18 10:19:03 -04:00
Gilbert Chen
97bae5f1a3
Close the response body when 301 is returned in the WebDAV backend
2020-09-14 11:36:22 -04:00
Gilbert Chen
40243fb043
Fixed a compile error in previous checkin
2020-09-12 11:39:46 -04:00
Gilbert Chen
403df1fd06
Added a call to discard http response where it was missed in previous fixes.
...
Also added a missing import "io/ioutil".
2020-09-09 21:47:01 -04:00
gilbertchen
4369bcfc0b
Merge pull request #549 from Jos635/master
...
Improve WebDAV performance
2020-09-09 16:02:36 -04:00
gilbertchen
d2b08aebee
Merge pull request #594 from alecuyer/fix/swift
...
Call Authenticate() before using Swift storage
2020-09-09 15:48:36 -04:00
gilbertchen
948994c2b6
Merge pull request #595 from twlee79/add_persist_pr
...
Adds -persist option to check and restore commands to continue despite errors
2020-09-09 15:42:46 -04:00
gilbertchen
ca4d004aca
Merge branch 'master' into add_persist_pr
2020-09-09 15:42:01 -04:00
Gilbert Chen
ce472fe375
Show erasure coding/rsa encryption if enabled for backup and copy
2020-09-04 10:43:37 -04:00
Gilbert Chen
923a6fbc5b
Implement Erasure Coding
2020-09-03 12:54:48 -04:00
Gilbert Chen
670cbcd776
Bump version to 2.6.2
2020-08-30 22:14:58 -04:00
Gilbert Chen
fd469bae9e
Check the returned value of Close() when uploading a chunk file via SFTP.
2020-08-29 22:22:51 -04:00
Gilbert Chen
acef01770a
Bump version to 2.6.1
2020-07-07 23:27:11 -04:00
Gilbert Chen
1eb1fb14a8
Don't throw an error on 0-byte chunk files with suffix '.tmp'.
2020-07-07 23:25:09 -04:00
Gilbert Chen
8b489f04eb
Bump version to 2.6.0
2020-07-05 21:27:44 -04:00
Gilbert Chen
089e19f8e6
Add a -key-passphrase option to pass in passphrase for RSA private key
...
This option is mainly for the web GUI which currently doesn't have a way to
specify the passphrase to decrypt the RSA private key.
2020-07-05 20:58:07 -04:00
Gilbert Chen
1da7e2b536
Fix a crash when a username is not specified with the WebDAV backend
2020-07-03 12:29:53 -04:00
Gilbert Chen
ed8b4393be
Add a new backend for StorageMadeEasy's File Fabric storage.
...
The storage url is fabric://username@storagemadeeasy.com/path/to/storage.
2020-07-03 12:07:33 -04:00
Gilbert Chen
5e28dc4911
Update go-dropbox dependency to fix 0-byte file uploading.
...
Incorporate gilbertchen/go-dropbox/commit/0baa9015ac2547d8b69b2e88c709aa90cfb8fbc1.
2020-07-03 11:52:16 -04:00
Gilbert Chen
f2f07a120d
Retry on "unexpected EOF" errors for the webdav backend.
2020-07-03 11:48:30 -04:00
Gilbert Chen
153f6a2d20
Use multiple threads to list the chunks directory for Google Drive
2020-06-15 12:49:13 -04:00
Gilbert Chen
5d45999077
Clear the loaded content after a snapshot has been verified
...
The snapshot content is loaded before verifying the snapshot, but after that
it isn't used anymore so it should be released to save memory.
2020-06-10 10:08:53 -04:00
Gilbert Chen
1adcf56890
Add an SFTP backend that supports more ciphers and kex algorithms.
...
"sftpc://" supports all algorithms implemented in golang.org/x/crypto/ssh,
especially including those weak ones that are excluded from the defaults.
2020-06-08 11:24:20 -04:00
Gilbert Chen
09e3cdfebf
Ignore 0-byte chunks passed in by the chunk reader.
...
The fixed-size chunk reader may create 0-byte chunks from empty files. This
may cause validation errors when preparing the snapshot file as the last step
of a backup.
2020-06-08 10:53:01 -04:00
Gilbert Chen
fe854d469d
Error out in the check command if there are 0-size chunks.
2020-06-02 11:37:12 -04:00
Gilbert Chen
76f1274e13
Bump version to 2.5.2
2020-05-10 21:41:30 -04:00
Gilbert Chen
9c3122b814
Fixed a bug causing the OneDrive Business token file path to be not saved
...
The file path was saved under the name `one_token` in Keychain/keyring, rather
than the correct name 'odb_token'.
2020-05-10 20:22:49 -04:00
Gilbert Chen
6ca8b8dff0
Disable snapshot cache when checking chunks
...
Otherwise every chunk will be stored to the snapshot cache when the `-chunks`
option is specified.
2020-05-10 00:26:47 -04:00
Tet Woo Lee
4ae16dec7f
add -persist in check and restore mode (for PR)
2020-05-06 18:39:52 +12:00
Alexandre Lécuyer
dae040681d
Call Authenticate() before using Swift storage
...
Failing to call Authenticate() before using the swift connection will
cause a panic in recent versions of the swift library.
2020-05-05 23:09:28 +02:00
Gilbert Chen
51cbf73caa
Bump version to 2.5.1
2020-04-17 15:57:08 -04:00
Gilbert Chen
835af11334
Fixed a bug in ssh login with encrypted private key
...
Check the type of the returned error instead of the error message to
determine if the private key file is encrypted by a passphrase.
2020-04-17 15:55:30 -04:00
Gilbert Chen
4c3557eb80
Bump version to 2.5.0
2020-04-09 23:22:32 -04:00
Gilbert Chen
eebcece9e0
Update github.com/aws/aws-sdk-go and google.golang.org/api to the latest
2020-04-09 23:21:55 -04:00
gilbertchen
8c80470c29
Merge pull request #593 from freaksdotcom/readall
...
Call ReadAll() on the body io.ReadCloser to allow the http keepalive connection to be reused.
2020-04-09 21:12:41 -04:00
Gilbert Chen
bcb889272d
Add test for Google Shared Drive
2020-04-09 00:04:30 -04:00
Gilbert Chen
79d8654a12
Allow the name of Google Shared Drive to be used in the storage url
...
The previous PR only accepts the id of the shared drive, which is not very
memorable. This commit makes it able to specify the drive by id or by name.
2020-04-08 23:59:15 -04:00
Brandon High
6bf0d2265c
Call ReadAll() on the body io.ReadCloser to allow the http keepalive connection to be reused.
2020-04-07 22:07:41 -07:00
