jgaunt/duplicacy
1
0
Fork 0
mirror of https://github.com/gilbertchen/duplicacy synced 2025-12-15 15:53:26 +00:00
Code Issues Releases Wiki Activity

Updated Storage Backends (markdown)

Mark Feit
2018-03-19 12:22:17 -04:00
parent 11a81edde8
commit 9c8bf69459
1 changed files with 12 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
Storage-Backends.md

@@ -60,16 +60,26 @@ Storage URL: s3c://region@host/bucket/path/to/storage
<details> <summary>Wasabi</summary>
```
Storage URL: s3://us-east-1@s3.wasabisys.com/bucket/path/to/storage
Storage URL: wasabi://region@s3.wasabisys.com/bucket/path
```
Where `region` is the storage region (currently only `us-east-1`), `bucket` is the name of the bucket and `path` is the path to the top of the Duplicacy storage within the bucket.
[Wasabi](https://wasabi.com) is a relatively new cloud storage service providing a S3-compatible API. It is well-suited for storing backups, because it is much cheaper than Amazon S3 with a storage cost of $0.0039/GB/month (see note below) and a download fee of $0.04/GB, and no additional charges on API calls.
### S3 and Billing
#### Short Version
The `s3` storage backend renames objects with a copy and delete which is inexpensive for AWS but more expensive for Wasabi. Use the `wasabi` backend for it to be handled properly.
#### Long Version
Wasabi's billing model differs from Amazon's in that any object created incurs charges for 90 days of storage, even if the object is deleted earlier than that, and then the monthly rate thereafter.
As part of the [process for purging data which is no longer needed](https://github.com/gilbertchen/duplicacy/wiki/Lock-Free-Deduplication#two-step-fossil-collection), Duplicacy renames objects. Because S3 does not support renaming objects, Duplicacy does the equivalent by using S3's copy operation to create a second object with the new name then deleting the one with the old name. S3-style renaming with Wasabi will incur additional charges during fossilization becasue of the additional objects it creates. For example, if a new 1 GB file is backed up in chunks on day 1, the initial storage will incur fees of at least $0.0117 (three months at $0.0039 each). If the file goes away and all snapshots that contained it are pruned on day 50, renaming the chunks will create an additional 1 GB of objects with a newly-started 90-day clock at a cost of $0.0117.
As part of the [process for purging data which is no longer needed](https://github.com/gilbertchen/duplicacy/wiki/Lock-Free-Deduplication#two-step-fossil-collection), Duplicacy renames objects. Because S3 does not support renaming objects, Duplicacy's `s3` backend does the equivalent by using S3's copy operation to create a second object with the new name then deleting the one with the old name. S3-style renaming with Wasabi will incur additional charges during fossilization becasue of the additional objects it creates. For example, if a new 1 GB file is backed up in chunks on day 1, the initial storage will incur fees of at least $0.0117 (three months at $0.0039 each). If the file goes away and all snapshots that contained it are pruned on day 50, renaming the chunks will create an additional 1 GB of objects with a newly-started 90-day clock at a cost of $0.0117.
The `wasabi` backend uses Wasabi's rename operation to avoid these extra charges.
### Snapshot Pruning