This bug leaks a chunk every time files in a revision are listed. Not a big
deal for backup and restore, but it becomes problematic when listing files in
many revisions for commands such check and history.
The version bit should not be set to 1 when encoding a snapshot. Instead,
it must be set to 1 on snapshot creation.
To correctly process old snapshots encoded incorrectly with version bit set
to 1, the first byte of the encoded file list is also checked. If the first
byte is `[`, then it must be an old snapshot, since the file list in the new
snapshot format always starts with a string encoded in msgpack, the first
byte of which can't be `[`.
This option is useful only when erasure coding is enabled. It will
download and re-upload chunks that contain corruption but are
generally recoverable. It can also be used to fix chunks that
are created by 3.0.1 on arm64 machines with wrong hashes.
highwayhash 1.0.1 contains a bug leading to incorrect hashes on arm64 machines.
The 1.0.1 version is retained in github.com/gilbertchen/highwayhash so the hash
can be checked again if a mismatch is detected by 1.0.2.
This commit fixed 2 bugs. The first bug occurs when an incomplete backup
contains a zero-byte file and no chunks. The second bug occurs when the
repository contains only zero-byte files.
The url format is storj://satellite/bucket/path. You can get the
satellite along with the api access key when requesting an Access
Grant of type API Access.
Main changes:
* Change the listing order of files/directories so that the local and remote
snapshots can be compared on-the-fly.
* Introduce a new struct called EntryList that maintains a list of
files/directories, which are kept in memory when the number is lower, and
serialized into a file when there are too many.
* EntryList can also be turned into an on-disk incomplete snapshot quickly,
to support fast-resume on next run.
* ChunkOperator can now download and upload chunks, thus replacing original
ChunkDownloader and ChunkUploader. The new ChunkDownloader is only used
to prefetch chunks during the restore operation.
This is to avoid the read-after-rename consistency issue where the effect
of renaming may not be observed by the subsequent attempt to download the
just renamed chunk.
When connecting to Google Drive with a service account key, only files in the
service account's own hidden drive space are listable. This change finds
the given storage path among shared folders first so that folders from the user
space can be made accessible via service account.
