Gilbert Chen
4da7f7b6f9
Check -files may download a chunk multple times
...
This commit fixed a bug that caused 'check -files' to download the same chunk
multiple times if shared by multiple small files.
2019-06-13 14:47:21 -04:00
Gilbert Chen
9d4ac34f4b
Don't compare hashes of empty files in the diff command
...
Empty files may or may not have a hash depending if the -hash option is used
during backup.
2019-06-06 12:35:34 -04:00
Gilbert Chen
458687d543
The cat command doesn't need to load the entire file into memory
...
It can print out the chunk as soon as a chunk is retrieved. This avoids
reconstructing the file in the memory which can be an issue with large files.
2019-05-03 11:33:16 -04:00
Gilbert Chen
4eb174cec5
Remove a few util functions that aren't necessary
2019-04-26 23:47:25 -04:00
Gilbert Chen
1da151f9d9
Add an additional lookup for a chunk that isn't in the chunk list
...
A chunk not in the chunk list may actually exists in two scenarios:
* the chunk may be a special snapshot chunk that contains the chunk sequence,
so it may be resurrected by the chunk downloader if it had been turned into
a fossil before
* if the API to list all chunks doesn't return the complete list due to some
bug
This additional lookup avoid reporting the missing chunk prematurely.
2019-04-21 20:32:21 -04:00
Gilbert Chen
4b69c1162e
Fix a memory issue that check -tabular uses too much memory with many revisions
...
The call to GetSnapshotChunks in ShowStatisticsTabular sets keepChunkHashes to
true -- this can cause too much memory consumption with hundreds of revisions.
2019-04-20 22:47:03 -04:00
Michael Cook
0762c448c4
gofmt -s
2018-12-29 13:20:10 +01:00
Michael Cook
741644b575
spelling
2018-12-29 13:04:40 +01:00
Gilbert Chen
244b797a1c
Print the number of files if available in the snapshot file
2018-11-03 10:38:35 -04:00
Gilbert Chen
073292018c
Don't show snapshots whose tags don't match the given one
2018-10-28 23:30:22 -04:00
Gilbert Chen
15f15aa2ca
Show more statistics in the check command
2018-10-28 23:27:36 -04:00
Gilbert Chen
22a0b222db
Align snapshot times to the beginning of days when calculating the differences
2018-09-08 20:31:49 -04:00
Gilbert Chen
e8b8922754
Continue to check other snapshots when one snapshot has missing chunks
2018-08-06 21:20:04 -04:00
Gilbert Chen
93cc632021
Record deleted snapshots in the fossil collection and if any deleted snapshot still exist nude the fossil collection
2018-08-04 22:59:25 -04:00
Gilbert Chen
f304b64b3f
Removed a redundant call to manager.chunkOperator.Resurrect
2018-08-03 11:32:24 -04:00
Gilbert Chen
8ae7d2a97d
Remove extra newline in the PRUNE_NEWSNAPSHOT log message
2018-07-26 21:24:33 -04:00
Gilbert Chen
72dfaa8b6b
Fixed a bug causing a new snapshot to be not counted when deciding which fossils can be deleted
2018-07-23 22:08:08 -04:00
Gilbert Chen
f68eb13584
A few fixes for multi-threaded pruning
2018-06-05 16:09:12 -04:00
Gilbert Chen
dd53b4797e
Implement multi-threaded pruning
2018-06-04 21:52:07 -04:00
Gilbert Chen
0e585e4be4
Fixed a crashing bug when showing the history of excluded files
2018-05-30 12:05:40 -04:00
Gilbert Chen
e03cd2a880
Add unreferenced fossils to the fossil collection instead of deleting them immediately
2018-05-29 12:57:38 -04:00
Frank
4dd5c43307
Add nobackup-file preference.
...
Directories containing a file with this name will not be backed up. I find it easier to drop a .nobackup file in directories I don't want backed up instead of maintaining a file of exclusions. This is also useful for scripts that create data in the repository but don't want it to be backed up.
2018-04-01 12:50:00 -07:00
Gilbert Chen
5d2242d39d
Preserve the list of chunk hashes for the latest snapshot when cleaning local snapshot cache
2018-03-27 23:34:40 -04:00
gilbertchen
13fffc2a11
Merge pull request #329 from pdf/prune_memory
...
Reduce memory consumption for prune operation
2018-03-20 14:32:12 -04:00
Gilbert Chen
e07226bd62
Retention policy erroneously apply to snapshots without the specified tags
2018-02-10 21:33:01 -05:00
Gilbert Chen
7230ddbef5
Clear the attributes from last snapshot after loading to save memory
2018-01-28 16:54:06 -05:00
Gilbert Chen
d330f61d25
Limit derivation key to 64 bytes since snapshot file path used as key may be longer
2018-01-20 23:52:35 -05:00
Peter Fern
57082cd1d2
Reduce memory consumption for prune operation
...
For non-exhaustive prune, consider only target chunks instead of mapping
all chunks in repository.
2018-01-21 10:12:09 +11:00
Peter Fern
bd5a689b7d
Add keepChunkHashes flag to GetSnapshotChunks, allowing reduced memory
2018-01-21 10:11:59 +11:00
Gilbert Chen
f944e01a02
Fix typos
2017-12-15 08:24:15 -05:00
Gilbert Chen
6f69aff712
Disable caching when retriving files in SnapshotManager
2017-11-27 23:36:09 -05:00
Gilbert Chen
86767b3df6
Implement new chunk directory structure
2017-11-07 12:05:39 -05:00
Gilbert Chen
9d38b49e42
Remove the extra new line for the cat command
2017-10-10 22:35:09 -04:00
gilbertchen
a342431b3c
Merge pull request #223 from lowne/prune-logging
...
add more detailed revision deletion logging on prune
2017-10-05 22:35:02 -04:00
Mark Lowne
86a6ededab
add more detailed revision deletion logging on prune
2017-10-03 16:17:23 +02:00
Gilbert Chen
7162d8916e
Use math.MaxInt32 to avoid a build error on 32-bit platforms
2017-09-29 22:20:37 -04:00
Gilbert Chen
923cd0aa63
Run goimports on all source files
2017-09-20 23:07:43 -04:00
gilbertchen
978212fd75
Merge pull request #196 from fracai/more-check-stats
...
print additional, table-formatted stats for CHECK
2017-09-20 17:16:23 -04:00
Arno Hautala
45bc778898
restored original stats output, enabled option to switch to tabular
2017-09-19 02:07:35 -04:00
Arno Hautala
d5d7649041
added -tabular to check options
2017-09-19 01:13:38 -04:00
Arno Hautala
f1fe64b9cc
moving "func min" to MinInt in utils
2017-09-19 01:06:08 -04:00
Arno Hautala
e2fe57e959
reorder check -stats columns
2017-09-19 00:59:38 -04:00
gilbertchen
fab9cc77c6
Merge pull request #180 from niknah/dry_run
...
Added backup --dry-run option.
2017-09-18 22:32:21 -04:00
niknah
f25783d59d
Comments from @gilbertchen, thanks.
2017-09-18 01:38:51 +10:00
Arno Hautala
d3cea2c7d0
print additional, table-formatted stats for CHECK
2017-09-15 23:34:06 -04:00
niknah
f74ea0368e
ListFiles Don't delete or create directories in dryRun
2017-09-15 02:21:07 +10:00
gilbertchen
554f63263f
Merge pull request #183 from jt70471/prune-patch-1
...
fix prune bug when last snapshot is removed for issue #182
2017-09-14 09:33:39 -04:00
niknah
03c2a190ee
Don't create extra directories on --dry-run
2017-09-14 14:20:58 +10:00
Jeff Thompson
84fc1343a7
fix prune bug when last snapshot is removed for issue #182
2017-09-12 12:38:27 -05:00
Emanuele Trombetta
c42a5a86a4
Fix typos
...
* Update duplicacy_main.go
* Update duplicacy_snapshotmanager.go
2017-09-12 07:49:07 +02:00
