diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 8e8c142..d095821 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -120,7 +120,7 @@ Three potential recovery strategies are supported to provide some flexibility fo - retain - on compaction KeyDeltas are retained in the Journal, only values are removed. -- recalc (not yet implemented) - the compaction rules assume that on recovery the key changes will be recalculated by comparing the change with the current database state. In recovery the key changes will be recalculated by comparing the change with the current database state. +- recalc - the compaction rules assume that on recovery the key changes will be recalculated by comparing the change with the current database state. In recovery the key changes will be recalculated by comparing the change with the current database state. A user-defined function should be passed in at startup to achieve this recalculation (to override `leveled_head:diff_indexspeacs/3`). ### Hot Backups diff --git a/docs/STARTUP_OPTIONS.md b/docs/STARTUP_OPTIONS.md index e73a021..c802f73 100644 --- a/docs/STARTUP_OPTIONS.md +++ b/docs/STARTUP_OPTIONS.md @@ -77,11 +77,11 @@ However, what if the Ledger had been erased? This could happen due to some corr The are three potential strategies: - - `skip` - don't worry about this scenario, require the Ledger to be backed up; + - `recovr` - don't worry about this scenario, require the Ledger to be backed up; - `retain` - discard the object itself on compaction but keep the key changes; - `recalc` - recalculate the indexes on reload by comparing the information on the object with the current state of the Ledger (as would be required by the PUT process when comparing IndexSpecs at PUT time). -There is no code for `recalc` at present it is simply a logical possibility. So to set a reload strategy there should be an entry like `{reload_strategy, [{TagName, skip|retain}]}`. By default tags are pre-set to `retain`. If there is no need to handle a corrupted Ledger, then all tags could be set to `skip`. +To set a reload strategy requires a list of tuples to match tag names to strategy `{reload_strategy, [{TagName, recovr|retain|recalc}]}`. By default tags are pre-set to `retain`. If there is no need to handle a corrupted Ledger, then all tags could be set to `recovr` - this assumes that either the ledger files are protected by some other means from corruption, or an external anti-entropy mechanism will recover the lost data. ## Compression Method diff --git a/src/leveled_bookie.erl b/src/leveled_bookie.erl index 27c158d..1aeea16 100644 --- a/src/leveled_bookie.erl +++ b/src/leveled_bookie.erl @@ -309,10 +309,14 @@ % resilience outside of the store), or retain (retain a history of % key changes, even when the object value has been compacted). % - % There is a third, theoretical and untested strategy, which is - % recalc - which would require when reloading the Ledger from the - % Journal, to recalculate the index changes based on the current - % state of the Ledger and the object metadata. + % There is a third strategy, which is recalc, where on reloading + % the Ledger from the Journal, the key changes are recalculated by + % comparing the extracted metadata from the Journal object, with the + % extracted metadata from the current Ledger object it is set to + % replace (should one be present). Implementing the recalc + % strategy requires a override function for + % `leveled_head:diff_indexspecs/3`. + % A function for the ?RIAK_TAG is provided and tested. % % reload_strategy options are a list - to map from a tag to the % strategy (recovr|retain|recalc). Defualt strategies are: