Going to abandond this branch for now. The change is beoming
excessively time consuming, and it is not clear that a smaller change
might not achieve more of the objectives.
All this is broken - but perhaps could get picke dup another day.
The manifest had previously been a list for eveyr leevl of the manifest,
and keys were found by folding over the list. By Level 4 the list will
be 4096 items long, and so the fold would be expensive, and would be
required many times.
To make this less expensive an ETS table is to use. However, the ETS
table needs to be shared between snapshots and so in order to use the
ETS the entries to the table need to support multi-versioning - whereby
each clone can see a version of the table at the Manifest SQN the clone
is supporting.
Prior to this refactor, the slot and been made up of four blocks with
an external binary index. Although the form of the index has changed
again, micro-benchmarking once again showed that this was a relatively
efficient mechanism.
the Skiplist range code was needlessly complicated. It may be faster
than the new code, but the complexity delta cannot be support for such a
small change.
This was incovered whilst troubleshooting the initial kv range test.
When rolling we already know the last_key - no need to seek for it on
startup.
The time it takes for this seek needs to be considered with regards to
startup time. Can we do without knowing lastkey?
Performance has regressed following the hashtable change. Speculation
that the hashtable format might not be right, and so there is more
cycling around the hashtree. Logging added.
Move to using the DJ Bernstein Magic Hash consistently, and trying to
make sure we only hash once for each operation (as the hash is more
expensive than phash2).
The improved lookup time for missing keys should allow for the L0 index
to be removed, and hence speed up the completion time for push_mem
operations.
It is expected there will be a second stage of creating a tinybloom as
part of the SFT creation process, and then adding that tinybloom to the
manifest. This will then reduce the message passing required for a GET
not in the cache or higher levels
The penciller attempt to close the L0 file if pending was unpredictable
in behaviour. If a L0 file is still pending it will be lost - but this
is at least a predictable event.
This allows for deleted journals to be retained for a period (the
waste_retnetion_period). The idea being that a backup strategy can
ensure that all journals are backed up, even ones created and removed
from within a backup period - so that any restore pont is possible.
This is also a pre-cursor to removing some of the PromptDelete
complexity from the Inker Clerk - all compactions can prompt deletion as
deletion is now deferred.
There were issues with how the Penciller behaves under ehavy write
pressure - most particularly where there are a large number of keys per
update (i.e. 2i heavy objects). Most immediately the attempt to chekc
whether the l0 file was ready slowed down the process of producing the
L0 file - so back-pressure created more back-pressure.
Going forward want to alter this most significantly as also the work
queue can build up unsustainably. there needs to be some pausing
prompted by the bookie on 'returned', and the use of 'returend when the
work queue exceeds a threshold.
