Change to 5 blocks is intended to make the blocks in lookup slots
fractionally smaller, but more importantly to introduce a middle block
that cna be opened in a binary-split style fashion to reduce the number
of blocks that need to be opened for range queries. Worst case for
full slots is 3 blocks now not 4.
Still not clear if yielding is the cause of memory problems, but taking
it away universally has impacted throughput. At the very least we
should continue to yield on high-contention files (those at higher
levels), where the processes are more likely to be quickly terminated
anyway allowing GC to be invoked.
There was complicated and confusing code that achieved nothing for
effiency when trimming slots. the expensive part (binary_to_term) was
still needed on every block, and it was hard to get code coverage and
make sense of what it was really trying to achieve.
This is now much simpler - and may set us up for potential further
indexing help.
This log under 2i load appears thousands of times per second. Not
sustainable as an info log. Will need to think about how to manage
this, but setting back to debug for now
RTrim only worked in special case of key matching, that would never
occur in real world range query. RTrim should really check for key
passing.
Returning empty list should not be possible - unless the query is
outside of the range entirely (and such a query should never go to this
SST).
No evidence from valume test that the scan width has made a posiitve
difference - so reverting, but not fully as slots may now be twice as
big, so sticking to half previous value
When testing with large numbers of 2i terms (and hence more Riak
Metadata), there is a surge in slow response times when there are
multiple concurrent merge events.
This could be veyr short term CPU starvation because of the merge
process. Perhaps it is delays waiting for the scan to complete -
smaller scanwidth may mena more interleaving and less latency?
A mistake meant resetting to lookup on a skipped key would cause issues
if the skipped key ocurred under a no_lookup slot after the ?SLOT_SIZE
had been reached. This caused the slot to switch to lookup, but beyond
the maximum size
The manifest and the logs are bloated by having the full file path for
every filename in there - given the root path is constant.
Could also cause issues if the mount point is ever changed.
