the new code requires bucket listing to be on binary keys not just
binary buckets. As this is only intended for use within Riak (where
all keys are buckets are binaries), this constraint seems OK.
A test needed changing to ensure it had a binary key in the bucket.
The CDB FSM process can be blocked by get_positions for all positions,
so loop around the index outside of the FSM process to allow for other
messages to interleave.
This at least checks the file is present, and the Key exists in the
index of that file. If the value is corrupt it will be removed by
compation, and then this will fail (unless the file is never compacted).
TODO: resolve issus of files which are corrupt - but never compacted
- a job for backup?
If the clerk updates the manifest - it might not recognise changes to
the manifest made since the clerk took the manifest. So the penciller
must merge its view of the snapshots back in to the updated manifest
fold objects which snaps in the fold was implemented incorrectly - it
took information from the LedgeCache at the point of the request, not
at the point of the fold. So the LedgerCache SQN may have been
surpassed in the Penciller by the time the fold was called.
riak_kv_sweeper gets the async fold function, then determines if the
function can be called. If the system is busy the fold may be queued,
and may never be acted upon.
This may cause issues with snapshot timeouts etc.
Increase this to 90 minutes. The first time all the snapshots are
rebuilt it may take a long time, but they all get scheduled together -
and queued until concurrency limits allow it to be completed.
currently the snapshot is made on initialisation, and only released
when completed (which may be after the queue). so the last couple of
snapshots were over-shooting the 1 hour.
Cmpaction is overly aggressive. It is a lot of work to compact a run of
files for just 20% reduction in disk space, when disk space for the
Journal (i.e. low IOPS disk space should be relatively inexpensive).
Require at least a 40% reduction for a compaction job.
