Check SQN order fold does not fold beyond end of snapshot

the Journla snapshot is not a true snapshot, in that the active file in the snapshot can still be taking appends.  So when getting a snapshot it is necessary to check if folding over the snapshot that the SQN is <= JournalSQN when the snapshot is taken.

Normally consistency of the snapshot is managed as the operation depends on the penciller, and the penciller *is* a snapshot.  Not in this case, as the penciller will return true on a sqn check if the pcl SQN is behind the Journal.  So the Journal folder, has been given an additionla check to stop at the JournalSQN.

This is perhaps a fault in the pcl check sqn, which should only return true on an exact match?  I'm nervous about changing this though, so we have a less pure fix for now.
This commit is contained in:
Martin Sumner 2019-02-14 21:14:11 +00:00
parent 5799f06452
commit db1486fa36
4 changed files with 102 additions and 14 deletions

View file

@ -363,12 +363,18 @@ foldobjects_allkeys(SnapFun, Tag, FoldObjectsFun, sqn_order) ->
fun() ->
{ok, LedgerSnapshot, JournalSnapshot} = SnapFun(),
{ok, JournalSQN} = leveled_inker:ink_getjournalsqn(JournalSnapshot),
IsValidFun =
fun(Bucket, Key, SQN) ->
LedgerKey = leveled_codec:to_ledgerkey(Bucket, Key, Tag),
leveled_penciller:pcl_checksequencenumber(LedgerSnapshot,
LedgerKey,
SQN)
CheckSQN =
leveled_penciller:pcl_checksequencenumber(LedgerSnapshot,
LedgerKey,
SQN),
% Need to check that we have not folded past the point
% at which the snapshot was taken
(JournalSQN >= SQN) and CheckSQN
end,
BatchFoldFun =