umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions

Check SQN order fold does not fold beyond end of snapshot

Browse source 
the Journla snapshot is not a true snapshot, in that the active file in the snapshot can still be taking appends.  So when getting a snapshot it is necessary to check if folding over the snapshot that the SQN is <= JournalSQN when the snapshot is taken.

Normally consistency of the snapshot is managed as the operation depends on the penciller, and the penciller *is* a snapshot.  Not in this case, as the penciller will return true on a sqn check if the pcl SQN is behind the Journal.  So the Journal folder, has been given an additionla check to stop at the JournalSQN.

This is perhaps a fault in the pcl check sqn, which should only return true on an exact match?  I'm nervous about changing this though, so we have a less pure fix for now.
This commit is contained in:
Martin Sumner 2019-02-14 21:14:11 +00:00
parent 5799f06452
commit db1486fa36
4 changed files with 102 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

65
src/leveled_bookie.erl
View file

@ -3021,6 +3021,71 @@ erase_journal_test() ->
?assertMatch(500, HeadsNotFound2),
ok = book_destroy(Bookie2).
sqnorder_fold_test() ->
RootPath = reset_filestructure(),
{ok, Bookie1} = book_start([{root_path, RootPath},
{max_journalsize, 1000000},
{cache_size, 500}]),
ok = book_put(Bookie1,
<<"B">>, <<"K1">>, {value, <<"V1">>}, [],
?STD_TAG),
ok = book_put(Bookie1,
<<"B">>, <<"K2">>, {value, <<"V2">>}, [],
?STD_TAG),
FoldObjectsFun = fun(B, K, V, Acc) -> Acc ++ [{B, K, V}] end,
{async, ObjFPre} =
book_objectfold(Bookie1,
?STD_TAG, {FoldObjectsFun, []}, true, sqn_order),
{async, ObjFPost} =
book_objectfold(Bookie1,
?STD_TAG, {FoldObjectsFun, []}, false, sqn_order),
ok = book_put(Bookie1,
<<"B">>, <<"K3">>, {value, <<"V3">>}, [],
?STD_TAG),
ObjLPre = ObjFPre(),
?assertMatch([{<<"B">>, <<"K1">>, {value, <<"V1">>}},
{<<"B">>, <<"K2">>, {value, <<"V2">>}}], ObjLPre),
ObjLPost = ObjFPost(),
?assertMatch([{<<"B">>, <<"K1">>, {value, <<"V1">>}},
{<<"B">>, <<"K2">>, {value, <<"V2">>}},
{<<"B">>, <<"K3">>, {value, <<"V3">>}}], ObjLPost),
ok = book_destroy(Bookie1).
sqnorder_mutatefold_test() ->
RootPath = reset_filestructure(),
{ok, Bookie1} = book_start([{root_path, RootPath},
{max_journalsize, 1000000},
{cache_size, 500}]),
ok = book_put(Bookie1,
<<"B">>, <<"K1">>, {value, <<"V1">>}, [],
?STD_TAG),
ok = book_put(Bookie1,
<<"B">>, <<"K1">>, {value, <<"V2">>}, [],
?STD_TAG),
FoldObjectsFun = fun(B, K, V, Acc) -> Acc ++ [{B, K, V}] end,
{async, ObjFPre} =
book_objectfold(Bookie1,
?STD_TAG, {FoldObjectsFun, []}, true, sqn_order),
{async, ObjFPost} =
book_objectfold(Bookie1,
?STD_TAG, {FoldObjectsFun, []}, false, sqn_order),
ok = book_put(Bookie1,
<<"B">>, <<"K1">>, {value, <<"V3">>}, [],
?STD_TAG),
ObjLPre = ObjFPre(),
?assertMatch([{<<"B">>, <<"K1">>, {value, <<"V2">>}}], ObjLPre),
ObjLPost = ObjFPost(),
?assertMatch([{<<"B">>, <<"K1">>, {value, <<"V3">>}}], ObjLPost),
ok = book_destroy(Bookie1).
check_notfound_test() ->
ProbablyFun = fun() -> probably end,
MissingFun = fun() -> missing end,