umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
1612 commits 1 branch 0 tags 25 MiB
Commit graph

1104 commits

Author SHA1 Message Date
martinsumner
fce7cbd2ef Add sync_strategy comments 2017-02-27 17:32:38 +00:00
martinsumner
2db28c3028 Tidy line length 2017-02-27 12:24:26 +00:00
martinsumner
8d3f4bd4dc Comment API on Bookie 
Make it easier to understand how to work the store
 2017-02-27 11:58:16 +00:00
martinsumner
27fdbf2ac8 Syntax 2017-02-26 22:46:57 +00:00
martinsumner
a9101e4781 SibCount must be non-zero 2017-02-26 22:45:20 +00:00
martinsumner
44471ecc0a ManEntry must be a ManEntry? 
What else could it be?
 2017-02-26 22:42:44 +00:00
martinsumner
aba385869f Test long running clause 2017-02-26 22:41:25 +00:00
martinsumner
4e6c22d7e6 Dumb test 
Can't test for slow not_present outside of simple unit test ... will be
very rare event
 2017-02-26 21:48:04 +00:00
martinsumner
19534122a2 Coverage checks 2017-02-26 21:37:47 +00:00
martinsumner
100d675472 iManifest -test empty journal 2017-02-26 21:13:07 +00:00
martinsumner
41e5686a66 Test coverage - missing 1% 2017-02-26 20:52:40 +00:00
martinsumner
fb54d29873 Timestamps no longer required 2017-02-26 19:09:56 +00:00
martinsumner
fad9bfbff1 Trigger log points, deprecate lower PUT log points 
Only record PUT timings at the Bookie, and trigger them correctly on the
right log point
 2017-02-26 19:07:55 +00:00
martinsumner
8077c70486 More conservative approach to ongoing work monitoring 
As per comments though - if we auto-restart pclerk in the future this
will have to be re-considered.

Perhaps a re-starting pclerk should force some reset of this boolean on
startup perhaps by making a different work_for_clerk if in a virgin
state.
 2017-02-09 23:49:12 +00:00
martinsumner
793977b76c Prevent double work ongoing 
This was the cause of the pclerk crash.  The concurrent work would lead
to the attempt to re-fetch the prompted deletions after they had been
deleted.
 2017-02-09 23:41:28 +00:00
martinsumner
05b2d8faaf is_empty not in OTP16 
bloody OTP16 strikes again - is_empty isn't there
 2017-02-09 21:12:01 +00:00
martinsumner
adb78f5c5a Resolve pclerk crash 
Need to add extra logging to understand why pclerk crashes in some
volume tests.

Penciller's Clerk <0.813.0> shutdown now complete for reason
{badarg,[{dict,fetch,[63,{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}}],[{file,[100,105,99,116,46,101,114,108]},{line,126}]},{leveled_pclerk,handle_cast,2,[{file,[115,114,99,47,108,101,118,101,108,101,100,95,112,99,108,101,114,107,46,101,114,108]},{line,96}]},{gen_server,handle_msg,5,[{file,[103,101,110,95,115,101,114,118,101,114,46,101,114,108]},{line,604}]},{proc_lib,init_p_do_apply,3,[{file,[112,114,111,99,95,108,105,98,46,101,114,108]},{line,239}]}]}

Should only be prompted after prompt deletions had bene updated.
Perhaps a race whereby somehow it is prompted again after it is emptied.
 2017-02-09 14:33:39 +00:00
martinsumner
7320b34681 Comment update 2017-01-25 12:38:33 +00:00
martinsumner
684f1bcd99 Increase bloom size 2017-01-25 01:07:15 +00:00
martinsumner
10a64b86be Remove rogue timestamp 2017-01-24 21:53:43 +00:00
martinsumner
d57b74d967 Re-introduce tinybloom to SST 
This had been removed due to the CPU cost of adding - however then the
tinybloom wa simplemented by directly manipulating bits through binary
comprehension - rather than applying bor band bsl bsr operations.

With these operations the cost of producing and checking the bloom is
<10% by comparison.
 2017-01-24 21:51:12 +00:00
Martin Sumner
f8f2e02d92 Use bel for powers of two 
D’oh.  That’s much much faster of course!
 2017-01-24 18:09:51 +00:00
martinsumner
8c3d0fc493 Alternate implementation 2017-01-24 17:15:39 +00:00
martinsumner
9d95057518 Do it daft way - will it be faster? 2017-01-24 17:00:30 +00:00
martinsumner
93448b76ba Add some extra length checks 2017-01-24 15:49:19 +00:00
martinsumner
a8488663c7 Alternate tinybloom 
Previously the code had used a tiny bloom - but this proved to be
expensive to build.  Looking at the alternative of a slot-size only tiny
bloom
 2017-01-24 15:48:12 +00:00
martinsumner
266e851a96 Merge pull request #18 from martinsumner/mas-leveledtree 
Mas leveledtree
 2017-01-24 02:39:24 +00:00
martinsumner
d225f4d7f5 Add use of leveled_tree to sst summary 2017-01-23 22:58:51 +00:00
martinsumner
94762c79ca Improve test coverage 
Add tests for other scenarios
 2017-01-23 21:43:33 +00:00
martinsumner
861cedf45e Add back missed space in logs 2017-01-23 19:16:17 +00:00
martinsumner
fb896f13b1 Improve logging - add timestamp to logs 2017-01-23 18:56:01 +00:00
martinsumner
90c920fe86 Additional unit test work 
Reverts a previous ct test fix
 2017-01-23 15:15:40 +00:00
martinsumner
5105df1cd6 Add replace capability to manifest 2017-01-23 11:02:54 +00:00
Martin Sumner
2c4c5c9597 Corrections 
Support an empty list of entries being added.  Also specify a tree
correctly in all from_orderedlist scenarios
 2017-01-23 00:22:53 +00:00
martinsumner
cc9494735b Use an index tree for the lower levels of penciller manifest 2017-01-22 23:36:16 +00:00
Martin Sumner
58cda7d157 Switch to using skip lists from leveled_tree 
Remove now unused leveled_skiplist and leveled_tinybloom
 2017-01-21 22:34:56 +00:00
martinsumner
6d2eb1d57c Added skiplist to datatypes 2017-01-21 21:51:35 +00:00
martinsumner
efec232e71 Adjust test size to match SST summary 2017-01-21 16:26:23 +00:00
martinsumner
0fa22ab4b3 Add skiplist into leveled_tree 
Getting ready to remove seperate skiplist module.  Need to add the
skiplist support in leveled_tree for relative testing.
 2017-01-21 16:23:24 +00:00
martinsumner
f031558b28 Make idxt active implementation 
Use idxt for ledger_cache - ready for volume test
 2017-01-21 13:45:19 +00:00
martinsumner
917473dfdc Alternative tree 
As an alternatuve to a tree - try a tree where the list of elements and
the tree to lookup the list of elements are kept in seperate objects
 2017-01-21 13:41:41 +00:00
martinsumner
d6761259e1 Experiment with seperating index from elements 2017-01-21 12:54:32 +00:00
martinsumner
ed96d0ca7a Change so that type of cache is parameterised 
The Tree doesn't seem to be better than the skiplist.  Get ready to make
this switchable
 2017-01-21 11:38:26 +00:00
Martin Sumner
4846272393 Build more efficiently 
Don’t use ++ to build the tree
 2017-01-21 01:48:38 +00:00
Martin Sumner
51f0402292 Change Skip Width 2017-01-20 21:31:17 +00:00
Martin Sumner
9e32e228bb Include gb_trees code 
The gb_trees in OTP16 doesn’t have iterative from support.  Included
here so that we can continue to use it regardless in Riak.
 2017-01-20 17:34:57 +00:00
martinsumner
3d99036093 Switch the LM1 cache to be a tree 
Use a tree of lists not a skiplist
 2017-01-20 16:36:20 +00:00
Martin Sumner
1745ba6863 Improve from_list speed 2017-01-20 16:35:38 +00:00
martinsumner
220d493b5a Tree changes 2017-01-20 16:15:00 +00:00
Martin Sumner
860a8a95e3 Width of 32 may be better 
With 4000 keys - build time dropped significantly
 2017-01-19 22:59:53 +00:00