umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
493 commits 1 branch 0 tags 25 MiB
Commit graph

493 commits

This branch
This branch
All branches
Author SHA1 Message Date
martinsumner
cbe663ebb6 Clean-up readme 2017-02-02 11:16:21 +00:00
martinsumner
3330b39b73 Ongoing work on documents 2017-02-01 13:58:09 +00:00
martinsumner
f55753cfc3 First draft 2017-01-31 13:21:52 +00:00
martinsumner
67d372e051 Merge pull request #19 from martinsumner/mas-tinybloom 
Mas tinybloom
 2017-01-25 12:46:11 +00:00
martinsumner
7320b34681 Comment update 2017-01-25 12:38:33 +00:00
martinsumner
684f1bcd99 Increase bloom size 2017-01-25 01:07:15 +00:00
martinsumner
10a64b86be Remove rogue timestamp 2017-01-24 21:53:43 +00:00
martinsumner
d57b74d967 Re-introduce tinybloom to SST 
This had been removed due to the CPU cost of adding - however then the
tinybloom wa simplemented by directly manipulating bits through binary
comprehension - rather than applying bor band bsl bsr operations.

With these operations the cost of producing and checking the bloom is
<10% by comparison.
 2017-01-24 21:51:12 +00:00
Martin Sumner
f8f2e02d92 Use bel for powers of two 
D’oh.  That’s much much faster of course!
 2017-01-24 18:09:51 +00:00
martinsumner
8c3d0fc493 Alternate implementation 2017-01-24 17:15:39 +00:00
martinsumner
9d95057518 Do it daft way - will it be faster? 2017-01-24 17:00:30 +00:00
martinsumner
93448b76ba Add some extra length checks 2017-01-24 15:49:19 +00:00
martinsumner
a8488663c7 Alternate tinybloom 
Previously the code had used a tiny bloom - but this proved to be
expensive to build.  Looking at the alternative of a slot-size only tiny
bloom
 2017-01-24 15:48:12 +00:00
martinsumner
266e851a96 Merge pull request #18 from martinsumner/mas-leveledtree 
Mas leveledtree
 2017-01-24 02:39:24 +00:00
martinsumner
d225f4d7f5 Add use of leveled_tree to sst summary 2017-01-23 22:58:51 +00:00
martinsumner
1102133723 Use skpl as ledger_cache 
Under assumption that it generates less GC noise (based on
micro-benchmark in leveled_tree eunit testing).

Note to confirm, needed to swap around the test order, and this showed
less collections in each position for skpl - and a < 10% performance hit
 2017-01-23 21:45:34 +00:00
martinsumner
94762c79ca Improve test coverage 
Add tests for other scenarios
 2017-01-23 21:43:33 +00:00
martinsumner
861cedf45e Add back missed space in logs 2017-01-23 19:16:17 +00:00
martinsumner
fb896f13b1 Improve logging - add timestamp to logs 2017-01-23 18:56:01 +00:00
martinsumner
90c920fe86 Additional unit test work 
Reverts a previous ct test fix
 2017-01-23 15:15:40 +00:00
martinsumner
5105df1cd6 Add replace capability to manifest 2017-01-23 11:02:54 +00:00
Martin Sumner
2c4c5c9597 Corrections 
Support an empty list of entries being added.  Also specify a tree
correctly in all from_orderedlist scenarios
 2017-01-23 00:22:53 +00:00
martinsumner
cc9494735b Use an index tree for the lower levels of penciller manifest 2017-01-22 23:36:16 +00:00
Martin Sumner
58cda7d157 Switch to using skip lists from leveled_tree 
Remove now unused leveled_skiplist and leveled_tinybloom
 2017-01-21 22:34:56 +00:00
martinsumner
6d2eb1d57c Added skiplist to datatypes 2017-01-21 21:51:35 +00:00
martinsumner
efec232e71 Adjust test size to match SST summary 2017-01-21 16:26:23 +00:00
martinsumner
0fa22ab4b3 Add skiplist into leveled_tree 
Getting ready to remove seperate skiplist module.  Need to add the
skiplist support in leveled_tree for relative testing.
 2017-01-21 16:23:24 +00:00
martinsumner
f031558b28 Make idxt active implementation 
Use idxt for ledger_cache - ready for volume test
 2017-01-21 13:45:19 +00:00
martinsumner
917473dfdc Alternative tree 
As an alternatuve to a tree - try a tree where the list of elements and
the tree to lookup the list of elements are kept in seperate objects
 2017-01-21 13:41:41 +00:00
martinsumner
d6761259e1 Experiment with seperating index from elements 2017-01-21 12:54:32 +00:00
martinsumner
ed96d0ca7a Change so that type of cache is parameterised 
The Tree doesn't seem to be better than the skiplist.  Get ready to make
this switchable
 2017-01-21 11:38:26 +00:00
Martin Sumner
4846272393 Build more efficiently 
Don’t use ++ to build the tree
 2017-01-21 01:48:38 +00:00
Martin Sumner
51f0402292 Change Skip Width 2017-01-20 21:31:17 +00:00
Martin Sumner
9e32e228bb Include gb_trees code 
The gb_trees in OTP16 doesn’t have iterative from support.  Included
here so that we can continue to use it regardless in Riak.
 2017-01-20 17:34:57 +00:00
martinsumner
3d99036093 Switch the LM1 cache to be a tree 
Use a tree of lists not a skiplist
 2017-01-20 16:36:20 +00:00
Martin Sumner
1745ba6863 Improve from_list speed 2017-01-20 16:35:38 +00:00
martinsumner
220d493b5a Tree changes 2017-01-20 16:15:00 +00:00
martinsumner
30c72c3bd6 Merge pull request #17 from martinsumner/mas-manifesttidy-inker 
Mas manifesttidy inker
 2017-01-19 23:01:27 +00:00
Martin Sumner
860a8a95e3 Width of 32 may be better 
With 4000 keys - build time dropped significantly
 2017-01-19 22:59:53 +00:00
Martin Sumner
12d06efa10 Set log to debug 2017-01-19 22:49:59 +00:00
martinsumner
c722f3132d Initial version of potential new tree module 
with basic Unit Tests.  Intended to replace skiplist
 2017-01-19 22:49:32 +00:00
martinsumner
853f113ee5 Inker Manifest to be a two-level list 
To eas seraching from front to back, change the inker manifest to be a
two-level list
 2017-01-19 12:23:28 +00:00
Martin Sumner
1f406c76dd Resolve test issues 2017-01-19 09:47:56 +00:00
martinsumner
3ca928629c Split out inker manifest - load last key from manifest 
The process of re-opening file included an expensive search for the
LastKey - now the LastKey can be provided out of the manifest.
 2017-01-18 15:23:06 +00:00
martinsumner
4e53128a2d Merge remote-tracking branch 'refs/remotes/origin/master' into mas-manifesttidy-inker 2017-01-18 10:13:21 +00:00
martinsumner
213a2e93fb Merge pull request #16 from martinsumner/mas-manifesttidy-simple 
Penciller Manifest - Clean API
 2017-01-17 21:14:12 +00:00
martinsumner
f97e02975a Add comments 2017-01-17 21:09:42 +00:00
martinsumner
59c0c06b60 OTP16 type again 2017-01-17 19:01:59 +00:00
martinsumner
5345242e27 Take out type definition for OTP 16 2017-01-17 19:00:30 +00:00
martinsumner
6298886592 Empty non-persisted parts of record on save 
Some of the manifest should not be preserved between restarts - so empty
these parts on save
 2017-01-17 17:15:04 +00:00