martinsumner
3330b39b73
Ongoing work on documents
2017-02-01 13:58:09 +00:00
martinsumner
f55753cfc3
First draft
2017-01-31 13:21:52 +00:00
martinsumner
67d372e051
Merge pull request #19 from martinsumner/mas-tinybloom
...
Mas tinybloom
2017-01-25 12:46:11 +00:00
martinsumner
7320b34681
Comment update
2017-01-25 12:38:33 +00:00
martinsumner
684f1bcd99
Increase bloom size
2017-01-25 01:07:15 +00:00
martinsumner
10a64b86be
Remove rogue timestamp
2017-01-24 21:53:43 +00:00
martinsumner
d57b74d967
Re-introduce tinybloom to SST
...
This had been removed due to the CPU cost of adding - however then the
tinybloom wa simplemented by directly manipulating bits through binary
comprehension - rather than applying bor band bsl bsr operations.
With these operations the cost of producing and checking the bloom is
<10% by comparison.
2017-01-24 21:51:12 +00:00
Martin Sumner
f8f2e02d92
Use bel for powers of two
...
D’oh. That’s much much faster of course!
2017-01-24 18:09:51 +00:00
martinsumner
8c3d0fc493
Alternate implementation
2017-01-24 17:15:39 +00:00
martinsumner
9d95057518
Do it daft way - will it be faster?
2017-01-24 17:00:30 +00:00
martinsumner
93448b76ba
Add some extra length checks
2017-01-24 15:49:19 +00:00
martinsumner
a8488663c7
Alternate tinybloom
...
Previously the code had used a tiny bloom - but this proved to be
expensive to build. Looking at the alternative of a slot-size only tiny
bloom
2017-01-24 15:48:12 +00:00
martinsumner
266e851a96
Merge pull request #18 from martinsumner/mas-leveledtree
...
Mas leveledtree
2017-01-24 02:39:24 +00:00
martinsumner
d225f4d7f5
Add use of leveled_tree to sst summary
2017-01-23 22:58:51 +00:00
martinsumner
1102133723
Use skpl as ledger_cache
...
Under assumption that it generates less GC noise (based on
micro-benchmark in leveled_tree eunit testing).
Note to confirm, needed to swap around the test order, and this showed
less collections in each position for skpl - and a < 10% performance hit
2017-01-23 21:45:34 +00:00
martinsumner
94762c79ca
Improve test coverage
...
Add tests for other scenarios
2017-01-23 21:43:33 +00:00
martinsumner
861cedf45e
Add back missed space in logs
2017-01-23 19:16:17 +00:00
martinsumner
fb896f13b1
Improve logging - add timestamp to logs
2017-01-23 18:56:01 +00:00
martinsumner
90c920fe86
Additional unit test work
...
Reverts a previous ct test fix
2017-01-23 15:15:40 +00:00
martinsumner
5105df1cd6
Add replace capability to manifest
2017-01-23 11:02:54 +00:00
Martin Sumner
2c4c5c9597
Corrections
...
Support an empty list of entries being added. Also specify a tree
correctly in all from_orderedlist scenarios
2017-01-23 00:22:53 +00:00
martinsumner
cc9494735b
Use an index tree for the lower levels of penciller manifest
2017-01-22 23:36:16 +00:00
Martin Sumner
58cda7d157
Switch to using skip lists from leveled_tree
...
Remove now unused leveled_skiplist and leveled_tinybloom
2017-01-21 22:34:56 +00:00
martinsumner
6d2eb1d57c
Added skiplist to datatypes
2017-01-21 21:51:35 +00:00
martinsumner
efec232e71
Adjust test size to match SST summary
2017-01-21 16:26:23 +00:00
martinsumner
0fa22ab4b3
Add skiplist into leveled_tree
...
Getting ready to remove seperate skiplist module. Need to add the
skiplist support in leveled_tree for relative testing.
2017-01-21 16:23:24 +00:00
martinsumner
f031558b28
Make idxt active implementation
...
Use idxt for ledger_cache - ready for volume test
2017-01-21 13:45:19 +00:00
martinsumner
917473dfdc
Alternative tree
...
As an alternatuve to a tree - try a tree where the list of elements and
the tree to lookup the list of elements are kept in seperate objects
2017-01-21 13:41:41 +00:00
martinsumner
d6761259e1
Experiment with seperating index from elements
2017-01-21 12:54:32 +00:00
martinsumner
ed96d0ca7a
Change so that type of cache is parameterised
...
The Tree doesn't seem to be better than the skiplist. Get ready to make
this switchable
2017-01-21 11:38:26 +00:00
Martin Sumner
4846272393
Build more efficiently
...
Don’t use ++ to build the tree
2017-01-21 01:48:38 +00:00
Martin Sumner
51f0402292
Change Skip Width
2017-01-20 21:31:17 +00:00
Martin Sumner
9e32e228bb
Include gb_trees code
...
The gb_trees in OTP16 doesn’t have iterative from support. Included
here so that we can continue to use it regardless in Riak.
2017-01-20 17:34:57 +00:00
martinsumner
3d99036093
Switch the LM1 cache to be a tree
...
Use a tree of lists not a skiplist
2017-01-20 16:36:20 +00:00
Martin Sumner
1745ba6863
Improve from_list speed
2017-01-20 16:35:38 +00:00
martinsumner
220d493b5a
Tree changes
2017-01-20 16:15:00 +00:00
martinsumner
30c72c3bd6
Merge pull request #17 from martinsumner/mas-manifesttidy-inker
...
Mas manifesttidy inker
2017-01-19 23:01:27 +00:00
Martin Sumner
860a8a95e3
Width of 32 may be better
...
With 4000 keys - build time dropped significantly
2017-01-19 22:59:53 +00:00
Martin Sumner
12d06efa10
Set log to debug
2017-01-19 22:49:59 +00:00
martinsumner
c722f3132d
Initial version of potential new tree module
...
with basic Unit Tests. Intended to replace skiplist
2017-01-19 22:49:32 +00:00
martinsumner
853f113ee5
Inker Manifest to be a two-level list
...
To eas seraching from front to back, change the inker manifest to be a
two-level list
2017-01-19 12:23:28 +00:00
Martin Sumner
1f406c76dd
Resolve test issues
2017-01-19 09:47:56 +00:00
martinsumner
3ca928629c
Split out inker manifest - load last key from manifest
...
The process of re-opening file included an expensive search for the
LastKey - now the LastKey can be provided out of the manifest.
2017-01-18 15:23:06 +00:00
martinsumner
4e53128a2d
Merge remote-tracking branch 'refs/remotes/origin/master' into mas-manifesttidy-inker
2017-01-18 10:13:21 +00:00
martinsumner
213a2e93fb
Merge pull request #16 from martinsumner/mas-manifesttidy-simple
...
Penciller Manifest - Clean API
2017-01-17 21:14:12 +00:00
martinsumner
f97e02975a
Add comments
2017-01-17 21:09:42 +00:00
martinsumner
59c0c06b60
OTP16 type again
2017-01-17 19:01:59 +00:00
martinsumner
5345242e27
Take out type definition for OTP 16
2017-01-17 19:00:30 +00:00
martinsumner
6298886592
Empty non-persisted parts of record on save
...
Some of the manifest should not be preserved between restarts - so empty
these parts on save
2017-01-17 17:15:04 +00:00
martinsumner
c098eca06e
Put tests back in
2017-01-17 16:55:35 +00:00
