umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
828 commits 1 branch 0 tags 25 MiB
Commit graph

828 commits

This branch
This branch
All branches
Author SHA1 Message Date
martinsumner
1102133723 Use skpl as ledger_cache 
Under assumption that it generates less GC noise (based on
micro-benchmark in leveled_tree eunit testing).

Note to confirm, needed to swap around the test order, and this showed
less collections in each position for skpl - and a < 10% performance hit
 2017-01-23 21:45:34 +00:00
martinsumner
94762c79ca Improve test coverage 
Add tests for other scenarios
 2017-01-23 21:43:33 +00:00
martinsumner
861cedf45e Add back missed space in logs 2017-01-23 19:16:17 +00:00
martinsumner
fb896f13b1 Improve logging - add timestamp to logs 2017-01-23 18:56:01 +00:00
martinsumner
90c920fe86 Additional unit test work 
Reverts a previous ct test fix
 2017-01-23 15:15:40 +00:00
martinsumner
5105df1cd6 Add replace capability to manifest 2017-01-23 11:02:54 +00:00
Martin Sumner
2c4c5c9597 Corrections 
Support an empty list of entries being added.  Also specify a tree
correctly in all from_orderedlist scenarios
 2017-01-23 00:22:53 +00:00
martinsumner
cc9494735b Use an index tree for the lower levels of penciller manifest 2017-01-22 23:36:16 +00:00
Martin Sumner
58cda7d157 Switch to using skip lists from leveled_tree 
Remove now unused leveled_skiplist and leveled_tinybloom
 2017-01-21 22:34:56 +00:00
martinsumner
6d2eb1d57c Added skiplist to datatypes 2017-01-21 21:51:35 +00:00
martinsumner
efec232e71 Adjust test size to match SST summary 2017-01-21 16:26:23 +00:00
martinsumner
0fa22ab4b3 Add skiplist into leveled_tree 
Getting ready to remove seperate skiplist module.  Need to add the
skiplist support in leveled_tree for relative testing.
 2017-01-21 16:23:24 +00:00
martinsumner
f031558b28 Make idxt active implementation 
Use idxt for ledger_cache - ready for volume test
 2017-01-21 13:45:19 +00:00
martinsumner
917473dfdc Alternative tree 
As an alternatuve to a tree - try a tree where the list of elements and
the tree to lookup the list of elements are kept in seperate objects
 2017-01-21 13:41:41 +00:00
martinsumner
d6761259e1 Experiment with seperating index from elements 2017-01-21 12:54:32 +00:00
martinsumner
ed96d0ca7a Change so that type of cache is parameterised 
The Tree doesn't seem to be better than the skiplist.  Get ready to make
this switchable
 2017-01-21 11:38:26 +00:00
Martin Sumner
4846272393 Build more efficiently 
Don’t use ++ to build the tree
 2017-01-21 01:48:38 +00:00
Martin Sumner
51f0402292 Change Skip Width 2017-01-20 21:31:17 +00:00
Martin Sumner
9e32e228bb Include gb_trees code 
The gb_trees in OTP16 doesn’t have iterative from support.  Included
here so that we can continue to use it regardless in Riak.
 2017-01-20 17:34:57 +00:00
martinsumner
3d99036093 Switch the LM1 cache to be a tree 
Use a tree of lists not a skiplist
 2017-01-20 16:36:20 +00:00
Martin Sumner
1745ba6863 Improve from_list speed 2017-01-20 16:35:38 +00:00
martinsumner
220d493b5a Tree changes 2017-01-20 16:15:00 +00:00
martinsumner
30c72c3bd6 Merge pull request #17 from martinsumner/mas-manifesttidy-inker 
Mas manifesttidy inker
 2017-01-19 23:01:27 +00:00
Martin Sumner
860a8a95e3 Width of 32 may be better 
With 4000 keys - build time dropped significantly
 2017-01-19 22:59:53 +00:00
Martin Sumner
12d06efa10 Set log to debug 2017-01-19 22:49:59 +00:00
martinsumner
c722f3132d Initial version of potential new tree module 
with basic Unit Tests.  Intended to replace skiplist
 2017-01-19 22:49:32 +00:00
martinsumner
853f113ee5 Inker Manifest to be a two-level list 
To eas seraching from front to back, change the inker manifest to be a
two-level list
 2017-01-19 12:23:28 +00:00
Martin Sumner
1f406c76dd Resolve test issues 2017-01-19 09:47:56 +00:00
martinsumner
3ca928629c Split out inker manifest - load last key from manifest 
The process of re-opening file included an expensive search for the
LastKey - now the LastKey can be provided out of the manifest.
 2017-01-18 15:23:06 +00:00
martinsumner
4e53128a2d Merge remote-tracking branch 'refs/remotes/origin/master' into mas-manifesttidy-inker 2017-01-18 10:13:21 +00:00
martinsumner
213a2e93fb Merge pull request #16 from martinsumner/mas-manifesttidy-simple 
Penciller Manifest - Clean API
 2017-01-17 21:14:12 +00:00
martinsumner
f97e02975a Add comments 2017-01-17 21:09:42 +00:00
martinsumner
59c0c06b60 OTP16 type again 2017-01-17 19:01:59 +00:00
martinsumner
5345242e27 Take out type definition for OTP 16 2017-01-17 19:00:30 +00:00
martinsumner
6298886592 Empty non-persisted parts of record on save 
Some of the manifest should not be preserved between restarts - so empty
these parts on save
 2017-01-17 17:15:04 +00:00
martinsumner
c098eca06e Put tests back in 2017-01-17 16:55:35 +00:00
martinsumner
850a524ad3 Close Pending Deletes on shutdown 
Pending deletes are not in the Manifest, and so were not being closed on
shutdown
 2017-01-17 16:54:54 +00:00
martinsumner
3712c62a50 Broken WIP 2017-01-17 16:30:04 +00:00
martinsumner
7086717765 Rename to pmanifest 
There may be a seperate imanifest in the future
 2017-01-17 14:11:50 +00:00
martinsumner
2e734ff501 Corrected snapshot releases 2017-01-17 14:02:21 +00:00
martinsumner
c99c50ce6e Fix-up message exchange on confirm delete 2017-01-17 11:18:58 +00:00
martinsumner
14ebf94e56 Correct botched manifest entry removal 2017-01-17 10:53:13 +00:00
martinsumner
5076187545 Reduce log level of troubleshooting log 2017-01-17 10:39:31 +00:00
martinsumner
ec08d1ab97 Must remove before we insert - cannot safely if overlapping with insertions 2017-01-17 10:37:46 +00:00
martinsumner
5adfb5c5ef Extra logging for troubleshooting 2017-01-17 10:32:15 +00:00
martinsumner
59dc451e70 Address confusion over type in DeletionsList 2017-01-17 10:21:29 +00:00
martinsumner
c32fd3fb4c Change to use manifest_entry not straight PID in unit test 2017-01-17 10:14:40 +00:00
martinsumner
9832ecc369 Manifest now back to a simple list 
This has refactored code with the implementation of the manifest
isolated in to a seperate module, and the pure async relationship
between penciller and their clerk.  However, the manifest is just a
simple list at each level.
 2017-01-17 10:12:15 +00:00
martinsumner
72d16af2b1 Switch out implementation of manifest 
This gives a new manifest implementation that is an array of lists.

Just basic unit testing of lookup insertion and removal in this module.
The API was changed subtly, and so nothing broader will work at this
stage
 2017-01-16 23:27:42 +00:00
martinsumner
38c7c9be9b Attempt to fix manifets issues 
ETS manifest is making eveyrthing SLOW.  Marked basement.  Perhaps fixed
some GC issues.
 2017-01-15 12:41:16 +00:00