umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
463 commits 1 branch 0 tags 25 MiB
Commit graph

463 commits

This branch
This branch
All branches
Author SHA1 Message Date
Martin Sumner
58cda7d157 Switch to using skip lists from leveled_tree 
Remove now unused leveled_skiplist and leveled_tinybloom
 2017-01-21 22:34:56 +00:00
martinsumner
6d2eb1d57c Added skiplist to datatypes 2017-01-21 21:51:35 +00:00
martinsumner
efec232e71 Adjust test size to match SST summary 2017-01-21 16:26:23 +00:00
martinsumner
0fa22ab4b3 Add skiplist into leveled_tree 
Getting ready to remove seperate skiplist module.  Need to add the
skiplist support in leveled_tree for relative testing.
 2017-01-21 16:23:24 +00:00
martinsumner
f031558b28 Make idxt active implementation 
Use idxt for ledger_cache - ready for volume test
 2017-01-21 13:45:19 +00:00
martinsumner
917473dfdc Alternative tree 
As an alternatuve to a tree - try a tree where the list of elements and
the tree to lookup the list of elements are kept in seperate objects
 2017-01-21 13:41:41 +00:00
martinsumner
d6761259e1 Experiment with seperating index from elements 2017-01-21 12:54:32 +00:00
martinsumner
ed96d0ca7a Change so that type of cache is parameterised 
The Tree doesn't seem to be better than the skiplist.  Get ready to make
this switchable
 2017-01-21 11:38:26 +00:00
Martin Sumner
4846272393 Build more efficiently 
Don’t use ++ to build the tree
 2017-01-21 01:48:38 +00:00
Martin Sumner
51f0402292 Change Skip Width 2017-01-20 21:31:17 +00:00
Martin Sumner
9e32e228bb Include gb_trees code 
The gb_trees in OTP16 doesn’t have iterative from support.  Included
here so that we can continue to use it regardless in Riak.
 2017-01-20 17:34:57 +00:00
martinsumner
3d99036093 Switch the LM1 cache to be a tree 
Use a tree of lists not a skiplist
 2017-01-20 16:36:20 +00:00
Martin Sumner
1745ba6863 Improve from_list speed 2017-01-20 16:35:38 +00:00
martinsumner
220d493b5a Tree changes 2017-01-20 16:15:00 +00:00
Martin Sumner
860a8a95e3 Width of 32 may be better 
With 4000 keys - build time dropped significantly
 2017-01-19 22:59:53 +00:00
martinsumner
c722f3132d Initial version of potential new tree module 
with basic Unit Tests.  Intended to replace skiplist
 2017-01-19 22:49:32 +00:00
martinsumner
213a2e93fb Merge pull request #16 from martinsumner/mas-manifesttidy-simple 
Penciller Manifest - Clean API
 2017-01-17 21:14:12 +00:00
martinsumner
f97e02975a Add comments 2017-01-17 21:09:42 +00:00
martinsumner
59c0c06b60 OTP16 type again 2017-01-17 19:01:59 +00:00
martinsumner
5345242e27 Take out type definition for OTP 16 2017-01-17 19:00:30 +00:00
martinsumner
6298886592 Empty non-persisted parts of record on save 
Some of the manifest should not be preserved between restarts - so empty
these parts on save
 2017-01-17 17:15:04 +00:00
martinsumner
c098eca06e Put tests back in 2017-01-17 16:55:35 +00:00
martinsumner
850a524ad3 Close Pending Deletes on shutdown 
Pending deletes are not in the Manifest, and so were not being closed on
shutdown
 2017-01-17 16:54:54 +00:00
martinsumner
7086717765 Rename to pmanifest 
There may be a seperate imanifest in the future
 2017-01-17 14:11:50 +00:00
martinsumner
2e734ff501 Corrected snapshot releases 2017-01-17 14:02:21 +00:00
martinsumner
c99c50ce6e Fix-up message exchange on confirm delete 2017-01-17 11:18:58 +00:00
martinsumner
14ebf94e56 Correct botched manifest entry removal 2017-01-17 10:53:13 +00:00
martinsumner
5076187545 Reduce log level of troubleshooting log 2017-01-17 10:39:31 +00:00
martinsumner
ec08d1ab97 Must remove before we insert - cannot safely if overlapping with insertions 2017-01-17 10:37:46 +00:00
martinsumner
5adfb5c5ef Extra logging for troubleshooting 2017-01-17 10:32:15 +00:00
martinsumner
59dc451e70 Address confusion over type in DeletionsList 2017-01-17 10:21:29 +00:00
martinsumner
c32fd3fb4c Change to use manifest_entry not straight PID in unit test 2017-01-17 10:14:40 +00:00
martinsumner
9832ecc369 Manifest now back to a simple list 
This has refactored code with the implementation of the manifest
isolated in to a seperate module, and the pure async relationship
between penciller and their clerk.  However, the manifest is just a
simple list at each level.
 2017-01-17 10:12:15 +00:00
martinsumner
72d16af2b1 Switch out implementation of manifest 
This gives a new manifest implementation that is an array of lists.

Just basic unit testing of lookup insertion and removal in this module.
The API was changed subtly, and so nothing broader will work at this
stage
 2017-01-16 23:27:42 +00:00
martinsumner
38c7c9be9b Attempt to fix manifets issues 
ETS manifest is making eveyrthing SLOW.  Marked basement.  Perhaps fixed
some GC issues.
 2017-01-15 12:41:16 +00:00
martinsumner
d16defa597 Remove non-compatible OTP16 option 2017-01-15 11:29:19 +00:00
martinsumner
e6c4c9eff8 Log deletions from mnaifest (via GC) 2017-01-15 11:04:26 +00:00
martinsumner
13c5cc8899 Add timing points 
Add some timing points to manifest updates
 2017-01-15 10:36:50 +00:00
martinsumner
8e92a2c563 Fix pclerk unit test 2017-01-15 01:47:23 +00:00
Martin Sumner
f868be8086 Change Penciller to Clerk handoff 
correct the previous change to make sure that deletes are not confirmed
when work is outstanding, but also make sure that the clerk only ever
casts the Penciller so no deadlocks can ensue.
 2017-01-15 00:52:43 +00:00
martinsumner
9577d24be0 Wait on close of penciller clerk 
The clerk never calls the Penciller, so cannot deadlock.  Will wait for
a merge to be complete
 2017-01-14 22:59:04 +00:00
martinsumner
0c01a90f96 Manifest SQN at startup 
An L0 file outside the Manifest will be written at a Manifest SQN one
higher - so need to look at stratup for one at one higher
 2017-01-14 22:31:45 +00:00
martinsumner
7b0b3e9b83 Add logging of Manifest SQN at startup 2017-01-14 22:26:26 +00:00
martinsumner
ea171cb13b Attempt to stop race - between level zero and ongoing merge 2017-01-14 22:03:57 +00:00
martinsumner
dcf3afc056 Log basement setting when creating files 2017-01-14 21:19:51 +00:00
martinsumner
8f0a096dd2 More work on timeout 2017-01-14 20:57:43 +00:00
martinsumner
86af99e511 Set timeout correctly 2017-01-14 20:19:08 +00:00
martinsumner
85c03a61f9 Log changes 2017-01-14 20:11:01 +00:00
martinsumner
13c81f0ed1 Basic working 
Some basic tests working - but still outstanding issues.
 2017-01-14 19:41:09 +00:00
martinsumner
76bdd83346 Manifest refactor - STILL BROKEN 
Some working tests now, but sitll broken
 2017-01-14 16:36:05 +00:00