umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
1406 commits 1 branch 0 tags 25 MiB
Commit graph

1406 commits

This branch
This branch
All branches
Author SHA1 Message Date
Martin Sumner
15a5db5055 Sort tuned seg list 2018-05-15 10:36:07 +01:00
Martin Sumner
6160db3377 Further log 2018-05-15 10:30:09 +01:00
Martin Sumner
1e40648c83 Add initial log 2018-05-15 10:11:37 +01:00
Martin Sumner
83d5a2c7a3
Merge pull request #136 from martinsumner/mas-i134-compileflag 
Mas i134 compileflag
 2018-05-04 18:42:12 +01:00
Martin Sumner
eb5950a179 Merge branch 'master' into mas-i134-compileflag 2018-05-04 17:14:21 +01:00
Martin Sumner
96cee2a64c Add specific compile flag to avoid deprecation warning 
Will see us through OTP 20 and 21.
 2018-05-04 17:14:12 +01:00
Martin Sumner
073fc909ed
Merge pull request #135 from martinsumner/mas-i61-codectidy 
Mas i61 codectidy
 2018-05-04 17:13:20 +01:00
Martin Sumner
6a20b2ce66 Use leveled_codec types 
... and exporting them.

Previously types wer enot exported, and it appears dialyzer treated tham as any() when they were unexported types ??!!??
 2018-05-04 15:24:08 +01:00
Martin Sumner
2063cacd8f More spec/doc work in leveled_codec 
Note that at some stage KeyChanges got overloaded to mean {KeyChanges, TTL}, and the spec now tries to make this a bit clearer
 2018-05-04 11:19:37 +01:00
Martin Sumner
aa34ffda5b Crash not skip on corrupted key 2018-05-03 20:14:36 +01:00
Martin Sumner
2cd20fcb47 Missed generate_uuid reference 2018-05-03 18:26:02 +01:00
Martin Sumner
f88f511df3 leveled_codec spec/doc 
Try and make this code a little bit more organised andd easier to follow
 2018-05-03 17:18:13 +01:00
Martin Sumner
931129d25f
Merge pull request #133 from martinsumner/mas-i61-dialyzertidy 
Mas i61 dialyzertidy
 2018-05-02 16:06:27 +01:00
Martin Sumner
416aa7c0d3 Reference startup writeup 2018-05-02 16:03:03 +01:00
Martin Sumner
1ed37a8ab0 Typos 2018-05-02 15:50:10 +01:00
Martin Sumner
ef9ac57c5e Add information on startup options 2018-05-02 15:48:16 +01:00
Martin Sumner
a9db4a0082 Spec and comment Inker 2018-05-02 10:57:27 +01:00
Martin Sumner
0185b3aa7d Fix Accumulators 
Correct inefficient list accumulators
 2018-05-02 00:23:26 +01:00
Martin Sumner
82a1449447 Fix bad accumulator in test 
Test takes a long time due to sleep (still need to work on that), but also FoldKeysFun uses ++ rather than [{B, K}|Acc] to extend the list.  Order of magnitude speed-up for these queries by changing the way this accumulates
 2018-05-01 22:30:28 +01:00
Martin Sumner
dd7b753688 Add spec and comments to penciller 2018-05-01 21:28:40 +01:00
Martin Sumner
ee5dc43cfd Add doc/spec to bookie 
Improve the comments and dialyzer hints within the bookie.
 2018-05-01 12:03:24 +01:00
Martin Sumner
8123e49c7a
Merge pull request #132 from martinsumner/mas-i126-intermittentfailure 
Mas i126 intermittentfailure
 2018-04-30 20:37:22 +01:00
Martin Sumner
37b067e61b Remove failure detection from eunit test 
As it now consistently passes
 2018-04-30 17:48:34 +01:00
Martin Sumner
941d4d2577 Fix issue with use of ets:next 
ets:next is not StartInclusive - so we shouldn't use ets:next when querying the ledger cache, until it has been confirmed thta a matching key to the StartKey is not present.

This reoslves the intermittently failing unit test which highlighted this (and also makes that intermittent failure a permanent failure by expanding the test cases it covers).
 2018-04-30 15:20:37 +01:00
Martin Sumner
e182f8c95c
Merge pull request #130 from russelldb/rdb/backend_eqc_test 
WIP: changes for backend_eqc test
 2018-04-16 22:24:27 +01:00
Russell Brown
10659bfbd5 Add CT tests for riak $ indexes 
The project has been at 100% coverage, don't ruin it now!
 2018-04-16 17:19:20 +01:00
Russell Brown
92582de4dd WIP: changes for backend_eqc test 
Enable support for $key and $bucket index
 2018-04-12 10:06:29 +01:00
Martin Sumner
12dae9209a
Merge pull request #129 from martinsumner/mas-i128-determinate 
Mas i128 determinate
 2018-04-10 15:18:35 +01:00
Martin Sumner
6e51cbc853 is_alive not needed 
Throwback when this was in terminate.
 2018-04-10 12:22:00 +01:00
Martin Sumner
45ec837813 de-terminate cdb 2018-04-10 10:32:52 +01:00
Martin Sumner
293bd8fb78 de-terminate inker 2018-04-10 10:05:46 +01:00
Martin Sumner
bb752f1025 de-terminate bookie 2018-04-10 09:54:35 +01:00
Martin Sumner
11ba7029aa de-terminate penciller 2018-04-10 09:51:21 +01:00
Martin Sumner
c6769c6659
Merge pull request #127 from martinsumner/mas-i115-simplettcache 
Mas i115 simplettcache
 2018-04-05 09:15:51 +01:00
Martin Sumner
54d81220b2 Merge branch 'master' into mas-i115-simplettcache 2018-04-05 09:14:02 +01:00
Martin Sumner
766cee50d7 Expand test for join_segment 2018-04-04 22:37:01 +01:00
Martin Sumner
c8552d5bd5 Add exports required by kn_index_tictactree 2018-03-28 14:01:40 +01:00
Martin Sumner
e0d63209a1 Export merge binaries 
For use by processes wanting to merge roots or branches
 2018-03-26 09:39:34 +01:00
Martin Sumner
c1cd00b498 Allow ignore non-binary subkey for hash 
This allows the subkey to be an integer, that will be gnored for hashing purposes
 2018-03-22 22:07:24 +00:00
Martin Sumner
b81caf7dee segment_hash -> tictac 
the concept of the segment hash belongs to the leveled_tictac module, not the codec.

Previously the alignment of tictac and store was accidental, this change makes it explicit.
 2018-03-22 19:03:52 +00:00
Martin Sumner
6ce903ad2b Change segment_hahs for HEAD 
Needs toaling with ?RIAK tag, and so that AAE key-ordered and segment-ordered stores also agree on definition of Segment ID
 2018-03-22 17:12:58 +00:00
Martin Sumner
cda412508a IsEmpty check 
Previously there was no is_empty check, and there was a workaround using binary_bucketlist.  But what if there were many buckets - this is a slow seek (using get next key over and over).

Instead have a proper is_empty check.
 2018-03-21 15:31:00 +00:00
Martin Sumner
ef22aabe85 Alter comment 2018-03-20 11:11:41 +00:00
Martin Sumner
096f49bb1d Add missing comment 2018-03-19 11:40:49 +00:00
Martin Sumner
6ed40ad959 Add some further details - IndexNs 
Following discussions with Russell, try and explain the as-is status of hashtrees a bit more clearly
 2018-03-16 14:45:17 +00:00
Martin Sumner
669a859d75
Merge pull request #123 from martinsumner/bug/rdb/gh122 
Fix failing test, I don't understand how
 2018-03-15 11:16:28 +00:00
Russell Brown
d487c1bc51 Fix failing test, I don't understand how 
See #122 for details
 2018-03-15 11:09:28 +00:00
Martin Sumner
25be7ef535 Change TicTac tree sizing 
Previously TicTac trees were of equal width at both levels.  This seemed reasonably efficient way of sclaing up key sizes (e.g. at each increment both level widths doubled giving a 4 x increase in size).

However if we're to do comparisons with fetch_root followed by fetch_branches this might not be as efficient.  As with a 1024 x 1024 tree a 1000 deltas requires the whole tree to eb exchanged, where as with a 4096 * 256 tree a 1000 deltas only requires a quarter of the tree to be sent over the network.

The whole tree, with a 4 byte hash size is 4MB in this case - requiring 30ms of network time on a Gbps link. There may be longer blocking of inter-node communication on LFNs (as per the tests in Meiklejohn's Partisan paper - https://arxiv.org/abs/1802.02652).

This  changes makes TicTac tree tests run slower.  But perhaps this is a price worth paying for smoothing out the potential impact on inter-node communication?
 2018-03-14 10:36:17 +00:00
Martin Sumner
a9a20c9150 Smoother temrinate on destroy 
don't try and terminate a dead process
 2018-03-07 16:20:41 +00:00
Martin Sumner
e29743310d Make destroy "normal" 
Put the special actions in the handle_call not the terminate
 2018-03-07 16:14:50 +00:00