umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
1123 commits 1 branch 0 tags 25 MiB
Commit graph

1123 commits

This branch
This branch
All branches
Author SHA1 Message Date
Martin Sumner
d468dd32a8 More consistency in extract value functions 
Try to make this a little bit cleaner and more obvious.  Make sure the value passed is always a value and not a crc/value.
 2018-05-18 13:29:19 +01:00
Martin Sumner
7cc5512a0a safe_read 
There are now two cases of safe_read_next_key/2

the first case using the original function name will return just the Key when positive (not the Key and KeyBinary).  If the Key and KeyBinary is required (e.g. for CRC checking purposes), then use safe_read_next_keyint/2.
 2018-05-18 11:49:19 +01:00
Martin Sumner
d56ff6efbc Make CRC check cover key and value 
So that a corrupted key is detected through a CRC check without crashing leveled (e.g. because it binary_to_term/1 failes somewhere or it is a tuple of the wrong length).

Previously the CRC covered the value only.

Note if you only wish to extract the value, the key cannot be independently validated.  Perhaps a process extrating key only could hit issues.
 2018-05-18 11:17:04 +01:00
Martin Sumner
501b7806e9
Merge pull request #139 from martinsumner/mas-i138-useblockindexonfetchrange 
Mas i138 useblockindexonfetchrange
 2018-05-17 19:32:31 +01:00
Martin Sumner
eb5d9596b2 Add comments to test 
Also loads some more objects to potentially make results clearer
 2018-05-17 18:20:16 +01:00
Martin Sumner
989f23bca6 Add cache population for non-yielding range fetches 
More likely to require caching at lower levels.
 2018-05-17 16:38:52 +01:00
Martin Sumner
779ccd9c2a Add use of block index when not cached (for fetch range) 2018-05-17 14:56:15 +01:00
Martin Sumner
d8dd952cba
Merge pull request #137 from martinsumner/mas-i11-backenddelete 
Mas i11 backenddelete
 2018-05-16 19:26:05 +01:00
Martin Sumner
18aabb49ba Segment filter and multiple keys in slot 
An issue was spotted.  If we use a segment filter in a query, and there are multiple matches within a given slot - only the first match is returned.

Tests didn't detect this.  Now they do, and the issue is resolved.
 2018-05-16 17:24:23 +01:00
Martin Sumner
cbf6e26fc8 Revert "Accumulate keys in check_blocks" 
This reverts commit 5414e18047.
 2018-05-16 15:11:18 +01:00
Martin Sumner
9cc27469c1 Add test to invoke segment filter issue 
there is an issue if you have a segment filter, and there is more than one match in the slot.
 2018-05-16 15:11:08 +01:00
Martin Sumner
5414e18047 Accumulate keys in check_blocks 
Previously couldn't accumulate keys using check-blocks - so if a key was found in the first position, and there were other positions to check for other keys, those other positions wouldn't be checked.
 2018-05-16 11:38:26 +01:00
Martin Sumner
2cc088fc64 More debug 2018-05-16 11:24:57 +01:00
Martin Sumner
a7cda7213f Debug logs 2018-05-16 11:06:51 +01:00
Martin Sumner
8b4adcccaf Debug logs 2018-05-16 10:51:59 +01:00
Martin Sumner
06fd9856b5 Add debug log 2018-05-16 10:34:47 +01:00
Martin Sumner
3d1c085af3 Revert log additions 2018-05-15 10:41:56 +01:00
Martin Sumner
15a5db5055 Sort tuned seg list 2018-05-15 10:36:07 +01:00
Martin Sumner
6160db3377 Further log 2018-05-15 10:30:09 +01:00
Martin Sumner
1e40648c83 Add initial log 2018-05-15 10:11:37 +01:00
Martin Sumner
83d5a2c7a3
Merge pull request #136 from martinsumner/mas-i134-compileflag 
Mas i134 compileflag
 2018-05-04 18:42:12 +01:00
Martin Sumner
eb5950a179 Merge branch 'master' into mas-i134-compileflag 2018-05-04 17:14:21 +01:00
Martin Sumner
96cee2a64c Add specific compile flag to avoid deprecation warning 
Will see us through OTP 20 and 21.
 2018-05-04 17:14:12 +01:00
Martin Sumner
073fc909ed
Merge pull request #135 from martinsumner/mas-i61-codectidy 
Mas i61 codectidy
 2018-05-04 17:13:20 +01:00
Martin Sumner
6a20b2ce66 Use leveled_codec types 
... and exporting them.

Previously types wer enot exported, and it appears dialyzer treated tham as any() when they were unexported types ??!!??
 2018-05-04 15:24:08 +01:00
Martin Sumner
2063cacd8f More spec/doc work in leveled_codec 
Note that at some stage KeyChanges got overloaded to mean {KeyChanges, TTL}, and the spec now tries to make this a bit clearer
 2018-05-04 11:19:37 +01:00
Martin Sumner
aa34ffda5b Crash not skip on corrupted key 2018-05-03 20:14:36 +01:00
Martin Sumner
2cd20fcb47 Missed generate_uuid reference 2018-05-03 18:26:02 +01:00
Martin Sumner
f88f511df3 leveled_codec spec/doc 
Try and make this code a little bit more organised andd easier to follow
 2018-05-03 17:18:13 +01:00
Martin Sumner
931129d25f
Merge pull request #133 from martinsumner/mas-i61-dialyzertidy 
Mas i61 dialyzertidy
 2018-05-02 16:06:27 +01:00
Martin Sumner
416aa7c0d3 Reference startup writeup 2018-05-02 16:03:03 +01:00
Martin Sumner
1ed37a8ab0 Typos 2018-05-02 15:50:10 +01:00
Martin Sumner
ef9ac57c5e Add information on startup options 2018-05-02 15:48:16 +01:00
Martin Sumner
a9db4a0082 Spec and comment Inker 2018-05-02 10:57:27 +01:00
Martin Sumner
0185b3aa7d Fix Accumulators 
Correct inefficient list accumulators
 2018-05-02 00:23:26 +01:00
Martin Sumner
82a1449447 Fix bad accumulator in test 
Test takes a long time due to sleep (still need to work on that), but also FoldKeysFun uses ++ rather than [{B, K}|Acc] to extend the list.  Order of magnitude speed-up for these queries by changing the way this accumulates
 2018-05-01 22:30:28 +01:00
Martin Sumner
dd7b753688 Add spec and comments to penciller 2018-05-01 21:28:40 +01:00
Martin Sumner
ee5dc43cfd Add doc/spec to bookie 
Improve the comments and dialyzer hints within the bookie.
 2018-05-01 12:03:24 +01:00
Martin Sumner
8123e49c7a
Merge pull request #132 from martinsumner/mas-i126-intermittentfailure 
Mas i126 intermittentfailure
 2018-04-30 20:37:22 +01:00
Martin Sumner
37b067e61b Remove failure detection from eunit test 
As it now consistently passes
 2018-04-30 17:48:34 +01:00
Martin Sumner
941d4d2577 Fix issue with use of ets:next 
ets:next is not StartInclusive - so we shouldn't use ets:next when querying the ledger cache, until it has been confirmed thta a matching key to the StartKey is not present.

This reoslves the intermittently failing unit test which highlighted this (and also makes that intermittent failure a permanent failure by expanding the test cases it covers).
 2018-04-30 15:20:37 +01:00
Martin Sumner
e182f8c95c
Merge pull request #130 from russelldb/rdb/backend_eqc_test 
WIP: changes for backend_eqc test
 2018-04-16 22:24:27 +01:00
Russell Brown
10659bfbd5 Add CT tests for riak $ indexes 
The project has been at 100% coverage, don't ruin it now!
 2018-04-16 17:19:20 +01:00
Russell Brown
92582de4dd WIP: changes for backend_eqc test 
Enable support for $key and $bucket index
 2018-04-12 10:06:29 +01:00
Martin Sumner
12dae9209a
Merge pull request #129 from martinsumner/mas-i128-determinate 
Mas i128 determinate
 2018-04-10 15:18:35 +01:00
Martin Sumner
6e51cbc853 is_alive not needed 
Throwback when this was in terminate.
 2018-04-10 12:22:00 +01:00
Martin Sumner
45ec837813 de-terminate cdb 2018-04-10 10:32:52 +01:00
Martin Sumner
293bd8fb78 de-terminate inker 2018-04-10 10:05:46 +01:00
Martin Sumner
bb752f1025 de-terminate bookie 2018-04-10 09:54:35 +01:00
Martin Sumner
11ba7029aa de-terminate penciller 2018-04-10 09:51:21 +01:00