umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
1461 commits 1 branch 0 tags 25 MiB
Commit graph

1461 commits

This branch
This branch
All branches
Author SHA1 Message Date
Martin Sumner
f4c65597ea log 2 compat 
Need to support math:log2 in early versions of OTP (for Riak)
 2018-10-24 15:42:21 +01:00
Martin Sumner
6d62803567 Test at higher volume 
Increase probability of hit
 2018-10-24 12:06:42 +01:00
Martin Sumner
d171fda705 Format comment 2018-10-24 11:58:25 +01:00
Martin Sumner
d5d54cc13a Add timing test to guide selection of approach 2018-10-24 11:50:36 +01:00
Martin Sumner
4981cdfc07 Expand segment lists when matches are produced form smaller trees 
Let us say a store has precalculated segments based on large tree size, but mismatched segmentIDs are found on a small tree size (for query efficiency).

the list of segment IDs need to be expanded out for matching.  This may be more efficient that running match_segment for each check (expand out once and uses lists:member/2).  Also this is necessary for leveled_so AAE stores (segment-ordered stores)
 2018-10-24 10:19:54 +01:00
Martin Sumner
5cbda7c2f7
Merge pull request #197 from martinsumner/mas-i196-segfilter 
Add segment matching function
 2018-10-24 07:34:14 +01:00
Martin Sumner
dd9a6f0015 Add segment matching function 
For when matching between different tree sizes
 2018-10-23 18:46:10 +01:00
Martin Sumner
5f5255142d
Merge pull request #195 from martinsumner/mas-i194-startupsqn 
Mas i194 startupsqn
 2018-10-03 21:05:57 +01:00
Martin Sumner
0bbaf1f25e Handle missing journal files without data loss 
Make sure data added after the journal has been lost will not be lost (even though lost data may stay lost)
 2018-10-03 18:38:56 +01:00
Martin Sumner
9a0082db4e Test to expose startup issue 
See #194
 2018-10-03 18:29:20 +01:00
Martin Sumner
cc676190c4
Merge pull request #193 from martinsumner/mas-192-journalcheck4head 
Mas 192 journalcheck4head
 2018-10-01 11:59:41 +01:00
Martin Sumner
b7c8304d99 Never check inker if head_only 2018-09-28 16:58:28 +01:00
Martin Sumner
b1ae7837a1 Add unit test of frequency 
but a bit hard to test due to randomness
 2018-09-28 16:16:19 +01:00
Martin Sumner
575397229e Infrequent check of Journal for key presence 
with decreasing frequency if there are repeated successes when checking
 2018-09-28 15:46:43 +01:00
Martin Sumner
8be873f776
Merge pull request #191 from martinsumner/mas-i189-tuplebuckets 
Mas i189 tuplebuckets
 2018-09-27 18:14:30 +01:00
Martin Sumner
f95f078929 Merge branch 'master' into mas-i189-tuplebuckets 2018-09-27 16:53:18 +01:00
Martin Sumner
ee172468ce
Merge pull request #190 from martinsumner/mas-qi18-saferead 
Mas qi18 saferead
 2018-09-27 16:52:43 +01:00
Martin Sumner
1fc2c7ff2b Switch log to warning 
As the database will continue to function
 2018-09-27 15:54:16 +01:00
Martin Sumner
88e4cf2361 Add unit test with bad write 
If a partial write is made at the beginning,  the file was not being truncated.  Consequently writes would be accepted after it opens, and the those writes would be after the corruption so would never be read in the future.
 2018-09-27 15:26:45 +01:00
Martin Sumner
b15fef8cf6 Add unit test of corrupted keys 
To be detected in direct fetch
 2018-09-27 14:20:13 +01:00
Martin Sumner
39e366a31b Add ct test 
Which passes
 2018-09-27 11:32:42 +01:00
Martin Sumner
0fb35e658f Add support for buckets that are tuples 
Only {binary(), binary()} tuples
 2018-09-27 09:34:40 +01:00
Martin Sumner
cbd7713c71
Merge pull request #188 from martinsumner/mas-i186-iclerkscore 
Mas i186 iclerkscore
 2018-09-26 17:36:43 +01:00
Martin Sumner
36537b4ae2 May need to wait longer for completion 
As esting more keys
 2018-09-26 15:38:43 +01:00
Martin Sumner
b2fce95b14 Add explainer to NaiveBestRun 2018-09-26 14:11:02 +01:00
Martin Sumner
7885f93ba6 Boost docs 
Try and make this easier to follow with docs and specs
 2018-09-26 14:07:44 +01:00
Martin Sumner
979c65f0af Merge branch 'mas-i185-docupdate' into mas-i186-iclerkscore 2018-09-26 13:33:07 +01:00
Martin Sumner
d9229c9782 Change score assessment 
Wans't actually calculating scores correctly previously.
 2018-09-26 12:56:28 +01:00
Martin Sumner
3df481941f Record time to calculate compation run 
As this is a factor in assessing suitability of Journal size
 2018-09-26 10:19:24 +01:00
Martin Sumner
8d8a0a3f5b Improve explanation of max_journal_size 
Tighten guidance with regards to very small values, which aren't expected to make sense
 2018-09-26 09:37:50 +01:00
Martin Sumner
908f762a48
Merge pull request #186 from martinsumner/mas-i185-emptyjournal 
Mas i185 emptyjournal
 2018-09-25 19:54:57 +01:00
Martin Sumner
0772317247 Test mistake 
If random integer was low, total could be below threshold - so calculate total correctly.

Should make value re-generate random uniform, but test is still valid without this
 2018-09-25 18:32:48 +01:00
Martin Sumner
051fcd672c Add test of switching from a big journal to a little journal 
And sending objects over the journal size
 2018-09-25 16:56:24 +01:00
Martin Sumner
ab74d5c0dd
Merge pull request #184 from martinsumner/mas-i183-thowinasyncfold 
Mas i183 thowinasyncfold
 2018-09-24 22:41:12 +01:00
Martin Sumner
f4b365438c Further comments in API docs 2018-09-24 20:43:21 +01:00
Martin Sumner
bed155761b Added comments 
This is still a clumsy feature, in terms of implementation.

Is the fact that some folds handle a throw, and some don't an issue?
 2018-09-24 20:05:48 +01:00
Martin Sumner
a9b097e392 Add a wrapper to fold_keys queries 
Queries that in Riak will be based on fold_keys need to be able to catch throws, and re-throw them to be detected by the worker (whilst still clearing up the snapshot)
 2018-09-24 19:54:28 +01:00
Martin Sumner
bcda272fb9
Merge pull request #182 from martinsumner/mas-i180-dollarbucketregex 
Mas i180 dollarbucketregex
 2018-09-24 16:37:25 +01:00
Martin Sumner
8f320a054e Add object deletion to test of dolerkey index 
Related to tracking a failure in riak_test
 2018-09-21 17:08:59 +01:00
Martin Sumner
37f784f041 Add regex tests for dollar bucket queries 
Proce this works as well as for dollar key.  This isn't tested in riak_test currently, but seems illogical not to support it her eif it is supported on $key queries.
 2018-09-21 15:17:29 +01:00
Martin Sumner
041e86fed9
Merge pull request #181 from martinsumner/mas-i180-dollarkeyregex 
Mas i180 dollarkeyregex
 2018-09-21 13:27:53 +01:00
Martin Sumner
ba93ea149f Account for slower tests in older Erlang versions 
Tests run slower in OTP16 - so do less work in this version.
 2018-09-21 13:27:10 +01:00
Martin Sumner
1a3d3daa89 Add regex support to $key index 
Regex to be applied to key only
 2018-09-21 12:04:32 +01:00
Martin Sumner
d27be07f67
Merge pull request #179 from martinsumner/mas-i178-bookhead 
Mas i178 bookhead
 2018-09-20 14:24:17 +01:00
Martin Sumner
d999bc1e4a Merge branch 'master' into mas-i178-bookhead 2018-09-20 12:08:46 +01:00
Martin Sumner
c439e4144a Add new book_headonly/4 API 
To address special situation of performing a head requets in head_only mode - where a sub-key is a required input.
 2018-09-20 12:08:33 +01:00
Martin Sumner
9c15a39023
Merge pull request #177 from martinsumner/rdb/quviq-i13-startkey-doc 
Update docs for start key
 2018-09-20 12:02:08 +01:00
Russell Brown
f12d277875 Update docs for start key 
Quviq/leveled/issues/13 shows the confusion around the purpose of
start key. Hopefully this commit clarifies that start key is a further
refinement to the range start. It's essentially and AND.
 2018-09-20 11:21:55 +01:00
Martin Sumner
f4a7a729ae
Merge pull request #175 from martinsumner/rdb/quviq-i15-regex-doc 
Update docs to clarify that regex cannot be run on integers
 2018-09-19 18:31:39 +01:00
Martin Sumner
572befbcfa
Merge pull request #176 from martinsumner/mas-i174-refactorsaferead 
Refactor safe_read
 2018-09-19 18:30:15 +01:00