umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
986 commits 1 branch 0 tags 25 MiB
Commit graph

696 commits

Author SHA1 Message Date
Martin Sumner
5b4bc1ce59 Merge branch 'master' into mas-i108-cdbtimings 2017-11-20 17:34:50 +00:00
Martin Sumner
51f504fec5 Add extra slow_fetch test 
sometimes ct tests don’t hit this - surprisingly
 2017-11-20 17:29:57 +00:00
Martin Sumner
464baaa252 no_timing on key_check 
reader was out of step with delete_pending state
 2017-11-20 16:36:26 +00:00
Martin Sumner
faa5ef82aa Test logging of samples 
To prompt the log the journal size needs to be reduced
 2017-11-20 15:31:31 +00:00
Martin Sumner
de60a55be2 Add missing logref 2017-11-20 15:19:30 +00:00
Martin Sumner
fe0fc21461 Fix R16 dialzyer errors 2017-11-20 15:14:02 +00:00
Martin Sumner
8a43539090 Take sample timings from CDB files 
Periodically get a CDB file process to take samples of how long fetching keys/values takes - and record  those samples
 2017-11-20 14:58:43 +00:00
Martin Sumner
62a84b95bb Add fadvise help to scan 2017-11-20 10:40:09 +00:00
Martin Sumner
0e071d078e fold_objects in SQN order 
This adds a test that fold_objects works in SQN order
 2017-11-17 18:30:51 +00:00
Martin Sumner
50c81d0626 Make ink fold more generic 
Also makes the fold_from_sequence loop much easier to follow
 2017-11-17 14:54:53 +00:00
Martin Sumner
39ad5c9680 Make inker fold generic 
ink_loadpcl is in effect an inker fold - so abstract out the inker fold part to make this a generic capability
 2017-11-15 16:08:24 +00:00
Martin Sumner
f55cbbeac3 OTP 19 requires defaults in dialyzer 2017-11-13 14:02:39 +00:00
Martin Sumner
0208cde803 Drop sample size 
Scoring is relatively inaccurate - doesn't seem necessary to test sucha  bing sample, especially as eveyr file needs a score
 2017-11-10 14:40:07 +00:00
Martin Sumner
c4456568be Expand testing 
Reasoned that the case clause (RHS of []) that was previously uncovered by tests - is so as it is not a possible outcome.

So case clause removed.
 2017-11-10 12:56:12 +00:00
Martin Sumner
0c498f293d Test out-of-date update 
Check no recent_aae index is created
 2017-11-10 10:08:30 +00:00
Martin Sumner
46262e2105 Improve testing of safe reading 
Test didn't test happy day scenario correctly - and so bit flipping scenarios couldn't be trusted either.
 2017-11-09 20:22:43 +00:00
Martin Sumner
31505c1f5f get_nextkey no longer used 
So removed.  Also now, no reason to call read_next_key directly - and all errors all seem to be try/caught.
 2017-11-09 19:28:38 +00:00
Martin Sumner
ae70af5eec Add unit test to hit safe read edge cases 
... and they can't be hit.  So remove branches and let them fail if something unexplicable happens
 2017-11-09 17:12:47 +00:00
Martin Sumner
69f7e4c12a Add doc statement 2017-11-09 12:48:48 +00:00
Martin Sumner
ee47d62c24 iclerk testing 
Not sure if this scenario is unlikely or impossible - but filtering it seems harmless.

Abstracting out function makes testing of scoring scenarios a bit easier.

Catch-up on setting specs for external functions in the iclerk.
 2017-11-09 12:42:49 +00:00
Martin Sumner
53afd70174 Remove unnecessary logs 2017-11-09 11:17:50 +00:00
Martin Sumner
15a74fac5c Remove debug logs 2017-11-08 21:03:04 +00:00
Martin Sumner
4c05dc79f9 Merge branch 'master' into mas-aae-segementfoldplus 2017-11-08 18:38:49 +00:00
Martin Sumner
7de4dccbd9 Extend journal compaction test 
to cover with and without  waste retention.  Also makes sure that CDB files in a restarted store will respect the wast retention period set.
 2017-11-08 16:18:48 +00:00
Martin Sumner
1d2effc773 Improve docs and specs 
Focus on leveled_cdb
 2017-11-08 15:09:23 +00:00
Martin Sumner
22e894c928 Allow waste retnetion to be ignored 
If wast retention period is undefined, then it should be ignored - and no waste retained (rather than retaining waste for 24 hours as at present).

This wasn't working anyway - as reopen reader didn't get the cdb options (which didn't have the waste path on anyway) - so waste would not eb retained if the file had been opened after a stop/start.
 2017-11-08 12:58:09 +00:00
Martin Sumner
e8bd712fb8 Tidy up test shutdown 2017-11-08 11:20:22 +00:00
Martin Sumner
5cee3a8e4e Tidy up spec 
Also remove _app _sup originally added for dialyzer (due to false understanding they were needed for dialyzer)
 2017-11-07 19:41:39 +00:00
Martin Sumner
bb15c1f780 Take out OTP files 
Were only there for show
 2017-11-07 16:22:49 +00:00
Martin Sumner
bea094aaf5 no non-binary objects in inker 2017-11-07 13:43:29 +00:00
Martin Sumner
332286f35c From inker kv - value cannot be a term 2017-11-07 13:42:12 +00:00
Martin Sumner
8f27b3b628 Merge branch 'master' into mas-aae-segementfoldplus 2017-11-07 11:22:56 +00:00
Martin Sumner
0af0d85239 Add option description 
Add documentation of new options
 2017-11-07 10:22:27 +00:00
Martin Sumner
f358bd7622 Switch to using passed in compression method for maybe_compress 
When the compaction discovers compression is required it will used the passed in method at startup - not the method which had been previously defined.
 2017-11-06 21:16:46 +00:00
Martin Sumner
1d475235d1 Improve test coverage 
Make compress on receipt/compaction configurable
 2017-11-06 18:44:08 +00:00
Martin Sumner
61b7be5039 Make compression algorithm an option 
Compression can be switched between LZ4 and zlib (native).

The setting to determine if compression should happen on receipt is now a macro definition in leveled_codec.
 2017-11-06 15:54:58 +00:00
Martin Sumner
4c44e86eab Enable compression on receipt to Journal 
Rather than always deferring compression until compaction
 2017-11-05 22:33:32 +00:00
Martin Sumner
830906c552 Compress journal with lz4 
When the value is a binary (which should be the case with Riak)
 2017-11-05 21:48:57 +00:00
Martin Sumner
9fa8ed6cca Add LZ4 2017-11-03 14:18:49 +00:00
Martin Sumner
4fbb770a8c Revert "Failed attempt to hack in LZ4" 
This reverts commit 912920a53c.
 2017-11-03 11:47:25 +00:00
Martin Sumner
912920a53c Failed attempt to hack in LZ4 2017-11-03 11:47:00 +00:00
Martin Sumner
c6749e61a9 Split out block serialisation 
To allow for alternate compression scenarios to be more easily tested
 2017-11-03 11:04:31 +00:00
Martin Sumner
c8ad39b33b foldheads_bybucket adds segment list support 
Accelerate queries for foldheads_bybucket as well
 2017-11-01 22:00:12 +00:00
Martin Sumner
6beeadc7d8 Simplify SnapSQN check 
Less ugly
 2017-11-01 19:31:20 +00:00
Martin Sumner
6e0bf7bce3 Remove empty check 
It is not obvious why empty binaries can't be merged this way.  BothEmpty seems a pointless expression - and it never gets hit by test coverage.
 2017-11-01 18:19:37 +00:00
Martin Sumner
400202e38d Simpliy test assertion 2017-11-01 17:50:01 +00:00
Martin Sumner
5b5b4a3a29 Test coverage 
Code no longer requires LongRunning to be undefined so that it can be decided through bext guess.

Also cover branches of tictac tree code.
 2017-11-01 17:14:19 +00:00
Martin Sumner
53c3bf6c37 Remove get_slotid 
Had been used in some debug logging - now not called
 2017-11-01 17:05:35 +00:00
Martin Sumner
ee7f9ee4e0 Test coverage 
... and column width formatting
 2017-11-01 15:11:14 +00:00
Martin Sumner
033cf1954d Add check for too-small trees 
Provide a function for generating segmentfilter lists so that it can handle trees that are "too small".

Test those smaller trees - plus also false positives and cold caches
 2017-11-01 13:18:01 +00:00