umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
290 commits 1 branch 0 tags 25 MiB
Commit graph

250 commits

Author SHA1 Message Date
martinsumner
b0a515553f Merge remote-tracking branch 'origin/OTP16' into OTP16 
# Conflicts:
#	src/leveled_cdb.erl
 2016-11-27 00:18:20 +00:00
martinsumner
45f7fcb41a Datasync added for riak_sync strategy 2016-11-27 00:16:49 +00:00
Martin Sumner
0cb5fa39d9 Basho Erlang didn't like o_sync 
Or some options, trying sync.
 2016-11-26 23:03:07 +00:00
Martin Sumner
8f83c5226d Add log of write ops 2016-11-26 22:59:33 +00:00
martinsumner
2fdf70ddf6 Add further skiplist test 2016-11-26 12:52:10 +00:00
Martin Sumner
fe080895fd Revert type definition 
Can’t find a type definition support din both OTP 16 and OTP 18, so
reverting to not defining type
 2016-11-25 18:22:35 +00:00
Martin Sumner
ec32a7e3eb OTP16 compliance - array type 2016-11-25 18:20:17 +00:00
martinsumner
196c807b5e Pass through sync_strategy 
Allow to switch for Riak to use o_sync as the sync flag rather than sync
 2016-11-25 17:41:08 +00:00
martinsumner
03d025d581 Replace ledger-side gb_trees 
Try to make minimal change to replace gb_trees with gb_tree API-like
skiplists
 2016-11-25 14:50:13 +00:00
martinsumner
c9afe34fea Missing key tming test 2016-11-24 23:31:13 +00:00
martinsumner
54eedb8282 Add out of range tests for skiplist range 2016-11-24 23:22:19 +00:00
martinsumner
3bbfd8b923 Tighten SkipList range 
Make SkipList range accurate - trim the edges
 2016-11-24 23:09:17 +00:00
martinsumner
2d3b1bbf2c Ongoing work on skip lists 
Initial rough implementation with some timing tests
 2016-11-24 20:16:41 +00:00
martinsumner
96b9e1faa3 Initial Commit for SkipLists 
To speed check on alternate platform
 2016-11-22 23:21:47 +00:00
martinsumner
638fc69e01 Correctly set array type 
Otherwise cannot compile in both OTP 16 and 17
 2016-11-21 22:26:12 +00:00
martinsumner
51dbad95c0 Change FoldBucketsFun to take just bucket 
FoldBucketsFun does not take keys should be a 2-arity function (Bucket,
Acc).
 2016-11-21 14:12:17 +00:00
martinsumner
acdc702e7d Check not snap 
Make sure snapshots cannot trigger deletion
 2016-11-21 12:35:20 +00:00
martinsumner
0f7e421371 Add destruction 
Allow a store to be cleared out and destroyed
 2016-11-21 12:34:40 +00:00
martinsumner
386d40928b Fast List Buckets 
Copied the technique from HanoiDB to speed up list buckets.
 2016-11-20 21:21:31 +00:00
martinsumner
f40ecdd529 Pick-up test misses 
There were some coverage misses in tests, so check in unit test coverage
or remove branches not currently needed.
 2016-11-18 21:35:45 +00:00
martinsumner
6684e8e1d3 Refine query to accept fold functions 
Need to be able to pass external fold functions into different queries,
to work as a Riak backend
 2016-11-18 15:53:22 +00:00
martinsumner
ac223ced68 Add FoldKeysFun 
Add the capability to pass FoldKeysFun into the index_query to allow for
compatability with riak backend requirements.
 2016-11-18 11:53:14 +00:00
martinsumner
e3004faa6e Add support for folding keys on a single bucket 2016-11-17 15:55:29 +00:00
martinsumner
aa355a0aae Coverage cheat pt 2 2016-11-14 20:56:59 +00:00
martinsumner
8cbe2ef93a Coverage cheats 
You juke the stats, and majors become colonels.  I've been here before
 2016-11-14 20:43:38 +00:00
martinsumner
630f802780 Inker Close nastiness 
Try to stop some of the potential deadlocking around Inker close and
prove that snapshots at higher Manifest SQNs can be ignored
 2016-11-14 19:34:11 +00:00
martinsumner
75d6af75c6 Penciller review 
The penciller attempt to close the L0 file if pending was unpredictable
in behaviour.  If a L0 file is still pending it will be lost - but this
is at least a predictable event.
 2016-11-14 17:18:28 +00:00
martinsumner
eb76a6dbcd Multi-key test 
Now failing with timeout coverage enabled.  100 keys is unnecessray so
reduce to 20 per loop, but suspect this is another issue
 2016-11-14 12:43:45 +00:00
martinsumner
c0e1455430 Remove PromptDelete 
Now that deleted journals are stored in waste for the retention period -
don't worry about deleting them
 2016-11-14 11:40:02 +00:00
martinsumner
44738f7c75 Deferred Deletion of Journals 
This allows for deleted journals to be retained for a period (the
waste_retnetion_period).  The idea being that a backup strategy can
ensure that all journals are backed up, even ones created and removed
from within a backup period - so that any restore pont is possible.

This is also a pre-cursor to removing some of the PromptDelete
complexity from the Inker Clerk - all compactions can prompt deletion as
deletion is now deferred.
 2016-11-14 11:17:14 +00:00
martinsumner
3d737dcf3c More sensible test assertion 
Also better for test coverage stats
 2016-11-09 19:38:02 +00:00
martinsumner
2458a90fb3 Remove overlapped case clause 2016-11-09 16:35:13 +00:00
martinsumner
feb3c2a524 Add recovr strategy test 2016-11-09 16:27:47 +00:00
martinsumner
3a77591da9 Removed unnecessary timeout condition from rolling state in CDB 2016-11-09 00:14:57 +00:00
martinsumner
0cf96e7198 Removed daft last function from SFT 2016-11-09 00:13:52 +00:00
martinsumner
0905ea51e2 Further corrupt file unit test for CDB 2016-11-08 23:07:03 +00:00
martinsumner
cc3cbc983b Tidy up closure of CDB Files 2016-11-08 22:43:22 +00:00
martinsumner
8b8652e2d7 Coverage Tests 
Add nonsense tests for nonsense coverage on standard methods.

Look at CDB search_hash_table - looks like it doubled-up on break-outs
so that one would never get hit
 2016-11-08 01:03:09 +00:00
martinsumner
b43ec0de7e Fix the fix 2016-11-08 00:46:01 +00:00
martinsumner
6f0abea401 Stop unncessary handling of pending_delete in test 2016-11-08 00:44:51 +00:00
martinsumner
105d3b66f2 Add hash clash unit tests 
Add unit tests for keys with clashing hashes
 2016-11-07 23:53:14 +00:00
martinsumner
b47f8963d5 Speed of corrupted sft unit test 
Test too slow when running with coverage
 2016-11-07 19:20:46 +00:00
martinsumner
5a36f053d9 End-to-end testing of corrupted bloom 
Add a unit test calling the API of a SFT file clerk with a corrupted
bloom
 2016-11-07 18:26:55 +00:00
martinsumner
05338a1e85 Switch SFT to fsm 
To amke it easier to context switch between the two file formats, make
the SFT file clerk a FSM like the CDB one.
 2016-11-07 17:08:50 +00:00
martinsumner
8035583301 Comment review 2016-11-07 11:17:13 +00:00
martinsumner
079e713e16 Comment fiddling 2016-11-07 11:07:12 +00:00
martinsumner
6bf72849bc Comments reviewed 2016-11-07 11:02:38 +00:00
martinsumner
8f068c1280 Fixup API to allow for Tags to be passed 
plus fixes to previous commits moving API functions into testutil
 2016-11-07 10:42:49 +00:00
martinsumner
37c23a5b38 Shift pause out of leveled 
Leveled will now signal the need for a pause due to back-pressure, but
not actually pause itself.  The hope is that in a riak implementation
this pause can be managed by the put_fsm, and so not lock the store.
 2016-11-07 10:27:38 +00:00
martinsumner
4583460328 Clean API of Riak-specific Methods 
Clena the API of Riak specific methods, and also resolve timing issue in
simple_server unit test.  Previously this would end up with missing data
(and a lower sequence number after start) because of the penciller_clerk
timeout being relatively large in the context of this test.  Now the
timeout has bene reduced the L0 slot is cleared by the time of the
close.  To make sure an extra sleep has been added as a precaution to
avoid any intermittent issues.
 2016-11-07 10:11:57 +00:00