umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
1612 commits 1 branch 0 tags 25 MiB
Commit graph

1612 commits

This branch
This branch
All branches
Author SHA1 Message Date
martinsumner
d2bd01eaf1 Add fast fail to skiplist 
Add a bloom filter to the skiplist, to make it faster at returning not
found.  The SkipList is now encapsulated within a dict().
 2016-12-09 18:30:40 +00:00
martinsumner
f0db730f07 Adjust jitter settings 2016-12-09 16:34:15 +00:00
martinsumner
82cb49638a Attempt at performance improvement 
Try to add some extra jitter in to the process of L0 writes, and also
make L0 writes delayed to help with bufferring
 2016-12-09 14:36:03 +00:00
martinsumner
349d194a7c Increase jitter slightly 2016-12-09 09:52:31 +00:00
martinsumner
5bdb7fd7fa Alter Riak HEAD 
Change the extract of Riak metadata.

In Riak-based volume tests hte writing of SFT files is tanking.  Could
this be the "extra" metadata.  i.e. There are only current plans to look
at the vclock.  Sibling count is free to fetch, what if we just get
these two items, will it be less CPU to extract the metadata, but also
will the reduced weight reduce the downstream impact?
 2016-12-08 23:38:50 +00:00
martinsumner
2f4013f430 Set Jitter correctly this time 2016-12-08 21:02:39 +00:00
martinsumner
b03ef664c8 Experiment with SFT settings 2016-12-08 20:52:17 +00:00
martinsumner
f07ab85a81 Oops 2016-12-08 18:36:57 +00:00
martinsumner
34e7ce170a Jitter co-ordination issues 
Experiment to try and reoslve jitter co-orsination issue
 2016-12-08 18:35:20 +00:00
martinsumner
ac45f95559 Investigate jitter coordination 2016-12-08 18:30:45 +00:00
martinsumner
c40e5d2d30 Reduce log noise 2016-12-08 16:43:35 +00:00
martinsumner
e2af60f72b Add cache size jitter 
First pan-riak volume requests showed coordination issues
 2016-12-08 16:38:44 +00:00
martinsumner
441fff13e5 Merge pull request #6 from martinsumner/OTP16 
OTP16
 2016-12-03 15:01:28 +00:00
martinsumner
3417baa3b8 Simple test 
To try and pinpoint any issue with _int index (as seen in Riak
integrtaion testing)
 2016-12-02 17:39:28 +00:00
Martin Sumner
d07cad34b7 Why lists reverse? 
It doesn’t make sense as the Acc may not be a list.  Not failing any
tests.
 2016-12-02 00:40:00 +00:00
martinsumner
3ec144d33d Remove unnecessary branch 2016-12-01 11:31:25 +00:00
martinsumner
dd0e9de050 Edit comment 2016-11-30 23:24:12 +00:00
martinsumner
c0384d7a86 Extend small skiplist test further 2016-11-30 23:23:10 +00:00
martinsumner
39f61db92b Further testing of skiplists 
Suspected at some stage there was a problem with small lists that never
hit a marker.  Can't find it yet through test
 2016-11-30 23:21:39 +00:00
martinsumner
364527f3b8 Add potential support for deeper layers 
Unproven attempt to make skiplist code more generic
 2016-11-30 22:35:23 +00:00
martinsumner
743d59c71b Make SkipList Two-Level 
Read and write times were increasing as the siz eof skiplist got larger
(e.g. over 2000).  Tried instead with a smaller skip width and a
two-tier skiplist and this gave much more regular eprformance as the
size of the skiplist went up.
 2016-11-30 15:36:27 +00:00
martinsumner
e3783485de Minor SkipList speed-up 2016-11-29 11:13:08 +00:00
martinsumner
dd6201b34b Try to avoid crashing on invalid key length 
May occurr in corrupted files.
 2016-11-29 00:27:23 +00:00
martinsumner
b62b38caa6 Tidy up test at end 
May cause issues otherwise if there is an attempt to corrupt the journal
running in another bookie
 2016-11-28 23:00:12 +00:00
martinsumner
e8c1d39df9 Switch to binary format Riak object 
Initial change to try and test assuming that leveled received the binary
format of Riak objects (and parses that for metadata).
 2016-11-28 22:26:09 +00:00
martinsumner
b0a515553f Merge remote-tracking branch 'origin/OTP16' into OTP16 
# Conflicts:
#	src/leveled_cdb.erl
 2016-11-27 00:18:20 +00:00
martinsumner
45f7fcb41a Datasync added for riak_sync strategy 2016-11-27 00:16:49 +00:00
Martin Sumner
0cb5fa39d9 Basho Erlang didn't like o_sync 
Or some options, trying sync.
 2016-11-26 23:03:07 +00:00
Martin Sumner
8f83c5226d Add log of write ops 2016-11-26 22:59:33 +00:00
Martin Sumner
feb7149a8d Add schema file 2016-11-26 22:10:51 +00:00
martinsumner
2fdf70ddf6 Add further skiplist test 2016-11-26 12:52:10 +00:00
Martin Sumner
fe080895fd Revert type definition 
Can’t find a type definition support din both OTP 16 and OTP 18, so
reverting to not defining type
 2016-11-25 18:22:35 +00:00
Martin Sumner
ec32a7e3eb OTP16 compliance - array type 2016-11-25 18:20:17 +00:00
martinsumner
196c807b5e Pass through sync_strategy 
Allow to switch for Riak to use o_sync as the sync flag rather than sync
 2016-11-25 17:41:08 +00:00
martinsumner
03d025d581 Replace ledger-side gb_trees 
Try to make minimal change to replace gb_trees with gb_tree API-like
skiplists
 2016-11-25 14:50:13 +00:00
martinsumner
c9afe34fea Missing key tming test 2016-11-24 23:31:13 +00:00
martinsumner
54eedb8282 Add out of range tests for skiplist range 2016-11-24 23:22:19 +00:00
martinsumner
3bbfd8b923 Tighten SkipList range 
Make SkipList range accurate - trim the edges
 2016-11-24 23:09:17 +00:00
martinsumner
2d3b1bbf2c Ongoing work on skip lists 
Initial rough implementation with some timing tests
 2016-11-24 20:16:41 +00:00
martinsumner
96b9e1faa3 Initial Commit for SkipLists 
To speed check on alternate platform
 2016-11-22 23:21:47 +00:00
martinsumner
e06888ad58 Merge pull request #5 from martinsumner/initial-setup 
Correctly set array type
 2016-11-21 22:27:42 +00:00
martinsumner
638fc69e01 Correctly set array type 
Otherwise cannot compile in both OTP 16 and 17
 2016-11-21 22:26:12 +00:00
martinsumner
4099f05616 Merge pull request #4 from martinsumner/initial-setup 
Initial setup
 2016-11-21 19:29:25 +00:00
martinsumner
51dbad95c0 Change FoldBucketsFun to take just bucket 
FoldBucketsFun does not take keys should be a 2-arity function (Bucket,
Acc).
 2016-11-21 14:12:17 +00:00
martinsumner
acdc702e7d Check not snap 
Make sure snapshots cannot trigger deletion
 2016-11-21 12:35:20 +00:00
martinsumner
0f7e421371 Add destruction 
Allow a store to be cleared out and destroyed
 2016-11-21 12:34:40 +00:00
martinsumner
386d40928b Fast List Buckets 
Copied the technique from HanoiDB to speed up list buckets.
 2016-11-20 21:21:31 +00:00
martinsumner
f40ecdd529 Pick-up test misses 
There were some coverage misses in tests, so check in unit test coverage
or remove branches not currently needed.
 2016-11-18 21:35:45 +00:00
martinsumner
427b97873c Change FoldKeysFun 2016-11-18 17:58:43 +00:00
martinsumner
57727e74f9 D'oh 2016-11-18 17:14:01 +00:00