umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
749 commits 1 branch 0 tags 25 MiB
Commit graph

749 commits

This branch
This branch
All branches
Author SHA1 Message Date
martinsumner
569b498727 Resolve dialyzer warnings 
Botched switch to leveled_log in list - so reoslved dialyzer warnings
 2017-06-01 22:03:51 +01:00
martinsumner
32612dfe4a Yet another array type OTP16 issue 2017-06-01 21:39:01 +01:00
martinsumner
afbf918f2c Change from using array type 
Won't compile in OTP16
 2017-06-01 21:37:23 +01:00
Martin Sumner
1a051eab35 Merge pull request #73 from martinsumner/mas-logbase-i72 
Change log to keyfind
 2017-06-01 10:30:53 +01:00
martinsumner
204a5a8caa Change log to keyfind 
Save doing dict:to_list each call to log
 2017-05-31 10:56:48 +01:00
Martin Sumner
92275cfc2f Merge pull request #71 from martinsumner/mas-specs-i61b 
Mas specs i61b
 2017-05-30 20:34:25 +01:00
martinsumner
e66e326c2b Dialyzer help 
Add specs and docs
 2017-05-30 16:50:23 +01:00
martinsumner
e86d91dcf6 Give dialyzer a hand 
add specs and docs
 2017-05-30 16:25:55 +01:00
martinsumner
c073344163 Dialyzer helper 
Add specs and docs to external API
 2017-05-30 15:36:50 +01:00
martinsumner
fc4802c539 Dialyzer help 
Add specs and docs
 2017-05-30 11:37:10 +01:00
martinsumner
0459591fd7 spec and doc - leveled_pmanifest 
Give the dialyzer some help
 2017-05-26 15:11:33 +01:00
martinsumner
f23072fb54 Merge remote-tracking branch 'refs/remotes/origin/master' into mas-specs-i61b 2017-05-26 13:45:20 +01:00
Martin Sumner
041da54c62 Merge pull request #70 from martinsumner/mas-iclerkrelease-i69 
Release penciller snapshot after journal compaction
 2017-05-26 13:44:49 +01:00
martinsumner
c664176247 Release penciller snapshot after journal compaction 
As otherwise memory consumption beocmes an issue, as they will take an
hour to timeout naturally.
 2017-05-26 10:51:30 +01:00
Martin Sumner
5c5fa8ee8d Merge pull request #68 from martinsumner/mas-isempty-i67 
Mas isempty i67
 2017-05-23 15:54:47 +01:00
Martin Sumner
96a548e17a Change tests - binary keys 
the new code requires bucket listing to be on binary keys not just
binary buckets.  As this is only intended for use within Riak (where
all keys are buckets are binaries), this constraint seems OK.

A test needed changing to ensure it had a binary key in the bucket.
 2017-05-23 15:54:11 +01:00
martinsumner
0d8ab0899e Add test for is_empty 
Bucket listing didn't care if keys were active - now does.
 2017-05-23 11:59:44 +01:00
martinsumner
2738ee951c Add specs and docs for Penciller 
Help the dialyzer
 2017-05-22 18:09:12 +01:00
martinsumner
366253831a Add dpecs to inker 
Give the dialyzer a hand
 2017-05-22 15:31:42 +01:00
Martin Sumner
7ca672d7bc Merge pull request #66 from martinsumner/mas-cdbindex-i65 
Switch IndexCache to tuple
 2017-05-22 13:12:56 +01:00
martinsumner
dfcb30a33e Switch IndexCache to tuple 
So that direct element fetch should be faster?
 2017-05-22 10:24:49 +01:00
Martin Sumner
f45681610e Merge pull request #64 from martinsumner/mas-specs-i61a 
Mas specs i61a
 2017-05-22 10:03:07 +01:00
martinsumner
c90e0f824d Spelling error change 2017-05-22 10:00:34 +01:00
martinsumner
a81dd2839e Merge remote-tracking branch 'refs/remotes/origin/master' into mas-specs-i61a 2017-05-22 09:58:41 +01:00
Martin Sumner
8e334c0b5a Merge pull request #63 from martinsumner/mas-compactscore-i35 
Mas compactscore i35
 2017-05-22 09:57:32 +01:00
Martin Sumner
2bbb504c02 Up max file compactions per run 
Try and do more compaction work for each run
 2017-05-21 22:06:41 +01:00
martinsumner
695638f34c Loop for get_positions 
The CDB FSM process can be blocked by get_positions for all positions,
so loop around the index outside of the FSM process to allow for other
messages to interleave.
 2017-05-20 12:25:06 +01:00
martinsumner
319dc5f388 CDB changes to add timing log 
Forgot to save before commit last time!
 2017-05-19 14:28:55 +01:00
martinsumner
36a48c16e5 Add timing log to position scan 
Position scan may be very expensive, add timing log to confirm
 2017-05-19 13:59:57 +01:00
martinsumner
18a12ff9ff Improve comments 2017-05-18 14:09:45 +01:00
martinsumner
8b3ca78d49 spec help for SST file 2017-05-18 12:29:56 +01:00
martinsumner
28f45749d7 Add specs to exported API of CDB files 
Time to give the dialyzer some help
 2017-05-17 12:54:02 +01:00
Martin Sumner
e4c6dc7119 Update VOLUME.md 
Minor edits for clarity
 2017-04-24 11:56:11 +01:00
Martin Sumner
b0c91172f4 Merge pull request #60 from martinsumner/mas-sweeperfold-i59 
Mas sweeperfold i59
 2017-04-24 11:39:15 +01:00
martinsumner
d5860790be Update future branch descriptions 2017-04-24 11:33:05 +01:00
martinsumner
a052edaea0 Add volume test information 
Volume testing with AAE rebuilds
 2017-04-24 11:24:14 +01:00
martinsumner
fbb4879d81 Change fold_heads to do basic Journal presence check 
This at least checks the file is present, and the Key exists in the
index of that file.  If the value is corrupt it will be removed by
compation, and then this will fail (unless the file is never compacted).

TODO: resolve issus of files which are corrupt - but never compacted
- a job for backup?
 2017-04-21 15:55:03 +01:00
martinsumner
4d12dfe0ab Returning snapshots 
If the clerk updates the manifest - it might not recognise changes to
the manifest made since the clerk took the manifest.  So the penciller
must merge its view of the snapshots back in to the updated manifest
 2017-04-19 22:46:37 +01:00
Martin Sumner
fa9daf8696 Correct async fold 
fold objects which snaps in the fold was implemented incorrectly - it
took information from the LedgeCache at the point of the request, not
at the point of the fold.  So the LedgerCache SQN may have been
surpassed in the Penciller by the time the fold was called.
 2017-04-17 23:01:55 +01:00
martinsumner
7cba182951 Merge remote-tracking branch 'origin/mas-sweeperfold-i59' into mas-sweeperfold-i59 
# Conflicts:
#	src/leveled_bookie.erl
 2017-04-17 15:07:35 +01:00
martinsumner
50d95ef6aa Move snapshot inside of the fold function 
riak_kv_sweeper gets the async fold function, then determines if the
function can be called.  If the system is busy the fold may be queued,
and may never be acted upon.

This may cause issues with snapshot timeouts etc.
 2017-04-17 15:03:03 +01:00
Martin Sumner
e01efe02f6 Long snaphsot timeout increase 
Increase this to 90 minutes.  The first time all the snapshots are
rebuilt it may take a long time, but they all get scheduled together -
and queued until concurrency limits allow it to be completed.

currently the snapshot is made on initialisation, and only released
when completed (which may be after the queue).  so the last couple of
snapshots were over-shooting the 1 hour.
 2017-04-13 22:43:29 +01:00
Martin Sumner
618d9cf53b FoldHeads to output binary 
so that byte_size will work in sweeper
 2017-04-11 11:17:27 +01:00
Martin Sumner
9badc8fbe7 Merge branch 'master' into mas-sweeperfold-i59 2017-04-10 21:49:08 +01:00
Martin Sumner
13a1574fa0 Merge pull request #57 from martinsumner/mas-metadatabin 
Mas metadatabin
 2017-04-10 21:47:45 +01:00
martinsumner
60b0b88226 Switch to binary response 
Force header in fold_heads to be a binary ... as this is what KV expects
 2017-04-07 17:08:40 +00:00
martinsumner
6d42abbc1a Add bybucket to fold_heads 
give fold_heads equivalent functionality to fold_objects - both can now
be done for allkeys and bybucket
 2017-04-07 14:56:28 +00:00
martinsumner
9375baf636 Add unit test for foldheads 
compare foldheads foldobjects and hahstree_query output
 2017-04-07 14:19:25 +00:00
martinsumner
b464e2e28c Extend foldobjects to support proxy object 
To allow for folds which probbaly don't need values to not always to
have to fetch the value
 2017-04-07 12:09:11 +00:00
Martin Sumner
4e9fa2a206 Timeout long-running snapshots 
Add logic to timeout long-running snapshots.
 2017-04-05 09:16:01 +01:00