umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
958 commits 1 branch 0 tags 25 MiB
Commit graph

958 commits

This branch
This branch
All branches
Author SHA1 Message Date
martinsumner
86b11803c9 Build and compare 
Build and compare of tictac trees.  These are mergable merkle trees that
are not cryptographically secure.
 2017-06-15 15:40:23 +01:00
Martin Sumner
f2beee6c35 Merge pull request #77 from martinsumner/mas-compactiontesting 
Mas compactiontesting
 2017-06-09 09:53:36 +01:00
martinsumner
b69ff38e30 Update docs 2017-06-06 16:49:50 +01:00
martinsumner
af69f946cf Add further compaction comments 
Link to the compaction branch and add further description
 2017-06-06 16:44:05 +01:00
martinsumner
e59b67d1be Table rendering 
seems to work in atom markdown not github markdown :-(
 2017-06-06 16:36:36 +01:00
martinsumner
94f3e036ea Add journal compaction testing 2017-06-06 16:30:02 +01:00
Martin Sumner
4cf3eea1eb Merge pull request #76 from martinsumner/mas-compactionconfig 
Change default compaction settings.

The change is as a result of the 12h volume tests, which showed a need to do more compaction work per cycle.
 2017-06-02 08:42:20 +01:00
Martin Sumner
15c52ae118 Change default compaction settings 
Need to allow specific settings to be passed into unit tests.

Also, too much journal compaction may lead to intermittent failures on
the basic_SUITE space_clear_on_delete test.  think this is because
there are less “deletes” to reload in on startup to trigger the cascade
down and clear up?
 2017-06-02 08:37:57 +01:00
Martin Sumner
39fd3e5c87 Merge pull request #75 from martinsumner/mas-arraytype-i73 
Mas arraytype i73
 2017-06-01 22:28:17 +01:00
martinsumner
569b498727 Resolve dialyzer warnings 
Botched switch to leveled_log in list - so reoslved dialyzer warnings
 2017-06-01 22:03:51 +01:00
martinsumner
32612dfe4a Yet another array type OTP16 issue 2017-06-01 21:39:01 +01:00
martinsumner
afbf918f2c Change from using array type 
Won't compile in OTP16
 2017-06-01 21:37:23 +01:00
Martin Sumner
1a051eab35 Merge pull request #73 from martinsumner/mas-logbase-i72 
Change log to keyfind
 2017-06-01 10:30:53 +01:00
martinsumner
204a5a8caa Change log to keyfind 
Save doing dict:to_list each call to log
 2017-05-31 10:56:48 +01:00
Martin Sumner
92275cfc2f Merge pull request #71 from martinsumner/mas-specs-i61b 
Mas specs i61b
 2017-05-30 20:34:25 +01:00
martinsumner
e66e326c2b Dialyzer help 
Add specs and docs
 2017-05-30 16:50:23 +01:00
martinsumner
e86d91dcf6 Give dialyzer a hand 
add specs and docs
 2017-05-30 16:25:55 +01:00
martinsumner
c073344163 Dialyzer helper 
Add specs and docs to external API
 2017-05-30 15:36:50 +01:00
martinsumner
fc4802c539 Dialyzer help 
Add specs and docs
 2017-05-30 11:37:10 +01:00
martinsumner
0459591fd7 spec and doc - leveled_pmanifest 
Give the dialyzer some help
 2017-05-26 15:11:33 +01:00
martinsumner
f23072fb54 Merge remote-tracking branch 'refs/remotes/origin/master' into mas-specs-i61b 2017-05-26 13:45:20 +01:00
Martin Sumner
041da54c62 Merge pull request #70 from martinsumner/mas-iclerkrelease-i69 
Release penciller snapshot after journal compaction
 2017-05-26 13:44:49 +01:00
martinsumner
c664176247 Release penciller snapshot after journal compaction 
As otherwise memory consumption beocmes an issue, as they will take an
hour to timeout naturally.
 2017-05-26 10:51:30 +01:00
Martin Sumner
5c5fa8ee8d Merge pull request #68 from martinsumner/mas-isempty-i67 
Mas isempty i67
 2017-05-23 15:54:47 +01:00
Martin Sumner
96a548e17a Change tests - binary keys 
the new code requires bucket listing to be on binary keys not just
binary buckets.  As this is only intended for use within Riak (where
all keys are buckets are binaries), this constraint seems OK.

A test needed changing to ensure it had a binary key in the bucket.
 2017-05-23 15:54:11 +01:00
martinsumner
0d8ab0899e Add test for is_empty 
Bucket listing didn't care if keys were active - now does.
 2017-05-23 11:59:44 +01:00
martinsumner
2738ee951c Add specs and docs for Penciller 
Help the dialyzer
 2017-05-22 18:09:12 +01:00
martinsumner
366253831a Add dpecs to inker 
Give the dialyzer a hand
 2017-05-22 15:31:42 +01:00
Martin Sumner
7ca672d7bc Merge pull request #66 from martinsumner/mas-cdbindex-i65 
Switch IndexCache to tuple
 2017-05-22 13:12:56 +01:00
martinsumner
dfcb30a33e Switch IndexCache to tuple 
So that direct element fetch should be faster?
 2017-05-22 10:24:49 +01:00
Martin Sumner
f45681610e Merge pull request #64 from martinsumner/mas-specs-i61a 
Mas specs i61a
 2017-05-22 10:03:07 +01:00
martinsumner
c90e0f824d Spelling error change 2017-05-22 10:00:34 +01:00
martinsumner
a81dd2839e Merge remote-tracking branch 'refs/remotes/origin/master' into mas-specs-i61a 2017-05-22 09:58:41 +01:00
Martin Sumner
8e334c0b5a Merge pull request #63 from martinsumner/mas-compactscore-i35 
Mas compactscore i35
 2017-05-22 09:57:32 +01:00
Martin Sumner
2bbb504c02 Up max file compactions per run 
Try and do more compaction work for each run
 2017-05-21 22:06:41 +01:00
martinsumner
695638f34c Loop for get_positions 
The CDB FSM process can be blocked by get_positions for all positions,
so loop around the index outside of the FSM process to allow for other
messages to interleave.
 2017-05-20 12:25:06 +01:00
martinsumner
319dc5f388 CDB changes to add timing log 
Forgot to save before commit last time!
 2017-05-19 14:28:55 +01:00
martinsumner
36a48c16e5 Add timing log to position scan 
Position scan may be very expensive, add timing log to confirm
 2017-05-19 13:59:57 +01:00
martinsumner
18a12ff9ff Improve comments 2017-05-18 14:09:45 +01:00
martinsumner
8b3ca78d49 spec help for SST file 2017-05-18 12:29:56 +01:00
martinsumner
28f45749d7 Add specs to exported API of CDB files 
Time to give the dialyzer some help
 2017-05-17 12:54:02 +01:00
Martin Sumner
e4c6dc7119 Update VOLUME.md 
Minor edits for clarity
 2017-04-24 11:56:11 +01:00
Martin Sumner
b0c91172f4 Merge pull request #60 from martinsumner/mas-sweeperfold-i59 
Mas sweeperfold i59
 2017-04-24 11:39:15 +01:00
martinsumner
d5860790be Update future branch descriptions 2017-04-24 11:33:05 +01:00
martinsumner
a052edaea0 Add volume test information 
Volume testing with AAE rebuilds
 2017-04-24 11:24:14 +01:00
martinsumner
fbb4879d81 Change fold_heads to do basic Journal presence check 
This at least checks the file is present, and the Key exists in the
index of that file.  If the value is corrupt it will be removed by
compation, and then this will fail (unless the file is never compacted).

TODO: resolve issus of files which are corrupt - but never compacted
- a job for backup?
 2017-04-21 15:55:03 +01:00
martinsumner
4d12dfe0ab Returning snapshots 
If the clerk updates the manifest - it might not recognise changes to
the manifest made since the clerk took the manifest.  So the penciller
must merge its view of the snapshots back in to the updated manifest
 2017-04-19 22:46:37 +01:00
Martin Sumner
fa9daf8696 Correct async fold 
fold objects which snaps in the fold was implemented incorrectly - it
took information from the LedgeCache at the point of the request, not
at the point of the fold.  So the LedgerCache SQN may have been
surpassed in the Penciller by the time the fold was called.
 2017-04-17 23:01:55 +01:00
martinsumner
7cba182951 Merge remote-tracking branch 'origin/mas-sweeperfold-i59' into mas-sweeperfold-i59 
# Conflicts:
#	src/leveled_bookie.erl
 2017-04-17 15:07:35 +01:00
martinsumner
50d95ef6aa Move snapshot inside of the fold function 
riak_kv_sweeper gets the async fold function, then determines if the
function can be called.  If the system is busy the fold may be queued,
and may never be acted upon.

This may cause issues with snapshot timeouts etc.
 2017-04-17 15:03:03 +01:00