umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions

Comment update

Browse source
This commit is contained in:
martinsumner 2017-01-25 12:38:33 +00:00
parent 684f1bcd99
commit 7320b34681
1 changed files with 8 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

20
src/leveled_sst.erl
View file

@ -8,7 +8,7 @@
%% -------- Slots --------- %% -------- Slots ---------
%% %%
%% The view is built from sublists referred to as slot. Each slot is up to 128 %% The view is built from sublists referred to as slot. Each slot is up to 128
%% keys and values in size. Three strategis have been benchmarked for the %% keys and values in size. Three strategies have been benchmarked for the
%% slot: a skiplist, a gb-tree, four blocks of flat lists with an index. %% slot: a skiplist, a gb-tree, four blocks of flat lists with an index.
%% %%
%% Skiplist: %% Skiplist:
@ -23,7 +23,7 @@
%% %%
%% Indexed Blocks: %% Indexed Blocks:
%% build and serialise slot 342 microseconds %% build and serialise slot 342 microseconds
%% de-deriaise and check * 128 - 6746 microseconds %% de-deserialise and check * 128 - 6746 microseconds
%% flatten back to list - 187 microseconds %% flatten back to list - 187 microseconds
%% %%
%% The negative side of using Indexed Blocks is the storage of the index. In %% The negative side of using Indexed Blocks is the storage of the index. In
@ -34,23 +34,19 @@
%% %%
%% -------- Blooms --------- %% -------- Blooms ---------
%% %%
%% There is a summary bloom for the table. the summary bloom is split by the %% There is a bloom for each slot - based on two hashes and 8 bits per key.
%% first byte of the hash, and consists of two hashes (derived from the
%% remainder of the hash). This is the top bloom, and the size varies by
%% level.
%% Level 0 has 8 bits per key - 0.05 fpr
%% Level 1 has 6 bits per key - 0.08 fpr
%% Other Levels have 4 bits per key - 0.15 fpr
%% %%
%% With the indexed block implementation of the slot a second slot-level bloom %% Hashing for blooms is a challenge, as the slot is a slice of an ordered
%% is unnecessary (as the index itself yields a 0.003 % fpr). %% list of keys with a fixed format. It is likely that the keys may vary by
%% only one or two ascii characters, and there is a desire to avoid the
%% overhead of cryptographic hash functions that may be able to handle this.
%% %%
%% -------- Summary --------- %% -------- Summary ---------
%% %%
%% Each file has a summary - which is the 128 keys at the top of each slot in %% Each file has a summary - which is the 128 keys at the top of each slot in
%% a skiplist, with some basic metadata about the slot stored as the value. %% a skiplist, with some basic metadata about the slot stored as the value.
%% %%
%% The summary is stored seperately to the slots (wihtin the same file). %% The summary is stored seperately to the slots (within the same file).
%% %%
%% -------- CRC Checks --------- %% -------- CRC Checks ---------
%% %%