umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions
643 commits 1 branch 0 tags 25 MiB
Commit graph

643 commits

This branch
This branch
All branches
Author SHA1 Message Date
martinsumner
a1c49b668a Fix empty file again 
No special definition of empty required, as now an empty list when empty
 2017-03-14 00:17:09 +00:00
martinsumner
2b0ec1d9cc Don't double-loop on slots 
Previous version built a list of slots, then iterated over it to build a
list of binaries.  This converts the slot to a binary before building
the list
 2017-03-13 23:51:48 +00:00
martinsumner
4f0622d2ac Merge remote-tracking branch 'refs/remotes/origin/mas-sstblock-i42' into mas-sstblockv2-i42 2017-03-13 21:09:13 +00:00
martinsumner
c5bb150f97 Drop some logs 
Not found to be interesting so far
 2017-03-13 20:30:33 +00:00
martinsumner
54534e725f Experiment with smaller scan width 
When testing with large numbers of 2i terms (and hence more Riak
Metadata), there is a surge in slow response times when there are
multiple concurrent merge events.

This could be veyr short term CPU starvation because of the merge
process.  Perhaps it is delays waiting for the scan to complete -
smaller scanwidth may mena more interleaving and less latency?
 2017-03-13 19:53:12 +00:00
martinsumner
5311a157d5 Merge remote-tracking branch 'refs/remotes/origin/mas-sstblock-i42' into mas-sstblockv2-i42 2017-03-13 19:22:41 +00:00
martinsumner
c787e0cd78 Handle corrupted Ledger Key when applying recovery strategy 
Otherwise may blow up in journal_compaction_bustedjournal test
 2017-03-13 14:32:46 +00:00
martinsumner
f3e962c43a Add level to SST slow fetch log 2017-03-13 12:16:36 +00:00
martinsumner
f2cd9b3f33 Consistency of empty slotlist references 
Need to return an empty slotlist in a consistent way
 2017-03-11 13:04:55 +00:00
martinsumner
1f8de798bd Fix empty slot issue 2017-03-11 12:41:30 +00:00
martinsumner
a07770a3df Unit tets of lookup over-size issue 
A mistake meant resetting to lookup on a skipped key would cause issues
if the skipped key ocurred under a no_lookup slot after the ?SLOT_SIZE
had been reached.  This caused the slot to switch to lookup, but beyond
the maximum size
 2017-03-11 00:03:55 +00:00
martinsumner
4e4f498f20 Correctly set no_lookup on skip_key 
Otherwise could change to lookup after the size limit has been reached
 2017-03-10 23:48:17 +00:00
martinsumner
1813317121 Correctly identify empty slotlist 2017-03-10 22:49:00 +00:00
martinsumner
b2f3d882a9 Draft of branch to condense range_only keys 2017-03-10 20:43:37 +00:00
martinsumner
730ab2ec48 tidy out io:format 2017-03-10 11:10:15 +00:00
martinsumner
601f43de3d Merge remote-tracking branch 'refs/remotes/origin/master' into mas-sstblock-i42 2017-03-10 10:24:51 +00:00
Martin Sumner
121f410b8f Merge pull request #45 from martinsumner/mas-shortenfilepath 
Mas shortenfilepath
 2017-03-10 08:59:41 +00:00
martinsumner
d7eee2f9c9 Remove rogue log 2017-03-09 22:24:11 +00:00
martinsumner
39a005a8d0 Try and be consistent in flilename format 
i.e. ./filename.sst
 2017-03-09 21:52:29 +00:00
martinsumner
bfcf981485 Correct root path setting in pclerk 2017-03-09 21:32:36 +00:00
martinsumner
4c59342600 Change SST reference to split filename 
The manifest and the logs are bloated by having the full file path for
every filename in there - given the root path is constant.

Could also cause issues if the mount point is ever changed.
 2017-03-09 21:23:09 +00:00
Martin Sumner
3582f4bc34 Merge pull request #44 from martinsumner/mas-manifestbug-i43 
Mas manifestbug i43
 2017-03-09 09:42:11 +00:00
martinsumner
3b2d8691b2 Make sure selection is just the manifest entry 2017-03-08 15:10:08 +00:00
martinsumner
90b587b083 Allow manifest entry to be selected from level 2 2017-03-08 12:09:10 +00:00
martinsumner
04cfb453c4 Fetch specific block only 
Rely on CRC check in zlib.  Still need to catch on failure
 2017-03-07 20:19:11 +00:00
Martin Sumner
3c5740e7bf Merge pull request #41 from martinsumner/mas-2iphase2-i34 
Mas 2iphase2 i34
 2017-03-07 11:59:05 +00:00
martinsumner
cd82bba6cd Use cache size - don't recalc length 2017-03-06 21:44:20 +00:00
martinsumner
9ad6969b0d Seed randomnes at Actor startup 2017-03-06 21:35:02 +00:00
martinsumner
32cdf26ffb Correct to catch new response to snapshot request 2017-03-06 19:37:43 +00:00
martinsumner
87f2c5d7ae Merge remote-tracking branch 'origin/mas-2iphase2-i34' into mas-2iphase2-i34 
# Conflicts:
#	src/leveled_log.erl
 2017-03-06 18:44:22 +00:00
martinsumner
c92107e4b4 2i order of events 
When running a load of mainly 2i queries, there is a huge cost in the
previous snapshot code.  The time taken to create a clone of the
Penciller (duplicating all the LoopState) varied between 1 and 200ms
depedning on the size of the LoopState.

For 2i queries, most of that LoopState was then being thrown away after
running the query against the levelzero_cache.  This was taking < 1ms on
average.  It would be better to avoid the o(100)ms of CPU  burning and
block for o(1)ms - so th eorder of events have been changed ot filter
first so only the small part of the LoopState actually required is
copied to the clone.
 2017-03-06 18:42:32 +00:00
Martin Sumner
c1dc92720c Random, random, random 
well random had me foxed.  As the clone was short-lived process it only
called random once - and so always got the same answer.

random has to be seeded to give different answers when called once from
a process - so this is now seeded in leveed_log
 2017-03-06 13:51:38 +00:00
martinsumner
eb6f668fcd Use log at random 
Easy way to sample frequent things - especially when they'r ein ocverag
equeries
 2017-03-06 10:34:56 +00:00
martinsumner
94a39e4c4d Fix double logging of B0004 2017-03-06 10:27:48 +00:00
martinsumner
5c2f05858d Alter logging to help understand performance factors 
Change logging of the snapshots to better understand performance
 2017-03-06 10:17:51 +00:00
Martin Sumner
b1e12a3bc5 Update PATHS.md 2017-03-04 20:56:32 +00:00
Martin Sumner
c696d27d6e Merge pull request #40 from martinsumner/mas-docs6 
Mas docs6
 2017-03-04 20:51:32 +00:00
martinsumner
7ceb6256c7 Properly attempt to fix link 2017-03-04 20:50:30 +00:00
martinsumner
542c519cd1 Attempt to fix link 2017-03-04 20:49:38 +00:00
martinsumner
bc5388710b Update SST comments 2017-03-04 20:47:46 +00:00
martinsumner
4ec7d835bd Add further info to HEAD path - and link to it 2017-03-04 20:45:28 +00:00
martinsumner
9190699021 Add HEAD path description 2017-03-04 20:36:53 +00:00
Martin Sumner
47060740b6 Merge pull request #39 from martinsumner/mas-refine2i-34 
Mas refine2i 34
 2017-03-02 22:18:46 +00:00
martinsumner
426f5df75c Fix dialyzer warnings 2017-03-02 21:24:40 +00:00
martinsumner
30e23fe514 Add no_lookup feature to snapshots 2017-03-02 18:23:47 +00:00
martinsumner
b01f7d23df Add pre-filter for 2i queries 2017-03-02 17:49:43 +00:00
Martin Sumner
84d53295cc Merge pull request #38 from martinsumner/mas-bitcaskcompare 
Mas bitcaskcompare
 2017-02-28 21:28:03 +00:00
martinsumner
8482f5acc9 Add result from first hour 2017-02-28 21:26:50 +00:00
martinsumner
a82ce44255 Add clarifying statement 2017-02-28 21:23:54 +00:00
martinsumner
db73ecf26c Add bitcask volume test results 2017-02-28 21:19:46 +00:00