umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions

Notes testing on non-compressed L1

Browse source
This commit is contained in:
Martin Sumner 2017-12-04 13:08:24 +00:00
parent f8ceedc9bb
commit fea87c21ec
1 changed files with 2 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/LATENCY.md
View file

@ -108,3 +108,5 @@ Looking at the number of files per second (across the cluster)
A potential way forward is to remove the compression from Levels 0 and 1. These levels will make a small fraction of the overall space in any reasonably sized store - so losing compression should not have a major impact on page_cache coverage. This would reduce both the slot_serialise and fold_toslot delays during merge. A potential way forward is to remove the compression from Levels 0 and 1. These levels will make a small fraction of the overall space in any reasonably sized store - so losing compression should not have a major impact on page_cache coverage. This would reduce both the slot_serialise and fold_toslot delays during merge.
A downside of removing compression will be the time to complete the CRC check on fetching and serialising. Initial unit tests indicated this more than doubled. Trying non-native CRC checks (such as xxHash) didn't change this impact. However, the side effect seems to have been mitigated by making the CRC check per block (and CRC checking the header separately). A downside of removing compression will be the time to complete the CRC check on fetching and serialising. Initial unit tests indicated this more than doubled. Trying non-native CRC checks (such as xxHash) didn't change this impact. However, the side effect seems to have been mitigated by making the CRC check per block (and CRC checking the header separately).
Testing the approach of not compressing L1 in a full-scale test was not successful though. Although the slot_serialise time dropped significantly, overall performance was worse - almost certainly due to increased disk write activity.