Mas d34 ms.i446 plusplus (#448)

* Minor optimisations

Try to reduce the calls to ++, and ensure that where possible the shorted list is being copied.

* Pass Acc into function

So that the list can be accumulated efficiently, without an additional copy to add back the accumulator at the end.

* prepend to accumulators

Code review to make sure we prepend to accumulators everywhere, to reduce the copying involved.

attempt to further optimise in leveled_sst (where most expensive ++ occurs).  This optimises for the case when Acc is [], and enforces a series of '++' to start from the right, prepending in turn.  Some shell testing indicated that this was not necessarily the case (although this doesn't seem tobe consistently reproducible).

```
6> element(1, timer:tc(fun() -> KL1 ++ KL2 ++ KL3 ++ KL4 end)).
28
7> element(1, timer:tc(fun() -> KL1 ++ KL2 ++ KL3 ++ KL4 end)).
174
8> element(1, timer:tc(fun() -> KL1 ++ KL2 ++ KL3 ++ KL4 end)).
96
9> element(1, timer:tc(fun() -> KL1 ++ KL2 ++ KL3 ++ KL4 end)).
106
10> element(1, timer:tc(fun() -> KL1 ++ KL2 ++ KL3 ++ KL4 end)).
112

17> element(1, timer:tc(fun() -> lists:foldr(fun(KL0, KLAcc) -> KL0 ++ KLAcc end, [], [KL1, KL2, KL3, KL4]) end)).
21
18> element(1, timer:tc(fun() -> lists:foldr(fun(KL0, KLAcc) -> KL0 ++ KLAcc end, [], [KL1, KL2, KL3, KL4]) end)).
17
19> element(1, timer:tc(fun() -> lists:foldr(fun(KL0, KLAcc) -> KL0 ++ KLAcc end, [], [KL1, KL2, KL3, KL4]) end)).
12
20> element(1, timer:tc(fun() -> lists:foldr(fun(KL0, KLAcc) -> KL0 ++ KLAcc end, [], [KL1, KL2, KL3, KL4]) end)).
11
```

running eprof indicates that '++' and lists:reverse have been reduced (however impact had only previously been 1-2%)

* Add unit test to confirm (limited) merit of optimised list function

No difference in unit test with/without inline compilation, so this has been removed

* Update src/leveled_sst.erl

These functions had previously used inline compilation - but this didn't appear to improve performance

Co-authored-by: Thomas Arts <thomas.arts@quviq.com>

* Update src/leveled_sst.erl

Co-authored-by: Thomas Arts <thomas.arts@quviq.com>

* Update src/leveled_ebloom.erl

Co-authored-by: Thomas Arts <thomas.arts@quviq.com>

* Update following review

Also fix code coverage issues

* Update src/leveled_sst.erl

Co-authored-by: Thomas Arts <thomas.arts@quviq.com>

---------

Co-authored-by: Thomas Arts <thomas.arts@quviq.com>
This commit is contained in:
Martin Sumner 2024-09-05 15:08:05 +01:00 committed by GitHub
parent 30ec9214ac
commit 5db277b82d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 311 additions and 176 deletions

View file

@ -2002,6 +2002,9 @@ format_status_test() ->
?assertMatch(redacted, ST#state.levelzero_cache),
?assertMatch(redacted, ST#state.levelzero_index),
?assertMatch(redacted, ST#state.levelzero_astree),
NormStatus = format_status(#{reason => normal, state => S}),
NST = maps:get(state, NormStatus),
?assert(is_integer(array:size(element(2, NST#state.manifest)))),
clean_testdir(RootPath).
close_no_crash_test_() ->