Make sure there is no change pending regardless of why maybe_roll_memory has been called.
Also, check that the manifest SQN has been incremented before accepting the change.
Conflict here would lead to data loss in the penciller, so extra safety is important.
These logs duplicate information being received from other logs, so reduced to debug.
The long running test needs to change with the LONG_RUNNING macro
Warn at startup if this ratio is high. Not sure how snapshots will perform if there are a lot of ledger cache sin the list. However, it should still work. basic_SUITE/load_count test intended to demonstrate that a large ratio is still functional
A test thta will cause leveled to crash due to a low cache size being set - but protect against this (as well as the general scenario of the cache being full).
There could be a potential case where a L0 file present (post pending) without work backlog being set. In this case we want to roll the level zero to memory, but don't accept the cache update if the L0 cache is already full.
logs should always at least have a log_level defined which is in the ?LOG_LEVELS - so the final case of
`-is_active_level([] , _, _) -> false.`
is unnecessary (even if a bad log level is passed in).
correct Bucket and Key definition to hit expected eunit test cases in leveled_pmem
Will not lead to immediate run time changes in SST or CDB logs. These log settings will only change once the new files are re-written.
To completely change the log level - a restart of the store is necessary with new startup options.
Both log level and forced_logs. Allows for log_level to be changed at startup ad runtime. Also allow for a list of forced logs, so if log_level is set > info, individual info logs can be forced to be seen (such as to see stats logs).
If ther are backups made to the same folder, need to remove any files from that folder that are not included in this backup.
Some initial testing, needs more.
Added a test with back-to-back backups. This caused issues with the empty CDB file it created (on opening, couldn't cope with last key of empty).
So now backup won't roll the active journal if it is empty.
Previouslythe tinybloom was used within the SST file as an extra check to remove false fetches.
However the SST already has a low FPR check in the slot_index. If the newebloom was used (which is no longer per slot, but per sst), this can be shared with the penciller and then the penciller could use it and avoid the message pass.
the message pass may be blocked by a 2i query or a slot fetch request for a merge. So this should make performance within the Penciller snappier.
This is as a result of taking sst_timings within a volume test - where there was an average of + 100microsecs for each level that was dropped down. Given the bloom/slot checks were < 20 microsecs - there seems to be some further delay.
The bloom is a binary of > 64 bytes - so passing it around should not require a copy.
Producing the list of all slots to try appeared to be expensive. In volume tests taking 150 - 250 microseconds per GET. Perhaps the list could be long (>1000), with a split and append, so not surprising.
Instead loop and count.
If wast retention period is undefined, then it should be ignored - and no waste retained (rather than retaining waste for 24 hours as at present).
This wasn't working anyway - as reopen reader didn't get the cdb options (which didn't have the waste path on anyway) - so waste would not eb retained if the file had been opened after a stop/start.
Discovered a bug with search ranges in leveled_tree - this was uncovered by an intermittently fialing 19.3 test.
Test case added and bug fixed. It was due to a fialure to use end_key passed causing issues with particular manifests and full bucket ranges.
As descibed in https://github.com/martinsumner/leveled/issues/92
Only the first fix was made.
Just to eb safe - archiving means renaming to another file with a different extension. Assumption is that renamed files cna be manually reaped if necessary.
If the clerk updates the manifest - it might not recognise changes to
the manifest made since the clerk took the manifest. So the penciller
must merge its view of the snapshots back in to the updated manifest
