Commit graph

7 commits

Author SHA1 Message Date
martinsumner
254183369e CDB - switch to gen_fsm
The CDB file management server has distinct states, and was growing case
logic to prevent certain messages from being handled in ceratin states,
and to handle different messages differently.  So this has now been
converted to a gen_fsm.

As part of resolving this, the space_clear_ondelete test has been
completed, and completing this revealed that the Penciller could not
cope with a change which emptied the ledger.  So a series of changes has
been handled to allow it to smoothly progress to an empty manifest.
2016-10-26 20:39:16 +01:00
martinsumner
c78b5bca7d Basement Tombstones
Further progress towards the tidying up of basement tombstones in the
Ledger, with support added for key-listing to help with testing (and as
a potentially required feature).

The test is incomplete, but committing at this stage as the last commit
broke some tests (within the test code).

There are some outstanding questions about the handling of tombstones in
the Journal during compaction.  There exists a condition whereby values
could return if a recent journal is compacted and tombstones are removed
(as they are no longer present), but older journals have not been
compacted.  Now on stop/start - if the Ledger is wiped the removal of
the keys will be forgotten but the original PUTs would still remain.

The safest thing maybe to have rule that tombstones are never deleted
from the Inker's Journal - and accept the build-up of garbage.  Or there
could be an addition to the compaction process that checks back through
all the inker files to check that the Key of a tombstone is not present
in the past, before it is removed in the compaction.
2016-10-23 22:45:43 +01:00
martinsumner
e9c568a8b3 Test fix-up
There was a test that failed to close down a bookie and that caused some
issues.  The issues are double-reoslved, the close down was tidied as
well as the forgotten close being added back in.

There is some generla tidy around in anticipation of TTL support.
2016-10-21 21:26:28 +01:00
martinsumner
b2089baa1e Correct tombstone handling
Prepare SFT files for handling tombstones correctly (without expiry
dates).

Also some work as it can be seen from tests that some SFT files ar enot
be cleared out correctly.  Pausing before trying t clear out the fles to
experiment and trial the possibility that there is a timing issue.
2016-10-21 15:21:37 +01:00
martinsumner
0324edd6f6 Rotating object tests
Recent fixes have been made to problems associated with rapidly changing
objexts especially on re-opening of the bookie.  Test of rotating
objects from both an index query and a fetch perspective added to better
detect such issues in the future.
2016-10-20 12:16:17 +01:00
martinsumner
f16f71ae81 Revert ominshambles performance refactoring
To try and improve performance index entries had been removed from the
Ledger Cache, and a shadow list of the LedgerCache (in SQN order) was
kept to avoid gb_trees:to_list on push_mem.

This did not go well.  The issue was that ets does not deal with
duplicate keys in the list when inserting (it will only insert one, but
it is not clear which one).

This has been reverted back out.

The ETS parameters have been changed to [set, private].  It is not used
as an iterator, and is no longer passed out of the process (the
memtable_copy is sent instead).  This also avoids the tab2list function
being called.
2016-10-19 00:10:48 +01:00
martinsumner
3e475f46e8 Support for 2i query part1
Added basic support for 2i query.  This involved some refactoring of the
test code to share functions between suites.

There is sill a need for a Part 2 as no tests currently cover removal of
index entries.
2016-10-18 01:59:18 +01:00