Commit graph

269 commits

Author SHA1 Message Date
Martin Sumner
833c7a80cb corrected test
differing object was in wrong bucket
2017-06-19 13:11:43 +01:00
martinsumner
c586b78f45 Initial code with busted ct test
Initiat comparison made betwene trees externally - but ct test is bust.
2017-06-19 11:36:57 +01:00
martinsumner
f5dd154cee Rename hashtree query
Naming is now confusing now we have TicTac Trees.  This query builds a
list of keys and hashes not a tree - so it was misleading anyaway.  Now
renamed hashlist_query.
2017-06-16 12:38:59 +01:00
Martin Sumner
7642aac2cc Change Riak object hash approach
Change the riak object hash being kept in the metadata, to being a hash
of the vector clock
2017-06-16 10:14:24 +01:00
martinsumner
94f3e036ea Add journal compaction testing 2017-06-06 16:30:02 +01:00
Martin Sumner
15c52ae118 Change default compaction settings
Need to allow specific settings to be passed into unit tests.

Also, too much journal compaction may lead to intermittent failures on
the basic_SUITE space_clear_on_delete test.  think this is because
there are less “deletes” to reload in on startup to trigger the cascade
down and clear up?
2017-06-02 08:37:57 +01:00
Martin Sumner
96a548e17a Change tests - binary keys
the new code requires bucket listing to be on binary keys not just
binary buckets.  As this is only intended for use within Riak (where
all keys are buckets are binaries), this constraint seems OK.

A test needed changing to ensure it had a binary key in the bucket.
2017-05-23 15:54:11 +01:00
martinsumner
0d8ab0899e Add test for is_empty
Bucket listing didn't care if keys were active - now does.
2017-05-23 11:59:44 +01:00
martinsumner
a052edaea0 Add volume test information
Volume testing with AAE rebuilds
2017-04-24 11:24:14 +01:00
martinsumner
fbb4879d81 Change fold_heads to do basic Journal presence check
This at least checks the file is present, and the Key exists in the
index of that file.  If the value is corrupt it will be removed by
compation, and then this will fail (unless the file is never compacted).

TODO: resolve issus of files which are corrupt - but never compacted
- a job for backup?
2017-04-21 15:55:03 +01:00
Martin Sumner
fa9daf8696 Correct async fold
fold objects which snaps in the fold was implemented incorrectly - it
took information from the LedgeCache at the point of the request, not
at the point of the fold.  So the LedgerCache SQN may have been
surpassed in the Penciller by the time the fold was called.
2017-04-17 23:01:55 +01:00
martinsumner
400f65f557 Switch to binary metadata
Trya nd maintain binary format when stored in Ledger so less
swapping/changing as added and removed.
2017-04-04 10:02:35 +00:00
Martin Sumner
1335b11d82 Merge pull request #55 from martinsumner/mas-docs2i
Mas docs2i
2017-03-24 16:09:17 +00:00
martinsumner
5e1482d71b 2i query test results
Write-up of 2i query test results
2017-03-24 15:20:03 +00:00
martinsumner
97312e1356 Test to reflect vclock not binary 2017-03-21 18:28:28 +00:00
Martin Sumner
a2c164e1dd Switch to binary vclock 2017-03-20 22:44:19 +00:00
Martin Sumner
994d9758b0 Object now 4 bytes larger
due to addition of key changes length
2017-03-20 16:21:49 +00:00
martinsumner
32cdf26ffb Correct to catch new response to snapshot request 2017-03-06 19:37:43 +00:00
martinsumner
db73ecf26c Add bitcask volume test results 2017-02-28 21:19:46 +00:00
martinsumner
842639a90a Add smaller/larger object size test results 2017-02-27 23:04:31 +00:00
martinsumner
41e5686a66 Test coverage - missing 1% 2017-02-26 20:52:40 +00:00
martinsumner
27a819c04e Chart fix 2017-02-26 12:52:56 +00:00
martinsumner
c45e50e195 Update volume testing results 2017-02-26 12:50:16 +00:00
martinsumner
7ee26fc78c Filename change 2017-02-22 18:25:54 +00:00
martinsumner
5ad1e39f69 Alter volume charts to fixed axis with clearer delta 2017-02-22 18:22:08 +00:00
martinsumner
ec021d73e3 Fixed axis charts
Also compariosn betwen 6 hour and 8 hour tests
2017-02-21 12:10:05 +00:00
martinsumner
ede93c5bb6 Putting charts on fixed axis 2017-02-21 11:49:06 +00:00
martinsumner
79f927c4fc Change charts to be fixed axis comparison
Also removed 99.9th percentile
2017-02-21 11:29:57 +00:00
martinsumner
0bbcac0460 Add more volume test details 2017-02-20 12:06:24 +00:00
martinsumner
ea9a5fab26 More volume results 2017-02-08 17:01:27 +00:00
martinsumner
c533e829ff Add charts from Riak cluster testing 2017-02-02 12:57:50 +00:00
martinsumner
946edaef30 Initial draft of volume test writeup 2017-02-02 12:25:59 +00:00
martinsumner
0ddaaf9ac3 Stopped unnecessary seek for last_key
When rolling we already know the last_key - no need to seek for it on
startup.

The time it takes for this seek needs to be considered with regards to
startup time.  Can we do without knowing lastkey?
2016-12-22 19:51:39 +00:00
martinsumner
9e28287231 Resolve failing recovery test
Now passing consistently with a number of different corruptions catered
for (including corruption of the Tag in the Inker Key)
2016-12-16 23:18:55 +00:00
martinsumner
3417baa3b8 Simple test
To try and pinpoint any issue with _int index (as seen in Riak
integrtaion testing)
2016-12-02 17:39:28 +00:00
martinsumner
b62b38caa6 Tidy up test at end
May cause issues otherwise if there is an attempt to corrupt the journal
running in another bookie
2016-11-28 23:00:12 +00:00
martinsumner
e8c1d39df9 Switch to binary format Riak object
Initial change to try and test assuming that leveled received the binary
format of Riak objects (and parses that for metadata).
2016-11-28 22:26:09 +00:00
martinsumner
45f7fcb41a Datasync added for riak_sync strategy 2016-11-27 00:16:49 +00:00
martinsumner
196c807b5e Pass through sync_strategy
Allow to switch for Riak to use o_sync as the sync flag rather than sync
2016-11-25 17:41:08 +00:00
martinsumner
51dbad95c0 Change FoldBucketsFun to take just bucket
FoldBucketsFun does not take keys should be a 2-arity function (Bucket,
Acc).
2016-11-21 14:12:17 +00:00
martinsumner
0f7e421371 Add destruction
Allow a store to be cleared out and destroyed
2016-11-21 12:34:40 +00:00
martinsumner
386d40928b Fast List Buckets
Copied the technique from HanoiDB to speed up list buckets.
2016-11-20 21:21:31 +00:00
martinsumner
427b97873c Change FoldKeysFun 2016-11-18 17:58:43 +00:00
martinsumner
57727e74f9 D'oh 2016-11-18 17:14:01 +00:00
martinsumner
f5944f2466 Change test foler to return (Bucket, Key} as expeceted 2016-11-18 16:52:53 +00:00
martinsumner
ec18f9ab4c Uncomment test 2016-11-18 16:34:16 +00:00
martinsumner
6684e8e1d3 Refine query to accept fold functions
Need to be able to pass external fold functions into different queries,
to work as a Riak backend
2016-11-18 15:53:22 +00:00
martinsumner
ac223ced68 Add FoldKeysFun
Add the capability to pass FoldKeysFun into the index_query to allow for
compatability with riak backend requirements.
2016-11-18 11:53:14 +00:00
martinsumner
e3004faa6e Add support for folding keys on a single bucket 2016-11-17 15:55:29 +00:00
Martin Sumner
67a9f5bae6 Initial Load Tests with results
Initial tests run comparing throughput when first populating and then
loading data into levelled and eleveledb.

The tests were run in series, populating first and then loading.  The
population tests were run again in-between to try and add a roughly
even underlying volume into the stores.

The initial tests were run on on a quad core iMac with 8GB of RAM and a
fusion drive.  Due to the limited footprint of the hardware, the number
of concurrent database instances was reduced to 12, rather than the 32
in the off0the-shelf leveldb test.
2016-11-17 08:20:17 +00:00