Commit graph

1152 commits

Author SHA1 Message Date
Martin Sumner
dd20132892 Add test with fold_heads
Build the AAE tree equally using fold_heads.  This is a pre-cursor to running this within Riak.

In part this leans on some of the work done to improve standard Riak AAE with leveled.  When rebuilding the standard AAE store only the head is required, and so this process was switched in riak_kv_sweeper to make a fold_heads request if supported by the backend.

The head response is a proxy object, which when loaded into a riak_object will allow for access to object metadata, but will use the passed function if access to object contents is requested.
2017-08-05 16:43:03 +01:00
Martin Sumner
22709abcd6 Buit what if build initiated during fold 2017-08-04 18:35:16 +01:00
Martin Sumner
bea40ffc72 More info on hashtree locks 2017-08-04 18:30:29 +01:00
Martin Sumner
76692ae589 The problem of the lock
Still need to investigate behaviour under get_lock()
2017-08-04 16:30:25 +01:00
Martin Sumner
b99539fc8a Add most recent thoughts on implementation
Especially how to manage alternate backends to leveled
2017-08-04 16:19:40 +01:00
Martin Sumner
ff5e85700d Update gitignore 2017-08-04 15:32:25 +01:00
Martin Sumner
f9b61fb5e7 Minor edit 2017-08-01 14:01:41 +01:00
Martin Sumner
82a1619846 Mini edit 2017-08-01 13:25:50 +01:00
Martin Sumner
a046666e4a Add further stats 2017-08-01 13:23:10 +01:00
Martin Sumner
676bf12dc3 Add figures on rate of list_keys operations 2017-08-01 13:16:49 +01:00
Martin Sumner
6107ac1748 Add response time comparison for listkeys 2017-08-01 13:04:05 +01:00
Martin Sumner
66d85efe62 Switch back to pure markdown
But image still appears to have lost clarity
2017-08-01 12:57:10 +01:00
Martin Sumner
48fd2c76f0 Use screenshot for clarity
Detail lost when saving image direct from google sheets
2017-08-01 12:55:09 +01:00
Martin Sumner
b9139b3116 Sizing experiment 2017-08-01 12:50:53 +01:00
Martin Sumner
43547cff41 Experiment with image insertion and sizing 2017-08-01 12:50:02 +01:00
Martin Sumner
a8c3403a9d Experiment with image sizing 2017-08-01 12:47:52 +01:00
Martin Sumner
89b5c43fd3 Add volume chart 2017-08-01 12:45:01 +01:00
Martin Sumner
a85c853fe0 Merge branch 'master' into mas-aae-riakimplnotes 2017-08-01 12:28:37 +01:00
Martin Sumner
714adf279a Merge pull request #81 from project-fifo/random
Add compatibility for old and new random / rand functions
2017-08-01 12:24:23 +01:00
Martin Sumner
968161a36d Initial start to write-up
Charts etc still to add
2017-08-01 11:19:21 +01:00
Heinz N. Gies
b319386210 Fix strong_rand to rand_bytes 2017-08-01 11:55:05 +02:00
Heinz N. Gies
bbe763514b Remove uniform_s/2 from old random code 2017-08-01 11:37:18 +02:00
Heinz N. Gies
38e9b0e80a Add missing uniform/0 function 2017-08-01 11:24:12 +02:00
Heinz N. Gies
25389893cf Add compatibility for old and new random / rand functions 2017-08-01 11:24:12 +02:00
Martin Sumner
a49d3bbfec Merge pull request #84 from martinsumner/mas-aaeexpiry-i82
Add extra second to sleep
2017-08-01 00:14:48 +01:00
Martin Sumner
8748fef28c Add extra second to sleep
Sleep for just one more second to resolve intermittent failure
2017-08-01 00:14:31 +01:00
Martin Sumner
7f6f51efcd Merge pull request #80 from project-fifo/dialyzer
Fix dialyzer warnings for newer erlang releases
2017-07-31 21:42:50 +01:00
Heinz N. Gies
379e33ba84 Cleanup dialyzer errrors in leveled_bookie 2017-07-31 19:58:56 +02:00
Heinz N. Gies
8717d42ffe Cleanup dialyzer errrors in leveled_cdb 2017-07-31 19:55:09 +02:00
Heinz N. Gies
e8ed7954cc Cleanup dialyzer errrors in leveled_iclerk 2017-07-31 19:53:01 +02:00
Heinz N. Gies
eece253222 Cleanup most dialyzer errrors in leveled_inker 2017-07-31 19:47:58 +02:00
Heinz N. Gies
44fd603474 Cleanup dialyzer errrors in leveled_pclerk 2017-07-31 19:41:26 +02:00
Heinz N. Gies
369bdece5f Cleanup dialyzer errrors in leveled_penciller 2017-07-31 19:39:40 +02:00
Heinz N. Gies
858ee9a915 Cleanup dialyzer errrors in leveled_pmanifest 2017-07-31 19:32:06 +02:00
Heinz N. Gies
5e6df539cb Cleanup dialyzer errrors in leveled_sst 2017-07-31 19:30:29 +02:00
Martin Sumner
4ccd67a3c2 Merge pull request #79 from martinsumner/mas-tictac
TicTac Merkle Tree for AAE
2017-07-21 11:10:46 +01:00
Martin Sumner
2297c0fab1 Change snap_async to snap
As currently implemented in riak_core branch
2017-07-21 10:27:43 +01:00
Martin Sumner
9c4910fe26 Edits to AAE write-up
Further re-read before push
2017-07-21 10:21:54 +01:00
Martin Sumner
2ad1ac0baf node_worker_pool
Having a separate vnode_worker_pool wouldn't reoslve he parallelism issue obviously!  Need a node_worker_pool instead.
2017-07-13 17:40:06 +01:00
martinsumner
cb5f09496f Update thoughts on concurrency management
After reviewing the code it looks easier to have  separate pool of vnode
workers to manage concurrency, rather than trying to alter the coverage
FSM itself.  This will make it easier to adjust for different backend
capabilities (i.e. worst case it would fallback to unthrottled).
2017-07-12 14:50:38 +01:00
martinsumner
2adf60e974 Look at super-async folds and rollingcoverage_fsm 2017-07-11 19:48:12 +01:00
Martin Sumner
65fd029ca6 typo - backlist/blacklist 2017-07-11 12:25:06 +01:00
martinsumner
80fd2615f6 Implement blacklist/whitelist
Change from the all/whitelist ebhavior to the blacklist/whitelist
behaviour documented in the write-up
2017-07-11 11:44:01 +01:00
martinsumner
7c86de2491 Clarification on NRT query 2017-07-11 10:29:50 +01:00
martinsumner
e4a2a8ecea Stuff on read repair 2017-07-11 10:28:34 +01:00
martinsumner
b5ca2a8541 Word wrangling 2017-07-10 16:47:39 +01:00
martinsumner
185d4e9747 Notes on Riak Implementation options 2017-07-10 16:36:01 +01:00
martinsumner
03d0b913d5 Add new background section 2017-07-10 11:45:35 +01:00
Martin Sumner
3ed41a0e74 Merge pull request #78 from martinsumner/rdb/review/mas-tictac
Some proposed edits
2017-07-10 09:50:40 +01:00
martinsumner
e7c322d956 Clarify pointer to research 2017-07-10 09:49:20 +01:00