Commit graph

1328 commits

Author SHA1 Message Date
Martin Sumner
bfddb53e31
Merge pull request #221 from martinsumner/mas-i219-tuplebuckets
Deprecate use of Bucket as a constraint in indexfold
2018-12-05 20:47:40 +00:00
Martin Sumner
835955adb9 Deprecate use of Bucket as a constraint in indexfold
Must also give a StartKey.  Otherwise there may be conflict when tuplebuckets are used
2018-12-05 20:46:17 +00:00
Martin Sumner
f415d22ef0
Merge pull request #220 from martinsumner/mas-i157-indexttl
Add test of index expiry
2018-12-05 15:42:43 +00:00
Martin Sumner
510994233e Add check that index disappears
Check I0 count goes down when that index is removed
2018-12-05 15:42:21 +00:00
Martin Sumner
cf1fcaeef2 Add test of index expiry
To show how this works, and prove that it does work thta way.

Test may require adjusting if tested on a slow node (e.g. reduce KeyCount or increase TTL)
2018-12-05 15:18:20 +00:00
Martin Sumner
150fcf15d6
Merge pull request #217 from martinsumner/mas-i216-nomod
Remove incorrect mod entry
2018-12-05 09:31:57 +00:00
Martin Sumner
d438c29200
Merge pull request #218 from martinsumner/mas-uw-configurableloglevel
Mas uw configurableloglevel
2018-12-05 09:31:31 +00:00
Martin Sumner
9735661d98 Typos as per PR comments 2018-12-05 09:31:03 +00:00
Martin Sumner
578a9f88e0 Support for log settings at startup
Both log level and forced_logs.  Allows for log_level to be changed at startup ad runtime.  Also allow for a list of forced logs, so if log_level is set > info, individual info logs can be forced to be seen (such as to see stats logs).
2018-12-05 00:17:39 +00:00
Martin Sumner
968be4c4be Remove incorrect mod entry 2018-12-04 20:15:29 +00:00
Ulf Wiger
88c233f911 dynamically check log_level 2018-11-30 10:38:03 +01:00
Martin Sumner
aacc2adc54
Merge pull request #212 from martinsumner/mas-i211-breakingfolds
Mas i211 breakingfolds
2018-11-24 10:33:50 +00:00
Martin Sumner
6d2d0694e3 Reverse necessary on bucket list
The function should see the buckets in order, so it accumulates in such a way to reverse the order - it makes sense that the outcome should be in reverse.
2018-11-23 19:03:24 +00:00
Martin Sumner
a9aa23bc9c Bucket list
update the docs to advertise throw capability.  Test it for bucket list (and fix ordering of bucket lists)
2018-11-23 18:56:30 +00:00
Martin Sumner
ef2a8c62af Add capability to exit a head or object fold with a throw
This allows for all fold functions to throw an exception to exit out of a fold with all dependencies still closed down as expected.

This was previously available for key folds, which was necessary for the folds to work in Riak (as max_results in index queries depends one xiting the fold with an exception).  This change now adds a ct test, and adds support for head folds, object folds (key order) and object folds (sqn order)
2018-11-23 16:00:11 +00:00
Martin Sumner
da5abb4e4c
Merge pull request #210 from martinsumner/mas-i209-rangeissue
Mas i209 rangeissue
2018-11-15 08:56:25 +00:00
Martin Sumner
fab11bc2d2 Update comments to assist with clarity 2018-11-15 08:54:06 +00:00
Martin Sumner
a12931b430 Add comments 2018-11-15 01:06:37 +00:00
Martin Sumner
b571be9e43 Check that seglist-filtered keys are actually in range 2018-11-15 00:00:18 +00:00
Martin Sumner
3c02901ab3
Merge pull request #208 from martinsumner/mas-i207-althashkey
Mas i207 althashkey
2018-11-09 19:13:49 +00:00
Martin Sumner
0aab208e9b Additional export
Export changes required to support kv_index_tictactree.  This will call tictac_hahs, but also needs to know hash of key used in bxor - so that it can control alterations.
2018-11-09 16:05:48 +00:00
Martin Sumner
bcdf553223 Export function 2018-11-09 15:53:15 +00:00
Martin Sumner
a7773b148d Split hash - seperate key has for bxor with value 2018-11-09 14:51:38 +00:00
Martin Sumner
4756d402a0
Merge pull request #206 from martinsumner/mas-i204-largeseglists
Mas i204 largeseglists
2018-11-07 22:57:25 +00:00
Martin Sumner
2afb160a12 Add test - large seglist 2018-11-07 21:35:21 +00:00
Martin Sumner
ea7aa3086d Refactor membership check
To change to set membership when size beyond threshold
2018-11-07 17:43:26 +00:00
Martin Sumner
aaf10135a1
Merge pull request #205 from martinsumner/mas-xxx-timingtests
Mas xxx timingtests
2018-11-07 17:03:26 +00:00
Martin Sumner
787b02e4d8 Switch to leveled_rand for compatibility 2018-11-07 17:01:51 +00:00
Martin Sumner
6f4f1b9560 Add membership timing tests - sets vs lists 2018-11-07 16:57:00 +00:00
Martin Sumner
62f0351906
Merge pull request #203 from martinsumner/mas-i202-fixuptypes
Tidy up unexported types
2018-11-05 16:18:29 +00:00
Martin Sumner
174a40aab2 Tidy up unexported types
also re:mp may not be exported in R16
2018-11-05 16:02:19 +00:00
Martin Sumner
d3c9104336
Merge pull request #201 from martinsumner/mas-i189-tuplebuckets-take2
Mas i189 tuplebuckets take2
2018-11-05 12:22:36 +00:00
Martin Sumner
e9fb893ea0 Check segment is as expected with tuplebuckets
In head_only mode
2018-11-05 10:31:15 +00:00
Martin Sumner
e72a946f43 TupleBuckets in Riak objects
Adds support with test for tuplebuckets in Riak keys.

This exposed that there was no filter using the seglist on the in-mmemory keys.  This means that if there is no filter applied in the fold_function, many false positives may emerge.

This is probably not a big performance benefit (and indeed for performance it may be better to apply during the leveled_pmem:merge_trees).

Some thought still required as to what is more likely to contribute to future bugs: an extra location using the hash matching found in leveled_sst, or the extra results in the query.
2018-11-05 01:21:08 +00:00
Martin Sumner
37cdb22979
Merge pull request #200 from martinsumner/mas-i198-recentaae
Mas i198 recentaae
2018-11-04 20:51:13 +00:00
Martin Sumner
4b8f493fd6 Allow compariosn with empty binaries
If trees are non-existent, then fetch_root may return an empty binary (in tictac aae) - still need to compare
2018-11-02 17:34:58 +00:00
Martin Sumner
2eec8a5378 MaxCount monitoring and responding
Stop issue of {no_more_keys, Acc} being passed on fold over list of ranges to next range (and blowing up)
2018-11-01 23:40:28 +00:00
Martin Sumner
dc84eabe0c Revert "Temp log"
This reverts commit 2b57ff831c.
2018-11-01 20:16:08 +00:00
Martin Sumner
c126872fab Revert "More temp logs"
This reverts commit 7e959f1353.
2018-11-01 20:16:04 +00:00
Martin Sumner
7e959f1353 More temp logs 2018-11-01 20:04:22 +00:00
Martin Sumner
2b57ff831c Temp log 2018-11-01 19:58:32 +00:00
Martin Sumner
71fa1447e0 Allow for all keys head folds to used modifed range
This helps with kv_index_tictcatree with the leveled_so backend.  Now this cna do folds over ranges of keys with modified filters (as folds over ranges of keys must go over lal keys if the backend is segment_ordered)
2018-11-01 17:30:18 +00:00
Martin Sumner
aa123a80a7 Allow for backwards/forwards compatibility in specs 2018-11-01 12:40:24 +00:00
Martin Sumner
acd5db779a Merge branch 'master' into mas-i198-recentaae 2018-11-01 10:59:05 +00:00
Martin Sumner
8e752ce93c
Merge pull request #199 from martinsumner/mas-i196-expandseglist
Mas i196 expandseglist
2018-11-01 10:52:29 +00:00
Martin Sumner
f77dc8c3a5 Add object_spec type
Initial refactor to prepare to allow for a new version object_spec type that will support LMD being promoted as an accessible item.
2018-11-01 10:41:46 +00:00
Martin Sumner
376407fa66 Extend testing
In particular discover that after objects are modifed (outside of the LMD window) - they no longer appear in results for queries in the LMD window.
2018-10-31 21:37:53 +00:00
Martin Sumner
62f1302305 Add test comparing filter with non-filter
How do timings differ - does the SST filter accelerate?  The answer appears to be yes - the filtered version takes half the time.
2018-10-31 18:34:27 +00:00
Martin Sumner
19bfe48564 Initial ct test
Which exposed it wasn't working.  If there is no segment list passed - just a modification filter, you don't need to check the position list (as checking the position list returns an empty position so sipping all the matching results!)
2018-10-31 16:35:53 +00:00
Martin Sumner
aaccd09a98 Allow for setting max_keys to wrap Acc
Acc in response is now of form {Reason, Acc} not just Acc so that the application can understand the reason for the results ending - and take appropriate action (e.g. restart again from the LastKey to return more results).
2018-10-31 14:22:28 +00:00