Commit graph

1389 commits

Author SHA1 Message Date
Martin Sumner
2c1503b6b5
Merge pull request #250 from martinsumner/mas-i249-iclerkshutdown
Mas i249 iclerkshutdown
2019-01-25 09:54:14 +00:00
Martin Sumner
00a59f4f8f
Merge branch 'mas-qc-inkercompaction' into mas-i249-iclerkshutdown 2019-01-25 09:53:56 +00:00
Martin Sumner
0333604fd9 Change to cast in inker/iclerk interaction
This allows for leveled_iclerk:clerk_stop to be a sync call, so that files will only be closed once the iclerk has stopped.  This is designed ot prevent iclerk crashes during shutdowns when files it is depnding on are closed mid shutdown.
2019-01-24 21:32:54 +00:00
Martin Sumner
28d0aef5fe Make check that compaction not ongoing before accepting new compaction
Respond 'busy' if compaction is ongoing
2019-01-24 15:46:17 +00:00
Martin Sumner
c9bf43953b Expect TS in snapshot references from manifest
Add type to prevent re-occurence.  This is also detected by failure in eqc tests.
2019-01-24 14:32:01 +00:00
Martin Sumner
a13a6ae45f Updated model
This has inappropriate default parameter changes.
2019-01-22 12:53:31 +00:00
Martin Sumner
b713ce60a8 Initial eqc setup 2019-01-21 10:51:07 +00:00
Martin Sumner
67104c85c1
Merge pull request #248 from martinsumner/mas-i246-dotdotwhy
Change file references in unit tests
2019-01-17 21:03:20 +00:00
Martin Sumner
7f08fd5a68 Change file references in unit tests
Write into test folder within the repo, not outside of it.  Try and resolve issues wiht make test in riak
2019-01-17 21:02:29 +00:00
Martin Sumner
3e75eb3647
Merge pull request #247 from martinsumner/mas-i246-maketestfail
Remove pointless test
2019-01-17 11:31:17 +00:00
Martin Sumner
86128ddf56 Remove pointless test
Was testing a test admin function for coverage reasons
2019-01-17 11:30:50 +00:00
Martin Sumner
267e103e58
Merge pull request #245 from martinsumner/mas-i244-startupcachesizes
Mas i244 startupcachesizes
2019-01-14 18:48:39 +00:00
Martin Sumner
a4d89ad6d1 Add log of higher than expected ratio of cache sizes
Warn at startup if this ratio is high.  Not sure how snapshots will perform if there are a lot of ledger cache sin the list.  However, it should still work.  basic_SUITE/load_count test intended to demonstrate that a large ratio is still functional
2019-01-14 16:11:04 +00:00
Martin Sumner
c060c0e41d Handle L0 cache being full
A test thta will cause leveled to crash due to a low cache size being set - but protect against this (as well as the general scenario of the cache being full).

There could be a potential case where a L0 file present (post pending) without work backlog being set.  In this case we want to roll the level zero to memory, but don't accept the cache update if the L0 cache is already full.
2019-01-14 12:27:51 +00:00
Martin Sumner
d5a9f2e8b7
Merge pull request #242 from martinsumner/mas-i241-eunitfailure
Creating the file is unnecessary
2018-12-18 16:27:38 +00:00
Martin Sumner
e77451f60b Creating the file is unnecessary 2018-12-18 16:26:48 +00:00
Martin Sumner
0a43d696f9
Merge pull request #240 from martinsumner/mas-i239-schemaalign
Correct defaults
2018-12-18 11:23:27 +00:00
Martin Sumner
32ea4380b2 correct defaults 2018-12-18 11:22:25 +00:00
Martin Sumner
420322c14f
Merge pull request #238 from martinsumner/mas-innn-schemamixup
Typo
2018-12-14 22:43:46 +00:00
Martin Sumner
8ccb985775 Typo 2018-12-14 22:43:21 +00:00
Martin Sumner
d246fd363e
Merge pull request #237 from martinsumner/mas-i233-multibackend
Mas i233 multibackend
2018-12-14 22:00:03 +00:00
Martin Sumner
5b578ea2bb Add new snapshot timeout to multi-backend 2018-12-14 19:57:32 +00:00
Martin Sumner
e6d868f8cd Merge branch 'master' into mas-i233-multibackend 2018-12-14 19:55:34 +00:00
Martin Sumner
534fcdb1b6
Merge pull request #236 from martinsumner/mas-i234-snapshottimeout
Mas i234 snapshottimeout
2018-12-14 19:49:06 +00:00
Martin Sumner
ceea196cc0 Update priv/leveled.schema
The snapshottimeouts would not normally eb changed - so make them hidden
2018-12-14 14:27:44 +00:00
Martin Sumner
ef068326a0 Update priv/leveled.schema 2018-12-14 14:13:44 +00:00
Martin Sumner
2741c46daa Add timeout for inker snapshots
So that they can also be released if they silently crash without closing neatly.
2018-12-14 13:53:36 +00:00
Martin Sumner
8bf36214e1 Make snapshot timeout configurable
Also increase defaults.  In riak snapshots may be used to fold over all objects, and with delays at the receiving end, this could take significant time
2018-12-14 11:23:04 +00:00
Martin Sumner
da17d6c9a4 Merge branch 'master' into mas-i233-multibackend 2018-12-14 10:01:52 +00:00
Martin Sumner
b344b7d827
Merge pull request #232 from martinsumner/mas-pr231-review
Mas pr231 review
2018-12-14 10:01:16 +00:00
Martin Sumner
45f62ffa56 Make all multi schema hidden 2018-12-13 16:24:33 +00:00
Martin Sumner
f211e587f7 Make multi_schema mainly hidden
As wiht other schemas - make the multi_backend schema mainly hidden
2018-12-12 16:16:47 +00:00
Martin Sumner
52c523dbab Add multi_backend schema 2018-12-12 09:56:06 +00:00
Martin Sumner
7a876066e2 Resolve coverage gaps
logs should always at least have a log_level defined which is in the ?LOG_LEVELS - so the final case of

`-is_active_level([]   , _, _) -> false.`

is unnecessary (even if a bad log level is passed in).

correct Bucket and Key definition to hit expected eunit test cases in leveled_pmem
2018-12-11 22:25:37 +00:00
Martin Sumner
672cfd4fcd Allow for run-time changes to log_level and forced_logs
Will not lead to immediate run time changes in SST or CDB logs.  These log settings will only change once the new files are re-written.

To completely change the log level - a restart of the store is necessary with new startup options.
2018-12-11 21:59:57 +00:00
Martin Sumner
6677f2e5c6 Push log update through to cdb/sst
Using the cdb_options and sst_options records
2018-12-11 20:42:00 +00:00
Martin Sumner
9ca6b499e1 Remove use of string rather than straddle OTP version
string functions were used in unit tetss only, and were replaceable with io_lib:format
2018-12-11 15:44:37 +00:00
Martin Sumner
90574122c9 Merge remote-tracking branch 'aeternity/uw-avoid-set_env' into mas-pr231-review 2018-12-10 18:33:23 +00:00
Ulf Wiger
d30ca16a17 store log opts in pdict + inherit at proc start 2018-12-10 16:09:11 +01:00
Martin Sumner
4387837279
Merge pull request #230 from martinsumner/mas-i229-loadrepeatedkey
Mas i229 loadrepeatedkey
2018-12-10 14:08:24 +00:00
Martin Sumner
f274d2a63a Tighten acceptable duration
even with cover, passes in 30s.
2018-12-10 13:23:39 +00:00
Martin Sumner
b38d845c3a Update src/leveled_pmem.erl
Simple change to check for presence of objetc in list before adding it:

```Head check took 124416 microseconds checking list of length 5000

Head check took 130114 microseconds checking list of length 5000

Check for presence of repeated objects

Head check took 1725 microseconds checking list of length 5```
2018-12-10 12:01:11 +00:00
Martin Sumner
e73f48a18b Add failing test
Test fails as fetching repeated object is too slow.

```Head check took 124301 microseconds checking list of length 5000

Head check took 112286 microseconds checking list of length 5000

Head check took 1336512 microseconds checking list of length 5

2018-12-10T11:54:41.342 B0013 <0.2459.0> Long running task took 260788 microseconds with task of type pcl_head

2018-12-10T11:54:41.618 B0013 <0.2459.0> Long running task took 276508 microseconds with task of type pcl_head

2018-12-10T11:54:41.894 B0013 <0.2459.0> Long running task took 275225 microseconds with task of type pcl_head

2018-12-10T11:54:42.173 B0013 <0.2459.0> Long running task took 278836 microseconds with task of type pcl_head

2018-12-10T11:54:42.477 B0013 <0.2459.0> Long running task took 304524 microseconds with task of type pcl_head```

It taks twice as long to check for one repeated object as it does to check for 5K non-repeated objects
2018-12-10 11:58:21 +00:00
Martin Sumner
401228c84e
Merge pull request #228 from martinsumner/mas-i223-dialyzer
Fix dialyzer warnings
2018-12-07 14:36:42 +00:00
Martin Sumner
4b4b774c0d Fix dialyzer warnings
Dialyzer got smarter in OTP 21 and spotted that the output type was wrong from tune_seglist
2018-12-07 14:36:18 +00:00
Martin Sumner
bb0d088aae
Update leveled.app.src
Switch release reference
2018-12-07 12:50:24 +00:00
Martin Sumner
7821aed6c6
Merge pull request #227 from martinsumner/mas-lz4ref
Change lz4 ref
2018-12-07 12:48:19 +00:00
Martin Sumner
c1b3e7c793 Change lz4 ref
there were no local modifications - so use tagged version of szktty (which is in hex)
2018-12-07 12:47:48 +00:00
Martin Sumner
43033a5e19
Merge pull request #226 from martinsumner/mas-appsrc
Mas appsrc
2018-12-07 12:29:18 +00:00
Martin Sumner
1fc73f9092 Update src/leveled.app.src 2018-12-07 12:26:41 +00:00