umgeher/leveled
0
Fork 0
Code Issues Pull requests Releases Activity Actions

Add ZSTD compression (#430)

Browse source 
* Add support for zstd and split compression

Add support for using zstd as an alternative to native, lz4.

Upgrade lz4 to v1.9.4 (with ARM enhancements).

Allow for split compression algorithms - i.e. use native on journal, but lz4 on ledger.

* Switch to AdRoll zstd

Development appears to be active and ongoing.  No issues running on different linux flavours.

* Use realistic bucket name

* Update README.md

* Switch branch

* Add comment following review
This commit is contained in:
Martin Sumner 2024-01-23 16:25:03 +00:00 • committed by GitHub
parent c294570bce
commit 999ce8ba5b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 156 additions and 73 deletions
Download patch file Download diff file Expand all files Collapse all files

16
priv/leveled.schema
View file

@ -43,13 +43,23 @@
]}.
%% @doc Compression method
%% Can be lz4 or native (which will use the Erlang native zlib compression)
%% within term_to_binary
%% Can be lz4, zstd or native (which will use the Erlang native zlib
%% compression) within term_to_binary
{mapping, "leveled.compression_method", "leveled.compression_method", [
{datatype, {enum, [native, lz4, none]}},
{datatype, {enum, [native, lz4, zstd, none]}},
{default, native}
]}.
%% @doc Ledger compression
%% If an alternative compression method is preferred specifically for the
%% ledger, it can be specified here. Default is as_store - use whatever method
%% has been defined in leveled.compression.method. Alternatives are native,
%% lz4, ztsd and none
{mapping, "leveled.ledger_compression", "leveled.ledger_compression", [
{datatype, {enum, [as_store, native, lz4, zstd, none]}},
{default, as_store}
]}.
%% @doc Compression point
%% The point at which compression is applied to the Journal (the Ledger is
%% always compressed). Use on_receipt or on_compact. on_compact is suitable