Hi Sergi,
we are planning to on-board Ignite persistence store and since we are having
huge volume of data we need to have page compression. After checking few
links I would like to confirm that does any critical issue still pending or
do we need to wait for specific version to release? It would
Hello Igniters!
I've implemented compression of WAL page snapshot records. Ticket [1], PR
[2]. I've used page compression module implemented by Sergi Vladykin for
page store.
To configure WAL page records compression there are 2 properties added to
DataStorageConfiguration: walPageCompression
Denis,
See inline.
пн, 19 нояб. 2018 г. в 20:17, Denis Magda :
> Hi Sergi,
>
> Didn't know you were cooking this dish in the background ) Excellent. Just
> to be sure, that's part of this IEP, right?
>
>
Hi Sergi,
Didn't know you were cooking this dish in the background ) Excellent. Just
to be sure, that's part of this IEP, right?
https://cwiki.apache.org/confluence/display/IGNITE/IEP-20%3A+Data+Compression+in+Ignite#IEP-20:DataCompressioninIgnite-Withoutin-memorycompression
Since we can
Ilya,
Zstd itself has default compression level 3. I just used that number to be
consistent with the library defaults.
I will check if there is a significant difference in performance.
Sergi
пн, 19 нояб. 2018 г. в 14:59, Ilya Kasnacheev :
> Hello!
>
> You have zstd default level of 3. In my
Hello!
You have zstd default level of 3. In my tests, zstd usually performed much
better with compression level 2. Please consider.
I admire your effort!
Regards,
--
Ilya Kasnacheev
пн, 19 нояб. 2018 г. в 14:02, Sergi Vladykin :
> Right now the functionality has nothing to do with WAL, but
Right now the functionality has nothing to do with WAL, but your idea
definitely makes sense and worth being implemented as a next step.
Sergi
пн, 19 нояб. 2018 г. в 13:58, Andrey Mashenkov :
> Hi Sergi,
>
> It is not clear for me will your changes affect PageSnapshot WAL record.
> Is it
Hi Sergi,
It is not clear for me will your changes affect PageSnapshot WAL record.
Is it possible to add compression support for PageSnapshot WAL record as
well, to reduce WAL size?
Thanks.
On Mon, Nov 19, 2018 at 1:01 PM Sergi Vladykin
wrote:
> Folks,
>
> I've implemented page compression
Folks,
I've implemented page compression for persistent store and going to merge
it to master.
https://github.com/apache/ignite/pull/5200
Some design notes:
It employs "hole punching" approach, it means that the pages are kept
uncompressed in memory,
but when they get written to disk, they
