c: arzamas...@mail.ru
>Subject: Re: ignite PureJavaCrc32 vs java.util.zip.CRC32 bench.
>Date: Tue, 21 Aug 2018 23:17:20 +0300
>
>Dmitriy, yes, you correctly understood my idea.
>
>вт, 21 авг. 2018 г. в 22:03, Vladimir Ozerov < voze...@gridgain.com >:
>
>> Gu
Dmitriy, yes, you correctly understood my idea.
вт, 21 авг. 2018 г. в 22:03, Vladimir Ozerov :
> Guys,
>
> Do we have any numbers for real workloads? Or at least Yardstick.
>
> вт, 21 авг. 2018 г. в 21:04, Dmitriy Pavlov :
>
> > Let us double-check current options. Maybe I'm mistaken about Alex
Guys,
Do we have any numbers for real workloads? Or at least Yardstick.
вт, 21 авг. 2018 г. в 21:04, Dmitriy Pavlov :
> Let us double-check current options. Maybe I'm mistaken about Alex
> Plehanov's idea.
>
> Let's introduce the following variables and functions:
> data - our block.
>
Let us double-check current options. Maybe I'm mistaken about Alex
Plehanov's idea.
Let's introduce the following variables and functions:
data - our block.
OldImpl(data) = X
NewImpl(data) = Y
NewImpl(data) ^ C = Y ^ C, where C is constant, ^ is bitwise XOR operation.
And NewImpl(data) ^ C = X =
Alex
In that case the data becomes in unpredictable state (mix of new and old
methods) and there's a chance that some partitions will be never touched
and never converted. It will force us always support old compression.
I suppose the explicit conversion that clearly say what is happening now
and
Sergey, converting data also force us to introduce some flag to WAL
segments/records or convert WAL segments too. WAL segments can be archived,
they also can be compressed.
IMO we better should not introduce any compatibility modes, left data as is
and always just convert crc value returned by
Dmitriy
Due to significant improvement and to reduce the number supported
modes/options would be good to convert the data at the moment of upgrade.
On Tue, Aug 21, 2018 at 12:03 AM, Dmitriy Setrakyan
wrote:
> Sergey, that was precisely my comment in the ticket:
>
> Can we add this option
Igniters,
let's come back to the idea from Alex Plehanov.
We can try to provide custom xor mask at the end of CRC calc. We can do a
compatible solution using fast implementation, can't we?
Sincerely,
Dmitriy Pavlov
вт, 21 авг. 2018 г. в 0:04, Dmitriy Setrakyan :
> Sergey, that was precisely
Sergey, that was precisely my comment in the ticket:
Can we add this option without breaking compatibility with previous
page/storage formats? If not, then this should support both implementation.
The default should be the new fastest implementation, but if we encounter
the older, slower one,
Hi Igniters
I suppose that'll break compatibility for LFS (PDS).
Do we plan to provide a migration guide w/o data loss for upgrade AI 2.x to
3.0?
On Mon, Aug 20, 2018 at 11:46 PM, Dmitriy Setrakyan
wrote:
> I commented in the ticket: https://issues.apache.org/
> jira/browse/IGNITE-9272
>
> It
I commented in the ticket: https://issues.apache.org/jira/browse/IGNITE-9272
It if can integrate it correctly, according to my comment, in 2.7 release,
it would be great. Otherwise, let's plan this change for 3.0 release.
D.
On Mon, Aug 20, 2018 at 3:50 AM, Eduard Shangareev <
Hi,
I have checked the benchmark and it shows great performance boost on my
laptop!
+1 for this change.
On Tue, Aug 14, 2018 at 9:01 PM Dmitriy Pavlov
wrote:
> Hi Evgeniy,
>
> Thank you. I see that the ticket is unassigned.
>
> Would you like to contribute PR to be macro-benchmarked with
Hi Evgeniy,
Thank you. I see that the ticket is unassigned.
Would you like to contribute PR to be macro-benchmarked with Ignite?
Sincerely,
Dmitriy Pavlov
вт, 14 авг. 2018 г. в 20:57, Евгений Станиловский
:
> I fill the ticket, bench code attached there.
>
I fill the ticket, bench code attached there.
https://issues.apache.org/jira/browse/IGNITE-9272
Thanks!
>Has anyone else run the benchmark and reproduced the performance
>difference?
>
>On Tue, Aug 14, 2018 at 8:16 AM, Dmitriy Pavlov < dpavlov@gmail.com >
>wrote:
>
>> It depends.
>>
>> CRC
Has anyone else run the benchmark and reproduced the performance difference?
On Tue, Aug 14, 2018 at 8:16 AM, Dmitriy Pavlov
wrote:
> It depends.
>
> CRC is a CPU-intensive operation, while WAL logging and page store write
> are mostly about IO speed.
>
> In the same time, it can make the huge
It depends.
CRC is a CPU-intensive operation, while WAL logging and page store write
are mostly about IO speed.
In the same time, it can make the huge impact on machines with fast IO and
slow CPU. So if we can apply change proposed by Evgeniy and Alexey it could
benefit performance because we
Guys, what time in % does crc calculation take in WAL logging process?
--Yakov
2018-08-14 13:37 GMT+03:00 Dmitriy Pavlov :
> Hi Alex, thank you for this idea.
>
> Evgeniy, Alex, would you like to submit the patch with bypassing
> implementation differences to keep compatibility?
>
> Sincerely,
Hi Alex, thank you for this idea.
Evgeniy, Alex, would you like to submit the patch with bypassing
implementation differences to keep compatibility?
Sincerely,
Dmitriy Pavlov
вт, 14 авг. 2018 г. в 12:06, Alex Plehanov :
> Hello, Igniters!
>
> In java8 java.lang.zip.CRC32 methods become
Hello, Igniters!
In java8 java.lang.zip.CRC32 methods become intrinsic, moreover new
"update" method, which use ByteBuffer was introduced. Since we moved to
java8, perhaps we really can get performance boost by using standard
java.lang.zip.CRC32 instead of PureJavaCrc32.
About compatibility:
Evgeniy,
Could you share benchmark code? And please share what version of JVM
you have used.
On Mon, Aug 13, 2018 at 10:44 PM Zhenya wrote:
> I think it would break backward compatibility, as Nikolay mentioned above
> we would take exception here:
>
> [1]
>
>
I think it would break backward compatibility, as Nikolay mentioned above
we would take exception here:
[1]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/file/FilePageStore.java#L372
thats why i question for
Hi Evgeniy,
would you like to submit a patch with CRC32 implementation change?
Sincerely,
Dmitriy Pavlov
пн, 13 авг. 2018 г. в 22:08, Евгений Станиловский
:
> Hi, igniters, i wrote a simple bench, looks like PureJavaCrc32 has
> performance problems in compatible with zip.CRC32.
>
> Benchmark
Hello, Denis.
AFAIK CRC calculated:
1. For each page that are readed or written to the persistence store [1].
2. For each read or write of WAL record [2].
[1]
Hello Evgeniy,
What is PureJavaCrc32 used for in Ignite? Is it on a critical path
somewhere?
--
Denis
On Mon, Aug 13, 2018 at 12:08 PM Евгений Станиловский
wrote:
> Hi, igniters, i wrote a simple bench, looks like PureJavaCrc32 has
> performance problems in compatible with zip.CRC32.
>
>
Hi, igniters, i wrote a simple bench, looks like PureJavaCrc32 has performance
problems in compatible with zip.CRC32.
Benchmark Mode Cnt Score Error Units
BenchmarkCRC.Crc32 avgt 5 1088914.540 ± 368851.822 ns/op
BenchmarkCRC.pureJavaCrc32 avgt 5 6619408.049 ± 3746712.210 ns/op
thoughts?
25 matches
Mail list logo
