[jira] [Comment Edited] (FLINK-35215) The performance of serializerKryo and serializerKryoWithoutRegistration are regressed
[ https://issues.apache.org/jira/browse/FLINK-35215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848416#comment-17848416 ] Rui Fan edited comment on FLINK-35215 at 5/25/24 3:17 PM: -- Merged to master via: * 46db92641602313c4f5c1390807222c344880e2d (Revert the commit of FLINK-34954) * c1baf07d7601a683f42997dc35dfaef4e41bc928 * 5b535e1410782a2d42127e00d77815d6ab7068ed (Add test) I will follow the benchmark result in the following days, and close this Jira after the performance is recovered. was (Author: fanrui): Merged to master via: * 46db92641602313c4f5c1390807222c344880e2d * c1baf07d7601a683f42997dc35dfaef4e41bc928 I will follow the benchmark result in the following days, and close this Jira after the performance is recovered. > The performance of serializerKryo and serializerKryoWithoutRegistration are > regressed > - > > Key: FLINK-35215 > URL: https://issues.apache.org/jira/browse/FLINK-35215 > Project: Flink > Issue Type: Bug > Components: API / Type Serialization System >Affects Versions: 1.20.0 >Reporter: Rui Fan >Assignee: Rui Fan >Priority: Blocker > Labels: pull-request-available > Fix For: 1.20.0 > > Attachments: image-2024-04-25-14-57-55-231.png, > image-2024-04-25-15-00-32-410.png > > > The performance of serializerKryo and serializerKryoWithoutRegistration are > regressed[1][2], I checked recent commits, and found FLINK-34954 changed > related logic. > > [1] > [http://flink-speed.xyz/timeline/#/?exe=1,6,12&ben=serializerKryo&extr=on&quarts=on&equid=off&env=3&revs=50] > [2] > http://flink-speed.xyz/timeline/#/?exe=1,6,12&ben=serializerKryoWithoutRegistration&extr=on&quarts=on&equid=off&env=3&revs=50 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (FLINK-35215) The performance of serializerKryo and serializerKryoWithoutRegistration are regressed
[ https://issues.apache.org/jira/browse/FLINK-35215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840693#comment-17840693 ] Rui Fan edited comment on FLINK-35215 at 4/25/24 8:00 AM: -- Here is the benchmark result for my test: ||Code||serializerKryo||serializerKryoWithoutRegistration||Comment|| |Case1: current master branch|167.16333 ops/ms|135.03 ops/ms|Lower about 10% than reverts FLINK-34954| |Case2: master branch reverts FLINK-34954 |190.636025667 ops/ms|147.184592333 ops/ms| | |Case3: reverts FLINK-34954 and with [https://github.com/apache/flink/pull/24717]|189.621157333 ops/ms|146.914767 ops/ms|Lower about 0.5% than reverts FLINK-34954 | Note: I choose 3 samples for each cases. h1. Case1: current master branch data from benchmark result WebUI. * serializerKryo (avg: 167.16333 ops/ms) ** [http://flink-speed.xyz/timeline/#/?exe=6&ben=serializerKryo&extr=on&quarts=on&equid=off&env=3&revs=50] ** 164.68 ops/ms (Apr 22) ** 168.80 ops/ms (Apr 23) ** 168.01 ops/ms (Apr 24) * serializerKryoWithoutRegistration (avg: 135.03 ops/ms) ** [http://flink-speed.xyz/timeline/#/?exe=12&ben=serializerKryoWithoutRegistration&extr=on&quarts=on&equid=off&env=3&revs=50] ** 134.93 ops/ms (Apr 22) ** 134.06 ops/ms (Apr 23) ** 136.10 ops/ms (Apr 24) h1. Case2: master branch reverts FLINK-34954 I trigger it manually, the code branch is : [https://github.com/1996fanrui/flink/tree/revert-kyro-fix] Note: the result of case2 and case3 isn't public, I paste the result from the Flink Community jenkins. * serializerKryo (avg: 190.636025667 ops/ms) ** 194.929708 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/81/artifact/jmh-result.csv/*view*/] ** 186.023176 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/83/artifact/jmh-result.csv/*view*/] ** 190.955193 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/85/artifact/jmh-result.csv/*view*/] * serializerKryoWithoutRegistration (avg: 147.184592333 ops/ms) ** 149.782267 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/81/artifact/jmh-result.csv/*view*/] ** 146.312711 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/83/artifact/jmh-result.csv/*view*/] ** 145.458799 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/85/artifact/jmh-result.csv/*view*/] h1. Case3: reverts FLINK-34954 and with [https://github.com/apache/flink/pull/24717] I trigger it manually, the code branch is : [https://github.com/1996fanrui/flink/tree/revert-kyro-fix-and-return-if-0] * serializerKryo (avg: 189.621157333 ops/ms) ** 182.683889 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/82/artifact/jmh-result.csv/*view*/] ** 191.713318 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/84/artifact/jmh-result.csv/*view*/] ** 194.466265 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/86/artifact/jmh-result.csv/*view*/] * serializerKryoWithoutRegistration (avg: 146.914767 ops/ms) ** 146.514048 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/82/artifact/jmh-result.csv/*view*/] ** 145.825161 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/84/artifact/jmh-result.csv/*view*/] ** 148.405092 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/86/artifact/jmh-result.csv/*view*/] Note: I try to use other solution to fix the bug that FLINK-34954 mentioned, but I'm not sure whether the logic is right/suitable. If the solution is fine, the benchmark result is better than FLINK-34954. Code change: [https://github.com/1996fanrui/flink/commit/ebce171a806a579225897a2d29e4cd28e9164c4f] The change is pretty easy. was (Author: fanrui): Here is the benchmark result for my test: ||Code||serializerKryo||serializerKryoWithoutRegistration||Comment|| |Case1: current master branch|167.16333 ops/ms|135.03 ops/ms|Lower about 10% than reverts FLINK-34954 | |Case2: master branch reverts FLINK-34954 |190.636025667 ops/ms|147.184592333 ops/ms| | |Case3: reverts FLINK-34954 and with https://github.com/apache/flink/pull/24717|189.621157333 ops/ms|146.914767 ops/ms|Lower about 0.5% than reverts FLINK-34954 | Note: I choose 3 samples for each cases. h1. Case1: current master branch data from benchmark result WebUI. * serializerKryo (avg: 167.16333 ops/ms) ** [http://flink-speed.xyz/timeline/#/?exe=6&ben=serializerKryo&extr=on&quarts=on&equid=off&env=3&revs=50] ** 164.68 ops/ms (Apr 22) ** 168.80 ops/ms (Apr 23) ** 168.01 ops/ms (Apr 24) * serializerKryoWithoutRegistration (avg: 135.03 ops/ms) ** [http://flink-speed.xyz/timeline/#/?exe=12&ben=serializerKryoWithoutRegistration&extr=on&quarts=on&equid=of
