[
https://issues.apache.org/jira/browse/BEAM-10542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162379#comment-17162379
]
Valentyn Tymofieiev edited comment on BEAM-10542 at 7/21/20, 10:51 PM:
-----------------------------------------------------------------------
I took another look at Flink runner regression I mentioned earlier. Looking at
the dashboard around June 1, we can actually see an *improvement* in
metrics[1]. In the default configuration for SMOKE nexmark suite we use
numEvents=10000. Since I didn't see any regression on my machine in default
configuration when I started to investigate this, I increased
numEvents=1000000.
I confirmed that benchmark performance for numEvents=10000 actually improves in
2.23.0, as can also be seen on Nexmark graphs. I don't know whether a decrease
in performance on numEvents=1000000 is an intended behavior for[2] or not, will
let [~dmvk] decide whether that needs to be investigated further.
However to reiterate, the [2] change does not affect other runners and the
impact is localized to Flink runner.
[1]
http://104.154.241.245/d/ahuaA_zGz/nexmark?orgId=1&from=1587595097904&to=1595371097905&var-processingType=batch&var-ID=All&var-runner=FlinkRunner
[2] https://github.com/apache/beam/pull/11858
was (Author: tvalentyn):
I took another look at Flink runner regression I mentioned earlier. Looking at
the dashboard around June 1, we can actually see an *improvement* in
metrics[1]. In the default configuration for SMOKE nexmark suite we use a
numEvents=10000. Since I didn't see any regression in default configuration, I
increased numEvents=1000000. I confirmed that benchmark performance for
numEvents=10000 actually improves in 2.23.0, as can also be seen on Nexmark
graphs. I don't know whether a decrease in performance on 1000000 is an
intended behavior for https://github.com/apache/beam/pull/11858 or not, will
let [~dmvk] decide whether that needs to be investigated or not.
However to reiterate that change does not affect other runners and the impact
is localized to Flink runner.
[1]
http://104.154.241.245/d/ahuaA_zGz/nexmark?orgId=1&from=1587595097904&to=1595371097905&var-processingType=batch&var-ID=All&var-runner=FlinkRunner
> Investigate a possible Nexmark performance regression around 06/16
> --------------------------------------------------------------------
>
> Key: BEAM-10542
> URL: https://issues.apache.org/jira/browse/BEAM-10542
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Valentyn Tymofieiev
> Priority: P0
> Fix For: 2.23.0
>
> Attachments: nexmark.png
>
>
> There is a jump in benchmark metric visible on the Dashboard:
> http://104.154.241.245/d/ahuaA_zGz/nexmark?orgId=1&from=now-90d&to=now
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
