Chesnay Schepler created FLINK-23392:
----------------------------------------
Summary: Benchmarks should exclude startup/shutdown time
Key: FLINK-23392
URL: https://issues.apache.org/jira/browse/FLINK-23392
Project: Flink
Issue Type: Bug
Components: Benchmarks
Reporter: Chesnay Schepler
The benchmarks currently simply run against a local environment, which means
that the benchmark time includes how long it takes for Flink to start and shut
down. For most of the benchmarks the actual processing only accounts for a
small fraction of the current benchmark duration.
This creates noise in the benchmark results, and can result in serious
performance regressions being reported that are not not actually performance
related.
Instead, the benchmarks should setup separate setup a MiniCluster and run the
job against that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
