Hi Yuxin, Thanks for the update. LGTM. @Zakelly Please please be aware of that and make sure the benchmarks test will include that fix. Thanks!
Best regards, Jing On Wed, Sep 20, 2023 at 8:51 AM Yuxin Tan <tanyuxinw...@gmail.com> wrote: > Hi, dear community and release managers, > > Thanks for bringing this up. > > When testing the release candidate #0 for the batch scenario, I found an > issue of frequent flushing in Hybrid shuffle. It is a new bug introduced by > 1.18 and may significantly impact the performance of shuffle writing. > > The fix tickit FLINK-33044[1] has been reviewed and approved. To prevent > any performance issues, I would like to backport the fix to version 1.18 > unless there are any objections. > > Glad to hear your suggestions. > > [1] https://issues.apache.org/jira/browse/FLINK-33044 > > Best, > Yuxin > > > Zakelly Lan <zakelly....@gmail.com> 于2023年9月19日周二 17:26写道： > >> Hi Yuan and Jing, >> >> Thank you for sharing your thoughts. I completely agree that it is our >> top priority to ensure that there are no regressions from the last >> commit the previous benchmark pipeline covered to the final commit of >> this release. I will try to get this result first. >> >> >> Best, >> Zakelly >> >> On Tue, Sep 19, 2023 at 4:55 PM Jing Ge <j...@ververica.com.invalid> >> wrote: >> > >> > Hi >> > >> > Thanks Zakelly and Yuan for your effort and update. Since we changed the >> > hardware, IMHO, if we are able to reach a consensus in the community >> that >> > there is no regression with the benchmarks, we could consider releasing >> rc1 >> > without waiting for the new baseline scores which might take days. >> > >> > Best regards, >> > Jing >> > >> > On Tue, Sep 19, 2023 at 10:42 AM Yuan Mei <yuanmei.w...@gmail.com> >> wrote: >> > >> > > Hey Zakelly, >> > > >> > > Thanks very much for the efforts to re-build the entire benchmark >> > > environment. >> > > >> > > As long as we have >> > > 1) the pipeline set up and ready (no need for the entire portal >> ready), >> > > 2) get benchmark comparison numbers (comparing with the commit just >> before >> > > the benchmark pipeline is down) and >> > > 3) confirmed no-regression, it should be good enough. >> > > >> > > Thanks again! >> > > >> > > Best >> > > Yuan >> > > >> > > On Tue, Sep 19, 2023 at 4:26 PM Zakelly Lan <zakelly....@gmail.com> >> wrote: >> > > >> > > > Hi everyone, >> > > > >> > > > I am working on rebuilding the benchmark pipeline and it's almost >> > > > done. However, due to the change in machines for benchmarking, I >> will >> > > > need a few more days to run tests and gather the baseline scores for >> > > > further comparison. Once the pipeline is fully ready, we will >> proceed >> > > > with the performance test for release 1.18.0. >> > > > >> > > > Please let me know if you have any concerns. Thank you all for your >> > > > patience. >> > > > >> > > > Best, >> > > > Zakelly >> > > > >> > > > On Mon, Sep 18, 2023 at 6:57 PM Jing Ge <j...@ververica.com.invalid >> > >> > > > wrote: >> > > > > >> > > > > Hi everyone, >> > > > > >> > > > > The RC0 for Apache Flink 1.18.0 has been created. This RC is >> currently >> > > > for >> > > > > preview only to facilitate the integrated testing since the >> benchmark >> > > > tests >> > > > > are not available yet[1] and the release announcement is still >> under >> > > > > review. The RC1 will be released after all benchmarks tests are >> passed. >> > > > The >> > > > > related voting process will be triggered once the announcement is >> > > ready. >> > > > > The RC0 has all the artifacts that we would typically have for a >> > > release, >> > > > > except for the release note and the website pull request for the >> > > release >> > > > > announcement. >> > > > > >> > > > > The following contents are available for your review: >> > > > > >> > > > > - The preview source release and binary convenience releases [2], >> which >> > > > > are signed with the key with fingerprint 96AE0E32CBE6E0753CE6 [3]. >> > > > > - all artifacts that would normally be deployed to the Maven >> > > > > Central Repository [4]. >> > > > > - source code tag "release-1.18.0-rc0" [5] >> > > > > >> > > > > Your help testing the release will be greatly appreciated! And >> we'll >> > > > > create the rc1 release and the voting thread as soon as all the >> efforts >> > > > are >> > > > > finished. >> > > > > >> > > > > [1] https://issues.apache.org/jira/browse/FLINK-33052 >> > > > > [2] >> https://dist.apache.org/repos/dist/dev/flink/flink-1.18.0-rc0/ >> > > > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS >> > > > > [4] >> > > > >> https://repository.apache.org/content/repositories/orgapacheflink-1656/ >> > > > > [5] >> https://github.com/apache/flink/releases/tag/release-1.18.0-rc0 >> > > > > >> > > > > Best regards, >> > > > > Qingsheng, Sergei, Konstantin and Jing >> > > > >> > > >> >
