Hi,
+1 (non-binding) Thanks for driving this, Gary & Yu. There is an unfriendly error here: "OutOfMemoryError: Direct buffer memory" in FileChannelBoundedData$FileBufferReader. It forces our batch users to configure "taskmanager.memory.task.off-heap.size" in production jobs. And users are hard to know how much memory they need configure. Even for us developers, it is hard to say how much memory, it depends on tasks left over from the previous stage and the parallelism. It is not a blocker, but hope to resolve it in 1.11. - Verified signatures and checksums - Maven build from source skip tests - Verified pom files point to the 1.10.0 version - Test Hive integration and SQL client: work well Best, Jingsong Lee On Mon, Feb 10, 2020 at 12:28 PM Zhu Zhu <reed...@gmail.com> wrote: > My bad. The missing commit info is caused by building from the src code zip > which does not contain the git info. > So this is not a problem. > > +1 (binding) for rc3 > Here's what's were verified : > * built successfully from the source code > * run a sample streaming and a batch job with parallelism=1000 on yarn > cluster, with the new scheduler and legacy scheduler, the job runs well > (tuned some resource configs to enable the jobs to work well) > * killed TMs to trigger failures, the jobs can finally recover from the > failures > > Thanks, > Zhu Zhu > > Zhu Zhu <reed...@gmail.com> 于2020年2月10日周一 上午12:31写道： > > > The commit info is shown as <unknown> on the web UI and in logs. > > Not sure if it's a common issue or just happens to my build only. > > > > Thanks, > > Zhu Zhu > > > > aihua li <liaihua1...@gmail.com> 于2020年2月9日周日 下午7:42写道： > > > >> Yes, but the results you see in the Performance Code Speed Center [3] > >> skip FLIP-49. > >> The results of the default configurations are overwritten by the latest > >> results. > >> > >> > 2020年2月9日 下午5:29，Yu Li <car...@gmail.com> 写道： > >> > > >> > Thanks for the efforts Aihua! These could definitely improve our RC > >> test coverage! > >> > > >> > Just to confirm, that the stability tests were executed with the same > >> test suite for Alibaba production usage, and the e2e performance one was > >> executed with the test suite proposed in FLIP-83 [1] and FLINK-14917 > [2], > >> and the result could also be observed from our performance code-speed > >> center [3], right? > >> > > >> > Thanks. > >> > > >> > Best Regards, > >> > Yu > >> > > >> > [1] > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-83%3A+Flink+End-to-end+Performance+Testing+Framework > >> < > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-83%3A+Flink+End-to-end+Performance+Testing+Framework > >> > > >> > [2] https://issues.apache.org/jira/browse/FLINK-14917 < > >> https://issues.apache.org/jira/browse/FLINK-14917> > >> > [3] https://s.apache.org/nglhm <https://s.apache.org/nglhm> > >> > > >> > On Sun, 9 Feb 2020 at 11:20, aihua li <liaihua1...@gmail.com <mailto: > >> liaihua1...@gmail.com>> wrote: > >> > +1 (non-binging) > >> > > >> > I ran stability tests and end-to-end performance tests in branch > >> release-1.10.0-rc3,both of them passed. > >> > > >> > Stability test: It mainly checks The flink job can revover from > >> various abnormal situations which concluding disk full, > >> > network interruption, zk unable to connect, rpc message timeout, etc. > >> > If job can't be recoverd it means test failed. > >> > The test passed after running 5 hours. > >> > > >> > End-to-end performance test: It containes 32 test scenarios which > >> designed in FLIP-83. > >> > Test results: The performance regressions about 3% from 1.9.1 if uses > >> default parameters; > >> > The result: > >> > > >> > if skips FLIP-49 (add parameters:taskmanager.memory.managed.fraction: > >> 0,taskmanager.memory.flink.size: 1568m in flink-conf.yaml), > >> > the performance improves about 5% from 1.9.1. The result: > >> > > >> > > >> > I confirm it with @Xintong Song < > >> https://cwiki.apache.org/confluence/display/~xintongsong> that the > >> result makes sense. > >> > > >> >> 2020年2月8日 上午5:54，Gary Yao <g...@apache.org <mailto:g...@apache.org>> > >> 写道： > >> >> > >> >> Hi everyone, > >> >> Please review and vote on the release candidate #3 for the version > >> 1.10.0, > >> >> as follows: > >> >> [ ] +1, Approve the release > >> >> [ ] -1, Do not approve the release (please provide specific comments) > >> >> > >> >> > >> >> The complete staging area is available for your review, which > includes: > >> >> * JIRA release notes [1], > >> >> * the official Apache source release and binary convenience releases > >> to be > >> >> deployed to dist.apache.org <http://dist.apache.org/> [2], which are > >> signed with the key with > >> >> fingerprint BB137807CEFBE7DD2616556710B12A1F89C115E8 [3], > >> >> * all artifacts to be deployed to the Maven Central Repository [4], > >> >> * source code tag "release-1.10.0-rc3" [5], > >> >> * website pull request listing the new release and adding > announcement > >> blog > >> >> post [6][7]. > >> >> > >> >> The vote will be open for at least 72 hours. It is adopted by > majority > >> >> approval, with at least 3 PMC affirmative votes. > >> >> > >> >> Thanks, > >> >> Yu & Gary > >> >> > >> >> [1] > >> >> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12345845 > >> < > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12345845 > >> > > >> >> [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.10.0-rc3/ < > >> https://dist.apache.org/repos/dist/dev/flink/flink-1.10.0-rc3/> > >> >> [3] https://dist.apache.org/repos/dist/release/flink/KEYS < > >> https://dist.apache.org/repos/dist/release/flink/KEYS> > >> >> [4] > >> https://repository.apache.org/content/repositories/orgapacheflink-1333 > < > >> https://repository.apache.org/content/repositories/orgapacheflink-1333> > >> >> [5] https://github.com/apache/flink/releases/tag/release-1.10.0-rc3 > < > >> https://github.com/apache/flink/releases/tag/release-1.10.0-rc3> > >> >> [6] https://github.com/apache/flink-web/pull/302 < > >> https://github.com/apache/flink-web/pull/302> > >> >> [7] https://github.com/apache/flink-web/pull/301 < > >> https://github.com/apache/flink-web/pull/301> > >> > > >> > >> > -- Best, Jingsong Lee
