Thanks for the comments Biao! bq. It seems this proposal is separated into several stages. Is there a more detailed plan? Good point! For stage one we'd like to try introducing the benchmark first, so we could guard the release (hopefully starting from 1.10). For other stages, we don't have detailed plan yet, but will add child FLIPs when moving on and open new discussion/voting separately. I have updated the FLIP document to better reflect this, please check it and let me know what you think. Thanks.
Best Regards, Yu On Tue, 5 Nov 2019 at 10:16, Biao Liu <mmyy1...@gmail.com> wrote: > Thanks Yu for bringing this topic. > > +1 for this proposal. Glad to have an e2e performance testing. > > It seems this proposal is separated into several stages. Is there a more > detailed plan? > > Thanks, > Biao /'bɪ.aʊ/ > > > > On Mon, 4 Nov 2019 at 19:54, Congxian Qiu <qcx978132...@gmail.com> wrote: > > > +1 for this idea. > > > > Currently, we have the micro benchmark for flink, which can help us find > > the regressions. And I think the e2e jobs performance testing can also > help > > us to cover more scenarios. > > > > Best, > > Congxian > > > > > > Jingsong Li <jingsongl...@gmail.com> 于2019年11月4日周一 下午5:37写道： > > > > > +1 for the idea. Thanks Yu for driving this. > > > Just curious about that can we collect the metrics about Job scheduling > > and > > > task launch. the speed of this part is also important. > > > We can add tests for watch it too. > > > > > > Look forward to more batch test support. > > > > > > Best, > > > Jingsong Lee > > > > > > On Mon, Nov 4, 2019 at 10:00 AM OpenInx <open...@gmail.com> wrote: > > > > > > > > The test cases are written in java and scripts in python. We > propose > > a > > > > separate directory/module in parallel with flink-end-to-end-tests, > with > > > the > > > > > name of flink-end-to-end-perf-tests. > > > > > > > > Glad to see that the newly introduced e2e test will be written in > Java. > > > > because I'm re-working on the existed e2e tests suites from BASH > > scripts > > > > to Java test cases so that we can support more external system , such > > as > > > > running the testing job on yarn+flink, docker+flink, > standalone+flink, > > > > distributed kafka cluster etc. > > > > BTW, I think the perf e2e test suites will also need to be designed > as > > > > supporting running on both standalone env and distributed env. will > be > > > > helpful > > > > for developing & evaluating the perf. > > > > Thanks. > > > > > > > > On Mon, Nov 4, 2019 at 9:31 AM aihua li <liaihua1...@gmail.com> > wrote: > > > > > > > > > In stage1, the checkpoint mode isn't disabled,and uses heap as the > > > > > statebackend. > > > > > I think there should be some special scenarios to test checkpoint > and > > > > > statebackend, which will be discussed and added in the release-1.11 > > > > > > > > > > > 在 2019年11月2日，上午12:13，Yun Tang <myas...@live.com> 写道： > > > > > > > > > > > > By the way, do you think it's worthy to add a checkpoint mode > which > > > > just > > > > > disable checkpoint to run end-to-end jobs? And when will stage2 and > > > > stage3 > > > > > be discussed in more details? > > > > > > > > > > > > > > > > > > > > > > > -- > > > Best, Jingsong Lee > > > > > >
