Updates for 1.10.1 watched issues (we are in good progress and almost there to produce the first RC, thanks all for the efforts):
* Blockers (3 left) - [Closed] FLINK-16018 Improve error reporting when submitting batch job (instead of AskTimeoutException) - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job submission - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException when use flink-sql-connector-elasticsearch7 - [Closed] FLINK-16262 Class loader problem with FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory - [Closed] FLINK-16406 Increase default value for JVM Metaspace to minimise its OutOfMemoryError - [Closed] FLINK-16454 Update the copyright year in NOTICE files - [PR reviewed] FLINK-16576 State inconsistency on restore with memory state backends - [Under Discussion] FLINK-16626 Exception encountered when cancelling a job in yarn per-job mode - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before client can retrieve JobResult - [PR reviewed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum throws UnsupportedOperationException * Critical (1 left) - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership" error - [Closed] FLINK-16047 Blink planner produces wrong aggregate results with state clean up - [Closed] FLINK-16070 Blink planner can not extract correct unique key for UpsertStreamTableSink - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should be handled as Fatal Error in TaskManager - [May Postpone] FLINK-16408 Bind user code class loader to lifetime of a slot Please let me know if you find any missing ones, thanks. Best Regards, Yu On Fri, 27 Mar 2020 at 21:06, Yu Li <car...@gmail.com> wrote: > Here comes the latest status of issues in 1.10.1 watch list: > > * Blockers (4 left) > - [Under Discussion] FLINK-16018 Improve error reporting when submitting > batch job (instead of AskTimeoutException) > - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on > repeated job submission > - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException when > use flink-sql-connector-elasticsearch7 > - [PR Approved] FLINK-16262 Class loader problem with > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to > minimise its OutOfMemoryError > - [Closed] FLINK-16454 Update the copyright year in NOTICE files > - [In Progress] FLINK-16576 State inconsistency on restore with memory > state backends > - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down MiniCluster > before client can retrieve JobResult > > * Critical (2 left) > - [Closed] FLINK-16047 Blink planner produces wrong aggregate results > with state clean up > - [PR Submitted] FLINK-16070 Blink planner can not extract correct > unique key for UpsertStreamTableSink > - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory should be > handled as Fatal Error in TaskManager > - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot > > There's a new blocker added. Please let me know if you find any other > issues should be put into the watch list. Thanks. > > Best Regards, > Yu > > > On Sat, 21 Mar 2020 at 14:37, Yu Li <car...@gmail.com> wrote: > >> Hi All, >> >> Here is the status update of issues in 1.10.1 watch list: >> >> * Blockers (7) >> >> - [Under Discussion] FLINK-16018 Improve error reporting when >> submitting batch job (instead of AskTimeoutException) >> >> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM error >> on repeated job submission >> >> - [PR Submitted] FLINK-16170 SearchTemplateRequest >> ClassNotFoundException when use flink-sql-connector-elasticsearch7 >> >> - [PR Submitted] FLINK-16262 Class loader problem with >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory >> >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to >> minimise its OutOfMemoryError >> >> - [Closed] FLINK-16454 Update the copyright year in NOTICE files >> >> - [Open] FLINK-16576 State inconsistency on restore with memory state >> backends >> >> >> * Critical (4) >> >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results >> with state clean up >> >> - [PR Submitted] FLINK-16070 Blink planner can not extract correct >> unique key for UpsertStreamTableSink >> >> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be >> handled as Fatal Error in TaskManager >> >> - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot >> >> Please let me know if you see any new blockers to add to the list. Thanks. >> >> Best Regards, >> Yu >> >> >> On Wed, 18 Mar 2020 at 00:11, Yu Li <car...@gmail.com> wrote: >> >>> Thanks for the updates Till! >>> >>> For FLINK-16018, maybe we could create two sub-tasks for easy and >>> complete fix separately, and only include the easy one in 1.10.1? Or please >>> just feel free to postpone the whole task to 1.10.2 if "all or nothing" >>> policy is preferred (smile). Thanks. >>> >>> Best Regards, >>> Yu >>> >>> >>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <trohrm...@apache.org> >>> wrote: >>> >>>> +1 for a soonish bug fix release. Thanks for volunteering as our release >>>> manager Yu. >>>> >>>> I think we can soon merge the increase of metaspace size and improving >>>> the >>>> error message. The assumption is that we currently don't have too many >>>> small Flink 1.10 deployments with a process size <= 1GB. Of course, the >>>> sooner we release the bug fix release, the fewer deployments will be >>>> affected by this change. >>>> >>>> For FLINK-16018, I think there would be an easy solution which we could >>>> include in the bug fix release. The proper fix will most likely take a >>>> bit >>>> longer, though. >>>> >>>> Cheers, >>>> Till >>>> >>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <azagre...@apache.org> >>>> wrote: >>>> >>>> > > @Andrey and @Xintong - could we have a quick poll on the user >>>> mailing >>>> > list >>>> > > about increasing the metaspace size in Flink 1.10.1? Specifically >>>> asking >>>> > > for who has very small TM setups? >>>> > >>>> > There has been a survey about this topic since 10 days: >>>> > >>>> > `[Survey] Default size for the new JVM Metaspace limit in 1.10` >>>> > I can ask there about the small TM setups. >>>> > >>>> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <car...@gmail.com> wrote: >>>> > >>>> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency on >>>> restore >>>> > with >>>> > > memory state backends >>>> > > >>>> > > Let me recompile the watching list with recent feedbacks. There're >>>> > totally >>>> > > 45 issues with Blocker/Critical priority for 1.10.1, out of which 14 >>>> > > already resolved and 31 left, and the below ones are watched >>>> (meaning >>>> > that >>>> > > once the below ones got fixed, we will kick of the releasing with >>>> left >>>> > ones >>>> > > postponed to 1.10.2 unless objections): >>>> > > >>>> > > * Blockers (7) >>>> > > - [Under Discussion] FLINK-16018 Improve error reporting when >>>> > submitting >>>> > > batch job (instead of AskTimeoutException) >>>> > > - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM >>>> error >>>> > > on repeated job submission >>>> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest >>>> > ClassNotFoundException >>>> > > when use flink-sql-connector-elasticsearch7 >>>> > > - [PR Submitted] FLINK-16262 Class loader problem with >>>> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory >>>> > > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to >>>> > > minimise its OutOfMemoryError >>>> > > - [Open] FLINK-16454 Update the copyright year in NOTICE files >>>> > > - [Open] FLINK-16576 State inconsistency on restore with memory >>>> state >>>> > > backends >>>> > > >>>> > > * Critical (4) >>>> > > - [Open] FLINK-16047 Blink planner produces wrong aggregate >>>> results >>>> > with >>>> > > state clean up >>>> > > - [PR Submitted] FLINK-16070 Blink planner can not extract correct >>>> > unique >>>> > > key for UpsertStreamTableSink >>>> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be >>>> > > handled as Fatal Error in TaskManager >>>> > > - [Open] FLINK-16408 Bind user code class loader to lifetime of a >>>> slot >>>> > > >>>> > > Please raise your hand if you find any other issues should be put >>>> into >>>> > this >>>> > > list, thanks. >>>> > > >>>> > > Please also note that 1.10.2 version is already created (thanks for >>>> the >>>> > > help @jincheng), and feel free to adjust/assign fix version to it >>>> when >>>> > > necessary. >>>> > > >>>> > > Best Regards, >>>> > > Yu >>>> > > >>>> > > >>>> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org> >>>> wrote: >>>> > > >>>> > > > @Andrey and @Xintong - could we have a quick poll on the user >>>> mailing >>>> > > list >>>> > > > about increasing the metaspace size in Flink 1.10.1? Specifically >>>> > asking >>>> > > > for who has very small TM setups? >>>> > > > >>>> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <car...@gmail.com> wrote: >>>> > > > >>>> > > > > Thanks for the reminder Stephan and the inputs/discussion >>>> Andrey and >>>> > > > > Xintong. I totally agree that we should try best not to >>>> introduce new >>>> > > > > issues when resolving the existing one. Will watch closely >>>> about the >>>> > > > > discussion and conclusion around metaspace configuration. >>>> > > > > >>>> > > > > From the point as a developer instead of RM, I'm optimistic that >>>> > > > reserving >>>> > > > > 160MB more memory to JVM metaspace by default won't cause a big >>>> > > problem, >>>> > > > > based on the assumption that nowadays assigning 2GB/4GB memory >>>> per TM >>>> > > is >>>> > > > > the common case. >>>> > > > > >>>> > > > > Best Regards, >>>> > > > > Yu >>>> > > > > >>>> > > > > >>>> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song < >>>> tonysong...@gmail.com> >>>> > > > wrote: >>>> > > > > >>>> > > > > > I think in FLINK-16406 we have not only increased the default >>>> value >>>> > > for >>>> > > > > > metaspace, but also increased the 'process.size' in default >>>> > > > > > 'flink-conf.yaml'. >>>> > > > > > >>>> > > > > > That means: >>>> > > > > > >>>> > > > > > - For people using the default 'flink-conf.yaml', it's not >>>> > really >>>> > > > > taking >>>> > > > > > memory from heap / managed for metaspace. >>>> > > > > > - For people using custom configuration files: >>>> > > > > > - If they are specifying 'flink.size', then metaspace >>>> does >>>> > not >>>> > > > > really >>>> > > > > > affect the flink memory internal components (heap, >>>> managed, >>>> > > > etc.). >>>> > > > > > - If they are specifying 'process.size' (most likely >>>> active >>>> > > > > setups), >>>> > > > > > then they will have less heap / managed memory, or >>>> worse, >>>> > their >>>> > > > > > configuration may not work without adjustment. >>>> > > > > > - For a TM with large process memory (e.g., 4 GB or >>>> more), >>>> > > it >>>> > > > is >>>> > > > > > probably ok that the metaspace takes 160 MB more >>>> memory. >>>> > > > > > - For a TM with smaller total memory (1 or 2 GB) the >>>> > > metaspace >>>> > > > > > increase is more likely to cause problem. >>>> > > > > > >>>> > > > > > So basically only people have small 'process.size' in custom >>>> config >>>> > > > file >>>> > > > > > are really affected. I'm not sure what is the proportion of >>>> such >>>> > use >>>> > > > > cases >>>> > > > > > though. (From questions asked on the user ML, probably not >>>> much). >>>> > > > > > >>>> > > > > > Thank you~ >>>> > > > > > >>>> > > > > > Xintong Song >>>> > > > > > >>>> > > > > > >>>> > > > > > >>>> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen < >>>> se...@apache.org> >>>> > > > wrote: >>>> > > > > > >>>> > > > > > > No need to revert it now - I am not saying it should not go >>>> into >>>> > > > > 1.10.1, >>>> > > > > > I >>>> > > > > > > am just saying this is not clear to me yet. >>>> > > > > > > >>>> > > > > > > We have to trade off the fact that we may break some >>>> deployments >>>> > > with >>>> > > > > the >>>> > > > > > > fact that we will "safe" a lot of new deployments. >>>> > > > > > > I simply lack the data points / insight at the moment to >>>> > understand >>>> > > > how >>>> > > > > > > significant both cases are, meaning how many users would be >>>> > > affected >>>> > > > > and >>>> > > > > > > how badly. >>>> > > > > > > >>>> > > > > > > Independent of that, improving the error message is always >>>> > helpful. >>>> > > > > > > >>>> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin < >>>> > > > > > > azagrebin.apa...@gmail.com> >>>> > > > > > > wrote: >>>> > > > > > > >>>> > > > > > > > > - For 1.10.1 I am not completely sure, because users >>>> expect >>>> > > to >>>> > > > > > > upgrade >>>> > > > > > > > > that without config adjustments. That might not be >>>> possible >>>> > > with >>>> > > > > that >>>> > > > > > > > > change. >>>> > > > > > > > >>>> > > > > > > > Ok, makes sense, I will revert it for 1.10 and only try to >>>> > > improve >>>> > > > > > error >>>> > > > > > > > message and docs. >>>> > > > > > > > >>>> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen < >>>> se...@apache.org> >>>> > > wrote: >>>> > > > > > > > > >>>> > > > > > > > > @Andrey about the increase in metaspace size >>>> > > > > > > > > - I have no concerns for 1.11.0. >>>> > > > > > > > > - For 1.10.1 I am not completely sure, because users >>>> expect >>>> > > to >>>> > > > > > > upgrade >>>> > > > > > > > > that without config adjustments. That might not be >>>> possible >>>> > > with >>>> > > > > that >>>> > > > > > > > > change. >>>> > > > > > > > > >>>> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin < >>>> > > > > > > > azagrebin.apa...@gmail.com> >>>> > > > > > > > > wrote: >>>> > > > > > > > > >>>> > > > > > > > >> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM >>>> error >>>> > on >>>> > > > > > repeated >>>> > > > > > > > >> job” >>>> > > > > > > > >> >>>> > > > > > > > >> My understanding that the issue is basically covered >>>> by: >>>> > > > > > > > >> >>>> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be >>>> handled as >>>> > > > Fatal >>>> > > > > > > Error >>>> > > > > > > > >> in TaskManager >>>> > > > > > > > >> no full consensus there but improving error message >>>> for >>>> > > > existing >>>> > > > > > > task >>>> > > > > > > > >> thread fatal handling could be done at least >>>> > > > > > > > >> >>>> > > > > > > > >> - [FLINK-16406] Increase default value for JVM >>>> Metaspace to >>>> > > > > minimise >>>> > > > > > > its >>>> > > > > > > > >> OutOfMemoryError >>>> > > > > > > > >> see further >>>> > > > > > > > >> >>>> > > > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from >>>> > > > dynamically >>>> > > > > > > > loaded >>>> > > > > > > > >> AWS connectors >>>> > > > > > > > >> not sure whether this is a blocker but looks close to >>>> be >>>> > > > resolved >>>> > > > > > > > >> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM >>>> > Metaspace" >>>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix >>>> release? >>>> > > It >>>> > > > > > > changes >>>> > > > > > > > >>> setups, takes away memory from heap / managed memory >>>> on >>>> > > > existing >>>> > > > > > > setups >>>> > > > > > > > >>> that keep their flink-conf.yaml. >>>> > > > > > > > >> >>>> > > > > > > > >> My understanding was that increasing to 256m resolved >>>> the >>>> > > > reported >>>> > > > > > > > problems >>>> > > > > > > > >> and we decided to make the change so I have merged it >>>> today >>>> > as >>>> > > > > there >>>> > > > > > > > were >>>> > > > > > > > >> no more concerns. >>>> > > > > > > > >> If there are concerns I can revert it. >>>> > > > > > > > >> >>>> > > > > > > > >> On the other hand, I think improving the message error >>>> with >>>> > > > > > reference >>>> > > > > > > to >>>> > > > > > > > >> the metaspace option should help the most >>>> > > > > > > > >> because user would not have to read all docs to fix it >>>> > > > > > > > >> then maybe this change is not even needed. >>>> > > > > > > > >> >>>> > > > > > > > >> Best, >>>> > > > > > > > >> Andrey >>>> > > > > > > > >> >>>> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen < >>>> se...@apache.org> >>>> > > > wrote: >>>> > > > > > > > >>> >>>> > > > > > > > >>> Good idea to go ahead with 1.10.1 >>>> > > > > > > > >>> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM >>>> error >>>> > on >>>> > > > > > repeated >>>> > > > > > > > >> job" >>>> > > > > > > > >>> - I don't think we have consensus on the exact >>>> solution, >>>> > yet, >>>> > > > and >>>> > > > > > > some >>>> > > > > > > > >> of >>>> > > > > > > > >>> the changes might also have side effects that are >>>> hard to >>>> > > > > predict, >>>> > > > > > > so I >>>> > > > > > > > >> am >>>> > > > > > > > >>> not sure we should rush this in. >>>> > > > > > > > >>> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM >>>> > Metaspace" >>>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix >>>> release? >>>> > > It >>>> > > > > > > changes >>>> > > > > > > > >>> setups, takes away memory from heap / managed memory >>>> on >>>> > > > existing >>>> > > > > > > setups >>>> > > > > > > > >>> that keep their flink-conf.yaml. >>>> > > > > > > > >>> >>>> > > > > > > > >>> We may need to unblock the release form these two >>>> issues >>>> > and >>>> > > > > think >>>> > > > > > > > about >>>> > > > > > > > >>> having 1.10.2 in the near future. >>>> > > > > > > > >>> >>>> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li < >>>> car...@gmail.com> >>>> > > > wrote: >>>> > > > > > > > >>> >>>> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an eye on >>>> these >>>> > two. >>>> > > > > > > > >>>> >>>> > > > > > > > >>>> Best Regards, >>>> > > > > > > > >>>> Yu >>>> > > > > > > > >>>> >>>> > > > > > > > >>>> >>>> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu < >>>> imj...@gmail.com> >>>> > > > wrote: >>>> > > > > > > > >>>> >>>> > > > > > > > >>>>> Thanks for driving this release, Yu! >>>> > > > > > > > >>>>> +1 to start 1.10.1 release cycle. >>>> > > > > > > > >>>>> >>>> > > > > > > > >>>>> From the Table SQL module, I think we should also >>>> try to >>>> > > get >>>> > > > in >>>> > > > > > the >>>> > > > > > > > >>>>> following issues: >>>> > > > > > > > >>>>> - FLINK-16441: Allow users to override flink-conf >>>> > > parameters >>>> > > > > from >>>> > > > > > > SQL >>>> > > > > > > > >> CLI >>>> > > > > > > > >>>>> environment >>>> > > > > > > > >>>>> this allows users to set e.g. statebackend, >>>> watermark >>>> > > > interval, >>>> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI >>>> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched >>>> > > > exception >>>> > > > > is >>>> > > > > > > > >> thrown >>>> > > > > > > > >>>>> this will improve much experience when using SQL CLI >>>> > > > > > > > >>>>> >>>> > > > > > > > >>>>> Best, >>>> > > > > > > > >>>>> Jark >>>> > > > > > > > >>>>> >>>> > > > > > > > >>>>> >>>> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li < >>>> car...@gmail.com> >>>> > > > wrote: >>>> > > > > > > > >>>>> >>>> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've added >>>> 1.10.1 into >>>> > > > > > > FLINK-16225 >>>> > > > > > > > >>>> fix >>>> > > > > > > > >>>>>> versions and promoted its priority to Critical. >>>> Will >>>> > also >>>> > > > > watch >>>> > > > > > > the >>>> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408. >>>> > > > > > > > >>>>>> >>>> > > > > > > > >>>>>> Best Regards, >>>> > > > > > > > >>>>>> Yu >>>> > > > > > > > >>>>>> >>>> > > > > > > > >>>>>> >>>> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin < >>>> > > > > > > azagre...@apache.org >>>> > > > > > > > > >>>> > > > > > > > >>>>>> wrote: >>>> > > > > > > > >>>>>> >>>> > > > > > > > >>>>>>> Hi Yu, >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release >>>> discussion! >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>>> Apart from >>>> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM >>>> Metaspace >>>> > to >>>> > > > > > > minimise >>>> > > > > > > > >>>> its >>>> > > > > > > > >>>>>>> OutOfMemoryError >>>> > > > > > > > >>>>>>> which should be merged soon >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>>> I think we should also try to get in the following >>>> > > issues: >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be >>>> > handled >>>> > > > as >>>> > > > > > > Fatal >>>> > > > > > > > >>>>> Error >>>> > > > > > > > >>>>>> in >>>> > > > > > > > >>>>>>> TaskManager >>>> > > > > > > > >>>>>>> This should solve the Metaspace problem even in a >>>> > better >>>> > > > way >>>> > > > > > > > because >>>> > > > > > > > >>>>> OOM >>>> > > > > > > > >>>>>>> failure should point users to the docs immediately >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to >>>> lifetime >>>> > > of >>>> > > > a >>>> > > > > > slot >>>> > > > > > > > >>>>>>> This should give a better protection against class >>>> > > loading >>>> > > > > > leaks >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when >>>> submitting >>>> > > > batch >>>> > > > > > job >>>> > > > > > > > >>>>>> (instead >>>> > > > > > > > >>>>>>> of AskTimeoutException) >>>> > > > > > > > >>>>>>> This problem has recently happened for multiple >>>> users >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>>> Best, >>>> > > > > > > > >>>>>>> Andrey >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li < >>>> > > > > > > > jingsongl...@gmail.com> >>>> > > > > > > > >>>>>>> wrote: >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the >>>> 1.10.1 >>>> > > > release. >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>>> Some issues are very important, Users are looking >>>> > > forward >>>> > > > to >>>> > > > > > > them. >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>>> Best, >>>> > > > > > > > >>>>>>>> Jingsong Lee >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo < >>>> > > > > > karma...@gmail.com> >>>> > > > > > > > >>>>> wrote: >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu! >>>> > > > > > > > >>>>>>>>> >>>> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle. >>>> > > > > > > > >>>>>>>>> >>>> > > > > > > > >>>>>>>>> Best, >>>> > > > > > > > >>>>>>>>> Yangze Guo >>>> > > > > > > > >>>>>>>>> >>>> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song < >>>> > > > > > > > >>>>> tonysong...@gmail.com> >>>> > > > > > > > >>>>>>>>> wrote: >>>> > > > > > > > >>>>>>>>>> >>>> > > > > > > > >>>>>>>>>> Yu, >>>> > > > > > > > >>>>>>>>>> Thanks for the explanation. >>>> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to get some >>>> > inputs >>>> > > > for >>>> > > > > > > > >>>>>>> prioritizing >>>> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to >>>> me. >>>> > > > > > > > >>>>>>>>>> >>>> > > > > > > > >>>>>>>>>> >>>> > > > > > > > >>>>>>>>>> Thank you~ >>>> > > > > > > > >>>>>>>>>> >>>> > > > > > > > >>>>>>>>>> Xintong Song >>>> > > > > > > > >>>>>>>>>> >>>> > > > > > > > >>>>>>>>>> >>>> > > > > > > > >>>>>>>>>> >>>> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li < >>>> > > > car...@gmail.com> >>>> > > > > > > > >>>> wrote: >>>> > > > > > > > >>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1? >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> According to the history, the first patch >>>> release >>>> > of >>>> > > a >>>> > > > > > major >>>> > > > > > > > >>>>>>> version >>>> > > > > > > > >>>>>>>>> will >>>> > > > > > > > >>>>>>>>>>> take ~1month from discussion started, >>>> depending on >>>> > > the >>>> > > > > > speed >>>> > > > > > > > >>>> of >>>> > > > > > > > >>>>>>>> blocker >>>> > > > > > > > >>>>>>>>>>> issue resolving: >>>> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1], >>>> > released >>>> > > > on >>>> > > > > > Jul >>>> > > > > > > > >>>>> 3rd >>>> > > > > > > > >>>>>>> [2] >>>> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], >>>> > released >>>> > > > on >>>> > > > > > Oct >>>> > > > > > > > >>>>> 19th >>>> > > > > > > > >>>>>>> [4] >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> We won't rush to match the history of course, >>>> but >>>> > > could >>>> > > > > use >>>> > > > > > > > >>>> it >>>> > > > > > > > >>>>>> as a >>>> > > > > > > > >>>>>>>>>>> reference. And please feel free to let me >>>> know if >>>> > any >>>> > > > > > > > >>>> concerns >>>> > > > > > > > >>>>>>>> Xintong. >>>> > > > > > > > >>>>>>>>>>> Thanks. >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> Best Regards, >>>> > > > > > > > >>>>>>>>>>> Yu >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> [1] >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>> >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>> >>>> > > > > > > > >>>>> >>>> > > > > > > > >>>> >>>> > > > > > > > >> >>>> > > > > > > > >>>> > > > > > > >>>> > > > > > >>>> > > > > >>>> > > > >>>> > > >>>> > >>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html >>>> > > > > > > > >>>>>>>>>>> [2] >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>> >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>> >>>> > > > > > > > >>>>> >>>> > > > > > > > >>>> >>>> > > > > > > > >> >>>> > > > > > > > >>>> > > > > > > >>>> > > > > > >>>> > > > > >>>> > > > >>>> > > >>>> > >>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html >>>> > > > > > > > >>>>>>>>>>> [3] >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>> >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>> >>>> > > > > > > > >>>>> >>>> > > > > > > > >>>> >>>> > > > > > > > >> >>>> > > > > > > > >>>> > > > > > > >>>> > > > > > >>>> > > > > >>>> > > > >>>> > > >>>> > >>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html >>>> > > > > > > > >>>>>>>>>>> [4] >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>> >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>> >>>> > > > > > > > >>>>> >>>> > > > > > > > >>>> >>>> > > > > > > > >> >>>> > > > > > > > >>>> > > > > > > >>>> > > > > > >>>> > > > > >>>> > > > >>>> > > >>>> > >>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song < >>>> > > > > > > > >>>>>> tonysong...@gmail.com> >>>> > > > > > > > >>>>>>>>> wrote: >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering >>>> to be >>>> > > the >>>> > > > > > > > >>>>> release >>>> > > > > > > > >>>>>>>>> manager. >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> +1 for the proposal. >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> One quick question, what is the time plan for >>>> > > 1.10.1? >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> Thank you~ >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> Xintong Song >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang >>>> > > > > > > > >>>>>>>>>>>> <wangzhijiang...@aliyun.com.invalid> wrote: >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu! >>>> > > > > > > > >>>>>>>>>>>>> +1 on my side >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> Best, >>>> > > > > > > > >>>>>>>>>>>>> Zhijiang >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>> >>>> > > > > > > >>>> > ------------------------------------------------------------------ >>>> > > > > > > > >>>>>>>>>>>>> From:Yu Li <car...@gmail.com> >>>> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25 >>>> > > > > > > > >>>>>>>>>>>>> To:dev <dev@flink.apache.org> >>>> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1 >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will >>>> also >>>> > keep >>>> > > > an >>>> > > > > > > > >>>> eye >>>> > > > > > > > >>>>> on >>>> > > > > > > > >>>>>>>> these >>>> > > > > > > > >>>>>>>>>>>>> existing blocker issues. >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> Best Regards, >>>> > > > > > > > >>>>>>>>>>>>> Yu >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng < >>>> > > > > > > > >>>>> he...@apache.org> >>>> > > > > > > > >>>>>>>>> wrote: >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>> Hi Yu, >>>> > > > > > > > >>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and >>>> > > > volunteer >>>> > > > > > > > >>>> as >>>> > > > > > > > >>>>>> the >>>> > > > > > > > >>>>>>>>> release >>>> > > > > > > > >>>>>>>>>>>>>> manager! >>>> > > > > > > > >>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>> I found there are some other issues[1] >>>> which are >>>> > > > > marked >>>> > > > > > > > >>>>> as >>>> > > > > > > > >>>>>> a >>>> > > > > > > > >>>>>>>>> blocker: >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in >>>> > NOTICE >>>> > > > > files >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with >>>> > > > > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE >>>> and >>>> > > usrlib >>>> > > > > > > > >>>>>> directory >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest >>>> > > > > > > > >>>>> ClassNotFoundException >>>> > > > > > > > >>>>>>> when >>>> > > > > > > > >>>>>>>>> use >>>> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7 >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when >>>> > > > submitting >>>> > > > > > > > >>>>> batch >>>> > > > > > > > >>>>>>> job >>>> > > > > > > > >>>>>>>>>>>> (instead >>>> > > > > > > > >>>>>>>>>>>>> of >>>> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException) >>>> > > > > > > > >>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved in >>>> 1.10.1. >>>> > > > > > > > >>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>> Best, >>>> > > > > > > > >>>>>>>>>>>>>> Hequn >>>> > > > > > > > >>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>> [1] >>>> > > > > > > > >>>>>>>>> >>>> > > > > > > >>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891 >>>> > > > > > > > >>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li < >>>> > > > > > > > >>>> car...@gmail.com> >>>> > > > > > > > >>>>>>>> wrote: >>>> > > > > > > > >>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng, >>>> > > > > > > > >>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful. >>>> Thanks a >>>> > > lot! >>>> > > > > > > > >>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>> Best Regards, >>>> > > > > > > > >>>>>>>>>>>>>>> Yu >>>> > > > > > > > >>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng >>>> sun < >>>> > > > > > > > >>>>>>>>>>> sunjincheng...@gmail.com >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>> wrote: >>>> > > > > > > > >>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I >>>> would >>>> > > > like >>>> > > > > > > > >>>>> to >>>> > > > > > > > >>>>>>> give >>>> > > > > > > > >>>>>>>>> you a >>>> > > > > > > > >>>>>>>>>>>>> hand >>>> > > > > > > > >>>>>>>>>>>>>> at >>>> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is >>>> finished.(If you >>>> > > > > > > > >>>> need) >>>> > > > > > > > >>>>>> :) >>>> > > > > > > > >>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>> Best, >>>> > > > > > > > >>>>>>>>>>>>>>>> Jincheng >>>> > > > > > > > >>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <car...@gmail.com> 于2020年3月10日周二 >>>> > > 下午5:49写道： >>>> > > > > > > > >>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Hi All, >>>> > > > > > > > >>>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we >>>> > released >>>> > > > > > > > >>>>> Flink >>>> > > > > > > > >>>>>>>>> 1.10.0. We >>>> > > > > > > > >>>>>>>>>>>>>> already >>>> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved >>>> improvements/bugs >>>> > in >>>> > > > > > > > >>>> the >>>> > > > > > > > >>>>>>>>>>> release-1.10 >>>> > > > > > > > >>>>>>>>>>>>>>> branch, >>>> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 >>>> release >>>> > > cycle. >>>> > > > > > > > >>>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are: >>>> > > > > > > > >>>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the >>>> license and >>>> > > > > > > > >>>> notice >>>> > > > > > > > >>>>>>> file >>>> > > > > > > > >>>>>>>> in >>>> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib >>>> > > > > > > > >>>>>>>>>>>>>>>>> module >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak >>>> in >>>> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be >>>> known >>>> > > in >>>> > > > > > > > >>>>>>>>> HiveCatalog >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336 >>>> ArrayIndexOufOBoundsException in >>>> > > > > > > > >>>>>>>> TypeExtractor >>>> > > > > > > > >>>>>>>>>>> when >>>> > > > > > > > >>>>>>>>>>>>>>> mapping >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is >>>> failed if >>>> > > > > > > > >>>> running >>>> > > > > > > > >>>>>> in >>>> > > > > > > > >>>>>>>>> blink >>>> > > > > > > > >>>>>>>>>>>>> planner >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints >>>> are not >>>> > > > > > > > >>>> reset >>>> > > > > > > > >>>>>> on >>>> > > > > > > > >>>>>>>> task >>>> > > > > > > > >>>>>>>>>>>>> recovery >>>> > > > > > > > >>>>>>>>>>>>>> in >>>> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function >>>> using >>>> > > sql >>>> > > > > > > > >>>>>>> casuing >>>> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation >>>> failed >>>> > > > > > > > >>>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following >>>> issues >>>> > > should >>>> > > > > > > > >>>> be >>>> > > > > > > > >>>>>>>> merged >>>> > > > > > > > >>>>>>>>>>> before >>>> > > > > > > > >>>>>>>>>>>>>>> 1.10.1 >>>> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM >>>> issue): >>>> > > > > > > > >>>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes >>>> Metaspace >>>> > OOM >>>> > > > > > > > >>>>> error >>>> > > > > > > > >>>>>> on >>>> > > > > > > > >>>>>>>>>>> repeated >>>> > > > > > > > >>>>>>>>>>>>> job >>>> > > > > > > > >>>>>>>>>>>>>>>>> submission >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value >>>> for JVM >>>> > > > > > > > >>>>>> Metaspace >>>> > > > > > > > >>>>>>> to >>>> > > > > > > > >>>>>>>>>>>> minimise >>>> > > > > > > > >>>>>>>>>>>>>> its >>>> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces >>>> wrong >>>> > > > > > > > >>>>> aggregate >>>> > > > > > > > >>>>>>>>> results >>>> > > > > > > > >>>>>>>>>>> with >>>> > > > > > > > >>>>>>>>>>>>>> state >>>> > > > > > > > >>>>>>>>>>>>>>>>> clean up >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not >>>> extract >>>> > > > > > > > >>>> correct >>>> > > > > > > > >>>>>>>> unique >>>> > > > > > > > >>>>>>>>> key >>>> > > > > > > > >>>>>>>>>>>> for >>>> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink >>>> > > > > > > > >>>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release >>>> manager and >>>> > > kick >>>> > > > > > > > >>>>> off >>>> > > > > > > > >>>>>>> the >>>> > > > > > > > >>>>>>>>>>> release >>>> > > > > > > > >>>>>>>>>>>>>>> process >>>> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do >>>> you >>>> > > > > > > > >>>> think? >>>> > > > > > > > >>>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing >>>> blocker >>>> > > > > > > > >>>> issues >>>> > > > > > > > >>>>>>> need >>>> > > > > > > > >>>>>>>>> to be >>>> > > > > > > > >>>>>>>>>>>>> fixed >>>> > > > > > > > >>>>>>>>>>>>>> in >>>> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks. >>>> > > > > > > > >>>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards, >>>> > > > > > > > >>>>>>>>>>>>>>>>> Yu >>>> > > > > > > > >>>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>>> >>>> > > > > > > > >>>>>>>>>>> >>>> > > > > > > > >>>>>>>>> >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>>> -- >>>> > > > > > > > >>>>>>>> Best, Jingsong Lee >>>> > > > > > > > >>>>>>>> >>>> > > > > > > > >>>>>>> >>>> > > > > > > > >>>>>> >>>> > > > > > > > >>>>> >>>> > > > > > > > >>>> >>>> > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > > >>>> > > > > > > > >>>> > > > > > > >>>> > > > > > >>>> > > > > >>>> > > > >>>> > > >>>> > >>>> >>>
