Another blocker for 1.10.1: FLINK-16576 State inconsistency on restore with memory state backends
Let me recompile the watching list with recent feedbacks. There're totally 45 issues with Blocker/Critical priority for 1.10.1, out of which 14 already resolved and 31 left, and the below ones are watched (meaning that once the below ones got fixed, we will kick of the releasing with left ones postponed to 1.10.2 unless objections): * Blockers (7) - [Under Discussion] FLINK-16018 Improve error reporting when submitting batch job (instead of AskTimeoutException) - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job submission - [PR Submitted] FLINK-16170 SearchTemplateRequest ClassNotFoundException when use flink-sql-connector-elasticsearch7 - [PR Submitted] FLINK-16262 Class loader problem with FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory - [Closed] FLINK-16406 Increase default value for JVM Metaspace to minimise its OutOfMemoryError - [Open] FLINK-16454 Update the copyright year in NOTICE files - [Open] FLINK-16576 State inconsistency on restore with memory state backends * Critical (4) - [Open] FLINK-16047 Blink planner produces wrong aggregate results with state clean up - [PR Submitted] FLINK-16070 Blink planner can not extract correct unique key for UpsertStreamTableSink - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be handled as Fatal Error in TaskManager - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot Please raise your hand if you find any other issues should be put into this list, thanks. Please also note that 1.10.2 version is already created (thanks for the help @jincheng), and feel free to adjust/assign fix version to it when necessary. Best Regards, Yu On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org> wrote: > @Andrey and @Xintong - could we have a quick poll on the user mailing list > about increasing the metaspace size in Flink 1.10.1? Specifically asking > for who has very small TM setups? > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <car...@gmail.com> wrote: > > > Thanks for the reminder Stephan and the inputs/discussion Andrey and > > Xintong. I totally agree that we should try best not to introduce new > > issues when resolving the existing one. Will watch closely about the > > discussion and conclusion around metaspace configuration. > > > > From the point as a developer instead of RM, I'm optimistic that > reserving > > 160MB more memory to JVM metaspace by default won't cause a big problem, > > based on the assumption that nowadays assigning 2GB/4GB memory per TM is > > the common case. > > > > Best Regards, > > Yu > > > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <tonysong...@gmail.com> > wrote: > > > > > I think in FLINK-16406 we have not only increased the default value for > > > metaspace, but also increased the 'process.size' in default > > > 'flink-conf.yaml'. > > > > > > That means: > > > > > > - For people using the default 'flink-conf.yaml', it's not really > > taking > > > memory from heap / managed for metaspace. > > > - For people using custom configuration files: > > > - If they are specifying 'flink.size', then metaspace does not > > really > > > affect the flink memory internal components (heap, managed, > etc.). > > > - If they are specifying 'process.size' (most likely active > > setups), > > > then they will have less heap / managed memory, or worse, their > > > configuration may not work without adjustment. > > > - For a TM with large process memory (e.g., 4 GB or more), it > is > > > probably ok that the metaspace takes 160 MB more memory. > > > - For a TM with smaller total memory (1 or 2 GB) the metaspace > > > increase is more likely to cause problem. > > > > > > So basically only people have small 'process.size' in custom config > file > > > are really affected. I'm not sure what is the proportion of such use > > cases > > > though. (From questions asked on the user ML, probably not much). > > > > > > Thank you~ > > > > > > Xintong Song > > > > > > > > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <se...@apache.org> > wrote: > > > > > > > No need to revert it now - I am not saying it should not go into > > 1.10.1, > > > I > > > > am just saying this is not clear to me yet. > > > > > > > > We have to trade off the fact that we may break some deployments with > > the > > > > fact that we will "safe" a lot of new deployments. > > > > I simply lack the data points / insight at the moment to understand > how > > > > significant both cases are, meaning how many users would be affected > > and > > > > how badly. > > > > > > > > Independent of that, improving the error message is always helpful. > > > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin < > > > > azagrebin.apa...@gmail.com> > > > > wrote: > > > > > > > > > > - For 1.10.1 I am not completely sure, because users expect to > > > > upgrade > > > > > > that without config adjustments. That might not be possible with > > that > > > > > > change. > > > > > > > > > > Ok, makes sense, I will revert it for 1.10 and only try to improve > > > error > > > > > message and docs. > > > > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org> wrote: > > > > > > > > > > > > @Andrey about the increase in metaspace size > > > > > > - I have no concerns for 1.11.0. > > > > > > - For 1.10.1 I am not completely sure, because users expect to > > > > upgrade > > > > > > that without config adjustments. That might not be possible with > > that > > > > > > change. > > > > > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin < > > > > > azagrebin.apa...@gmail.com> > > > > > > wrote: > > > > > > > > > > > >> > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on > > > repeated > > > > > >> job” > > > > > >> > > > > > >> My understanding that the issue is basically covered by: > > > > > >> > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be handled as > Fatal > > > > Error > > > > > >> in TaskManager > > > > > >> no full consensus there but improving error message for > existing > > > > task > > > > > >> thread fatal handling could be done at least > > > > > >> > > > > > >> - [FLINK-16406] Increase default value for JVM Metaspace to > > minimise > > > > its > > > > > >> OutOfMemoryError > > > > > >> see further > > > > > >> > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from > dynamically > > > > > loaded > > > > > >> AWS connectors > > > > > >> not sure whether this is a blocker but looks close to be > resolved > > > > > >> > > > > > >>> About "FLINK-16406 Increase default value for JVM Metaspace" > > > > > >>> - Have we consensus that this is okay for a bugfix release? It > > > > changes > > > > > >>> setups, takes away memory from heap / managed memory on > existing > > > > setups > > > > > >>> that keep their flink-conf.yaml. > > > > > >> > > > > > >> My understanding was that increasing to 256m resolved the > reported > > > > > problems > > > > > >> and we decided to make the change so I have merged it today as > > there > > > > > were > > > > > >> no more concerns. > > > > > >> If there are concerns I can revert it. > > > > > >> > > > > > >> On the other hand, I think improving the message error with > > > reference > > > > to > > > > > >> the metaspace option should help the most > > > > > >> because user would not have to read all docs to fix it > > > > > >> then maybe this change is not even needed. > > > > > >> > > > > > >> Best, > > > > > >> Andrey > > > > > >> > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org> > wrote: > > > > > >>> > > > > > >>> Good idea to go ahead with 1.10.1 > > > > > >>> > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on > > > repeated > > > > > >> job" > > > > > >>> - I don't think we have consensus on the exact solution, yet, > and > > > > some > > > > > >> of > > > > > >>> the changes might also have side effects that are hard to > > predict, > > > > so I > > > > > >> am > > > > > >>> not sure we should rush this in. > > > > > >>> > > > > > >>> About "FLINK-16406 Increase default value for JVM Metaspace" > > > > > >>> - Have we consensus that this is okay for a bugfix release? It > > > > changes > > > > > >>> setups, takes away memory from heap / managed memory on > existing > > > > setups > > > > > >>> that keep their flink-conf.yaml. > > > > > >>> > > > > > >>> We may need to unblock the release form these two issues and > > think > > > > > about > > > > > >>> having 1.10.2 in the near future. > > > > > >>> > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <car...@gmail.com> > wrote: > > > > > >>> > > > > > >>>> Thanks for the reminder Jark. Will keep an eye on these two. > > > > > >>>> > > > > > >>>> Best Regards, > > > > > >>>> Yu > > > > > >>>> > > > > > >>>> > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <imj...@gmail.com> > wrote: > > > > > >>>> > > > > > >>>>> Thanks for driving this release, Yu! > > > > > >>>>> +1 to start 1.10.1 release cycle. > > > > > >>>>> > > > > > >>>>> From the Table SQL module, I think we should also try to get > in > > > the > > > > > >>>>> following issues: > > > > > >>>>> - FLINK-16441: Allow users to override flink-conf parameters > > from > > > > SQL > > > > > >> CLI > > > > > >>>>> environment > > > > > >>>>> this allows users to set e.g. statebackend, watermark > interval, > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI > > > > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched > exception > > is > > > > > >> thrown > > > > > >>>>> this will improve much experience when using SQL CLI > > > > > >>>>> > > > > > >>>>> Best, > > > > > >>>>> Jark > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <car...@gmail.com> > wrote: > > > > > >>>>> > > > > > >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into > > > > FLINK-16225 > > > > > >>>> fix > > > > > >>>>>> versions and promoted its priority to Critical. Will also > > watch > > > > the > > > > > >>>>>> progress of FLINK-16108/FLINK-16408. > > > > > >>>>>> > > > > > >>>>>> Best Regards, > > > > > >>>>>> Yu > > > > > >>>>>> > > > > > >>>>>> > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin < > > > > azagre...@apache.org > > > > > > > > > > > >>>>>> wrote: > > > > > >>>>>> > > > > > >>>>>>> Hi Yu, > > > > > >>>>>>> > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release discussion! > > > > > >>>>>>> > > > > > >>>>>>> Apart from > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM Metaspace to > > > > minimise > > > > > >>>> its > > > > > >>>>>>> OutOfMemoryError > > > > > >>>>>>> which should be merged soon > > > > > >>>>>>> > > > > > >>>>>>> I think we should also try to get in the following issues: > > > > > >>>>>>> > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be handled > as > > > > Fatal > > > > > >>>>> Error > > > > > >>>>>> in > > > > > >>>>>>> TaskManager > > > > > >>>>>>> This should solve the Metaspace problem even in a better > way > > > > > because > > > > > >>>>> OOM > > > > > >>>>>>> failure should point users to the docs immediately > > > > > >>>>>>> > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to lifetime of > a > > > slot > > > > > >>>>>>> This should give a better protection against class loading > > > leaks > > > > > >>>>>>> > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when submitting > batch > > > job > > > > > >>>>>> (instead > > > > > >>>>>>> of AskTimeoutException) > > > > > >>>>>>> This problem has recently happened for multiple users > > > > > >>>>>>> > > > > > >>>>>>> Best, > > > > > >>>>>>> Andrey > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li < > > > > > jingsongl...@gmail.com> > > > > > >>>>>>> wrote: > > > > > >>>>>>> > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1 > release. > > > > > >>>>>>>> > > > > > >>>>>>>> Some issues are very important, Users are looking forward > to > > > > them. > > > > > >>>>>>>> > > > > > >>>>>>>> Best, > > > > > >>>>>>>> Jingsong Lee > > > > > >>>>>>>> > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo < > > > karma...@gmail.com> > > > > > >>>>> wrote: > > > > > >>>>>>>> > > > > > >>>>>>>>> Thanks for driving this release, Yu! > > > > > >>>>>>>>> > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle. > > > > > >>>>>>>>> > > > > > >>>>>>>>> Best, > > > > > >>>>>>>>> Yangze Guo > > > > > >>>>>>>>> > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song < > > > > > >>>>> tonysong...@gmail.com> > > > > > >>>>>>>>> wrote: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Yu, > > > > > >>>>>>>>>> Thanks for the explanation. > > > > > >>>>>>>>>> I've no concerns. I was just trying to get some inputs > for > > > > > >>>>>>> prioritizing > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to me. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Thank you~ > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Xintong Song > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li < > car...@gmail.com> > > > > > >>>> wrote: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1? > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> According to the history, the first patch release of a > > > major > > > > > >>>>>>> version > > > > > >>>>>>>>> will > > > > > >>>>>>>>>>> take ~1month from discussion started, depending on the > > > speed > > > > > >>>> of > > > > > >>>>>>>> blocker > > > > > >>>>>>>>>>> issue resolving: > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1], released > on > > > Jul > > > > > >>>>> 3rd > > > > > >>>>>>> [2] > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], released > on > > > Oct > > > > > >>>>> 19th > > > > > >>>>>>> [4] > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> We won't rush to match the history of course, but could > > use > > > > > >>>> it > > > > > >>>>>> as a > > > > > >>>>>>>>>>> reference. And please feel free to let me know if any > > > > > >>>> concerns > > > > > >>>>>>>> Xintong. > > > > > >>>>>>>>>>> Thanks. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> Best Regards, > > > > > >>>>>>>>>>> Yu > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> [1] > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>>> > > > > > >> > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html > > > > > >>>>>>>>>>> [2] > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>>> > > > > > >> > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html > > > > > >>>>>>>>>>> [3] > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>>> > > > > > >> > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html > > > > > >>>>>>>>>>> [4] > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>>> > > > > > >> > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song < > > > > > >>>>>> tonysong...@gmail.com> > > > > > >>>>>>>>> wrote: > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering to be the > > > > > >>>>> release > > > > > >>>>>>>>> manager. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> +1 for the proposal. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> One quick question, what is the time plan for 1.10.1? > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Thank you~ > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Xintong Song > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang > > > > > >>>>>>>>>>>> <wangzhijiang...@aliyun.com.invalid> wrote: > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu! > > > > > >>>>>>>>>>>>> +1 on my side > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Best, > > > > > >>>>>>>>>>>>> Zhijiang > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>> > > > > ------------------------------------------------------------------ > > > > > >>>>>>>>>>>>> From:Yu Li <car...@gmail.com> > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25 > > > > > >>>>>>>>>>>>> To:dev <dev@flink.apache.org> > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1 > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also keep > an > > > > > >>>> eye > > > > > >>>>> on > > > > > >>>>>>>> these > > > > > >>>>>>>>>>>>> existing blocker issues. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Best Regards, > > > > > >>>>>>>>>>>>> Yu > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng < > > > > > >>>>> he...@apache.org> > > > > > >>>>>>>>> wrote: > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Hi Yu, > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and > volunteer > > > > > >>>> as > > > > > >>>>>> the > > > > > >>>>>>>>> release > > > > > >>>>>>>>>>>>>> manager! > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> I found there are some other issues[1] which are > > marked > > > > > >>>>> as > > > > > >>>>>> a > > > > > >>>>>>>>> blocker: > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in NOTICE > > files > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with > > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib > > > > > >>>>>> directory > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest > > > > > >>>>> ClassNotFoundException > > > > > >>>>>>> when > > > > > >>>>>>>>> use > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7 > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when > submitting > > > > > >>>>> batch > > > > > >>>>>>> job > > > > > >>>>>>>>>>>> (instead > > > > > >>>>>>>>>>>>> of > > > > > >>>>>>>>>>>>>> AskTimeoutException) > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> These may also need to be resolved in 1.10.1. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Best, > > > > > >>>>>>>>>>>>>> Hequn > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> [1] > > > > > >>>>>>>>> > > > > https://issues.apache.org/jira/projects/FLINK/versions/12346891 > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li < > > > > > >>>> car...@gmail.com> > > > > > >>>>>>>> wrote: > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Hi Jincheng, > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a lot! > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Best Regards, > > > > > >>>>>>>>>>>>>>> Yu > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun < > > > > > >>>>>>>>>>> sunjincheng...@gmail.com > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would > like > > > > > >>>>> to > > > > > >>>>>>> give > > > > > >>>>>>>>> you a > > > > > >>>>>>>>>>>>> hand > > > > > >>>>>>>>>>>>>> at > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is finished.(If you > > > > > >>>> need) > > > > > >>>>>> :) > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Best, > > > > > >>>>>>>>>>>>>>>> Jincheng > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Yu Li <car...@gmail.com> 于2020年3月10日周二 下午5:49写道： > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Hi All, > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we released > > > > > >>>>> Flink > > > > > >>>>>>>>> 1.10.0. We > > > > > >>>>>>>>>>>>>> already > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs in > > > > > >>>> the > > > > > >>>>>>>>>>> release-1.10 > > > > > >>>>>>>>>>>>>>> branch, > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release cycle. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are: > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and > > > > > >>>> notice > > > > > >>>>>>> file > > > > > >>>>>>>> in > > > > > >>>>>>>>>>>>>>> flink-ml-lib > > > > > >>>>>>>>>>>>>>>>> module > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in > > > > > >>>>>>>>>>>>> flink-state-processor-api > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known in > > > > > >>>>>>>>> HiveCatalog > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in > > > > > >>>>>>>> TypeExtractor > > > > > >>>>>>>>>>> when > > > > > >>>>>>>>>>>>>>> mapping > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if > > > > > >>>> running > > > > > >>>>>> in > > > > > >>>>>>>>> blink > > > > > >>>>>>>>>>>>> planner > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not > > > > > >>>> reset > > > > > >>>>>> on > > > > > >>>>>>>> task > > > > > >>>>>>>>>>>>> recovery > > > > > >>>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using sql > > > > > >>>>>>> casuing > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation failed > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following issues should > > > > > >>>> be > > > > > >>>>>>>> merged > > > > > >>>>>>>>>>> before > > > > > >>>>>>>>>>>>>>> 1.10.1 > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM issue): > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace OOM > > > > > >>>>> error > > > > > >>>>>> on > > > > > >>>>>>>>>>> repeated > > > > > >>>>>>>>>>>>> job > > > > > >>>>>>>>>>>>>>>>> submission > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM > > > > > >>>>>> Metaspace > > > > > >>>>>>> to > > > > > >>>>>>>>>>>> minimise > > > > > >>>>>>>>>>>>>> its > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong > > > > > >>>>> aggregate > > > > > >>>>>>>>> results > > > > > >>>>>>>>>>> with > > > > > >>>>>>>>>>>>>> state > > > > > >>>>>>>>>>>>>>>>> clean up > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract > > > > > >>>> correct > > > > > >>>>>>>> unique > > > > > >>>>>>>>> key > > > > > >>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release manager and kick > > > > > >>>>> off > > > > > >>>>>>> the > > > > > >>>>>>>>>>> release > > > > > >>>>>>>>>>>>>>> process > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do you > > > > > >>>> think? > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing blocker > > > > > >>>> issues > > > > > >>>>>>> need > > > > > >>>>>>>>> to be > > > > > >>>>>>>>>>>>> fixed > > > > > >>>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Best Regards, > > > > > >>>>>>>>>>>>>>>>> Yu > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>>> -- > > > > > >>>>>>>> Best, Jingsong Lee > > > > > >>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>>> > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > >
