FLINK-20033 has been merged. Cheers, Till
On Mon, Nov 9, 2020 at 9:39 AM Till Rohrmann <trohrm...@apache.org> wrote: > I would like to bring one more critical issue to your attention which is > FLINK-20033 [1]. I believe that this issue is actually causing what has > been reported in FLINK-19816 [2]. I hope to have it fixed by the end of > today. Once FLINK-20033 is fixed, I think that we don't have to block the > release on FLINK-19816. > > [1] https://issues.apache.org/jira/browse/FLINK-20033 > [2] https://issues.apache.org/jira/browse/FLINK-19816 > > Cheers, > Till > > On Mon, Nov 9, 2020 at 4:05 AM Xintong Song <tonysong...@gmail.com> wrote: > >> Hi devs, >> >> I'd like to provide an update on the progress of preparing release 1.11.3. >> >> *Blockers* >> We currently have 3 remaining blockers. (3 resolved and 1 emerged compared >> to last week) >> >> - [FLINK-19698] Add close() method and onCheckpointComplete() to the >> Source. >> The issue has been fixed on the master branch. It's currently blocked on >> the FLIP-27 backportings to backport it to the 1.11 branch. >> >> - [FLINK-19717] SourceReaderBase.pollNext may return END_OF_INPUT if >> SplitReader.fetch throws >> A PR has been opened and reviewed. From the discussions on the PR, it >> looks >> close to mergeable. >> >> - [FLINK-19816] Flink restored from a wrong checkpoint (a very old one and >> not the last completed one) >> This is a newly emerged blocker and Matthias is working on it. >> >> *Test Instabilities* >> We currently have 27 test instabilities[1]. >> AFAIK, none of them are as serious as to block the 1.11.3 release. >> >> *FLIP-27 Backprotings* >> >> I noticed that there's no jira issues opened on the FLIP-27 backporting >> efforts, which is part of the major efforts planned for the 1.11.3 >> release, >> making it hard to track the progress. >> >> >> @Stephan and @Becket, could you please share the updates on the >> backporting >> efforts? How is the progress and when are the efforts expected to be >> finished? It would be appreciated and helpful if we can have a jira ticket >> to track the backporing progress publicly. >> >> >> Thank you~ >> >> Xintong Song >> >> >> [1] >> >> https://issues.apache.org/jira/browse/FLINK-16908?filter=12348580&jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20affectedVersion%20in%20(1.11.0%2C%201.11.1%2C%201.11.2%2C%201.11.3)%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20updated%20DESC%2C%20created%20DESC >> >> On Wed, Nov 4, 2020 at 3:58 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org> >> wrote: >> >> > Thanks for the efforts so far with pushing for 1.11.3. >> > >> > I'd like to provide a mid-week update on how we're looking with all the >> > pending blockers and backports: >> > >> > *Blockers**:* >> > >> > Currently all blockers either have an open PR, or have been merged. I'll >> > highlight below the blockers *that still require reviewing efforts for >> them >> > to move forward:* >> > >> > - [FLINK-19909] Flink application in attach mode could not terminate >> when >> > the only job is canceled. PR: >> https://github.com/apache/flink/pull/13911 >> > - [FLINK-19717] SourceReaderBase.pollNext may return END_OF_INPUT if >> > SplitReader.fetch throws. PR: >> https://github.com/apache/flink/pull/13776 >> > >> > The above PRs currently have no reviews at all yet. They seem to already >> > have designated reviewers. >> > >> > >> > *Backports:* >> > - FLIP-27 robustness improvement backports: Stephan is currently >> working on >> > backporting several FLIP-27 changes. There are no PRs yet for the >> > backported changes. >> > - DataStreamUtils.collect() refactorings backport: Steven Wu mentioned >> to >> > backport this, but AFAIK this isn't assigned to anyone yet. >> > >> > *ETAs*: >> > >> > I'd like to request ETAs for the remaining backports, to prevent a >> creep in >> > the scope of this bugfix release. >> > >> > We already have Flink users that would benefit from fixes that have been >> > merged to release-1.11 already, so technically speaking the backports >> > should be considered "nice-to-have" (to the best of my knowledge of the >> > changes) and could potentially be moved to a follow-up 1.11.4. >> > Most notably, the Stateful Functions project is already waiting on Flink >> > 1.11.3 to address critical recovery issues (please see the StateFun >> 2.2.1 >> > release discussion thread [1]). >> > >> > @Stephan Ewen <se...@apache.org> @Becket Qin <becket....@gmail.com> >> could >> > you provide an ETA for the FLIP-27 backports? It would help to get a >> better >> > estimate to decide how we proceed here. >> > >> > Cheers, >> > Gordon >> > >> > [1] >> > >> > >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-StateFun-hotfix-version-2-2-1-td46239.html >> > >> > On Wed, Nov 4, 2020 at 3:16 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org >> > >> > wrote: >> > >> > > > The collect() utils can be picked back, I see no issue with that >> (it is >> > > isolated utilities). >> > > >> > > Just checking on all the requested backports mentioned in this thread, >> > and >> > > figuring out which ones seem to still be unassigned / open. >> > > >> > > Is someone working on backporting >> > > >> > >> https://github.com/apache/flink/commit/09a7a66b7313fea64817fe960a8da1265b428efc >> > > to release-1.11 at the moment? >> > > >> > > On Wed, Nov 4, 2020 at 1:39 AM Steven Wu <stevenz...@gmail.com> >> wrote: >> > > >> > >> @Stephan Ewen <se...@apache.org> yeah, we can do that. don't worry >> > about >> > >> it. your earlier email had the perfect explanation on why file source >> > >> shouldn't be backported. >> > >> >> > >> On Tue, Nov 3, 2020 at 3:37 AM Stephan Ewen <se...@apache.org> >> wrote: >> > >> >> > >> > @Steven would it be possible to initially copy some of the code >> into >> > the >> > >> > iceberg source and later replace it by a dependency on the Flink >> file >> > >> > source? >> > >> > >> > >> > On Mon, Nov 2, 2020 at 8:33 PM Steven Wu <stevenz...@gmail.com> >> > wrote: >> > >> > >> > >> > > Stephan, thanks a lot for explaining the file connector. that >> makes >> > >> > sense. >> > >> > > >> > >> > > I was asking because we were trying to reuse some of the >> > >> implementations >> > >> > in >> > >> > > the file source for Iceberg source. Flink Iceberg source lives in >> > the >> > >> > > Iceberg repo, which is not possible to code against the master >> > branch >> > >> of >> > >> > > the Flink code. >> > >> > > >> > >> > > On Mon, Nov 2, 2020 at 3:31 AM Stephan Ewen <se...@apache.org> >> > wrote: >> > >> > > >> > >> > > > Hi Steven! >> > >> > > > >> > >> > > > So far there are no plans to pick back the file system >> connector >> > >> code. >> > >> > > This >> > >> > > > is still evolving and not finalized for 1.12, so I don't feel >> it >> > is >> > >> a >> > >> > > good >> > >> > > > candidate to be backported. >> > >> > > > However, with the base connector changes backported, you >> should be >> > >> able >> > >> > > to >> > >> > > > run the file connector code from master against 1.11.3. >> > >> > > > >> > >> > > > The collect() utils can be picked back, I see no issue with >> that >> > >> (it is >> > >> > > > isolated utilities). >> > >> > > > >> > >> > > > Best, >> > >> > > > Stephan >> > >> > > > >> > >> > > > >> > >> > > > On Mon, Nov 2, 2020 at 3:02 AM Steven Wu <stevenz...@gmail.com >> > >> > >> wrote: >> > >> > > > >> > >> > > > > Basically, it would be great to get the latest code in the >> > >> > > > > flink-connector-files (FLIP-27). >> > >> > > > > >> > >> > > > > On Sat, Oct 31, 2020 at 9:57 AM Steven Wu < >> stevenz...@gmail.com >> > > >> > >> > > wrote: >> > >> > > > > >> > >> > > > > > Stephan, it will be great if we can also backport the >> > >> > DataStreamUtils >> > >> > > > > > related commits that help with collecting output from >> > unbounded >> > >> > > > streams. >> > >> > > > > > e.g. >> > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> https://github.com/apache/flink/commit/09a7a66b7313fea64817fe960a8da1265b428efc >> > >> > > > > > >> > >> > > > > > I tried to copy and paste the code to unblock myself. but >> it >> > >> > quickly >> > >> > > > got >> > >> > > > > > into the rabbit hole of more and more code. >> > >> > > > > > >> > >> > > > > > On Fri, Oct 30, 2020 at 11:02 AM Stephan Ewen < >> > se...@apache.org >> > >> > >> > >> > > > wrote: >> > >> > > > > > >> > >> > > > > >> I have started with backporting the source API changes. >> Some >> > >> minor >> > >> > > > > >> conflicts to solve, will need a bit more to finish this. >> > >> > > > > >> >> > >> > > > > >> On Fri, Oct 30, 2020 at 7:25 AM Tzu-Li (Gordon) Tai < >> > >> > > > > tzuli...@apache.org> >> > >> > > > > >> wrote: >> > >> > > > > >> >> > >> > > > > >> > @Stephan Ewen <se...@apache.org> >> > >> > > > > >> > Are there already plans or ongoing efforts for >> backporting >> > >> the >> > >> > > list >> > >> > > > of >> > >> > > > > >> > FLIP-27 changes that you posted? >> > >> > > > > >> > >> > >> > > > > >> > On Thu, Oct 29, 2020 at 7:08 PM Xintong Song < >> > >> > > tonysong...@gmail.com >> > >> > > > > >> > >> > > > > >> > wrote: >> > >> > > > > >> > >> > >> > > > > >> >> Hi folks, >> > >> > > > > >> >> >> > >> > > > > >> >> Just to provide some updates concerning the status on >> the >> > >> > > > > >> >> test instabilities. >> > >> > > > > >> >> >> > >> > > > > >> >> Currently, we have 30 unresolved tickets labeled with >> > >> `Affects >> > >> > > > > Version` >> > >> > > > > >> >> 1.11.x. >> > >> > > > > >> >> >> > >> > > > > >> >> >> > >> > > > > >> >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> https://issues.apache.org/jira/browse/FLINK-19775?filter=12348580&jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20affectedVersion%20in%20(1.11.0%2C%201.11.1%2C%201.11.2%2C%201.11.3)%20AND%20labels%20%3D%20test-stability%20ORDER%20BY%20created%20DESC >> > >> > > > > >> >> >> > >> > > > > >> >> Among the 30 tickets, 11 of them are: >> > >> > > > > >> >> - Have occured in the recent 3 months >> > >> > > > > >> >> - Not confirmed to be pure testability issues >> > >> > > > > >> >> - Not confirmed to be rare condition cases >> > >> > > > > >> >> >> > >> > > > > >> >> It would be nice if someone familiar with these >> components >> > >> can >> > >> > > > take a >> > >> > > > > >> look >> > >> > > > > >> >> into these issues. >> > >> > > > > >> >> >> > >> > > > > >> >> - https://issues.apache.org/jira/browse/FLINK-17159 >> (ES6) >> > >> > > > > >> >> - https://issues.apache.org/jira/browse/FLINK-17912 >> > (Kafka) >> > >> > > > > >> >> - https://issues.apache.org/jira/browse/FLINK-17949 >> > (Kafka) >> > >> > > > > >> >> ⁃ https://issues.apache.org/jira/browse/FLINK-18444 >> > (Kafka) >> > >> > > > > >> >> - https://issues.apache.org/jira/browse/FLINK-18634 >> > (Kafka) >> > >> > > > > >> >> - https://issues.apache.org/jira/browse/FLINK-18648 >> > (Kafka) >> > >> > > > > >> >> - https://issues.apache.org/jira/browse/FLINK-18807 >> > (Kafka) >> > >> > > > > >> >> - https://issues.apache.org/jira/browse/FLINK-19369 >> > >> > > > (BlobClientTest) >> > >> > > > > >> >> - https://issues.apache.org/jira/browse/FLINK-19436 >> > (TPCDS) >> > >> > > > > >> >> - https://issues.apache.org/jira/browse/FLINK-19690 >> > >> > > > (Format/Parquet) >> > >> > > > > >> >> - https://issues.apache.org/jira/browse/FLINK-19775 >> > >> > > > > >> >> (SystemProcessingTimeServiceTest) >> > >> > > > > >> >> >> > >> > > > > >> >> Thank you~ >> > >> > > > > >> >> >> > >> > > > > >> >> Xintong Song >> > >> > > > > >> >> >> > >> > > > > >> >> >> > >> > > > > >> >> >> > >> > > > > >> >> On Thu, Oct 29, 2020 at 10:21 AM Jingsong Li < >> > >> > > > jingsongl...@gmail.com >> > >> > > > > > >> > >> > > > > >> >> wrote: >> > >> > > > > >> >> >> > >> > > > > >> >> > +1 to backport the FLIP-27 adjustments to 1.11.x. >> > >> > > > > >> >> > >> > >> > > > > >> >> > If possible, that would be great. Many people are >> > looking >> > >> > > forward >> > >> > > > > to >> > >> > > > > >> the >> > >> > > > > >> >> > FLIP-27 interface, but they don't want to take the >> risk >> > to >> > >> > > > upgrade >> > >> > > > > to >> > >> > > > > >> >> 1.12 >> > >> > > > > >> >> > (And wait 1.12). After all, 1.11 is a relatively >> stable >> > >> > > version. >> > >> > > > > >> >> > >> > >> > > > > >> >> > Best, >> > >> > > > > >> >> > Jingsong >> > >> > > > > >> >> > >> > >> > > > > >> >> > On Thu, Oct 29, 2020 at 1:24 AM Stephan Ewen < >> > >> > se...@apache.org >> > >> > > > >> > >> > > > > >> wrote: >> > >> > > > > >> >> > >> > >> > > > > >> >> > > Thanks for starting this. >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > +1 form my side to backport the FLIP-27 >> adjustments to >> > >> > > 1.11.x. >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > There were quite a few changes, and I think we >> need to >> > >> > > > > cherry-pick >> > >> > > > > >> >> them >> > >> > > > > >> >> > all >> > >> > > > > >> >> > > to not get some inconsistent mix of changes and >> many >> > >> merge >> > >> > > > > >> conflicts. >> > >> > > > > >> >> > > I made a list below of what we need to add to >> > >> > "release-1.11". >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > *#### Core Source API Changes to backport (in >> REVERSE >> > >> > order)* >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > (Use: "git log >> > >> > > > > >> >> > > >> > >> > > > >> flink-core/src/main/java/org/apache/flink/api/connector/source") >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 162c072e9265a7b6dd9d6f5459eb7974091c4c4e >> > >> > > > > >> >> > > [FLINK-19492][core] Consolidate Source Events >> between >> > >> > Source >> > >> > > > API >> > >> > > > > >> and >> > >> > > > > >> >> > Split >> > >> > > > > >> >> > > Reader API >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 428fbbad85379f71a9fbf1a1ab3e3dcfa9006436 >> > >> > > > > >> >> > > [FLINK-19205][core] Add access to configuration and >> > >> > hostname >> > >> > > in >> > >> > > > > the >> > >> > > > > >> >> > > SourceReaderContext >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 723e1790744ddc78d19c7c978442af1383f38d33 >> > >> > > > > >> >> > > [hotfix][core] Add to Source Enumerator convenience >> > >> methods >> > >> > > to >> > >> > > > > >> assign >> > >> > > > > >> >> > > single split >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > *#### Connector Base Changes to Backport (in >> REVERSE >> > >> > order)* >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > (Use: "git log >> > flink-connectors/flink-connector-base") >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 401f56fe9d6b0271260edf9787cdcbfe4d03874d >> > >> > > > > >> >> > > [FLINK-19427][FLINK-19489][tests] Fix test >> conditions >> > >> for >> > >> > > > > >> >> > > >> > 'SplitFetcherTest.testNotifiesWhenGoingIdleConcurrent()' >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 68c5c2ff779d82a1ff81ffaf60d8a1b283797db1 >> > >> > > > > >> >> > > [FLINK-19448][connector base] Explicitly check for >> > >> > > un-expected >> > >> > > > > >> >> condition >> > >> > > > > >> >> > > that would leave an inconsistent state >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 162c072e9265a7b6dd9d6f5459eb7974091c4c4e >> > >> > > > > >> >> > > [FLINK-19492][core] Consolidate Source Events >> between >> > >> > Source >> > >> > > > API >> > >> > > > > >> and >> > >> > > > > >> >> > Split >> > >> > > > > >> >> > > Reader API >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit c1ca7a4c7c21ec8868c14cdddd43c559625b794c >> > >> > > > > >> >> > > [refactor][tests] Move some source test utils from >> > >> > > > > >> >> flink-connector-base >> > >> > > > > >> >> > to >> > >> > > > > >> >> > > flink-core >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit ee5c4c211c35c70d28252363bbc8400453609977 >> > >> > > > > >> >> > > [FLINK-19251][connectors] Avoid confusing queue >> > >> handling in >> > >> > > > > >> >> > > "SplitReader.handleSplitsChanges()" >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 5abef56b2bf85bcac786f6b16b6899b6cced7176 >> > >> > > > > >> >> > > [FLINK-19250][connectors] Fix error propagation in >> > >> > connector >> > >> > > > base >> > >> > > > > >> >> > > (SplitFetcherManager). >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 8fcca837c55a9216595ee4c03038b52747098dbb >> > >> > > > > >> >> > > [hotfix][connectors] Improve JavaDocs for >> > >> > > > > >> SingleThreadFetcherManager >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 4700bb5dde3303cbe98882f6beb7379425717b01 >> > >> > > > > >> >> > > [FLINK-19225][connectors] Various small >> improvements >> > to >> > >> > > > > >> >> SourceReaderBase >> > >> > > > > >> >> > > (part 2) >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 12261c6b7ed6478a9b9f6a69cb58246b83cab9b7 >> > >> > > > > >> >> > > [FLINK-17393][connectors] (follow-up) Wakeup the >> > >> > > SplitFetchers >> > >> > > > > more >> > >> > > > > >> >> > > elegantly. >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit c60aaff0249bfd6b5871b7f82e03efc487a54d6b >> > >> > > > > >> >> > > [hotfix][tests] Extend test coverage for >> > >> > > > > >> FutureCompletingBlockingQueue >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit cef8a587d7fd2fe64cc644da5ed095d82e46f631 >> > >> > > > > >> >> > > [FLINK-19245][connectors] Set default capacity for >> > >> > > > > >> >> > > FutureCompletingBlockingQueue. >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 4ea95782b4c6a2538153d4d16ad3f4839c7de0fb >> > >> > > > > >> >> > > [FLINK-19223][connectors] Simplify Availability >> Future >> > >> > Model >> > >> > > in >> > >> > > > > >> Base >> > >> > > > > >> >> > > Connector >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 511857049ba30c8ff0ee56da551fa4a479dc583e >> > >> > > > > >> >> > > [FLINK-18128][connectors] Ensure idle split >> fetchers >> > >> lead >> > >> > to >> > >> > > > > >> >> availability >> > >> > > > > >> >> > > notifications. >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit a8206467af0830dcb89623ea068b5ca3b3450c92 >> > >> > > > > >> >> > > [refactor][core] Eagerly initialize the FetchTask >> to >> > >> > support >> > >> > > > > proper >> > >> > > > > >> >> unit >> > >> > > > > >> >> > > testing >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 3b2f54bcb437f98e6137c904045cc51072b5c06b >> > >> > > > > >> >> > > [hotfix][tests] Move constants in SplitFetcherTest >> > >> relevant >> > >> > > to >> > >> > > > > only >> > >> > > > > >> >> one >> > >> > > > > >> >> > > test into test method >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit d7625760a75a508bf05bcddc380bb4d62ee1743e >> > >> > > > > >> >> > > [FLINK-19225][connectors] Various small >> improvements >> > to >> > >> > > > > >> >> SourceReaderBase >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit a5b0d3297748c1be47ad579a88f24df2255a8df1 >> > >> > > > > >> >> > > [FLINK-17393][connectors] Wakeup the SplitFetchers >> > more >> > >> > > > > elegantly. >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit f42a3ebc3e81a034b7221a803c153636fef34903 >> > >> > > > > >> >> > > [FLINK-18680][connectors] Make connector base >> > >> > > > RecordsWithSplitIds >> > >> > > > > >> more >> > >> > > > > >> >> > > lightweight. >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit e3d273de822b085183d09b275a445879ff94b350 >> > >> > > > > >> >> > > [FLINK-19162][connectors] Add 'recycle()' to the >> > >> > > > > >> RecordsWithSplitIds >> > >> > > > > >> >> to >> > >> > > > > >> >> > > support reuse of heavy objects. >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 8ebc464c2520453a70001cd712abc8dee6ee89e0 >> > >> > > > > >> >> > > [hotfix][testing] Add a set of parameterizable >> testing >> > >> > mocks >> > >> > > > for >> > >> > > > > >> the >> > >> > > > > >> >> > Split >> > >> > > > > >> >> > > Reader API >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit 930a07438be1185388d7150640f294dfe2a5d378 >> > >> > > > > >> >> > > [hotfix][connectors] Add RequestSplitEvent to >> > >> > > > > >> 'flink-connector-base' >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > commit e945ce8a933bc378844782f784ca473c767ca159 >> > >> > > > > >> >> > > [refactor][connectors] Backport of the >> connector-base >> > >> > > exception >> > >> > > > > >> >> handling >> > >> > > > > >> >> > > from the Kafka Connector Pull Request >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > On Wed, Oct 28, 2020 at 5:49 PM Steven Wu < >> > >> > > > stevenz...@gmail.com> >> > >> > > > > >> >> wrote: >> > >> > > > > >> >> > > >> > >> > > > > >> >> > > > I would love to see this FLIP-27 source interface >> > >> > > improvement >> > >> > > > > [1] >> > >> > > > > >> >> made >> > >> > > > > >> >> > to >> > >> > > > > >> >> > > > 1.11.3. >> > >> > > > > >> >> > > > >> > >> > > > > >> >> > > > [1] >> > https://issues.apache.org/jira/browse/FLINK-19698 >> > >> > > > > >> >> > > > >> > >> > > > > >> >> > > > On Wed, Oct 28, 2020 at 12:32 AM Tzu-Li (Gordon) >> > Tai < >> > >> > > > > >> >> > > tzuli...@apache.org> >> > >> > > > > >> >> > > > wrote: >> > >> > > > > >> >> > > > >> > >> > > > > >> >> > > > > Thanks for the replies so far! >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > Just to provide a brief update on the status of >> > >> > blockers >> > >> > > > for >> > >> > > > > >> >> 1.11.3 >> > >> > > > > >> >> > so >> > >> > > > > >> >> > > > far: >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > *PR opened, pending reviewer*- [FLINK-19717] >> > >> > > > > >> >> > SourceReaderBase.pollNext >> > >> > > > > >> >> > > > may >> > >> > > > > >> >> > > > > return END_OF_INPUT if SplitReader.fetch >> throws ( >> > >> > > > > >> >> > > > > https://github.com/apache/flink/pull/13776) >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > *PR opened, reviewed + close to being merged* >> > >> > > > > >> >> > > > > - [FLINK-19741] Timer service should skip >> > restoring >> > >> > from >> > >> > > > raw >> > >> > > > > >> keyed >> > >> > > > > >> >> > > stream >> > >> > > > > >> >> > > > > if it isn't the writer ( >> > >> > > > > >> >> https://github.com/apache/flink/pull/13761) >> > >> > > > > >> >> > > > > - [FLINK-19748] Raw keyed stream key group >> > iterator >> > >> > > should >> > >> > > > be >> > >> > > > > >> >> > skipping >> > >> > > > > >> >> > > > > unwritten key groups ( >> > >> > > > > >> https://github.com/apache/flink/pull/13772) >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > *Merged* >> > >> > > > > >> >> > > > > - [FLINK-19154] Application mode deletes HA >> data >> > in >> > >> > case >> > >> > > of >> > >> > > > > >> >> suspended >> > >> > > > > >> >> > > > > ZooKeeper connection >> > >> > > > > >> >> > > > > - [FLINK-19569] Upgrade ICU4J to 67.1+ >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > Right now as it seems, progress is mainly >> blocked >> > >> on a >> > >> > > > > reviewer >> > >> > > > > >> >> for >> > >> > > > > >> >> > > > > FLINK-19717. >> > >> > > > > >> >> > > > > Meanwhile, Xintong is keeping an eye on test >> > >> > > instabilities >> > >> > > > > [1] >> > >> > > > > >> to >> > >> > > > > >> >> see >> > >> > > > > >> >> > > if >> > >> > > > > >> >> > > > > there are any fixes that should be applied to >> > >> > > > `release-1.11`. >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > This is also a reminder, that if there are >> other >> > >> > blockers >> > >> > > > > that >> > >> > > > > >> we >> > >> > > > > >> >> > need >> > >> > > > > >> >> > > to >> > >> > > > > >> >> > > > > be aware of, or a need to re-establish >> estimated >> > >> time >> > >> > for >> > >> > > > > >> getting >> > >> > > > > >> >> > fixes >> > >> > > > > >> >> > > > in >> > >> > > > > >> >> > > > > and delay the RC for 1.11.3, please do let us >> > know! >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > Cheers, >> > >> > > > > >> >> > > > > Gordon >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > [1] >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > >> > >> > > > > >> >> > > >> > >> > > > > >> >> > >> > >> > > > > >> >> >> > >> > > > > >> >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20affectedVersion%20in%20(1.11.0%2C%201.11.1%2C%201.11.2)%20AND%20labels%20%3D%20test-stability >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > On Mon, Oct 26, 2020 at 9:43 PM Kostas Kloudas >> < >> > >> > > > > >> >> kklou...@apache.org> >> > >> > > > > >> >> > > > > wrote: >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > > > +1 for releasing Flink 1.11.3 as it contains >> a >> > >> number >> > >> > > of >> > >> > > > > >> >> important >> > >> > > > > >> >> > > > > > fixes and thanks Gordon and Xintong for >> > >> volunteering. >> > >> > > > > >> >> > > > > > >> > >> > > > > >> >> > > > > > Cheers, >> > >> > > > > >> >> > > > > > Kostas >> > >> > > > > >> >> > > > > > >> > >> > > > > >> >> > > > > > On Mon, Oct 26, 2020 at 4:37 AM Yu Li < >> > >> > > car...@gmail.com> >> > >> > > > > >> wrote: >> > >> > > > > >> >> > > > > > > >> > >> > > > > >> >> > > > > > > +1 for releasing Flink 1.11.3, and thanks >> > Gordon >> > >> > and >> > >> > > > > >> Xintong >> > >> > > > > >> >> for >> > >> > > > > >> >> > > > > > > volunteering as our release managers. >> > >> > > > > >> >> > > > > > > >> > >> > > > > >> >> > > > > > > Best Regards, >> > >> > > > > >> >> > > > > > > Yu >> > >> > > > > >> >> > > > > > > >> > >> > > > > >> >> > > > > > > >> > >> > > > > >> >> > > > > > > On Mon, 26 Oct 2020 at 09:45, Xintong Song >> < >> > >> > > > > >> >> > tonysong...@gmail.com> >> > >> > > > > >> >> > > > > > wrote: >> > >> > > > > >> >> > > > > > > >> > >> > > > > >> >> > > > > > > > Thanks Gordan for starting this >> discussion. >> > >> > > > > >> >> > > > > > > > My pleasure to help with the release >> > process. >> > >> > > > > >> >> > > > > > > > >> > >> > > > > >> >> > > > > > > > Thank you~ >> > >> > > > > >> >> > > > > > > > >> > >> > > > > >> >> > > > > > > > Xintong Song >> > >> > > > > >> >> > > > > > > > >> > >> > > > > >> >> > > > > > > > >> > >> > > > > >> >> > > > > > > > >> > >> > > > > >> >> > > > > > > > On Fri, Oct 23, 2020 at 11:29 PM Till >> > >> Rohrmann < >> > >> > > > > >> >> > > > trohrm...@apache.org >> > >> > > > > >> >> > > > > > >> > >> > > > > >> >> > > > > > > > wrote: >> > >> > > > > >> >> > > > > > > > >> > >> > > > > >> >> > > > > > > > > Thanks for starting this discussion >> > Gordon. >> > >> > There >> > >> > > > are >> > >> > > > > >> over >> > >> > > > > >> >> > 100 >> > >> > > > > >> >> > > > > issues >> > >> > > > > >> >> > > > > > > > > which are fixed for 1.11.3. Hence +1 >> for a >> > >> > > soonish >> > >> > > > > >> 1.11.3 >> > >> > > > > >> >> > > > release. >> > >> > > > > >> >> > > > > > Thanks >> > >> > > > > >> >> > > > > > > > > for volunteering as our release >> managers >> > >> Gordon >> > >> > > and >> > >> > > > > >> >> Xintong! >> > >> > > > > >> >> > > > > > > > > >> > >> > > > > >> >> > > > > > > > > Cheers, >> > >> > > > > >> >> > > > > > > > > Till >> > >> > > > > >> >> > > > > > > > > >> > >> > > > > >> >> > > > > > > > > On Fri, Oct 23, 2020 at 5:02 PM Tzu-Li >> > >> (Gordon) >> > >> > > > Tai < >> > >> > > > > >> >> > > > > > tzuli...@apache.org >> > >> > > > > >> >> > > > > > > > > >> > >> > > > > >> >> > > > > > > > > wrote: >> > >> > > > > >> >> > > > > > > > > >> > >> > > > > >> >> > > > > > > > >> Hi, >> > >> > > > > >> >> > > > > > > > >> >> > >> > > > > >> >> > > > > > > > >> Xintong and I would like to start a >> > >> discussion >> > >> > > for >> > >> > > > > >> >> releasing >> > >> > > > > >> >> > > > Flink >> > >> > > > > >> >> > > > > > > > 1.11.3 >> > >> > > > > >> >> > > > > > > > >> soon. >> > >> > > > > >> >> > > > > > > > >> >> > >> > > > > >> >> > > > > > > > >> It seems like we already have a few >> > >> pressing >> > >> > > > issues >> > >> > > > > >> that >> > >> > > > > >> >> > needs >> > >> > > > > >> >> > > > to >> > >> > > > > >> >> > > > > be >> > >> > > > > >> >> > > > > > > > >> included in a new hotfix release: >> > >> > > > > >> >> > > > > > > > >> >> > >> > > > > >> >> > > > > > > > >> - Heap-based timers’ restore >> behaviour >> > >> is >> > >> > > > > causing a >> > >> > > > > >> >> > > critical >> > >> > > > > >> >> > > > > > recovery >> > >> > > > > >> >> > > > > > > > >> issue for StateFun [1] [2] [3]. >> > >> > > > > >> >> > > > > > > > >> - There are several robustness >> issues >> > >> for >> > >> > the >> > >> > > > > >> FLIP-27 >> > >> > > > > >> >> new >> > >> > > > > >> >> > > > > source >> > >> > > > > >> >> > > > > > API, >> > >> > > > > >> >> > > > > > > > >> such as [4]. We already have some >> > users >> > >> > using >> > >> > > > the >> > >> > > > > >> >> FLIP-27 >> > >> > > > > >> >> > > API >> > >> > > > > >> >> > > > > > with >> > >> > > > > >> >> > > > > > > > >> 1.11.x, >> > >> > > > > >> >> > > > > > > > >> so it would be important to get >> those >> > >> fixes >> > >> > > in >> > >> > > > > for >> > >> > > > > >> >> 1.11.x >> > >> > > > > >> >> > > as >> > >> > > > > >> >> > > > > > well. >> > >> > > > > >> >> > > > > > > > >> >> > >> > > > > >> >> > > > > > > > >> Apart from the issues that are already >> > >> marked >> > >> > as >> > >> > > > > >> blocker >> > >> > > > > >> >> for >> > >> > > > > >> >> > > > > 1.11.3 >> > >> > > > > >> >> > > > > > in >> > >> > > > > >> >> > > > > > > > our >> > >> > > > > >> >> > > > > > > > >> JIRA [5], please let us know in this >> > >> thread if >> > >> > > > there >> > >> > > > > >> is >> > >> > > > > >> >> > > already >> > >> > > > > >> >> > > > > > ongoing >> > >> > > > > >> >> > > > > > > > >> work for other important fixes that we >> > >> should >> > >> > > try >> > >> > > > to >> > >> > > > > >> >> > include. >> > >> > > > > >> >> > > > > > > > >> >> > >> > > > > >> >> > > > > > > > >> Xintong and I would like to volunteer >> for >> > >> > > managing >> > >> > > > > >> this >> > >> > > > > >> >> > > release, >> > >> > > > > >> >> > > > > and >> > >> > > > > >> >> > > > > > > > will >> > >> > > > > >> >> > > > > > > > >> try to communicate the priority of >> > pending >> > >> > > > blockers >> > >> > > > > >> over >> > >> > > > > >> >> the >> > >> > > > > >> >> > > > next >> > >> > > > > >> >> > > > > > few >> > >> > > > > >> >> > > > > > > > >> days. >> > >> > > > > >> >> > > > > > > > >> Since the aforementioned issues are >> quite >> > >> > > > critical, >> > >> > > > > >> we’d >> > >> > > > > >> >> > like >> > >> > > > > >> >> > > to >> > >> > > > > >> >> > > > > aim >> > >> > > > > >> >> > > > > > > > >> for a *feature >> > >> > > > > >> >> > > > > > > > >> freeze by the end of next week (Oct. >> > 30th)* >> > >> > and >> > >> > > > > start >> > >> > > > > >> the >> > >> > > > > >> >> > > > release >> > >> > > > > >> >> > > > > > voting >> > >> > > > > >> >> > > > > > > > >> process the week after. >> > >> > > > > >> >> > > > > > > > >> If that is too short of a notice and >> you >> > >> might >> > >> > > > need >> > >> > > > > >> more >> > >> > > > > >> >> > time, >> > >> > > > > >> >> > > > > > please >> > >> > > > > >> >> > > > > > > > let >> > >> > > > > >> >> > > > > > > > >> us know! >> > >> > > > > >> >> > > > > > > > >> >> > >> > > > > >> >> > > > > > > > >> Cheers, >> > >> > > > > >> >> > > > > > > > >> Gordon >> > >> > > > > >> >> > > > > > > > >> >> > >> > > > > >> >> > > > > > > > >> [1] >> > >> > > > > https://issues.apache.org/jira/browse/FLINK-19692 >> > >> > > > > >> >> > > > > > > > >> [2] >> > >> > > > > https://issues.apache.org/jira/browse/FLINK-19741 >> > >> > > > > >> >> > > > > > > > >> [3] >> > >> > > > > https://issues.apache.org/jira/browse/FLINK-19748 >> > >> > > > > >> >> > > > > > > > >> [4] >> > >> > > > > https://issues.apache.org/jira/browse/FLINK-19717 >> > >> > > > > >> >> > > > > > > > >> [5] >> > >> > > > > >> >> > > > > > > > >> >> > >> > > > > >> >> > > > > > > > >> >> > >> > > > > >> >> > > > > > > > >> > >> > > > > >> >> > > > > > >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > >> > >> > > > > >> >> > > >> > >> > > > > >> >> > >> > >> > > > > >> >> >> > >> > > > > >> >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > >> https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%20AND%20fixVersion%20%3D%201.11.3 >> > >> > > > > >> >> > > > > > > > >> >> > >> > > > > >> >> > > > > > > > > >> > >> > > > > >> >> > > > > > > > >> > >> > > > > >> >> > > > > > >> > >> > > > > >> >> > > > > >> > >> > > > > >> >> > > > >> > >> > > > > >> >> > > >> > >> > > > > >> >> > >> > >> > > > > >> >> > >> > >> > > > > >> >> > -- >> > >> > > > > >> >> > Best, Jingsong Lee >> > >> > > > > >> >> > >> > >> > > > > >> >> >> > >> > > > > >> > >> > >> > > > > >> >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > > >> > >> >
