Re: [DISCUSS] FLIP-423 ～FLIP-428: Introduce Disaggregated State Storage and Management in Flink 2.0
Hi team, Thanks for your discussion. Regarding FLIP-425, we have supplemented several updates to answer high-frequency questions: 1. We captured a flame graph of the Hashmap state backend in "Synchronous execution with asynchronous APIs"[1], which reveals that the framework overhead (including reference counting, future-related code and so on) consumes about 9% of the keyed operator CPU time. 2. We added a set of comparative experiments for watermark processing, the performance of Out-Of-Order mode is 70% better than strictly-ordered mode under ~140MB state size. Instructions on how to run this test have also been added[2]. 3. Regarding the order of StreamRecord, whether it has state access or not. We supplemented a new *Strict order of 'processElement'*[3]. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-425%3A+Asynchronous+Execution+Model#FLIP425:AsynchronousExecutionModel-SynchronousexecutionwithasynchronousAPIs [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-425%3A+Asynchronous+Execution+Model#FLIP425:AsynchronousExecutionModel-Strictly-orderedmodevs.Out-of-ordermodeforwatermarkprocessing [3] https://cwiki.apache.org/confluence/display/FLINK/FLIP-425%3A+Asynchronous+Execution+Model#FLIP425:AsynchronousExecutionModel-ElementOrder Best regards, Yanfei Yunfeng Zhou 于2024年3月5日周二 09:25写道： > > Hi Zakelly, > > > 5. I'm not very sure ... revisiting this later since it is not important. > > It seems that we still have some details to confirm about this > question. Let's postpone this to after the critical parts of the > design are settled. > > > 8. Yes, we had considered ... metrics should be like afterwards. > > Oh sorry I missed FLIP-431. I'm fine with discussing this topic in milestone > 2. > > Looking forward to the detailed design about the strict mode between > same-key records and the benchmark results about the epoch mechanism. > > Best regards, > Yunfeng > > On Mon, Mar 4, 2024 at 7:59 PM Zakelly Lan wrote: > > > > Hi Yunfeng, > > > > For 1: > > I had a discussion with Lincoln Lee, and I realize it is a common case the > > same-key record should be blocked before the `processElement`. It is easier > > for users to understand. Thus I will introduce a strict mode for this and > > make it default. My rough idea is just like yours, by invoking some method > > of AEC instance before `processElement`. The detailed design will be > > described in FLIP later. > > > > For 2: > > I agree with you. We could throw exceptions for now and optimize this later. > > > > For 5: > >> > >> It might be better to move the default values to the Proposed Changes > >> section instead of making them public for now, as there will be > >> compatibility issues once we want to dynamically adjust the thresholds > >> and timeouts in future. > > > > Agreed. The whole framework is under experiment until we think it is > > complete in 2.0 or later. The default value should be better determined > > with more testing results and production experience. > > > >> The configuration execution.async-state.enabled seems unnecessary, as > >> the infrastructure may automatically get this information from the > >> detailed state backend configurations. We may revisit this part after > >> the core designs have reached an agreement. WDYT? > > > > > > I'm not very sure if there is any use case where users write their code > > using async APIs but run their job in a synchronous way. The first two > > scenarios that come to me are for benchmarking or for a small state, while > > they don't want to rewrite their code. Actually it is easy to support, so > > I'd suggest providing it. But I'm fine with revisiting this later since it > > is not important. WDYT? > > > > For 8: > > Yes, we had considered the I/O metrics group especially the back-pressure, > > idle and task busy per second. In the current plan we can do state access > > during back-pressure, meaning that those metrics for input would better be > > redefined. I suggest we discuss these existing metrics as well as some new > > metrics that should be introduced in FLIP-431 later in milestone 2, since > > we have basically finished the framework thus we will have a better view of > > what metrics should be like afterwards. WDYT? > > > > > > Best, > > Zakelly > > > > On Mon, Mar 4, 2024 at 6:49 PM Yunfeng Zhou > > wrote: > >> > >> Hi Zakelly, > >> > >> Thanks for the responses! > >> > >> > 1. I will discuss this with some expert SQL developers. ... mode for > >> > StreamRecord processing. > >> > >> In DataStream API there should also be use cases when the order of > >> output is strictly required. I agree with it that SQL experts may help > >> provide more concrete use cases that can accelerate our discussion, > >> but please allow me to search for DataStream use cases that can prove > >> the necessity of this strict order preservation mode, if answers from > >> SQL experts are shown to be negative. > >> > >> For your convenience, my current
[jira] [Created] (FLINK-34585) [JUnit5 Migration] Module: Flink CDC
Hang Ruan created FLINK-34585: - Summary: [JUnit5 Migration] Module: Flink CDC Key: FLINK-34585 URL: https://issues.apache.org/jira/browse/FLINK-34585 Project: Flink Issue Type: Sub-task Components: Flink CDC Reporter: Hang Ruan Most tests in Flink CDC are still using Junit 4. We need to use Junit 5 instead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34584) Change package name to org.apache.flink.cdc
Hang Ruan created FLINK-34584: - Summary: Change package name to org.apache.flink.cdc Key: FLINK-34584 URL: https://issues.apache.org/jira/browse/FLINK-34584 Project: Flink Issue Type: Sub-task Components: Flink CDC Reporter: Hang Ruan Flink CDC need to change its package name to org.apache.flink.cdc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] Release 1.19.0, release candidate #1
Hi, Lincoln, We are currently addressing the FLINK-34105, where we have encountered an Akka timeout during the TPC-DS benchmarks. We plan to revert the changes made in FLINK-33532, and we anticipate that the corresponding pull request will be ready today. Could you include this patch into the rc2? Best, Yangze Guo On Wed, Mar 6, 2024 at 2:00 PM Lincoln Lee wrote: > > Hi everyone, > > This rc is officially cancelled because of the incompleteness of python > wheel packages(lost 2 newly added py3.11 packages for mac, both 10.9 & > 11.0, will be fixed in [1]). > > A new rc will be created. > > [1] https://issues.apache.org/jira/browse/FLINK-34582 > > Best, > Yun, Jing, Martijn and Lincoln > > Lincoln Lee 于2024年3月6日周三 00:24写道： > > > Hi everyone, > > > > Please review and vote on the release candidate #1 for the version 1.19.0, > > as follows: > > [ ] +1, Approve the release > > [ ] -1, Do not approve the release (please provide specific comments) > > > > The complete staging area is available for your review, which includes: > > > > * JIRA release notes [1], and the pull request adding release note for > > users [2] > > * the official Apache source release and binary convenience releases to be > > deployed to dist.apache.org [3], which are signed with the key with > > fingerprint E57D30ABEE75CA06 [4], > > * all artifacts to be deployed to the Maven Central Repository [5], > > * source code tag "release-1.19.0-rc1" [6], > > * website pull request listing the new release and adding announcement > > blog post [7]. > > > > The vote will be open for at least 72 hours. It is adopted by majority > > approval, with at least 3 PMC affirmative votes. > > > > [1] > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353282 > > [2] https://github.com/apache/flink/pull/24394 > > [3] https://dist.apache.org/repos/dist/dev/flink/flink-1.19.0-rc1/ > > [4] https://dist.apache.org/repos/dist/release/flink/KEYS > > [5] https://repository.apache.org/content/repositories/orgapacheflink-1708 > > [6] https://github.com/apache/flink/releases/tag/release-1.19.0-rc1 > > [7] https://github.com/apache/flink-web/pull/721 > > > > Best, > > Yun, Jing, Martijn and Lincoln > >
[ANNOUNCE] Flink 1.19.0 rc1 cancelled & sync summary on 03/05/2024
Hi devs, As mentioned in the vote thread[1] rc1 has been cancelled due to the incompleteness of python wheel packages(lost 2 newly added py3.11 packages for mac, both 10.9 & 11.0, will be fixed in [2]). If no new blockers, we'll start a new rc soon. Also share some highlights from the release sync on 03/05/2024 - Test instabilities A new failure [3] comes up and has been confirmed not a blocker. - Release notes The release note[4] and announcement[5] prs are ready to review, please help review before 1.19 release, thanks! - Sync meeting (https://meet.google.com/vcx-arzs-trv) We've already switched to weekly release sync, so the next release sync will be on Mar 12th, 2024. Feel free to join! [1] https://lists.apache.org/thread/10bxy1zhzy6hycjyohyl3pzx3xs3zh34 [2] https://issues.apache.org/jira/browse/FLINK-34582 [3] https://issues.apache.org/jira/browse/FLINK-34571 [4] https://github.com/apache/flink/pull/24394 [5] https://github.com/apache/flink-web/pull/721 Best, Yun, Jing, Martijn and Lincoln
Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website
> a rare way different than > individuals (few individuals can donate such resources) Theoretically, if an individual donates so, we list list him/her as well. I've seen such donations in The Perl Foundation like [1]. But since a PMC doesn't have a fundraising office, we may not accept raw money anyway; it's already out of the thread :D Best, tison. [1] https://news.perlfoundation.org/post/announcement_of_the_ian_hague Yun Tang 于2024年3月6日周三 13:58写道： > > Thanks for Jark's proposal, and I'm +1 for adding such a page. > > The CI infrastructure helps the Apache Flink project to run well. I cannot > imagine how insufficient CI machines would impact the development progress, > especially when the feature freeze date is close. And I believe that most > guys who contributed to the community would not know Alibaba and Ververica > had ever donated several machines to make the community work smoothly for > years. > > > Best > Yun Tang > > From: Jark Wu > Sent: Wednesday, March 6, 2024 11:35 > To: dev@flink.apache.org > Subject: Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website > > Hi Max, > > Thank you for your input. > > According to ASF policy[1], the Thank Page is intended to thank third > parties > that provide physical resources like machines, services, and software that > the committers > or the project truly needs. I agree with Tison, such donation is countable > and that's why > I started this discussion to collect the full list. The thank Page is not > intended to thank working > hours or contributions from individual volunteers which I think > is recognized in other ways > (e.g., credit of committer and PMC member). > > Best, > Jark > > [1]: https://www.apache.org/foundation/marks/linking#projectthanks > > On Wed, 6 Mar 2024 at 01:14, tison wrote: > > > Hi Max, > > > > Thanks for sharing your concerns :D > > > > I'd elaborate a bit on this topic with an example, that Apache Airflow > > has a small section for its special sponsor who offers machines for CI > > also [1]. > > > > In my understanding, companies employ developers to invest time in the > > development of Flink and that is large, vague, and hard to be fair to > > list all of the companies. > > > > However, physical resources like CI machines are countable and they > > help the sustainability of our project in a rare way different than > > individuals (few individuals can donate such resources). We can > > maintain such a section or page for those sponsors so that it also > > decreases the friction when the company asks "what we can gain" (for > > explicit credits, at least, and easy understanding). > > > > Any entity is welcome to add themselves as long as it's valid. > > > > For the fair part, I'm not an employee of both companies listed on the > > demo page and I don't feel uncomfortable. Those companies do invest a > > lot on our project and I'd regard it as a chance to encourage other > > companies to follow. > > > > Best, > > tison. > > > > [1] https://github.com/apache/airflow?tab=readme-ov-file#sponsors > > > > Maximilian Michels 于2024年3月6日周三 00:49写道： > > > > > > I'm a bit torn on this idea. On the one hand, it makes sense to thank > > > sponsors and entities who have supported Flink in the past. On other > > > hand, this list is bound to be incomplete and maybe also biased, even > > > if not intended to be so. I think the power of open-source comes from > > > the unconditional donation of code and knowledge. Infrastructure costs > > > are a reality and donations in that area are meaningful, but they are > > > just one piece of the total sum which consists of many volunteers and > > > working hours. In my eyes, a Thank You page would have to display each > > > entity fairly which is going to be hard to achieve. > > > > > > -Max > > > > > > On Tue, Mar 5, 2024 at 2:30 PM Jingsong Li > > wrote: > > > > > > > > +1 for setting up > > > > > > > > On Tue, Mar 5, 2024 at 5:39 PM Jing Ge > > wrote: > > > > > > > > > > +1 and thanks for the proposal! > > > > > > > > > > Best regards, > > > > > Jing > > > > > > > > > > On Tue, Mar 5, 2024 at 10:26 AM tison wrote: > > > > > > > > > > > I like this idea, so +1 for setting up. > > > > > > > > > > > > For anyone who have the access, this is a related thread about > > > > > > project-wise sponsor in the foundation level [1]. > > > > > > > > > > > > Best, > > > > > > tison. > > > > > > > > > > > > [1] > > https://lists.apache.org/thread/2nv0x9gfk9lfnpb2315xgywyx84y97v6 > > > > > > > > > > > > Jark Wu 于2024年3月5日周二 17:17写道： > > > > > > > > > > > > > > Sorry, I posted the wrong [7] link. The Flink benchmark ML link > > is: > > > > > > > https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 > > > > > > > > > > > > > > > > > > > > > On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I want to propose adding a "Special Thanks" page to our
Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website
Thanks for Jark's proposal, and I'm +1 for adding such a page. The CI infrastructure helps the Apache Flink project to run well. I cannot imagine how insufficient CI machines would impact the development progress, especially when the feature freeze date is close. And I believe that most guys who contributed to the community would not know Alibaba and Ververica had ever donated several machines to make the community work smoothly for years. Best Yun Tang From: Jark Wu Sent: Wednesday, March 6, 2024 11:35 To: dev@flink.apache.org Subject: Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website Hi Max, Thank you for your input. According to ASF policy[1], the Thank Page is intended to thank third parties that provide physical resources like machines, services, and software that the committers or the project truly needs. I agree with Tison, such donation is countable and that's why I started this discussion to collect the full list. The thank Page is not intended to thank working hours or contributions from individual volunteers which I think is recognized in other ways (e.g., credit of committer and PMC member). Best, Jark [1]: https://www.apache.org/foundation/marks/linking#projectthanks On Wed, 6 Mar 2024 at 01:14, tison wrote: > Hi Max, > > Thanks for sharing your concerns :D > > I'd elaborate a bit on this topic with an example, that Apache Airflow > has a small section for its special sponsor who offers machines for CI > also [1]. > > In my understanding, companies employ developers to invest time in the > development of Flink and that is large, vague, and hard to be fair to > list all of the companies. > > However, physical resources like CI machines are countable and they > help the sustainability of our project in a rare way different than > individuals (few individuals can donate such resources). We can > maintain such a section or page for those sponsors so that it also > decreases the friction when the company asks "what we can gain" (for > explicit credits, at least, and easy understanding). > > Any entity is welcome to add themselves as long as it's valid. > > For the fair part, I'm not an employee of both companies listed on the > demo page and I don't feel uncomfortable. Those companies do invest a > lot on our project and I'd regard it as a chance to encourage other > companies to follow. > > Best, > tison. > > [1] https://github.com/apache/airflow?tab=readme-ov-file#sponsors > > Maximilian Michels 于2024年3月6日周三 00:49写道： > > > > I'm a bit torn on this idea. On the one hand, it makes sense to thank > > sponsors and entities who have supported Flink in the past. On other > > hand, this list is bound to be incomplete and maybe also biased, even > > if not intended to be so. I think the power of open-source comes from > > the unconditional donation of code and knowledge. Infrastructure costs > > are a reality and donations in that area are meaningful, but they are > > just one piece of the total sum which consists of many volunteers and > > working hours. In my eyes, a Thank You page would have to display each > > entity fairly which is going to be hard to achieve. > > > > -Max > > > > On Tue, Mar 5, 2024 at 2:30 PM Jingsong Li > wrote: > > > > > > +1 for setting up > > > > > > On Tue, Mar 5, 2024 at 5:39 PM Jing Ge > wrote: > > > > > > > > +1 and thanks for the proposal! > > > > > > > > Best regards, > > > > Jing > > > > > > > > On Tue, Mar 5, 2024 at 10:26 AM tison wrote: > > > > > > > > > I like this idea, so +1 for setting up. > > > > > > > > > > For anyone who have the access, this is a related thread about > > > > > project-wise sponsor in the foundation level [1]. > > > > > > > > > > Best, > > > > > tison. > > > > > > > > > > [1] > https://lists.apache.org/thread/2nv0x9gfk9lfnpb2315xgywyx84y97v6 > > > > > > > > > > Jark Wu 于2024年3月5日周二 17:17写道： > > > > > > > > > > > > Sorry, I posted the wrong [7] link. The Flink benchmark ML link > is: > > > > > > https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 > > > > > > > > > > > > > > > > > > On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > I want to propose adding a "Special Thanks" page to our Apache > Flink > > > > > website [1] > > > > > > > > > > > > > > to honor and appreciate the > > > > > > > companies and organizations that have sponsored > > > > > > > > > > > > > > > > > > > > > machines or services for our project. The establishment of > such a page > > > > > serves as > > > > > > > > > > > > > > > > > > > > > a public acknowledgment of our sponsors' contributions and > > > > > simultaneously acts > > > > > > > > > > > > > > > > > > > > > as a positive encouragement for other entities to consider > supporting > > > > > our project. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Adding Per-Project Thanks pages is allowed by ASF policy[2], > which > > > > > says "PMCs > > >
Re: [VOTE] Release 1.19.0, release candidate #1
Hi everyone, This rc is officially cancelled because of the incompleteness of python wheel packages(lost 2 newly added py3.11 packages for mac, both 10.9 & 11.0, will be fixed in [1]). A new rc will be created. [1] https://issues.apache.org/jira/browse/FLINK-34582 Best, Yun, Jing, Martijn and Lincoln Lincoln Lee 于2024年3月6日周三 00:24写道： > Hi everyone, > > Please review and vote on the release candidate #1 for the version 1.19.0, > as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > The complete staging area is available for your review, which includes: > > * JIRA release notes [1], and the pull request adding release note for > users [2] > * the official Apache source release and binary convenience releases to be > deployed to dist.apache.org [3], which are signed with the key with > fingerprint E57D30ABEE75CA06 [4], > * all artifacts to be deployed to the Maven Central Repository [5], > * source code tag "release-1.19.0-rc1" [6], > * website pull request listing the new release and adding announcement > blog post [7]. > > The vote will be open for at least 72 hours. It is adopted by majority > approval, with at least 3 PMC affirmative votes. > > [1] > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353282 > [2] https://github.com/apache/flink/pull/24394 > [3] https://dist.apache.org/repos/dist/dev/flink/flink-1.19.0-rc1/ > [4] https://dist.apache.org/repos/dist/release/flink/KEYS > [5] https://repository.apache.org/content/repositories/orgapacheflink-1708 > [6] https://github.com/apache/flink/releases/tag/release-1.19.0-rc1 > [7] https://github.com/apache/flink-web/pull/721 > > Best, > Yun, Jing, Martijn and Lincoln >
Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website
Thanks for Jark's proposal, and I'm +1 for adding such a page. The CI infrastructure helps the Apache Flink project to run well. I cannot imagine how insufficient CI machines would impact the development progress, especially when the feature freeze date is close. And I believe that most guys who contributed to the community would not know Alibaba and Ververica had ever donated several machines to make the community work smoothly for years. Best Yun Tang From: Jark Wu Sent: Wednesday, March 6, 2024 11:35 To: dev@flink.apache.org Subject: Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website Hi Max, Thank you for your input. According to ASF policy[1], the Thank Page is intended to thank third parties that provide physical resources like machines, services, and software that the committers or the project truly needs. I agree with Tison, such donation is countable and that's why I started this discussion to collect the full list. The thank Page is not intended to thank working hours or contributions from individual volunteers which I think is recognized in other ways (e.g., credit of committer and PMC member). Best, Jark [1]: https://www.apache.org/foundation/marks/linking#projectthanks On Wed, 6 Mar 2024 at 01:14, tison wrote: > Hi Max, > > Thanks for sharing your concerns :D > > I'd elaborate a bit on this topic with an example, that Apache Airflow > has a small section for its special sponsor who offers machines for CI > also [1]. > > In my understanding, companies employ developers to invest time in the > development of Flink and that is large, vague, and hard to be fair to > list all of the companies. > > However, physical resources like CI machines are countable and they > help the sustainability of our project in a rare way different than > individuals (few individuals can donate such resources). We can > maintain such a section or page for those sponsors so that it also > decreases the friction when the company asks "what we can gain" (for > explicit credits, at least, and easy understanding). > > Any entity is welcome to add themselves as long as it's valid. > > For the fair part, I'm not an employee of both companies listed on the > demo page and I don't feel uncomfortable. Those companies do invest a > lot on our project and I'd regard it as a chance to encourage other > companies to follow. > > Best, > tison. > > [1] https://github.com/apache/airflow?tab=readme-ov-file#sponsors > > Maximilian Michels 于2024年3月6日周三 00:49写道： > > > > I'm a bit torn on this idea. On the one hand, it makes sense to thank > > sponsors and entities who have supported Flink in the past. On other > > hand, this list is bound to be incomplete and maybe also biased, even > > if not intended to be so. I think the power of open-source comes from > > the unconditional donation of code and knowledge. Infrastructure costs > > are a reality and donations in that area are meaningful, but they are > > just one piece of the total sum which consists of many volunteers and > > working hours. In my eyes, a Thank You page would have to display each > > entity fairly which is going to be hard to achieve. > > > > -Max > > > > On Tue, Mar 5, 2024 at 2:30 PM Jingsong Li > wrote: > > > > > > +1 for setting up > > > > > > On Tue, Mar 5, 2024 at 5:39 PM Jing Ge > wrote: > > > > > > > > +1 and thanks for the proposal! > > > > > > > > Best regards, > > > > Jing > > > > > > > > On Tue, Mar 5, 2024 at 10:26 AM tison wrote: > > > > > > > > > I like this idea, so +1 for setting up. > > > > > > > > > > For anyone who have the access, this is a related thread about > > > > > project-wise sponsor in the foundation level [1]. > > > > > > > > > > Best, > > > > > tison. > > > > > > > > > > [1] > https://lists.apache.org/thread/2nv0x9gfk9lfnpb2315xgywyx84y97v6 > > > > > > > > > > Jark Wu 于2024年3月5日周二 17:17写道： > > > > > > > > > > > > Sorry, I posted the wrong [7] link. The Flink benchmark ML link > is: > > > > > > https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 > > > > > > > > > > > > > > > > > > On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > I want to propose adding a "Special Thanks" page to our Apache > Flink > > > > > website [1] > > > > > > > > > > > > > > to honor and appreciate the > > > > > > > companies and organizations that have sponsored > > > > > > > > > > > > > > > > > > > > > machines or services for our project. The establishment of > such a page > > > > > serves as > > > > > > > > > > > > > > > > > > > > > a public acknowledgment of our sponsors' contributions and > > > > > simultaneously acts > > > > > > > > > > > > > > > > > > > > > as a positive encouragement for other entities to consider > supporting > > > > > our project. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Adding Per-Project Thanks pages is allowed by ASF policy[2], > which > > > > > says "PMCs > > >
[jira] [Created] (FLINK-34583) Bug for dynamic table option hints with multiple CTEs
Xingcan Cui created FLINK-34583:
---
Summary: Bug for dynamic table option hints with multiple CTEs
Key: FLINK-34583
URL: https://issues.apache.org/jira/browse/FLINK-34583
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Affects Versions: 1.18.1
Reporter: Xingcan Cui
The table options hints don't work well with multiple WITH clauses referring to
the same table. Please see the following example.
The following query with hints works well.
{code:java}
SELECT * FROM T1 /*+ OPTIONS('foo' = 'bar') */ WHERE...;{code}
The following query with multiple WITH clauses also works well.
{code:java}
WITH T2 AS (SELECT * FROM T1 /*+ OPTIONS('foo' = 'bar') */ WHERE...)
T3 AS (SELECT ... FROM T2 WHERE...)
SELECT * FROM T3;{code}
The following query with multiple WITH clauses referring to the same original
table failed to recognize the hints.
{code:java}
WITH T2 AS (SELECT * FROM T1 /*+ OPTIONS('foo' = 'bar') */ WHERE...),
T3 AS (SELECT ... FROM T2 WHERE...),
T4 AS (SELECT ... FROM T2 WHERE...),
T5 AS (SELECT ... FROM T3 JOIN T4 ON...)
SELECT * FROM T5;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website
Hi Max, Thank you for your input. According to ASF policy[1], the Thank Page is intended to thank third parties that provide physical resources like machines, services, and software that the committers or the project truly needs. I agree with Tison, such donation is countable and that's why I started this discussion to collect the full list. The thank Page is not intended to thank working hours or contributions from individual volunteers which I think is recognized in other ways (e.g., credit of committer and PMC member). Best, Jark [1]: https://www.apache.org/foundation/marks/linking#projectthanks On Wed, 6 Mar 2024 at 01:14, tison wrote: > Hi Max, > > Thanks for sharing your concerns :D > > I'd elaborate a bit on this topic with an example, that Apache Airflow > has a small section for its special sponsor who offers machines for CI > also [1]. > > In my understanding, companies employ developers to invest time in the > development of Flink and that is large, vague, and hard to be fair to > list all of the companies. > > However, physical resources like CI machines are countable and they > help the sustainability of our project in a rare way different than > individuals (few individuals can donate such resources). We can > maintain such a section or page for those sponsors so that it also > decreases the friction when the company asks "what we can gain" (for > explicit credits, at least, and easy understanding). > > Any entity is welcome to add themselves as long as it's valid. > > For the fair part, I'm not an employee of both companies listed on the > demo page and I don't feel uncomfortable. Those companies do invest a > lot on our project and I'd regard it as a chance to encourage other > companies to follow. > > Best, > tison. > > [1] https://github.com/apache/airflow?tab=readme-ov-file#sponsors > > Maximilian Michels 于2024年3月6日周三 00:49写道： > > > > I'm a bit torn on this idea. On the one hand, it makes sense to thank > > sponsors and entities who have supported Flink in the past. On other > > hand, this list is bound to be incomplete and maybe also biased, even > > if not intended to be so. I think the power of open-source comes from > > the unconditional donation of code and knowledge. Infrastructure costs > > are a reality and donations in that area are meaningful, but they are > > just one piece of the total sum which consists of many volunteers and > > working hours. In my eyes, a Thank You page would have to display each > > entity fairly which is going to be hard to achieve. > > > > -Max > > > > On Tue, Mar 5, 2024 at 2:30 PM Jingsong Li > wrote: > > > > > > +1 for setting up > > > > > > On Tue, Mar 5, 2024 at 5:39 PM Jing Ge > wrote: > > > > > > > > +1 and thanks for the proposal! > > > > > > > > Best regards, > > > > Jing > > > > > > > > On Tue, Mar 5, 2024 at 10:26 AM tison wrote: > > > > > > > > > I like this idea, so +1 for setting up. > > > > > > > > > > For anyone who have the access, this is a related thread about > > > > > project-wise sponsor in the foundation level [1]. > > > > > > > > > > Best, > > > > > tison. > > > > > > > > > > [1] > https://lists.apache.org/thread/2nv0x9gfk9lfnpb2315xgywyx84y97v6 > > > > > > > > > > Jark Wu 于2024年3月5日周二 17:17写道： > > > > > > > > > > > > Sorry, I posted the wrong [7] link. The Flink benchmark ML link > is: > > > > > > https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 > > > > > > > > > > > > > > > > > > On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > I want to propose adding a "Special Thanks" page to our Apache > Flink > > > > > website [1] > > > > > > > > > > > > > > to honor and appreciate the > > > > > > > companies and organizations that have sponsored > > > > > > > > > > > > > > > > > > > > > machines or services for our project. The establishment of > such a page > > > > > serves as > > > > > > > > > > > > > > > > > > > > > a public acknowledgment of our sponsors' contributions and > > > > > simultaneously acts > > > > > > > > > > > > > > > > > > > > > as a positive encouragement for other entities to consider > supporting > > > > > our project. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Adding Per-Project Thanks pages is allowed by ASF policy[2], > which > > > > > says "PMCs > > > > > > > > > > > > > > > > > > > > > may wish to provide recognition for third parties that provide > > > > > software or services > > > > > > > > > > > > > > > > > > > > > to the project's committers to further the goals of the > project. These > > > > > are typically > > > > > > > > > > > > > > called Per-Project Thanks pages". Many Apache projects have > added such > > > > > > > > > > > > > > pages, for example, Apache HBase[3] and Apache Mina[4]. > > > > > > > > > > > > > > > > > > > > > To initiate this idea, I have drafted a preliminary page under > the > > > > > > > "About" menu > > > > > > > > > > > > > > on the > > > > > > >
[jira] [Created] (FLINK-34582) release build tools lost the newly added py3.11 packages for mac
lincoln lee created FLINK-34582: --- Summary: release build tools lost the newly added py3.11 packages for mac Key: FLINK-34582 URL: https://issues.apache.org/jira/browse/FLINK-34582 Project: Flink Issue Type: Bug Affects Versions: 1.19.0, 1.20.0 Reporter: lincoln lee Assignee: Xingbo Huang during 1.19.0-rc1 building binaries via tools/releasing/create_binary_release.sh lost the newly added py3.11 2 packages for mac -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34581) streaming code throws java.lang.reflect.InaccessibleObjectException
Henning Schmiedehausen created FLINK-34581:
--
Summary: streaming code throws
java.lang.reflect.InaccessibleObjectException
Key: FLINK-34581
URL: https://issues.apache.org/jira/browse/FLINK-34581
Project: Flink
Issue Type: Bug
Components: API / DataStream
Affects Versions: 1.18.1
Reporter: Henning Schmiedehausen
I have a pretty simple test pipeline (read a bunch of tables from Apache Kafka,
join and project them, then write to Apache Iceberg) that I run locally with
Junit5 and the MiniClusterExtension. This works ok with Java 11 and Flink
1.17.2 and Flink 1.18.1.
With Java 17, I see
{{[WARN ] IcebergStreamWriter (1/1)#1
(78d9251dbbab3aae84bf303dfc080d23_626c1e687bcaad0c13c507629a5894a8_0_1)
switched from RUNNING to FAILED with failure cause:}}
{{java.io.IOException: Could not perform checkpoint 2 for operator
IcebergStreamWriter (1/1)#1.}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1275)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.notifyCheckpoint(CheckpointBarrierHandler.java:147)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.triggerCheckpoint(SingleCheckpointBarrierHandler.java:287)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.access$100(SingleCheckpointBarrierHandler.java:64)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler$ControllerImpl.triggerGlobalCheckpoint(SingleCheckpointBarrierHandler.java:488)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.triggerGlobalCheckpoint(AbstractAlignedBarrierHandlerState.java:74)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.barrierReceived(AbstractAlignedBarrierHandlerState.java:66)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.lambda$processBarrier$2(SingleCheckpointBarrierHandler.java:234)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.markCheckpointAlignedAndTransformState(SingleCheckpointBarrierHandler.java:262)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.processBarrier(SingleCheckpointBarrierHandler.java:231)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.handleEvent(CheckpointedInputGate.java:181)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.pollNext(CheckpointedInputGate.java:159)}}
{{at
org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:118)}}
{{at
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)}}
{{at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)}}
{{at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)}}
{{at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)}}
{{at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)}}
{{at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)}}
{{at java.base/java.lang.Thread.run(Thread.java:840)}}
{{Caused by: java.lang.RuntimeException:
java.lang.reflect.InaccessibleObjectException: Unable to make field private
final java.lang.Object[] java.util.Arrays$ArrayList.a accessible: module
java.base does not "opens java.util" to unnamed module @6af93788}}
{{at
com.twitter.chill.java.ArraysAsListSerializer.(ArraysAsListSerializer.java:69)}}
{{at
org.apache.flink.api.java.typeutils.runtime.kryo.FlinkChillPackageRegistrar.registerSerializers(FlinkChillPackageRegistrar.java:67)}}
{{at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.getKryoInstance(KryoSerializer.java:513)}}
{{at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.checkKryoInitialized(KryoSerializer.java:522)}}
{{at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.serialize(KryoSerializer.java:348)}}
{{at
org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:165)}}
{{at
org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:43)}}
{{at
org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:54)}}
{{at
Re: [DISCUSS] FLIP-399: Flink Connector Doris
Hi Di, Thanks for your proposal. +1 for the contribution. I'd like to know your thoughts about the following questions: 1. According to your clarification of the exactly-once, thanks for it BTW, no PreCommitTopology is required. Does it make sense to let DorisSink[1] implement SupportsCommitter, since the TwoPhaseCommittingSink is deprecated[2] before turning the Doris connector into a Flink connector? 2. OLAP engines are commonly used as the tail/downstream of a data pipeline to support further e.g. ad-hoc query or cube with feasible pre-aggregation. Just out of curiosity, would you like to share some real use cases that will use OLAP engines as the source of a streaming data pipeline? Or it will only be used as the source for the batch? 3. The E2E test only covered sink[3], if I am not mistaken. Would you like to test the source in E2E too? [1] https://github.com/apache/doris-flink-connector/blob/43e0e5cf9b832854ea228fb093077872e3a311b6/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/DorisSink.java#L55 [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-372%3A+Enhance+and+synchronize+Sink+API+to+match+the+Source+API [3] https://github.com/apache/doris-flink-connector/blob/43e0e5cf9b832854ea228fb093077872e3a311b6/flink-doris-connector/src/test/java/org/apache/doris/flink/tools/cdc/MySQLDorisE2ECase.java#L96 Best regards, Jing On Tue, Mar 5, 2024 at 11:18 AM wudi <676366...@qq.com.invalid> wrote: > Hi, Jeyhun Karimov. > Thanks for your question. > > - How to ensure Exactly-Once? > 1. When the Checkpoint Barrier arrives, DorisSink will trigger the > precommit api of StreamLoad to complete the persistence of data in Doris > (the data will not be visible at this time), and will also pass this TxnID > to the Committer. > 2. When this Checkpoint of the entire Job is completed, the Committer will > call the commit api of StreamLoad and commit TxnID to complete the > visibility of the transaction. > 3. When the task is restarted, the Txn with successful precommit and > failed commit will be aborted based on the label-prefix, and Doris' abort > API will be called. (At the same time, Doris will also abort transactions > that have not been committed for a long time) > > ps: At the same time, this part of the content has been updated in FLIP > > - Because the default table model in Doris is Duplicate ( > https://doris.apache.org/docs/data-table/data-model/), which does not > have a primary key, batch writing may cause data duplication, but UNIQ The > model has a primary key, which ensures the idempotence of writing, thus > achieving Exactly-Once > > Brs, > di.wu > > > > 2024年3月2日 17:50，Jeyhun Karimov 写道： > > > > Hi, > > > > Thanks for the proposal. +1 for the FLIP. > > I have a few questions: > > > > - How exactly the two (Stream Load's two-phase commit and Flink's > two-phase > > commit) combination will ensure the e2e exactly-once semantics? > > > > - The FLIP proposes to combine Doris's batch writing with the primary key > > table to achieve Exactly-Once semantics. Could you elaborate more on > that? > > Why it is not the default behavior but a workaround? > > > > Regards, > > Jeyhun > > > > On Sat, Mar 2, 2024 at 10:14 AM Yanquan Lv wrote: > > > >> Thanks for driving this. > >> The content is very detailed, it is recommended to add a section on Test > >> Plan for more completeness. > >> > >> Di Wu 于2024年1月25日周四 15:40写道： > >> > >>> Hi all, > >>> > >>> Previously, we had some discussions about contributing Flink Doris > >>> Connector to the Flink community [1]. I want to further promote this > >> work. > >>> I hope everyone will help participate in this FLIP discussion and > provide > >>> more valuable opinions and suggestions. > >>> Thanks. > >>> > >>> [1] https://lists.apache.org/thread/lvh8g9o6qj8bt3oh60q81z0o1cv3nn8p > >>> > >>> Brs, > >>> di.wu > >>> > >>> > >>> > >>> On 2023/12/07 05:02:46 wudi wrote: > > Hi all, > > As discussed in the previous email [1], about contributing the Flink > >>> Doris Connector to the Flink community. > > > Apache Doris[2] is a high-performance, real-time analytical database > >>> based on MPP architecture, for scenarios where Flink is used for data > >>> analysis, processing, or real-time writing on Doris, Flink Doris > >> Connector > >>> is an effective tool. > > At the same time, Contributing Flink Doris Connector to the Flink > >>> community will further expand the Flink Connectors ecosystem. > > So I would like to start an official discussion FLIP-399: Flink > >>> Connector Doris[3]. > > Looking forward to comments, feedbacks and suggestions from the > >>> community on the proposal. > > [1] https://lists.apache.org/thread/lvh8g9o6qj8bt3oh60q81z0o1cv3nn8p > [2] > >> https://doris.apache.org/docs/dev/get-starting/what-is-apache-doris/ > [3] > >>> > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-399%3A+Flink+Connector+Doris > > > Brs,
Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website
Hi Max, Thanks for sharing your concerns :D I'd elaborate a bit on this topic with an example, that Apache Airflow has a small section for its special sponsor who offers machines for CI also [1]. In my understanding, companies employ developers to invest time in the development of Flink and that is large, vague, and hard to be fair to list all of the companies. However, physical resources like CI machines are countable and they help the sustainability of our project in a rare way different than individuals (few individuals can donate such resources). We can maintain such a section or page for those sponsors so that it also decreases the friction when the company asks "what we can gain" (for explicit credits, at least, and easy understanding). Any entity is welcome to add themselves as long as it's valid. For the fair part, I'm not an employee of both companies listed on the demo page and I don't feel uncomfortable. Those companies do invest a lot on our project and I'd regard it as a chance to encourage other companies to follow. Best, tison. [1] https://github.com/apache/airflow?tab=readme-ov-file#sponsors Maximilian Michels 于2024年3月6日周三 00:49写道： > > I'm a bit torn on this idea. On the one hand, it makes sense to thank > sponsors and entities who have supported Flink in the past. On other > hand, this list is bound to be incomplete and maybe also biased, even > if not intended to be so. I think the power of open-source comes from > the unconditional donation of code and knowledge. Infrastructure costs > are a reality and donations in that area are meaningful, but they are > just one piece of the total sum which consists of many volunteers and > working hours. In my eyes, a Thank You page would have to display each > entity fairly which is going to be hard to achieve. > > -Max > > On Tue, Mar 5, 2024 at 2:30 PM Jingsong Li wrote: > > > > +1 for setting up > > > > On Tue, Mar 5, 2024 at 5:39 PM Jing Ge wrote: > > > > > > +1 and thanks for the proposal! > > > > > > Best regards, > > > Jing > > > > > > On Tue, Mar 5, 2024 at 10:26 AM tison wrote: > > > > > > > I like this idea, so +1 for setting up. > > > > > > > > For anyone who have the access, this is a related thread about > > > > project-wise sponsor in the foundation level [1]. > > > > > > > > Best, > > > > tison. > > > > > > > > [1] https://lists.apache.org/thread/2nv0x9gfk9lfnpb2315xgywyx84y97v6 > > > > > > > > Jark Wu 于2024年3月5日周二 17:17写道： > > > > > > > > > > Sorry, I posted the wrong [7] link. The Flink benchmark ML link is: > > > > > https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 > > > > > > > > > > > > > > > On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > I want to propose adding a "Special Thanks" page to our Apache Flink > > > > website [1] > > > > > > > > > > > > to honor and appreciate the > > > > > > companies and organizations that have sponsored > > > > > > > > > > > > > > > > > > machines or services for our project. The establishment of such a > > > > > > page > > > > serves as > > > > > > > > > > > > > > > > > > a public acknowledgment of our sponsors' contributions and > > > > simultaneously acts > > > > > > > > > > > > > > > > > > as a positive encouragement for other entities to consider > > > > > > supporting > > > > our project. > > > > > > > > > > > > > > > > > > > > > > > > Adding Per-Project Thanks pages is allowed by ASF policy[2], which > > > > says "PMCs > > > > > > > > > > > > > > > > > > may wish to provide recognition for third parties that provide > > > > software or services > > > > > > > > > > > > > > > > > > to the project's committers to further the goals of the project. > > > > > > These > > > > are typically > > > > > > > > > > > > called Per-Project Thanks pages". Many Apache projects have added > > > > > > such > > > > > > > > > > > > pages, for example, Apache HBase[3] and Apache Mina[4]. > > > > > > > > > > > > > > > > > > To initiate this idea, I have drafted a preliminary page under the > > > > > > "About" menu > > > > > > > > > > > > on the > > > > > > Flink website to specifically thank Alibaba and Ververica, by > > > > > > following > > > > > > > > > > > > the ASF guidelines and the Apache Mina project. > > > > > > > > > > > > > > > > > > page image: > > > > > > > > > > https://github.com/apache/flink/assets/5378924/e51aaffe-565e-46d1-90af-3900904afcc0 > > > > > > > > > > > > > > > > > > > > > > > > Below companies are on the thanks list for their donation to Flink > > > > testing infrastructure: > > > > > > > > > > > > - Alibaba donated 8 machines (32vCPU,64GB) for running Flink CI > > > > > > builds > > > > [5]. > > > > > > > > > > > > > > > > > > - Ververica donated 2 machines for hosting flink-ci repositories [6] > > > > and running Flink benchmarks [7]. > > > > > > > > > > > > > > > > > > I may miss some other donations or companies, please add them if you > > > > know. > > > > > > > > > >
Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website
I'm a bit torn on this idea. On the one hand, it makes sense to thank sponsors and entities who have supported Flink in the past. On other hand, this list is bound to be incomplete and maybe also biased, even if not intended to be so. I think the power of open-source comes from the unconditional donation of code and knowledge. Infrastructure costs are a reality and donations in that area are meaningful, but they are just one piece of the total sum which consists of many volunteers and working hours. In my eyes, a Thank You page would have to display each entity fairly which is going to be hard to achieve. -Max On Tue, Mar 5, 2024 at 2:30 PM Jingsong Li wrote: > > +1 for setting up > > On Tue, Mar 5, 2024 at 5:39 PM Jing Ge wrote: > > > > +1 and thanks for the proposal! > > > > Best regards, > > Jing > > > > On Tue, Mar 5, 2024 at 10:26 AM tison wrote: > > > > > I like this idea, so +1 for setting up. > > > > > > For anyone who have the access, this is a related thread about > > > project-wise sponsor in the foundation level [1]. > > > > > > Best, > > > tison. > > > > > > [1] https://lists.apache.org/thread/2nv0x9gfk9lfnpb2315xgywyx84y97v6 > > > > > > Jark Wu 于2024年3月5日周二 17:17写道： > > > > > > > > Sorry, I posted the wrong [7] link. The Flink benchmark ML link is: > > > > https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 > > > > > > > > > > > > On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > I want to propose adding a "Special Thanks" page to our Apache Flink > > > website [1] > > > > > > > > > > to honor and appreciate the > > > > > companies and organizations that have sponsored > > > > > > > > > > > > > > > machines or services for our project. The establishment of such a page > > > serves as > > > > > > > > > > > > > > > a public acknowledgment of our sponsors' contributions and > > > simultaneously acts > > > > > > > > > > > > > > > as a positive encouragement for other entities to consider supporting > > > our project. > > > > > > > > > > > > > > > > > > > > Adding Per-Project Thanks pages is allowed by ASF policy[2], which > > > says "PMCs > > > > > > > > > > > > > > > may wish to provide recognition for third parties that provide > > > software or services > > > > > > > > > > > > > > > to the project's committers to further the goals of the project. These > > > are typically > > > > > > > > > > called Per-Project Thanks pages". Many Apache projects have added > > > > > such > > > > > > > > > > pages, for example, Apache HBase[3] and Apache Mina[4]. > > > > > > > > > > > > > > > To initiate this idea, I have drafted a preliminary page under the > > > > > "About" menu > > > > > > > > > > on the > > > > > Flink website to specifically thank Alibaba and Ververica, by > > > > > following > > > > > > > > > > the ASF guidelines and the Apache Mina project. > > > > > > > > > > > > > > > page image: > > > > > > > > https://github.com/apache/flink/assets/5378924/e51aaffe-565e-46d1-90af-3900904afcc0 > > > > > > > > > > > > > > > > > > > > Below companies are on the thanks list for their donation to Flink > > > testing infrastructure: > > > > > > > > > > - Alibaba donated 8 machines (32vCPU,64GB) for running Flink CI builds > > > [5]. > > > > > > > > > > > > > > > - Ververica donated 2 machines for hosting flink-ci repositories [6] > > > and running Flink benchmarks [7]. > > > > > > > > > > > > > > > I may miss some other donations or companies, please add them if you > > > know. > > > > > > > > > > Looking forward to your feedback about this proposal! > > > > > > > > > > > > > > > Best, > > > > > > > > > > Jark > > > > > > > > > > > > > > > [1]: https://flink.apache.org/ > > > > > > > > > > [2]: https://www.apache.org/foundation/marks/linking#projectthanks > > > > > > > > > > [3]: https://hbase.apache.org/sponsors.html > > > > > > > > > > [4]: https://mina.apache.org/special-thanks.html > > > > > > > > > > [5]: > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-AvailableCustomBuildMachines > > > > > > > > > > [6]: > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration > > > > > > > > > > [7]: > > > > > > > > https://lists.apache.org/thread.html/41a68c775753a7841896690c75438e0a497634102e676db880f30225@%3Cdev.flink.apache.org%3E > > > > > > > >
[VOTE] Release 1.19.0, release candidate #1
Hi everyone, Please review and vote on the release candidate #1 for the version 1.19.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], and the pull request adding release note for users [2] * the official Apache source release and binary convenience releases to be deployed to dist.apache.org [3], which are signed with the key with fingerprint E57D30ABEE75CA06 [4], * all artifacts to be deployed to the Maven Central Repository [5], * source code tag "release-1.19.0-rc1" [6], * website pull request listing the new release and adding announcement blog post [7]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353282 [2] https://github.com/apache/flink/pull/24394 [3] https://dist.apache.org/repos/dist/dev/flink/flink-1.19.0-rc1/ [4] https://dist.apache.org/repos/dist/release/flink/KEYS [5] https://repository.apache.org/content/repositories/orgapacheflink-1708 [6] https://github.com/apache/flink/releases/tag/release-1.19.0-rc1 [7] https://github.com/apache/flink-web/pull/721 Best, Yun, Jing, Martijn and Lincoln
Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website
+1 for setting up On Tue, Mar 5, 2024 at 5:39 PM Jing Ge wrote: > > +1 and thanks for the proposal! > > Best regards, > Jing > > On Tue, Mar 5, 2024 at 10:26 AM tison wrote: > > > I like this idea, so +1 for setting up. > > > > For anyone who have the access, this is a related thread about > > project-wise sponsor in the foundation level [1]. > > > > Best, > > tison. > > > > [1] https://lists.apache.org/thread/2nv0x9gfk9lfnpb2315xgywyx84y97v6 > > > > Jark Wu 于2024年3月5日周二 17:17写道： > > > > > > Sorry, I posted the wrong [7] link. The Flink benchmark ML link is: > > > https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 > > > > > > > > > On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > > > > > > > Hi all, > > > > > > > > > > > > > > > > I want to propose adding a "Special Thanks" page to our Apache Flink > > website [1] > > > > > > > > to honor and appreciate the > > > > companies and organizations that have sponsored > > > > > > > > > > > > machines or services for our project. The establishment of such a page > > serves as > > > > > > > > > > > > a public acknowledgment of our sponsors' contributions and > > simultaneously acts > > > > > > > > > > > > as a positive encouragement for other entities to consider supporting > > our project. > > > > > > > > > > > > > > > > Adding Per-Project Thanks pages is allowed by ASF policy[2], which > > says "PMCs > > > > > > > > > > > > may wish to provide recognition for third parties that provide > > software or services > > > > > > > > > > > > to the project's committers to further the goals of the project. These > > are typically > > > > > > > > called Per-Project Thanks pages". Many Apache projects have added such > > > > > > > > pages, for example, Apache HBase[3] and Apache Mina[4]. > > > > > > > > > > > > To initiate this idea, I have drafted a preliminary page under the > > > > "About" menu > > > > > > > > on the > > > > Flink website to specifically thank Alibaba and Ververica, by following > > > > > > > > the ASF guidelines and the Apache Mina project. > > > > > > > > > > > > page image: > > > > > > https://github.com/apache/flink/assets/5378924/e51aaffe-565e-46d1-90af-3900904afcc0 > > > > > > > > > > > > > > > > Below companies are on the thanks list for their donation to Flink > > testing infrastructure: > > > > > > > > - Alibaba donated 8 machines (32vCPU,64GB) for running Flink CI builds > > [5]. > > > > > > > > > > > > - Ververica donated 2 machines for hosting flink-ci repositories [6] > > and running Flink benchmarks [7]. > > > > > > > > > > > > I may miss some other donations or companies, please add them if you > > know. > > > > > > > > Looking forward to your feedback about this proposal! > > > > > > > > > > > > Best, > > > > > > > > Jark > > > > > > > > > > > > [1]: https://flink.apache.org/ > > > > > > > > [2]: https://www.apache.org/foundation/marks/linking#projectthanks > > > > > > > > [3]: https://hbase.apache.org/sponsors.html > > > > > > > > [4]: https://mina.apache.org/special-thanks.html > > > > > > > > [5]: > > > > > > https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-AvailableCustomBuildMachines > > > > > > > > [6]: > > > > > > https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration > > > > > > > > [7]: > > > > > > https://lists.apache.org/thread.html/41a68c775753a7841896690c75438e0a497634102e676db880f30225@%3Cdev.flink.apache.org%3E > > > > > >
[jira] [Created] (FLINK-34580) Job run via REST erases "pipeline.classpaths" config
Ferenc Csaky created FLINK-34580:
Summary: Job run via REST erases "pipeline.classpaths" config
Key: FLINK-34580
URL: https://issues.apache.org/jira/browse/FLINK-34580
Project: Flink
Issue Type: Bug
Components: Runtime / REST
Affects Versions: 1.18.1, 1.17.2, 1.19.0
Reporter: Ferenc Csaky
Fix For: 1.20.0
The
[{{JarHandlerContext#applyToConfiguration}}|https://github.com/apache/flink/blob/e0b6c121eaf7aeb2974a45d199e452b022f07d29/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/utils/JarHandlerUtils.java#L134]
creates a {{PackagedProgram}} and then overwrites the {{pipeline.jars}} and
{{pipeline.classpaths}} values according to that newly created
{{{}PackagedProgram{}}}.
Although that [{{PackagedProgram}}
init|https://github.com/apache/flink/blob/e0b6c121eaf7aeb2974a45d199e452b022f07d29/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/utils/JarHandlerUtils.java#L185]
does not set {{classpaths}} at all, so it will always overwrites the effective
configuration with an empty value, even if it had something previously.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34579) Introduce metric for time since last completed checkpoint
Stefan Richter created FLINK-34579: -- Summary: Introduce metric for time since last completed checkpoint Key: FLINK-34579 URL: https://issues.apache.org/jira/browse/FLINK-34579 Project: Flink Issue Type: Improvement Components: Runtime / Checkpointing Reporter: Stefan Richter Assignee: Stefan Richter Fix For: 1.20.0 This metric will help us to identify jobs with checkpointing problems without first requiring to complete or fail the checkpoint first before the problem surfaces. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] FLIP-399: Flink Connector Doris
Hi, Jeyhun Karimov. Thanks for your question. - How to ensure Exactly-Once? 1. When the Checkpoint Barrier arrives, DorisSink will trigger the precommit api of StreamLoad to complete the persistence of data in Doris (the data will not be visible at this time), and will also pass this TxnID to the Committer. 2. When this Checkpoint of the entire Job is completed, the Committer will call the commit api of StreamLoad and commit TxnID to complete the visibility of the transaction. 3. When the task is restarted, the Txn with successful precommit and failed commit will be aborted based on the label-prefix, and Doris' abort API will be called. (At the same time, Doris will also abort transactions that have not been committed for a long time) ps: At the same time, this part of the content has been updated in FLIP - Because the default table model in Doris is Duplicate (https://doris.apache.org/docs/data-table/data-model/), which does not have a primary key, batch writing may cause data duplication, but UNIQ The model has a primary key, which ensures the idempotence of writing, thus achieving Exactly-Once Brs, di.wu > 2024年3月2日 17:50，Jeyhun Karimov 写道： > > Hi, > > Thanks for the proposal. +1 for the FLIP. > I have a few questions: > > - How exactly the two (Stream Load's two-phase commit and Flink's two-phase > commit) combination will ensure the e2e exactly-once semantics? > > - The FLIP proposes to combine Doris's batch writing with the primary key > table to achieve Exactly-Once semantics. Could you elaborate more on that? > Why it is not the default behavior but a workaround? > > Regards, > Jeyhun > > On Sat, Mar 2, 2024 at 10:14 AM Yanquan Lv wrote: > >> Thanks for driving this. >> The content is very detailed, it is recommended to add a section on Test >> Plan for more completeness. >> >> Di Wu 于2024年1月25日周四 15:40写道： >> >>> Hi all, >>> >>> Previously, we had some discussions about contributing Flink Doris >>> Connector to the Flink community [1]. I want to further promote this >> work. >>> I hope everyone will help participate in this FLIP discussion and provide >>> more valuable opinions and suggestions. >>> Thanks. >>> >>> [1] https://lists.apache.org/thread/lvh8g9o6qj8bt3oh60q81z0o1cv3nn8p >>> >>> Brs, >>> di.wu >>> >>> >>> >>> On 2023/12/07 05:02:46 wudi wrote: Hi all, As discussed in the previous email [1], about contributing the Flink >>> Doris Connector to the Flink community. Apache Doris[2] is a high-performance, real-time analytical database >>> based on MPP architecture, for scenarios where Flink is used for data >>> analysis, processing, or real-time writing on Doris, Flink Doris >> Connector >>> is an effective tool. At the same time, Contributing Flink Doris Connector to the Flink >>> community will further expand the Flink Connectors ecosystem. So I would like to start an official discussion FLIP-399: Flink >>> Connector Doris[3]. Looking forward to comments, feedbacks and suggestions from the >>> community on the proposal. [1] https://lists.apache.org/thread/lvh8g9o6qj8bt3oh60q81z0o1cv3nn8p [2] >> https://doris.apache.org/docs/dev/get-starting/what-is-apache-doris/ [3] >>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-399%3A+Flink+Connector+Doris Brs, di.wu >>> >>
Re: [DISCUSS] FLIP-399: Flink Connector Doris
Thanks, Yanquan, Test Plan has been added at the end of FLIP[1]. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-399%3A+Flink+Connector+Doris Brs, di.wu > 2024年3月2日 17:13，Yanquan Lv 写道： > > Thanks for driving this. > The content is very detailed, it is recommended to add a section on Test > Plan for more completeness. > > Di Wu 于2024年1月25日周四 15:40写道： > >> Hi all, >> >> Previously, we had some discussions about contributing Flink Doris >> Connector to the Flink community [1]. I want to further promote this work. >> I hope everyone will help participate in this FLIP discussion and provide >> more valuable opinions and suggestions. >> Thanks. >> >> [1] https://lists.apache.org/thread/lvh8g9o6qj8bt3oh60q81z0o1cv3nn8p >> >> Brs, >> di.wu >> >> >> >> On 2023/12/07 05:02:46 wudi wrote: >>> >>> Hi all, >>> >>> As discussed in the previous email [1], about contributing the Flink >> Doris Connector to the Flink community. >>> >>> >>> Apache Doris[2] is a high-performance, real-time analytical database >> based on MPP architecture, for scenarios where Flink is used for data >> analysis, processing, or real-time writing on Doris, Flink Doris Connector >> is an effective tool. >>> >>> At the same time, Contributing Flink Doris Connector to the Flink >> community will further expand the Flink Connectors ecosystem. >>> >>> So I would like to start an official discussion FLIP-399: Flink >> Connector Doris[3]. >>> >>> Looking forward to comments, feedbacks and suggestions from the >> community on the proposal. >>> >>> [1] https://lists.apache.org/thread/lvh8g9o6qj8bt3oh60q81z0o1cv3nn8p >>> [2] https://doris.apache.org/docs/dev/get-starting/what-is-apache-doris/ >>> [3] >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-399%3A+Flink+Connector+Doris >>> >>> >>> Brs, >>> >>> di.wu >>> >>
Re: Temporal join on rolling aggregate
Hi Everyone! I have discussed this with Sébastien Chevalley, he is going to prepare and drive the FLIP while I will assist him along the way. Thanks Gyula On Tue, Mar 5, 2024 at 9:57 AM wrote: > I do agree with Ron Liu. > This would definitely need a FLIP as it would impact SQL and extend it > with the equivalent of TimestampAssigners in the Java API. > > Is there any existing JIRA here, or is anybody willing to drive a FLIP? > On Feb 26, 2024 at 02:36 +0100, Ron liu , wrote: > > +1, > But I think this should be a more general requirement, that is, support for > declaring watermarks in query, which can be declared for any type of > source, such as table, view. Similar to databricks provided [1], this needs > a FLIP. > > [1] > > https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-qry-select-watermark.html > > Best, > Ron > >
Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website
+1 and thanks for the proposal! Best regards, Jing On Tue, Mar 5, 2024 at 10:26 AM tison wrote: > I like this idea, so +1 for setting up. > > For anyone who have the access, this is a related thread about > project-wise sponsor in the foundation level [1]. > > Best, > tison. > > [1] https://lists.apache.org/thread/2nv0x9gfk9lfnpb2315xgywyx84y97v6 > > Jark Wu 于2024年3月5日周二 17:17写道： > > > > Sorry, I posted the wrong [7] link. The Flink benchmark ML link is: > > https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 > > > > > > On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > > > > > Hi all, > > > > > > > > > > > > I want to propose adding a "Special Thanks" page to our Apache Flink > website [1] > > > > > > to honor and appreciate the > > > companies and organizations that have sponsored > > > > > > > > > machines or services for our project. The establishment of such a page > serves as > > > > > > > > > a public acknowledgment of our sponsors' contributions and > simultaneously acts > > > > > > > > > as a positive encouragement for other entities to consider supporting > our project. > > > > > > > > > > > > Adding Per-Project Thanks pages is allowed by ASF policy[2], which > says "PMCs > > > > > > > > > may wish to provide recognition for third parties that provide > software or services > > > > > > > > > to the project's committers to further the goals of the project. These > are typically > > > > > > called Per-Project Thanks pages". Many Apache projects have added such > > > > > > pages, for example, Apache HBase[3] and Apache Mina[4]. > > > > > > > > > To initiate this idea, I have drafted a preliminary page under the > > > "About" menu > > > > > > on the > > > Flink website to specifically thank Alibaba and Ververica, by following > > > > > > the ASF guidelines and the Apache Mina project. > > > > > > > > > page image: > > > > https://github.com/apache/flink/assets/5378924/e51aaffe-565e-46d1-90af-3900904afcc0 > > > > > > > > > > > > Below companies are on the thanks list for their donation to Flink > testing infrastructure: > > > > > > - Alibaba donated 8 machines (32vCPU,64GB) for running Flink CI builds > [5]. > > > > > > > > > - Ververica donated 2 machines for hosting flink-ci repositories [6] > and running Flink benchmarks [7]. > > > > > > > > > I may miss some other donations or companies, please add them if you > know. > > > > > > Looking forward to your feedback about this proposal! > > > > > > > > > Best, > > > > > > Jark > > > > > > > > > [1]: https://flink.apache.org/ > > > > > > [2]: https://www.apache.org/foundation/marks/linking#projectthanks > > > > > > [3]: https://hbase.apache.org/sponsors.html > > > > > > [4]: https://mina.apache.org/special-thanks.html > > > > > > [5]: > > > > https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-AvailableCustomBuildMachines > > > > > > [6]: > > > > https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration > > > > > > [7]: > > > > https://lists.apache.org/thread.html/41a68c775753a7841896690c75438e0a497634102e676db880f30225@%3Cdev.flink.apache.org%3E > > > >
Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website
I like this idea, so +1 for setting up. For anyone who have the access, this is a related thread about project-wise sponsor in the foundation level [1]. Best, tison. [1] https://lists.apache.org/thread/2nv0x9gfk9lfnpb2315xgywyx84y97v6 Jark Wu 于2024年3月5日周二 17:17写道： > > Sorry, I posted the wrong [7] link. The Flink benchmark ML link is: > https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 > > > On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > > > Hi all, > > > > > > > > I want to propose adding a "Special Thanks" page to our Apache Flink > > website [1] > > > > to honor and appreciate the > > companies and organizations that have sponsored > > > > > > machines or services for our project. The establishment of such a page > > serves as > > > > > > a public acknowledgment of our sponsors' contributions and simultaneously > > acts > > > > > > as a positive encouragement for other entities to consider supporting our > > project. > > > > > > > > Adding Per-Project Thanks pages is allowed by ASF policy[2], which says > > "PMCs > > > > > > may wish to provide recognition for third parties that provide software or > > services > > > > > > to the project's committers to further the goals of the project. These are > > typically > > > > called Per-Project Thanks pages". Many Apache projects have added such > > > > pages, for example, Apache HBase[3] and Apache Mina[4]. > > > > > > To initiate this idea, I have drafted a preliminary page under the > > "About" menu > > > > on the > > Flink website to specifically thank Alibaba and Ververica, by following > > > > the ASF guidelines and the Apache Mina project. > > > > > > page image: > > https://github.com/apache/flink/assets/5378924/e51aaffe-565e-46d1-90af-3900904afcc0 > > > > > > > > Below companies are on the thanks list for their donation to Flink testing > > infrastructure: > > > > - Alibaba donated 8 machines (32vCPU,64GB) for running Flink CI builds [5]. > > > > > > - Ververica donated 2 machines for hosting flink-ci repositories [6] and > > running Flink benchmarks [7]. > > > > > > I may miss some other donations or companies, please add them if you know. > > > > Looking forward to your feedback about this proposal! > > > > > > Best, > > > > Jark > > > > > > [1]: https://flink.apache.org/ > > > > [2]: https://www.apache.org/foundation/marks/linking#projectthanks > > > > [3]: https://hbase.apache.org/sponsors.html > > > > [4]: https://mina.apache.org/special-thanks.html > > > > [5]: > > https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-AvailableCustomBuildMachines > > > > [6]: > > https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration > > > > [7]: > > https://lists.apache.org/thread.html/41a68c775753a7841896690c75438e0a497634102e676db880f30225@%3Cdev.flink.apache.org%3E > >
Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website
Sorry, I posted the wrong [7] link. The Flink benchmark ML link is: https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 On Tue, 5 Mar 2024 at 16:56, Jark Wu wrote: > Hi all, > > > > I want to propose adding a "Special Thanks" page to our Apache Flink website > [1] > > to honor and appreciate the > companies and organizations that have sponsored > > > machines or services for our project. The establishment of such a page serves > as > > > a public acknowledgment of our sponsors' contributions and simultaneously acts > > > as a positive encouragement for other entities to consider supporting our > project. > > > > Adding Per-Project Thanks pages is allowed by ASF policy[2], which says "PMCs > > > may wish to provide recognition for third parties that provide software or > services > > > to the project's committers to further the goals of the project. These are > typically > > called Per-Project Thanks pages". Many Apache projects have added such > > pages, for example, Apache HBase[3] and Apache Mina[4]. > > > To initiate this idea, I have drafted a preliminary page under the > "About" menu > > on the > Flink website to specifically thank Alibaba and Ververica, by following > > the ASF guidelines and the Apache Mina project. > > > page image: > https://github.com/apache/flink/assets/5378924/e51aaffe-565e-46d1-90af-3900904afcc0 > > > > Below companies are on the thanks list for their donation to Flink testing > infrastructure: > > - Alibaba donated 8 machines (32vCPU,64GB) for running Flink CI builds [5]. > > > - Ververica donated 2 machines for hosting flink-ci repositories [6] and > running Flink benchmarks [7]. > > > I may miss some other donations or companies, please add them if you know. > > Looking forward to your feedback about this proposal! > > > Best, > > Jark > > > [1]: https://flink.apache.org/ > > [2]: https://www.apache.org/foundation/marks/linking#projectthanks > > [3]: https://hbase.apache.org/sponsors.html > > [4]: https://mina.apache.org/special-thanks.html > > [5]: > https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-AvailableCustomBuildMachines > > [6]: > https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration > > [7]: > https://lists.apache.org/thread.html/41a68c775753a7841896690c75438e0a497634102e676db880f30225@%3Cdev.flink.apache.org%3E >
Re: Temporal join on rolling aggregate
I do agree with Ron Liu. This would definitely need a FLIP as it would impact SQL and extend it with the equivalent of TimestampAssigners in the Java API. Is there any existing JIRA here, or is anybody willing to drive a FLIP? On Feb 26, 2024 at 02:36 +0100, Ron liu , wrote: > +1, > But I think this should be a more general requirement, that is, support for > declaring watermarks in query, which can be declared for any type of > source, such as table, view. Similar to databricks provided [1], this needs > a FLIP. > > [1] > https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-qry-select-watermark.html > > Best, > Ron
[DISCUSS] Add "Special Thanks" Page on the Flink Website
Hi all, I want to propose adding a "Special Thanks" page to our Apache Flink website [1] to honor and appreciate the companies and organizations that have sponsored machines or services for our project. The establishment of such a page serves as a public acknowledgment of our sponsors' contributions and simultaneously acts as a positive encouragement for other entities to consider supporting our project. Adding Per-Project Thanks pages is allowed by ASF policy[2], which says "PMCs may wish to provide recognition for third parties that provide software or services to the project's committers to further the goals of the project. These are typically called Per-Project Thanks pages". Many Apache projects have added such pages, for example, Apache HBase[3] and Apache Mina[4]. To initiate this idea, I have drafted a preliminary page under the "About" menu on the Flink website to specifically thank Alibaba and Ververica, by following the ASF guidelines and the Apache Mina project. page image: https://github.com/apache/flink/assets/5378924/e51aaffe-565e-46d1-90af-3900904afcc0 Below companies are on the thanks list for their donation to Flink testing infrastructure: - Alibaba donated 8 machines (32vCPU,64GB) for running Flink CI builds [5]. - Ververica donated 2 machines for hosting flink-ci repositories [6] and running Flink benchmarks [7]. I may miss some other donations or companies, please add them if you know. Looking forward to your feedback about this proposal! Best, Jark [1]: https://flink.apache.org/ [2]: https://www.apache.org/foundation/marks/linking#projectthanks [3]: https://hbase.apache.org/sponsors.html [4]: https://mina.apache.org/special-thanks.html [5]: https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-AvailableCustomBuildMachines [6]: https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration [7]: https://lists.apache.org/thread.html/41a68c775753a7841896690c75438e0a497634102e676db880f30225@%3Cdev.flink.apache.org%3E
[jira] [Created] (FLINK-34578) Remove issue template
Zhongqiang Gong created FLINK-34578: --- Summary: Remove issue template Key: FLINK-34578 URL: https://issues.apache.org/jira/browse/FLINK-34578 Project: Flink Issue Type: Sub-task Components: Flink CDC Reporter: Zhongqiang Gong Flink cdc had done repo migration. So we should remove issue template. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34577) Add IssueNavigationLink for IDEA git log
Zhongqiang Gong created FLINK-34577: --- Summary: Add IssueNavigationLink for IDEA git log Key: FLINK-34577 URL: https://issues.apache.org/jira/browse/FLINK-34577 Project: Flink Issue Type: Sub-task Components: Flink CDC Reporter: Zhongqiang Gong Add IssueNavigationLink for IDEA git log -- This message was sent by Atlassian Jira (v8.20.10#820010)
