Re: Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

Congratulations! Thank you! All of the Flink community! Best, Jingsong On Tue, Jul 4, 2023 at 1:24 PM tison wrote: > > Congrats and with honor :D > > Best, > tison. > > > Mang Zhang 于2023年7月4日周二 11:08写道： > > > Congratulations!-- > > > > Best regards, > > Mang Zhang > > > > > > > > > > > > 在

Re: Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

Congrats and with honor :D Best, tison. Mang Zhang 于2023年7月4日周二 11:08写道： > Congratulations!-- > > Best regards, > Mang Zhang > > > > > > 在 2023-07-04 01:53:46，"liu ron" 写道： > >Congrats everyone > > > >Best, > >Ron > > > >Jark Wu 于2023年7月3日周一 22:48写道： > > > >> Congrats everyone! > >> > >>

Re: [VOTE] FLIP-321: introduce an API deprecation process

+1 binding On Tue, Jul 4, 2023 at 10:40 AM Zhu Zhu wrote: > > +1 (binding) > > Thanks, > Zhu > > ConradJam 于2023年7月3日周一 22:39写道： > > > > +1 (no-binding) > > > > Matthias Pohl 于2023年7月3日周一 22:33写道： > > > > > Thanks, Becket > > > > > > +1 (binding) > > > > > > On Mon, Jul 3, 2023 at 10:44 AM

Re: [DISCUSS] Deprecate SourceFunction APIs

> do we have any plan to offer a lighter Source API to decrease the connector > development cost? I remember mentioning it many times, but no contributor did it. ToT Best, Jingsong On Tue, Jul 4, 2023 at 11:01 AM Leonard Xu wrote: > > +1 to deprecate. > +1 for David’s points. > > I’ve one

Re:Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

Congratulations!-- Best regards, Mang Zhang 在 2023-07-04 01:53:46，"liu ron" 写道： >Congrats everyone > >Best, >Ron > >Jark Wu 于2023年7月3日周一 22:48写道： > >> Congrats everyone! >> >> Best, >> Jark >> >> > 2023年7月3日 22:37，Yuval Itzchakov 写道： >> > >> > Congrats team! >> > >> > On Mon, Jul 3, 2023,

Re: [DISCUSS] Deprecate SourceFunction APIs

+1 to deprecate. +1 for David’s points. I’ve one related question, do we have any plan to offer a lighter Source API to decrease the connector development cost? New Source API is good but too heavy for use cases like tests or even some simple connectors. Best, Leonard > On Jun 6, 2022, at

Re: [VOTE] FLIP-321: introduce an API deprecation process

+1 (binding) Thanks, Zhu ConradJam 于2023年7月3日周一 22:39写道： > > +1 (no-binding) > > Matthias Pohl 于2023年7月3日周一 22:33写道： > > > Thanks, Becket > > > > +1 (binding) > > > > On Mon, Jul 3, 2023 at 10:44 AM Jing Ge > > wrote: > > > > > +1(binding) > > > > > > On Mon, Jul 3, 2023 at 10:19 AM Stefan

[jira] [Created] (FLINK-32523) NotifyCheckpointAbortedITCase.testNotifyCheckpointAborted fails with timeout on AZP

Sergey Nuyanzin created FLINK-32523: --- Summary: NotifyCheckpointAbortedITCase.testNotifyCheckpointAborted fails with timeout on AZP Key: FLINK-32523 URL: https://issues.apache.org/jira/browse/FLINK-32523

Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

Congrats everyone Best, Ron Jark Wu 于2023年7月3日周一 22:48写道： > Congrats everyone! > > Best, > Jark > > > 2023年7月3日 22:37，Yuval Itzchakov 写道： > > > > Congrats team! > > > > On Mon, Jul 3, 2023, 17:28 Jing Ge via user > wrote: > >> Congratulations! > >> > >> Best

Re: Working improving the REST API

Just adding the updates from the Slack thread - @SamratDeb and @JulietLee have expressed interest, and there is a PR ready for review! https://github.com/apache/flink/pull/22901 Regards, Hong On 3 Jul 2023, at 16:56, Teoh, Hong wrote: CAUTION: This email originated from outside of the

Re: [DISCUSS] Deprecate SourceFunction APIs

I am happy to announce that the blocker has been resolved and SourceFunction is now marked as @Deprecated [1]. The work continues to remove the dependencies on the SourceFunction API in Flink internals in order to prepare for dropping it completely in Flink 2.0. I'd like to get some opinions on

Working improving the REST API

Hi all, (This is a cross-post from a Slack message posted in #dev [1]. Posting here for visibility) Recently I’ve been looking into improving the REST API that Flink has, to make it more generally available for programmatic access (e.g. control functions) rather than just for the Flink

Re: [DISCUSS] Flink and Externalized connectors leads to block and circular dependency problems

@Martijn thanks for clear explanations. If we follow the line you specified (Connectors shouldn't rely on dependencies that may or may not be available in Flink itself) It seems that we should add a certain dependency if we need(such as commons-io, commons-collection) in connector pom explicitly.

Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

Congrats everyone! Best, Jark > 2023年7月3日 22:37，Yuval Itzchakov 写道： > > Congrats team! > > On Mon, Jul 3, 2023, 17:28 Jing Ge via user > wrote: >> Congratulations! >> >> Best regards, >> Jing >> >> >> On Mon, Jul 3, 2023 at 3:21 PM yuxia >

Re: [VOTE] FLIP-322 Cooldown period for adaptive scheduler

+1 (no-binding) Etienne Chauchot 于2023年7月3日周一 15:57写道： > Hi all, > > The vote closes within 6 hours and, as for now, there was no vote. This > is a very short FLIP, that takes a few minutes to read. > > Please cast your vote so that the development could start. > > Thanks. > > Best > > Etienne

Re: [VOTE] FLIP-321: introduce an API deprecation process

+1 (no-binding) Matthias Pohl 于2023年7月3日周一 22:33写道： > Thanks, Becket > > +1 (binding) > > On Mon, Jul 3, 2023 at 10:44 AM Jing Ge > wrote: > > > +1(binding) > > > > On Mon, Jul 3, 2023 at 10:19 AM Stefan Richter > > wrote: > > > > > +1 (binding) > > > > > > > > > > On 3. Jul 2023, at 10:08,

Re: [DISCUSS] Release 2.0 Work Items

Hi Community: I see some tasks in the 2.0 list that haven't been assigned yet. I want to take the initiative to take on some tasks that I can complete. How do I apply to the community for this part of the task? I am interested in the following parts of FLINK-32377

Re: [VOTE] FLIP-321: introduce an API deprecation process

Thanks, Becket +1 (binding) On Mon, Jul 3, 2023 at 10:44 AM Jing Ge wrote: > +1(binding) > > On Mon, Jul 3, 2023 at 10:19 AM Stefan Richter > wrote: > > > +1 (binding) > > > > > > > On 3. Jul 2023, at 10:08, Martijn Visser > > wrote: > > > > > > +1 (binding) > > > > > > > > > > > > On Mon,

Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

Congratulations! Best regards, Jing On Mon, Jul 3, 2023 at 3:21 PM yuxia wrote: > Congratulations! > > Best regards, > Yuxia > > -- > *发件人: *"Pushpa Ramakrishnan" > *收件人: *"Xintong Song" > *抄送: *"dev" , "User" > *发送时间: *星期一, 2023年 7 月 03日 下午 8:36:30 > *主题: *Re:

Re: [DISCUSS] Flink and Externalized connectors leads to block and circular dependency problems

Hi Ran Tao, Thanks for opening this topic. I think there's a couple of things at hand: 1. Connectors shouldn't rely on dependencies that may or may not be available in Flink itself, like we've seen with flink-shaded. That avoids a tight coupling between Flink and connectors, which is exactly what

[DISCUSS] Flink and Externalized connectors leads to block and circular dependency problems

I have an issue here that needs to upgrade commons-collections[1] (this is an example), but PR ci fails because flink-python test cases depend on flink-sql-connector-kafka, but kafka-sql-connector is a small jar, does not include this dependency, so flink ci cause exception[2]. Current my solution

Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

Congratulations! Best regards, Yuxia 发件人: "Pushpa Ramakrishnan" 收件人: "Xintong Song" 抄送: "dev" , "User" 发送时间: 星期一, 2023年 7 月 03日 下午 8:36:30 主题: Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award Congratulations \uD83E\uDD73 On 03-Jul-2023, at 3:30 PM, Xintong Song

[jira] [Created] (FLINK-32522) Flink sql connector kafka should include commons-collections in shade phase

Ran Tao created FLINK-32522: --- Summary: Flink sql connector kafka should include commons-collections in shade phase Key: FLINK-32522 URL: https://issues.apache.org/jira/browse/FLINK-32522 Project: Flink

[jira] [Created] (FLINK-32521) Support GRPC Read/Write via GCS Connector

Jayadeep Jayaraman created FLINK-32521: -- Summary: Support GRPC Read/Write via GCS Connector Key: FLINK-32521 URL: https://issues.apache.org/jira/browse/FLINK-32521 Project: Flink Issue

Re: [DISCUSS] Graduate the FileSink to @PublicEvolving

Hi, If there are no other concerns, I will start voting. Thanks! Best Regards, Jing On Mon, Jun 26, 2023 at 11:35 AM Jing Ge wrote: > Hi, > > @Galen @Yuxia > > Your points are valid. Speaking of removing deprecated API, I have the > same concern. As a matter of fact, I have been raising it in

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Xingtong, Option 1, secure plan would be: 1. graduate kafka, File, JDBC connectors to @Public 2. graduate SinkV2 to @Public 3. remove SinkFunction. Option 2, risky plan but at a fast pace: 1. graduate SinkV2 to @Public and expecting more maintenance effort since there are many known and

Re: [DISCUSS] Release 2.0 Work Items

Thanks Xintong! I am +1 on the change. Best Yuan On Mon, Jul 3, 2023 at 6:20 PM Jing Ge wrote: > Hi Sergey, > > Thanks for the clarification! I will not hijack this thread to discuss > Scala code strategy. > > Best regards, > Jing > > On Mon, Jul 3, 2023 at 10:51 AM Sergey Nuyanzin > wrote:

[jira] [Created] (FLINK-32520) FlinkDeployment recovered states from an obsolete savepoint

Ruibin Xing created FLINK-32520: --- Summary: FlinkDeployment recovered states from an obsolete savepoint Key: FLINK-32520 URL: https://issues.apache.org/jira/browse/FLINK-32520 Project: Flink

Re: [DISCUSS] FLIP-309: Enable operators to trigger checkpoints dynamically

Hi Piotr, Please see my comments inline. On Mon, Jul 3, 2023 at 5:19 PM Piotr Nowojski wrote: > Hi Dong, > > Starting from the end: > > > It seems that the only benefit of this approach is to avoid" > > adding SplitEnumeratorContext#setIsProcessingBacklog." > > Yes, that's the major benefit of

[jira] [Created] (FLINK-32518) Enable atomicity for [CREATE OR] REPLACE table as statement

luoyuxia created FLINK-32518: Summary: Enable atomicity for [CREATE OR] REPLACE table as statement Key: FLINK-32518 URL: https://issues.apache.org/jira/browse/FLINK-32518 Project: Flink Issue

[jira] [Created] (FLINK-32519) Add doc for [CREATE OR] REPLACE TABLE AS statement

luoyuxia created FLINK-32519: Summary: Add doc for [CREATE OR] REPLACE TABLE AS statement Key: FLINK-32519 URL: https://issues.apache.org/jira/browse/FLINK-32519 Project: Flink Issue Type:

[jira] [Created] (FLINK-32517) Support to execute [CREATE OR] REPLACE TABLE AS statement

luoyuxia created FLINK-32517: Summary: Support to execute [CREATE OR] REPLACE TABLE AS statement Key: FLINK-32517 URL: https://issues.apache.org/jira/browse/FLINK-32517 Project: Flink Issue

[jira] [Created] (FLINK-32516) Support to parse REPLACE TABLE AS statement

luoyuxia created FLINK-32516: Summary: Support to parse REPLACE TABLE AS statement Key: FLINK-32516 URL: https://issues.apache.org/jira/browse/FLINK-32516 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32515) FLIP-303: Support REPLACE TABLE AS SELECT statement

luoyuxia created FLINK-32515: Summary: FLIP-303: Support REPLACE TABLE AS SELECT statement Key: FLINK-32515 URL: https://issues.apache.org/jira/browse/FLINK-32515 Project: Flink Issue Type: New

Re: [DISCUSS] Release 2.0 Work Items

Hi Sergey, Thanks for the clarification! I will not hijack this thread to discuss Scala code strategy. Best regards, Jing On Mon, Jul 3, 2023 at 10:51 AM Sergey Nuyanzin wrote: > Hi Jing, > > Maybe I was not clear enough, sorry. > However the main reason for this item about Calcite rules is

Re: [VOTE] FLIP-303: Support REPLACE TABLE AS SELECT statement

Hi, all. FLIP-303[1] has been accepted. There are 3 binding votes, and 3 non-binding votes: - Jing Ge (binding) - Lincoln (binding) - Yuxia Luo (binding) - Mang Zhang (non-binding) - Ron (non-binding) - Feng Jin (non-binding) Thanks everyone. [1]:

[jira] [Created] (FLINK-32514) FLIP-309: Support using larger checkpointing interval when source is processing backlog

Yunfeng Zhou created FLINK-32514: Summary: FLIP-309: Support using larger checkpointing interval when source is processing backlog Key: FLINK-32514 URL: https://issues.apache.org/jira/browse/FLINK-32514

[ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

Dear Community, I'm pleased to share this good news with everyone. As some of you may have already heard, Apache Flink has won the 2023 SIGMOD Systems Award [1]. "Apache Flink greatly expanded the use of stream data-processing." -- SIGMOD Awards Committee SIGMOD is one of the most influential

[jira] [Created] (FLINK-32513) Job in BATCH-mode with a significant number of transformations freezes on method StreamGraphGenerator.existsUnboundedSource()

Vladislav Keda created FLINK-32513: -- Summary: Job in BATCH-mode with a significant number of transformations freezes on method StreamGraphGenerator.existsUnboundedSource() Key: FLINK-32513 URL:

Re: [DISCUSS] FLIP-309: Enable operators to trigger checkpoints dynamically

Hi Dong, Starting from the end: > It seems that the only benefit of this approach is to avoid" > adding SplitEnumeratorContext#setIsProcessingBacklog." Yes, that's the major benefit of this counter-proposal. > In the target use-case, user still want to do checkpoint (though at a" > larger

Re: [DISCUSS] Release 2.0 Work Items

Hi Jing, Maybe I was not clear enough, sorry. However the main reason for this item about Calcite rules is not abandoning Scala. The main reason are changes in Calcite itself where there was introduced code generator framework (immutables) to generate config java classes for rules and old api

Re: [VOTE] FLIP-321: introduce an API deprecation process

+1(binding) On Mon, Jul 3, 2023 at 10:19 AM Stefan Richter wrote: > +1 (binding) > > > > On 3. Jul 2023, at 10:08, Martijn Visser > wrote: > > > > +1 (binding) > > > > > > > > On Mon, Jul 3, 2023 at 10:03 AM Xintong Song > wrote: > > > >> +1 (binding) > >> > >>

Re: [DISCUSS] Release 2.0 Work Items

Hi, Speaking of "Move Calcite rules from Scala to Java", I was wondering if this thread is the right place to talk about it. Afaik, the Flink community has decided to abandon Scala. That is the reason, I guess, we want to move those Calcite rules from Scala to Java. On the other side, new Scala

Re: [DISCUSS] Release 2.0 Work Items

Thanks all for the discussion. IIUC, we need to make the following changes. Please correct me if I get it wrong. 1. Disaggregated State Management - Clarify that only the public API related part is must-have for 2.0. 2. Java version support - Split it into 3 items: a) make java 17 the default

Re: [VOTE] FLIP-321: introduce an API deprecation process

+1 (binding) > On 3. Jul 2023, at 10:08, Martijn Visser wrote: > > +1 (binding) > > > > On Mon, Jul 3, 2023 at 10:03 AM Xintong Song > wrote: > >> +1 (binding) >> >> Best, >> >> Xintong >> >> >> >> On Sat, Jul 1, 2023 at 11:26 PM Dong Lin wrote: >> >>>

Re: [VOTE] FLIP-321: introduce an API deprecation process

+1 (binding) On Mon, Jul 3, 2023 at 10:03 AM Xintong Song wrote: > +1 (binding) > > Best, > > Xintong > > > > On Sat, Jul 1, 2023 at 11:26 PM Dong Lin wrote: > > > Thanks for the FLIP. > > > > +1 (binding) > > > > On Fri, Jun 30, 2023 at 5:39 PM Becket Qin wrote: > > > > > Hi folks, > > > >

Re: [DISCUSSION] test connectors against Flink master in PRs

Hi all, I wanted to post here the result of a discussion I had in private with Chesnay related to this subject. The question was regarding archunit with connectors: "How to deal with different archunit violations between 2 versions of Flink ?  If a violation is normal and should be added to

Re: [VOTE] FLIP-321: introduce an API deprecation process

+1 (binding) Best, Xintong On Sat, Jul 1, 2023 at 11:26 PM Dong Lin wrote: > Thanks for the FLIP. > > +1 (binding) > > On Fri, Jun 30, 2023 at 5:39 PM Becket Qin wrote: > > > Hi folks, > > > > I'd like to start the VOTE for FLIP-321[1] which proposes to introduce an > > API deprecation

Re: [VOTE] FLIP-322 Cooldown period for adaptive scheduler

Hi all, The vote closes within 6 hours and, as for now, there was no vote. This is a very short FLIP, that takes a few minutes to read. Please cast your vote so that the development could start. Thanks. Best Etienne Le 29/06/2023 à 15:51, Etienne Chauchot a écrit : Hi all, Thanks for

[jira] [Created] (FLINK-32512) SHOW JARS should not show the jars for temporary function

Fang Yong created FLINK-32512: - Summary: SHOW JARS should not show the jars for temporary function Key: FLINK-32512 URL: https://issues.apache.org/jira/browse/FLINK-32512 Project: Flink Issue