Re: [DISCUSS] Flink and Externalized connectors leads to block and circular dependency problems

2023-07-06 Thread Mason Chen
Hi all, I also agree with what's been said above. +1, I think the Table API delegation is a good suggestion--it essentially allows a connector to get Python support for free. We've seen that Table/SQL and Python APIs complement each other well and are ideal for data scientists. With respect to un

[jira] [Created] (FLINK-32556) Renames contenderID into componentId

2023-07-06 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32556: - Summary: Renames contenderID into componentId Key: FLINK-32556 URL: https://issues.apache.org/jira/browse/FLINK-32556 Project: Flink Issue Type: Sub-task

Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

2023-07-06 Thread Yuan Mei
Congrats everyone :-) Best Yuan On Fri, Jul 7, 2023 at 11:29 AM Hang Ruan wrote: > Hi, Leonard. > > I would like to help to add this page. Please assign this issue to me. > Thanks. > > Best, > Hang > > Leonard Xu 于2023年7月7日周五 11:26写道: > >> Congrats to all ! >> >> It will be helpful to promote

Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

2023-07-06 Thread Hang Ruan
Hi, Leonard. I would like to help to add this page. Please assign this issue to me. Thanks. Best, Hang Leonard Xu 于2023年7月7日周五 11:26写道: > Congrats to all ! > > It will be helpful to promote Apache Flink if we can add a page to our > website like others[2]. I’ve created an issue to improve this

Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

2023-07-06 Thread Leonard Xu
Congrats to all ! It will be helpful to promote Apache Flink if we can add a page to our website like others[2]. I’ve created an issue to improve this. Best, Leonard [1] https://issues.apache.org/jira/browse/FLINK-32555 [2] https://spark.apache.org/news/sigmod-system-award.html

[jira] [Created] (FLINK-32555) Add a page to show SIGMOD System Awards for Apache Flink

2023-07-06 Thread Leonard Xu (Jira)
Leonard Xu created FLINK-32555: -- Summary: Add a page to show SIGMOD System Awards for Apache Flink Key: FLINK-32555 URL: https://issues.apache.org/jira/browse/FLINK-32555 Project: Flink Issue Ty

Re: [ANNOUNCE] Flink 1.18 Feature Freeze Extended until July 24th, 2023

2023-07-06 Thread Shammon FY
Thanks for driving the release and sharing the update, looking forward to 1.18 Best, Shammon FY On Fri, Jul 7, 2023 at 10:56 AM Yun Tang wrote: > Thanks for driving this release and sharing the update on the feature > freeze extension. > > > Best > Yun Tang > >

Re: [ANNOUNCE] Flink 1.18 Feature Freeze Extended until July 24th, 2023

2023-07-06 Thread Yun Tang
Thanks for driving this release and sharing the update on the feature freeze extension. Best Yun Tang From: Jing Ge Sent: Thursday, July 6, 2023 17:13 To: yuxia Cc: dev ; re...@apache.org ; snuyan...@gmail.com ; Konstantin Knauf Subject: Re: [ANNOUNCE] Flink

Re: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

2023-07-06 Thread Shammon FY
Thanks Jing, sounds good to me. I have updated the FLIP and renamed the lineage related classes to `LineageGraph`, `LineageVertex` and `LineageEdge` and keep it consistent with the job definition in Flink. Best, Shammon FY On Thu, Jul 6, 2023 at 8:25 PM Jing Ge wrote: > Hi Shammon, > > Thanks

Re: [DISCUSS] FLIP-313 Add support of User Defined AsyncTableFunction

2023-07-06 Thread Aitozi
Hi guys, Since there are no further comments, Kindly ping for the vote thread [1] :D Thanks, Aitozi. [1]: https://lists.apache.org/thread/7g5n2vshosom2dj9bp7x4n01okrnx4xx Aitozi 于2023年6月26日周一 10:31写道: > Hi Lincoln, > Thanks for your confirmation. I have updated the consensus to the FLI

Re: Re: [DISCUSS] FLIP-313 Add support of User Defined AsyncTableFunction

2023-07-06 Thread Aitozi
Hi Awake, Thanks for your good point, updated Best, Aitozi. 宇航 李 于2023年7月5日周三 11:29写道: > Hi Aitozi, > > I think it is necessary to add the following description in FLIP to > express the difference between user-defined asynchronous table function and > AsyncTableFunction: > > User-defined as

Re: [DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2023-07-06 Thread Dong Lin
Hi Jing, Thank you for the suggestion. Yes, we can extend it to support null if in the future we find any use-case for this flexibility. Best, Dong On Thu, Jul 6, 2023 at 7:55 PM Jing Ge wrote: > Hi Dong, > > one scenario I could imagine is that users could enable global object > reuse feature

Re: [DISCUSS] FLIP-325: Support configuring end-to-end allowed latency

2023-07-06 Thread Dong Lin
Hi Jing, Thanks for the comments. Please see my reply inline. On Fri, Jul 7, 2023 at 5:40 AM Jing Ge wrote: > Hi, > > Thank you all for the inspired discussion. Really appreciate it! > > @Dong I'd like to ask some (stupid) questions to make sure I understand > your thoughts correctly. > > 1. It

Re: [VOTE] FLIP-321: introduce an API deprecation process

2023-07-06 Thread John Roesler
+1 (non-binding) Thanks for the FLIP! -John On Mon, Jul 3, 2023, at 22:21, Jingsong Li wrote: > +1 binding > > On Tue, Jul 4, 2023 at 10:40 AM Zhu Zhu wrote: >> >> +1 (binding) >> >> Thanks, >> Zhu >> >> ConradJam 于2023年7月3日周一 22:39写道: >> > >> > +1 (no-binding) >> > >> > Matthias Pohl 于2023年7月

Re: [DISCUSS] FLIP-325: Support configuring end-to-end allowed latency

2023-07-06 Thread Jing Ge
Hi, Thank you all for the inspired discussion. Really appreciate it! @Dong I'd like to ask some (stupid) questions to make sure I understand your thoughts correctly. 1. It will make no sense to send the same type of RecordAttributes right? e.g. if one RecordAttributes(isBacklog=true) has been s

[jira] [Created] (FLINK-32554) Facilitate slot isolation and resource management for global committer

2023-07-06 Thread Allen Wang (Jira)
Allen Wang created FLINK-32554: -- Summary: Facilitate slot isolation and resource management for global committer Key: FLINK-32554 URL: https://issues.apache.org/jira/browse/FLINK-32554 Project: Flink

[jira] [Created] (FLINK-32552) Mixed up Flink session job deployments

2023-07-06 Thread Fabio Wanner (Jira)
Fabio Wanner created FLINK-32552: Summary: Mixed up Flink session job deployments Key: FLINK-32552 URL: https://issues.apache.org/jira/browse/FLINK-32552 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-32553) ClusterEntrypointTest.testCloseAsyncShouldNotDeregisterApp failed on AZP

2023-07-06 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32553: --- Summary: ClusterEntrypointTest.testCloseAsyncShouldNotDeregisterApp failed on AZP Key: FLINK-32553 URL: https://issues.apache.org/jira/browse/FLINK-32553 Projec

Re: [DISCUSS] FLIP-325: Support configuring end-to-end allowed latency

2023-07-06 Thread Dong Lin
Hi Shammon, Thanks for your comments. Please see my reply inline. On Thu, Jul 6, 2023 at 12:47 PM Shammon FY wrote: > Hi, > > Thanks for your replay @Dong. I really agree with Piotr's points and I > would like to share some thoughts from my side. > > About the latency for mini-batch mechanism

Re: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

2023-07-06 Thread Jing Ge
Hi Shammon, Thanks for the clarification. Atlas might have his historical reason back to the hadoop era or maybe even back to the hibernate where Entity and Relation were commonly used. Flink already used Vertex and Edge to describe DAG. Some popular tools like dbt are also using this convention[1

Re: [DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2023-07-06 Thread Jing Ge
Hi Dong, one scenario I could imagine is that users could enable global object reuse features but force deep copy for some user defined specific functions because of any limitations. But that is only my gut feeling. And agree, we could keep the solution simple for now as FLIP described and upgrade

Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

2023-07-06 Thread Piotr Nowojski
Congratulations to everyone :) Piotrek czw., 6 lip 2023 o 12:23 Jane Chan napisał(a): > Congratulations! > > Best, > Jane > > On Thu, Jul 6, 2023 at 2:15 PM Jiadong Lu wrote: > > > Congratulations! > > > > Best regards, > > Jiadong Lu > > > > On 2023/7/6 13:26, Weihua Hu wrote: > > > Congratul

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

2023-07-06 Thread Dong Lin
Hi Piotr, Thanks for your comments! Please see my reply inline. On Wed, Jul 5, 2023 at 11:44 PM Piotr Nowojski wrote: > Hi Dong, > > I have a couple of questions. > > Could you explain why those properties > > @Nullable private Boolean isOutputOnEOF = null; > @Nullable private Boolean i

[VOTE] FLIP-322 Cooldown period for adaptive scheduler. Second vote.

2023-07-06 Thread Etienne Chauchot
Hi all, Thanks for your feedback about the FLIP-322: Cooldown period for adaptive scheduler [1]. This FLIP was discussed in [2]. I'd like to start a vote for it. The vote will be open for at least 72 hours (until July 9th 15:00 GMT) unless there is an objection or insufficient votes. [1] ht

Re: [DISCUSS] FLIP-322 Cooldown period for adaptive scheduler

2023-07-06 Thread Etienne Chauchot
Hi, I think we have reached a consensus here. I have updated the FLIP to reflect recent suggestions. I will start a new vote. Best Etienne Le 05/07/2023 à 14:42, Etienne Chauchot a écrit : Hi all, Thanks David for your suggestions. Comments inline. Le 04/07/2023 à 13:35, David Morávek a

Re: [DISCUSS] FLIP-329: Add operator attribute to specify support for object-reuse

2023-07-06 Thread Dong Lin
Hi Jing, Thank you for the detailed explanation. Please see my reply inline. On Thu, Jul 6, 2023 at 3:17 AM Jing Ge wrote: > Hi Xuannan, Hi Dong, > > Thanks for your clarification. > > @Xuannan > > A Jira ticket has been created for the doc update: > https://issues.apache.org/jira/browse/FLINK-

Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

2023-07-06 Thread Jane Chan
Congratulations! Best, Jane On Thu, Jul 6, 2023 at 2:15 PM Jiadong Lu wrote: > Congratulations! > > Best regards, > Jiadong Lu > > On 2023/7/6 13:26, Weihua Hu wrote: > > Congratulations! > > > > Best, > > Weihua > > > > > > On Wed, Jul 5, 2023 at 5:46 PM Shammon FY wrote: > > > >> Congratulat

[jira] [Created] (FLINK-32551) Provide the possibility to have a savepoint taken by the operator when deleting a flinkdeployment

2023-07-06 Thread Nicolas Fraison (Jira)
Nicolas Fraison created FLINK-32551: --- Summary: Provide the possibility to have a savepoint taken by the operator when deleting a flinkdeployment Key: FLINK-32551 URL: https://issues.apache.org/jira/browse/FLINK-

Re: [ANNOUNCE] Flink 1.18 Feature Freeze Extended until July 24th, 2023

2023-07-06 Thread Jing Ge
Thanks for driving it and sharing the update! Best regards, Jing On Thu, Jul 6, 2023 at 9:21 AM yuxia wrote: > Thanks for the update and thanks for your efforts. > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Rui Fan" <1996fan...@gmail.com> > 收件人: "dev" , re...@apache.org > 抄送: "Jing Ge

Re: [DISCUSS] FLIP-325: Support configuring end-to-end allowed latency

2023-07-06 Thread Dong Lin
Hi Piotr, Thanks for your comments. Please see my reply inline. On Thu, Jul 6, 2023 at 1:54 AM Piotr Nowojski wrote: > Hi, > > Thanks for this proposal, this is a very much needed thing that should be > addressed in Flink. > > I think there is one thing that hasn't been discussed neither here

[jira] [Created] (FLINK-32550) Single-parallel task consumes multi-partition data exception in pulsar

2023-07-06 Thread jiangchunyang (Jira)
jiangchunyang created FLINK-32550: - Summary: Single-parallel task consumes multi-partition data exception in pulsar Key: FLINK-32550 URL: https://issues.apache.org/jira/browse/FLINK-32550 Project: Fli

[jira] [Created] (FLINK-32549) Tiered storage memory manager supports ownership transfer for buffers

2023-07-06 Thread Yuxin Tan (Jira)
Yuxin Tan created FLINK-32549: - Summary: Tiered storage memory manager supports ownership transfer for buffers Key: FLINK-32549 URL: https://issues.apache.org/jira/browse/FLINK-32549 Project: Flink

Re: [ANNOUNCE] Flink 1.18 Feature Freeze Extended until July 24th, 2023

2023-07-06 Thread yuxia
Thanks for the update and thanks for your efforts. Best regards, Yuxia - 原始邮件 - 发件人: "Rui Fan" <1996fan...@gmail.com> 收件人: "dev" , re...@apache.org 抄送: "Jing Ge" , snuyan...@gmail.com, "Konstantin Knauf" 发送时间: 星期四, 2023年 7 月 06日 下午 3:06:28 主题: Re: [ANNOUNCE] Flink 1.18 Feature Freeze Ex

Re: [ANNOUNCE] Flink 1.18 Feature Freeze Extended until July 24th, 2023

2023-07-06 Thread Rui Fan
Thanks for the update, and thank you for your efforts for the 1.18 release! Best, Rui Fan On Thu, Jul 6, 2023 at 2:40 PM Qingsheng Ren wrote: > Hi devs, > > Recently we collected some feedback from developers, and in order to give > more time for polishing some important features in 1.18, we de