Re: [DISCUSS] Planning Flink 1.20

2024-03-21 Thread Jark Wu
Thanks for kicking this off. +1 for the volunteered release managers (Weijie Guo, Rui Fan) and the targeting date (feature freeze: June 15). Best, Jark On Fri, 22 Mar 2024 at 14:00, Rui Fan <1996fan...@gmail.com> wrote: > Thanks Leonard for this feedback and help! > > Best, > Rui > > On Fri

Re: [DISCUSS] Planning Flink 1.20

2024-03-21 Thread Rui Fan
Thanks Leonard for this feedback and help! Best, Rui On Fri, Mar 22, 2024 at 12:36 PM weijie guo wrote: > Thanks Leonard! > > > I'd like to help you if you need some help like permissions from PMC > side, please feel free to ping me. > > Nice to know. It'll help a lot! > > Best regards, > > Wei

Re: [DISCUSS] Planning Flink 1.20

2024-03-21 Thread weijie guo
Thanks Leonard! > I'd like to help you if you need some help like permissions from PMC side, please feel free to ping me. Nice to know. It'll help a lot! Best regards, Weijie Leonard Xu 于2024年3月22日周五 12:09写道: > +1 for the proposed release managers (Weijie Guo, Rui Fan), both the two > candi

Re: [DISCUSS] Planning Flink 1.20

2024-03-21 Thread Leonard Xu
+1 for the proposed release managers (Weijie Guo, Rui Fan), both the two candidates are pretty active committers thus I believe they know the community development process well. The recent releases have four release managers, and I am also looking forward to having other volunteers join the man

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Hangxiang Yu
Congratulations! Thanks for the efforts. On Fri, Mar 22, 2024 at 10:00 AM Yanfei Lei wrote: > Congratulations! > > Best regards, > Yanfei > > Xuannan Su 于2024年3月22日周五 09:21写道: > > > > Congratulations! > > > > Best regards, > > Xuannan > > > > On Fri, Mar 22, 2024 at 9:17 AM Charles Zhang > wro

Re: [DISCUSS] FLIP-428: Fault Tolerance/Rescale Integration for Disaggregated State

2024-03-21 Thread Jinzhong Li
Hi Jeyhun, Thanks for your thoughtful feedback! > Why dont we consider an option where checkpoint directory just contains > metadata. So, we do not need to copy the data all the time from working > directory to the checkpointing directory. > Basically, when checkpointing, 1) we mark files in work

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

2024-03-21 Thread Yuepeng Pan
+1 (non-binding) Regards, Yuepeng Pan 在 2024-03-22 04:11:32,"Jeyhun Karimov" 写道: >+1 (non-binding) > >Regards, >Jeyhun > >On Thu, Mar 21, 2024 at 2:04 PM Márton Balassi >wrote: > >> +1(binding) >> >> On Thu, Mar 21, 2024 at 1:24 PM Leonard Xu wrote: >> >> > +1(binding) >> > >>

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Yanfei Lei
Congratulations! Best regards, Yanfei Xuannan Su 于2024年3月22日周五 09:21写道: > > Congratulations! > > Best regards, > Xuannan > > On Fri, Mar 22, 2024 at 9:17 AM Charles Zhang wrote: > > > > Congratulations! > > > > Best wishes, > > Charles Zhang > > from Apache InLong > > > > > > Jeyhun Karimov 于2

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Xuannan Su
Congratulations! Best regards, Xuannan On Fri, Mar 22, 2024 at 9:17 AM Charles Zhang wrote: > > Congratulations! > > Best wishes, > Charles Zhang > from Apache InLong > > > Jeyhun Karimov 于2024年3月22日周五 04:16写道: > > > Great news! Congratulations! > > > > Regards, > > Jeyhun > > > > On Thu, Mar 2

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Charles Zhang
Congratulations! Best wishes, Charles Zhang from Apache InLong Jeyhun Karimov 于2024年3月22日周五 04:16写道: > Great news! Congratulations! > > Regards, > Jeyhun > > On Thu, Mar 21, 2024 at 2:00 PM Yuxin Tan wrote: > > > Congratulations! Thanks for the efforts. > > > > > > Best, > > Yuxin > > > > > >

Re: [DISCUSS] FLIP-428: Fault Tolerance/Rescale Integration for Disaggregated State

2024-03-21 Thread Jeyhun Karimov
Hi Jinzhong, Thanks for the FLIP. +1 for it. I have a few questions: - Why dont we consider an option where checkpoint directory just contains metadata. So, we do not need to copy the data all the time from working directory to the checkpointing directory. Basically, when checkpointing, 1) we ma

Re: [DISCUSS] FLIP-XXX Apicurio-avro format

2024-03-21 Thread Jeyhun Karimov
Hi David, Thanks for the FLIP. +1 for it. I have a minor comment. Can you please elaborate more on mechanisms in place to ensure data consistency and integrity, particularly in the event of schema conflicts? Since each message includes a schema ID for inbound and outbound messages, can you elabor

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Jeyhun Karimov
Great news! Congratulations! Regards, Jeyhun On Thu, Mar 21, 2024 at 2:00 PM Yuxin Tan wrote: > Congratulations! Thanks for the efforts. > > > Best, > Yuxin > > > Samrat Deb 于2024年3月21日周四 20:28写道: > > > Congratulations ! > > > > Bests > > Samrat > > > > On Thu, 21 Mar 2024 at 5:52 PM, Ahmed Ha

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

2024-03-21 Thread Jeyhun Karimov
+1 (non-binding) Regards, Jeyhun On Thu, Mar 21, 2024 at 2:04 PM Márton Balassi wrote: > +1(binding) > > On Thu, Mar 21, 2024 at 1:24 PM Leonard Xu wrote: > > > +1(binding) > > > > Best, > > Leonard > > > > > 2024年3月21日 下午5:21,Martijn Visser 写道: > > > > > > +1 (binding) > > > > > > On Thu, Ma

Re: Flink Kubernetes Operator Failing Over FlinkDeployments to a New Cluster

2024-03-21 Thread Kevin Lam
No worries, thanks for the reply Gyula. Ah yes, I see how those points you raised make the feature tricky to implement. Could this be considered for a FLIP (or two) in the future? On Wed, Mar 20, 2024 at 2:21 PM Gyula Fóra wrote: > Sorry for the late reply Kevin. > > I think what you are sugges

Re: [DISCUSS] FLIP-434: Support optimizations for pre-partitioned data sources

2024-03-21 Thread Jeyhun Karimov
Hi Lorenzo, Thanks a lot for your comments. Please find my answers below: For the interface `SupportsPartitioning`, why returning `Optional`? > If one decides to implement that, partitions must exist (at maximum, > return and empty list). Returning `Optional` seem just to complicate the > logic

[jira] [Created] (FLINK-34911) ChangelogRecoveryRescaleITCase failed fatally with 127 exit code

2024-03-21 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-34911: --- Summary: ChangelogRecoveryRescaleITCase failed fatally with 127 exit code Key: FLINK-34911 URL: https://issues.apache.org/jira/browse/FLINK-34911 Project: Flink

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.8.0, release candidate #1

2024-03-21 Thread Maximilian Michels
The vote is now closed. I'm happy to announce that we have unanimously approved this release. There are 6 approving votes, 3 of which are binding: * Gyula Fora (binding) * Marton Balassi (binding) * Maximilian Michels (binding) * Rui Fan (non-binding) * Alexander Fedulov (non-binding) * Mate Cza

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.8.0, release candidate #1

2024-03-21 Thread Maximilian Michels
+1 (binding) 1. Verified the archives, checksums, and signatures 2. Extracted and inspected the source code for binaries 3. Compiled and tested the source code via mvn verify 4. Verified license files / headers 5. Deployed helm chart to test cluster 6. Ran example job 7. Tested autoscaling without

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-21 Thread Yanfei Lei
Thanks for your reading and valuable comments! > 1) About locking VS reference counting: I would like to clear out which > mechanism prevents what: The `KeyAccountingUnit` implements locking behavior on keys and ensures 2 state requests on the same key happen in order. Double-locking the same key

Re: [DISCUSS] FLIP-434: Support optimizations for pre-partitioned data sources

2024-03-21 Thread Benchao Li
Jeyhun, Sorry for the delay. And thanks for the explanation, it sounds good to me! Jeyhun Karimov 于2024年3月16日周六 05:09写道: > > Hi Benchao, > > Thanks for your comments. > > 1. What the parallelism would you take? E.g., 128 + 256 => 128? What > > if we cannot have a good greatest common divisor, li

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

2024-03-21 Thread Márton Balassi
+1(binding) On Thu, Mar 21, 2024 at 1:24 PM Leonard Xu wrote: > +1(binding) > > Best, > Leonard > > > 2024年3月21日 下午5:21,Martijn Visser 写道: > > > > +1 (binding) > > > > On Thu, Mar 21, 2024 at 8:01 AM gongzhongqiang < > gongzhongqi...@apache.org> > > wrote: > > > >> +1 (non-binding) > >> > >> Be

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Yuxin Tan
Congratulations! Thanks for the efforts. Best, Yuxin Samrat Deb 于2024年3月21日周四 20:28写道: > Congratulations ! > > Bests > Samrat > > On Thu, 21 Mar 2024 at 5:52 PM, Ahmed Hamdy wrote: > > > Congratulations, great work and great news. > > Best Regards > > Ahmed Hamdy > > > > > > On Thu, 21 Mar 2

[jira] [Created] (FLINK-34910) Can not plan window join without projections

2024-03-21 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-34910: Summary: Can not plan window join without projections Key: FLINK-34910 URL: https://issues.apache.org/jira/browse/FLINK-34910 Project: Flink Issue Ty

[jira] [Created] (FLINK-34909) OceanBase事务ID需求

2024-03-21 Thread xiaotouming (Jira)
xiaotouming created FLINK-34909: --- Summary: OceanBase事务ID需求 Key: FLINK-34909 URL: https://issues.apache.org/jira/browse/FLINK-34909 Project: Flink Issue Type: New Feature Components: F

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Samrat Deb
Congratulations ! Bests Samrat On Thu, 21 Mar 2024 at 5:52 PM, Ahmed Hamdy wrote: > Congratulations, great work and great news. > Best Regards > Ahmed Hamdy > > > On Thu, 21 Mar 2024 at 11:41, Benchao Li wrote: > > > Congratulations, and thanks for the great work! > > > > Yuan Mei 于2024年3月21日

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

2024-03-21 Thread Leonard Xu
+1(binding) Best, Leonard > 2024年3月21日 下午5:21,Martijn Visser 写道: > > +1 (binding) > > On Thu, Mar 21, 2024 at 8:01 AM gongzhongqiang > wrote: > >> +1 (non-binding) >> >> Bests, >> Zhongqiang Gong >> >> Ferenc Csaky 于2024年3月20日周三 22:11写道: >> >>> Hello devs, >>> >>> I would like to start

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Ahmed Hamdy
Congratulations, great work and great news. Best Regards Ahmed Hamdy On Thu, 21 Mar 2024 at 11:41, Benchao Li wrote: > Congratulations, and thanks for the great work! > > Yuan Mei 于2024年3月21日周四 18:31写道: > > > > Thanks for driving these efforts! > > > > Congratulations > > > > Best > > Yuan > >

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Benchao Li
Congratulations, and thanks for the great work! Yuan Mei 于2024年3月21日周四 18:31写道: > > Thanks for driving these efforts! > > Congratulations > > Best > Yuan > > On Thu, Mar 21, 2024 at 4:35 PM Yu Li wrote: > > > Congratulations and look forward to its further development! > > > > Best Regards, > >

Re: [DISCUSS] FLIP-425: Asynchronous Execution Model

2024-03-21 Thread lorenzo . affetti
Thank you everybody for the questions and answers (especially Yanfei Lei), it was very instructive to go over the discussion. I am gonna add some questions on top of what happened and add some thoughts as well below. 1) About locking VS reference counting: I would like to clear out which mechani

[jira] [Created] (FLINK-34908) mysql pipeline to doris and starrocks will lost precision for timestamp

2024-03-21 Thread Xin Gong (Jira)
Xin Gong created FLINK-34908: Summary: mysql pipeline to doris and starrocks will lost precision for timestamp Key: FLINK-34908 URL: https://issues.apache.org/jira/browse/FLINK-34908 Project: Flink

Re: [VOTE] FLIP-433: State Access on DataStream API V2

2024-03-21 Thread Jinzhong Li
+1 (non-binding) Best, Jinzhong On Thu, Mar 21, 2024 at 6:15 PM Zakelly Lan wrote: > +1 non-binding > > > Best, > Zakelly > > On Thu, Mar 21, 2024 at 5:34 PM Gyula Fóra wrote: > > > +1 (binding) > > > > Gyula > > > > On Thu, Mar 21, 2024 at 3:33 AM Rui Fan <1996fan...@gmail.com> wrote: > > > >

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Yuan Mei
Thanks for driving these efforts! Congratulations Best Yuan On Thu, Mar 21, 2024 at 4:35 PM Yu Li wrote: > Congratulations and look forward to its further development! > > Best Regards, > Yu > > On Thu, 21 Mar 2024 at 15:54, ConradJam wrote: > > > > Congrattulations! > > > > Leonard Xu 于2024

Re: [VOTE] FLIP-433: State Access on DataStream API V2

2024-03-21 Thread Zakelly Lan
+1 non-binding Best, Zakelly On Thu, Mar 21, 2024 at 5:34 PM Gyula Fóra wrote: > +1 (binding) > > Gyula > > On Thu, Mar 21, 2024 at 3:33 AM Rui Fan <1996fan...@gmail.com> wrote: > > > +1(binding) > > > > Thanks to Weijie for driving this proposal, which solves the problem > that I > > raised i

[jira] [Created] (FLINK-34907) jobRunningTs should be the timestamp that all tasks are running

2024-03-21 Thread Rui Fan (Jira)
Rui Fan created FLINK-34907: --- Summary: jobRunningTs should be the timestamp that all tasks are running Key: FLINK-34907 URL: https://issues.apache.org/jira/browse/FLINK-34907 Project: Flink Issue

Re: [VOTE] FLIP-402: Extend ZooKeeper Curator configurations

2024-03-21 Thread gongzhongqiang
+1 (non-binding) Best, Zhongqiang Gong Alex Nitavsky 于2024年3月7日周四 23:09写道: > Hi everyone, > > I'd like to start a vote on FLIP-402 [1]. It introduces new configuration > options for Apache Flink's ZooKeeper integration for high availability by > reflecting existing Apache Curator configuration

[jira] [Created] (FLINK-34906) Don't start autoscaling when some tasks are not running

2024-03-21 Thread Rui Fan (Jira)
Rui Fan created FLINK-34906: --- Summary: Don't start autoscaling when some tasks are not running Key: FLINK-34906 URL: https://issues.apache.org/jira/browse/FLINK-34906 Project: Flink Issue Type: Imp

Re: [VOTE] FLIP-433: State Access on DataStream API V2

2024-03-21 Thread Gyula Fóra
+1 (binding) Gyula On Thu, Mar 21, 2024 at 3:33 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Thanks to Weijie for driving this proposal, which solves the problem that I > raised in FLIP-359. > > Best, > Rui > > On Thu, Mar 21, 2024 at 10:10 AM Hangxiang Yu wrote: > > > +1 (binding

Re: [VOTE] FLIP-402: Extend ZooKeeper Curator configurations

2024-03-21 Thread Martijn Visser
+1 (binding) On Wed, Mar 20, 2024 at 1:19 PM Ferenc Csaky wrote: > +1 (non-binding), thanks for driving this! > > Best, > Ferenc > > > On Wednesday, March 20th, 2024 at 10:57, Yang Wang < > wangyang0...@apache.org> wrote: > > > > > > > +1 (binding) since ZK HA is still widely used. > > > > > > B

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

2024-03-21 Thread Martijn Visser
+1 (binding) On Thu, Mar 21, 2024 at 8:01 AM gongzhongqiang wrote: > +1 (non-binding) > > Bests, > Zhongqiang Gong > > Ferenc Csaky 于2024年3月20日周三 22:11写道: > > > Hello devs, > > > > I would like to start a vote about FLIP-439 [1]. The FLIP is about to > > externalize the Kudu > > connector from

Re: [DISCUSS] FLIP-434: Support optimizations for pre-partitioned data sources

2024-03-21 Thread lorenzo . affetti
Hello Jeyhun, I really like the proposal and definitely makes sense to me. I have a couple of nits here and there: For the interface `SupportsPartitioning`, why returning `Optional`? If one decides to implement that, partitions must exist (at maximum, return and empty list). Returning `Optional`

[jira] [Created] (FLINK-34905) The default length of CHAR/BINARY data type of Add column DDL

2024-03-21 Thread Qishang Zhong (Jira)
Qishang Zhong created FLINK-34905: - Summary: The default length of CHAR/BINARY data type of Add column DDL Key: FLINK-34905 URL: https://issues.apache.org/jira/browse/FLINK-34905 Project: Flink

[jira] [Created] (FLINK-34904) [Feature] submit Flink CDC pipeline job to yarn application cluster.

2024-03-21 Thread ZhengYu Chen (Jira)
ZhengYu Chen created FLINK-34904: Summary: [Feature] submit Flink CDC pipeline job to yarn application cluster. Key: FLINK-34904 URL: https://issues.apache.org/jira/browse/FLINK-34904 Project: Flink

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Yu Li
Congratulations and look forward to its further development! Best Regards, Yu On Thu, 21 Mar 2024 at 15:54, ConradJam wrote: > > Congrattulations! > > Leonard Xu 于2024年3月20日周三 21:36写道: > > > Hi devs and users, > > > > We are thrilled to announce that the donation of Flink CDC as a > > sub-proje

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

2024-03-21 Thread gongzhongqiang
+1 (non-binding) Bests, Zhongqiang Gong Ferenc Csaky 于2024年3月20日周三 22:11写道: > Hello devs, > > I would like to start a vote about FLIP-439 [1]. The FLIP is about to > externalize the Kudu > connector from the recently retired Apache Bahir project [2] to keep it > maintainable and > make it up to

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread ConradJam
Congrattulations! Leonard Xu 于2024年3月20日周三 21:36写道: > Hi devs and users, > > We are thrilled to announce that the donation of Flink CDC as a > sub-project of Apache Flink has completed. We invite you to explore the new > resources available: > > - GitHub Repository: https://github.com/apache/fli

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.8.0, release candidate #1

2024-03-21 Thread Márton Balassi
+1 (binding) As per Gyula's suggestion above verified with " ghcr.io/apache/flink-kubernetes-operator:91d67d9 ". - Verified Helm repo works as expected, points to correct image tag, build, version - Verified basic examples + checked operator logs everything looks as expected - Verified hashes, si

[jira] [Created] (FLINK-34903) Add mysql-pipeline-connector with table.exclude.list option to exclude unnecessary tables

2024-03-21 Thread shiyuyang (Jira)
shiyuyang created FLINK-34903: - Summary: Add mysql-pipeline-connector with table.exclude.list option to exclude unnecessary tables Key: FLINK-34903 URL: https://issues.apache.org/jira/browse/FLINK-34903

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

2024-03-21 Thread Samrat Deb
+1 (non-binding ) Bests, Samrat On Thu, Mar 21, 2024 at 12:55 PM Hang Ruan wrote: > +1 (non-binding) > > Best, > Hang > > Őrhidi Mátyás 于2024年3月21日周四 00:00写道: > > > +1 (binding) > > > > On Wed, Mar 20, 2024 at 8:37 AM Gabor Somogyi > > > wrote: > > > > > +1 (binding) > > > > > > G > > > > > >

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread gongzhongqiang
Congrattulations! Thanks for the great work! Best, Zhongqiang Gong Leonard Xu 于2024年3月20日周三 21:36写道: > Hi devs and users, > > We are thrilled to announce that the donation of Flink CDC as a > sub-project of Apache Flink has completed. We invite you to explore the new > resources available: > >

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

2024-03-21 Thread Hang Ruan
+1 (non-binding) Best, Hang Őrhidi Mátyás 于2024年3月21日周四 00:00写道: > +1 (binding) > > On Wed, Mar 20, 2024 at 8:37 AM Gabor Somogyi > wrote: > > > +1 (binding) > > > > G > > > > > > On Wed, Mar 20, 2024 at 3:59 PM Gyula Fóra wrote: > > > > > +1 (binding) > > > > > > Thanks! > > > Gyula > > > >

[jira] [Created] (FLINK-34902) INSERT INTO column mismatch leads to IndexOutOfBoundsException

2024-03-21 Thread Timo Walther (Jira)
Timo Walther created FLINK-34902: Summary: INSERT INTO column mismatch leads to IndexOutOfBoundsException Key: FLINK-34902 URL: https://issues.apache.org/jira/browse/FLINK-34902 Project: Flink