Re:[VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-04-09 Thread Feifan Wang
+1 (non-binding) -- —— Best regards, Feifan Wang At 2024-04-10 12:36:00, "Rui Fan" <1996fan...@gmail.com> wrote: >Hi devs, > >Thank you to everyone for the feedback on FLIP-441: Show >the JobType and remove Execution Mode on Flink WebUI[1] >which has been discussed

Re: [VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-04-09 Thread Zhu Zhu
+1 Thanks, Zhu gongzhongqiang 于2024年4月10日周三 13:11写道: > +1 (non binding) > > > Bests, > > Zhongqiang Gong > > Rui Fan <1996fan...@gmail.com> 于2024年4月10日周三 12:36写道: > > > Hi devs, > > > > Thank you to everyone for the feedback on FLIP-441: Show > > the JobType and remove Execution Mode on Flink

Re: [DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-09 Thread Muhammet Orazov
Hey Xuannan, Thanks for the FLIP and your efforts! Minor clarification from my side: We will relocate these ConfigOptions to a class that is included in the documentation generation. Would it make sense to define also in the FLIP the options class for these variables? For example,

[VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-04-09 Thread Rui Fan
Hi devs, Thank you to everyone for the feedback on FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI[1] which has been discussed in this thread [2]. I would like to start a vote for it. The vote will be open for at least 72 hours unless there is an objection or not enough

Re: [VOTE] FLIP-399: Flink Connector Doris

2024-04-09 Thread Jing Ge
+1(binding) Best regards, Jing On Tue, Apr 9, 2024 at 8:54 PM Feng Jin wrote: > +1 (non-binding) > > Best, > Feng > > On Tue, Apr 9, 2024 at 5:56 PM gongzhongqiang > wrote: > > > +1 (non-binding) > > > > Best, > > > > Zhongqiang Gong > > > > wudi <676366...@qq.com.invalid> 于2024年4月9日周二

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-09 Thread Jark Wu
I have been following up on the discussion, it's a great FLIP to further unify stream and batch ETL pipelines. Thanks for the proposal! Here is my ranking: 1. Materialized Table -> "The table materializes the results of a query that you specify", this can reflect what we want and doesn't

Re: [VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-04-09 Thread gongzhongqiang
+1 (non binding) Bests, Zhongqiang Gong Rui Fan <1996fan...@gmail.com> 于2024年4月10日周三 12:36写道: > Hi devs, > > Thank you to everyone for the feedback on FLIP-441: Show > the JobType and remove Execution Mode on Flink WebUI[1] > which has been discussed in this thread [2]. > > I would like to

Re: [DISCUSS] FLIP-438: Amazon SQS Sink Connector

2024-04-09 Thread Dhingra, Priya
On 4/9/24, 3:57 PM, "Dhingra, Priya" mailto:dhipr...@amazon.com.inva>LID> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi Ahmed and Samrat, Thanks a lot

Re: [DISCUSS] FLIP-438: Amazon SQS Sink Connector

2024-04-09 Thread Dhingra, Priya
 Hi Ahmed and Samrat, Thanks a lot for all the feedbacks, this is my first ever contribution to apache Flink, hence I was bit unaware about few things but updated all of them as per your suggestions, thanks again for all the support here, much appreciated!! 1)  I am not sure why we need to

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-09 Thread Becket Qin
Thanks for the proposal. I like the FLIP. My ranking: 1. Refresh(ing) / Live Table -> easy to understand and implies the dynamic characteristic 2. Derived Table -> easy to understand. 3. Materialized Table -> sounds like just a table with physical data stored somewhere. 4. Materialized View

Re: [DISCUSS] FLIP-434: Support optimizations for pre-partitioned data sources

2024-04-09 Thread Jeyhun Karimov
Hi Lincoln, I think I was misunderstood. My approach was not to use MiniBatchLocalGroupAggFunction directly but use the similar approach to it. Currently, local and global aggregate functions are used together in query plans. In my quick PoC, I verified that my modified version of

Re: [DISCUSS] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-04-09 Thread Zakelly Lan
Thanks Rui for driving this! +1 for this idea. Best, Zakelly On Mon, Apr 8, 2024 at 7:17 PM Ahmed Hamdy wrote: > Acknowledged, +1 to start a vote. > Best Regards > Ahmed Hamdy > > > On Mon, 8 Apr 2024 at 12:04, Rui Fan <1996fan...@gmail.com> wrote: > > > Sorry, it's a typo. It should be

Re: [DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-09 Thread Zakelly Lan
Thanks Xuannan for driving this! +1 for cleaning these up. And minor comments: It seems the StateBackendOptions is already annotated with @PublicEvolving. Best, Zakelly On Tue, Apr 9, 2024 at 4:21 PM Xuannan Su wrote: > Hi all, > > I'd like to start a discussion on FLIP-442: General

Re: [VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-04-09 Thread Samrat Deb
+1 (non binding) Bests, Samrat On Wed, 10 Apr 2024 at 10:26 AM, Feifan Wang wrote: > > > +1 (non-binding) > > > > > > > > -- > > > > > —— > > Best regards, > > Feifan Wang > > > > > At 2024-04-10 12:36:00, "Rui Fan" <1996fan...@gmail.com> wrote: > >Hi devs, > > > >Thank you to

[jira] [Created] (FLINK-35063) Flink sql connector pulsar/hive com.fasterxml.jackson.annotation.JsonFormat$Value conflict

2024-04-09 Thread chendaping (Jira)
chendaping created FLINK-35063: -- Summary: Flink sql connector pulsar/hive com.fasterxml.jackson.annotation.JsonFormat$Value conflict Key: FLINK-35063 URL: https://issues.apache.org/jira/browse/FLINK-35063

Re: [VOTE] FLIP-399: Flink Connector Doris

2024-04-09 Thread Muhammet Orazov
Hey Wudi, Thanks for your efforts. +1 (non-binding) Best, Muhammet On 2024-04-09 02:47, wudi wrote: Hi devs, I would like to start a vote about FLIP-399 [1]. The FLIP is about contributing the Flink Doris Connector[2] to the Flink community. Discussion thread [3]. The vote will be open

Re: [External] Inquiry Regarding Azure Pipelines

2024-04-09 Thread Yisha Zhou
Thanks Robert. Manual trigger didn’t work. I’ll push a new commit to trigger it. > 2024年4月9日 16:33,Robert Metzger 写道: > > I'm not 100% sure, but I think the manual trigger via a command will use > the latest commit (e.g. your second commit) to trigger a build. > Historically, manual

Re: [VOTE] FLIP-399: Flink Connector Doris

2024-04-09 Thread gongzhongqiang
+1 (non-binding) Best, Zhongqiang Gong wudi <676366...@qq.com.invalid> 于2024年4月9日周二 10:48写道: > Hi devs, > > I would like to start a vote about FLIP-399 [1]. The FLIP is about > contributing the Flink Doris Connector[2] to the Flink community. > Discussion thread [3]. > > The vote will be open

[jira] [Created] (FLINK-35062) Migrate RewriteMultiJoinConditionRule

2024-04-09 Thread Jacky Lau (Jira)
Jacky Lau created FLINK-35062: - Summary: Migrate RewriteMultiJoinConditionRule Key: FLINK-35062 URL: https://issues.apache.org/jira/browse/FLINK-35062 Project: Flink Issue Type: Sub-task

RE: [DISCUSS] FLIP-437: Support ML Models in Flink SQL

2024-04-09 Thread David Radley
Hi Han, Thanks for getting back to me. I am curious about the valid characters in a model name – I assume any characters are valid as it is a quoted string in SQL. So $ could be in the model name. I would think that the model would be determined then the model is deployed, ( there could be

Re: [External] Inquiry Regarding Azure Pipelines

2024-04-09 Thread Robert Metzger
I'm not 100% sure, but I think the manual trigger via a command will use the latest commit (e.g. your second commit) to trigger a build. Historically, manual triggering has not been very reliable. If the manual triggering isn't working, you can also just push a commit with a tiny change to trigger

Re: [VOTE] FLIP-399: Flink Connector Doris

2024-04-09 Thread Ahmed Hamdy
Hi Wudi, +1 (non-binding). Best Regards Ahmed Hamdy On Tue, 9 Apr 2024 at 09:21, Yuepeng Pan wrote: > Hi, Di. > > Thank you for driving it ! > > +1 (non-binding). > > Best, > Yuepeng Pan > > On 2024/04/09 02:47:55 wudi wrote: > > Hi devs, > > > > I would like to start a vote about FLIP-399

Re: [VOTE] FLIP-399: Flink Connector Doris

2024-04-09 Thread Robert Metzger
+1 (binding) On Tue, Apr 9, 2024 at 10:33 AM Ahmed Hamdy wrote: > Hi Wudi, > > +1 (non-binding). > > Best Regards > Ahmed Hamdy > > > On Tue, 9 Apr 2024 at 09:21, Yuepeng Pan wrote: > > > Hi, Di. > > > > Thank you for driving it ! > > > > +1 (non-binding). > > > > Best, > > Yuepeng Pan > > > >

Re: [VOTE] FLIP-399: Flink Connector Doris

2024-04-09 Thread Leonard Xu
+1 (binding) Best, Leonard > 2024年4月9日 下午5:11,Muhammet Orazov 写道: > > Hey Wudi, > > Thanks for your efforts. > > +1 (non-binding) > > Best, > Muhammet > > On 2024-04-09 02:47, wudi wrote: >> Hi devs, >> I would like to start a vote about FLIP-399 [1]. The FLIP is about >> contributing the

Re: Two potential bugs in Flink ML

2024-04-09 Thread Yunfeng Zhou
Hi Komal, Thanks for your example code! I found that Flink ML has a bug when it comes to keyed two input operators. I have submitted a PR to fix this bug and you can build the Flink ML library for your program according to its document after this PR is approved. The bugfix PR:

[jira] [Created] (FLINK-35061) Migrate TemporalJoinUtil

2024-04-09 Thread Jacky Lau (Jira)
Jacky Lau created FLINK-35061: - Summary: Migrate TemporalJoinUtil Key: FLINK-35061 URL: https://issues.apache.org/jira/browse/FLINK-35061 Project: Flink Issue Type: Sub-task

[DISCUSSION] FLIP-442: FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-09 Thread Xuannan Su
Hi all, I'd like to start a discussion on FLIP-442: General Improvement to Configuration for Flink 2.0 [1]. As Flink moves toward 2.0, we aim to provide users with a better experience with the existing configuration. This FLIP proposes several general improvements to the current configuration.

Re: [VOTE] FLIP-399: Flink Connector Doris

2024-04-09 Thread Yuepeng Pan
Hi, Di. Thank you for driving it ! +1 (non-binding). Best, Yuepeng Pan On 2024/04/09 02:47:55 wudi wrote: > Hi devs, > > I would like to start a vote about FLIP-399 [1]. The FLIP is about > contributing the Flink Doris Connector[2] to the Flink community. Discussion > thread [3]. > > The

[jira] [Created] (FLINK-35064) Flink sql connector pulsar/hive com.fasterxml.jackson.annotation.JsonFormat$Value conflict

2024-04-09 Thread chendaping (Jira)
chendaping created FLINK-35064: -- Summary: Flink sql connector pulsar/hive com.fasterxml.jackson.annotation.JsonFormat$Value conflict Key: FLINK-35064 URL: https://issues.apache.org/jira/browse/FLINK-35064

Re: [VOTE] FLIP-399: Flink Connector Doris

2024-04-09 Thread Jiabao Sun
+1 (binding) Best, Jiabao Leonard Xu 于2024年4月9日周二 17:35写道: > +1 (binding) > > Best, > Leonard > > > 2024年4月9日 下午5:11,Muhammet Orazov 写道: > > > > Hey Wudi, > > > > Thanks for your efforts. > > > > +1 (non-binding) > > > > Best, > > Muhammet > > > > On 2024-04-09 02:47, wudi wrote: > >> Hi

[jira] [Created] (FLINK-35067) Support metadata 'op_type' virtual column for Postgres CDC Connector.

2024-04-09 Thread Hongshun Wang (Jira)
Hongshun Wang created FLINK-35067: - Summary: Support metadata 'op_type' virtual column for Postgres CDC Connector. Key: FLINK-35067 URL: https://issues.apache.org/jira/browse/FLINK-35067 Project:

[jira] [Created] (FLINK-35065) Add numFiredTimers and numFiredTimersPerSecond metrics

2024-04-09 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-35065: -- Summary: Add numFiredTimers and numFiredTimersPerSecond metrics Key: FLINK-35065 URL: https://issues.apache.org/jira/browse/FLINK-35065 Project: Flink

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-09 Thread Timo Walther
Hi Ron, yes naming is hard. But it will have large impact on trainings, presentations, and the mental model of users. Maybe the easiest is to collect ranking by everyone with some short justification: My ranking (from good to not so good): 1. Refresh Table -> states what it does 2.

Re: Inquiry Regarding Azure Pipelines

2024-04-09 Thread Muhammet Orazov
Hey Robert, Thanks for fixing the flinkbot issue. Thanks for your efforts (including Mathias and Lorenzo)! Best, Muhammet p.s. Mandatory XKCD: https://xkcd.com/2347/ On 2024-04-03 16:52, Robert Metzger wrote: Hi Yisha, flinkbot is currently not active, so new PRs are not triggering any

Re: Iceberg table maintenance

2024-04-09 Thread Ahmed Hamdy
Thanks Peter for forwarding this. Best Regards Ahmed Hamdy On Tue, 9 Apr 2024 at 05:51, Péter Váry wrote: > Forwarding the invite for the discussion we plan to do with the Iceberg > folks, as some of you might be interested in this. > > -- Forwarded message - > From: Brian

Re: [VOTE] FLIP-399: Flink Connector Doris

2024-04-09 Thread Ferenc Csaky
+1 (non-binding) Best, Ferenc On Tuesday, April 9th, 2024 at 10:32, Ahmed Hamdy wrote: > > > Hi Wudi, > > +1 (non-binding). > > Best Regards > Ahmed Hamdy > > > On Tue, 9 Apr 2024 at 09:21, Yuepeng Pan panyuep...@apache.org wrote: > > > Hi, Di. > > > > Thank you for driving it ! > >

[jira] [Created] (FLINK-35066) TwoInputOperator in IterationBody cannot use keyBy

2024-04-09 Thread Yunfeng Zhou (Jira)
Yunfeng Zhou created FLINK-35066: Summary: TwoInputOperator in IterationBody cannot use keyBy Key: FLINK-35066 URL: https://issues.apache.org/jira/browse/FLINK-35066 Project: Flink Issue

[jira] [Created] (FLINK-35060) Provide compatibility of old CheckpointMode for connector testing framework

2024-04-09 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-35060: --- Summary: Provide compatibility of old CheckpointMode for connector testing framework Key: FLINK-35060 URL: https://issues.apache.org/jira/browse/FLINK-35060 Project:

[jira] [Created] (FLINK-35059) Bump org.postgresql:postgresql from 42.5.1 to 42.5.5 in flink-connector-jdbc

2024-04-09 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-35059: --- Summary: Bump org.postgresql:postgresql from 42.5.1 to 42.5.5 in flink-connector-jdbc Key: FLINK-35059 URL: https://issues.apache.org/jira/browse/FLINK-35059

[jira] [Created] (FLINK-35068) Introduce built-in serialization support for Set

2024-04-09 Thread Zhanghao Chen (Jira)
Zhanghao Chen created FLINK-35068: - Summary: Introduce built-in serialization support for Set Key: FLINK-35068 URL: https://issues.apache.org/jira/browse/FLINK-35068 Project: Flink Issue

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-09 Thread Ron liu
Hi, Dev My rankings are: 1. Derived Table 2. Materialized Table 3. Live Table 4. Materialized View Best, Ron Ron liu 于2024年4月9日周二 20:07写道: > Hi, Dev > > After several rounds of discussion, there is currently no consensus on the > name of the new concept. Timo has proposed that we decide

[jira] [Created] (FLINK-35070) remove unused duplicate code

2024-04-09 Thread niliushall (Jira)
niliushall created FLINK-35070: -- Summary: remove unused duplicate code Key: FLINK-35070 URL: https://issues.apache.org/jira/browse/FLINK-35070 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-09 Thread Lincoln Lee
Thanks Ron and Timo for your proposal! Here is my ranking: 1. Derived table -> extend the persistent semantics of derived table in SQL standard, with a strong association with query, and has industry precedents such as Google Looker. 2. Live Table -> an alternative for 'dynamic table'

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-09 Thread Ron liu
Hi, Timo Thanks for your reply. I agree with you that sometimes naming is more difficult. When no one has a clear preference, voting on the name is a good solution, so I'll send a separate email for the vote, clarify the rules for the vote, then let everyone vote. One other point to confirm, in

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-09 Thread Ron liu
Hi, Dev Sorry for my previous statement was not quite accurate. We will hold a vote for the name within this thread. Best, Ron Ron liu 于2024年4月9日周二 19:29写道: > Hi, Timo > > Thanks for your reply. > > I agree with you that sometimes naming is more difficult. When no one has > a clear

Re: [VOTE] FLIP-399: Flink Connector Doris

2024-04-09 Thread Feng Jin
+1 (non-binding) Best, Feng On Tue, Apr 9, 2024 at 5:56 PM gongzhongqiang wrote: > +1 (non-binding) > > Best, > > Zhongqiang Gong > > wudi <676366...@qq.com.invalid> 于2024年4月9日周二 10:48写道: > > > Hi devs, > > > > I would like to start a vote about FLIP-399 [1]. The FLIP is about > > contributing

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-09 Thread Ron liu
Hi, Dev After several rounds of discussion, there is currently no consensus on the name of the new concept. Timo has proposed that we decide the name through a vote. This is a good solution when there is no clear preference, so we will adopt this approach. Regarding the name of the new concept,

[jira] [Created] (FLINK-35069) ContinuousProcessingTimeTrigger continuously registers timers in a loop at the end of the window

2024-04-09 Thread Jinzhong Li (Jira)
Jinzhong Li created FLINK-35069: --- Summary: ContinuousProcessingTimeTrigger continuously registers timers in a loop at the end of the window Key: FLINK-35069 URL: https://issues.apache.org/jira/browse/FLINK-35069

Re: [DISCUSS] FLIP-434: Support optimizations for pre-partitioned data sources

2024-04-09 Thread Lincoln Lee
Thanks Jeyhun for your reply! Unfortunately, MiniBatchLocalGroupAggFunction only works for local agg in two-phase aggregation, while global aggregation (which is actually handled by the KeyedMapBundleOperator) still relies on the KeyedStream, meaning that consistency of the partitioner and state