Re: [DISCUSS] FLIP-315: Support Operator Fusion Codegen for Flink SQL

Hi, Lincoln Thanks for your appreciation of this design. Regarding your question: > do we consider adding a benchmark for the operators to intuitively understand the improvement brought by each improvement? I think it makes sense to add a benchmark, Spark also has this benchmark framework. But I

[jira] [Created] (FLINK-32252) SELECT COUNT(*) will return nothing when source no data return

tanjialiang created FLINK-32252: --- Summary: SELECT COUNT(*) will return nothing when source no data return Key: FLINK-32252 URL: https://issues.apache.org/jira/browse/FLINK-32252 Project: Flink

Re: [DISCUSS] FLIP-308: Support Time Travel In Batch Mode

Hi Feng, I think this FLIP would provide one important feature to unify the stream-SQL and batch-SQL when we backfill the historical data in batch mode. For the "Syntax" session, I think you could add descriptions of how to align backfill time travel with querying the latest data. And I think y

Re: [DISCUSS] FLIP-315: Support Operator Fusion Codegen for Flink SQL

Hi, Yun, Jinsong, Benchao Thanks for your valuable input about this FLIP. First of all, let me emphasize that from the technical implementation point of view, this design is feasible in both stream and batch scenarios, so I consider both stream and batch mode in FLIP. In the stream scenario, for

Re: [DISCUSS] FLIP-313 Add support of User Defined AsyncTableFunction

One more thing for discussion: In our internal implementation, we reuse the option `table.exec.async-lookup.buffer-capacity` and `table.exec.async-lookup.timeout` to config the async udtf. Do you think we should add two extra option to distinguish from the lookup option such as `table.exec.async-

Re: [DISCUSS] FLIP-316: Introduce SQL Driver

Hi Jark, Thanks for your input! Please see my comments inline. > Isn't Table API the same way as DataSream jobs to submit Flink SQL? > DataStream API also doesn't provide a default main class for users, > why do we need to provide such one for SQL? Sorry for the confusion I caused. By DataStream

[DISCUSS] Make Preconditions/VisibleForTesting public

Hi all, As part of fixing the architecture-tests-production Connector rule in this PR [1], it appeared that (external) connectors depending on the non-public Preconditions/VisibleForTesting classes was a violation to the "connectors should not depend on non-public classes that are not in the

[jira] [Created] (FLINK-32253) Blocklist unblockResources does not update for pending containers

Prabhu Joseph created FLINK-32253: - Summary: Blocklist unblockResources does not update for pending containers Key: FLINK-32253 URL: https://issues.apache.org/jira/browse/FLINK-32253 Project: Flink

Re: [DISCUSS] FLIP-311: Support Call Stored Procedure

Hi Yuxia, Thanks for the FLIP. I have a couple of questions: 1. The syntax talks about how to CALL or SHOW the available stored procedures, but not on how to create one. Will there not be a SQL syntax to create/save a stored procedure? 2. Is there a default syntax in Calcite for stored procedures

Re: [DISCUSS] FLIP-317: Upgrade Kryo from 2.24.0 to 5.5.0

Here I have a suggestion, because I mentioned Flink2.0 earlier, I am wondering if there is a possibility: whether the user can perform the migration of all states to Kryo5 when performing the first start-up task of migrating to version 2.0 in the future, until we give up maintaining Kryo2 later Do

Re: [DISCUSS] FLIP-307: Flink connector Redshift

Hi Samrat, Thanks for the FLIP. If I understand this correctly, the Redshift sink would not be able to support exactly-once, is that correct? Best regards, Martijn On Sat, Jun 3, 2023 at 9:18 PM Samrat Deb wrote: > Hi Jing Ge, > > >>> Do you already have any prototype? I'd like to join the re

Re: [DISCUSS] Hive dialect shouldn't fall back to Flink's default dialect

+1 for anything that helps us with externalizing the Hive connector :D On Thu, Jun 1, 2023 at 3:34 PM Lincoln Lee wrote: > +1, thanks yuxia for driving the hive decoupling work! > Since the 1.16 release, the compatibility of Hive queries has reached a > relatively high level, so it is time to ab

Re: [DISCUSS] FLIP-317: Upgrade Kryo from 2.24.0 to 5.5.0

Hi ConradJam, That assumes that it will be possible to upgrade statefully to Flink 2.0: given that it is a major breaking change, I wouldn't assume that will be possible. Best regards, Martijn On Mon, Jun 5, 2023 at 2:37 PM ConradJam wrote: > Here I have a suggestion, because I mentioned Flin

[jira] [Created] (FLINK-32254) FineGrainedSlotManager may not allocate enough taskmangers if maxSlotNum is configured

Weijie Guo created FLINK-32254: -- Summary: FineGrainedSlotManager may not allocate enough taskmangers if maxSlotNum is configured Key: FLINK-32254 URL: https://issues.apache.org/jira/browse/FLINK-32254 Pr

Re: [DISCUSS] FLIP-307: Flink connector Redshift

Hi Samrat, Thanks for the feedback. I would suggest adding that information into the FLIP. +1 Looking forward to your PR :-) Best regards, Jing On Sat, Jun 3, 2023 at 9:19 PM Samrat Deb wrote: > Hi Jing Ge, > > >>> Do you already have any prototype? I'd like to join the reviews. > The prototy

Call for Presentations, Community Over Code Asia 2023

You are receiving this message because you are subscribed to one more more developer mailing lists at the Apache Software Foundation. The call for presentations is now open at "https://apachecon.com/acasia2023/cfp.html";, and will be closed by Sunday, Jun 18th, 2023 11:59 PM GMT. The event will b

[VOTE] FLIP-312: Add Yarn ACLs to Flink Containers

Hi everyone, Thanks for all the feedback for FLIP-312: Add Yarn ACLs to Flink Containers. Following is the discussion thread : Link

[jira] [Created] (FLINK-32255) Pre-deployed idle Task Managers

Christophe Bornet created FLINK-32255: - Summary: Pre-deployed idle Task Managers Key: FLINK-32255 URL: https://issues.apache.org/jira/browse/FLINK-32255 Project: Flink Issue Type: New Fea

[jira] [Created] (FLINK-32256) Add ARRAY_MAX support in SQL & Table API

Bonnie Varghese created FLINK-32256: --- Summary: Add ARRAY_MAX support in SQL & Table API Key: FLINK-32256 URL: https://issues.apache.org/jira/browse/FLINK-32256 Project: Flink Issue Type: Bu

[jira] [Created] (FLINK-32257) Add ARRAY_MIN support in SQL & Table API

Bonnie Varghese created FLINK-32257: --- Summary: Add ARRAY_MIN support in SQL & Table API Key: FLINK-32257 URL: https://issues.apache.org/jira/browse/FLINK-32257 Project: Flink Issue Type: Im

[jira] [Created] (FLINK-32258) Add ARRAY_SORT support in SQL & Table API

Bonnie Varghese created FLINK-32258: --- Summary: Add ARRAY_SORT support in SQL & Table API Key: FLINK-32258 URL: https://issues.apache.org/jira/browse/FLINK-32258 Project: Flink Issue Type: I

[jira] [Created] (FLINK-32259) Add ARRAY_JOIN support in SQL & Table API

Bonnie Varghese created FLINK-32259: --- Summary: Add ARRAY_JOIN support in SQL & Table API Key: FLINK-32259 URL: https://issues.apache.org/jira/browse/FLINK-32259 Project: Flink Issue Type: I

[jira] [Created] (FLINK-32260) Add SLICE support in SQL & Table API

Bonnie Varghese created FLINK-32260: --- Summary: Add SLICE support in SQL & Table API Key: FLINK-32260 URL: https://issues.apache.org/jira/browse/FLINK-32260 Project: Flink Issue Type: Improv

[jira] [Created] (FLINK-32261) Add MAP_UNION support in SQL & Table API

Bonnie Varghese created FLINK-32261: --- Summary: Add MAP_UNION support in SQL & Table API Key: FLINK-32261 URL: https://issues.apache.org/jira/browse/FLINK-32261 Project: Flink Issue Type: Im

[jira] [Created] (FLINK-32262) Add MAP_ENTRIES support in SQL & Table API

Bonnie Varghese created FLINK-32262: --- Summary: Add MAP_ENTRIES support in SQL & Table API Key: FLINK-32262 URL: https://issues.apache.org/jira/browse/FLINK-32262 Project: Flink Issue Type:

[jira] [Created] (FLINK-32263) Add ELT support in SQL & Table API

Bonnie Varghese created FLINK-32263: --- Summary: Add ELT support in SQL & Table API Key: FLINK-32263 URL: https://issues.apache.org/jira/browse/FLINK-32263 Project: Flink Issue Type: Improvem

[jira] [Created] (FLINK-32264) Add FIELD support in SQL & Table API

Bonnie Varghese created FLINK-32264: --- Summary: Add FIELD support in SQL & Table API Key: FLINK-32264 URL: https://issues.apache.org/jira/browse/FLINK-32264 Project: Flink Issue Type: Improv

Re: [VOTE] FLIP-312: Add Yarn ACLs to Flink Containers

Thanks for starting the vote on this one, Archit. +1 (non-binding) Regards Venkata krishnan On Mon, Jun 5, 2023 at 9:55 AM Archit Goyal wrote: > Hi everyone, > > Thanks for all the feedback for FLIP-312: Add Yarn ACLs to Flink > Containers< > https://urldefense.com/v3/__https://cwiki.apache.o

Re: [DISCUSS] FLIP-246: Multi Cluster Kafka Source

Hi all, I'm working on FLIP-246 again, for the Multi Cluster Kafka Source contribution. The document has been updated with some more context about how it can solve the Kafka topic removal scenario and a sequence diagram to illustrate how the components interact. Looking forward to any feedback!

Re: [DISCUSS] FLIP-316: Introduce SQL Driver

Hi Paul, +1 for this feature and supporting SQL file + JSON plans. We get a lot of requests to just be able to submit a SQL file, but the JSON plan optimizations make sense. +1 for init containers or a more generalized way of obtaining arbitrary files. File fetching isn't specific to just SQL--it

[jira] [Created] (FLINK-32265) Use default classloader in jobmanager when there are no user jars for job

Fang Yong created FLINK-32265: - Summary: Use default classloader in jobmanager when there are no user jars for job Key: FLINK-32265 URL: https://issues.apache.org/jira/browse/FLINK-32265 Project: Flink

Re: [DISCUSS] FLIP-311: Support Call Stored Procedure

Hi, Martijn. Thanks for you feedback. 1: In this FLIP we don't intend to allow users to customize their own stored procedure for we don't want to expose too much to users too early as the FLIP said. The procedures are supposed to be provided only by Catalog. Catalog devs can write their build-in

Re: [VOTE] FLIP-312: Add Yarn ACLs to Flink Containers

+1 (binding) Thanks for driving the FLIP, Archit. Cheers, Jiangjie (Becket) Qin On Tue, Jun 6, 2023 at 4:33 AM Venkatakrishnan Sowrirajan wrote: > Thanks for starting the vote on this one, Archit. > > +1 (non-binding) > > Regards > Venkata krishnan > > > On Mon, Jun 5, 2023 at 9:55 AM Archit

Re: [VOTE] FLIP-312: Add Yarn ACLs to Flink Containers

+1 (binding) Best, Yang Becket Qin 于2023年6月6日周二 10:35写道： > +1 (binding) > > Thanks for driving the FLIP, Archit. > > Cheers, > > Jiangjie (Becket) Qin > > On Tue, Jun 6, 2023 at 4:33 AM Venkatakrishnan Sowrirajan < > vsowr...@asu.edu> > wrote: > > > Thanks for starting the vote on this one, Arc

[jira] [Created] (FLINK-32266) Kafka Source Continues Consuming Previous Topic After Loading Savepoint

xiechenling created FLINK-32266: --- Summary: Kafka Source Continues Consuming Previous Topic After Loading Savepoint Key: FLINK-32266 URL: https://issues.apache.org/jira/browse/FLINK-32266 Project: Flink

Re: [DISCUSS] FLIP-294: Support Customized Job Meta Data Listener

Hi Martijn, Thanks for your attention, I will soon initiate a discussion about FLIP-314. Best, Shammon FY On Fri, Jun 2, 2023 at 2:55 AM Martijn Visser wrote: > Hi Shammon, > > Just wanted to chip-in that I like the overall FLIP. Will be interesting to > see the follow-up discussion on FLIP-3

Re: [DISCUSS] FLIP-294: Support Customized Job Meta Data Listener

Hi devs: Thanks for all the feedback, and if there are no more comments, I will start a vote on FLIP-294 [1] later. Thanks again. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-294%3A+Support+Customized+Job+Meta+Data+Listener Best, Shammon FY On Tue, Jun 6, 2023 at 1:40 PM Shammon F

Re: [DISCUSSION] Improved USER/SYSTEM exception wrapping in Flink code base

Thanks for bringing this up Hong! Classifying exceptions was also the main driving factor behind pluggable failure enrichers . However, we could do a much better job maintaining a hierarchy of System and User exceptions thus making the classificat

Re: [DISCUSS] FLIP-294: Support Customized Job Meta Data Listener

Hi Shammon, Thanks for driving it! It is a really interesting proposal. Looking forward to the follow-up FLIP for the lineage feature, users will love it :-) There are some inconsistencies in the content. In the very below example, listener.onEvent(CatalogModificationEvent) is called, while in th