Re: [EXTERNAL] Re: flink s3[parquet] -> s3[iceberg]

Hi Dan I think using Flink SQL should be able to meet your needs. You can write a Flink Jar program. Accept different directories, schemas, mappings, and sink tables to generate DDL and DML. Assuming you have two directories: directory1 -> f1, f2, f3, f4 -> iceberg1 directory2 -> f1, f2, f3 ->

Re: dependency error with latest Kafka connector

Hi all, I've cherry-picked FLINK-30400 onto v3.0 branch of flink-connector-kafka. Treating this thread as justification to start a vote for 3.0.2 RC #1 immediately so we can get out a new release ASAP. Please see the vote thread here [1]. @guenterh.lists Would you be able to test this RC and

Doubts about state and table API

Hi, We are having a job in production where we use table API to join multiple topics. The query looks like this: SELECT * FROM topic1 AS t1 JOIN topic2 AS t2 ON t1.userId = t2.userId JOIN topic3 AS t3 ON t1.userId = t3.accountUserId This works and produces an EnrichedActivity any time any of

Elasticsearch connector support?

Hi From the documentation I can see there isn’t any ES support in Flink 1.18 right now and Flink-26088 (ES 8 support) is still open. Does anyone has an idea when ES connector support will be available in 1.18 Please let me know. Med venlig hilsen / Best regards Lasse Nedergaard

Re: dependency error with latest Kafka connector

> built a fat uber jar from quickstart with Flink 1.18.0 for > flink-streaming-java and flink-clients, and flink-connector-kafka version > 3.0.1-1.18 > then submitted to local Flink cluster 1.18.0. Things worked as expected and > the job ran fine. Hey，@Gordan I guess things may work as

Re: 关于Flink SQL语句内的函数名和内建函数名称对应不上的问题

感谢大佬，我找到了。 所以说SQL类的内建函数实际上使用的是calcite的能力，而flink自己的内建函数是在table api中使用 > 2023年11月24日 17:07，Xuyang 写道： > > Hi, > 关于你举的例子，如果编译了源码的话，可以在FlinkSqlParserImpl这个被动态生成的词法解析器类中找到PostfixRowOperator方法，大致是通过识别到IS > NOT NULL这三个关键字，转化为Calcite的这个内置函数SqlStdOperatorTable.IS_NOT_NULL > > > > > -- > >

Re: Confluent Kafka conection error

Hi Hang, Few more points regarding this issue 1. The issue does not replicate in my local installation and occurs only on kubernetes server installation. 2. We have used kubernetes operator to install flink on the server Please let me know if any other info is required here. Thanks, Tauseef

Re: Confluent Kafka conection error

Hi Hang, I cross checked this issue multiple times. I also upgraded to flink 1.18 but the issue persists. Can you please let me know a few guidelines on how to investigate this and fix it positively. Thanks, Tauseef On Thu, 23 Nov 2023 at 18:08, Tauseef Janvekar wrote: > Thanks Hang. > > I

Re: dependency error with latest Kafka connector

Hi Gordon, I'm wondering if this might be a difference between how Maven and Gradle build their projects, since you've done your validations with Maven, but Günter uses Gradle. In the end, the quickest fix would be to backport FLINK-30400 to the Flink Kafka 3.0 release branch. Best regards,

Re:关于Flink SQL语句内的函数名和内建函数名称对应不上的问题

Hi, 关于你举的例子，如果编译了源码的话，可以在FlinkSqlParserImpl这个被动态生成的词法解析器类中找到PostfixRowOperator方法，大致是通过识别到IS NOT NULL这三个关键字，转化为Calcite的这个内置函数SqlStdOperatorTable.IS_NOT_NULL -- Best！ Xuyang 在 2023-11-24 15:15:04，"jinzhuguang" 写道： >flink 1.18.0 > > >例如我写下一条SQL： > select * from KafkaTable where