Re: Monitoring File Processing Progress in Flink Jobs

Hi arjun Flink will save the currently processed file and its corresponding offset in Flink state [1]. You may need to use the Flink state process API[1] to access it. However, I don't think this is a good approach. I suggest adding relevant metrics to the FileSystem connector to report the

Re: Issues about removed topics with KafkaSource

Hi, That's by design: you can't dynamically add and remove topics from an existing Flink job that is being restarted from a snapshot. The feature you're looking for is being planned as part of FLIP-246 [1] Best regards, Martijn [1]

Re: [DISCUSS][FLINK-33240] Document deprecated options as well

Hi Samrat and Ruan, Thanks for the suggestion. I'm actually in favor of adding the deprecated options in the same section as the non-deprecated ones. This would make user search for descriptions of the replacement options more easily. It would be a different story for options deprecated

Re: [DISCUSS][FLINK-33240] Document deprecated options as well

Hi Alexander, I haven't done a complete analysis yet. But through simple code search, roughly 35 options would be added with this change. Also note that some old options defined in a ConfigConstant class won's be added here as flink-doc won't discover these constant-based options. Best,

Re: Re: Query on Flink SQL create DDL primary key for nested field

Hi Xuyang, Thanks again for giving me some insights on how to use the Datastream API for my use case, I will explore it and experiment with it. I wanted to use the value inside the row datatype as a primary key because, I might get multiple records for the same id and when I try to make a join

Re:flink的sql gateway支持自定义的UDF吗？

Hi, 你指的是sql gateway上 ADD JAR这种方式来上传自定义UDF的jar包[1]么？ [1] https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/dev/table/sql/jar/ -- Best！ Xuyang 在 2023-11-01 14:21:04，"RS" 写道： >Hi >flink的sql gateway支持自定义的UDF吗？包括java和python的，有示例可以参考下吗？

Re:Re: Query on Flink SQL create DDL primary key for nested field

Hi, Elakiya, I think you can get what you need here[1] with many examples briging DataStream api and Table API. There may be some redundancy, and I'm not sure this is a best way to resolve the question. First, use the StreamTableEnvironment to execute your original ddl without pk. Second,

Re: [DISCUSS][FLINK-33240] Document deprecated options as well

Thanks for the proposal. +1 from my side and +1 for putting them to a separate section. Best, Hang Samrat Deb 于2023年11月1日周三 15:32写道： > Thanks for the proposal , > +1 for adding deprecated identifier > > [Thought] Can we have seperate section / page for deprecated configs ? Wdut > ? > > >

Re: Query on Flink SQL create DDL primary key for nested field

Hi Xuyang, Thank you for your response. Since, I have no access to create a ticket in the ASF jira I have requested for the access and once I get the access will raise a ticket for the same. Also, you have asked me to use Datastream API to extract the id and then use the TableAPI feature, since

Issues about removed topics with KafkaSource

Hey We have a flinkapp which is subscribing to multiple topics, we recently upgraded our application from 1.13 to 1.15, which we started to use KafkaSource instead of FlinkKafkaConsumer (deprecated). But we noticed some weird issue with KafkaSource after the upgrade, we are setting the topics

flink的sql gateway支持自定义的UDF吗？

Hi flink的sql gateway支持自定义的UDF吗？包括java和python的，有示例可以参考下吗？

Re:flink 1.18.0的sql gateway支持提交作业到k8s上运行吗？

Hi， 提交到本地是flink配置文件里面配置的jobmanager的地址，所以肯定也是提交到K8S的吧 yarn的不太清楚。 在 2023-10-30 14:36:23，"casel.chen" 写道： >想问一下目前flink 1.18.0的sql gateway只支持提交作业到本地运行吗？能否支持提交作业到yarn或k8s上运行呢？