Re: Can temporal table functions only be registered using the table API?

I've found more examples here: https://www.ververica.com/blog/a-journey-to-beating-flinks-sql-performance where a fact table is enriched using several dimension tables, but again the temporal table functions are registered using Table API like so: ```java tEnv.registerFunction(

Re: ClassNotFoundException when loading protobuf message class in Flink SQL

Hi James, Your steps seem right. Could you check your jar file '~/repos/simple_protobuf/SimpleTest/SimpleTest.jar' that it does contain 'com.example.SimpleTest.class'? Besides that, to use Kafka connector in sql-client, you should use 'flink-sql-connector-kafka' instead of

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

Hey Martijn, Well, as a user I think that Scala API still adds a tremendous value, with all its issues. But I'm not a committer and I don't know what effort it takes to keep maintaining it... so I prepare for the worst :) Regarding the proposed timeline, I don't know all the specifics around

Re:Question about Flink Broadcast State event ordering

Hi Qing: > I think this is refering to the order between broadcasted element and non > broadcasted element, right? No, as broadcast and nonbroadcast stream are different streams, they will usually transfer with different tcp connection, we can not control the order of elements in different

ClassNotFoundException when loading protobuf message class in Flink SQL

Hi Flink Community, I am trying to prove out the new protobuf functionality added to 1.16 ([1]). I have built master locally and have attempted following the Protobuf Format doc ([2]) to create a table with the kafka connector using the protobuf format. I compiled the sample .proto file using

Re: Flink KafkaSource still referencing deleted topic

Hi Mason, Definitely! Feel free to open a PR and ping me for a review. Cheers, Martijn On Tue, Oct 4, 2022 at 3:51 PM Mason Chen wrote: > Hi Martjin, > > I notice that this question comes up quite often. Would this be a good > addition to the KafkaSource documentation? I'd be happy to

Re: Flink KafkaSource still referencing deleted topic

Hi Martjin, I notice that this question comes up quite often. Would this be a good addition to the KafkaSource documentation? I'd be happy to contribute to the documentation. Best, Mason On Tue, Oct 4, 2022 at 11:23 AM Martijn Visser wrote: > Hi Robert, > > Based on >

Support avro-confluent format for FileSystem SQL connector

Hi Flink community, According to flink doc, avro-confluent([1]) is only supported for kafka sql connector and upsert kafka sql connector. I'm wondering if there is any reason this format is not supported for Filesystem sql connector ([2]) ? We are looking to use FileSystem sink to write to s3

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

Hi Yaroslav, If I could summarize your suggestion, would it mean that you would only be in favour of dropping Scala API support if we introduce Java 17 support exactly at the same time (say Flink 2.0). I was first thinking that an alternative would be to have a Flink 2.0 which supports Java 17

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

Hi Martijn, The 2.0 argument makes sense (I agree it's easier to introduce more breaking changes in one major release), but I think my comment about Java 17 also makes sense in this case: 1) easier to introduce because breaking changes are possible 2) you'd need to give some syntax sugar as an

Re: Can temporal table functions only be registered using the table API?

By looking at the docs for older versions of Flink, e.g., https://nightlies.apache.org/flink/flink-docs-release-1.8/dev/table/streaming/joins.html it seems that it's possible to rewrite this query ``` SELECT o.amount * r.rate AS amount FROM Orders AS o, LATERAL TABLE (Rates(o.rowtime)) AS

Re: Can temporal table functions only be registered using the table API?

It doesn't seem the case with processing time unless I'm mistaken: https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/joins/#processing-time-temporal-join This case seems to require a different syntax based on LATERAL TABLE and a temporal table function (FOR

Re: Flink KafkaSource still referencing deleted topic

Hi Robert, Based on https://stackoverflow.com/questions/72870074/apache-flink-restoring-state-from-checkpoint-with-changes-kafka-topic I think you'll need to change the UID for your KafkaSource and restart your job with allowNonRestoredState enabled. Best regards, Martijn On Tue, Oct 4, 2022

Flink KafkaSource still referencing deleted topic

We've changed the KafkaSource to ingest from a new topic but the old name is still being referenced: 2022-10-04 07:03:41org.apache.flink.util.FlinkException: Global failure triggered by OperatorCoordinator for 'Source: Grokfailures' (operator feca28aff5a3958840bee985ee7de4d3).at

Re: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout

Hi Ori, Thanks for reaching out! I do fear that there's not much that we can help out with. As you mentioned, it looks like there's a network issue which would be on the Google side of issues. I'm assuming that the mentioned Flink version corresponds with Flink 1.12 [1], which isn't supported in

Can temporal table functions only be registered using the table API?

Based on this: https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/concepts/temporal_table_function/ It seems that the only way of registering temporal table functions is via the Table API. If that is the case, is there a way to make this example work ``` SELECT SUM(amount *

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

Hi Yaroslav, Thanks for the feedback, that's much appreciated! Regarding Java 17 as a prerequisite, we would have to break compatibility already since Scala 2.12.7 doesn't compile on Java 17 [1]. Given that we can only remove Scala APIs with the next major Flink (2.0) version, would that still

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

+1 At my employer, we maintain several Flink jobs in Scala. We've been writing newer jobs in Java, and we'd be fine with porting our Scala jobs over to the Java API. I'd like to request Java 17 support. Specifically, Java records is a feature our Flink code would use a lot of and make the Java

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

Hi Martijn, As a Scala user, this change would affect me a lot and I'm not looking forward to rewriting my codebase, and it's not even a very large one :) I'd like to suggest supporting Java 17 as a prerequisite ( https://issues.apache.org/jira/browse/FLINK-15736). Things like switch expressions

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

Hi Martijn, Thanks for bringing this up. It is generally a great idea, so +1. Since both scala extension projects mentioned in the FLIP are still very young and I don't think they will attract more scala developers as Flink could just because they are external projects. It will be a big issue

Question about Flink Broadcast State event ordering

Hi Flink user group, I have a question around broadcast. Reading the docs https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/broadcast_state/#important-considerations, it says the following: > Order of events in Broadcast State may differ across tasks:

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

Hi Marton, You're making a good point, I originally wanted to include already the User mailing list to get their feedback but forgot to do so. I'll do some more outreach via other channels as well. @Users of Flink, I've made a proposal to deprecate and remove Scala API support in a future

Re: Does kubernetes operator support manually triggering savepoint with canceling the job?

This should do the trick: job: upgradeMode: savepoint state: suspended In the CR you should see something similar, after applying the above change. lastSavepoint: formatType: CANONICAL location: file:/flink-data/savepoints/savepoint-fc61e1-b9cf089c260e

Re: 提交新的flink-sql的最佳方案是什么？

Hi， 能麻烦问下你之前使用过哪个支持动态修改SQL的框架或系统吗？ Best， Biao Geng 获取 Outlook for iOS 发件人: Zeguang ZHANG 发送时间: Friday, September 30, 2022 3:26:22 PM 收件人: user-zh@flink.apache.org 主题: 提交新的flink-sql的最佳方案是什么？ 您好，