Unaligned checkpoint waiting in 'start delay' with AsyncDataStream

2022-07-01 Thread Nathan Sharp
I am attempting to use unaligned checkpointing with AsyncDataStream, but the checkpoints sit in "start delay" until the job finishes. I am brand new to Flink, so it is entirely reasonable to assume the problem is with my code. I published my test code at https://github.com/phxnsharp/AsyncDataS

Alternate Forms of Deserialization in Flink SQL CLI

2022-07-01 Thread Eric Carlson
Hello, What would be the most natural way to plug in a custom deserializer or deserialization format to the Flink SQL CLI (or to do something similar which achieves the same effect)? There is no support for Protobuf, for instance, and regardless it should make the CLI much more useful. It is strai

Re: influxdb metrics reporter - 4k series per job restart

2022-07-01 Thread Mason Chen
Hi all, If you can wait for Flink 1.16, there is a new feature to filter metrics (includes/excludes filter). Additionally, you can already take advantage of dropping unnecessary labels with `scope.variables.excludes` in the current release. Link to 1.16 metric features: https://nightlies.apache.or

Does Flink 1.14 support comsume Kafka 0.9?

2022-07-01 Thread hjw
Flink  1.14 Document  display Flink 1.14 kafka connector only  backwards compatible with broker versions 0.10.0 or later. Unfortunately, I have to use Flink 1.14 comsume Kafka 0.9 version? How can I do it ??thx.  

Re: Flink 1.12 StreamRecordQueueEntry is not public class

2022-07-01 Thread Martijn Visser
What I'm interested in is understanding why you're relying on this class; what is the problem you're trying to solve? That input could be useful to consider making that interface public or there could be another way to solve your problem. Best regards, Martijn Op wo 29 jun. 2022 om 18:12 schreef

Re: Flink Elasticsearch connector 7 with SSL

2022-07-01 Thread Martijn Visser
Hi Shishal, Please don't cross-post your question in multiple channels. As answered in Slack, SSL connections to Elasticsearch7 are currently not supported and are tracked under https://issues.apache.org/jira/browse/FLINK-27054 An Elasticsearch8 connector is being tracked under https://issues.apa

Re: RuntimeContextInitializationContextAdapters: ClassNotFoundException

2022-07-01 Thread Martijn Visser
Hi Harald, Are you using Maven? Do you use IntelliJ? And have you followed the project configuration page, with regards to setting dependencies as ? [1] Best regards, Martijn [1] https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/configuration/maven/#importing-the-project-into

Re: influxdb metrics reporter - 4k series per job restart

2022-07-01 Thread Martijn Visser
Have you considered setting the value for some of the series to a fixed value? For example, if you're not interested in the value for , you could consider setting that to a fixed value "task_id" [1] ? Best regards, Martijn [1] https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics