Re: How to read flinkSQL job state

Hi Yifan, AFAIK, if you want to query a job’s state from outside Flink, you can use Queryable State[1]. Hope this helps. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/queryable_state/ Xiangyu Yifan He via user 于2023年9月6日周三 13:10写道： > Hi team, >

How to read flinkSQL job state

Hi team, We are investigating why the checkpoint size of our FlinkSQL jobs keeps growing and we want to look into the checkpoint file to know what is causing the problem. I know we can use the state processor api to read the state of jobs using datastream api, but how can I read the state of jobs

Re: Question regarding asyncIO timeout

Hi Ken, Thanks for the suggestion. Definitely a good call to just wrap the retry inside the client code. I'll give it a try. Besides that, Do you know if the async timeout is actually a global timeout? meaning it accounts for the time of each attempt call plus any interval time in between. I incre

Re: Question regarding asyncIO timeout

Hi Leon, Normally I try to handle retrying in the client being used to call the server, as you have more control/context. If that’s not an option for you, then normally (un)orderedWaitWithRetry() should work - when you say “it doesn’t seem to help much”, are you saying that even with retry you

Question regarding asyncIO timeout

Hi Flink users, We are using Flink AsyncIO to call a grpc-based service in our Flink job. And from time to time we are experiencing Async function timeout issues, here's the exception. ``` java.lang.Exception: Could not complete the stream element: Record @ 169393916 : [B@cadc5b3. Caused by: j

Disabling Job Submissions through Flink UI

Hello Team, I hope this email finds you well. I've been working on disabling job submissions from the Flink UI, and I am facing an issue with the configuration. Specifically, I've set web.submit.enable = false to achieve this. However, it seems that this configuration change is preventing the Flin

Re: In Flink, is there a way to merge two streams in stateful manner

Thank you for the detailed explanation. On Tue, 29 Aug 2023 at 22:45, Ken Krugler wrote: > If you need a specific output order, then merge the three streams, key by > a constant (like 1), and run that into a KeyedProcessFunction. > > That function can buffer out-of-order records, and set up a ti

Re: Keytab Setup on Kubernetes

hi Chirag, Flink now supports 2 ways to have TGT which is a Kerberos ticket and has nothing to do with the "until 7 days renewable" HDFS TGS ticket (with default config). * Keytab: if one mounts a keytab for at least the JobManager pod then it can create TGT infinitely (or until the user's passwo

Keytab Setup on Kubernetes

Hi, I am trying to use the FileSource to collect files from HDFS. The HDFS cluster is secured and has Kerberos enabled. My Flink cluster runs on Kubernetes (not using the Flink operator) with 2 Job Managers in HA and 3 Task Managers. I wanted to understand the correct way to configure the keytab

Re: Flink K8S operator does not support IPv6

FYI, adding environment variables of ` KUBERNETES_DISABLE_HOSTNAME_VERIFICATION=true` works for me. This env variable needs to be added to both the Flink operator and the Flink job definition. On Tue, Aug 8, 2023 at 12:03 PM Xiaolong Wang wrote: > Ok, thank you. > > On Tue, Aug 8, 2023 at 11:22