Re: kerberos token expire

The ticket cache will be expired after its lifespan. You can try to set the security.kerberos.login.use-ticket-cache to false as you provide the keytab. Best, Yangze Guo On Tue, Jul 6, 2021 at 10:02 AM 谢扬成 wrote: > > Hi, > > I processed data with flink which version is 1.12.2, data source read f

Flink + Kafka Dynamic Table

Hi, flink community! I have below scenario in medical field - record is frequently modified and must not be lost - when record is modified the results previously produced by this record should also be modified. e.g. table A, B, C. A join B and result is table D, A join C and result is table

Re: Job Recovery Time on TM Lost

I know that there are retry strategies for akka rpc frameworks. I was just considering that, since the environment is shared by JM and TMs, and the connections among TMs (using netty) are flaky in unstable environments, which will also cause the job failure, is it necessary to build a strongly guar

Re: Yarn doesn't deploy multple TMs; -yn option missing in newer versions

Hi, Marzi, > But only one Task manager spins up and even when I submit a second job which > requires more resources, The job is stuck in SCHEDULED state and no > additional TMs get automatically assigned. Would you like to check if there are enough resources in your yarn cluster? Also, the JM lo

kerberos token expire

Hi, I processed data with flink which version is 1.12.2, data source read from kafka, after logic processing, then write into HDFS with parquet format, the Hadoop cluster opened kerberos authentication mechanism. flink-conf.yml like below: security.kerberos.login.use-ticket-cache: true security

How Do I Specify the Encryption Algorithm Suite of the Flink REST Service?

Hi all: How Do I Specify the Encryption Algorithm Suite of the REST Service After the SSL Option Is Enabled for Flink? 王辉 （WangHui） 智慧园区解决方案部 [云与计算BG] 华为技术有限公司 Tel : +86 15940836304 Email : wanghu...@huawei.com [cid:image001.png@01D7724B.C3FCB070] This e-mail and its

Yarn doesn't deploy multple TMs; -yn option missing in newer versions

Hi All, I am exploring running Flink 1.12.0 on yarn and so far I have been able to start a yarn session and submit a job. But only one Task manager spins up and even when I submit a second job which requires more resources, The job is stuck in SCHEDULED state and no additional TMs get automatic

Savepoints with bootstraping a datastream function

I'm trying to bootstrap state into a KeyedProcessFunction equivalent that takes in a DataStream but I'm unable to find a reference for the same. I found this gist https://gist.github.com/alpinegizmo/ff3d2e748287853c88f21259830b29cf But it seems to only apply for DataSet. My usecase is to manually t

Flink 1.13 fails to load log4j2 yaml configuration file via jackson-dataformat-yaml

Hi, I am attempting to upgrade Flink from 1.9 to 1.13.1 I am using a YAML based log4j file. In 1.9, it worked perfectly fine by adding the following dependency to the classpath (I deploy with an uber JAR): "com.fasterxml.jackson.dataformat" % "jackson-dataformat-yaml" % "2.12.3" However, with Fl

答复: WELCOME to user@flink.apache.org

Hello, I find that security.ssl.rest.enabled: true is configured in flink, and the Java keystore password is stored in plaintext in the configuration file. Can the keystore password be encrypted for storage?

The best way to read historical data in a stream

Hi I’m looking for some advice for the “right” way to load historical data into a stream. The case is as follow. I have a stream, sometimes I need to match the current live stream data up against data stored in database, let say elasticsearch, I generate a side output with the query informat

Unsubscribe

Unsubscibe pon., 5 lip 2021, 08:19 użytkownik Dan Pettersson < dan.pettersso...@gmail.com> napisał: > Unsubscribe >

Re: Corrupted unaligned checkpoints in Flink 1.11.1

Hey Alex, Sorry, I've missed your previous email. I've spent a bit more time searching our Jira for relevant bugs and maybe you were hit by this one: https://issues.apache.org/jira/browse/FLINK-21351 ? > T2: Job1 was savepointed, brought down and replaced with Job2. This in combination with FLINK

Re: Job Recovery Time on TM Lost

I think for RPC communication there are retry strategies used by the underlying Akka ActorSystem. So a RpcEndpoint can reconnect to a remote ActorSystem and resume communication. Moreover, there are also reconciliation protocols in place which reconcile the states between the components because of

Re: Job Recovery Time on TM Lost

As far as I know, a TM will report connection failure once its connected TM is lost. I suppose JM can believe the report and fail the tasks in the lost TM if it also encounters a connection failure. Of course, it won't work if the lost TM is standalone. But I suppose we can use the same strategy a

Re: Suspected SPAM - RE: FW: Hadoop3 with Flink

Hi Suchithra, I suspect you want to connect to something like HDFS from the running k8s application. Is this correct? Yangze already gave some good hints you have to make sure that the necessary Hadoop java libraries and configurations are part of your Flink docker images or are mounted from an