pyflink aggfunction in tvf question

2023-09-12 Thread faronzz
hi flink community~ I came across a problem I didn't understand,I can't use pyflink aggfuction function properly in window tvf, The following are available: java aggfuntion flink system aggfunction window (not window tvf) I want to know if this is a bug or if I'm using it the wrong way?Tha

Re: Checkpoint jitter?

2023-09-12 Thread Hangxiang Yu
Hi, Matyas. Do you mean something like adjusting checkpoint intervals dynamically or frequency of uploading files according to the pressure of the durable storage ? On Wed, Sep 13, 2023 at 9:12 AM Őrhidi Mátyás wrote: > Hey folks, > > Is it possible to add some sort of jitter to the checkpointin

Re: Reading parquet files using Flink

2023-09-12 Thread liu ron
Hi, Are you using the DataStream API to read parquet file? Why not use Flink SQL to read the it? The ParquetRowInputFormat has been removed, you can use ParquetColumnarRowInputFormat in 1.17.1. Best, Ron Hou, Lijuan via user 于2023年9月12日周二 05:49写道: > Hi team, > > > > Is there any defined way t

Re: Order of Execution in KeyedBroadcastProcessFunction

2023-09-12 Thread liu ron
Hi, The KeyedBroadcastProcessFunction#processBroadcastElement is called in outer CoBroadcastWithKeyedOperator[1], CoBroadcastWithKeyedOperator is a two-input operator. As David says, this operator is run in a single thread, so the processBroadcastElement method will run to completion before any fu

Checkpoint jitter?

2023-09-12 Thread Őrhidi Mátyás
Hey folks, Is it possible to add some sort of jitter to the checkpointing logic for massively parallel jobs to mitigate the burst impact on the durable storage when a checkpoint is triggered? Thanks, Matyas

Modifying default config when deploying operator with OLM

2023-09-12 Thread Zach Lorimer
Hi, Is there any way to override or modify the default configuration that is deployed in the Flink Operator when using OLM (operator lifecycle manager)? I can't seem to find anything and the only solution I've come up with is to patch the ConfigMap after deployment and wait 5 minutes for the dynami

Re: Flink kubernets operator delete HA metadata after resuming from suspend

2023-09-12 Thread Gyula Fóra
Hi! I think this issue is the same as https://issues.apache.org/jira/browse/FLINK-33011 Not sure what exactly is the underlying cause as I could not repro it, but the fix should be simple. Also I believe it's not 1.6.0 related unless a JOSDK/Fabric8 upgrade caused it. Cheers, Gyula On Mon, Sep