[Flink operator] Flink Autoscale - Limit the max number of scale ups

Hi, I am trying the autoscale provided by the operator. I found that Autoscale keeps happening even after reaching max-parallelism. I have a question. How can we set the limit for the scale-up process? Because in my cluster it reached the max limit. So, there won't be any more resources

Re: Facing issue when using S3 in Flink 1.17

Thanks a lot, Gabor. Do you know by any chance when this fix will be released? Thanks in advance. On Thu, Apr 20, 2023 at 9:06 PM Gabor Somogyi wrote: > Hi Sriram, > > This has been fixed in https://issues.apache.org/jira/browse/FLINK-31839 > > G > > > On Thu, Apr 20

Facing issue when using S3 in Flink 1.17

Hi Team, I am using S3 as FileSystem to write data from Flink. I am getting the below error in Flink 1.17. The same code works in Flink 1.16. Could someone please help? Caused by: java.lang.IllegalStateException: Delegation token provider with service name {} has multiple implementations [s3]

Re: Using S3 as stream source in Flink

tem/ > and > https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/filesystem/ > > Best regards, > > Martijn > > Op di 24 jan. 2023 om 10:26 schreef Sriram Ganesh : > >> Hi Everyone, >> >> I am thinking of switching my input source f

Using S3 as stream source in Flink

Hi Everyone, I am thinking of switching my input source from Kafka to S3. First, I couldn't find any streaming source connector for S3. I have some basic questions about 1. How S3 will work as a streaming source with proper checkpointing.? 2. How Flink will manage the last offset processed from

Re: Modify savepoints in Flink

> [1] > https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/libs/state_processor_api/#keyed-state > > pt., 21 paź 2022 o 14:37 Sriram Ganesh napisał(a): > >> I have question on this. Different connector can have different >> serialisation and de-serlisation tec

Re: Modify savepoints in Flink

ot exposed via Public API) - > best to use StateProcessor API. > > Best, > Piotrek > > pt., 21 paź 2022 o 10:54 Sriram Ganesh napisał(a): > >> Thanks !. Will try this. >> >> On Fri, Oct 21, 2022 at 2:22 PM Piotr Nowojski >> wrote: >> >>> Hi Sriram, &

Re: Modify savepoints in Flink

he next savepoint. > > Best, > Piotrek > > [1] > https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/libs/state_processor_api/ > > pt., 21 paź 2022 o 10:05 Sriram Ganesh napisał(a): > >> Hi All, >> >> I am working on a scenario where I need

Modify savepoints in Flink

Hi All, I am working on a scenario where I need to modify the existing savepoint operator state. Ex: Wanted to remove some offset of the savepoint. What is the better practice for these scenarios?. Could you please help me with any example as such? Thanks in advance. -- *Sriram G* *Tech*