Re: Unbalanced distribution of keyed stream to downstream parallel operators

2022-04-07 Thread Isidoros Ioannou
k if the distribution looks better. Have a look at > KeyGroupRangeAssignment [1] to test that out without running Flink itself. > > [1] > https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/state/KeyGroupRangeAssignment.java > > On

Re: Unbalanced distribution of keyed stream to downstream parallel operators

2022-04-04 Thread Isidoros Ioannou
t; Hope this would be helpful! > > Qingsheng > > > On Apr 1, 2022, at 17:37, Isidoros Ioannou wrote: > > > > Hello, > > > > we ran a flink application version 1.13.2 that consists of a kafka > source with one partition so far > > then we filter the da

Unbalanced distribution of keyed stream to downstream parallel operators

2022-04-01 Thread Isidoros Ioannou
Hello, we ran a flink application version 1.13.2 that consists of a kafka source with one partition so far then we filter the data based on some conditions, mapped to POJOS and we transform to a KeyedStream based on an accountId long property from the POJO. The downstream operators are 10 CEP oper

Re: Flink kafka consumer disconnection, application processing stays behind

2022-03-24 Thread Isidoros Ioannou
ut drops, or dump > the record to see if there’s something special in it. > > Hope these could be helpful! > > Best regards, > > Qingsheng > > > On Mar 23, 2022, at 19:19, Isidoros Ioannou wrote: > > > > Hi, we are running flink 1.13.2 version on Kinesis Analyt

Flink kafka consumer disconnection, application processing stays behind

2022-03-23 Thread Isidoros Ioannou
Hi, we are running flink 1.13.2 version on Kinesis Analytics. Our source is a kafka topic with one partition so far and we are using the FlinkKafkaConsumer (kafka-connector-1.13.2) Sometimes we get some errors from the consumer like the below: "locationInformation":"org.apache.kafka.clients.FetchS

Low Watermark

2022-02-25 Thread Isidoros Ioannou
Hello, could someone please explain what the Low Watermark indicates in the Flink UI in the attached image? I have event time enabled with a boundOutOfOrdernessStrategy of 3s for the incoming events and I use CEP with a within window of 5 minutes.

Re: IterativeCondition instead of SimpleCondition not matching pattern

2021-11-04 Thread Isidoros Ioannou
does not work properly. Any suggestions? BR, Isidoros Στις Πέμ, 4 Νοε 2021 στις 11:27 μ.μ., ο/η Isidoros Ioannou < akis3...@gmail.com> έγραψε: > I am using Processing time characteristic. > > DataStream inputStream = env.fromElements( > Model.of(1, "

Re: IterativeCondition instead of SimpleCondition not matching pattern

2021-11-04 Thread Isidoros Ioannou
.@gmail.com> έγραψε: > Thanks for the update, the requirements make sense. > > Some follow up questions: > * What time characteristic are you using? Processing or Event? > * Can you describe a bit more what you mean by "input like the one I have > commented bellow"?

Fwd: IterativeCondition instead of SimpleCondition not matching pattern

2021-11-04 Thread Isidoros Ioannou
-- Forwarded message - Από: Isidoros Ioannou Date: Πέμ, 4 Νοε 2021 στις 10:01 μ.μ. Subject: Re: IterativeCondition instead of SimpleCondition not matching pattern To: Austin Cawley-Edwards Hi Austin, thank you for your answer and I really appreciate your willingness to help

Fwd: IterativeCondition instead of SimpleCondition not matching pattern

2021-11-04 Thread Isidoros Ioannou
I face an issue when try to match some elements in a Pattern sequence. Flink 1.11.1 version. Here is my case: final StreamExecutionEnvironment env = EnvironmentProvider.getEnvironment(); DataStream inputStream = env.fromElements( Model.of(1, "A", "US"), Model.of(2, "B", "US