Hi,
My issue has been resolved through discussion with AWS support.
It turns out that Kinesis Data Analytics reports to CloudWatch in a way I did
not expect. The way to view the accurate values for Flink counters is with
Average in CloudWatch metrics.
Below is the response from AWS support,
Hi Dawid,
My team's Flink application's primary purpose is not to count the number of SQS
messages received or the number of successful or failed S3 downloads. The
application's primary purpose is to process events and the corresponding data,
and for each event, create or update a new entry in
From: Zhanghao Chen
Sent: March 5, 2022 11:11 PM
To: Shane Bishop ; user@flink.apache.org
Subject: Re: Question about Flink counters
Hi Shane,
Could you share more information on what you would like to use the counter for?
The counter discussed here is primarily designed for exposing counts
If I used a thread-safe counter implementation, would that be enough to make
the count correct for a Flink cluster with multiple machines?
Best,
Shane
From: Zhanghao Chen
Sent: March 4, 2022 11:08 PM
To: Shane Bishop ; user@flink.apache.org
Subject: Re
Hi all,
For Flink counters [1], are increment operations guaranteed to be atomic across
all parallel tasks? I.e., is there a guarantee that the counter values will not
be higher than expected?
Thanks,
Shane
---
[1]
Hi all,
My team has started seeing the error "java.lang.IllegalStateException: No
operators defined in streaming topology. Cannot execute." However, even with
this error, the Flink application starts and runs fine, and the Flink job
renders fine in the Flink Dashboard.
Attached is the full
type is registered with Kryo or not.
[1] https://issues.apache.org/jira/browse/FLINK-25993
Best regards,
Shane
From: Chesnay Schepler
Sent: February 7, 2022 3:08 AM
To: Shane Bishop ; user@flink.apache.org
Subject: Re: Questions about Kryo
Sent: February 7, 2022 3:08 AM
To: Shane Bishop ; user@flink.apache.org
Subject: Re: Questions about Kryo setRegistrationRequired(false)
There isn't any setting to control setRegistrationRequired().
You can however turn Kryo off via ExecutionConfig#disableGenericTypes, although
this may
butor, so I might be missing safety mechanisms that are in
place to prevent the Kryo serialization/deserialization vulnerability even when
registration required is set to false. Are there any such safety mechanisms in
place?
Is there anything I can do as a user of Flink to protect myself against this
Kryo vulnerability?
Best regards,
Shane Bishop
