Hi,
I added some metrics on a step right after KafkaIO. When I compare the read
time difference between producer and KafkaIO it is 800ms for P99. However
somehow that step's opening and closing bundle difference is 18 seconds for
p99. The step itself does not do any specific thing. Do you have any
Thank you Luke and Reuven for helping me. Now I can see my pipeline
processing time for each record.
On Wed, Jun 3, 2020 at 9:25 AM Reuven Lax wrote:
> Note: you need to tag the timestamp parameter to @ProcessElement with
> the @Timestamp annotation.
>
> On Mon, Jun 1, 2020 at 3:31 PM Luke Cwik
Hi Brian,
Thanks so much for your quick response!
I've tried with both Apache Beam 2.20.0 and 2.21.0, both result in the
exact same error. Here is the full stacktrace:
(metadata-persistor) ➜ metadata-persistor git:(feature/DEV-1249) ✗
metadata_persistor --project X --environments X --wi
Quick update: this test code works just fine on Dataflow as well as the
DirectRunner. Looks like the FlinkRunner is problematic for some reason here.
On 2020/06/08 20:11:13, Pradip Thachile wrote:
> Hey folks,
>
> I posted this on the Flink user mailing list but didn't get any traction
> the
