What you say makes sense to me. I am not aware of any known bugs, hence,
it seems you are hitting some unknown bug.
Can you reproduce it reliably? Maybe you can share the code and input
data set so we can have a look? Best would be, if we could reproduce it
via TopologyTestDriver.
-Matthias
On
Hi Matthias,
when we say 2.4.0, we just built the kafka from source and marked it as 2.4.0
for our internal reference.
But I have taken latest Kafka 2.3.1 and tested & the issue still persists.
pseudo code:
.filter((key, value) -> ...)
.groupByKey()
.windowedBy(TimeWindows.of(Duration.ofMinutes
Did you try to test you code using `TopologyTestDriver`? Maybe this
helps to figure out the root cause of the issue.
We have many unit/integration tests in place and many people use
suppress() successfully in production. Hence, I am sure, it basically
works -- of course, they might still be an unk
Tried your suggestions and unable to get suppress emit anything. I can see the
SUPPRESS_STORES are created in Kafka nodes but nothing get outputted.
Looks like the grace period and window closing is not honored for some reason.
I can see lot of people having difficulty of getting suppress workin
Have you tried deleting the suppress changelog topic to see if you get
something after deleting it?
By per topic and not per key I mean that if you send for example an event
with timestamp equal to today's date with key 1 and that closes today's
window and data in the past with key 2 won't go thro
I tried with different timestamps in the near past but nothing coming out. I
went thru the article from Confluent about using the suppress but I don’t see
many people are successful with that.
What do you mean by “timestamp is per topic and not per key”. Can you please
elaborate?
> On Sep
Did you ever push any data with a greater timestamp than the current one
you're producing?
One thing took me a while to find out is that the suppress timestamp is per
topic and not per key
--
Alessandro Tagliapietra
On Wed, Sep 11, 2019 at 8:06 PM Thameem Ansari wrote:
> Yes I am able to see t
Yes I am able to see the output when I remove suppress.
> On Sep 11, 2019, at 9:58 PM, Matthias J. Sax wrote:
>
> Hard to say. Do you see output if you remove `suppress()` from your
> topology?
>
> -Matthias
>
>
> On 9/11/19 6:19 PM, Thameem Ansari wrote:
>> I am using a producer simulator
Hard to say. Do you see output if you remove `suppress()` from your
topology?
-Matthias
On 9/11/19 6:19 PM, Thameem Ansari wrote:
> I am using a producer simulator to simulate the events in the past and I can
> see my time advances and the topology is based on the event time. But even if
> I r
I am using a producer simulator to simulate the events in the past and I can
see my time advances and the topology is based on the event time. But even if I
run the producer for few hours nothing get emitted. Is there anyway to debug
this issue?
> On Sep 11, 2019, at 6:13 PM, Matthias J. Sax
Note that `suppress()` is event time based, and does not emit any data
if event time does not advance.
A common miss understanding is, that people stop to send data and expect
to see a result after some time, but that is not how it works. If you
stop sending data, event time cannot advance and thu
In my streaming topology, I am using the suppress dsl operator. As per the
documentation, it is supposed to output the final results after the window
closes. But I noticed it's not emitting anything at all. Here is the pseudo
code of my topology.
.filter((key, value) -> ...)
.flatMap((key, val
12 matches
Mail list logo
