How do you check if new data is in the topic and what happens if not?
On Sat, 12 Mar 2022 at 00:40, karan alang wrote:
> Hello All,
>
> I have a structured Streaming program, which reads data from Kafka topic,
> and does some processing, and finally puts data into target Kafka Topic.
>
> Note :
There are a number of flaws here.
You have defined your trigger based processing time within Spark
Structured Streaming (SSS) as below
trigger(processingTime='4 minutes')
SSS will trigger every 4 minutes, in other words within a micro-batch of 4
minutes. This is what is known as micro-batch int
hi,
Does anybody else have a better suggestion for my problem?.
Thanks,
Venkat
2016173438
On Fri, Mar 11, 2022 at 4:43 PM Venkatesan Muniappan <
m.venkatbe...@gmail.com> wrote:
> ok. I work for an org where such upgrades take a few months. Not an
> immediate task.
>
> Thanks,
> Venkat
> 2016173
