Yes, we did. But for some reason latest does not show them. The count is always 0.
On Sun, Apr 19, 2020 at 3:42 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> wrote: > Did you provide more records to topic "after" you started the query? > That's the only one I can imagine based on such information. > > On Fri, Apr 17, 2020 at 9:13 AM Ruijing Li <liruijin...@gmail.com> wrote: > >> Hi all, >> >> Apologies if this has been asked before, but I could not find the answer >> to this question. We have a structured streaming job, but for some reason, >> if we use startingOffsets = latest with foreachbatch mode, it doesn’t >> produce any data. >> >> Rather, in logs I see it repeats the message “ Fetcher [Consumer] >> Resetting offset for partition to offset” over and over again.. >> >> However with startingOffsets=earliest, we don’t get this issue. I’m >> wondering then how we can use startingOffsets=latest as I wish to start >> from the latest offset available. >> -- >> Cheers, >> Ruijing Li >> > -- Cheers, Ruijing Li
