For some reason, after restarting the app and trying again, latest now works as expected. Not sure why it didn’t work before.
On Tue, Apr 21, 2020 at 1:46 PM Ruijing Li <liruijin...@gmail.com> wrote: > Yes, we did. But for some reason latest does not show them. The count is > always 0. > > On Sun, Apr 19, 2020 at 3:42 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> > wrote: > >> Did you provide more records to topic "after" you started the query? >> That's the only one I can imagine based on such information. >> >> On Fri, Apr 17, 2020 at 9:13 AM Ruijing Li <liruijin...@gmail.com> wrote: >> >>> Hi all, >>> >>> Apologies if this has been asked before, but I could not find the answer >>> to this question. We have a structured streaming job, but for some reason, >>> if we use startingOffsets = latest with foreachbatch mode, it doesn’t >>> produce any data. >>> >>> Rather, in logs I see it repeats the message “ Fetcher [Consumer] >>> Resetting offset for partition to offset” over and over again.. >>> >>> However with startingOffsets=earliest, we don’t get this issue. I’m >>> wondering then how we can use startingOffsets=latest as I wish to start >>> from the latest offset available. >>> -- >>> Cheers, >>> Ruijing Li >>> >> -- > Cheers, > Ruijing Li > -- Cheers, Ruijing Li
