Yea have tried ENABLE_AUTO_COMMIT_CONFIG , its helping me before GroupByKey but not after that. In other words once the window is materialized and if any exception occurs while processing the elements in it, its dropping the remaining elements of that window when restarted.
On Jan 8, 2018 12:45 PM, "Reuven Lax" <re...@google.com> wrote: > Do you set ENABLE_AUTO_COMMIT_CONFIG? > > On Sun, Jan 7, 2018 at 10:57 PM, Sushil Ks <sushil...@gmail.com> wrote: > >> HI Aljoscha, >> The issue is let's say I consumed 100 elements in 5 >> mins Fixed Window with *GroupByKey* and later I applied *ParDO* for all >> those elements. If there is an issue while processing element 70 in >> *ParDo *and the pipeline restarts with *UserCodeException *it's skipping >> the rest 30 elements. Wanted to know if this is expected? In case if you >> still having doubt let me know will share a code snippet. >> >> Regards, >> Sushil Ks >> > >
