Could you clarify what you mean by "inconsistent" and "incorrect"? Are elements missing/duplicated, or just batched differently?
On Fri, Aug 9, 2019 at 2:18 AM rahul patwari <[email protected]> wrote: > > I only ran in Direct runner. I will run in other runners and let you know the > results. > I am not setting "streaming" when executing. > > On Fri 9 Aug, 2019, 2:56 AM Lukasz Cwik, <[email protected]> wrote: >> >> Have you tried running this on more than one runner (e.g. Dataflow, Flink, >> Direct)? >> >> Are you setting --streaming when executing? >> >> On Thu, Aug 8, 2019 at 10:23 AM rahul patwari <[email protected]> >> wrote: >>> >>> Hi, >>> >>> I am getting inconsistent results when using GroupIntoBatches PTransform. >>> I am using Create.of() PTransform to create a PCollection from in-memory. >>> When a coder is given with Create.of() PTransform, I am facing the issue. >>> If the coder is not provided, the results are consistent and correct(Maybe >>> this is just a coincidence and the problem is at some other place). >>> If Batch Size is 1, results are always consistent. >>> >>> Not sure if this is an issue with Serialization/Deserialization (or) >>> GroupIntoBatches (or) Create.of() PTransform. >>> >>> The Java code, expected correct results, and inconsistent results are >>> available at https://github.com/rahul8383/beam-examples >>> >>> Thanks, >>> Rahul
