Thanks! Appreciate the quick response!!
-Sameer.
On 2021/01/20 19:41:39, Boyuan Zhang wrote:
> I opened https://github.com/apache/beam/pull/13779 for exxposing built-in
> timestamp policy and commitOffsetInFinalize to ReadFromKafka.
>
> On Fri, Jan 15, 2021 at 3:09 PM Chamikara Jayalath
>
I opened https://github.com/apache/beam/pull/13779 for exxposing built-in
timestamp policy and commitOffsetInFinalize to ReadFromKafka.
On Fri, Jan 15, 2021 at 3:09 PM Chamikara Jayalath
wrote:
>
>
> On Fri, Jan 15, 2021 at 2:55 PM Boyuan Zhang wrote:
>
>> Re Cham,
>>
>>> Looking at
On Fri, Jan 15, 2021 at 2:55 PM Boyuan Zhang wrote:
> Re Cham,
>
>> Looking at https://github.com/apache/beam/pull/12572 seems like you just
>> need support for FixedWindows, right ? This should work now I believe since
>> Dataflow Python multi-language pipelines use portable job submission by
Re Cham,
> Looking at https://github.com/apache/beam/pull/12572 seems like you just
> need support for FixedWindows, right ? This should work now I believe since
> Dataflow Python multi-language pipelines use portable job submission by
> default.
The problem part is not the FixedWindows but the
Thanks for bringing this up Sameer.
On Fri, Jan 15, 2021 at 1:18 PM Boyuan Zhang wrote:
> +Chamikara Jayalath
>
> Hi Sameer,
>
> Thanks for reaching out!
>
> We will expose *commitOffsetsInFinalize *to py ReadFromKafka transform
> when we have CustomWindow support in python SDK, which should
Thanks for the quick response on this. Looking forward to these features in
future.
Regarding `TimestampPolicyFactory` I wanted to use the predefined `CreateTime`
policy instead of the default `processingTime`.
Cheers,
-Sameer.
On 2021/01/15 21:18:25, Boyuan Zhang wrote:
> +Chamikara
+Chamikara Jayalath
Hi Sameer,
Thanks for reaching out!
We will expose *commitOffsetsInFinalize *to py ReadFromKafka transform when
we have CustomWindow support in python SDK, which should be coming soon.
In terms of *TimestampPolicyFactory*,if you are using the built-in types,
like
