(due to asynchronous
advancement), so generally we encourage people to reason about timers and
windowing in their code instead.
Reuven
On Sun, Aug 9, 2020 at 9:39 AM jmac...@godaddy.com<mailto:jmac...@godaddy.com>
mailto:jmac...@godaddy.com>> wrote:
I understand that watermarks are con
and
windowing in their code instead.
Reuven
On Sun, Aug 9, 2020 at 9:39 AM jmac...@godaddy.com<mailto:jmac...@godaddy.com>
mailto:jmac...@godaddy.com>> wrote:
I understand that watermarks are concurrently advanced, and that they are
estimates and not precise. but I’m not sure this
Assuming I have understood the use case correctly.
Sorry I won't have time to try it out myself this week, but it's a worthwhile
pattern to explore and publish on the patterns page.
Cheers
Rez
On Mon, 10 Aug 2020, 00:30 jmac...@godaddy.com<mailto:jmac...@godaddy.com>,
mailto:jmac...@god
o Question
Notice: This email is from an external sender.
On Tue, Aug 4, 2020 at 1:08 PM jmac...@godaddy.com<mailto:jmac...@godaddy.com>
mailto:jmac...@godaddy.com>> wrote:
So, after some additional digging, it appears that Beam does not consistently
check for timer expiry before calli
2020 at 14:57, Reuven Lax
mailto:re...@google.com>> wrote:
On Tue, Aug 4, 2020 at 1:08 PM jmac...@godaddy.com<mailto:jmac...@godaddy.com>
mailto:jmac...@godaddy.com>> wrote:
So, after some additional digging, it appears that Beam does not consistently
check for timer e
called. Is there any reason why this isn’t happening? Am I
misunderstanding something?
From: "jmac...@godaddy.com"
Reply-To: "dev@beam.apache.org"
Date: Monday, August 3, 2020 at 10:51 AM
To: "dev@beam.apache.org"
Subject: Re: Stateful Pardo Question
Notice: This em
ardo Question
Notice: This email is from an external sender.
Are you sure that there is a 15 minute gap in your data?
On Mon, Aug 3, 2020 at 6:20 AM jmac...@godaddy.com<mailto:jmac...@godaddy.com>
mailto:jmac...@godaddy.com>> wrote:
I am confused about the behavior of timers on a simple st
I am confused about the behavior of timers on a simple stateful pardo. I have
put together a little repro here: https://github.com/randomsamples/pardo_repro
I basically want to build something like a session window, accumulating events
until quiescence of the stream for a given key and gap
com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileIO.java#L988
On Wed, Jul 29, 2020 at 12:28 PM
jmac...@godaddy.com<mailto:jmac...@godaddy.com>
mailto:jmac...@godaddy.com>> wrote:
We would like to use ParquetIO but limit individual files written out a ma
We would like to use ParquetIO but limit individual files written out a maximum
size. Don’t see any easy way to do this, and it seems like default behavior is
to split based on parallelism? Anyone have any guidance on this?
ed to relying on default names
that match class names.
On Tue, Jan 28, 2020 at 6:41 PM jmac...@godaddy.com<mailto:jmac...@godaddy.com>
mailto:jmac...@godaddy.com>> wrote:
Yeah it just seems like a lot of boiler plate to do builders with lots of
methods just to wrap a MapElements type for syntac
Yeah it just seems like a lot of boiler plate to do builders with lots of
methods just to wrap a MapElements type for syntactic convenience. After
thinking this over last night I’m wondering if it wouldn’t be better to use
static factory methods for these, so rather than the following, which
Hi Beam Community,
Our team has a number of PTransforms that are basically wrappers around
MapElements, which give us a concise syntax when specifying pipelines which
leverage shared map stages. One example that we are looking at currently is a
function which takes JSON and maps it into
13 matches
Mail list logo
