Hi,
I came across a case where using
PCollection#applyWindowingStrategyInternal seems legit in user core. The
case is roughly as follows:
a) compute some streaming statistics
b) apply the same transform (say ComputeWindowedAggregation) with
different parameters on these statistics
Lukavský wrote:
Hi,
I came across a case where using
PCollection#applyWindowingStrategyInternal seems legit in user core.
The case is roughly as follows:
a) compute some streaming statistics
b) apply the same transform (say ComputeWindowedAggregation) with
different parameters on these
t; PCollections with incompatible windowFns more user-friendly. The current
> approach where we require the same windowFn for all input PCollections
> creates some unnecessary boilerplate code needed on user side.
>
> Jan
>
> On 4/6/24 15:45, Jan Lukavský wrote:
>
current
approach where we require the same windowFn for all input
PCollections
creates some unnecessary boilerplate code needed on user side.
Jan
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>
> I came across a case where using
> PCollection#applyWindow
if we can make flattening
>> PCollections with incompatible windowFns more user-friendly. The current
>> approach where we require the same windowFn for all input PCollections
>> creates some unnecessary boilerplate code needed on user side.
>>
>> Jan
>>
>> On
input
PCollections
creates some unnecessary boilerplate code needed on user side.
Jan
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>
> I came across a case where using
> PCollection#applyWindowingStrategyInter
and very poorly documented, so I doubt many users
>> know about this!
>>
>> Reuven
>>
>> On Sat, Apr 6, 2024 at 7:09 AM Jan Lukavský wrote:
>>
>>> Immediate self-correction, although setting the strategy directly via
>>> setWindowingStrategyInte
pproach where we require the same windowFn for all
input PCollections
creates some unnecessary boilerplate code needed on user
side.
Jan
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>
> I cam
previous window 11-12). Users can do this by providing a
>>> WindowMappingFunction to the View - essentially a function from window to
>>> window. Unfortunately this is hard to use (one must create their own
>>> PCollectionView class) and very poorly docume
reates some unnecessary boilerplate code needed on
user side.
Jan
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>
> I came across a case where using
> PCollection#applyWindowingStrategyInternal seems
>>>>
>>>>
>>>> In Side input we also allow the user to control this mapping, so for
>>>> example side input elements could always map to the previous fixed window
>>>> (e.g. while processing window 12-1, you want to see summary data of all
>
the global-window elements into the main-input's
>>>>> fixed window.
>>>>>
>>>>> This is a one-sided merge function, there is a 'main' and 'side'
>>>>> input, but the generic symmetric merge might be possible as w
te code
needed on user side.
Jan
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>
> I came across a case where using
>
PCollection#applyWindow
t; However
>>>>>> main input elements are supposed to see side input elements in the same
>>>>>> window (and in fact main inputs are blocked until the side-input window
>>>>>> is
>>>>>> ready), so we must do a mapping. If for e
ections
creates some unnecessary boilerplate
code needed on user side.
Jan
On 4/6/24 15:45, Jan Lukavský wrote:
;>> construction
>>>>>>> time. Alternatively only as a small syntactic sugar, something like:
>>>>>>>
>>>>>>>
>>>>>>> Flatten.pCollections().withWindowingStrategy(WindowResolution.into(oneInput.getWindo
The current
approach where we require the
same windowFn for all input
PCollections
creates some unnecessary
boilerplate code needed on user
17 matches
Mail list logo
