tions
creates some unnecessary
boilerplate code needed on user
side.
Jan
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>>>>>>>
>>>>>>> Flatten.pCollections().withWindowingStrategy(WindowResolution.into(oneInput.getWindowingStrategy()))
>>>>>>>
>>>>>>> or anything similar. This can be done in user code, so it is not
>>&g
n
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>
> I came across a case where using
>
>>> window (and in fact main inputs are blocked until the side-input window
>>>>>> is
>>>>>> ready), so we must do a mapping. If for example (and very commonly!) the
>>>>>> side input is in the global window and the main input is in a f
rote:
> Hi,
>
> I came across a case where using
>
PCollection#applyWindowingStrategyInternal
seems legit in user core.
> The case is roughly as follows:
t;>> This is a one-sided merge function, there is a 'main' and 'side'
>>>>> input, but the generic symmetric merge might be possible as well. E.g. if
>>>>> one PCollection of Flatten is in GlobalWindow, I wonder if there are cases
>>>>> where user
>>>> example side input elements could always map to the previous fixed window
>>>> (e.g. while processing window 12-1, you want to see summary data of all
>>>> records in the previous window 11-12). Users can do this by providing a
>>>> WindowMappin
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>
> I came across a case where using
> PCollection#applyWindowingStrategyInternal seems
legit in user core.
> The case is roughly as follows:
>
the View - essentially a function from window to
>>> window. Unfortunately this is hard to use (one must create their own
>>> PCollectionView class) and very poorly documented, so I doubt many users
>>> know about this!
>>>
>>> Reuven
>
creates some unnecessary boilerplate code needed on user
side.
Jan
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>
> I came across a case where using
> PCollection#applyWindowingStrategyInternal
!
>>
>> Reuven
>>
>> On Sat, Apr 6, 2024 at 7:09 AM Jan Lukavský wrote:
>>
>>> Immediate self-correction, although setting the strategy directly via
>>> setWindowingStrategyInternal() *seemed* to be working during Pipeline
>>> construction tim
ilerplate code needed on user side.
Jan
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>
> I came across a case where using
> PCollection#applyWindowingStrategyInternal seems legit in
user core.
> The case is roughly as f
ns with incompatible windowFns more user-friendly. The current
>> approach where we require the same windowFn for all input PCollections
>> creates some unnecessary boilerplate code needed on user side.
>>
>> Jan
>>
>> On 4/6/24 15:45, Jan Lukavský wrote:
>
re the same windowFn for all input
PCollections
creates some unnecessary boilerplate code needed on user side.
Jan
On 4/6/24 15:45, Jan Lukavský wrote:
> Hi,
>
> I came across a case where using
> PCollection#applyWindowingStrategyInternal seems legit
ollections with incompatible windowFns more user-friendly. The current
> approach where we require the same windowFn for all input PCollections
> creates some unnecessary boilerplate code needed on user side.
>
> Jan
>
> On 4/6/24 15:45, Jan Lukavský wrote:
> >
Lukavský wrote:
Hi,
I came across a case where using
PCollection#applyWindowingStrategyInternal seems legit in user core.
The case is roughly as follows:
a) compute some streaming statistics
b) apply the same transform (say ComputeWindowedAggregation) with
different parameters
Hi,
I came across a case where using
PCollection#applyWindowingStrategyInternal seems legit in user core. The
case is roughly as follows:
a) compute some streaming statistics
b) apply the same transform (say ComputeWindowedAggregation) with
different parameters on these statistics
17 matches
Mail list logo
