Re: @RequiresStableInput and Pipeline fusion

2022-12-14 Thread Jan Lukavský
Filled https://github.com/apache/beam/issues/24655.  Jan On 12/14/22 00:52, Luke Cwik via dev wrote: This is definitely not working for portable pipelines since the GreedyPipelineFuser doesn't create a fusion boundary which as you pointed out causes a single stage that has a non-deterministic

Re: @RequiresStableInput and Pipeline fusion

2022-12-13 Thread Luke Cwik via dev
This is definitely not working for portable pipelines since the GreedyPipelineFuser doesn't create a fusion boundary which as you pointed out causes a single stage that has a non-deterministic function followed by one that requires stable input. It seems as though we should have runners check the

@RequiresStableInput and Pipeline fusion

2022-12-13 Thread Jan Lukavský
Hi, I have a question about @RequiresStableInput functionality. I'm trying to make it work for portable Flink runner [1], [2]. We have an integration test (which should probably be turned into Validates runner test, but that is a different story) [3]. The test creates random key for input