Hi,
No, it is a Java job.
This is an example code that causes the duplicate side input tag log entries
PCollectionView sideInput = sideCollection.apply(View.asSingleton());
inputCollection
.apply(ParDo.of(new MyFn1()).withSideInputs(sideInput))
.apply(ParDo.of(new MyFn2()).withSideInputs(sideInput));
But if I create two separated views like this then the duplicate side input
tag log entries won’t appear
PCollectionView sideInput1 = sideCollection.apply(View.asSingleton());
PCollectionView sideInput2 = sideCollection.apply(View.asSingleton());
inputCollection
.apply(ParDo.of(new MyFn1()).withSideInputs(sideInput1))
.apply(ParDo.of(new MyFn2()).withSideInputs(sideInput2));
-Binh
On Mon, Nov 7, 2022 at 10:50 AM Reuven Lax via user
wrote:
> Is this a Python job?
>
> On Mon, Nov 7, 2022 at 12:38 AM Binh Nguyen Van
> wrote:
>
>> Hi,
>>
>> I am writing a pipeline where I have one singleton side input that I want
>> to use in multiple different transforms. When I run the pipeline in Google
>> Dataflow I see multiple entries in the logs that have a message like this
>>
>> Deduplicating side input tags, found non-unique side input key
>> org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.:1204#4663620f501c9270
>>
>> Is this something that I should avoid? If so how can I do that?
>>
>> Thanks
>> -Binh
>>
>
