Hi,
No, it is a Java job.
This is an example code that causes the duplicate side input tag log entries
PCollectionView<MyData> sideInput = sideCollection.apply(View.asSingleton());
inputCollection
.apply(ParDo.of(new MyFn1()).withSideInputs(sideInput))
.apply(ParDo.of(new MyFn2()).withSideInputs(sideInput));
But if I create two separated views like this then the duplicate side input
tag log entries won’t appear
PCollectionView<MyData> sideInput1 = sideCollection.apply(View.asSingleton());
PCollectionView<MyData> sideInput2 = sideCollection.apply(View.asSingleton());
inputCollection
.apply(ParDo.of(new MyFn1()).withSideInputs(sideInput1))
.apply(ParDo.of(new MyFn2()).withSideInputs(sideInput2));
-Binh
On Mon, Nov 7, 2022 at 10:50 AM Reuven Lax via user <user@beam.apache.org>
wrote:
> Is this a Python job?
>
> On Mon, Nov 7, 2022 at 12:38 AM Binh Nguyen Van <binhn...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I am writing a pipeline where I have one singleton side input that I want
>> to use in multiple different transforms. When I run the pipeline in Google
>> Dataflow I see multiple entries in the logs that have a message like this
>>
>> Deduplicating side input tags, found non-unique side input key
>> org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1204#4663620f501c9270
>>
>> Is this something that I should avoid? If so how can I do that?
>>
>> Thanks
>> -Binh
>>
>
