[ https://issues.apache.org/jira/browse/BEAM-6858?focusedWorklogId=290596&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-290596 ]
ASF GitHub Bot logged work on BEAM-6858: ---------------------------------------- Author: ASF GitHub Bot Created on: 07/Aug/19 16:46 Start Date: 07/Aug/19 16:46 Worklog Time Spent: 10m Work Description: salmanVD commented on pull request #9275: [BEAM-6858] Support side inputs injected into a DoFn URL: https://github.com/apache/beam/pull/9275#discussion_r311655427 ########## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ParDo.java ########## @@ -652,6 +650,17 @@ public static DoFnSchemaInformation getDoFnSchemaInformation( return withSideInputs(Arrays.asList(sideInputs)); } + /** + * Returns a new {@link ParDo} {@link PTransform} that's like this {@link PTransform} but with + * the specified additional side inputs. Does not modify this {@link PTransform}. + * + * <p>See the discussion of Side Inputs above for more explanation. + */ + public SingleOutput<InputT, OutputT> withSideInput(String tagId, PCollectionView<?> sideInput) { + sideInput.setTagInternalId(tagId); Review comment: I tried to use SerializableUtils.clone to copy but since `PCollection`, `PValueBase` does not implement serializable we cannot clone it with this method. Cloning it with this method makes `PCollection`, `Pipeline` and couple of other fields as `null`. I tried to implement setter but for `Pipeline` is final in `PValueBase` which affects a lot of other classes as well. So, cloning the object and setting just the PCollection in that cloned object works but during the execution of pipeline it fails the condition in `SideInputContainer.createReaderForViews` method with an error `Can't create a SideInputReader with unknown views`. I think we need to set all the objects which are getting null after cloning the object. Do you have any other suggestion? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 290596) Time Spent: 1h 20m (was: 1h 10m) > Support side inputs injected into a DoFn > ---------------------------------------- > > Key: BEAM-6858 > URL: https://issues.apache.org/jira/browse/BEAM-6858 > Project: Beam > Issue Type: Bug > Components: sdk-java-core > Reporter: Reuven Lax > Assignee: Shehzaad Nakhoda > Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Beam currently supports injecting main inputs into a DoFn process method. A > user can write the following: > @ProcessElement public void process(@Element InputT element) > And Beam will (using ByteBuddy code generation) inject the input element into > the process method. > We would like to also support the same for side inputs. For example: > @ProcessElement public void process(@Element InputT element, > @SideInput("tag1") String input1, @SideInput("tag2") Integer input2) > This requires the existing process-method analysis framework to capture these > side inputs. The ParDo code would have to verify the type of the side input > and include them in the list of side inputs. This would also eliminate the > need for the user to explicitly call withSideInputs on the ParDo. > -- This message was sent by Atlassian JIRA (v7.6.14#76016)