[
https://issues.apache.org/jira/browse/BEAM-6386?focusedWorklogId=187057&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-187057
]
ASF GitHub Bot logged work on BEAM-6386:
----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Jan/19 21:53
Start Date: 18/Jan/19 21:53
Worklog Time Spent: 10m
Work Description: jklukas commented on pull request #7437: [BEAM-6386]
Add named variant of PTransform::compose()
URL: https://github.com/apache/beam/pull/7437#discussion_r249196723
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/PTransform.java
##########
@@ -319,4 +319,16 @@ public OutputT expand(InputT input) {
}
};
}
+
+ /** Like {@link #compose(SerializableFunction)}, but with a custom name. */
+ @Experimental
+ public static <InputT extends PInput, OutputT extends POutput>
Review comment:
Your example is spot-on and represents probably the majority use case of
`PTransform.compose`. I agree with you that in cases where you're defining a
composed transform inline as part of a pipeline, passing the name to
`PCollection.apply` is sufficient.
The specific case where I've found the lack of name a nuisance is when I
want to return a transform from a method. Consider the following enum, which we
use as a pipeline option:
```
public enum InputType {
pubsub {
/** Return a PTransform that reads from a Pubsub subscription. */
public PTransform<PBegin, ResultWithErrors<PCollection<PubsubMessage>>>
read(
SinkOptions.Parsed options) {
return PTransform.compose(input -> input
.apply(PubsubIO.readMessagesWithAttributes().fromSubscription(options.getInput()))
.apply(MapElementsWithErrors.ToPubsubMessageFrom.identity()));
}
},
file {
/** Return a PTransform that reads from local or remote files. */
public PTransform<PBegin, ResultWithErrors<PCollection<PubsubMessage>>>
read(
SinkOptions.Parsed options) {
return PTransform.compose(input ->
input.apply(TextIO.read().from(options.getInput()))
.apply(options.getInputFileFormat().decode()));
}
};
public abstract PTransform<PBegin,
ResultWithErrors<PCollection<PubsubMessage>>> read(
SinkOptions.Parsed options);
}
```
My main class then calls:
```
input.apply(options.getInputType().read())
```
I'd like the name of the transform to indicate what the input type was
rather than picking a static name that I pass into `apply`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 187057)
Time Spent: 0.5h (was: 20m)
> Add named variant of PTransform::compose
> ----------------------------------------
>
> Key: BEAM-6386
> URL: https://issues.apache.org/jira/browse/BEAM-6386
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Jeff Klukas
> Assignee: Jeff Klukas
> Priority: Minor
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> BEAM- 5413 introduced PTransform::compose as a concise way of creating a
> composition of transforms as a lambda. We should add a variant to allow
> specifying a name for the returned transform in the same way that {{apply}}
> can take an explicit name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
