[ https://issues.apache.org/jira/browse/BEAM-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16953250#comment-16953250 ]
Ning Kang commented on BEAM-646: -------------------------------- Hi, This is Ning. I'm working a project that supports InteractiveRunner (Python3+) when users use Beam in notebook environment. Would apply() interception in runner still be useful as a hook for non-Beam related but interactivity related features such as collecting IPython/notebook cell information when a PTransfrom is applied? Of course, we can also have all pipelines support interactivity by making the interception inside pipelines themselves. But it's unlikely that all runners would/could take a pipeline with interactivity logic at this moment. And those ipython/notebook dependencies probably shouldn't be introduced into pipeline itself. Would there be any APIs that support invoking arbitrary external logic at different stages of building a pipeline when pipeline is completely decoupled from runner? Thanks! > Get runners out of the apply() > ------------------------------ > > Key: BEAM-646 > URL: https://issues.apache.org/jira/browse/BEAM-646 > Project: Beam > Issue Type: Improvement > Components: beam-model, sdk-java-core > Reporter: Kenneth Knowles > Assignee: Thomas Groh > Priority: Major > Labels: backwards-incompatible > Fix For: 0.6.0 > > > Right now, the runner intercepts calls to apply() and replaces transforms as > we go. This means that there is no "original" user graph. For portability and > misc architectural benefits, we would like to build the original graph first, > and have the runner override later. > Some runners already work in this manner, but we could integrate it more > smoothly, with more validation, via some handy APIs on e.g. the Pipeline > object. -- This message was sent by Atlassian Jira (v8.3.4#803005)