[
https://issues.apache.org/jira/browse/BEAM-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043527#comment-16043527
]
ASF GitHub Bot commented on BEAM-1630:
--------------------------------------
GitHub user chamikaramj opened a pull request:
https://github.com/apache/beam/pull/3333
[BEAM-1630] Adds ability to dynamically replace PTransforms during runtime.
Adds two new interfaces, PTransformMatcher and PTransformOverride.
Currently only supports replacements where input and output types are an
exact match (we have to address complexities due to type hints before
supporting replacements with different types).
This can be used to dynamically update a populated pipeline at runtime.
Each runner can configure it's own overrides.
This will be used by SplittableDoFn where matching ParDo transforms will be
dynamically replaced by SplittableParDo.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chamikaramj/beam
sdf_direct_runner_ptransform_override
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3333.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3333
----
----
> Add Splittable DoFn to Python SDK
> ---------------------------------
>
> Key: BEAM-1630
> URL: https://issues.apache.org/jira/browse/BEAM-1630
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py
> Reporter: Chamikara Jayalath
> Assignee: Chamikara Jayalath
>
> Splittable DoFn [1] is currently being implemented for Java SDK [2]. We
> should add this to Python SDK as well.
> Following document proposes an API for this.
> https://docs.google.com/document/d/1h_zprJrOilivK2xfvl4L42vaX4DMYGfH1YDmi-s_ozM/edit?usp=sharing
> [1] https://s.apache.org/splittable-do-fn
> [2]
> https://lists.apache.org/thread.html/0ce61ac162460a149d5c93cdface37cc383f8030fe86ca09e5699b18@%3Cdev.beam.apache.org%3E
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
