[
https://issues.apache.org/jira/browse/BEAM-3221?focusedWorklogId=383170&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-383170
]
ASF GitHub Bot logged work on BEAM-3221:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Feb/20 21:33
Start Date: 06/Feb/20 21:33
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #10779: [BEAM-3221]
Clarify documentation for StandardTransforms.Primitives, Pipeline, and
PTransform.
URL: https://github.com/apache/beam/pull/10779#discussion_r376094205
##########
File path: model/pipeline/src/main/proto/beam_runner_api.proto
##########
@@ -78,31 +78,45 @@ message Components {
// A Pipeline is a hierarchical graph of PTransforms, linked
// by PCollections.
//
-// This is represented by a number of by-reference maps to nodes,
-// PCollections, SDK environments, UDF, etc., for
-// supporting compact reuse and arbitrary graph structure.
+// Impulse -> PCollection -> ParDo -> PCollection -> GroupByKey -> ...
+// \> PCollection -> ParDo -> ...
//
-// All of the keys in the maps here are arbitrary strings that are only
-// required to be internally consistent within this proto message.
+// This is represented by a number of by-reference maps to transforms,
+// PCollections, SDK environments, coders, etc., for
+// supporting compact reuse and arbitrary graph structure.
message Pipeline {
// (Required) The coders, UDFs, graph nodes, etc, that make up
// this pipeline.
Components components = 1;
- // (Required) The ids of all PTransforms that are not contained within
another PTransform.
- // These must be in shallow topological order, so that traversing them
recursively
- // in this order yields a recursively topological traversal.
+ // (Required) The ids of all PTransforms that are not contained within
another
+ // PTransform. These must be in shallow topological order, so that traversing
+ // them recursively in this order yields a recursively topological traversal.
repeated string root_transform_ids = 2;
// (Optional) Static display data for the pipeline. If there is none,
// it may be omitted.
DisplayData display_data = 3;
}
-// An applied PTransform! This does not contain the graph data, but only the
-// fields specific to a graph node that is a Runner API transform
-// between PCollections.
+// Transforms are the operations in your pipeline, and provide a generic
+// processing framework. You provide processing logic in the form of a function
+// object (colloquially referred to as “user code”), and your user code is
+// applied to each element of an input PCollection (or more than one
+// PCollection). Depending on the pipeline runner and back-end that you choose,
+// many different workers across a cluster may execute instances of your user
+// code in parallel. The user code running on each worker generates the output
+// elements that are ultimately added to the final output PCollection that the
+// transform produces.
+//
+//The Beam SDKs contain a number of different transforms that you can apply to
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 383170)
Time Spent: 2h 40m (was: 2.5h)
> Model pipeline representation improvements
> ------------------------------------------
>
> Key: BEAM-3221
> URL: https://issues.apache.org/jira/browse/BEAM-3221
> Project: Beam
> Issue Type: Improvement
> Components: beam-model
> Reporter: Henning Rohde
> Priority: Major
> Labels: portability
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Collections of various (breaking) tweaks to the Runner API, notably the
> pipeline representation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
