This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit 0b495f8c508389fed8a9f467104bc03b15d84b83
Author: melissa <meliss...@google.com>
AuthorDate: Mon Sep 25 13:52:15 2017 -0700
Add new execution model page and associated images
---
src/_includes/header.html | 1 +
src/documentation/execution-model.md | 163 +++++++++++++++++++++
src/documentation/index.md | 9 +-
src/images/execution_model_bundling.svg | 33 +++++
.../execution_model_bundling_coupled_failure.svg | 38 +++++
src/images/execution_model_bundling_gantt.svg | 33 +++++
src/images/execution_model_bundling_gantt_max.svg | 45 ++++++
src/images/execution_model_bundling_multi.svg | 35 +++++
.../execution_model_bundling_multi_gantt.svg | 34 +++++
src/images/execution_model_failure_retry.svg | 36 +++++
10 files changed, 423 insertions(+), 4 deletions(-)
diff --git a/src/_includes/header.html b/src/_includes/header.html
index 839b4ea..1981fb1 100644
--- a/src/_includes/header.html
+++ b/src/_includes/header.html
@@ -35,6 +35,7 @@
<li role="separator" class="divider"></li>
<li class="dropdown-header">Beam Concepts</li>
<li><a href="{{ site.baseurl
}}/documentation/programming-guide/">Programming Guide</a></li>
+ <li><a href="{{ site.baseurl
}}/documentation/execution-model/">Execution Model</a></li>
<li><a href="{{ site.baseurl
}}/documentation/resources/">Additional Resources</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">Pipeline Fundamentals</li>
diff --git a/src/documentation/execution-model.md
b/src/documentation/execution-model.md
new file mode 100644
index 0000000..a8ebee0
--- /dev/null
+++ b/src/documentation/execution-model.md
@@ -0,0 +1,163 @@
+---
+layout: default
+title: "Beam Execution Model"
+permalink: /documentation/execution-model/
+---
+
+# Apache Beam Execution Model
+
+The Beam model allows runners to execute your pipeline in different ways.
+Depending on the runner’s choices, you may observe various effects as a result.
+This page describes the effects of these choices so you can better understand
+how Beam pipelines execute.
+
+**Table of Contents:**
+* TOC
+{:toc}
+
+## Processing of elements
+
+The serialization and communication of elements between machines is one of the
+most expensive operations in a distributed execution of your pipeline. Avoiding
+this serialization may require re-processing elements after failures or may
+limit the distribution of output to other machines.
+
+The runner processes elements on many machines and may serialize elements in
+between machines for other communication and persistence reasons.
+
+### Serialization and communication
+
+The runner may serialize elements for communication or persistence.
+
+The runner may decide to transfer elements between transforms in a variety of
+ways, such as:
+
+1. Routing elements to a worker for processing as part of a grouping
operation.
+ This may involve serializing elements and sorting them by their key.
+1. Redistributing elements between workers to adjust parallelism. This may
+ involve serializing elements and communicating them to other workers.
+1. Using the elements in a side input to a `ParDo`. This may require
+ serializing the elements and broadcasting them to all the workers executing
+ the `ParDo`.
+1. Passing elements between transforms that are running on the same worker.
+ This may avoid having to serialize the elements, and instead just passing
+ them in memory.
+
+Additionally, the runner may serialize and persist elements for other reasons:
+
+1. When used as part of a Stateful `DoFn`, the runner may persist values to
some
+ state mechanism.
+1. When committing the results of processing, the runner may persist the
outputs
+ as a checkpoint.
+
+### Bundling and persistence
+
+Beam pipelines often focus on ["embarassingly
parallel"](https://en.wikipedia.org/wiki/Embarrassingly_parallel)
+problems. Because of this, the APIs emphasize processing elements in parallel,
+which makes it difficult to express things like "assign a sequence number to
+each element in a PCollection." This is intentional since such algorithms are
+much more likely to suffer from scalability problems.
+
+Processing all elements in parallel also has some drawbacks. Specifically, it
+makes it impossible to batch any operations, such as writing elements to a sink
+or checkpointing progress during processing.
+
+Instead of processing all elements simultaneously, the elements in a
+`PCollection` are processed in _bundles_. The division of the collection into
+bundles is arbitrary and selected by the runner. This allows the runner to
+choose an appropriate middle-ground between persisting results after every
+element, and having to retry everything if there is a failure.
+
+## Failures and parallelism within and between transforms
+
+In this section, we discuss how elements in the input collection are processed
+in parallel, and how transforms are retried when failures occur.
+
+### Data-parallelism within one transform {#data-parallelism}
+
+The bundling of elements when executing a single `ParDo` may look something
like
+this. In this diagram, a collection with 9 elements is divided into 2
+bundles:
+
+
+
+When the `ParDo` executes, these bundles may be processed in parallel worker
+threads. The elements in each bundle are processed in sequence, as shown in
this
+diagram:
+
+
+
+Since elements cannot be split, the maximum parallelism for a transform depends
+on the number of elements in the collection. In this case, the maximum
+parallelism is 9:
+
+
+
+Note: Splittable ParDo allows splitting the processing of a single input across
+multiple bundles. This feature is still a work in progress, but may already be
+useful in some cases.
+
+### Dependent-parallelism between transforms {#dependent-parallellism}
+
+When two transforms are connected as shown below, the runner may choose to
+execute them in a way such that the bundling of the two transforms are
+dependent.
+
+
+
+In this picture, `ParDo1` and `ParDo2` are _dependently parallel_ if the output
+of `ParDo1` for a given element must be processed on the same worker thread.
+
+
+
+For example, two `ParDo` transforms in sequence may be _dependently parallel_
if
+the runner chooses to execute the consuming transform on each output from the
+producing transform without altering the bundling in between.
+
+Executing transforms this way allows a runner to avoid redistributing elements
+between workers, saving on communication costs. However, the maximum
parallelism
+now depends on the maximum parallelism of the first of the dependently parallel
+steps.
+
+### Failures within one transform
+
+If processing of an element within a bundle fails, the entire bundle fails. The
+elements in the bundle must be retried (otherwise the entire pipeline fails),
+although they need not be retried with the same bundling.
+
+In the following illustration, two elements (in green) were successfully
+processed. The third element’s processing failed, and there are three elements
+(in yellow) still to be processed. We see that the same elements were retried
+and processing successfully completed. Note that as shown, the retry does not
+necessarily happen in the same worker thread as the original attempt.
+
+Because we encountered a failure while processing an element in the input
+bundle, we had to reprocess all of the elements in the input bundle. Thus, the
+entire output bundle must be thrown away since all of the results it contains
+will be recomputed.
+
+
+
+If the failed transform is a `ParDo`, then the `DoFn` instance is torn down and
+abandoned.
+
+### Coupled failure: Failures between transforms
+
+If a failure to process an element in `ParDo2` causes `ParDo1` to re-execute,
+these two steps are said to be _co-failing_.
+
+In the following illustration, two `ParDo` transforms are processing elements.
+While processing an element, the second `ParDo` fails (which is shown in red).
+As a result, the runner must discard and recompute the output of the second
+`ParDo`. Because the runner was executing the two `ParDo`s together, the output
+bundle from the first `ParDo` must also be thrown away, and the elements in the
+input bundle must be retried. These two `ParDo`s are co-failing.
+
+
+
+All `DoFns` that experience coupled failures are terminated and must be torn
+down since they aren’t following the normal `DoFn` lifecycle .
+
+Executing transforms this way allows a runner to avoid persisting elements
+between transforms, saving on persistence costs.
+
diff --git a/src/documentation/index.md b/src/documentation/index.md
index d9cb4ac..40ed972 100644
--- a/src/documentation/index.md
+++ b/src/documentation/index.md
@@ -11,12 +11,13 @@ redirect_from:
This section provides in-depth conceptual information and reference material
for the Beam Model, SDKs, and Runners:
-## Concepts
+## Concepts
Learn about the Beam Programming Model and the concepts common to all Beam
SDKs and Runners.
* The [Programming Guide]({{ site.baseurl }}/documentation/programming-guide/)
introduces all the key Beam concepts.
-* Visit [Additional Resources]({{ site.baseurl }}/documentation/resources/)
for some of our favorite articles and talks about Beam.
+* Learn about Beam's [execution model]({{ site.baseurl
}}/documentation/execution-model/) to better understand how pipelines execute.
+* Visit [Additional Resources]({{ site.baseurl }}/documentation/resources/)
for some of our favorite articles and talks about Beam.
## Pipeline Fundamentals
@@ -28,7 +29,7 @@ Learn about the Beam Programming Model and the concepts
common to all Beam SDKs
Find status and reference information on all of the available Beam SDKs.
-* [Java SDK]({{ site.baseurl }}/documentation/sdks/java/)
+* [Java SDK]({{ site.baseurl }}/documentation/sdks/java/)
* [Python SDK]({{ site.baseurl }}/documentation/sdks/python/)
## Runners
@@ -42,7 +43,7 @@ A Beam Runner runs a Beam pipeline on a specific (often
distributed) data proces
* [FlinkRunner]({{ site.baseurl }}/documentation/runners/flink/): Runs on
[Apache Flink](http://flink.apache.org).
* [SparkRunner]({{ site.baseurl }}/documentation/runners/spark/): Runs on
[Apache Spark](http://spark.apache.org).
* [DataflowRunner]({{ site.baseurl }}/documentation/runners/dataflow/): Runs
on [Google Cloud Dataflow](https://cloud.google.com/dataflow), a fully managed
service within [Google Cloud Platform](https://cloud.google.com/).
-* [GearpumpRunner]({{ site.baseurl }}/documentation/runners/gearpump/): Runs
on [Apache Gearpump (incubating)](http://gearpump.apache.org).
+* [GearpumpRunner]({{ site.baseurl }}/documentation/runners/gearpump/): Runs
on [Apache Gearpump (incubating)](http://gearpump.apache.org).
### Choosing a Runner
diff --git a/src/images/execution_model_bundling.svg
b/src/images/execution_model_bundling.svg
new file mode 100644
index 0000000..7a257e3
--- /dev/null
+++ b/src/images/execution_model_bundling.svg
@@ -0,0 +1,33 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<svg id="svg2" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
xmlns="http://www.w3.org/2000/svg"; height="144.38" width="419.8" version="1.1"
xmlns:cc="http://creativecommons.org/ns#";
xmlns:dc="http://purl.org/dc/elements/1.1/";>
+ <metadata id="metadata7">
+ <rdf:RDF>
+ <cc:Work rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+ <dc:title/>
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g id="layer1" transform="translate(25.429,-781.45258)">
+ <g id="g3135" transform="translate(8.1428566,0)">
+ <rect id="rect2987" ry="11.782" height="34.286" width="197.14" y="876.55"
x="26.429" fill="#ff7f2a"/>
+ <text id="text2991" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-family="Sans" xml:space="preserve" font-size="40px"
y="901.14917" x="88.930664" font-weight="normal" font-style="normal"
fill="#000000"><tspan id="tspan2993" y="901.14917" x="88.930664"
font-size="20px">ParDo1</tspan></text>
+ </g>
+ <rect id="rect3025" height="36.429" width="237.14" y="815.51" x="14.571"
fill="#afc6e9"/>
+ <path id="path3029"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(-14,812.36218)" fill="#59F"/>
+ <path id="path3029-9"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(94.85714,828.79075)" fill="#59F"/>
+ <path id="path3029-1"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(7.42857,826.64789)" fill="#59F"/>
+ <path id="path3029-2"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(40.285713,808.79075)" fill="#59F"/>
+ <path id="path3029-7"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(40.285713,824.50504)" fill="#59F"/>
+ <path id="path3029-0"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(75.285713,815.93361)" fill="#59F"/>
+ <path id="path3029-93"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(20.999998,814.50504)" fill="#59F"/>
+ <path id="path3029-6"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(108.14286,813.79075)" fill="#59F"/>
+ <path id="path3029-06"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(136,821.64789)" fill="#59F"/>
+ <text id="text3153" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-family="Sans" xml:space="preserve" font-size="20px"
y="841.17542" x="257.14285" font-weight="normal" font-style="normal"
fill="#000000"><tspan id="tspan3155" y="841.17542"
x="257.14285">Collection</tspan></text>
+ <rect id="rect3161" height="51.429" width="5" y="808.01" x="110.71"
fill="#1c2422"/>
+ <text id="text3163" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-family="Sans" xml:space="preserve" font-size="20px"
y="811.64789" x="15.000003" font-weight="normal" font-style="normal"
fill="#000000"><tspan id="tspan3165" y="811.64789" x="15.000003">Bundle
1</tspan></text>
+ <text id="text3163-1" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-family="Sans" xml:space="preserve" font-size="20px"
y="811.64789" x="128.29799" font-weight="normal" font-style="normal"
fill="#000000"><tspan id="tspan3165-8" y="811.64789" x="128.29799">Bundle
2</tspan></text>
+ </g>
+</svg>
diff --git a/src/images/execution_model_bundling_coupled_failure.svg
b/src/images/execution_model_bundling_coupled_failure.svg
new file mode 100644
index 0000000..3c5b015
--- /dev/null
+++ b/src/images/execution_model_bundling_coupled_failure.svg
@@ -0,0 +1,38 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<svg id="svg2" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
xmlns="http://www.w3.org/2000/svg"; height="303.81" width="589.09" version="1.1"
xmlns:cc="http://creativecommons.org/ns#";
xmlns:dc="http://purl.org/dc/elements/1.1/";>
+ <metadata id="metadata7">
+ <rdf:RDF>
+ <cc:Work rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+ <dc:title/>
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g id="layer1" transform="translate(20.046875,-752.34875)">
+ <path id="path3207" stroke-linejoin="miter" d="m50,805.93,0,196.43h436.43"
stroke="#000" stroke-linecap="butt" stroke-width="1px" fill="none"/>
+ <text id="text3209" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" xml:space="preserve" font-size="16px" font-style="normal"
y="779.505" x="20" font-family="Sans" line-height="125%" fill="#000000"><tspan
id="tspan3211" y="779.505" x="20">Worker</tspan><tspan id="tspan3219"
y="799.505" x="20">Thread</tspan></text>
+ <text id="text3213" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" xml:space="preserve" font-size="16px" font-style="normal"
y="1020.9336" x="445" font-family="Sans" line-height="125%"
fill="#000000"><tspan id="tspan3215" y="1020.9336"
x="445">Processing</tspan><tspan id="tspan3217" y="1040.9336"
x="445">Time</tspan></text>
+ <g id="g3140-54"
transform="matrix(2.3376624,0,0,1.2745098,24.367395,-85.011242)">
+ <rect id="rect3025-7-1-9-8-74" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-54-9"
transform="matrix(2.3376624,0,0,1.2745098,152.93886,-85.011242)" fill="#AFA">
+ <rect id="rect3025-7-1-9-8-74-5" height="36.429" width="55" y="806.93"
x="14.143" fill="#AFA"/>
+ </g>
+ <g id="g3140-54-1"
transform="matrix(1.6103896,0,0,1.2745098,37.081674,-143.43981)">
+ <rect id="rect3025-7-1-9-8-74-7" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-54-9-6"
transform="matrix(2.5324676,0,0,1.2745098,112.4695,-143.43981)" fill="#AFA">
+ <rect id="rect3025-7-1-9-8-74-5-1" height="36.429" width="55" y="806.93"
x="14.143" fill="#AFA"/>
+ </g>
+ <path id="path3029-6"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(226.3817,907.58872)" fill="#ff2a2a"/>
+ <g id="g3140-54-1-9"
transform="matrix(2.6753248,0,0,1.2745098,262.54504,-141.92755)">
+ <rect id="rect3025-7-1-9-8-74-7-4" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-54-9-6-8"
transform="matrix(1.5194805,0,0,1.2745098,423.03483,-141.92755)" fill="#AFA">
+ <rect id="rect3025-7-1-9-8-74-5-1-1" height="36.429" width="55" y="806.93"
x="14.143" fill="#AFA"/>
+ </g>
+ <flowRoot id="flowRoot5183" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-weight="normal" xml:space="preserve" font-size="20px"
transform="translate(203.62946,797.53031)" font-style="normal"
font-family="Sans" fill="#000000"><flowRegion id="flowRegion5185"><rect
id="rect5187" y="94.526" width="125.71" x="192.86"
height="30.714"/></flowRegion><flowPara id="flowPara5189"
font-size="24px">Retry</flowPara></flowRoot>
+ </g>
+</svg>
diff --git a/src/images/execution_model_bundling_gantt.svg
b/src/images/execution_model_bundling_gantt.svg
new file mode 100644
index 0000000..963ac6d
--- /dev/null
+++ b/src/images/execution_model_bundling_gantt.svg
@@ -0,0 +1,33 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<svg id="svg2" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
xmlns="http://www.w3.org/2000/svg"; height="303.81" width="589.09" version="1.1"
xmlns:cc="http://creativecommons.org/ns#";
xmlns:dc="http://purl.org/dc/elements/1.1/";>
+ <metadata id="metadata7">
+ <rdf:RDF>
+ <cc:Work rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+ <dc:title/>
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g id="layer1" transform="translate(20.046875,-752.34875)">
+ <path id="path3207" stroke-linejoin="miter" d="m50,805.93,0,196.43h436.43"
stroke="#000" stroke-linecap="butt" stroke-width="1px" fill="none"/>
+ <text id="text3209" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-family="Sans" xml:space="preserve" font-size="16px"
font-style="normal" y="779.505" x="20" font-weight="normal"
fill="#000000"><tspan id="tspan3211" y="779.505" x="20">Worker</tspan><tspan
id="tspan3219" y="799.505" x="20">Thread</tspan></text>
+ <text id="text3213" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-family="Sans" xml:space="preserve" font-size="16px"
font-style="normal" y="1020.9336" x="445" font-weight="normal"
fill="#000000"><tspan id="tspan3215" y="1020.9336"
x="445">Processing</tspan><tspan id="tspan3217" y="1040.9336"
x="445">Time</tspan></text>
+ <g id="g3140" transform="translate(43.285717,0.78568637)">
+ <rect id="rect3025-7-19" height="36.429" width="55" y="900.51" x="100.59"
fill="#afc6e9"/>
+ <rect id="rect3025-7-1-4" height="36.429" width="55" y="900.51" x="160.11"
fill="#afc6e9"/>
+ <rect id="rect3025-7-1-2-7" height="36.429" width="55" y="900.51"
x="219.62" fill="#afc6e9"/>
+ <rect id="rect3025-7-1-9-8" height="36.429" width="55" y="900.51"
x="279.14" fill="#afc6e9"/>
+ <rect id="rect3025-7" height="36.429" width="55" y="956.93" x="22.143"
fill="#afc6e9"/>
+ <rect id="rect3025-7-1" height="36.429" width="55" y="956.93" x="82.571"
fill="#afc6e9"/>
+ <rect id="rect3025-7-1-2" height="36.429" width="55" y="956.93" x="143"
fill="#afc6e9"/>
+ <rect id="rect3025-7-1-9" height="36.429" width="55" y="956.93" x="203.43"
fill="#afc6e9"/>
+ <text id="text3163" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-family="Sans" xml:space="preserve" font-size="20px"
font-style="normal" y="926.17542" x="375.81967" font-weight="normal"
fill="#000000"><tspan id="tspan3165" y="926.17542" x="375.81967">Bundle
1</tspan></text>
+ <text id="text3163-1" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-family="Sans" xml:space="preserve" font-size="20px"
font-style="normal" y="982.604" x="260.8978" font-weight="normal"
fill="#000000"><tspan id="tspan3165-8" y="982.604" x="260.8978">Bundle
2</tspan></text>
+ <text id="text3163-1-3" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-family="Sans" xml:space="preserve" font-size="20px"
font-style="normal" y="982.29639" x="100.64494" font-weight="normal"
fill="#000000"><tspan id="tspan3165-8-6" y="982.29639"
x="100.64494">ParDo1</tspan></text>
+ <text id="text3435" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-family="Sans" xml:space="preserve" font-size="20px"
font-style="normal" y="925.8678" x="164.21637" font-weight="normal"
fill="#000000"><tspan id="tspan3437" y="925.8678"
x="164.21637">ParDo1</tspan></text>
+ <rect id="rect3025-7-19-4" height="36.429" width="55" y="900.51" x="41.071"
fill="#afc6e9"/>
+ </g>
+ </g>
+</svg>
diff --git a/src/images/execution_model_bundling_gantt_max.svg
b/src/images/execution_model_bundling_gantt_max.svg
new file mode 100644
index 0000000..661eaaa
--- /dev/null
+++ b/src/images/execution_model_bundling_gantt_max.svg
@@ -0,0 +1,45 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<svg id="svg2" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
xmlns="http://www.w3.org/2000/svg"; height="303.81" width="589.09" version="1.1"
xmlns:cc="http://creativecommons.org/ns#";
xmlns:dc="http://purl.org/dc/elements/1.1/";>
+ <metadata id="metadata7">
+ <rdf:RDF>
+ <cc:Work rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+ <dc:title/>
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g id="layer1" transform="translate(20.046875,-752.34875)">
+ <path id="path3207" stroke-linejoin="miter" d="m50,805.93,0,196.43h436.43"
stroke="#000" stroke-linecap="butt" stroke-width="1px" fill="none"/>
+ <text id="text3209" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" xml:space="preserve" font-size="16px" font-style="normal"
y="779.505" x="20" font-family="Sans" line-height="125%" fill="#000000"><tspan
id="tspan3211" y="779.505" x="20">Worker</tspan><tspan id="tspan3219"
y="799.505" x="20">Thread</tspan></text>
+ <text id="text3213" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" xml:space="preserve" font-size="16px" font-style="normal"
y="1020.9336" x="445" font-family="Sans" line-height="125%"
fill="#000000"><tspan id="tspan3215" y="1020.9336"
x="445">Processing</tspan><tspan id="tspan3217" y="1040.9336"
x="445">Time</tspan></text>
+ <g id="g3140" transform="matrix(1,0,0,0.31372546,43.285717,554.56371)">
+ <rect id="rect3025-7-1-9-8" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-0" transform="matrix(1,0,0,0.31372546,43.285717,597.33157)">
+ <rect id="rect3025-7-1-9-8-6" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-1" transform="matrix(1,0,0,0.31372546,43.285717,682.86727)">
+ <rect id="rect3025-7-1-9-8-5" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-5" transform="matrix(1,0,0,0.31372546,53.285717,618.71549)">
+ <rect id="rect3025-7-1-9-8-4" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-7" transform="matrix(1,0,0,0.31372546,55.285717,704.2512)">
+ <rect id="rect3025-7-1-9-8-65" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-6" transform="matrix(1,0,0,0.31372546,51.285717,575.94764)">
+ <rect id="rect3025-7-1-9-8-9" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-3" transform="matrix(1,0,0,0.31372546,43.285717,640.09942)">
+ <rect id="rect3025-7-1-9-8-7" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-4" transform="matrix(1,0,0,0.31372546,85.285717,661.48335)">
+ <rect id="rect3025-7-1-9-8-52" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <g id="g3140-54" transform="matrix(1,0,0,0.31372546,43.285717,725.63512)">
+ <rect id="rect3025-7-1-9-8-74" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ </g>
+</svg>
diff --git a/src/images/execution_model_bundling_multi.svg
b/src/images/execution_model_bundling_multi.svg
new file mode 100644
index 0000000..8d0d09c
--- /dev/null
+++ b/src/images/execution_model_bundling_multi.svg
@@ -0,0 +1,35 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<svg id="svg2" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
xmlns="http://www.w3.org/2000/svg"; height="256.02" width="317.14" version="1.1"
xmlns:cc="http://creativecommons.org/ns#";
xmlns:dc="http://purl.org/dc/elements/1.1/";>
+ <metadata id="metadata7">
+ <rdf:RDF>
+ <cc:Work rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+ <dc:title/>
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g id="layer1" transform="translate(24.327568,-613.00507)">
+ <g id="g3135" transform="translate(9.24386,-180)">
+ <rect id="rect2987" ry="11.782" height="34.286" width="197.14" y="876.55"
x="26.429" fill="#ff7f2a"/>
+ <text id="text2991" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-weight="normal" font-size="40px" font-style="normal"
y="901.14917" x="88.930664" font-family="Sans" xml:space="preserve"
fill="#000000"><tspan id="tspan2993" font-size="20px" y="901.14917"
x="88.930664">ParDo1</tspan></text>
+ </g>
+ <g id="g3995" transform="translate(1.1010034,0)">
+ <rect id="rect3025" height="36.429" width="237.14" y="635.51" x="14.571"
fill="#afc6e9"/>
+ <rect id="rect3161" height="51.429" width="5" y="628.01" x="110.71"
fill="#1c2422"/>
+ <text id="text3163" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" font-size="20px" line-height="125%" xml:space="preserve"
font-style="normal" y="661.17542" x="20.000004" font-family="Sans"
fill="#000000"><tspan id="tspan3165" y="661.17542" x="20.000004">Bundle
1</tspan></text>
+ <text id="text3163-1" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" font-size="20px" line-height="125%" xml:space="preserve"
font-style="normal" y="661.17542" x="136.15514" font-family="Sans"
fill="#000000"><tspan id="tspan3165-8" y="661.17542" x="136.15514">Bundle
2</tspan></text>
+ </g>
+ <g id="g3135-4" transform="translate(9.24386,-56.808306)">
+ <rect id="rect2987-3" ry="11.782" height="34.286" width="197.14" y="876.55"
x="26.429" fill="#ff7f2a"/>
+ <text id="text2991-0" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-weight="normal" font-size="40px" font-style="normal"
y="901.14917" x="88.930664" font-family="Sans" xml:space="preserve"
fill="#000000"><tspan id="tspan2993-7" font-size="20px" y="901.14917"
x="88.930664">ParDo2</tspan></text>
+ </g>
+ <g id="g4003" transform="translate(1.5014038e-5,0)">
+ <rect id="rect3025-8" height="36.429" width="237.14" y="758.7" x="15.672"
fill="#afc6e9"/>
+ <rect id="rect3161-9" height="51.429" width="5" y="751.2" x="155.82"
fill="#1c2422"/>
+ <text id="text3163-2" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" font-size="20px" line-height="125%" xml:space="preserve"
font-style="normal" y="784.36713" x="41.8153" font-family="Sans"
fill="#000000"><tspan id="tspan3165-6" y="784.36713" x="41.8153">Bundle
3</tspan></text>
+ <text id="text3163-1-6" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" font-size="20px" line-height="125%" xml:space="preserve"
font-style="normal" y="784.36713" x="160.82756" font-family="Sans"
fill="#000000"><tspan id="tspan3165-8-4" y="784.36713" x="160.82756">Bundle
4</tspan></text>
+ </g>
+ </g>
+</svg>
diff --git a/src/images/execution_model_bundling_multi_gantt.svg
b/src/images/execution_model_bundling_multi_gantt.svg
new file mode 100644
index 0000000..689fcbc
--- /dev/null
+++ b/src/images/execution_model_bundling_multi_gantt.svg
@@ -0,0 +1,34 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<svg id="svg2" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
xmlns="http://www.w3.org/2000/svg"; height="303.81" width="589.09" version="1.1"
xmlns:cc="http://creativecommons.org/ns#";
xmlns:dc="http://purl.org/dc/elements/1.1/";>
+ <metadata id="metadata7">
+ <rdf:RDF>
+ <cc:Work rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+ <dc:title/>
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g id="layer1" transform="translate(20.046875,-752.34875)">
+ <path id="path3207" stroke-linejoin="miter" d="m50,805.93,0,196.43h436.43"
stroke="#000" stroke-linecap="butt" stroke-width="1px" fill="none"/>
+ <text id="text3209" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" xml:space="preserve" font-size="16px" font-style="normal"
y="779.505" x="20" font-family="Sans" line-height="125%" fill="#000000"><tspan
id="tspan3211" y="779.505" x="20">Worker</tspan><tspan id="tspan3219"
y="799.505" x="20">Thread</tspan></text>
+ <text id="text3213" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" xml:space="preserve" font-size="16px" font-style="normal"
y="1020.9336" x="445" font-family="Sans" line-height="125%"
fill="#000000"><tspan id="tspan3215" y="1020.9336"
x="445">Processing</tspan><tspan id="tspan3217" y="1040.9336"
x="445">Time</tspan></text>
+ <g id="g3140-54"
transform="matrix(2.3376624,0,0,1.2745098,24.367395,-85.011242)">
+ <rect id="rect3025-7-1-9-8-74" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <flowRoot id="flowRoot4030" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-weight="normal" xml:space="preserve" font-size="20px"
transform="translate(20.644914,751.7051)" font-style="normal"
font-family="Sans" fill="#000000"><flowRegion id="flowRegion4032"><rect
id="rect4034" y="202.14" width="107.14" x="65"
height="24.286"/></flowRegion><flowPara
id="flowPara4036">ParDo1</flowPara></flowRoot>
+ <g id="g3140-54-9"
transform="matrix(2.3376624,0,0,1.2745098,152.93886,-85.011242)" fill="#AFA">
+ <rect id="rect3025-7-1-9-8-74-5" height="36.429" width="55" y="806.93"
x="14.143" fill="#AFA"/>
+ </g>
+ <flowRoot id="flowRoot4030-0" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-weight="normal" xml:space="preserve" font-size="20px"
transform="translate(149.21638,751.7051)" font-style="normal"
font-family="Sans" fill="#000000"><flowRegion id="flowRegion4032-4"><rect
id="rect4034-8" y="202.14" width="107.14" x="65"
height="24.286"/></flowRegion><flowPara
id="flowPara4036-7">ParDo2</flowPara></flowRoot>
+ <g id="g3140-54-1"
transform="matrix(3.2727274,0,0,1.2745098,13.571526,-143.43981)">
+ <rect id="rect3025-7-1-9-8-74-7" height="36.429" width="55" y="806.93"
x="14.143" fill="#afc6e9"/>
+ </g>
+ <flowRoot id="flowRoot4030-2" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-weight="normal" xml:space="preserve" font-size="20px"
transform="translate(48.787789,693.27658)" font-style="normal"
font-family="Sans" fill="#000000"><flowRegion id="flowRegion4032-7"><rect
id="rect4034-2" y="202.14" width="107.14" x="65"
height="24.286"/></flowRegion><flowPara
id="flowPara4036-2">ParDo1</flowPara></flowRoot>
+ <g id="g3140-54-9-6"
transform="matrix(2.5324676,0,0,1.2745098,203.89807,-143.43981)" fill="#AFA">
+ <rect id="rect3025-7-1-9-8-74-5-1" height="36.429" width="55" y="806.93"
x="14.143" fill="#AFA"/>
+ </g>
+ <flowRoot id="flowRoot4030-0-0" style="letter-spacing:0px;word-spacing:0px;"
line-height="125%" font-weight="normal" xml:space="preserve" font-size="20px"
transform="translate(208.36595,693.27658)" font-style="normal"
font-family="Sans" fill="#000000"><flowRegion id="flowRegion4032-4-6"><rect
id="rect4034-8-1" y="202.14" width="107.14" x="65"
height="24.286"/></flowRegion><flowPara
id="flowPara4036-7-5">ParDo2</flowPara></flowRoot>
+ </g>
+</svg>
diff --git a/src/images/execution_model_failure_retry.svg
b/src/images/execution_model_failure_retry.svg
new file mode 100644
index 0000000..bc4c097
--- /dev/null
+++ b/src/images/execution_model_failure_retry.svg
@@ -0,0 +1,36 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<svg id="svg2" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
xmlns="http://www.w3.org/2000/svg"; height="303.81" width="589.09" version="1.1"
xmlns:cc="http://creativecommons.org/ns#";
xmlns:dc="http://purl.org/dc/elements/1.1/";>
+ <metadata id="metadata7">
+ <rdf:RDF>
+ <cc:Work rdf:about="">
+ <dc:format>image/svg+xml</dc:format>
+ <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+ <dc:title/>
+ </cc:Work>
+ </rdf:RDF>
+ </metadata>
+ <g id="layer1" transform="translate(20.046875,-752.34875)">
+ <path id="path3207" stroke-linejoin="miter" d="m50,805.93,0,196.43h436.43"
stroke="#000" stroke-linecap="butt" stroke-width="1px" fill="none"/>
+ <text id="text3209" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" xml:space="preserve" font-size="16px" font-style="normal"
y="779.505" x="20" font-family="Sans" line-height="125%" fill="#000000"><tspan
id="tspan3211" y="779.505" x="20">Worker</tspan><tspan id="tspan3219"
y="799.505" x="20">Thread</tspan></text>
+ <text id="text3213" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" xml:space="preserve" font-size="16px" font-style="normal"
y="1020.9336" x="445" font-family="Sans" line-height="125%"
fill="#000000"><tspan id="tspan3215" y="1020.9336"
x="445">Processing</tspan><tspan id="tspan3217" y="1040.9336"
x="445">Time</tspan></text>
+ <rect id="rect3025-7-19-4" height="36.429" width="114.29" y="900.8"
x="80.357" fill="#afc6e9"/>
+ <path id="path3029-9"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(48.453122,912.45817)" fill="#4A0"/>
+ <path id="path3029-0"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(37.453124,899.60097)" fill="#4A0"/>
+ <path id="path3029-6"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(70.310271,897.45817)" fill="#ff2a2a"/>
+ <path id="path3029-06"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(98.167409,905.31527)" fill="#ffe680"/>
+ <path id="path3029-06-4"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(131.38169,914.95814)" fill="#ffe680"/>
+ <rect id="rect3025-7-19-4-9" height="36.429" width="114.29" y="850.8"
x="57.81" fill="#afc6e9"/>
+ <path id="path3029-06-65"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(113.52455,899.01729)" fill="#ffe680"/>
+ <g id="g5265">
+ <rect id="rect3025-7-19-4-1" height="36.429" width="114.29" y="850.8"
x="215.67" fill="#afc6e9"/>
+ <path id="path3029-9-5"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(192.33482,862.45818)" fill="#4A0"/>
+ <path id="path3029-0-97"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(169.90625,858.17241)" fill="#4A0"/>
+ <path id="path3029-6-7"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(211.62054,853.45818)" fill="#4A0"/>
+ <path id="path3029-06-6"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(233.47768,855.31528)" fill="#4A0"/>
+ <path id="path3029-06-6-3"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(261.38169,864.0173)" fill="#4A0"/>
+ <flowRoot id="flowRoot5183" style="letter-spacing:0px;word-spacing:0px;"
font-weight="normal" xml:space="preserve"
transform="translate(47.200894,756.81605)" line-height="125%" font-size="20px"
font-style="normal" font-family="Sans" fill="#000000"><flowRegion
id="flowRegion5185"><rect id="rect5187" y="94.526" x="192.86" width="125.71"
height="30.714"/></flowRegion><flowPara id="flowPara5189"
font-size="24px">Retry</flowPara></flowRoot>
+ <path id="path3029-9-5-3"
d="m57.857,14.286c0,2.3669-1.9188,4.2857-4.2857,4.2857s-4.2857-1.9188-4.2857-4.2857c0.001-2.367,1.919-4.286,4.286-4.286,2.3669,0,4.2857,1.9188,4.2857,4.2857z"
transform="translate(183.52455,849.0173)" fill="#4A0"/>
+ </g>
+ </g>
+</svg>
--
To stop receiving notification emails like this one, please contact
"commits@beam.apache.org" <commits@beam.apache.org>.
