This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push: new 50035da Publishing website 2020/12/11 00:03:01 at commit 0424485 50035da is described below commit 50035da9795c4d36461b80045286affd2ddae185 Author: jenkins <bui...@apache.org> AuthorDate: Fri Dec 11 00:03:01 2020 +0000 Publishing website 2020/12/11 00:03:01 at commit 0424485 --- .../blog/capability-matrix/index.html | 40 ++ .../generated-content/categories/beam/index.xml | 420 +++++++++++++++++++++ .../categories/capability/index.xml | 420 +++++++++++++++++++++ .../extensions/create-external-table/index.html | 4 +- .../runners/capability-matrix/index.html | 258 +++++++++++-- website/generated-content/sitemap.xml | 2 +- 6 files changed, 1104 insertions(+), 40 deletions(-) diff --git a/website/generated-content/blog/capability-matrix/index.html b/website/generated-content/blog/capability-matrix/index.html index 3d2d808..5f07c1b 100644 --- a/website/generated-content/blog/capability-matrix/index.html +++ b/website/generated-content/blog/capability-matrix/index.html @@ -113,6 +113,46 @@ Tyler Akidau [<a href=https://twitter.com/takidau>@takidau</a>]</p></header><div <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-blog-what><th class="cap-summary color-metadata format-category" colspan=8 style=color:#ca1>Bounded Splittable DoFn Support Status</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" style=color:#ec3>Beam Model</th><th class="cap-summary color-platform format-platform" [...] +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Side Inputs</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Splittable DoFn Initiated Checkpointing</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Dynamic Splitting</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Bundle Finalization</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-blog-what><th class="cap-summary color-metadata format-category" colspan=8 style=color:#ca1>Unbounded Splittable DoFn Support Status</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" style=color:#ec3>Beam Model</th><th class="cap-summary color-platform format-platfor [...] +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Side Inputs</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Splittable DoFn Initiated Checkpointing</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Dynamic Splitting</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Bundle Finalization</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr></table></div></p></div></article></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=footer__cols__col><div class=footer__cols__col__logo><img src=/images/beam_logo_circle.svg class=footer__logo alt="Beam logo"></div><div class=footer__cols__col__logo><img src=/images/apache_logo_circle.svg class=footer__logo alt="Apache logo"></div></d [...] <a href=http://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> diff --git a/website/generated-content/categories/beam/index.xml b/website/generated-content/categories/beam/index.xml index e31574a..59cd62a 100644 --- a/website/generated-content/categories/beam/index.xml +++ b/website/generated-content/categories/beam/index.xml @@ -1221,6 +1221,426 @@ How do refinements relate? <tr class='cap-summary'> <td class='cap-summary color-blank cap-blank' colspan='5'></td> </tr> +<tr class='cap-summary' id='cap-blog-what'> +<th class='cap-summary color-metadata format-category' colspan='8' style='color:#ca1'> +Bounded Splittable DoFn Support Status +</th> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability'></th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Beam Model</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Google Cloud Dataflow</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Apache Flink</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Apache Spark</th> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Base</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Side Inputs</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Splittable DoFn Initiated Checkpointing</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Dynamic Splitting</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Bundle Finalization</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<td class='cap-summary color-blank cap-blank' colspan='5'></td> +</tr> +<tr class='cap-summary' id='cap-blog-what'> +<th class='cap-summary color-metadata format-category' colspan='8' style='color:#ca1'> +Unbounded Splittable DoFn Support Status +</th> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability'></th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Beam Model</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Google Cloud Dataflow</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Apache Flink</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Apache Spark</th> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Base</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Side Inputs</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Splittable DoFn Initiated Checkpointing</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Dynamic Splitting</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Bundle Finalization</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<td class='cap-summary color-blank cap-blank' colspan='5'></td> +</tr> </table> </div> </p></description></item><item><title>Blog: Dataflow Python SDK is now public!</title><link>/blog/python-sdk-now-public/</link><pubDate>Thu, 25 Feb 2016 13:00:00 -0800</pubDate><guid>/blog/python-sdk-now-public/</guid><description> diff --git a/website/generated-content/categories/capability/index.xml b/website/generated-content/categories/capability/index.xml index 77fa72a..ba15eec 100644 --- a/website/generated-content/categories/capability/index.xml +++ b/website/generated-content/categories/capability/index.xml @@ -1165,6 +1165,426 @@ How do refinements relate? <tr class='cap-summary'> <td class='cap-summary color-blank cap-blank' colspan='5'></td> </tr> +<tr class='cap-summary' id='cap-blog-what'> +<th class='cap-summary color-metadata format-category' colspan='8' style='color:#ca1'> +Bounded Splittable DoFn Support Status +</th> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability'></th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Beam Model</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Google Cloud Dataflow</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Apache Flink</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Apache Spark</th> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Base</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Side Inputs</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Splittable DoFn Initiated Checkpointing</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Dynamic Splitting</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Bundle Finalization</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<td class='cap-summary color-blank cap-blank' colspan='5'></td> +</tr> +<tr class='cap-summary' id='cap-blog-what'> +<th class='cap-summary color-metadata format-category' colspan='8' style='color:#ca1'> +Unbounded Splittable DoFn Support Status +</th> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability'></th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Beam Model</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Google Cloud Dataflow</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Apache Flink</th> +<th class='cap-summary color-platform format-platform' style='color:#ec3'>Apache Spark</th> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Base</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Side Inputs</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Splittable DoFn Initiated Checkpointing</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Dynamic Splitting</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<th class='cap-summary color-capability format-capability' style='color:#ec3'>Bundle Finalization</th> +<td width='25%' class='cap-summary' style='background-color:#ec3;border-color:#ca1'> +<center> +<b> +&#x2713; +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#fe5;border-color:#ca1'> +<center> +<b> +~ +</b> +<b> +</b> +</center> +</td> +<td width='25%' class='cap-summary' style='background-color:#ddd;border-color:#ca1'> +<center> +<b> +&#x2715; +</b> +<b> +</b> +</center> +</td> +</tr> +<tr class='cap-summary'> +<td class='cap-summary color-blank cap-blank' colspan='5'></td> +</tr> </table> </div> </p></description></item></channel></rss> \ No newline at end of file diff --git a/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html b/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html index 3eb26e0..cf6c730 100644 --- a/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html +++ b/website/generated-content/documentation/dsls/sql/extensions/create-external-table/index.html @@ -68,7 +68,9 @@ with at least one <code>qualifier</code>. Cells are represented as simple types ROW type with a mandatory <code>val</code> field, optional <code>timestampMicros</code> and optional <code>labels</code>. Both read the latest cell in the column. Cells specified as Arrays of simple types (<code>ARRAY<simpleType></code>) allow to read all the column’s values.</p><p>For flat schema only <code>SIMPLE_TYPE</code> values are allowed. Every field except for <code>key</code> must correspond -to the key-values pairs specified in <code>columnsMapping</code>.</p><p>Not all existing column families and qualifiers have to be provided to the schema.</p><h3 id=write-mode-1>Write Mode</h3><p>Supported for flat schema only.</p><h3 id=example-1>Example</h3><pre><code>CREATE EXTERNAL TABLE beamTable( +to the key-values pairs specified in <code>columnsMapping</code>.</p><p>Not all existing column families and qualifiers have to be provided to the schema.</p><p>Filters are only allowed by <code>key</code> field with single <code>LIKE</code> statement with +<a href=https://github.com/google/re2/wiki/Syntax>RE2 Syntax</a> regex, e.g. +<code>SELECT * FROM table WHERE key LIKE '^key[012]{1}'</code></p><h3 id=write-mode-1>Write Mode</h3><p>Supported for flat schema only.</p><h3 id=example-1>Example</h3><pre><code>CREATE EXTERNAL TABLE beamTable( key VARCHAR NOT NULL, beamFamily ROW< boolLatest BOOLEAN NOT NULL, diff --git a/website/generated-content/documentation/runners/capability-matrix/index.html b/website/generated-content/documentation/runners/capability-matrix/index.html index 994e9bc..90bc607 100644 --- a/website/generated-content/documentation/runners/capability-matrix/index.html +++ b/website/generated-content/documentation/runners/capability-matrix/index.html @@ -1,7 +1,7 @@ <!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Apache Beam Capability Matrix</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and [...] <span class=sr-only>Toggle navigation</span> <span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button> -<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...] +<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...] <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> @@ -13,6 +13,8 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>GroupByKey</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> @@ -25,6 +27,8 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Flatten</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> @@ -37,6 +41,8 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Combine</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> @@ -49,6 +55,8 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Composite Transforms</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> @@ -61,6 +69,8 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Side Inputs</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> @@ -73,6 +83,8 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Source API</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> @@ -85,43 +97,177 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> -<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Splittable DoFn (SDF)</th><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> -<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> -<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> -<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> -<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Metrics</th><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> -<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> -<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Metrics</th><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Stateful Processing</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-what><th class="cap-summary color-metadata format-category" colspan=8 style=color:#ca1><div class=cap-toggle onclick='ToggleTables(1,"cap-full-what")'>(click to expand details)</div>Bounded Splittable DoFn Support Status</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform forma [...] +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> -<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Stateful Processing</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Side Inputs</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Splittable DoFn Initiated Checkpointing</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Dynamic Splitting</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Bundle Finalization</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-what><th class="cap-summary color-metadata format-category" colspan=8 style=color:#ca1><div class=cap-toggle onclick='ToggleTables(1,"cap-full-what")'>(click to expand details)</div>Unbounded Splittable DoFn Support Status</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform for [...] +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Side Inputs</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Splittable DoFn Initiated Checkpointing</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Dynamic Splitting</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Bundle Finalization</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> -<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-where><th class="cap-summary color-metadata format-category" colspan=8 style=color:#37d><div class=cap-toggle onclick='ToggleTables(1,"cap-full-where")'>(click to expand details)</div>Where in event time?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" styl [...] +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b> +<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b> +<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-where><th class="cap-summary color-metadata format-category" colspan=8 style=color:#37d><div class=cap-toggle onclick='ToggleTables(1,"cap-full-where")'>(click to expand details)</div>Where in event time?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" styl [...] <b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b> @@ -205,7 +351,7 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b> -<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-when><th class="cap-summary color-metadata format-category" colspan=8 style=color:#6a4><div class=cap-toggle onclick='ToggleTables(1,"cap-full-when")'>(click to expand details)</div>When in processing time?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" st [...] +<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-when><th class="cap-summary color-metadata format-category" colspan=8 style=color:#6a4><div class=cap-toggle onclick='ToggleTables(1,"cap-full-when")'>(click to expand details)</div>When in processing time?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" st [...] <b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b> @@ -301,7 +447,7 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b> -<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-how><th class="cap-summary color-metadata format-category" colspan=8 style=color:#b55><div class=cap-toggle onclick='ToggleTables(1,"cap-full-how")'>(click to expand details)</div>How do refinements relate?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" st [...] +<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-how><th class="cap-summary color-metadata format-category" colspan=8 style=color:#b55><div class=cap-toggle onclick='ToggleTables(1,"cap-full-how")'>(click to expand details)</div>How do refinements relate?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" st [...] <b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b> @@ -337,7 +483,7 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b> -<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-misc><th class="cap-summary color-metadata format-category" colspan=8 style=color:#aaa><div class=cap-toggle onclick='ToggleTables(1,"cap-full-misc")'>(click to expand details)</div>Additional common features not yet part of the Beam model</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary c [...] +<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-misc><th class="cap-summary color-metadata format-category" colspan=8 style=color:#aaa><div class=cap-toggle onclick='ToggleTables(1,"cap-full-misc")'>(click to expand details)</div>Additional common features not yet part of the Beam model</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary c [...] <b></b></center></td><td width=25% class=cap-summary style=background-color:#ccc;border-color:#aaa><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ccc;border-color:#aaa><center><b>~</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b> @@ -361,7 +507,7 @@ <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b> <b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b> -<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr></table></div><div id=cap-full style=display:none><table class=cap><tr class=cap id=cap-full-what><th class="cap color-metadata format-category" colspan=8 style=color:#ca1><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-what")'>(click to collapse details)</div>What is being computed?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color- [...] +<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr></table></div><div id=cap-full style=display:none><table class=cap><tr class=cap id=cap-full-what><th class="cap color-metadata format-category" colspan=8 style=color:#ca1><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-what")'>(click to collapse details)</div>What is being computed?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color- [...] : element-wise processing</b></center><br>Element-wise transformation parameterized by a chunk of user code. Elements are processed in bundles, with initialization and termination hooks. Bundle size is chosen by the runner and cannot be controlled by user code. ParDo processes a main input PCollection one element at a time, but provides side input access to additional PCollections.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br>Batch mode uses large bundle sizes. Streaming uses smaller bundle sizes.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br>ParDo itself, as per-element transformation with UDFs, is fully supported by Flink for both batch and streaming.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes @@ -373,7 +519,7 @@ : fully supported</b></center><br>Supported with per-element transformation.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes -: fully supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>GroupByKey</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>GroupByKey</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : key grouping</b></center><br>Grouping of key-value pairs per key, window, and pane. (See also other tabs.)</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br>Uses Flink's keyBy for key grouping. When grouping by window in streaming (creating the panes) the Flink runner uses the Beam code. This guarantees support for all windowing and triggering mechanisms.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially @@ -385,7 +531,7 @@ : fully supported</b></center><br>Uses Samza's partitionBy for key grouping and Beam's logic for window aggregation and triggering.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes -: fully supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Flatten</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Flatten</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : collection concatenation</b></center><br>Concatenates multiple homogenously typed collections together.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes @@ -397,7 +543,7 @@ : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes -: fully supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Combine</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Combine</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : associative &amp; commutative aggregation</b></center><br>Application of an associative, commutative operation over all values ("globally") or over all values associated with each key ("per key"). Can be implemented using ParDo, but often more efficient implementations exist.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : efficient execution</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br>Uses a combiner for pre-aggregation for batch and streaming.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes @@ -409,7 +555,7 @@ : fully supported</b></center><br>Use combiner for efficient pre-aggregation.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br>Batch mode uses pre-aggregation</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br>Batch mode uses pre-aggregation</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes -: fully supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Composite Transforms</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Composite Transforms</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : user-defined transformation subgraphs</b></center><br>Allows easy extensibility for library writers. In the near future, we expect there to be more information provided at this level -- customized metadata hooks for monitoring, additional runtime/environment hooks, etc.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially : supported via inlining</b></center><br>Currently composite transformations are inlined during execution. The structure is later recreated from the names, but other transform level information (if added to the model) will be lost.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially : supported via inlining</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially @@ -421,7 +567,7 @@ : supported via inlining</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially : supported via inlining</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially -: supported via inlining</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Side Inputs</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +: supported via inlining</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Side Inputs</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : additional elements available during DoFn execution</b></center><br>Side inputs are additional <tt>PCollections</tt> whose contents are computed during pipeline execution and then made accessible to DoFn code. The exact shape of the side input depends both on the <tt>PCollectionView</tt> used to describe the access pattern (interable, map, singleton) and the window of the element from the main input that is currently being processed.</td><td width=25% class=cap style=backgr [...] : some size restrictions in streaming</b></center><br>Batch mode supports a distributed implementation, but streaming mode may force some size restrictions. Neither mode is able to push lookups directly up into key-based sources.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : some size restrictions in streaming</b></center><br>Batch mode supports a distributed implementation, but streaming mode may force some size restrictions. Neither mode is able to push lookups directly up into key-based sources.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes @@ -433,7 +579,7 @@ : fully supported</b></center><br>Uses Samza's broadcast operator to distribute the side inputs.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially : with restrictions</b></center><br>Supported only when the side input source is bounded and windowing uses global window</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes -: fully supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Source API</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Source API</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : user-defined sources</b></center><br>Allows users to provide additional input sources. Supports both bounded and unbounded data. Includes hooks necessary to provide efficient parallelization (size estimation, progress information, dynamic splitting, etc).</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br>Support includes autotuning features (https://cloud.google.com/dataflow/service/dataflow-service-desc#autotuning-features).</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes @@ -445,19 +591,7 @@ : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes -: fully supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Splittable DoFn (SDF)</th><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially -: DoFn where processing of each element can be split for parallelism, or suspended and resumed</b></center><br>Allows users to develop DoFn's that process a single element in portions ("restrictions"), executed in parallel or sequentially. This supersedes the unbounded and bounded `Source` APIs by supporting all of their features on a per-element basis. See http://s.apache.org/splittable-do-fn. Design is in progress on achieving parity with Source API regarding progress signals.</td><td [...] -:</b></center><br>Does not yet support autotuning features of the Source API.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes -:</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially -: supports bounded-per-element SDFs</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No -: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No -: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No -: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No -: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially -: supports bounded-per-element SDFs</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No -: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No -: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially -: supports bounded</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Metrics</th><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Metrics</th><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially : user-provided metrics</b></center><br>Allow transforms to gather simple metrics across bundles in a <tt>PTransform</tt>. Provide a mechanism to obtain both committed and attempted metrics. Semantically similar to using an additional output, but support partial results as the transform executes, and support both committed and attempted values. Will likely want to augment <tt>Metrics</tt> to be more useful for processing unbounded data by making them windowed.</td><td width=2 [...] : All metrics types are supported.</b></center><br>Only attempted values are supported. No committed values for metrics.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially : All metric types are supported.</b></center><br>Only attempted values are supported. No committed values for metrics.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially @@ -468,7 +602,7 @@ : Counter and Gauge are supported.</b></center><br>Only attempted values are supported. No committed values for metrics.</td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No : not implemented</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially : All metrics types supported, both in batching and streaming mode.</b></center><br>Doesn't differentiate between committed and attempted values.</td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No -: not implemented</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Stateful Processing</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Stateful Processing</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes : storage per key, per window</b></center><br>Allows fine-grained access to per-key, per-window persistent state. Necessary for certain use cases (e.g. high-volume windows which store large amounts of data, but typically only access small portions of it; complex state machines; etc.) that are not easily or efficiently addressed via <tt>Combine</tt> or <tt>GroupByKey</tt>+<tt>ParDo</tt>.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b [...] : non-merging windows</b></center><br>State is supported for non-merging windows. SetState and MapState are not yet supported.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially : non-merging windows</b></center><br>State is supported for non-merging windows. SetState and MapState are not yet supported.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially @@ -480,7 +614,55 @@ : non-merging windows</b></center><br>States are backed up by either rocksDb KV store or in-memory hash map, and persist using changelog.</td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No : not implemented</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially : non-merging windows</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No -: not implemented</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-where><th class="cap color-metadata format-category" colspan=8 style=color:#37d><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-where")'>(click to collapse details)</div>Where in event time?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#59f>Beam Model</th><th clas [...] +: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-what><th class="cap color-metadata format-category" colspan=8 style=color:#ca1><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-what [...] +:</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only Dataflow Runner V2 supports this.</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only portable Flink Runner supports this.</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=2 [...] +:</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Side Inputs</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only Dataflow Runner V2 supports this.</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only portable Flink Runner supports this.</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=2 [...] +:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Splittable DoFn Initiated Checkpointing</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only Dataflow Runner v2 supports this.</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only portable Flink Runner supports this.</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=2 [...] +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Dynamic Splitting</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only Dataflow Runner V2 supports this.</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;b [...] +: Only with Python SDK</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Bundle Finalization</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only Dataflow Runner V2 supports this.</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;b [...] +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-what><th class="cap color-metadata format-category" colspan=8 style=color:#ca1><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-what")'>(click to collapse details)</div>Unbounded Splittable DoFn Support Status</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#ec3>Beam Model</th><th cl [...] +:</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;b [...] +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Side Inputs</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only Dataflow Runner V2 supports this.</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only portable Flink Runner supports this.</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=2 [...] +:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Splittable DoFn Initiated Checkpointing</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;b [...] +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Dynamic Splitting</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;b [...] +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Bundle Finalization</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes +:</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only Dataflow Runner V2 supports this.</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially +: Only portable Flink Runner supports this with checkpointing enabled.</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>:</b></ [...] +:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No +:</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-where><th class="cap color-metadata format-category" colspan=8 style=color:#37d><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-where")'>(click to collapse details)</div>Where in event time?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#59f>Beam Model</th><th class="cap color-pla [...] : all time</b></center><br>The default window which covers all of time. (Basically how traditional batch cases fit in the model.)</td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes : default</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes : supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes @@ -564,7 +746,7 @@ : supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes : supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes : supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes -: supported</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-when><th class="cap color-metadata format-category" colspan=8 style=color:#6a4><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-when")'>(click to collapse details)</div>When in processing time?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#8c6>Beam Model</th><th class="c [...] +: supported</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-when><th class="cap color-metadata format-category" colspan=8 style=color:#6a4><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-when")'>(click to collapse details)</div>When in processing time?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#8c6>Beam Model</th><th class="c [...] : user customizable</b></center><br>Triggering may be specified by the user (instead of simply driven by hardcoded defaults).</td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes : fully supported</b></center><br>Fully supported in streaming mode. In batch mode, intermediate trigger firings are effectively meaningless.</td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes @@ -652,7 +834,7 @@ : non-merging windows</b></center><br>The Samza Runner supports timers in non-merging windows.</td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No : not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially : non-merging windows</b></center><br></td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially -:</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-how><th class="cap color-metadata format-category" colspan=8 style=color:#b55><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-how")'>(click to collapse details)</div>How do refinements relate?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#d77>Beam Model</th><th class="cap color-p [...] +:</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-how><th class="cap color-metadata format-category" colspan=8 style=color:#b55><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-how")'>(click to collapse details)</div>How do refinements relate?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#d77>Beam Model</th><th class="cap color-p [...] : panes discard elements when fired</b></center><br>Elements are discarded from accumulated state as their pane is fired.</td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes : fully supported</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes @@ -684,7 +866,7 @@ : pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No : pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No : pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No -: pending model support</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-misc><th class="cap color-metadata format-category" colspan=8 style=color:#aaa><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-misc")'>(click to collapse details)</div>Additional common features not yet part of the Beam model</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" [...] +: pending model support</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-misc><th class="cap color-metadata format-category" colspan=8 style=color:#aaa><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-misc")'>(click to collapse details)</div>Additional common features not yet part of the Beam model</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" [...] :</b></center><br>APIs and semantics for draining a pipeline are under discussion. This would cause incomplete aggregations to be emitted regardless of trigger and tagged with metadata indicating it is incompleted.</td><td width=25% class=cap style=background-color:#ccc;border-color:#aaa><center><b>Partially :</b></center><br>Dataflow has a native drain operation, but it does not work in the presence of event time timer loops. Final implemention pending model support.</td><td width=25% class=cap style=background-color:#ccc;border-color:#aaa><center><b>Partially :</b></center><br>Flink supports taking a "savepoint" of the pipeline and shutting the pipeline down after its completion.</td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;b [...] diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index a92a0bc..7278dbf 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.25.0/</loc><lastmod>2020-10-29T14:08:19-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-10-29T14:08:19-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-10-29T14:08:19-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-10-29T14:08:19-07:00</lastmod></url><url><loc>/blog/b [...] \ No newline at end of file +<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.25.0/</loc><lastmod>2020-10-29T14:08:19-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-10-29T14:08:19-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-10-29T14:08:19-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-10-29T14:08:19-07:00</lastmod></url><url><loc>/blog/b [...] \ No newline at end of file