This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push: new 6284fadbd58 Publishing website 2022/07/20 04:18:18 at commit ba62704 6284fadbd58 is described below commit 6284fadbd58d9569ad6f0bc70b79e1d2a44a754a Author: jenkins <bui...@apache.org> AuthorDate: Wed Jul 20 04:18:18 2022 +0000 Publishing website 2022/07/20 04:18:18 at commit ba62704 --- website/generated-content/documentation/index.xml | 20 ++++++++++++++------ .../documentation/programming-guide/index.html | 20 ++++++++++++++------ 2 files changed, 28 insertions(+), 12 deletions(-) diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml index 5c1862f31ac..b9582243350 100644 --- a/website/generated-content/documentation/index.xml +++ b/website/generated-content/documentation/index.xml @@ -5800,8 +5800,12 @@ data contains names and phone numbers.</p> <span class="p">{</span> <span class="nx">name</span><span class="o">:</span> <span class="s2">&#34;amy&#34;</span><span class="p">,</span> <span class="nx">phone</span><span class="o">:</span> <span class="s2">&#34;333-444-5555&#34;</span> <span class="p">},</span> <span class="p">{</span> <span class="nx">name</span><span class="o">:</span> <span class="s2">&#34;carl&#34;</span><span class="p">,</span> <span class="nx">phone</span><span class="o">:</span> <span class="s2">&#34;444-555-6666&#34;</span> <span class="p">},</span> <span class="p">];</span> -<span class="kr">const</span> <span class="nx">emails</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">(</span><span class="nx">emails_list</span><span class="p">));</span> -<span class="kr">const</span> <span class="nx">phones</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">(</span><span class="nx">phones_list</span><span class="p">));</span> +<span class="kr">const</span> <span class="nx">emails</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span> +<span class="nx">beam</span><span class="p">.</span><span class="nx">withName</span><span class="p">(</span><span class="s2">&#34;createEmails&#34;</span><span class="p">,</span> <span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">(</span><span class="nx">emails_list</span><span class="p">))</span> +<span class="p">);</span> +<span class="kr">const</span> <span class="nx">phones</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span> +<span class="nx">beam</span><span class="p">.</span><span class="nx">withName</span><span class="p">(</span><span class="s2">&#34;createPhones&#34;</span><span class="p">,</span> <span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">(</span><span class="nx">phones_list</span><span class="p">))</span> +<span class="p">);</span> </code></pre></div> </div> </div> @@ -6653,8 +6657,12 @@ is a Beam transform for <code>PCollection</code> objects that store the sa <div class="highlight"><pre class="chroma"><code class="language-typescript" data-lang="typescript"><span class="c1">// Flatten takem an array of PCollection objects, wrapped in beam.P(...) </span><span class="c1">// Returns a single PCollection that contains a union of all of the elements in all input PCollections. </span><span class="c1"></span> -<span class="kr">const</span> <span class="nx">fib</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <spa [...] -<span class="kr">const</span> <span class="nx">pow</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <spa [...] +<span class="kr">const</span> <span class="nx">fib</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span> +<span class="nx">beam</span><span class="p">.</span><span class="nx">withName</span><span class="p">(</span><span class="s2">&#34;createFib&#34;</span><span class="p">,</span> <span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="m [...] +<span class="p">);</span> +<span class="kr">const</span> <span class="nx">pow</span> <span class="o">=</span> <span class="nx">root</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span> +<span class="nx">beam</span><span class="p">.</span><span class="nx">withName</span><span class="p">(</span><span class="s2">&#34;createPow&#34;</span><span class="p">,</span> <span class="nx">beam</span><span class="p">.</span><span class="nx">create</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="m [...] +<span class="p">);</span> <span class="kr">const</span> <span class="nx">result</span> <span class="o">=</span> <span class="nx">beam</span><span class="p">.</span><span class="nx">P</span><span class="p">([</span><span class="nx">fib</span><span class="p">,</span> <span class="nx">pow</span><span class="p">]).</span><span class="nx">apply</span><span class="p">(</span><span class="nx">beam</span><span class="p">.</span>< [...] </code></pre></div> </div> @@ -7437,7 +7445,7 @@ function will be invoked multiple time for the element, once for each window.< <a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard"> <img src="/images/copy-icon.svg"/> </a> -<div class="highlight"><pre class="chroma"><code class="language-typescript" data-lang="typescript"><span class="nx">pcoll</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">processFn</span><span class="p">,</span> <span class="p">{</span> <span class="nx">timestamp</span>: <span class="kt">pardo.windowParam</span><span class="p">()</span> <span class="p">});</span> +<div class="highlight"><pre class="chroma"><code class="language-typescript" data-lang="typescript"><span class="nx">pcoll</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">processWithWindow</span><span class="p">,</span> <span class="p">{</span> <span class="nx">timestamp</span>: <span class="kt">pardo.windowParam</span><span class="p">()</span> <span class="p">});</span> </code></pre></div> </div> </div> @@ -7507,7 +7515,7 @@ and how many times this window has already fired for this key.</p> <a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard"> <img src="/images/copy-icon.svg"/> </a> -<div class="highlight"><pre class="chroma"><code class="language-typescript" data-lang="typescript"><span class="nx">pcoll</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">processFn</span><span class="p">,</span> <span class="p">{</span> <span class="nx">timestamp</span>: <span class="kt">pardo.paneInfoParam</span><span class="p">()</span> <span class="p">});</span> +<div class="highlight"><pre class="chroma"><code class="language-typescript" data-lang="typescript"><span class="nx">pcoll</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">processWithPaneInfo</span><span class="p">,</span> <span class="p">{</span> <span class="nx">timestamp</span>: <span class="kt">pardo.paneInfoParam</span><span class="p">()</span> <span class="p">}); [...] </code></pre></div> </div> </div> diff --git a/website/generated-content/documentation/programming-guide/index.html b/website/generated-content/documentation/programming-guide/index.html index 00769b0874d..c755000bbda 100644 --- a/website/generated-content/documentation/programming-guide/index.html +++ b/website/generated-content/documentation/programming-guide/index.html @@ -909,8 +909,12 @@ data contains names and phone numbers.</p><div class="language-java snippet"><di <span class=p>{</span> <span class=nx>name</span><span class=o>:</span> <span class=s2>"carl"</span><span class=p>,</span> <span class=nx>phone</span><span class=o>:</span> <span class=s2>"444-555-6666"</span> <span class=p>},</span> <span class=p>];</span> -<span class=kr>const</span> <span class=nx>emails</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span><span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>(</span><span class=nx>emails_list</span><span class=p>));</span> -<span class=kr>const</span> <span class=nx>phones</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span><span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>(</span><span class=nx>phones_list</span><span class=p>));</span> +<span class=kr>const</span> <span class=nx>emails</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span> + <span class=nx>beam</span><span class=p>.</span><span class=nx>withName</span><span class=p>(</span><span class=s2>"createEmails"</span><span class=p>,</span> <span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>(</span><span class=nx>emails_list</span><span class=p>))</span> +<span class=p>);</span> +<span class=kr>const</span> <span class=nx>phones</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span> + <span class=nx>beam</span><span class=p>.</span><span class=nx>withName</span><span class=p>(</span><span class=s2>"createPhones"</span><span class=p>,</span> <span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>(</span><span class=nx>phones_list</span><span class=p>))</span> +<span class=p>);</span> </code></pre></div></div></div><p>After <code>CoGroupByKey</code>, the resulting data contains all data associated with each unique key from any of the input collections.</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=kd>final</span> <span class=n>TupleTag</span><span class=o><</span><span class=n>String</span><span class=o>></span> <span class=n>emai [...] <span class=kd>final</span> <span class=n>TupleTag</span><span class=o><</span><span class=n>String</span><span class=o>></span> <span class=n>phonesTag</span> <span class=o>=</span> <span class=k>new</span> <span class=n>TupleTag</span><span class=o><>();</span> @@ -1457,8 +1461,12 @@ is a Beam transform for <code>PCollection</code> objects that store the same dat <span class=nx>merged</span> <span class=o>:=</span> <span class=nx>beam</span><span class=p>.</span><span class=nf>Flatten</span><span class=p>(</span><span class=nx>s</span><span class=p>,</span> <span class=nx>pcol1</span><span class=p>,</span> <span class=nx>pcol2</span><span class=p>,</span> <span class=nx>pcol3</span><span class=p>)</span></code></pre></div></div></div><div class="language-typescript snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs [...] </span><span class=c1>// Returns a single PCollection that contains a union of all of the elements in all input PCollections. </span><span class=c1></span> -<span class=kr>const</span> <span class=nx>fib</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span><span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>([</span><span class=mi>1</span><span class=p>,</span> <span class=mi>1</span><span class=p>,</span> <span class=mi>2</span><span class=p>,</span> <span class=mi>3</span><span class=p>,</span> <span class=mi>5</span><span class= [...] -<span class=kr>const</span> <span class=nx>pow</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span><span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>([</span><span class=mi>1</span><span class=p>,</span> <span class=mi>2</span><span class=p>,</span> <span class=mi>4</span><span class=p>,</span> <span class=mi>8</span><span class=p>,</span> <span class=mi>16</span><span class [...] +<span class=kr>const</span> <span class=nx>fib</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span> + <span class=nx>beam</span><span class=p>.</span><span class=nx>withName</span><span class=p>(</span><span class=s2>"createFib"</span><span class=p>,</span> <span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>([</span><span class=mi>1</span><span class=p>,</span> <span class=mi>1</span><span class=p>,</span> <span class=mi>2</span><span class=p>,</span> <span class=mi>3</span><span class=p>,</span> <span class=mi>5</span><span class=p>,</spa [...] +<span class=p>);</span> +<span class=kr>const</span> <span class=nx>pow</span> <span class=o>=</span> <span class=nx>root</span><span class=p>.</span><span class=nx>apply</span><span class=p>(</span> + <span class=nx>beam</span><span class=p>.</span><span class=nx>withName</span><span class=p>(</span><span class=s2>"createPow"</span><span class=p>,</span> <span class=nx>beam</span><span class=p>.</span><span class=nx>create</span><span class=p>([</span><span class=mi>1</span><span class=p>,</span> <span class=mi>2</span><span class=p>,</span> <span class=mi>4</span><span class=p>,</span> <span class=mi>8</span><span class=p>,</span> <span class=mi>16</span><span class=p>,</sp [...] +<span class=p>);</span> <span class=kr>const</span> <span class=nx>result</span> <span class=o>=</span> <span class=nx>beam</span><span class=p>.</span><span class=nx>P</span><span class=p>([</span><span class=nx>fib</span><span class=p>,</span> <span class=nx>pow</span><span class=p>]).</span><span class=nx>apply</span><span class=p>(</span><span class=nx>beam</span><span class=p>.</span><span class=nx>flatten</span><span class=p>());</span> </code></pre></div></div></div><h5 id=data-encoding-merged-collections>4.2.5.1. Data encoding in merged collections</h5><p>By default, the coder for the output <code>PCollection</code> is the same as the coder for the first <code>PCollection</code> in the input <code>PCollectionList</code>. However, the input @@ -2016,7 +2024,7 @@ function will be invoked multiple time for the element, once for each window.</p <span class=k>pass</span></code></pre></div></div></div><div class="language-go snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-go data-lang=go><span class=kd>func</span> <span class=nf>MyDoFn</span><span class=p>(</span><span class=nx>w</span> <span class=nx>beam</span><span class=p>.</spa [...] <span class=nx>iw</span> <span class=o>:=</span> <span class=nx>w</span><span class=p>.(</span><span class=nx>window</span><span class=p>.</span><span class=nx>IntervalWindow</span><span class=p>)</span> <span class=o>...</span> -<span class=p>}</span></code></pre></div></div></div><div class="language-typescript snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-typescript data-lang=typescript><span class=nx>pcoll</span><span class=p>.</span><span class=nx>map</span><span class=p>(</span><span class=nx>processFn</span><spa [...] +<span class=p>}</span></code></pre></div></div></div><div class="language-typescript snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-typescript data-lang=typescript><span class=nx>pcoll</span><span class=p>.</span><span class=nx>map</span><span class=p>(</span><span class=nx>processWithWindow</s [...] </code></pre></div></div></div><p class=language-java><strong>PaneInfo:</strong> When triggers are used, Beam provides a <code>PaneInfo</code> object that contains information about the current firing. Using <code>PaneInfo</code> you can determine whether this is an early or a late firing, and how many times this window has already fired for this key.</p><p class=language-py><strong>PaneInfo:</strong> @@ -2053,7 +2061,7 @@ and how many times this window has already fired for this key.</p><div class="la <span class=k>for</span> <span class=nx>_</span><span class=p>,</span> <span class=nx>w</span> <span class=o>:=</span> <span class=k>range</span> <span class=nx>words</span> <span class=p>{</span> <span class=nf>emitWords</span><span class=p>(</span><span class=nx>w</span><span class=p>)</span> <span class=p>}</span> -<span class=p>}</span></code></pre></div></div></div><div class="language-typescript snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-typescript data-lang=typescript><span class=nx>pcoll</span><span class=p>.</span><span class=nx>map</span><span class=p>(</span><span class=nx>processFn</span><spa [...] +<span class=p>}</span></code></pre></div></div></div><div class="language-typescript snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-typescript data-lang=typescript><span class=nx>pcoll</span><span class=p>.</span><span class=nx>map</span><span class=p>(</span><span class=nx>processWithPaneInfo< [...] </code></pre></div></div></div><p class=language-java><strong>PipelineOptions:</strong> The <code>PipelineOptions</code> for the current pipeline can always be accessed in a process method by adding it as a parameter:</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=k>new</span> <span class=n>DoFn</span><span class=o><</span><span class=n>String</span><span class=o [...]