This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push: new 23bf5d9 Publishing website 2020/06/02 00:02:56 at commit 89fc35b 23bf5d9 is described below commit 23bf5d98b21094c1b3f1432269d0ece83b7f292c Author: jenkins <bui...@apache.org> AuthorDate: Tue Jun 2 00:02:56 2020 +0000 Publishing website 2020/06/02 00:02:56 at commit 89fc35b --- .../documentation/dsls/sql/overview/index.html | 3 +-- website/generated-content/documentation/index.xml | 14 +++++++------- .../documentation/programming-guide/index.html | 12 ++++++------ website/generated-content/sitemap.xml | 2 +- 4 files changed, 15 insertions(+), 16 deletions(-) diff --git a/website/generated-content/documentation/dsls/sql/overview/index.html b/website/generated-content/documentation/dsls/sql/overview/index.html index 9ad4338..797a7f1 100644 --- a/website/generated-content/documentation/dsls/sql/overview/index.html +++ b/website/generated-content/documentation/dsls/sql/overview/index.html @@ -6,8 +6,7 @@ bounded and unbounded <code>PCollections</code> with SQL statements. Your SQL qu is translated to a <code>PTransform</code>, an encapsulated segment of a Beam pipeline. You can freely mix SQL <code>PTransforms</code> and other <code>PTransforms</code> in your pipeline.</p><p>Beam SQL includes the following dialects:</p><ul><li><a href=https://calcite.apache.org>Beam Calcite SQL</a></li><li><a href=https://github.com/google/zetasql>Beam ZetaSQL</a></li></ul><p>Beam Calcite SQL is a variant of Apache Calcite, a dialect widespread in big data processing. Beam Calcite SQL is the default Beam SQL dialect. Beam ZetaSQL is more compatible with BigQuery, so it’s especially useful in pipelines that <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html>write to or read from BigQuery tables</a>.</p><p>To change dialects, pass <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/extensions/sql/package-summary.html>the dialect’s full pac [...] -the type of elements that Beam SQL operates on. A <code>PCollection<Row></code> plays the role of a table. -2.21.0</li></ul><h2 id=walkthrough>Walkthrough</h2><p>The <a href=/documentation/dsls/sql/walkthrough>SQL pipeline walkthrough</a> works through how to use Beam SQL with example code.</p><h2 id=shell>Shell</h2><p>The Beam SQL shell allows you to write pipelines as SQL queries without using the Java SDK. +the type of elements that Beam SQL operates on. A <code>PCollection<Row></code> plays the role of a table.</li></ul><h2 id=walkthrough>Walkthrough</h2><p>The <a href=/documentation/dsls/sql/walkthrough>SQL pipeline walkthrough</a> works through how to use Beam SQL with example code.</p><h2 id=shell>Shell</h2><p>The Beam SQL shell allows you to write pipelines as SQL queries without using the Java SDK. The <a href=/documentation/dsls/sql/shell>Shell page</a> describes how to work with the interactive Beam SQL shell.</p><h2 id=apache-calcite-dialect>Apache Calcite dialect</h2><p>The <a href=/documentation/dsls/sql/calcite/overview>Beam Calcite SQL overview</a> summarizes Apache Calcite operators, functions, syntax, and data types supported by Beam Calcite SQL.</p><h2 id=zetasql-dialect>ZetaSQL dialect</h2><p>For more information on the ZetaSQL features in Beam SQL, see the <a href=/documentation/dsls/sql/zetasql/overview>Beam ZetaSQL dialect reference</a>.</p><p>To switch to Beam ZetaSQL, configure the <a href=https://beam.apache.org/releases/javadoc/2.15.0/org/apache/beam/sdk/options/PipelineOptions.html>pipeline options</a> as follows:</p><pre><code>setPlannerName("org.apa [...] </code></pre><h2 id=beam-sql-extensions>Beam SQL extensions</h2><p>Beam SQL has additional extensions leveraging Beam’s unified batch/streaming model and processing complex data types. You can use these extensions with all Beam SQL dialects.</p></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=footer__cols__col><div class=footer__cols__col__logo><img src=/images/beam_logo_circle.svg class=footer__logo alt="Beam logo"></div><div class=footer_ [...] diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml index 52a2b12..873fd6b 100644 --- a/website/generated-content/documentation/index.xml +++ b/website/generated-content/documentation/index.xml @@ -3885,7 +3885,7 @@ one to easily project out only the fields of interest. The resulting <code>PC field as a top-level field. Both top-level and nested fields can be selected. For example, in the Purchase schema, one could select only the userId and streetAddress fields as follows</p> <div class=language-java> -<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="n">purchases</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Select</span><span class="o">.</span><span class="na">fieldNames</span><span class="o">(</span><span class="s">&#34;userId&#34;</span><span class="o">,</span> <span class="n">shippingAddress</span [...] +<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="n">purchases</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Select</span><span class="o">.</span><span class="na">fieldNames</span><span class="o">(</span><span class="s">&#34;userId&#34;</span><span class="o">,</span> <span class="s">&#34;shippingAddress [...] </div> <p>The resulting <code>PCollection</code> will have the following schema</p> <table> @@ -3909,7 +3909,7 @@ could select only the userId and streetAddress fields as follows</p> <br/> <p>The same is true for wildcard selections. The following</p> <div class=language-java> -<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="n">purchases</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Select</span><span class="o">.</span><span class="na">fieldNames</span><span class="o">(</span><span class="s">&#34;userId&#34;</span><span class="o">,</span> <span class="n">shippingAddress</span [...] +<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="n">purchases</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Select</span><span class="o">.</span><span class="na">fieldNames</span><span class="o">(</span><span class="s">&#34;userId&#34;</span><span class="o">,</span> <span class="s">&#34;shippingAddress [...] </div> <p>Will result in the following schema</p> <table> @@ -3951,7 +3951,7 @@ could select only the userId and streetAddress fields as follows</p> top-level field in the resulting row. This means that if multiple fields are selected from the same nested row, each selected field will appear as its own array field. For example</p> <div class=language-java> -<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="n">purchases</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Select</span><span class="o">.</span><span class="na">fieldNames</span><span class="o">(</span> <span class="s">&#34;transactions.bank&#34;</span><span class="o">,</span> <span class="n">transactions [...] +<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="n">purchases</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Select</span><span class="o">.</span><span class="na">fieldNames</span><span class="o">(</span> <span class="s">&#34;transactions.bank&#34;</span><span class="o">,</span> <span class="s">&#34;tra [...] </div> <p>Will result in the following schema</p> <table> @@ -4043,7 +4043,7 @@ has a schema with one field corresponding to each aggregation performed.</p> <p>The simplest usage of <code>Group</code> specifies no aggregations, in which case all inputs matching the provided set of fields are grouped together into an <code>ITERABLE</code> field. For example</p> <div class=language-java> -<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="n">purchases</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Group</span><span class="o">.</span><span class="na">byFieldNames</span><span class="o">(</span><span class="s">&#34;userId&#34;</span><span class="o">,</span> <span class="n">shippingAddress</spa [...] +<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="n">purchases</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Group</span><span class="o">.</span><span class="na">byFieldNames</span><span class="o">(</span><span class="s">&#34;userId&#34;</span><span class="o">,</span> <span class="s">&#34;shippingAddres [...] </div> <p>The output schema of this is:</p> <table> @@ -4069,9 +4069,9 @@ are grouped together into an <code>ITERABLE</code> field. For example</ <p>The names of the key and values fields in the output schema can be controlled using this withKeyField and withValueField builders, as follows:</p> <div class=language-java> -<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="n">purchases</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Group</span><span class="o">.</span><span class="na">byFieldNames</span><span class="o">(</span><span class="s">&#34;userId&#34;</span><span class="o">,</span> <span class="n">shippingAddress</spa [...] -</span><span class="s"> .withKeyField(&#34;</span><span class="n">userAndStreet</span><span class="s">&#34;) -</span><span class="s"> .withValueField(&#34;</span><span class="n">matchingPurchases</span><span class="err">&#34;</span><span class="o">));</span></code></pre></div> +<div class="highlight"><pre class="chroma"><code class="language-java" data-lang="java"><span class="n">purchases</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Group</span><span class="o">.</span><span class="na">byFieldNames</span><span class="o">(</span><span class="s">&#34;userId&#34;</span><span class="o">,</span> <span class="s">&#34;shippingAddres [...] +<span class="o">.</span><span class="na">withKeyField</span><span class="o">(</span><span class="s">&#34;userAndStreet&#34;</span><span class="o">)</span> +<span class="o">.</span><span class="na">withValueField</span><span class="o">(</span><span class="s">&#34;matchingPurchases&#34;</span><span class="o">));</span></code></pre></div> </div> <p>It is quite common to apply one or more aggregations to the grouped result. Each aggregation can specify one or more fields to aggregate, an aggregation function, and the name of the resulting field in the output schema. For example, the diff --git a/website/generated-content/documentation/programming-guide/index.html b/website/generated-content/documentation/programming-guide/index.html index d2d8cee..39612c3 100644 --- a/website/generated-content/documentation/programming-guide/index.html +++ b/website/generated-content/documentation/programming-guide/index.html @@ -1559,19 +1559,19 @@ allowing selections and aggregations in terms of named schema fields. Following schema transforms.</p><h5 id=selecting-input><strong>Selecting input</strong></h5><p>Often a computation is only interested in a subset of the fields in an input <code>PCollection</code>. The <code>Select</code> transform allows one to easily project out only the fields of interest. The resulting <code>PCollection</code> has a schema containing each selected field as a top-level field. Both top-level and nested fields can be selected. For example, in the Purchase schema, one -could select only the userId and streetAddress fields as follows</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>purchases</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>Select</span><span class=o>.</span><span class=na>fieldNames</span><span class=o>(</span><span class=s>"userId"</span><span class=o>,</span> <span class=n>shippingAddress</span><span class=o>.</sp [...] +could select only the userId and streetAddress fields as follows</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>purchases</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>Select</span><span class=o>.</span><span class=na>fieldNames</span><span class=o>(</span><span class=s>"userId"</span><span class=o>,</span> <span class=s>"shippingAddress.streetAddress"</ [...] top-level field in the resulting row. This means that if multiple fields are selected from the same nested row, each -selected field will appear as its own array field. For example</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>purchases</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>Select</span><span class=o>.</span><span class=na>fieldNames</span><span class=o>(</span> <span class=s>"transactions.bank"</span><span class=o>,</span> <span class=n>transactions</span><span class= [...] +selected field will appear as its own array field. For example</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>purchases</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>Select</span><span class=o>.</span><span class=na>fieldNames</span><span class=o>(</span> <span class=s>"transactions.bank"</span><span class=o>,</span> <span class=s>"transactions.purchaseAmoun [...] , then each selected field will be expanded to its own map at the top level. This means that the set of map keys will be copied, once for each selected field.</p><p>Sometimes different nested rows will have fields with the same name. Selecting multiple of these fields would result in a name conflict, as all selected fields are put in the same row schema. When this situation arises, the <code>Select.withFieldNameAs</code> builder method can be used to provide an alternate name for the selected field.</p><p>Another use of the Select transform is to flatten a nested schema into a single flat schema. For example</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>purchases</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>Select</span><span class=o>.</span><span c [...] those groupings, and storing the result of those aggregations in a new schema field. The output of the <code>Group</code> transform has a schema with one field corresponding to each aggregation performed.</p><p>The simplest usage of <code>Group</code> specifies no aggregations, in which case all inputs matching the provided set of fields -are grouped together into an <code>ITERABLE</code> field. For example</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>purchases</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>Group</span><span class=o>.</span><span class=na>byFieldNames</span><span class=o>(</span><span class=s>"userId"</span><span class=o>,</span> <span class=n>shippingAddress</span><span class=o [...] -builders, as follows:</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>purchases</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>Group</span><span class=o>.</span><span class=na>byFieldNames</span><span class=o>(</span><span class=s>"userId"</span><span class=o>,</span> <span class=n>shippingAddress</span><span class=o>.</span><span class=na>streetAddress</span><spa [...] -</span><span class=s> .withKeyField("</span><span class=n>userAndStreet</span><span class=s>") -</span><span class=s> .withValueField("</span><span class=n>matchingPurchases</span><span class=err>"</span><span class=o>));</span></code></pre></div></div><p>It is quite common to apply one or more aggregations to the grouped result. Each aggregation can specify one or more fields +are grouped together into an <code>ITERABLE</code> field. For example</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>purchases</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>Group</span><span class=o>.</span><span class=na>byFieldNames</span><span class=o>(</span><span class=s>"userId"</span><span class=o>,</span> <span class=s>"shippingAddress.streetAddress& [...] +builders, as follows:</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>purchases</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>Group</span><span class=o>.</span><span class=na>byFieldNames</span><span class=o>(</span><span class=s>"userId"</span><span class=o>,</span> <span class=s>"shippingAddress.streetAddress"</span><span class=o>)</span> + <span class=o>.</span><span class=na>withKeyField</span><span class=o>(</span><span class=s>"userAndStreet"</span><span class=o>)</span> + <span class=o>.</span><span class=na>withValueField</span><span class=o>(</span><span class=s>"matchingPurchases"</span><span class=o>));</span></code></pre></div></div><p>It is quite common to apply one or more aggregations to the grouped result. Each aggregation can specify one or more fields to aggregate, an aggregation function, and the name of the resulting field in the output schema. For example, the following application computes three aggregations grouped by userId, with all aggregations represented in a single output schema:</p><div class=language-java><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>purchases</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>Group</span><span class=o>.</span><span class=na>byFieldNames</span><span class=o>(</span><span class=s>"userId"</span><span class=o>)</span> diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index b10c142..d544949 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/categories/blog/</loc><lastmod>2020-05-28T15:14:36-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-05-28T15:14:36-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-05-28T15:14:36-07:00</lastmod></url><url><loc>/categories/python/</loc><lastmod>2020-05-28T15:14:36-07:00</lastmod></url><url><loc>/blog/ [...] \ No newline at end of file +<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/categories/blog/</loc><lastmod>2020-05-28T15:14:36-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-05-28T15:14:36-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-05-28T15:14:36-07:00</lastmod></url><url><loc>/categories/python/</loc><lastmod>2020-05-28T15:14:36-07:00</lastmod></url><url><loc>/blog/ [...] \ No newline at end of file