This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push: new 0b642f8ef33 Publishing website 2022/12/22 16:17:51 at commit eb23b0a 0b642f8ef33 is described below commit 0b642f8ef33d43ccf78d21c244d212ef10c2cb39 Author: jenkins <bui...@apache.org> AuthorDate: Thu Dec 22 16:17:51 2022 +0000 Publishing website 2022/12/22 16:17:51 at commit eb23b0a --- .../blog/review-input-streaming-connectors/index.html | 2 +- website/generated-content/contribute/index.xml | 8 ++++---- .../generated-content/contribute/postcommits-policies/index.html | 4 ++-- .../contribute/precommit-triage-guide/index.html | 6 +++--- website/generated-content/contribute/runner-guide/index.html | 4 ++-- website/generated-content/documentation/index.xml | 4 ++-- .../generated-content/documentation/io/built-in/cdap/index.html | 4 ++-- .../documentation/io/built-in/sparkreceiver/index.html | 4 ++-- website/generated-content/documentation/io/connectors/index.html | 8 ++++---- .../generated-content/documentation/programming-guide/index.html | 4 ++-- website/generated-content/sitemap.xml | 2 +- 11 files changed, 25 insertions(+), 25 deletions(-) diff --git a/website/generated-content/blog/review-input-streaming-connectors/index.html b/website/generated-content/blog/review-input-streaming-connectors/index.html index 66b7e7012a3..24be3125924 100644 --- a/website/generated-content/blog/review-input-streaming-connectors/index.html +++ b/website/generated-content/blog/review-input-streaming-connectors/index.html @@ -23,7 +23,7 @@ function openMenu(){addPlaceholder();blockScroll();}</script><div class="body__c & Julien Phalip [<a href=https://twitter.com/julienphalip>@julienphalip</a>]</span></div></header><div class="arrow-list header-top-margin" itemprop=articleBody><p>In this post, you’ll learn about the current state of support for input streaming connectors in <a href=/>Apache Beam</a>. For more context, you’ll also learn about the corresponding state of support in <a href=https://spark.apache.org/>Apache Spark</a>.</p><p>With batch processing, you might load data from any sourc [...] and <a href=https://spark.apache.org/docs/latest/api/java/org/apache/spark/streaming/StreamingContext.html#textFileStream-java.lang.String->textFileStream</a></td></tr><tr><td>S3<br>(Using the <code>s3://</code> URI)</td><td><a href=https://beam.apache.org/releases/javadoc/2.43.0/org/apache/beam/sdk/io/FileIO.html>FileIO</a> + <a href=https://beam.apache.org/releases/javadoc/2.43.0/org/apache/beam/sdk/io/aws2/options/S3Options.html>S3Options</a></td></tr><tr><td rowspan=3>Messaging Queue [...] -and <a href=https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.streaming.StreamingContext.textFileStream.html>textFileStream</a></td></tr><tr><td rowspan=2>Object stores</td><td>Google Cloud Storage</td><td><a href=https://beam.apache.org/releases/pydoc/2.43.0/apache_beam.io.gcp.gcsio.html>io.gcp.gcsio</a></td><td rowspan=2><a href=https://spark.apache.org/docs/latest/api/python/pyspark.streaming.html#pyspark.streaming.StreamingContext.textFileStream>textFileStream</a> [...] +and <a href=https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.streaming.StreamingContext.textFileStream.html>textFileStream</a></td></tr><tr><td rowspan=2>Object stores</td><td>Google Cloud Storage</td><td><a href=https://beam.apache.org/releases/pydoc/2.43.0/apache_beam.io.gcp.gcsio.html>io.gcp.gcsio</a></td><td rowspan=2><a href=https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.streaming.StreamingContext.html#pyspark.streaming.StreamingContext>te [...] </p><p>2022/11/17</p></div><div class=post><p class=post-title>Apache Beam 2.43.0</p><p class=post-info>Chamikara Jayalath</p></div></a><a class=post-card href=/blog/ml-resources/ data-categories="blog python"><div class="post-info post-category"><p>blog & python diff --git a/website/generated-content/contribute/index.xml b/website/generated-content/contribute/index.xml index 289e683207e..2e07ae2373f 100644 --- a/website/generated-content/contribute/index.xml +++ b/website/generated-content/contribute/index.xml @@ -2379,7 +2379,7 @@ community follows these post-commit test policies:</p> <p>When a post-commit test fails, follow the provided steps for your situation.</p> <h3 id="found-failing-test">I found a test failure</h3> <ol> -<li>Create a <a href="https://github.com/apache/tag-beam-it/issues/new/choose">GitHub issue</a> and assign it to yourself.</li> +<li>Create a <a href="https://github.com/apache/beam/issues/new/choose">GitHub issue</a> and assign it to yourself.</li> </ol> <ul> <li>Components: testing, anything else relevant</li> @@ -2541,9 +2541,9 @@ down to a specific test suite by clicking on it.</p> <p><img src="/images/precommit_dashboard.png" alt="example pre-commit duration dashboard"></p> <h2 id="triage-process">Triage Process</h2> <ol> -<li><a href="https://github.com/apache/tag-beam-it/issues">Search for existing +<li><a href="https://github.com/apache/beam/issues">Search for existing issues</a></li> -<li>Create a new issue if needed: <a href="https://github.com/apache/tag-beam-it/issues/new/choose">GitHub Issues</a></li> +<li>Create a new issue if needed: <a href="https://github.com/apache/beam/issues/new/choose">GitHub Issues</a></li> </ol> <ul> <li>Components: testing, anything else relevant</li> @@ -3831,7 +3831,7 @@ language-independent manner</li> <p>By inspecting transforms only through these classes, your runner will not depend on the particulars of the Java SDK.</p> <h2 id="the-runner-api-protos">The Runner API protos</h2> -<p>The <a href="https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/beam_runner_api.proto">Runner +<p>The <a href="https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/beam_runner_api.proto">Runner API</a> refers to a specific manifestation of the concepts in the Beam model, as a protocol buffers schema. Even though you should not manipulate these messages diff --git a/website/generated-content/contribute/postcommits-policies/index.html b/website/generated-content/contribute/postcommits-policies/index.html index 41078dfa50d..a0edbe55425 100644 --- a/website/generated-content/contribute/postcommits-policies/index.html +++ b/website/generated-content/contribute/postcommits-policies/index.html @@ -26,7 +26,7 @@ it is important that the tests pass reliably. Jenkins executes post-commit tests against the HEAD of the <code>master</code> branch. If post-commit tests fail, there is a problem with the HEAD build. In addition, post-commit tests are time consuming to run, and it is often hard to triage test failures.</p><h2 id=policies>Policies</h2><p>To ensure that Beam’s post-commit tests are reliable and healthy, the Beam -community follows these post-commit test policies:</p><ul><li><a href=/contribute/postcommits-policies-details/index.html#rollback_first>Rollback first</a></li><li><a href=/contribute/postcommits-policies-details/index.html#failing_test_is_critical_bug>A failing test is a critical bug</a></li><li><a href=/contribute/postcommits-policies-details/index.html#flake_is_failing>A flaky test is a critical bug</a></li><li><a href=/contribute/postcommits-policies-details/index.html#remove_flake>F [...] +community follows these post-commit test policies:</p><ul><li><a href=/contribute/postcommits-policies-details/index.html#rollback_first>Rollback first</a></li><li><a href=/contribute/postcommits-policies-details/index.html#failing_test_is_critical_bug>A failing test is a critical bug</a></li><li><a href=/contribute/postcommits-policies-details/index.html#flake_is_failing>A flaky test is a critical bug</a></li><li><a href=/contribute/postcommits-policies-details/index.html#remove_flake>F [...] test temporarily</a> while you rollback or create a fix.</li></ol><blockquote><p>Note: Rollback is always the first course of action. If a fix is trivial, open a pull request with the proposed fix while doing rollback.</p></blockquote><h3 id=pr-rolled-back>My change was rolled back due to a test failure</h3><p>After rollback there is time for deeper investigation. Start by looking at the @@ -38,7 +38,7 @@ Remember that <a href=/contribute/postcommits-policies-details/index.html#flake_is_failing>a flaky test is a critical bug</a>. Other bad tests are similar: they may fail for arbitrary reasons having nothing to do with what is being tested, making our signal unreliable.</li><li>Mark the problematic test to be skipped, with a link to the GitHub issue.</li></ol><h2 id=useful-links>Useful links</h2><ul><li><a href=https://cwiki.apache.org/confluence/display/BEAM/Contribution+Testing+Guide#ContributionTestingGuide-Bestpracticesforwritingtests>Best practices for writing tests</a></li></ul><h2 id=references>References</h2><ol><li><a href=https://lists.apache.org/thread.html/3bb4aa777751da2e2d7e226 [...] -mailing list proposal thread.</li></ol><div class=feedback><p class=update>Last updated on 2022/05/09</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div [...] +mailing list proposal thread.</li></ol><div class=feedback><p class=update>Last updated on 2022/09/08</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div [...] <a href=https://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col--group><div class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...] \ No newline at end of file diff --git a/website/generated-content/contribute/precommit-triage-guide/index.html b/website/generated-content/contribute/precommit-triage-guide/index.html index 27b7f9fd580..caa5e015a7d 100644 --- a/website/generated-content/contribute/precommit-triage-guide/index.html +++ b/website/generated-content/contribute/precommit-triage-guide/index.html @@ -33,8 +33,8 @@ the console log of a job.</li><li>Pre-commits aren’t timing out, but the t is >30m.</li></ol><h3 id=pre-commit-dashboard>Pre-commit Dashboard</h3><p>The Beam Community Metrics site contains a <a href=http://metrics.beam.apache.org/d/_TNndF2iz/pre-commit-tests>Pre-Commit Tests</a> dashboard showing job timing trends. You can modify the time window (defaults to 7 days) or filter -down to a specific test suite by clicking on it.</p><p><img src=/images/precommit_dashboard.png alt="example pre-commit duration dashboard"></p><h2 id=triage-process>Triage Process</h2><ol><li><a href=https://github.com/apache/tag-beam-it/issues>Search for existing -issues</a></li><li>Create a new issue if needed: <a href=https://github.com/apache/tag-beam-it/issues/new/choose>GitHub Issues</a></li></ol><ul><li>Components: testing, anything else relevant</li><li>Label: precommit</li><li>Reference this page in the description.</li></ul><ol><li>Determine where the slowness is coming from and identify issues. Open +down to a specific test suite by clicking on it.</p><p><img src=/images/precommit_dashboard.png alt="example pre-commit duration dashboard"></p><h2 id=triage-process>Triage Process</h2><ol><li><a href=https://github.com/apache/beam/issues>Search for existing +issues</a></li><li>Create a new issue if needed: <a href=https://github.com/apache/beam/issues/new/choose>GitHub Issues</a></li></ol><ul><li>Components: testing, anything else relevant</li><li>Label: precommit</li><li>Reference this page in the description.</li></ul><ol><li>Determine where the slowness is coming from and identify issues. Open additional issues if needed (such as for multiple issues).</li><li>Assign the issue as appropriate, e.g., to the test’s or PR’s author.</li></ol><h2 id=resolution>Resolution</h2><p>It is important that we quickly fix slow pre-commit tests. See <a href=/contribute/precommit-policies/>pre-commit test policies</a> for details.</p><h2 id=possible-causes-and-solutions>Possible Causes and Solutions</h2><p>This section lists some starting points for fixing pre-commit slowness.</p><h3 id=resource-exhaustion>Resource Exhaustion</h3><p>Have a look at the graphs in the Jupyter notebook. Does the rise in total duration match the rise in queuing time? If so, the slowness might be unrelated @@ -49,7 +49,7 @@ time waiting (such as an integration test) could be made to run concurrently wit the other tests.</p><p>If a test is determined to be too slow to be part of pre-commit tests, it should be removed from pre-commit and placed in post-commit instead. In addition, ensure that the code covered by the removed test is <a href=/contribute/postcommits-policies-details/#precommit_for_postcommit>covered by a unit test in -pre-commit</a>.</p><h3 id=slow-integration-tests>Slow integration tests</h3><p>Integration test slowdowns may be caused by dependent services.</p><h2 id=references>References</h2><ul><li><a href="https://docs.google.com/document/d/1udtvggmS2LTMmdwjEtZCcUQy6aQAiYTI3OrTP8CLfJM/edit?usp=sharing">Beam Fast Precommits design doc</a></li></ul><div class=feedback><p class=update>Last updated on 2022/05/09</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all [...] +pre-commit</a>.</p><h3 id=slow-integration-tests>Slow integration tests</h3><p>Integration test slowdowns may be caused by dependent services.</p><h2 id=references>References</h2><ul><li><a href="https://docs.google.com/document/d/1udtvggmS2LTMmdwjEtZCcUQy6aQAiYTI3OrTP8CLfJM/edit?usp=sharing">Beam Fast Precommits design doc</a></li></ul><div class=feedback><p class=update>Last updated on 2022/09/08</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all [...] <a href=https://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col--group><div class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...] \ No newline at end of file diff --git a/website/generated-content/contribute/runner-guide/index.html b/website/generated-content/contribute/runner-guide/index.html index e77b8cb4eb3..91f59e41168 100644 --- a/website/generated-content/contribute/runner-guide/index.html +++ b/website/generated-content/contribute/runner-guide/index.html @@ -301,7 +301,7 @@ SDK-agnostic manner are in the <code>beam-runners-core-construction-java</code> artifact, in the <code>org.apache.beam.runners.core.construction</code> namespace. The utilities are named consistently, like so:</p><ul><li><code>PTransformTranslation</code> - registry of known transforms and standard URNs</li><li><code>ParDoTranslation</code> - utilities for working with <code>ParDo</code> in a language-independent manner</li><li><code>WindowIntoTranslation</code> - same for <code>Window</code></li><li><code>FlattenTranslation</code> - same for <code>Flatten</code></li><li><code>WindowingStrategyTranslation</code> - same for windowing strategies</li><li><code>CoderTranslation</code> - same for coders</li><li>… etc, etc …</li></ul><p>By inspecting transforms only through these classes, your runner will not -depend on the particulars of the Java SDK.</p><h2 id=the-runner-api-protos>The Runner API protos</h2><p>The <a href=https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/beam_runner_api.proto>Runner +depend on the particulars of the Java SDK.</p><h2 id=the-runner-api-protos>The Runner API protos</h2><p>The <a href=https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/beam_runner_api.proto>Runner API</a> refers to a specific manifestation of the concepts in the Beam model, as a protocol buffers schema. Even though you should not manipulate these messages @@ -438,7 +438,7 @@ message GetStateResponse { enum JobState { ... -}</code></pre></div></div><div class=feedback><p class=update>Last updated on 2021/02/05</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class="footer [...] +}</code></pre></div></div><div class=feedback><p class=update>Last updated on 2022/09/08</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class="footer [...] <a href=https://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col--group><div class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...] \ No newline at end of file diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml index b22dbf41941..f93ab96cb72 100644 --- a/website/generated-content/documentation/index.xml +++ b/website/generated-content/documentation/index.xml @@ -13411,7 +13411,7 @@ $ jar -jar /path/to/expansion_service.jar &lt;PORT_NUMBER&gt;</code>& </li> <li> <p>In the module, use one of the <a href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.external.html#apache_beam.transforms.external.PayloadBuilder"><code>PayloadBuilder</code></a> classes to build the payload for the initial cross-language transform expansion request.</p> -<p>The parameter names and types of the payload should map to parameter names and types of the configuration POJO provided to the Java <code>ExternalTransformBuilder</code>. Parameter types are mapped across SDKs using a <a href="https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/schema.proto">Beam schema</a>. Parameter names are mapped by simply converting Python underscore-separated variable names to camel-case (Java standard).</p> +<p>The parameter names and types of the payload should map to parameter names and types of the configuration POJO provided to the Java <code>ExternalTransformBuilder</code>. Parameter types are mapped across SDKs using a <a href="https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/schema.proto">Beam schema</a>. Parameter names are mapped by simply converting Python underscore-separated variable names to camel-case (Jav [...] <p>In the following example, <a href="https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/kafka.py">kafka.py</a> uses <code>NamedTupleBasedPayloadBuilder</code> to build the payload. The parameters map to the Java <a href="https://github.com/apache/beam/blob/master/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java">KafkaIO.External.Configuration</a> config object defined in the previous section.</p> <div class="highlight"><pre class="chroma"><code class="language-py" data-lang="py"><span class="k">class</span> <span class="nc">ReadFromKafkaSchema</span><span class="p">(</span><span class="n">typing</span><span class="o">.</span><span class="n">NamedTuple</span><span class="p">):</span> <span class="n">consumer_config</span><span class="p">:</span> <span class="n">typing</span><span class="o">.</span><span class="n">Mapping</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]</span> @@ -14241,7 +14241,7 @@ limitations under the License. <ul> <li>CDAP Streaming plugin should be based on <code>Spark Receiver</code> (Spark 2.4).</li> <li>CDAP Streaming plugin should support work with offsets.</li> -<li>Corresponding Spark Receiver should implement <a href="https://github.com/apache/beam/blob/master/sdks/java/io/sparkreceiver/src/main/java/org/apache/beam/sdk/io/sparkreceiver/HasOffset.java">HasOffset</a> interface.</li> +<li>Corresponding Spark Receiver should implement <a href="https://github.com/apache/beam/blob/master/sdks/java/io/sparkreceiver/2/src/main/java/org/apache/beam/sdk/io/sparkreceiver/HasOffset.java">HasOffset</a> interface.</li> <li>Records should have the numeric field that represents record offset.</li> </ul> <h2 id="batch-reading-using-cdapio">Batch reading using CdapIO</h2> diff --git a/website/generated-content/documentation/io/built-in/cdap/index.html b/website/generated-content/documentation/io/built-in/cdap/index.html index 1894610a5ca..635765351b1 100644 --- a/website/generated-content/documentation/io/built-in/cdap/index.html +++ b/website/generated-content/documentation/io/built-in/cdap/index.html @@ -19,7 +19,7 @@ function addPlaceholder(){$('input:text').attr('placeholder',"What are you looking for?");} function endSearch(){var search=document.querySelector(".searchBar");search.classList.add("disappear");var icons=document.querySelector("#iconsBar");icons.classList.remove("disappear");} function blockScroll(){$("body").toggleClass("fixedPosition");} -function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Documentation</span></li><li><a href=/documentation>Using the Documentation</a></li><li class=section-nav-item--collapsible><span class=section-nav-lis [...] +function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Documentation</span></li><li><a href=/documentation>Using the Documentation</a></li><li class=section-nav-item--collapsible><span class=section-nav-lis [...] <span class=c1>// Read plugin parameters (e.g. from PipelineOptions) and put them into 'myPluginConfigParams' map. </span><span class=c1></span><span class=n>myPluginConfigParams</span><span class=o>.</span><span class=na>put</span><span class=o>(</span><span class=n>MyPluginConstants</span><span class=o>.</span><span class=na>USERNAME_PARAMETER_NAME</span><span class=o>,</span> <span class=n>pipelineOptions</span><span class=o>.</span><span class=na>getUsername</span><span class=o>());</span> <span class=c1>// ... @@ -169,7 +169,7 @@ function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfi <span class=o>.</span><span class=na>withPluginConfig</span><span class=o>(</span><span class=n>pluginConfig</span><span class=o>)</span> <span class=o>.</span><span class=na>withKeyClass</span><span class=o>(</span><span class=n>NullWritable</span><span class=o>.</span><span class=na>class</span><span class=o>)</span> <span class=o>.</span><span class=na>withValueClass</span><span class=o>(</span><span class=n>String</span><span class=o>.</span><span class=na>class</span><span class=o>);</span> -<span class=n>p</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=s>"readFromSalesforcePlugin"</span><span class=o>,</span> <span class=n>readTransform</span><span class=o>);</span></code></pre></div></div></div><p>To learn more please check out <a href=https://github.com/apache/beam/tree/master/examples/java/cdap/src/main/java/org/apache/beam/examples/complete/cdap>complete examples</a>.</p><div class=feedback><p class=update>Last updated o [...] +<span class=n>p</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=s>"readFromSalesforcePlugin"</span><span class=o>,</span> <span class=n>readTransform</span><span class=o>);</span></code></pre></div></div></div><p>To learn more please check out <a href=https://github.com/apache/beam/tree/master/examples/java/cdap/src/main/java/org/apache/beam/examples/complete/cdap>complete examples</a>.</p><div class=feedback><p class=update>Last updated o [...] <a href=https://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col--group><div class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...] \ No newline at end of file diff --git a/website/generated-content/documentation/io/built-in/sparkreceiver/index.html b/website/generated-content/documentation/io/built-in/sparkreceiver/index.html index 715b3065bcc..0c444606389 100644 --- a/website/generated-content/documentation/io/built-in/sparkreceiver/index.html +++ b/website/generated-content/documentation/io/built-in/sparkreceiver/index.html @@ -19,7 +19,7 @@ function addPlaceholder(){$('input:text').attr('placeholder',"What are you looking for?");} function endSearch(){var search=document.querySelector(".searchBar");search.classList.add("disappear");var icons=document.querySelector("#iconsBar");icons.classList.remove("disappear");} function blockScroll(){$("body").toggleClass("fixedPosition");} -function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Documentation</span></li><li><a href=/documentation>Using the Documentation</a></li><li class=section-nav-item--collapsible><span class=section-nav-lis [...] +function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Documentation</span></li><li><a href=/documentation>Using the Documentation</a></li><li class=section-nav-item--collapsible><span class=section-nav-lis [...] </span><span class=c1></span><span class=n>MyConfig</span> <span class=n>myPluginConfig</span> <span class=o>=</span> <span class=k>new</span> <span class=n>MyConfig</span><span class=o>(</span><span class=n>authToken</span><span class=o>,</span> <span class=n>apiServerUrl</span><span class=o>);</span> <span class=n>Object</span><span class=o>[]</span> <span class=n>myConstructorArgs</span> <span class=o>=</span> <span class=k>new</span> <span class=n>Object</span><span class=o>[]</span> <span class=o>{</span><span class=n>myConfig</span><span class=o>};</span> <span class=n>ReceiverBuilder</span><span class=o><</span><span class=n>String</span><span class=o>,</span> <span class=n>MyReceiver</span><span class=o><</span><span class=n>String</span><span class=o>>></span> <span class=n>myReceiverBuilder</span> <span class=o>=</span> @@ -42,7 +42,7 @@ function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfi <span class=n>SparkReceiverIO</span><span class=o>.<</span><span class=n>String</span><span class=o>></span><span class=n>read</span><span class=o>()</span> <span class=o>.</span><span class=na>withGetOffsetFn</span><span class=o>(</span><span class=n>GetOffsetUtils</span><span class=o>.</span><span class=na>getOffsetFnForHubspot</span><span class=o>())</span> <span class=o>.</span><span class=na>withSparkReceiverBuilder</span><span class=o>(</span><span class=n>hubspotReceiverBuilder</span><span class=o>)</span> -<span class=n>p</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=s>"readFromHubspotReceiver"</span><span class=o>,</span> <span class=n>readTransform</span><span class=o>);</span></code></pre></div></div></div><div class=feedback><p class=update>Last updated on 2022/12/06</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!< [...] +<span class=n>p</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=s>"readFromHubspotReceiver"</span><span class=o>,</span> <span class=n>readTransform</span><span class=o>);</span></code></pre></div></div></div><div class=feedback><p class=update>Last updated on 2022/12/22</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!< [...] <a href=https://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col--group><div class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...] \ No newline at end of file diff --git a/website/generated-content/documentation/io/connectors/index.html b/website/generated-content/documentation/io/connectors/index.html index 0a490f40726..b5234918d22 100644 --- a/website/generated-content/documentation/io/connectors/index.html +++ b/website/generated-content/documentation/io/connectors/index.html @@ -101,9 +101,9 @@ function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfi native</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=absent>✘</td></tr><tr><td>Pub/Sub Lite</td><td class=present>✔</td><td class=present>✔</td><td class=present>✔ <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/pubsublite/PubsubLiteIO.html>native</a></td><td class=present>✔ <a href=https://beam.apache.org/releases/pydoc/current/apache_beam.io.gcp.pubsublite.html>via X-language</a></td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td>InfluxDB</td><td class=present>✔</td><td class=present>✔</td><td class=present>✔ -<a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/influxdb/InfluxDbIO.html>native</a></td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td>SparkReceiverIO (<a href=/documentation/io/built-in/sparkreceiver/>guide</a>)</td><td class=present>✔</td><td class=present>✘</td><td class=present>✔ -<a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/sparkreceiver/2/SparkReceiverIO.html>native</a></td><td>Not available</td><td>Not available</td><td class=present>✘</td><td class=present>✔</td></tr><tr><td>CdapIO (<a href=/documentation/io/built-in/cdap/>guide</a>)</td><td class=present>✔</td><td class=present>✔</td><td class=present>✔ -<a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/cdap/CdapIO.html>native</a></td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr></table></div><h2 id=other-io-connectors-for-apache-beam>Other I/O Connectors for Apache Beam</h2><div class=table-container-wrapper><table class="table table-bordered table-connectors"><tr><th>Connector Name</th><th>Source Supported</th><th>Sink Supported</th><th>Java</th><th>Pytho [...] +<a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/influxdb/InfluxDbIO.html>native</a></td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td>SparkReceiverIO (<a href=/documentation/io/built-in/sparkreceiver/>guide</a>)</td><td class=present>✔</td><td class=absent>✘</td><td class=present>✔ +native</td><td>Not available</td><td>Not available</td><td class=absent>✘</td><td class=present>✔</td></tr><tr><td>CdapIO (<a href=/documentation/io/built-in/cdap/>guide</a>)</td><td class=present>✔</td><td class=present>✔</td><td class=present>✔ +native</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr></table></div><h2 id=other-io-connectors-for-apache-beam>Other I/O Connectors for Apache Beam</h2><div class=table-container-wrapper><table class="table table-bordered table-connectors"><tr><th>Connector Name</th><th>Source Supported</th><th>Sink Supported</th><th>Java</th><th>Python</th><th>Go</th><th>Batch Supported</th><th>Streaming Supported</th></tr><tr><td><a href=https://git [...] native</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td><a href=https://github.com/google/hana-bq-beam-connector>SAP Hana to Google BigQuery</a></td><td class=present>✔</td><td class=present>✔</td><td class=present>✔ native</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=absent>✘</td></tr><tr><td><a href=https://github.com/esakik/beam-mysql-connector>MySQL</a></td><td class=present>✔</td><td class=present>✔</td><td>Not available</td><td class=present>✔ native</td><td>Not available</td><td class=present>✔</td><td class=absent>✘</td></tr><tr><td><a href=https://github.com/Refinitiv/trep-websockets-beam-io>TrepWsIO</a></td><td class=present>✔</td><td class=absent>✘</td><td class=present>✔ @@ -113,7 +113,7 @@ native</td><td>Not available</td><td>Not available</td><td class=present>✔</td native</td><td>Not available</td><td class=present>✔</td><td class=absent>✘</td></tr><tr><td><a href=https://github.com/amient/goconnect>Go Connect</a></td><td class=present>✔</td><td class=present>✔</td><td class=absent>✘</td><td class=absent>✘</td><td class=present>✔ native</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td><a href=https://github.com/tinybirdco/tinybird-beam>Tinybird</a></td><td class=absent>✘</td><td class=present>✔</td><td>Not available</td><td class=present>✔ native</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td><a href=https://github.com/jccatrinck/dataflow-cloud-sql-python>Cloud SQL</a></td><td class=present>✔</td><td class=absent>✘</td><td>Not available</td><td class=present>✔ -native</td><td>Not available</td><td class=present>✔</td><td class=absent>✘</td></tr></table></div><div class=feedback><p class=update>Last updated on 2022/12/15</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=foote [...] +native</td><td>Not available</td><td class=present>✔</td><td class=absent>✘</td></tr></table></div><div class=feedback><p class=update>Last updated on 2022/12/22</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=foote [...] <a href=https://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col--group><div class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...] \ No newline at end of file diff --git a/website/generated-content/documentation/programming-guide/index.html b/website/generated-content/documentation/programming-guide/index.html index 4e1d95840f6..62654034649 100644 --- a/website/generated-content/documentation/programming-guide/index.html +++ b/website/generated-content/documentation/programming-guide/index.html @@ -4985,7 +4985,7 @@ mentioned Java transform:</p><div class=highlight><pre class=chroma><code class= </code></pre></div><p>For additional examples, see <a href=https://github.com/apache/beam/blob/master/examples/multi-language/src/main/java/org/apache/beam/examples/multilanguage/JavaCountRegistrar.java>JavaCountRegistrar</a> and <a href=https://github.com/apache/beam/blob/master/examples/multi-language/src/main/java/org/apache/beam/examples/multilanguage/JavaPrefixRegistrar.java>JavaPrefixRegistrar</a>.</p></li></ol><p>After you have implemented the <code>ExternalTransformBuilder</code> [...] # Start the expansion service at the specified port. -$ jar -jar /path/to/expansion_service.jar <PORT_NUMBER></code></pre></div></div><p>The expansion service is now ready to serve transforms on the specified port.</p><p>When creating SDK-specific wrappers for your transform, you may be able to use SDK-provided utilities to start up an expansion service. For example, the Python SDK provides the utilities <a href=https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.external.html#apache_beam.transforms.external.JavaJarE [...] +$ jar -jar /path/to/expansion_service.jar <PORT_NUMBER></code></pre></div></div><p>The expansion service is now ready to serve transforms on the specified port.</p><p>When creating SDK-specific wrappers for your transform, you may be able to use SDK-provided utilities to start up an expansion service. For example, the Python SDK provides the utilities <a href=https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.external.html#apache_beam.transforms.external.JavaJarE [...] <span class=n>consumer_config</span><span class=p>:</span> <span class=n>typing</span><span class=o>.</span><span class=n>Mapping</span><span class=p>[</span><span class=nb>str</span><span class=p>,</span> <span class=nb>str</span><span class=p>]</span> <span class=n>topics</span><span class=p>:</span> <span class=n>typing</span><span class=o>.</span><span class=n>List</span><span class=p>[</span><span class=nb>str</span><span class=p>]</span> <span class=c1># Other properties omitted for clarity.</span> @@ -5271,7 +5271,7 @@ implementations. This information can be accessed by using the conventional <span class=o>...</span> <span class=k>def</span> <span class=nf>infer_output_type</span><span class=p>(</span><span class=bp>self</span><span class=p>,</span> <span class=n>input_type</span><span class=p>):</span> - <span class=k>return</span> <span class=n>input_type</span></code></pre></div></div></div><div class=feedback><p class=update>Last updated on 2022/12/09</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><di [...] + <span class=k>return</span> <span class=n>input_type</span></code></pre></div></div></div><div class=feedback><p class=update>Last updated on 2022/09/08</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><di [...] <a href=https://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col--group><div class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...] \ No newline at end of file diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 97e1f6ccbe4..9e03fedaf02 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/apacheplayground/</loc><lastmod>2022-12-01T11:37:53-05:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2022-12-20T13:06:48-08:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-12-20T13:06:48-08:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-12-20T13:06:48-08:00</lastmod></url><url><loc>/b [...] \ No newline at end of file +<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/apacheplayground/</loc><lastmod>2022-12-01T11:37:53-05:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2022-12-20T13:06:48-08:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-12-20T13:06:48-08:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-12-20T13:06:48-08:00</lastmod></url><url><loc>/b [...] \ No newline at end of file