This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push: new ddf66a3 Publishing website 2021/09/30 18:03:15 at commit e525449 ddf66a3 is described below commit ddf66a3f72c4fa3c6aa5bfaf1d03089a366cf7b8 Author: jenkins <bui...@apache.org> AuthorDate: Thu Sep 30 18:03:15 2021 +0000 Publishing website 2021/09/30 18:03:15 at commit e525449 --- .../documentation/glossary/index.html | 2 +- website/generated-content/documentation/index.xml | 54 ++++++++++++---------- website/generated-content/sitemap.xml | 2 +- 3 files changed, 31 insertions(+), 27 deletions(-) diff --git a/website/generated-content/documentation/glossary/index.html b/website/generated-content/documentation/glossary/index.html index d45fc93..d632365 100644 --- a/website/generated-content/documentation/glossary/index.html +++ b/website/generated-content/documentation/glossary/index.html @@ -18,7 +18,7 @@ function addPlaceholder(){$('input:text').attr('placeholder',"What are you looking for?");} function endSearch(){var search=document.querySelector(".searchBar");search.classList.add("disappear");var icons=document.querySelector("#iconsBar");icons.classList.remove("disappear");} function blockScroll(){$("body").toggleClass("fixedPosition");} -function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Documentation</span></li><li><a href=/documentation>Using the Documentation</a></li><li class=section-nav-item--collapsible><span class=section-nav-lis [...] +function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Documentation</span></li><li><a href=/documentation>Using the Documentation</a></li><li class=section-nav-item--collapsible><span class=section-nav-lis [...] <a href=http://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div></div></div></footer></body></html> \ No newline at end of file diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml index 716914b..9c698e5 100644 --- a/website/generated-content/documentation/index.xml +++ b/website/generated-content/documentation/index.xml @@ -3323,14 +3323,14 @@ limitations under the License. --> <h1 id="apache-beam-glossary">Apache Beam glossary</h1> <h2 id="aggregation">Aggregation</h2> -<p>A transform pattern for computing a value from multiple input elements. Aggregation is similar to the reduce operation in the <a href="https://en.wikipedia.org/wiki/MapReduce">MapReduce</a> model. Aggregation transforms include Count (computes the count of all elements in the aggregation), Max (computes the maximum element in the aggregation), and Sum (computes the sum of all elements in the aggregation).</p> -<p>For a complete list of aggregation transforms, see:</p> +<p>A transform pattern for computing a value from multiple input elements. Aggregation is similar to the reduce operation in the <a href="https://en.wikipedia.org/wiki/MapReduce">MapReduce</a> model. Aggregation transforms include Combine (applies a user-defined function to all elements in the aggregation), Count (computes the count of all elements in the aggregation), Max (computes the maximum element in the aggregation), and Sum (computes the sum of all elements in the aggrega [...] +<p>For a list of built-in aggregation transforms, see:</p> <ul> <li><a href="/documentation/transforms/java/overview/#aggregation">Java Transform catalog</a></li> <li><a href="/documentation/transforms/python/overview/#aggregation">Python Transform catalog</a></li> </ul> <h2 id="apply">Apply</h2> -<p>A method for invoking a transform on a PCollection. Each transform in the Beam SDKs has a generic <code>apply</code> method (or pipe operator <code>|</code>). Invoking multiple Beam transforms is similar to method chaining, but with a difference: You apply the transform to the input PCollection, passing the transform itself as an argument, and the operation returns the output PCollection. Because of Beam’s deferred execution model, applying a transform does not immediat [...] +<p>A method for invoking a transform on an input PCollection (or set of PCollections) to produce one or more output PCollections. The <code>apply</code> method is attached to the PCollection (or value). Invoking multiple Beam transforms is similar to method chaining, but with a difference: You apply the transform to the input PCollection, passing the transform itself as an argument, and the operation returns the output PCollection. Because of Beam’s deferred execution model, app [...] <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#applying-transforms">Applying transforms</a></li> @@ -3342,13 +3342,13 @@ limitations under the License. <li><a href="/documentation/programming-guide/#size-and-boundedness">Size and boundedness</a></li> </ul> <h2 id="bounded-data">Bounded data</h2> -<p>A dataset of a known, fixed size. A PCollection can be bounded or unbounded, depending on the source of the data that it represents. Reading from a batch data source, such as a file or a database, creates a bounded PCollection. Beam also supports reading a bounded amount of data from an unbounded source.</p> +<p>A dataset of a known, fixed size (alternatively, a dataset that is not growing over time). A PCollection can be bounded or unbounded, depending on the source of the data that it represents. Reading from a batch data source, such as a file or a database, creates a bounded PCollection. Beam also supports reading a bounded amount of data from an unbounded source.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#size-and-boundedness">Size and boundedness</a></li> </ul> <h2 id="bundle">Bundle</h2> -<p>The processing unit for elements in a PCollection. Instead of processing all elements in a PCollection simultaneously, Beam processes the elements in bundles. The runner handles the division of the collection into bundles, and in doing so it may optimize the bundle size for the use case. For example, a streaming runner might process smaller bundles than a batch runner.</p> +<p>The processing and commit/retry unit for elements in a PCollection. Instead of processing all elements in a PCollection simultaneously, Beam processes the elements in bundles. The runner handles the division of the collection into bundles, and in doing so it may optimize the bundle size for the use case. For example, a streaming runner might process smaller bundles than a batch runner.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/runtime/model/#bundling-and-persistence">Bundling and persistence</a></li> @@ -3380,7 +3380,7 @@ limitations under the License. <li><a href="/documentation/transforms/python/aggregation/combinevalues/">CombineValues (Python)</a></li> </ul> <h2 id="composite-transform">Composite transform</h2> -<p>A PTransform that expands into many PTransforms. Composite transforms have a nested structure, in which a complex transform applies one or more simpler transforms. These simpler transforms could be existing Beam operations like ParDo, Combine, or GroupByKey, or they could be other composite transforms. Nesting multiple transforms inside a single composite transform can make your pipeline more modular and easier to understand.</p> +<p>A PTransform that expands into many PTransforms. Composite transforms have a nested structure, in which a complex transform applies one or more simpler transforms. These simpler transforms could be existing Beam operations like ParDo, Combine, or GroupByKey, or they could be other composite transforms. Nesting multiple transforms inside a single composite transform can make your pipeline more modular and easier to understand. Many of the built-in transforms are composite transforms [...] <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#composite-transforms">Composite transforms</a></li> @@ -3398,7 +3398,7 @@ limitations under the License. <li><a href="/documentation/programming-guide/#multi-language-pipelines">Multi-language pipelines</a></li> </ul> <h2 id="deferred-execution">Deferred execution</h2> -<p>A feature of the Beam execution model. Beam operations are deferred, meaning that the result of a given operation may not be available for control flow. Deferred execution allows the Beam API to support parallel processing of data.</p> +<p>A feature of the Beam execution model. Beam operations are deferred, meaning that the result of a given operation may not be available for control flow. Deferred execution allows the Beam API to support parallel processing of data and perform pipeline-level optimizations.</p> <h2 id="distribution-metric">Distribution (metric)</h2> <p>A metric that reports information about the distribution of reported values. In the Beam model, metrics provide insight into the state of a pipeline, potentially while the pipeline is running.</p> <p>To learn more, see:</p> @@ -3406,7 +3406,7 @@ limitations under the License. <li><a href="/documentation/programming-guide/#types-of-metrics">Types of metrics</a></li> </ul> <h2 id="dofn">DoFn</h2> -<p>A function object used by ParDo (or some other transform) to process the elements of a PCollection. A DoFn is a user-defined function, meaning that it contains custom code that defines a data processing task in your pipeline. The Beam system invokes a DoFn one or more times to process some arbitrary bundle of elements, but Beam doesn’t guarantee an exact number of invocations.</p> +<p>A function object used by ParDo (or some other transform) to process the elements of a PCollection, often producing elements for an output PCollection. A DoFn is a user-defined function, meaning that it contains custom code that defines a data processing task in your pipeline. The Beam system invokes a DoFn one or more times to process some arbitrary bundle of elements, but Beam doesn’t guarantee an exact number of invocations.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#pardo">ParDo</a></li> @@ -3433,14 +3433,14 @@ limitations under the License. <h2 id="engine">Engine</h2> <p>A data-processing system, such as Dataflow, Spark, or Flink. A Beam runner for an engine executes a Beam pipeline on that engine.</p> <h2 id="event-time">Event time</h2> -<p>The time a data event occurs, determined by a timestamp on an element. This is in contrast to processing time, which is when an element is processed in a pipeline. An event could be, for example, a user interaction or a write to an error log. There’s no guarantee that events will appear in a pipeline in order of event time.</p> +<p>The time a data event occurs, determined by a timestamp on an element. This is in contrast to processing time, which is when an element is processed in a pipeline. An event could be, for example, a user interaction or a write to an error log. There’s no guarantee that events will appear in a pipeline in order of event time, but windowing and timers let you reason correctly about event time.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#watermarks-and-late-data">Watermarks and late data</a></li> <li><a href="/documentation/programming-guide/#triggers">Triggers</a></li> </ul> <h2 id="expansion-service">Expansion Service</h2> -<p>A service that enables a pipeline to apply (expand) cross-language transforms defined in other SDKs. For example, by connecting to a Java expansion service, the Python SDK can apply transforms implemented in Java. Currently SDKs define expansion services as local processes, but in the future Beam may support long-running expansion services. The development of expansion services is part of the ongoing effort to support multi-language pipelines.</p> +<p>A service that enables a pipeline to apply (expand) cross-language transforms defined in other SDKs. For example, by connecting to a Java expansion service, the Python SDK can apply transforms implemented in Java. Currently, SDKs typically start up expansion services as local processes, but in the future Beam may support long-running expansion services. The development of expansion services is part of the ongoing effort to support multi-language pipelines.</p> <h2 id="flatten">Flatten</h2> <p>One of the core PTransforms. Flatten merges multiple PCollections into a single logical PCollection.</p> <p>To learn more, see:</p> @@ -3449,8 +3449,10 @@ limitations under the License. <li><a href="/documentation/transforms/java/other/flatten/">Flatten (Java)</a></li> <li><a href="/documentation/transforms/python/other/flatten/">Flatten (Python)</a></li> </ul> +<h2 id="fn-api">Fn API</h2> +<p>An interface that lets a runner invoke SDK-specific user-defined functions. The Fn API, together with the Runner API, supports the ability to mix and match SDKs and runners. Used together, the Fn and Runner APIs let new SDKs run on every runner, and let new runners run pipelines from every SDK.</p> <h2 id="fusion">Fusion</h2> -<p>An optimization that Beam runners can apply before running a pipeline. When one transform outputs a PCollection that’s consumed by another transform, or when two or more transforms take the same PCollection as input, a runner may be able to fuse the transforms together into a single processing unit (a <em>stage</em> in Dataflow). Fusion can make pipeline execution more efficient by preventing I/O operations.</p> +<p>An optimization that Beam runners can apply before running a pipeline. When one transform outputs a PCollection that’s consumed by another transform, or when two or more transforms take the same PCollection as input, a runner may be able to fuse the transforms together into a single processing unit (a <em>stage</em> in Dataflow). The consuming DoFn processes elements as they are emitted by the producing DoFn, rather than waiting for the entire intermediate PCollection to be c [...] <h2 id="gauge-metric">Gauge (metric)</h2> <p>A metric that reports the latest value out of reported values. In the Beam model, metrics provide insight into the state of a pipeline, potentially while the pipeline is running. Because metrics are collected from many workers, the gauge value may not be the absolute last value, but it will be one of the latest values produced by one of the workers.</p> <p>To learn more, see:</p> @@ -3473,7 +3475,7 @@ limitations under the License. <li><a href="/documentation/io/built-in/">Built-in I/O Transforms</a></li> </ul> <h2 id="map">Map</h2> -<p>An element-wise PTransform that applies a user-defined function (UDF) to each element in a PCollection. Using Map, you can transform each individual element, but you can&rsquo;t change the number of elements.</p> +<p>An element-wise PTransform that applies a user-defined function (UDF) to each element in a PCollection. Using Map, you can transform each individual element into a new element, but you can&rsquo;t change the number of elements.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/transforms/python/elementwise/map/">Map (Python)</a></li> @@ -3492,7 +3494,7 @@ limitations under the License. <li><a href="/documentation/programming-guide/#multi-language-pipelines">Multi-language pipelines</a></li> </ul> <h2 id="pardo">ParDo</h2> -<p>The lowest-level element-wise PTransform. For each element in an input PCollection, ParDo applies a function and emits zero, one, or multiple elements to an output PCollection. “ParDo” is short for “Parallel Do.” It’s similar to the map operation in a <a href="https://en.wikipedia.org/wiki/MapReduce">MapReduce</a> algorithm, the <code>apply</code> method from a DataFrame, or the <code>UPDATE</code> keyword from SQL.</p> +<p>The lowest-level element-wise PTransform. For each element in an input PCollection, ParDo applies a function and emits zero, one, or multiple elements to an output PCollection. “ParDo” is short for “Parallel Do.” It’s similar to the map operation in a <a href="https://en.wikipedia.org/wiki/MapReduce">MapReduce</a> algorithm and the reduce operation when following a GroupByKey. ParDo is also comparable to the <code>apply</code> method from a DataFrame, or the <code>UP [...] <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#pardo">ParDo</a></li> @@ -3500,7 +3502,7 @@ limitations under the License. <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo (Python)</a></li> </ul> <h2 id="partition">Partition</h2> -<p>An element-wise PTransform that splits a single PCollection into a fixed number of smaller PCollections. Partition requires a user-defined function (UDF) to determine how to split up the elements of the input collection into the resulting output collections. The number of partitions must be determined at graph construction time, meaning that you can’t determine the number of partitions using data calculated by the running pipeline.</p> +<p>An element-wise PTransform that splits a single PCollection into a fixed number of smaller, disjoint PCollections. Partition requires a user-defined function (UDF) to determine how to split up the elements of the input collection into the resulting output collections. The number of partitions must be determined at graph construction time, meaning that you can’t determine the number of partitions using data calculated by the running pipeline.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#partition">Partition</a></li> @@ -3514,13 +3516,13 @@ limitations under the License. <li><a href="/documentation/programming-guide/#pcollections">PCollections</a></li> </ul> <h2 id="pipe-operator-">Pipe operator (<code>|</code>)</h2> -<p>Delimits a step in a Python pipeline. For example: <code>[Final Output PCollection] = ([Initial Input PCollection] | [First Transform] | [Second Transform] | [Third Transform])</code>. The output of each transform is passed from left to right as input to the next transform. The pipe operator in Python is equivalent to the <code>apply</code> method in Java (in other words, the pipe applies a transform to a PCollection).</p> +<p>Delimits a step in a Python pipeline. For example: <code>[Final Output PCollection] = ([Initial Input PCollection] | [First Transform] | [Second Transform] | [Third Transform])</code>. The output of each transform is passed from left to right as input to the next transform. The pipe operator in Python is equivalent to the <code>apply</code> method in Java (in other words, the pipe applies a transform to a PCollection), and usage is similar to the pipe operator in shell [...] <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#applying-transforms">Applying transforms</a></li> </ul> <h2 id="pipeline">Pipeline</h2> -<p>An encapsulation of your entire data processing task, including reading input data from a source, transforming that data, and writing output data to a sink. You can think of a pipeline as a Beam program that uses PTransforms to process PCollections. The transforms in a pipeline can be represented as a directed acyclic graph (DAG). All Beam driver programs must create a pipeline.</p> +<p>An encapsulation of your entire data processing task, including reading input data from a source, transforming that data, and writing output data to a sink. You can think of a pipeline as a Beam program that uses PTransforms to process PCollections. (Alternatively, you can think of it as a single, executable composite PTransform with no inputs or outputs.) The transforms in a pipeline can be represented as a directed acyclic graph (DAG). All Beam driver programs must create a pipel [...] <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#overview">Overview</a></li> @@ -3529,7 +3531,7 @@ limitations under the License. <li><a href="/documentation/pipelines/create-your-pipeline/">Create your pipeline</a></li> </ul> <h2 id="processing-time">Processing time</h2> -<p>The time at which an element is processed at some stage in a pipeline. Processing time is not the same as event time, which is the time at which a data event occurs. Processing time is determined by the clock on the system processing the element. There’s no guarantee that elements will be processed in order of event time.</p> +<p>The real-world time at which an element is processed at some stage in a pipeline. Processing time is not the same as event time, which is the time at which a data event occurs. Processing time is determined by the clock on the system processing the element. There’s no guarantee that elements will be processed in order of event time.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#watermarks-and-late-data">Watermarks and late data</a></li> @@ -3556,21 +3558,21 @@ limitations under the License. <li><a href="/documentation/runners/capability-matrix/">Beam Capability Matrix</a></li> </ul> <h2 id="schema">Schema</h2> -<p>A language-independent type definition for a PCollection. The schema for a PCollection defines elements of that PCollection as an ordered list of named fields. Each field has a name, a type, and possibly a set of user options. Schemas provide a way to reason about types across different programming-language APIs.</p> +<p>A language-independent type definition for the elements of a PCollection. The schema for a PCollection defines elements of that PCollection as an ordered list of named fields. Each field has a name, a type, and possibly a set of user options. Schemas provide a way to reason about types across different programming-language APIs. They also let you describe data transformations more succinctly and at a higher level.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#schemas">Schemas</a></li> <li><a href="/documentation/patterns/schema/">Schema Patterns</a></li> </ul> <h2 id="session">Session</h2> -<p>A time interval for grouping data events. A session is defined by some minimum gap duration between events. For example, a data stream representing user mouse activity may have periods with high concentrations of clicks followed by periods of inactivity. A session can represent such a pattern of activity followed by inactivity.</p> +<p>A time interval for grouping data events. A session is defined by some minimum gap duration between events. For example, a data stream representing user mouse activity may have periods with high concentrations of clicks followed by periods of inactivity. A session can represent such a pattern of activity delimited by inactivity.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#session-windows">Session windows</a></li> <li><a href="/get-started/mobile-gaming-example/#analyzing-usage-patterns">Analyzing Usage Patterns</a></li> </ul> <h2 id="side-input">Side input</h2> -<p>Additional input to a PTransform. Side input is input that you provide in addition to the main input PCollection. A DoFn can access side input each time it processes an element in the PCollection. Side inputs are useful if your transform needs to inject additional data at runtime.</p> +<p>Additional input to a PTransform that is provided in its entirety, rather than element-by-element. Side input is input that you provide in addition to the main input PCollection. A DoFn can access side input each time it processes an element in the PCollection.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#side-inputs">Side inputs</a></li> @@ -3599,8 +3601,10 @@ limitations under the License. <li><a href="/documentation/programming-guide/#splittable-dofns">Splittable DoFns</a></li> <li><a href="/blog/splittable-do-fn-is-available/">Splittable DoFn in Apache Beam is Ready to Use</a></li> </ul> +<h2 id="stage">Stage</h2> +<p>The unit of fused transforms in a pipeline. Runners can perform fusion optimization to make pipeline execution more efficient. In Dataflow, the pipeline is conceptualized as a graph of fused stages.</p> <h2 id="state">State</h2> -<p>Persistent values that a PTransform can access. The state API lets you augment element-wise operations (for example, ParDo or Map) with mutable state. Using the state API, you can read from, and write to, state as you process each element of a PCollection. You can use the state API together with the timer API to create processing tasks that give you fine-grained control over the workflow.</p> +<p>Persistent values that a PTransform can access. The state API lets you augment element-wise operations (for example, ParDo or Map) with mutable state. Using the state API, you can read from, and write to, state as you process each element of a PCollection. You can use the state API together with the timer API to create processing tasks that give you fine-grained control over the workflow. State is always local to a key and window.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#state-and-timers">State and Timers</a></li> @@ -3622,7 +3626,7 @@ limitations under the License. <li><a href="/blog/timely-processing/">Timely (and Stateful) Processing with Apache Beam</a></li> </ul> <h2 id="timestamp">Timestamp</h2> -<p>A point in time associated with an element in a PCollection and used to assign a window to the element. The source that creates the PCollection assigns each element an initial timestamp, often corresponding to when the element was read or added. But you can also manually assign timestamps. This can be useful if elements have an inherent timestamp, but the timestamp is somewhere in the structure of the element itself (for example, a time field in a server log entry).</p> +<p>A point in event time associated with an element in a PCollection and used to assign a window to the element. The source that creates the PCollection assigns each element an initial timestamp, often corresponding to when the element was read or added. But you can also manually assign timestamps. This can be useful if elements have an inherent timestamp, but the timestamp is somewhere in the structure of the element itself (for example, a time field in a server log entry).</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#element-timestamps">Element timestamps</a></li> @@ -3637,7 +3641,7 @@ limitations under the License. <li><a href="/documentation/programming-guide/#triggers">Triggers</a></li> </ul> <h2 id="unbounded-data">Unbounded data</h2> -<p>A dataset of unlimited size. A PCollection can be bounded or unbounded, depending on the source of the data that it represents. Reading from a streaming or continuously-updating data source, such as Pub/Sub or Kafka, typically creates an unbounded PCollection.</p> +<p>A dataset that grows over time, with elements processed as they arrive. A PCollection can be bounded or unbounded, depending on the source of the data that it represents. Reading from a streaming or continuously-updating data source, such as Pub/Sub or Kafka, typically creates an unbounded PCollection.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#size-and-boundedness">Size and boundedness</a></li> @@ -3651,7 +3655,7 @@ limitations under the License. <li><a href="/documentation/programming-guide/#requirements-for-writing-user-code-for-beam-transforms">Requirements for writing user code for Beam transforms</a></li> </ul> <h2 id="watermark">Watermark</h2> -<p>The point in event time when all data in a window can be expected to have arrived in the pipeline. Watermarks provide a way to estimate the completeness of input data. Every PCollection has an associated watermark. Once the watermark progresses past the end of a window, any element that arrives with a timestamp in that window is considered late data.</p> +<p>An estimate on the lower bound of the timestamps that will be seen (in the future) at this point of the pipeline. Watermarks provide a way to estimate the completeness of input data. Every PCollection has an associated watermark. Once the watermark progresses past the end of a window, any element that arrives with a timestamp in that window is considered late data.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/programming-guide/#watermarks-and-late-data">Watermarks and late data</a></li> @@ -3663,7 +3667,7 @@ limitations under the License. <li><a href="/documentation/programming-guide/#windowing">Windowing</a></li> </ul> <h2 id="worker">Worker</h2> -<p>A container, process, or virtual machine (VM) that handles some part of the parallel processing of a pipeline. The Beam model doesn’t support synchronizing shared state across worker machines. Instead, each worker node has its own independent copy of state. A Beam runner might serialize elements between machines for communication purposes and for other reasons such as persistence.</p> +<p>A container, process, or virtual machine (VM) that handles some part of the parallel processing of a pipeline. Each worker node has its own independent copy of state. A Beam runner might serialize elements between machines for communication purposes and for other reasons such as persistence.</p> <p>To learn more, see:</p> <ul> <li><a href="/documentation/runtime/model/">Execution model</a></li> diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 1b084af..265a3d4 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.32.0/</loc><lastmod>2021-09-16T12:21:14-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-09-16T12:21:14-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-09-16T12:21:14-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-09-17T14:05:48-07:00</lastmod></url><url><loc>/blog/b [...] \ No newline at end of file +<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.32.0/</loc><lastmod>2021-09-16T12:21:14-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-09-16T12:21:14-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-09-16T12:21:14-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-09-17T14:05:48-07:00</lastmod></url><url><loc>/blog/b [...] \ No newline at end of file