+1 on consolidating and being consistent with our terms. I've always considered them (Runner/Engine) synonymous. From a user perspective, an engine without a runner isn't any good for their beam pipeline. That there's an adapter is an implementation detail in some instances. I do appreciate not using Adapter a term, avoiding confusing descriptions.
However, if we make the change and there's a clear glossary of terms somewhere then That puts the lifecycle of a pipeline to be (loosely) something like... A Beam User authors Pipelines by writing DoFns, adding them as PTransforms connected by PCollections into a Pipeline using a Beam SDK. An SDK converts the pipeline into a portable representation, and submit it to the Job Management Service of a Beam Runner. A Beam Runner translates the portable pipeline representation into terms an underlying Engine understands for Execution. The Beam Runner also reverses this translation when the Engine delegates tasks to workers, so that the Beam SDKs can execute the user's DoFns in keeping with the Beam Semantics. (Not covered, bundles etc, but you get the idea...) On Wed, Jan 6, 2021, 11:16 AM Robert Bradshaw <rober...@google.com> wrote: > +1 to keeping the distinction between Runner and Engine as Kenn described, > and cleaning up the site with these in mind (I don't think the term engine > is widely used yet). > > On Wed, Jan 6, 2021 at 11:15 AM Yichi Zhang <zyi...@google.com> wrote: > >> I agree with what kenn said, in most cases I would refer to the term >> runner as the adapter for translating user's pipeline code into a job >> representation and submitting it to the execution engine. Though in some >> cases they may still be used interchangeably such as direct runner? >> >> On Wed, Jan 6, 2021 at 11:02 AM Kenneth Knowles <k...@apache.org> wrote: >> >>> I personally try to always distinguish two concepts: the thing doing the >>> computing (like Spark or Flink), and the adapter for running a Beam >>> pipeline (like SparkRunner or FlinkRunner). I use the term "runner" to mean >>> the adapter, and have been trying to use the term "engine" to refer to the >>> thing doing the computing. Do you think that users will use these two >>> interchangeably? Do you have recommendations about if these terms makes >>> sense to users? >>> >>> Kenn >>> >>> On Wed, Jan 6, 2021 at 10:23 AM Griselda Cuevas <g...@apache.org> wrote: >>> >>>> Hi dev@ community, Happy New Year! >>>> >>>> I'm working on updating the copy of a few website pages, and something >>>> that I want to solve is standardize how we refer to runners across the >>>> site. So far I've identified these definitions: >>>> >>>> - Back-end >>>> - Backend systems >>>> - Execution environments >>>> - Runtime >>>> - Runtime system >>>> - Runner >>>> >>>> Even when the majority of users will understand these concepts >>>> interchangeably, it's a good idea to be consistent so new users get >>>> familiar with how Beam works and its components. >>>> >>>> I'm going to start using the word "Runner" as I update the copy and >>>> will ask the team working in te UI revamp to do the same. Let me know if >>>> you have any questions/concerns. >>>> >>>
