Portable artifact retrieval using URLs

[Steve Niemitz]

We're working on running an "expansion service as a service" for xlang
transforms.  One of the things we'd really like is to serve the actual
required artifacts to the client (submitting the pipeline) from our
blobstore rather than streaming it through the artifact retrieval API
(GetArtifact).

I have the expansion service returning the required artifact URLs when
expanding the transforms, but after that I've run into some confusion on
how it's supposed to work from there.

Using the direct runner, the local artifact retrieval service [1] will
correctly download the URL resource. However, trying to run in dataflow
will cause the stager to break since it only supports FILE types [2].

I _think_ what should happen is that in
artifact_service.maybe_store_artifact [3] we should download URL artifacts
locally and replace them with a temp file resource (similar to how
store_resource) works.  I made these changes locally and was able to then
submit jobs both via the direct runner and dataflow.

Any thoughts here?

[1]
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/artifact_service.py#L76

[2]
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py#L579

[3]
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/artifact_service.py#L284

Re: Portable artifact retrieval using URLs

[Steve Niemitz]

> but it was always the intent to have Dataflow support artifact types
beyond FILE.

Would you envision dataflow re-staging the artifacts to GCS, or just
passing them through?  In our case I'd rather the former, since it keeps a
job self-contained to GCS and not dependent on external resources when it
launches.

What's the appetite for a PR that changes the behavior of
artifact_service.maybe_store_artifact to download the URL artifact locally
first? (or changing the dataflow stager to do so).  From what I can tell
it's currently ~impossible to actually provide a URL resource through any
standard mechanism, so I doubt that'd be a breaking change.

Custom containers are certainly another option for us here too, but seem
more complicated to manage, I was hoping to start simple and move on to
them later when we really needed the complexity.

On Tue, Nov 2, 2021 at 12:12 PM Luke Cwik  wrote:

> What you are suggesting will work but it was always the intent to have
> Dataflow support artifact types beyond FILE.
>
> Another option would be to use custom containers with Dataflow and instead
> build the container embedding all (or most) of the artifacts that you need.
> This will help speed up how fast the workers start in Dataflow in addition
> to not having to workaround the fact that only FILE is supported.
>
> On Tue, Nov 2, 2021 at 8:03 AM Steve Niemitz  wrote:
>
>> We're working on running an "expansion service as a service" for xlang
>> transforms.  One of the things we'd really like is to serve the actual
>> required artifacts to the client (submitting the pipeline) from our
>> blobstore rather than streaming it through the artifact retrieval API
>> (GetArtifact).
>>
>> I have the expansion service returning the required artifact URLs when
>> expanding the transforms, but after that I've run into some confusion on
>> how it's supposed to work from there.
>>
>> Using the direct runner, the local artifact retrieval service [1] will
>> correctly download the URL resource. However, trying to run in dataflow
>> will cause the stager to break since it only supports FILE types [2].
>>
>> I _think_ what should happen is that in
>> artifact_service.maybe_store_artifact [3] we should download URL artifacts
>> locally and replace them with a temp file resource (similar to how
>> store_resource) works.  I made these changes locally and was able to then
>> submit jobs both via the direct runner and dataflow.
>>
>> Any thoughts here?
>>
>> [1]
>> https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/artifact_service.py#L76
>>
>> [2]
>> https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py#L579
>>
>> [3]
>> https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/artifact_service.py#L284
>>
>

Beam Contributer

[Mostafa Aghajani]

Hi,

This is Mostafa Aghajani from Virta. I'm closely working on Java SDK (KafkaIO, 
RabbitMqIO).
Can someone add me as a contributor for Beam's Jira issue tracker?
I would like to create/assign tickets for my work.
My Jira ID: aghajani

+ BTW how can I join ASF on slack as suggested in the Beam Contribution Guide 
(https://beam.apache.org/contribute/)? It only accepts emails from @apache.org 🙂

Best Regards,
Mostafa Aghajani

Re: Beam Contributer

[Moritz Mack]

Hi,

Welcome 😊
I don’t have permissions to manage Jira, but you should have received an invite 
for Slack.

Best,
Moritz

From: Mostafa Aghajani 
Reply to: "dev@beam.apache.org" 
Date: Tuesday, 2. November 2021 at 17:27
To: "dev@beam.apache.org" 
Subject: Beam Contributer

Warning! External email. Exercise caution when opening attachments or clicking 
any links.




Hi,

This is Mostafa Aghajani from Virta. I'm closely working on Java SDK (KafkaIO, 
RabbitMqIO).
Can someone add me as a contributor for Beam's Jira issue tracker?
I would like to create/assign tickets for my work.
My Jira ID: aghajani

+ BTW how can I join ASF on slack as suggested in the Beam Contribution Guide 
(https://beam.apache.org/contribute/)?
 It only accepts emails from @apache.org 🙂

Best Regards,
Mostafa Aghajani

As a recipient of an email from Talend, your contact personal data will be on 
our systems. Please see our privacy notice (updated August 2020) at Talend, 
Inc. 

Re: Beam Contributer

[Mostafa Aghajani]

Yes I did and joined using the link! 🎉
Thanks

Best Regards
Mostafa Aghajani

From: Moritz Mack 
Sent: Tuesday, November 2, 2021 7:01:05 PM
To: dev@beam.apache.org 
Subject: Re: Beam Contributer


Hi,



Welcome 😊

I don’t have permissions to manage Jira, but you should have received an invite 
for Slack.



Best,

Moritz



From: Mostafa Aghajani 
Reply to: "dev@beam.apache.org" 
Date: Tuesday, 2. November 2021 at 17:27
To: "dev@beam.apache.org" 
Subject: Beam Contributer



Warning! External email. Exercise caution when opening attachments or clicking 
any links.





Hi,



This is Mostafa Aghajani from Virta. I'm closely working on Java SDK (KafkaIO, 
RabbitMqIO).

Can someone add me as a contributor for Beam's Jira issue tracker?

I would like to create/assign tickets for my work.

My Jira ID: aghajani



+ BTW how can I join ASF on slack as suggested in the Beam Contribution Guide 
(https://beam.apache.org/contribute/)?
 It only accepts emails from @apache.org 🙂



Best Regards,

Mostafa Aghajani

As a recipient of an email from Talend, your contact personal data will be on 
our systems. Please see our privacy notice (updated August 2020) at Talend, 
Inc. 

Re: Beam Contributer

[Alexey Romanenko]

Hi Mostafa, 

I added you to contributors list.
Welcome to Beam!

—
Alexey

> On 2 Nov 2021, at 18:17, Mostafa Aghajani  wrote:
> 
> Yes I did and joined using the link! 🎉
> Thanks
> 
> Best Regards
> Mostafa Aghajani
> From: Moritz Mack 
> Sent: Tuesday, November 2, 2021 7:01:05 PM
> To: dev@beam.apache.org 
> Subject: Re: Beam Contributer
>  
> Hi,
>  
> Welcome 😊
> I don’t have permissions to manage Jira, but you should have received an 
> invite for Slack.
>  
> Best, 
> Moritz 
>  
> From: Mostafa Aghajani 
> Reply to: "dev@beam.apache.org" 
> Date: Tuesday, 2. November 2021 at 17:27
> To: "dev@beam.apache.org" 
> Subject: Beam Contributer
>  
> Warning! External email. Exercise caution when opening attachments or 
> clicking any links.
> 
> 
> 
> 
> 
> Hi, 
>  
> This is Mostafa Aghajani from Virta. I'm closely working on Java SDK 
> (KafkaIO, RabbitMqIO).
> Can someone add me as a contributor for Beam's Jira issue tracker?
> I would like to create/assign tickets for my work.
> My Jira ID: aghajani
>  
> + BTW how can I join ASF on slack as suggested in the Beam Contribution Guide 
> (https://beam.apache.org/contribute/ 
> )?
>  It only accepts emails from @apache.org 🙂
>  
> Best Regards,
> Mostafa Aghajani
> As a recipient of an email from Talend, your contact personal data will be on 
> our systems. Please see our privacy notice (updated August 2020) at Talend, 
> Inc.  
> 

P1 issues report (56)

[Beam Jira Bot]

This is your daily summary of Beam's current P1 issues, not including flaky 
tests 
(https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20priority%20%3D%20P1%20AND%20(labels%20is%20EMPTY%20OR%20labels%20!%3D%20flake).

See https://beam.apache.org/contribute/jira-priorities/#p1-critical for the 
meaning and expectations around P1 issues.

https://issues.apache.org/jira/browse/BEAM-13164: 
beam_PostCommit_Java_PVR_Spark_Batch timing out (created 2021-11-01)
https://issues.apache.org/jira/browse/BEAM-13137: make 
ElasticsearchIO$BoundedElasticsearchSource#getEstimatedSizeBytes deterministic 
(created 2021-10-28)
https://issues.apache.org/jira/browse/BEAM-13118: Nightly build wheels jobs 
failing (created 2021-10-26)
https://issues.apache.org/jira/browse/BEAM-13098: Repeated fields not 
translated correctly in BigQuery Storage API sink (created 2021-10-21)
https://issues.apache.org/jira/browse/BEAM-13087: 
apache_beam.runners.portability.fn_api_runner.translations_test.TranslationsTest.test_run_packable_combine_globally
 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible 
(created 2021-10-20)
https://issues.apache.org/jira/browse/BEAM-13078: Python DirectRunner does 
not emit data at GC time (created 2021-10-18)
https://issues.apache.org/jira/browse/BEAM-13076: Python AfterAny, AfterAll 
do not follow spec (created 2021-10-18)
https://issues.apache.org/jira/browse/BEAM-13074: Metrics are not reported 
by the Flink runner (created 2021-10-18)
https://issues.apache.org/jira/browse/BEAM-13060: Daily Python SDK build is 
not publicly accessible (created 2021-10-15)
https://issues.apache.org/jira/browse/BEAM-13059: Migrate GKE workloads to 
Containerd (created 2021-10-15)
https://issues.apache.org/jira/browse/BEAM-13058: Upgrade Kubernetes APIs 
(created 2021-10-15)
https://issues.apache.org/jira/browse/BEAM-13056: Add method to fetch 
ProcessContext FieldAccess (created 2021-10-14)
https://issues.apache.org/jira/browse/BEAM-13025: 
beam_PostCommit_Java_DataflowV2 failing pubsublite.ReadWriteIT (created 
2021-10-08)
https://issues.apache.org/jira/browse/BEAM-13010: Delete orphaned files 
(created 2021-10-06)
https://issues.apache.org/jira/browse/BEAM-12995: Consumer group with 
random prefix (created 2021-10-04)
https://issues.apache.org/jira/browse/BEAM-12959: Dataflow error in 
CombinePerKey operation (created 2021-09-26)
https://issues.apache.org/jira/browse/BEAM-12867: Either Create or 
DirectRunner fails to produce all elements to the following transform (created 
2021-09-09)
https://issues.apache.org/jira/browse/BEAM-12843: (Broken Pipe induced) 
Bricked Dataflow Pipeline  (created 2021-09-06)
https://issues.apache.org/jira/browse/BEAM-12818: When writing to GCS, 
spread prefix of temporary files and reuse autoscaling of the temporary 
directory (created 2021-08-30)
https://issues.apache.org/jira/browse/BEAM-12807: Java creates an incorrect 
pipeline proto when core-construction-java jar is not in the CLASSPATH (created 
2021-08-26)
https://issues.apache.org/jira/browse/BEAM-12792: Beam worker only installs 
--extra_package once (created 2021-08-24)
https://issues.apache.org/jira/browse/BEAM-12766: Already Exists: Dataset 
apache-beam-testing:python_bq_file_loads_NNN (created 2021-08-16)
https://issues.apache.org/jira/browse/BEAM-12632: ElasticsearchIO: Enabling 
both User/Pass auth and SSL overwrites User/Pass (created 2021-07-16)
https://issues.apache.org/jira/browse/BEAM-12540: 
beam_PostRelease_NightlySnapshot - Task 
:runners:direct-java:runMobileGamingJavaDirect FAILED (created 2021-06-25)
https://issues.apache.org/jira/browse/BEAM-12525: SDF BoundedSource seems 
to execute significantly slower than 'normal' BoundedSource (created 2021-06-22)
https://issues.apache.org/jira/browse/BEAM-12505: codecov/patch has poor 
behavior (created 2021-06-17)
https://issues.apache.org/jira/browse/BEAM-12500: Dataflow SocketException 
(SSLException) error while trying to send message from Cloud Pub/Sub to 
BigQuery (created 2021-06-16)
https://issues.apache.org/jira/browse/BEAM-12484: JdbcIO date conversion is 
sensitive to OS (created 2021-06-14)
https://issues.apache.org/jira/browse/BEAM-12467: 
java.io.InvalidClassException With Flink Kafka (created 2021-06-09)
https://issues.apache.org/jira/browse/BEAM-12279: Implement 
destination-dependent sharding in FileIO.writeDynamic (created 2021-05-04)
https://issues.apache.org/jira/browse/BEAM-12256: 
PubsubIO.readAvroGenericRecord creates SchemaCoder that fails to decode some 
Avro logical types (created 2021-04-29)
https://issues.apache.org/jira/browse/BEAM-11959: Python Beam SDK Harness 
hangs when installing pip packages (created 2021-03-11)
https://issues.apache.org/jira/browse/BEAM-11906: No trigger early 
repeatedly for session windows (created 2021-03-01)
 

[GitHub] [beam-site] ibzib commented on pull request #620: Update beam-site for release 2.34.0.

[GitBox]

ibzib commented on pull request #620:
URL: https://github.com/apache/beam-site/pull/620#issuecomment-956562631


   R: @udim 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@beam.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Flaky test issue report (30)

[Beam Jira Bot]

This is your daily summary of Beam's current flaky tests 
(https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20labels%20%3D%20flake)

These are P1 issues because they have a major negative impact on the community 
and make it hard to determine the quality of the software.

https://issues.apache.org/jira/browse/BEAM-13083: XVR Flink and XVR Spark 
failing (created 2021-10-20)
https://issues.apache.org/jira/browse/BEAM-13025: 
beam_PostCommit_Java_DataflowV2 failing pubsublite.ReadWriteIT (created 
2021-10-08)
https://issues.apache.org/jira/browse/BEAM-12928: beam_PostCommit_Python36 
- CrossLanguageSpannerIOTest - flakey failing (created 2021-09-21)
https://issues.apache.org/jira/browse/BEAM-12859: 
org.apache.beam.runners.dataflow.worker.fn.logging.BeamFnLoggingServiceTest.testMultipleClientsFailingIsHandledGracefullyByServer
 is flaky (created 2021-09-08)
https://issues.apache.org/jira/browse/BEAM-12809: 
testTwoTimersSettingEachOtherWithCreateAsInputBounded flaky (created 2021-08-26)
https://issues.apache.org/jira/browse/BEAM-12794: 
PortableRunnerTestWithExternalEnv.test_pardo_timers flaky (created 2021-08-24)
https://issues.apache.org/jira/browse/BEAM-12766: Already Exists: Dataset 
apache-beam-testing:python_bq_file_loads_NNN (created 2021-08-16)
https://issues.apache.org/jira/browse/BEAM-12540: 
beam_PostRelease_NightlySnapshot - Task 
:runners:direct-java:runMobileGamingJavaDirect FAILED (created 2021-06-25)
https://issues.apache.org/jira/browse/BEAM-12515: Python PreCommit flaking 
in PipelineOptionsTest.test_display_data (created 2021-06-18)
https://issues.apache.org/jira/browse/BEAM-12322: Python precommit flaky: 
Failed to read inputs in the data plane (created 2021-05-10)
https://issues.apache.org/jira/browse/BEAM-12320: 
PubsubTableProviderIT.testSQLSelectsArrayAttributes[0] failing in SQL 
PostCommit (created 2021-05-10)
https://issues.apache.org/jira/browse/BEAM-12291: 
org.apache.beam.runners.flink.ReadSourcePortableTest.testExecution[streaming: 
false] is flaky (created 2021-05-05)
https://issues.apache.org/jira/browse/BEAM-12200: 
SamzaStoreStateInternalsTest is flaky (created 2021-04-20)
https://issues.apache.org/jira/browse/BEAM-12163: Python GHA PreCommits 
flake with grpc.FutureTimeoutError on SDK harness startup (created 2021-04-13)
https://issues.apache.org/jira/browse/BEAM-12061: beam_PostCommit_SQL 
failing on KafkaTableProviderIT.testFakeNested (created 2021-03-27)
https://issues.apache.org/jira/browse/BEAM-11837: Java build flakes: 
"Memory constraints are impeding performance" (created 2021-02-18)
https://issues.apache.org/jira/browse/BEAM-11661: hdfsIntegrationTest 
flake: network not found (py38 postcommit) (created 2021-01-19)
https://issues.apache.org/jira/browse/BEAM-11641: Bigquery Read tests are 
flaky on Flink runner in Python PostCommit suites (created 2021-01-15)
https://issues.apache.org/jira/browse/BEAM-11541: 
testTeardownCalledAfterExceptionInProcessElement flakes on direct runner. 
(created 2020-12-30)
https://issues.apache.org/jira/browse/BEAM-10955: Flink Java Runner test 
flake: Could not find Flink job (FlinkJobNotFoundException) (created 2020-09-23)
https://issues.apache.org/jira/browse/BEAM-10866: 
PortableRunnerTestWithSubprocesses.test_register_finalizations flaky on macOS 
(created 2020-09-09)
https://issues.apache.org/jira/browse/BEAM-10485: Failure / flake: 
ElasticsearchIOTest > testWriteWithIndexFn (created 2020-07-14)
https://issues.apache.org/jira/browse/BEAM-9649: 
beam_python_mongoio_load_test started failing due to mismatched results 
(created 2020-03-31)
https://issues.apache.org/jira/browse/BEAM-8101: Flakes in 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundleStateful for 
Direct, Spark, Flink (created 2019-08-27)
https://issues.apache.org/jira/browse/BEAM-8035: 
WatchTest.testMultiplePollsWithManyResults flake: Outputs must be in timestamp 
order (sickbayed) (created 2019-08-22)
https://issues.apache.org/jira/browse/BEAM-7827: 
MetricsTest$AttemptedMetricTests.testAllAttemptedMetrics is flaky on 
DirectRunner (created 2019-07-26)
https://issues.apache.org/jira/browse/BEAM-7752: Java Validates 
DirectRunner: testTeardownCalledAfterExceptionInFinishBundleStateful flaky 
(created 2019-07-16)
https://issues.apache.org/jira/browse/BEAM-6804: [beam_PostCommit_Java] 
[PubsubReadIT.testReadPublicData] Timeout waiting on Sub (created 2019-03-11)
https://issues.apache.org/jira/browse/BEAM-5286: 
[beam_PostCommit_Java_GradleBuild][org.apache.beam.examples.subprocess.ExampleEchoPipelineTest.testExampleEchoPipeline][Flake]
 .sh script: text file busy. (created 2018-09-01)
https://issues.apache.org/jira/browse/BEAM-5172: 
org.apache.beam.sdk.io.elasticsearch/ElasticsearchIOTest is flaky (created 
2018-08-20)

[GitHub] [beam-site] ibzib commented on pull request #620: Update beam-site for release 2.34.0.

[GitBox]

ibzib commented on pull request #620:
URL: https://github.com/apache/beam-site/pull/620#issuecomment-956562631


   R: @udim 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@beam.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: Best practices for upgrading installed dependencies on Jenkins VMs?

[Robert Burke]

TIL as well. Sounds like the right location. Thanks Valentyn!

On Tue, Nov 2, 2021, 11:00 AM Valentyn Tymofieiev 
wrote:

> Yeah,  .profile is only sourced by login shells. Adding the PATH in
> .bashrc can be a workaround, but since .bashrc is executed every time a new
> shell runs, PATH variable will be growing with every shell subprocess, so
> several sources recommend .profile instead, which does not always work.
> We should be able to fix this by updating  /etc/environment instead (TIL).
>
> This is the current content:
> cat /etc/environment
>
> PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
>
>
>
>
> On Mon, Nov 1, 2021 at 10:50 AM Robert Burke  wrote:
>
>> Looks like while .profile was edited to add in a PATH section pointing to
>> /snap/bin (where go is now installed), it doesn't seem like .profile is
>> executed by the jenkins login shells.
>>
>>
>>
>> On Fri, Oct 29, 2021, 6:23 PM Valentyn Tymofieiev 
>> wrote:
>>
>>>
>>>
>>> On Wed, Oct 20, 2021 at 11:16 AM Valentyn Tymofieiev <
>>> valen...@google.com> wrote:
>>>


 On Wed, Oct 20, 2021 at 11:12 AM Pablo Estrada 
 wrote:

> Thanks everyone for investigating and documenting this. I'll use it
> today : )
>
 Dan may be also in the middle of doing this, please coordinate.

>
> ahem - maybe we should rename the image name/image family names
> to jenkins-worker-boot-image ? Does anyone foresee issues if we do that?
> Does jenkins depend on these names in some undocumented way?
>
 +1. it should 'just work', need to update the wiki after the change.
 Jenkins also did a terminology adjustment.

>>> I had to reimage Jenkins workers again, took care of the rename and
>>> changed the instructions.
>>>
>>> I am not sure what is the status of Go Postcommit problem, but noticed
>>> that jenkins worker #1 had a different boot disk. I reimaged all workers
>>> building on top of the latest image from the image family. If Go tests
>>> start failing, we may need to get help from Dan again.
>>>
>>>

> On Tue, Oct 19, 2021 at 1:43 PM Daniel Oliveira <
> danolive...@google.com> wrote:
>
>> I'm ok with deciding to avoid the "lite" update option, feel free to
>> revise the instructions as it seems appropriate. As for the issue, I 
>> fixed
>> it with a workaround that should work until we need to add a new image to
>> the agents, and I'm currently investigating the root cause and prepare a
>> fixed image.
>>
>> That said, I think this issue would have still happened even if we
>> didn't perform the "lite" update. I'm still trying to figure out the 
>> exact
>> problem, but it looks to be a PATH issue that wasn't effectively caught 
>> by
>> the current process. I won't get into details too much in this thread 
>> (see
>> the Jira for that), but essentially everything works in my environment 
>> when
>> I SSH into the VMs, but because the location of the "go" command changed 
>> in
>> the PATH, it seems to have stopped working for every other user, 
>> including
>> the Jenkins agents. I actually did notice that would happen when I was
>> working on the image, but the solution seemed to be to reboot the 
>> machine,
>> which I assumed happened already since I shut down the VM to image it.
>>
>> On Tue, Oct 19, 2021 at 12:09 PM Robert Burke 
>> wrote:
>>
>>> +1 to only having one way to do things. The Lite option seems liable
>>> to cause more problems since it means it's changes can be blown away if 
>>> a
>>> new image isn't prepared anyway.
>>> I don't think we are changing the images often enough for it.
>>> Perhaps call it the option to test changes if anything?
>>>
>>> On Tue, Oct 19, 2021, 11:55 AM Valentyn Tymofieiev <
>>> valen...@google.com> wrote:
>>>
 All workers were updated to use jenkins-slave-boot-image-20211011,
 which should have had a go command, but it appears slightly 
 misconfigured.
 I reopened BEAM-13037 [1] and added some details there.

 I also added instructions to wiki [2] on how to perform an image
 swap and it is actually very straightforward. I think a lesson here is 
 that
 making 'lite' upgrades is brittle as misconfigurations could resurface 
 down
 the road when the context of the lite upgrade is no longer fresh in our
 memory.

 I suggest we revise the instructions to keep only image swap
 commands and remove the 'lite' update option. +Daniel Oliveira
 , WDYT?  In the meantime, we should also
 prepare an image that fixes the misconfiguration. Would you be able to 
 help
 with that? Thank you.

 [1] https://issues.apache.org/jira/browse/BEAM-13037
 [2]
 https://cwiki.apache