I also ran portable wordcount on Spark and Flink runners using docker container images that we are releasing with 2.16.0. I was using the SDK code from RC1 tag in github, and pulled the container image from Docker repo as follows:
git checkout tags/v2.16.0-RC1 ./gradlew :runners:spark:job-server:runShadow or ./gradlew :runners:flink:1.5:job-server:runShadow In a separate terminal: docker pull apachebeam/python3.5_sdk:2.16.0_rc1 docker tag apachebeam/python3.5_sdk:2.16.0_rc1 apachebeam/python3.5_sdk:2.16.0 ./gradlew :sdks:python:test-suites:portable:py35:portableWordCountBatch -PjobEndpoint=localhost:8099 -PenvirionmentType=DOCKER As soon as that is done: docker ps -a | grep apachebeam Note the container ID of a running container. It stays around only for a minute or so, and gets garbage-collected... docker exec 866fb8932207 /bin/bash -c 'cat /tmp/py-wordcount*' <wordcount output is printed> Still voting +1. On Fri, Oct 4, 2019 at 12:41 PM Kenneth Knowles <k...@apache.org> wrote: > +1 (binding) > > - Reviewed what verifications had been done. Nice. > - Also did a gradle build of some targets in the archival source release > > The source release still does not build as a whole, as it has not since > 2.9.0 it seems. It is not as simple as excluding website from the build, > because it fails at configure time. Since particular artifacts can build, > it is not a blocker, but I've taken > https://issues.apache.org/jira/browse/BEAM-6228 and upgraded to critical > and put 2.17.0 as Release Version. > > Kenn > > On Fri, Oct 4, 2019 at 10:27 AM Pablo Estrada <pabl...@google.com> wrote: > >> Hi all, >> I looked at https://issues.apache.org/jira/browse/BEAM-8303, and it >> seems like the user has a workaround - is that correct? >> If that's the case, then I vote +1. >> >> @Max - lmk if you'd like to discuss further, but for now my vote is on +1. >> Best >> -P. >> >> On Fri, Oct 4, 2019 at 9:29 AM Mark Liu <mark...@google.com> wrote: >> >>> +1 (forgot to vote) >>> >>> I also triggered Java Nexmark on direct, dataflow, spark and flink >>> runner. Didn't saw performance regression from the dashboard ( >>> https://apache-beam-testing.appspot.com/dashboard-admin) >>> >>> On Fri, Oct 4, 2019 at 8:23 AM Mark Liu <mark...@google.com> wrote: >>> >>>> Thanks for the validation work! I validated following: >>>> >>>> - Java Quickstart on direct, dataflow,spark local, flink local runner >>>> - Java mobile gaming on direct and dataflow runner >>>> - Python Quickstart in batch and streaming in py2/3.5/3.6/3.7 using >>>> wheals/zip >>>> - Python Mobile Game in batch/streaming in py2/3.5/3.6/3.7 using >>>> wheals/zip on direct and dataflow runner >>>> >>>> Mark >>>> >>>> On Thu, Oct 3, 2019 at 6:57 PM Ahmet Altay <al...@google.com> wrote: >>>> >>>>> I see most of the release validations have been completed and marked >>>>> in the spreadsheet. Thank you all for doing that. If you have not >>>>> validated/voted yet please take a look at the release candidate. >>>>> >>>>> On Thu, Oct 3, 2019 at 7:59 AM Thomas Weise <t...@apache.org> wrote: >>>>> >>>>>> I think there is a different reason why the release manager should >>>>>> probably merge/approve all PRs that go into the release branch while the >>>>>> release is in progress: >>>>>> >>>>>> If/when the need arises for another RC, then only those changes >>>>>> should be included that are deemed blockers or explicitly agreed. >>>>>> Otherwise >>>>>> the release can potentially be delayed by modifications that invalidate >>>>>> prior verification or introduce new instability. >>>>>> >>>>> >>>>> I agree with this reasoning. It expresses my concern in a more clear >>>>> way. >>>>> >>>>> >>>>>> >>>>>> Thomas >>>>>> >>>>>> >>>>>> On Thu, Oct 3, 2019 at 3:12 AM Maximilian Michels <m...@apache.org> >>>>>> wrote: >>>>>> >>>>>>> > For the next time, may I suggest asking release manager to do the >>>>>>> > merging to the release branch. We do not know whether there will >>>>>>> be an >>>>>>> > RC2 or not. And if there will not be an RC2 release branch as of >>>>>>> now >>>>>>> > does not directly correspond to what will be released. >>>>>>> >>>>>>> The ground truth for releases are the release tags, not the release >>>>>>> branches. Downstream projects should not depend on the release >>>>>>> branches. >>>>>>> Release branches are merely important for the process of creating a >>>>>>> release, but they lose validity after the RC has been created and >>>>>>> released. >>>>>>> >>>>>>> On 02.10.19 11:45, Ahmet Altay wrote: >>>>>>> > +1 (validated python quickstarts). Thank you Mark. >>>>>>> > >>>>>>> > On Wed, Oct 2, 2019 at 10:49 AM Maximilian Michels <m...@apache.org >>>>>>> > <mailto:m...@apache.org>> wrote: >>>>>>> > >>>>>>> > Thanks for preparing the release, Mark! I would like to address >>>>>>> > https://issues.apache.org/jira/browse/BEAM-8303 in the >>>>>>> release. I've >>>>>>> > already merged the fix to the release-2.16.0 branch. If we do >>>>>>> another >>>>>>> > RC, we could include it. As a user is blocked on this, I would >>>>>>> not vote >>>>>>> > +1 for this RC, but I also do not want to block the release >>>>>>> process. >>>>>>> > >>>>>>> > >>>>>>> > Max, thank you for the clear communication for the importance and >>>>>>> at the >>>>>>> > same time non-blocking status of the issue. >>>>>>> > >>>>>>> > For the next time, may I suggest asking release manager to do the >>>>>>> > merging to the release branch. We do not know whether there will >>>>>>> be an >>>>>>> > RC2 or not. And if there will not be an RC2 release branch as of >>>>>>> now >>>>>>> > does not directly correspond to what will be released. >>>>>>> > >>>>>>> > >>>>>>> > On 01.10.19 09:18, Mark Liu wrote: >>>>>>> > > Hi everyone, >>>>>>> > > >>>>>>> > > Please review and vote on the release candidate #1 for the >>>>>>> version >>>>>>> > > 2.16.0, as follows: >>>>>>> > > [ ] +1, Approve the release >>>>>>> > > [ ] -1, Do not approve the release (please provide specific >>>>>>> comments) >>>>>>> > > >>>>>>> > > >>>>>>> > > The complete staging area is available for your review, >>>>>>> which >>>>>>> > includes: >>>>>>> > > * JIRA release notes [1], >>>>>>> > > * the official Apache source release to be deployed to >>>>>>> > dist.apache.org <http://dist.apache.org> >>>>>>> > > <http://dist.apache.org> [2], which is signed with the key >>>>>>> with >>>>>>> > > fingerprint C110B1C82074883A4241D977599D6305FF3ABB32 [3], >>>>>>> > > * all artifacts to be deployed to the Maven Central >>>>>>> Repository [4], >>>>>>> > > * source code tag ""v2.16.0-RC1" [5], >>>>>>> > > * website pull request listing the release [6], publishing >>>>>>> the API >>>>>>> > > reference manual [7], and the blog post [8]. >>>>>>> > > * Python artifacts are deployed along with the source >>>>>>> release to the >>>>>>> > > dist.apache.org <http://dist.apache.org> < >>>>>>> http://dist.apache.org> >>>>>>> > [2]. >>>>>>> > > * Validation sheet with a tab for 2.16.0 release to help >>>>>>> with >>>>>>> > validation >>>>>>> > > [9]. >>>>>>> > > * Docker images published to Docker Hub [10]. >>>>>>> > > >>>>>>> > > The vote will be open for at least 72 hours. It is adopted >>>>>>> by >>>>>>> > majority >>>>>>> > > approval, with at least 3 PMC affirmative votes. >>>>>>> > > >>>>>>> > > Thanks, >>>>>>> > > Mark Liu, Release Manager >>>>>>> > > >>>>>>> > > [1] >>>>>>> > > >>>>>>> > >>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12345494 >>>>>>> > > [2] https://dist.apache.org/repos/dist/dev/beam/2.16.0/ >>>>>>> > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS >>>>>>> > > [4] >>>>>>> > >>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1085/ >>>>>>> > > [5] https://github.com/apache/beam/tree/v2.16.0-RC1 >>>>>>> > > [6] https://github.com/apache/beam/pull/9667 >>>>>>> > > [7] https://github.com/apache/beam-site/pull/593 >>>>>>> > > [8] https://github.com/apache/beam/pull/9671 >>>>>>> > > [9] >>>>>>> > > >>>>>>> > >>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=890914284 >>>>>>> > > [10] https://hub.docker.com/u/apachebeam >>>>>>> > >>>>>>> >>>>>>
