+1 for continuing with RC1 with a notice about this issue. I marked this as a blocker for 2.15.0.
Thanks, Cham On Wed, Jul 31, 2019 at 3:28 PM Anton Kedin <ke...@google.com> wrote: > I am planning to close the vote tonight and this is the only potential > blocker discovered so far. So I strongly prefer to finalize the > current candidate to not delay the release further, since, as Ahmet > mentioned, this is a new functionality and not a regression, and is > experimental. However, given the severity of the issue, cutting another > candidate is definitely an option and if we can go this path as well. > > Regards, > Anton > > On Wed, Jul 31, 2019 at 3:16 PM Eugene Kirpichov <kirpic...@google.com> > wrote: > >> I would recommend that the known issue notice about this source at least >> be strongly worded - this source in the current state should be marked "DO >> NOT USE" - it will produce data loss in *most* production use cases. >> That still leaves the risk that people will use it anyway; up to folks >> driving the release to decide whether it's worth cutting a new candidate >> for the sake of just temporarily removing this source, or a notice in >> release notes is sufficient. >> >> On Wed, Jul 31, 2019 at 2:59 PM Ahmet Altay <al...@google.com> wrote: >> >>> Since the python mongodb source is new in this release (not a >>> regression) and experimental, I agree with adding a known issues notice to >>> the release notes instead of starting a RC2 only for this issue. >>> >>> On Wed, Jul 31, 2019 at 2:47 PM Chamikara Jayalath <chamik...@google.com> >>> wrote: >>> >>>> FYI we found a critical issue with the Python MongoDB source that is >>>> included with this release: >>>> https://issues.apache.org/jira/browse/BEAM-7866 >>>> I suggest we include a clear notice in the release about this issue if >>>> the release vote has already been finalized or make this a blocker if we >>>> are going for a RC2. >>>> >>>> Thanks, >>>> Cham >>>> >>>> On Wed, Jul 31, 2019 at 2:31 AM Robert Bradshaw <rober...@google.com> >>>> wrote: >>>> >>>>> On Wed, Jul 31, 2019 at 11:22 AM Valentyn Tymofieiev < >>>>> valen...@google.com> wrote: >>>>> >>>>>> I have checked Portable Wordcount example on Flink and Spark on >>>>>> Python 2 and Python 3. >>>>>> >>>>>> To do so, I had to checkout Beam from git repo, since using the >>>>>> source distribution does not include gradlew, and gradelw_orig did not >>>>>> work >>>>>> for me. Commands I ran: >>>>>> >>>>>> git checkout tags/v2.14.0-RC1 >>>>>> ./gradlew :sdks:python:container:py3:docker >>>>>> ./gradlew :runners:flink:1.5:job-server:runShadow # Use ./gradlew >>>>>> :runners:spark:job-server:runShadow for Spark >>>>>> ./gradlew >>>>>> :sdks:python:test-suites:portable:py35:portableWordCountBatch >>>>>> -PjobEndpoint=localhost:8099 -PenvironmentType=LOOPBACK >>>>>> cat /tmp/py-wordcount-direct* # to verify results. >>>>>> >>>>>> Loopback scenarios worked, however DOCKER scenarios did not. Opened >>>>>> several Jiras to follow up: >>>>>> >>>>>> https://issues.apache.org/jira/browse/BEAM-7857 >>>>>> https://issues.apache.org/jira/browse/BEAM-7858 >>>>>> https://issues.apache.org/jira/browse/BEAM-7859 >>>>>> <https://issues.apache.org/jira/browse/BEAM-7859?filter=-2> >>>>>> >>>>> >>>>> I commented on the bugs, and I think this is due to trying to use >>>>> Docker mode with local files (a known issue). >>>>> >>>>> >>>>>> The gradle targets that were required to run these tests are not >>>>>> present in 2.13.0 branch, so I don't consider it a regression and still >>>>>> cast +1. >>>>>> >>>>> >>>>> Agreed. >>>>> >>>>> >>>>>> On Wed, Jul 31, 2019 at 11:31 AM Ismaël Mejía <ieme...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Oups Robert pointed to me that I have probably not counted correctly. >>>>>>> There were indeed already 3 PMC +1 votes. Pablo, Robert and Ahmet. >>>>>>> Please excuse me for the extra noise. >>>>>>> >>>>>>> On Wed, Jul 31, 2019 at 9:46 AM Ismaël Mejía <ieme...@gmail.com> >>>>>>> wrote: >>>>>>> > >>>>>>> > To complete the release we need to have at least three +1 binding >>>>>>> > votes (votes from PMC members) as stated in [1]. So far we have >>>>>>> only >>>>>>> > 2. >>>>>>> > >>>>>>> > Thomas (and the others). The blog post PR is now open [2] please >>>>>>> help >>>>>>> > us add missing features or maybe to highlight the ones you consider >>>>>>> > important in the PR comments. >>>>>>> > >>>>>>> > Here it is the missing +1 (binding). Validated SHAs+signatures, >>>>>>> > beam-samples and one internal company project with the new jars. >>>>>>> > Compared source file vs tagged git repo. Everything looks ok. >>>>>>> > >>>>>>> > [1] https://www.apache.org/foundation/voting.html#ReleaseVotes >>>>>>> > [2] https://github.com/apache/beam/pull/9201/files >>>>>>> > >>>>>>> > On Wed, Jul 31, 2019 at 6:27 AM Anton Kedin <ke...@google.com> >>>>>>> wrote: >>>>>>> > > >>>>>>> > > Ran various postcommits, validates runners, and nexmark against >>>>>>> the release branch. All looks good so far. >>>>>>> > > >>>>>>> > > Will take another look at the docs/blog and the nexmark numbers >>>>>>> tomorrow, but if nothing comes up I will close the vote tomorrow >>>>>>> (Wednesday) by 6pm PST (= Thursday 01:00am UTC) since it's over 72hours >>>>>>> since the vote has started and we have a number of +1s including PMC >>>>>>> members and no -1s. >>>>>>> > > >>>>>>> > > Regards, >>>>>>> > > Anton >>>>>>> > > >>>>>>> > > On Tue, Jul 30, 2019 at 8:13 PM Valentyn Tymofieiev < >>>>>>> valen...@google.com> wrote: >>>>>>> > >> >>>>>>> > >> I also ran unit tests for Python 3.7 and they passed as well. >>>>>>> Cython tests for python3.7 require `apt-get install python3.7-dev`. >>>>>>> > >> >>>>>>> > >> On Wed, Jul 31, 2019 at 3:16 AM Pablo Estrada < >>>>>>> pabl...@google.com> wrote: >>>>>>> > >>> >>>>>>> > >>> +1 >>>>>>> > >>> >>>>>>> > >>> I installed from source, and ran unit tests for Python in 2.7, >>>>>>> 3.5, 3.6. >>>>>>> > >>> >>>>>>> > >>> Also ran a number of integration tests on Py 3.5 on Dataflow >>>>>>> and DirectRunner. >>>>>>> > >>> Best >>>>>>> > >>> -P. >>>>>>> > >>> >>>>>>> > >>> On Tue, Jul 30, 2019 at 11:09 AM Hannah Jiang < >>>>>>> hannahji...@google.com> wrote: >>>>>>> > >>>> >>>>>>> > >>>> I checked Py3 tests using .zip, mainly with direct runners, >>>>>>> and everything looks good, so +1. >>>>>>> > >>>> >>>>>>> > >>>> On Tue, Jul 30, 2019 at 2:08 AM Robert Bradshaw < >>>>>>> rober...@google.com> wrote: >>>>>>> > >>>>> >>>>>>> > >>>>> I checked all the artifact signatures and ran a couple test >>>>>>> pipelines with the wheels (Py2 and Py3) and everything looked good to >>>>>>> me, >>>>>>> so +1. >>>>>>> > >>>>> >>>>>>> > >>>>> On Mon, Jul 29, 2019 at 8:29 PM Valentyn Tymofieiev < >>>>>>> valen...@google.com> wrote: >>>>>>> > >>>>>> >>>>>>> > >>>>>> I have checked Python 3 batch and streaming quickstarts on >>>>>>> Dataflow runner using .zip and wheel distributions. So far +1 from me. >>>>>>> > >>>>>> >>>>>>> > >>>>>> On Mon, Jul 29, 2019 at 7:53 PM Ahmet Altay < >>>>>>> al...@google.com> wrote: >>>>>>> > >>>>>>> >>>>>>> > >>>>>>> +1, validated python 2 quickstarts. >>>>>>> > >>>>>>> >>>>>>> > >>>>>>> On Fri, Jul 26, 2019 at 5:46 PM Ahmet Altay < >>>>>>> al...@google.com> wrote: >>>>>>> > >>>>>>>> >>>>>>> > >>>>>>>> To confirm, I manuall validated leader board on python. >>>>>>> It is working. >>>>>>> > >>>>>>>> >>>>>>> > >>>>>>>> On Fri, Jul 26, 2019 at 5:23 PM Yifan Zou < >>>>>>> yifan...@google.com> wrote: >>>>>>> > >>>>>>>>> >>>>>>> > >>>>>>>>> AFAIK, there should not be any special prerequisites for >>>>>>> this. Things the script does including: >>>>>>> > >>>>>>>>> 1. download the python rc in zip >>>>>>> > >>>>>>>>> 2. start virtualenv and install the sdk. >>>>>>> > >>>>>>>>> 3. verify hash. >>>>>>> > >>>>>>>>> 4. config settings.xml and start a Java pubsub message >>>>>>> injector. >>>>>>> > >>>>>>>>> 5. run game examples and validate. >>>>>>> > >>>>>>>>> >>>>>>> > >>>>>>>>> Could you double check if the sdk was installed properly >>>>>>> (step 1&2)? >>>>>>> > >>>>>>>> >>>>>>> > >>>>>>>> >>>>>>> > >>>>>>>> I also guessing this is the case. Probably something >>>>>>> earlier in the validation script did not run as expected. >>>>>>> > >>>>>>>> >>>>>>> > >>>>>>>>> >>>>>>> > >>>>>>>>> >>>>>>> > >>>>>>>>> >>>>>>> > >>>>>>>>> Yifan >>>>>>> > >>>>>>>>> >>>>>>> > >>>>>>>>> On Fri, Jul 26, 2019 at 2:38 PM Anton Kedin < >>>>>>> ke...@google.com> wrote: >>>>>>> > >>>>>>>>>> >>>>>>> > >>>>>>>>>> Validation script fails for me when I try to run [1] >>>>>>> python leaderboard with direct runner: >>>>>>> > >>>>>>>>>> >>>>>>> > >>>>>>>>>> ``` >>>>>>> > >>>>>>>>>> ***************************************************** >>>>>>> > >>>>>>>>>> * Running Python Leaderboard with DirectRunner >>>>>>> > >>>>>>>>>> ***************************************************** >>>>>>> > >>>>>>>>>> /usr/bin/python: No module named >>>>>>> apache_beam.examples.complete.game >>>>>>> > >>>>>>>>>> ``` >>>>>>> > >>>>>>>>>> >>>>>>> > >>>>>>>>>> If someone has more context, what are the prerequisites >>>>>>> for this step? How does it look up the module? >>>>>>> > >>>>>>>>>> >>>>>>> > >>>>>>>>>> [1] >>>>>>> https://github.com/apache/beam/blob/master/release/src/main/scripts/run_rc_validation.sh#L424 >>>>>>> > >>>>>>>>>> >>>>>>> > >>>>>>>>>> Regards, >>>>>>> > >>>>>>>>>> Anton >>>>>>> > >>>>>>>>>> >>>>>>> > >>>>>>>>>> On Fri, Jul 26, 2019 at 10:23 AM Anton Kedin < >>>>>>> ke...@google.com> wrote: >>>>>>> > >>>>>>>>>>> >>>>>>> > >>>>>>>>>>> Cool, will make the post and will update the release >>>>>>> guide as well then >>>>>>> > >>>>>>>>>>> >>>>>>> > >>>>>>>>>>> On Fri, Jul 26, 2019 at 10:20 AM Chad Dombrova < >>>>>>> chad...@gmail.com> wrote: >>>>>>> > >>>>>>>>>>>>> >>>>>>> > >>>>>>>>>>>>> I think the release guide needs to be updated to >>>>>>> remove the optionality of blog creation and avoid confusion. Thanks for >>>>>>> pointing that out. >>>>>>> > >>>>>>>>>>>> >>>>>>> > >>>>>>>>>>>> >>>>>>> > >>>>>>>>>>>> +1 >>>>>>> > >>>>>>>>>>>> >>>>>>> >>>>>>
