I am planning to close the vote tonight and this is the only potential blocker discovered so far. So I strongly prefer to finalize the current candidate to not delay the release further, since, as Ahmet mentioned, this is a new functionality and not a regression, and is experimental. However, given the severity of the issue, cutting another candidate is definitely an option and if we can go this path as well.
Regards, Anton On Wed, Jul 31, 2019 at 3:16 PM Eugene Kirpichov <kirpic...@google.com> wrote: > I would recommend that the known issue notice about this source at least > be strongly worded - this source in the current state should be marked "DO > NOT USE" - it will produce data loss in *most* production use cases. That > still leaves the risk that people will use it anyway; up to folks driving > the release to decide whether it's worth cutting a new candidate for the > sake of just temporarily removing this source, or a notice in release notes > is sufficient. > > On Wed, Jul 31, 2019 at 2:59 PM Ahmet Altay <al...@google.com> wrote: > >> Since the python mongodb source is new in this release (not a regression) >> and experimental, I agree with adding a known issues notice to the release >> notes instead of starting a RC2 only for this issue. >> >> On Wed, Jul 31, 2019 at 2:47 PM Chamikara Jayalath <chamik...@google.com> >> wrote: >> >>> FYI we found a critical issue with the Python MongoDB source that is >>> included with this release: >>> https://issues.apache.org/jira/browse/BEAM-7866 >>> I suggest we include a clear notice in the release about this issue if >>> the release vote has already been finalized or make this a blocker if we >>> are going for a RC2. >>> >>> Thanks, >>> Cham >>> >>> On Wed, Jul 31, 2019 at 2:31 AM Robert Bradshaw <rober...@google.com> >>> wrote: >>> >>>> On Wed, Jul 31, 2019 at 11:22 AM Valentyn Tymofieiev < >>>> valen...@google.com> wrote: >>>> >>>>> I have checked Portable Wordcount example on Flink and Spark on Python >>>>> 2 and Python 3. >>>>> >>>>> To do so, I had to checkout Beam from git repo, since using the source >>>>> distribution does not include gradlew, and gradelw_orig did not work for >>>>> me. Commands I ran: >>>>> >>>>> git checkout tags/v2.14.0-RC1 >>>>> ./gradlew :sdks:python:container:py3:docker >>>>> ./gradlew :runners:flink:1.5:job-server:runShadow # Use ./gradlew >>>>> :runners:spark:job-server:runShadow for Spark >>>>> ./gradlew >>>>> :sdks:python:test-suites:portable:py35:portableWordCountBatch >>>>> -PjobEndpoint=localhost:8099 -PenvironmentType=LOOPBACK >>>>> cat /tmp/py-wordcount-direct* # to verify results. >>>>> >>>>> Loopback scenarios worked, however DOCKER scenarios did not. Opened >>>>> several Jiras to follow up: >>>>> >>>>> https://issues.apache.org/jira/browse/BEAM-7857 >>>>> https://issues.apache.org/jira/browse/BEAM-7858 >>>>> https://issues.apache.org/jira/browse/BEAM-7859 >>>>> <https://issues.apache.org/jira/browse/BEAM-7859?filter=-2> >>>>> >>>> >>>> I commented on the bugs, and I think this is due to trying to use >>>> Docker mode with local files (a known issue). >>>> >>>> >>>>> The gradle targets that were required to run these tests are not >>>>> present in 2.13.0 branch, so I don't consider it a regression and still >>>>> cast +1. >>>>> >>>> >>>> Agreed. >>>> >>>> >>>>> On Wed, Jul 31, 2019 at 11:31 AM Ismaël Mejía <ieme...@gmail.com> >>>>> wrote: >>>>> >>>>>> Oups Robert pointed to me that I have probably not counted correctly. >>>>>> There were indeed already 3 PMC +1 votes. Pablo, Robert and Ahmet. >>>>>> Please excuse me for the extra noise. >>>>>> >>>>>> On Wed, Jul 31, 2019 at 9:46 AM Ismaël Mejía <ieme...@gmail.com> >>>>>> wrote: >>>>>> > >>>>>> > To complete the release we need to have at least three +1 binding >>>>>> > votes (votes from PMC members) as stated in [1]. So far we have only >>>>>> > 2. >>>>>> > >>>>>> > Thomas (and the others). The blog post PR is now open [2] please >>>>>> help >>>>>> > us add missing features or maybe to highlight the ones you consider >>>>>> > important in the PR comments. >>>>>> > >>>>>> > Here it is the missing +1 (binding). Validated SHAs+signatures, >>>>>> > beam-samples and one internal company project with the new jars. >>>>>> > Compared source file vs tagged git repo. Everything looks ok. >>>>>> > >>>>>> > [1] https://www.apache.org/foundation/voting.html#ReleaseVotes >>>>>> > [2] https://github.com/apache/beam/pull/9201/files >>>>>> > >>>>>> > On Wed, Jul 31, 2019 at 6:27 AM Anton Kedin <ke...@google.com> >>>>>> wrote: >>>>>> > > >>>>>> > > Ran various postcommits, validates runners, and nexmark against >>>>>> the release branch. All looks good so far. >>>>>> > > >>>>>> > > Will take another look at the docs/blog and the nexmark numbers >>>>>> tomorrow, but if nothing comes up I will close the vote tomorrow >>>>>> (Wednesday) by 6pm PST (= Thursday 01:00am UTC) since it's over 72hours >>>>>> since the vote has started and we have a number of +1s including PMC >>>>>> members and no -1s. >>>>>> > > >>>>>> > > Regards, >>>>>> > > Anton >>>>>> > > >>>>>> > > On Tue, Jul 30, 2019 at 8:13 PM Valentyn Tymofieiev < >>>>>> valen...@google.com> wrote: >>>>>> > >> >>>>>> > >> I also ran unit tests for Python 3.7 and they passed as well. >>>>>> Cython tests for python3.7 require `apt-get install python3.7-dev`. >>>>>> > >> >>>>>> > >> On Wed, Jul 31, 2019 at 3:16 AM Pablo Estrada < >>>>>> pabl...@google.com> wrote: >>>>>> > >>> >>>>>> > >>> +1 >>>>>> > >>> >>>>>> > >>> I installed from source, and ran unit tests for Python in 2.7, >>>>>> 3.5, 3.6. >>>>>> > >>> >>>>>> > >>> Also ran a number of integration tests on Py 3.5 on Dataflow >>>>>> and DirectRunner. >>>>>> > >>> Best >>>>>> > >>> -P. >>>>>> > >>> >>>>>> > >>> On Tue, Jul 30, 2019 at 11:09 AM Hannah Jiang < >>>>>> hannahji...@google.com> wrote: >>>>>> > >>>> >>>>>> > >>>> I checked Py3 tests using .zip, mainly with direct runners, >>>>>> and everything looks good, so +1. >>>>>> > >>>> >>>>>> > >>>> On Tue, Jul 30, 2019 at 2:08 AM Robert Bradshaw < >>>>>> rober...@google.com> wrote: >>>>>> > >>>>> >>>>>> > >>>>> I checked all the artifact signatures and ran a couple test >>>>>> pipelines with the wheels (Py2 and Py3) and everything looked good to me, >>>>>> so +1. >>>>>> > >>>>> >>>>>> > >>>>> On Mon, Jul 29, 2019 at 8:29 PM Valentyn Tymofieiev < >>>>>> valen...@google.com> wrote: >>>>>> > >>>>>> >>>>>> > >>>>>> I have checked Python 3 batch and streaming quickstarts on >>>>>> Dataflow runner using .zip and wheel distributions. So far +1 from me. >>>>>> > >>>>>> >>>>>> > >>>>>> On Mon, Jul 29, 2019 at 7:53 PM Ahmet Altay < >>>>>> al...@google.com> wrote: >>>>>> > >>>>>>> >>>>>> > >>>>>>> +1, validated python 2 quickstarts. >>>>>> > >>>>>>> >>>>>> > >>>>>>> On Fri, Jul 26, 2019 at 5:46 PM Ahmet Altay < >>>>>> al...@google.com> wrote: >>>>>> > >>>>>>>> >>>>>> > >>>>>>>> To confirm, I manuall validated leader board on python. It >>>>>> is working. >>>>>> > >>>>>>>> >>>>>> > >>>>>>>> On Fri, Jul 26, 2019 at 5:23 PM Yifan Zou < >>>>>> yifan...@google.com> wrote: >>>>>> > >>>>>>>>> >>>>>> > >>>>>>>>> AFAIK, there should not be any special prerequisites for >>>>>> this. Things the script does including: >>>>>> > >>>>>>>>> 1. download the python rc in zip >>>>>> > >>>>>>>>> 2. start virtualenv and install the sdk. >>>>>> > >>>>>>>>> 3. verify hash. >>>>>> > >>>>>>>>> 4. config settings.xml and start a Java pubsub message >>>>>> injector. >>>>>> > >>>>>>>>> 5. run game examples and validate. >>>>>> > >>>>>>>>> >>>>>> > >>>>>>>>> Could you double check if the sdk was installed properly >>>>>> (step 1&2)? >>>>>> > >>>>>>>> >>>>>> > >>>>>>>> >>>>>> > >>>>>>>> I also guessing this is the case. Probably something >>>>>> earlier in the validation script did not run as expected. >>>>>> > >>>>>>>> >>>>>> > >>>>>>>>> >>>>>> > >>>>>>>>> >>>>>> > >>>>>>>>> >>>>>> > >>>>>>>>> Yifan >>>>>> > >>>>>>>>> >>>>>> > >>>>>>>>> On Fri, Jul 26, 2019 at 2:38 PM Anton Kedin < >>>>>> ke...@google.com> wrote: >>>>>> > >>>>>>>>>> >>>>>> > >>>>>>>>>> Validation script fails for me when I try to run [1] >>>>>> python leaderboard with direct runner: >>>>>> > >>>>>>>>>> >>>>>> > >>>>>>>>>> ``` >>>>>> > >>>>>>>>>> ***************************************************** >>>>>> > >>>>>>>>>> * Running Python Leaderboard with DirectRunner >>>>>> > >>>>>>>>>> ***************************************************** >>>>>> > >>>>>>>>>> /usr/bin/python: No module named >>>>>> apache_beam.examples.complete.game >>>>>> > >>>>>>>>>> ``` >>>>>> > >>>>>>>>>> >>>>>> > >>>>>>>>>> If someone has more context, what are the prerequisites >>>>>> for this step? How does it look up the module? >>>>>> > >>>>>>>>>> >>>>>> > >>>>>>>>>> [1] >>>>>> https://github.com/apache/beam/blob/master/release/src/main/scripts/run_rc_validation.sh#L424 >>>>>> > >>>>>>>>>> >>>>>> > >>>>>>>>>> Regards, >>>>>> > >>>>>>>>>> Anton >>>>>> > >>>>>>>>>> >>>>>> > >>>>>>>>>> On Fri, Jul 26, 2019 at 10:23 AM Anton Kedin < >>>>>> ke...@google.com> wrote: >>>>>> > >>>>>>>>>>> >>>>>> > >>>>>>>>>>> Cool, will make the post and will update the release >>>>>> guide as well then >>>>>> > >>>>>>>>>>> >>>>>> > >>>>>>>>>>> On Fri, Jul 26, 2019 at 10:20 AM Chad Dombrova < >>>>>> chad...@gmail.com> wrote: >>>>>> > >>>>>>>>>>>>> >>>>>> > >>>>>>>>>>>>> I think the release guide needs to be updated to >>>>>> remove the optionality of blog creation and avoid confusion. Thanks for >>>>>> pointing that out. >>>>>> > >>>>>>>>>>>> >>>>>> > >>>>>>>>>>>> >>>>>> > >>>>>>>>>>>> +1 >>>>>> > >>>>>>>>>>>> >>>>>> >>>>>
