Agree. I just would like what changed exactly as I didn't have any issue when I did the 2.1.0 release.
Regards JB On Nov 8, 2017, 21:50, at 21:50, Kenneth Knowles <k...@google.com.INVALID> wrote: >Agree with everything Robert said. So if we just rebuild the Python zip >then this should g2g? > >On Wed, Nov 8, 2017 at 12:37 PM, Robert Bradshaw < >rober...@google.com.invalid> wrote: > >> Let me try to clarify the state of the world (with regards to Python >> and proto files). >> >> * When Python setup.py is run, it checks to see if the generated pb2 >> files exist. If not, it attempts to generate them by installing the >> proto compiler and looking up the .proto definitions in its parent >> directory. This works great for the developer that checked out the >> full pristine sources from git (or otherwise obtained them). >> >> * For the sdist tarball uploaded to PyPi (aka Python Artifact), we >> ship the generated pb2 files both because (1) we don't want to force >> the user to install the proto compiler and (2) the "parent" directory >> doesn't exist as we're just shipping the sdks/python/... portion of >> the full git repository. >> >> * All previous "releases" in >> https://dist.apache.org/repos/dist/release/beam/ post the Python >> artifact (which is Python sources + generated proto files, but >notably >> no source proto files) in addition to the full source artifact (which >> contains some snapshot of the full git repository, Python and proto >> files included). We also separately publish Java artifacts offsite >> which is what people will install from. >> >> So it seems the purpose of the -python.zip file is just to stage what >> we intend to release on PyPi (e.g. for testing); it is not a source >> distribution (that is taken care of by the adjacent -source.zip file) >> and so there's no issue with it containing generated files. It should >> be the output of "python setup.py sdist" (possibly invoked by the mvn >> release commands, if you can get those to work). On the other hand, >> creating a separate python-only source distribution would serve no >> purpose, as it would be redundant with the existing >> everything-source-distribution which is just a manually taken >snapshot >> of the entire git repository. The confusion is around the role of the >> -python.zip file, and if we clarify that it's the proposed Python >PyPi >> artifact, and *not* some kind of python-only source distribution, the >> release process is WAI. >> >> - Robert >> >> >> On Wed, Nov 8, 2017 at 11:55 AM, Jean-Baptiste Onofré ><j...@nanthrax.net> >> wrote: >> > Let me take a look. Afair I didn't touch those files in last >release. >> > >> > I keep you posted. >> > >> > Regards >> > JB >> > >> > On Nov 8, 2017, 20:50, at 20:50, Reuven Lax ><re...@google.com.INVALID> >> wrote: >> >>I explicitly removed the pb2 files as I thought we determined they >> >>shouldn't be in the source release, and they caused RAT failures. >What >> >>should I be doing here? >> >> >> >>On Wed, Nov 8, 2017 at 10:21 AM, Robert Bradshaw < >> >>rober...@google.com.invalid> wrote: >> >> >> >>> This is due to having removed the auto-generated pb2 files. >> >>> >> >>> On Wed, Nov 8, 2017 at 9:37 AM, Valentyn Tymofieiev >> >>> <valen...@google.com.invalid> wrote: >> >>> > Confirming Ismaël's finding - I also see this error and it did >not >> >>see it >> >>> > on a candidate that was in the staging area yesterday. >> >>> > >> >>> > On Wed, Nov 8, 2017 at 9:07 AM, Ismaël Mejía ><ieme...@gmail.com> >> >>wrote: >> >>> > >> >>> >> I tested the python version of the release I just created a >new >> >>> >> virtualenv and run >> >>> >> >> >>> >> python setup.py install and it gave me this message: >> >>> >> >> >>> >> Traceback (most recent call last): >> >>> >> File "setup.py", line 203, in <module> >> >>> >> 'test': generate_protos_first(test), >> >>> >> File "/usr/lib/python2.7/distutils/core.py", line 151, in >setup >> >>> >> dist.run_commands() >> >>> >> File "/usr/lib/python2.7/distutils/dist.py", line 953, in >> >>> run_commands >> >>> >> self.run_command(cmd) >> >>> >> File "/usr/lib/python2.7/distutils/dist.py", line 972, in >> >>run_command >> >>> >> cmd_obj.run() >> >>> >> File >> >>"/home/ismael/.virtualenvs/beam-vote2/local/lib/python2.7/si >> >>> >> te-packages/setuptools/command/install.py", >> >>> >> line 67, in run >> >>> >> self.do_egg_install() >> >>> >> File >> >>"/home/ismael/.virtualenvs/beam-vote2/local/lib/python2.7/si >> >>> >> te-packages/setuptools/command/install.py", >> >>> >> line 109, in do_egg_install >> >>> >> self.run_command('bdist_egg') >> >>> >> File "/usr/lib/python2.7/distutils/cmd.py", line 326, in >> >>run_command >> >>> >> self.distribution.run_command(command) >> >>> >> File "/usr/lib/python2.7/distutils/dist.py", line 972, in >> >>run_command >> >>> >> cmd_obj.run() >> >>> >> File >> >>"/home/ismael/.virtualenvs/beam-vote2/local/lib/python2.7/si >> >>> >> te-packages/setuptools/command/bdist_egg.py", >> >>> >> line 169, in run >> >>> >> cmd = self.call_command('install_lib', warn_dir=0) >> >>> >> File >> >>"/home/ismael/.virtualenvs/beam-vote2/local/lib/python2.7/si >> >>> >> te-packages/setuptools/command/bdist_egg.py", >> >>> >> line 155, in call_command >> >>> >> self.run_command(cmdname) >> >>> >> File "/usr/lib/python2.7/distutils/cmd.py", line 326, in >> >>run_command >> >>> >> self.distribution.run_command(command) >> >>> >> File "/usr/lib/python2.7/distutils/dist.py", line 972, in >> >>run_command >> >>> >> cmd_obj.run() >> >>> >> File >> >>"/home/ismael/.virtualenvs/beam-vote2/local/lib/python2.7/si >> >>> >> te-packages/setuptools/command/install_lib.py", >> >>> >> line 11, in run >> >>> >> self.build() >> >>> >> File "/usr/lib/python2.7/distutils/command/install_lib.py", >line >> >>109, >> >>> >> in build >> >>> >> self.run_command('build_py') >> >>> >> File "/usr/lib/python2.7/distutils/cmd.py", line 326, in >> >>run_command >> >>> >> self.distribution.run_command(command) >> >>> >> File "/usr/lib/python2.7/distutils/dist.py", line 972, in >> >>run_command >> >>> >> cmd_obj.run() >> >>> >> File "setup.py", line 143, in run >> >>> >> gen_protos.generate_proto_files() >> >>> >> File >> >>"/home/ismael/releases/votes/beam/apache-beam-2.2.0-python/g >> >>> >> en_protos.py", >> >>> >> line 66, in generate_proto_files >> >>> >> 'Not in apache git tree; unable to find proto >definitions.') >> >>> >> RuntimeError: Not in apache git tree; unable to find proto >> >>definitions. >> >>> >> >> >>> >> Not sure if this is something in my environment, but this >passed >> >>when >> >>> >> I validated the previous release (2.1.0). >> >>> >> >> >>> >> >> >>> >> On Wed, Nov 8, 2017 at 11:30 AM, Reuven Lax >> >><re...@google.com.invalid> >> >>> >> wrote: >> >>> >> > Hi everyone, >> >>> >> > >> >>> >> > Please review and vote on the release candidate #2 for the >> >>version >> >>> 2.2.0, >> >>> >> > as follows: >> >>> >> > [ ] +1, Approve the release >> >>> >> > [ ] -1, Do not approve the release (please provide >specific >> >>> comments) >> >>> >> > >> >>> >> > >> >>> >> > The complete staging area is available for your review, >which >> >>> includes: >> >>> >> > * JIRA release notes [1], >> >>> >> > * the official Apache source release to be deployed to >> >>> dist.apache.org >> >>> >> [2], >> >>> >> > which is signed with the key with fingerprint B98B7708 [3], >> >>> >> > * all artifacts to be deployed to the Maven Central >Repository >> >>[4], >> >>> >> > * source code tag "v2.2.0-RC3" [5], >> >>> >> > * website pull request listing the release and publishing >the >> >>API >> >>> >> > reference manual [6]. >> >>> >> > * Java artifacts were built with Maven 3.5.0 and >> >>OpenJDK/Oracle JDK >> >>> >> > 1.8.0_144. >> >>> >> > * Python artifacts are deployed along with the source >release >> >>to the >> >>> >> > dist.apache.org [2]. >> >>> >> > >> >>> >> > The vote will be open for at least 72 hours. It is adopted >by >> >>majority >> >>> >> > approval, with at least 3 PMC affirmative votes. >> >>> >> > >> >>> >> > Thanks, >> >>> >> > Reuven >> >>> >> > >> >>> >> > [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa? >> >>> >> > projectId=12319527&version=12341044 >> >>> >> > [2] https://dist.apache.org/repos/dist/dev/beam/2.2.0/ >> >>> >> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS >> >>> >> > [4] >https://repository.apache.org/content/repositories/orgapache >> >>> >> beam-1023/ >> >>> >> > <https://repository.apache.org/content/repositories/ >> >>> orgapachebeam-1022/> >> >>> >> > [5] https://github.com/apache/beam/tree/v2.2.0-RC >> >>> >> > <https://github.com/apache/beam/tree/v2.2.0-RC2>3 >> >>> >> > [6] https://github.com/apache/beam-site/pull/337 >> >>> >> >> >>> >>
