+1 (binding) On Mon, Mar 13, 2017 at 11:10 AM, Robert Bradshaw <[email protected]> wrote:
> On Sat, Mar 11, 2017 at 11:19 PM, Ahmet Altay <[email protected]> > wrote: > >> On Sat, Mar 11, 2017 at 11:48 AM, Robert Bradshaw < >> [email protected]> wrote: >> >> > On Fri, Mar 10, 2017 at 9:05 PM, Ahmet Altay <[email protected]> >> > wrote: >> > >> > > Hi everyone, >> > > >> > > Please review and vote on the release candidate #2 for the version >> 0.6.0, >> > > as follows: >> > > [ ] +1, Approve the release >> > > [ ] -1, Do not approve the release (please provide specific comments) >> > > >> > > >> > > The complete staging area is available for your review, which >> includes: >> > > * JIRA release notes [1], >> > > * the official Apache source release to be deployed to >> dist.apache.org >> > > [2], >> > > which is signed with the key with fingerprint 6096FA00 [3], >> > > * all artifacts to be deployed to the Maven Central Repository [4], >> > > * source code tag "v0.6.0-RC2" [5], >> > > * website pull request listing the release and publishing the API >> > reference >> > > manual [6]. >> > > * python artifacts are deployed along with the source release to to >> > > dist.apache.org [2]. >> > > >> > >> > Are there plans also to deploy this at PyPi, and if so, what are the >> > details? >> > >> >> Yes, I think we should publish on PyPI. It will make the package usefeul >> to >> many Python developers. >> >> My recommendation for details: >> * PyPI requires one time registeration of the package. The most important >> decision here is the package name. Our current package name apache_beam is >> available. (Unless we decide to change this, there is no need to make a >> change to this RC2.) >> > > That name makes sense to me. Any objections? Do we need a separate vote, > or should we just wait until this one passes. > > >> * PyPI packages support two roles Owner and Maintainer. A Maintainer can >> deploy new releases. A Owner (in addition to Maintainer permissions), can >> manage the package and add/remove Maintainers/Owners. (This is my best >> understanding, documentation is sparse [1].) I suggest that a PMC member >> registers the package and become an owner. After that PMC can decide to >> add >> more owners and maintainers. (Release manager needs to be at least a >> maintainer.) >> > > I'll be happy to do this. > > >> * One-line change to release guide to use a command line tool to upload >> approved release to PyPI. > > Note that there is no need to rebuild the RC2 to execute the above >> proposal. >> >> Thank you, >> Ahmet >> >> [1] https://docs.python.org/2/distutils/packageindex.html#pypi-overview >> >> >> > >> > >> > > A suite of Jenkins jobs: >> > > * PreCommit_Java_MavenInstall [7], >> > > * PostCommit_Java_MavenInstall [8], >> > > * PostCommit_Java_RunnableOnService_Apex [9], >> > > * PostCommit_Java_RunnableOnService_Flink [10], >> > > * PostCommit_Java_RunnableOnService_Spark [11], >> > > * PostCommit_Java_RunnableOnService_Dataflow [12] >> > > * PostCommit_Python_Verify [13] >> > > >> > > Compared to release candidate #1, this candidate contains pull >> requests >> > > #2217 [14], #2221 [15], #2222 [16], #2224 [17], and #2225 [18]; see >> the >> > > discussion for reasoning. >> > > I verified the Python package installs fine in a clean virtualenv. The > hashes look good too now. However, when running wordcount_minimal I get > > File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/ > local/lib/python2.7/site-packages/apache_beam/io/fileio.py", line 281, in > glob > return gcsio.GcsIO().glob(path, limit) > AttributeError: 'NoneType' object has no attribute 'GcsIO' > > It would be good to have a better error when gcp is not installed (the > default). Not sure if this should block the release, but should be an easy > fix (I'll create a PR). > https://github.com/apache/beam/pull/2233 (maybe shouldn't hold up this pre-stable release). > However, even after installing gcp I get > > Traceback (most recent call last): > File "wordcount_minimal.py", line 123, in <module> > run() > File "wordcount_minimal.py", line 118, in run > p.run().wait_until_finish() > File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/ > local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 163, in > run > return self.runner.run(self) > File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/ > local/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", > line 175, in run > self.dataflow_client.create_job(self.job), self) > File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/ > local/lib/python2.7/site-packages/apache_beam/utils/retry.py", line 174, > in wrapper > return fun(*args, **kwargs) > File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/ > local/lib/python2.7/site-packages/apache_beam/runners/dataflow/internal/apiclient.py", > line 411, in create_job > self.create_job_description(job) > File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/ > local/lib/python2.7/site-packages/apache_beam/runners/dataflow/internal/apiclient.py", > line 432, in create_job_description > job.options, file_copy=self._gcs_file_copy) > File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/ > local/lib/python2.7/site-packages/apache_beam/runners/dataflow/internal/dependency.py", > line 362, in stage_job_resources > _stage_beam_sdk_tarball(sdk_remote_location, staged_path, temp_dir) > File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/ > local/lib/python2.7/site-packages/apache_beam/runners/dataflow/internal/dependency.py", > line 447, in _stage_beam_sdk_tarball > _dependency_file_copy(_download_pypi_sdk_package(temp_dir), > staged_path) > File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/ > local/lib/python2.7/site-packages/apache_beam/runners/dataflow/internal/dependency.py", > line 510, in _download_pypi_sdk_package > processes.check_call(cmd_args) > File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/ > local/lib/python2.7/site-packages/apache_beam/utils/processes.py", line > 40, in check_call > return subprocess.check_call(*args, **kwargs) > File "/usr/lib/python2.7/subprocess.py", line 540, in check_call > raise CalledProcessError(retcode, cmd) > subprocess.CalledProcessError: Command '['/usr/local/google/home/robe > rtwb/scratch/beam-0.6.0-test/bin/python', '-m', 'pip', 'install', > '--download', '/tmp/tmpMscbhq', 'apache-beam==0.6.0', '--no-binary', > ':all:', '--no-deps']' returned non-zero exit status 1 > > which looks like it's trying to download apache-beam from pypi. > Once apache_beam is on pypi this will work...somewhat circular but OK. Verified this works on direct and dataflow runner when passing --sdk_location. I tested Java on the Direct and Dataflow runner as well, which passed after this change: https://github.com/apache/beam-site/pull/176 > > > > [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje >> > > ctId=12319527&version=12339256 >> > > [2] https://dist.apache.org/repos/dist/dev/beam/0.6.0/ >> > > [3] https://dist.apache.org/repos/dist/dev/beam/KEYS >> > > [4] https://repository.apache.org/content/repositories/ >> > orgapachebeam-1013/ >> > > [5] https://git-wip-us.apache.org/repos/asf?p=beam.git;a=tag;h=r >> > > efs/tags/v0.6.0-RC2 >> > > [6] https://github.com/apache/beam-site/pull/175 >> > > [7] https://builds.apache.org/view/Beam/job/beam_PreCommit_Java_ >> > > MavenInstall/8340/ >> > > [8] https://builds.apache.org/view/Beam/job/beam_PostCommit_ >> > > Java_MavenInstall/2877/ >> > > [9] https://builds.apache.org/view/Beam/job/beam_PostCommit_Java >> > > _RunnableOnService_Apex/736/ >> > > [10] https://builds.apache.org/view/Beam/job/beam_PostCommit_Java >> > > _RunnableOnService_Flink/1895/ >> > > [11] https://builds.apache.org/view/Beam/job/beam_PostCommit_Java >> > > _RunnableOnService_Spark/1207/ >> > > [12] https://builds.apache.org/view/Beam/job/beam_PostCommit_Java_ >> > > RunnableOnService_Dataflow/2526/ >> > > [13] https://builds.apache.org/view/Beam/job/beam_PostCommit_Pyth >> > > on_Verify/1481/ >> > > [14] https://github.com/apache/beam/pull/2217 >> > > [15] https://github.com/apache/beam/pull/2221 >> > > [16] https://github.com/apache/beam/pull/2222 >> > > [17] https://github.com/apache/beam/pull/2224 >> > > [18] https://github.com/apache/beam/pull/2225 >> > > >> > >> > >
