On Sat, Mar 11, 2017 at 11:19 PM, Ahmet Altay <al...@google.com.invalid> wrote:
> On Sat, Mar 11, 2017 at 11:48 AM, Robert Bradshaw < > rober...@google.com.invalid> wrote: > > > On Fri, Mar 10, 2017 at 9:05 PM, Ahmet Altay <al...@google.com.invalid> > > wrote: > > > > > Hi everyone, > > > > > > Please review and vote on the release candidate #2 for the version > 0.6.0, > > > as follows: > > > [ ] +1, Approve the release > > > [ ] -1, Do not approve the release (please provide specific comments) > > > > > > > > > The complete staging area is available for your review, which includes: > > > * JIRA release notes [1], > > > * the official Apache source release to be deployed to dist.apache.org > > > [2], > > > which is signed with the key with fingerprint 6096FA00 [3], > > > * all artifacts to be deployed to the Maven Central Repository [4], > > > * source code tag "v0.6.0-RC2" [5], > > > * website pull request listing the release and publishing the API > > reference > > > manual [6]. > > > * python artifacts are deployed along with the source release to to > > > dist.apache.org [2]. > > > > > > > Are there plans also to deploy this at PyPi, and if so, what are the > > details? > > > > Yes, I think we should publish on PyPI. It will make the package usefeul to > many Python developers. > > My recommendation for details: > * PyPI requires one time registeration of the package. The most important > decision here is the package name. Our current package name apache_beam is > available. (Unless we decide to change this, there is no need to make a > change to this RC2.) > That name makes sense to me. Any objections? Do we need a separate vote, or should we just wait until this one passes. > * PyPI packages support two roles Owner and Maintainer. A Maintainer can > deploy new releases. A Owner (in addition to Maintainer permissions), can > manage the package and add/remove Maintainers/Owners. (This is my best > understanding, documentation is sparse [1].) I suggest that a PMC member > registers the package and become an owner. After that PMC can decide to add > more owners and maintainers. (Release manager needs to be at least a > maintainer.) > I'll be happy to do this. > * One-line change to release guide to use a command line tool to upload > approved release to PyPI. Note that there is no need to rebuild the RC2 to execute the above proposal. > > Thank you, > Ahmet > > [1] https://docs.python.org/2/distutils/packageindex.html#pypi-overview > > > > > > > > > A suite of Jenkins jobs: > > > * PreCommit_Java_MavenInstall [7], > > > * PostCommit_Java_MavenInstall [8], > > > * PostCommit_Java_RunnableOnService_Apex [9], > > > * PostCommit_Java_RunnableOnService_Flink [10], > > > * PostCommit_Java_RunnableOnService_Spark [11], > > > * PostCommit_Java_RunnableOnService_Dataflow [12] > > > * PostCommit_Python_Verify [13] > > > > > > Compared to release candidate #1, this candidate contains pull requests > > > #2217 [14], #2221 [15], #2222 [16], #2224 [17], and #2225 [18]; see the > > > discussion for reasoning. > I verified the Python package installs fine in a clean virtualenv. The hashes look good too now. However, when running wordcount_minimal I get File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/local/lib/python2.7/site-packages/apache_beam/io/fileio.py", line 281, in glob return gcsio.GcsIO().glob(path, limit) AttributeError: 'NoneType' object has no attribute 'GcsIO' It would be good to have a better error when gcp is not installed (the default). Not sure if this should block the release, but should be an easy fix (I'll create a PR). However, even after installing gcp I get Traceback (most recent call last): File "wordcount_minimal.py", line 123, in <module> run() File "wordcount_minimal.py", line 118, in run p.run().wait_until_finish() File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 163, in run return self.runner.run(self) File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/local/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 175, in run self.dataflow_client.create_job(self.job), self) File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/local/lib/python2.7/site-packages/apache_beam/utils/retry.py", line 174, in wrapper return fun(*args, **kwargs) File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/local/lib/python2.7/site-packages/apache_beam/runners/dataflow/internal/apiclient.py", line 411, in create_job self.create_job_description(job) File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/local/lib/python2.7/site-packages/apache_beam/runners/dataflow/internal/apiclient.py", line 432, in create_job_description job.options, file_copy=self._gcs_file_copy) File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/local/lib/python2.7/site-packages/apache_beam/runners/dataflow/internal/dependency.py", line 362, in stage_job_resources _stage_beam_sdk_tarball(sdk_remote_location, staged_path, temp_dir) File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/local/lib/python2.7/site-packages/apache_beam/runners/dataflow/internal/dependency.py", line 447, in _stage_beam_sdk_tarball _dependency_file_copy(_download_pypi_sdk_package(temp_dir), staged_path) File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/local/lib/python2.7/site-packages/apache_beam/runners/dataflow/internal/dependency.py", line 510, in _download_pypi_sdk_package processes.check_call(cmd_args) File "/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/local/lib/python2.7/site-packages/apache_beam/utils/processes.py", line 40, in check_call return subprocess.check_call(*args, **kwargs) File "/usr/lib/python2.7/subprocess.py", line 540, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/usr/local/google/home/robertwb/scratch/beam-0.6.0-test/bin/python', '-m', 'pip', 'install', '--download', '/tmp/tmpMscbhq', 'apache-beam==0.6.0', '--no-binary', ':all:', '--no-deps']' returned non-zero exit status 1 which looks like it's trying to download apache-beam from pypi. > > [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje > > > ctId=12319527&version=12339256 > > > [2] https://dist.apache.org/repos/dist/dev/beam/0.6.0/ > > > [3] https://dist.apache.org/repos/dist/dev/beam/KEYS > > > [4] https://repository.apache.org/content/repositories/ > > orgapachebeam-1013/ > > > [5] https://git-wip-us.apache.org/repos/asf?p=beam.git;a=tag;h=r > > > efs/tags/v0.6.0-RC2 > > > [6] https://github.com/apache/beam-site/pull/175 > > > [7] https://builds.apache.org/view/Beam/job/beam_PreCommit_Java_ > > > MavenInstall/8340/ > > > [8] https://builds.apache.org/view/Beam/job/beam_PostCommit_ > > > Java_MavenInstall/2877/ > > > [9] https://builds.apache.org/view/Beam/job/beam_PostCommit_Java > > > _RunnableOnService_Apex/736/ > > > [10] https://builds.apache.org/view/Beam/job/beam_PostCommit_Java > > > _RunnableOnService_Flink/1895/ > > > [11] https://builds.apache.org/view/Beam/job/beam_PostCommit_Java > > > _RunnableOnService_Spark/1207/ > > > [12] https://builds.apache.org/view/Beam/job/beam_PostCommit_Java_ > > > RunnableOnService_Dataflow/2526/ > > > [13] https://builds.apache.org/view/Beam/job/beam_PostCommit_Pyth > > > on_Verify/1481/ > > > [14] https://github.com/apache/beam/pull/2217 > > > [15] https://github.com/apache/beam/pull/2221 > > > [16] https://github.com/apache/beam/pull/2222 > > > [17] https://github.com/apache/beam/pull/2224 > > > [18] https://github.com/apache/beam/pull/2225 > > > > > >
