Thanks for the update Dan.
I think we should move forward on this release as, as you said, we have
important improvements compared to 0.3.0-incubating release.
We can do a 0.4.1-incubating pretty soon to address the bigquery IO
issues. I'm volunteer to do that.
Regards
JB
On 12/19/2016 09:21 PM, Dan Halperin wrote:
I vetted the binary artifacts accompanying the release by running several
jobs on the Dataflow and Direct runners. At a high level, the release looks
fine -- I ran some of my favorite jobs and they all worked swimmingly.
There are some severe bugs in BigQueryIO in the release. Specifically, we
broke the ability to write to BigQuery using different tables for every
window. To a large degree, this makes BigQuery useless when working with
unbounded data (streaming pipelines). The bugs have been fixed (and
accompanying tests added) in PRs #1651 and #1400.
Conclusion: +0.8
* 0.4.0-incubating RC3 is largely an improvement over 0.3.0-incubating,
especially in the user getting started experience.
* The bugs in BigQueryIO are blockers for BigQuery users, but this is
likely a relatively small fraction of the Beam community. I would not
retract RC3 based on this alone. Unless we plan to cut an RC4 for other
reasons, we should move forward with RC3.
I'd hope that we hear from key users of the Apex, Flink, and Spark runners
before closing the vote, even though it's technically been 72+ hours. I
suggest we wait to ensure they have an opportunity to chime in.
Thanks,
Dan
Appendix: pom.xml changes to use binary releases from Apache Staging:
<repositories>
<repository>
<id>apache.staging</id>
<name>Apache Development Staging Repository</name>
<url>https://repository.apache.org/content/repositories/staging/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
On Sun, Dec 18, 2016 at 10:14 PM, Jean-Baptiste Onofré <j...@nanthrax.net>
wrote:
Hi guys,
The good thing is that my issue to access repository.apache.org Nexus is
now fixed.
To update the signature files, we have to drop the Nexus repository to
stage a new one,
meaning cancel the current vote to do a new RC4.
I can do that, up to you.
Anyway, regarding the release content, +1 (binding).
Regards
JB
On 12/18/2016 06:56 PM, Davor Bonaci wrote:
Indeed -- I did help JB with the release ever so slightly, due to the
networking connectivity issue reaching repository.apache.org, which JB
further described and is tracked in INFRA-13086 [1]. This is not
Beam-specific.
The current signature shouldn't be a problem at all, but, since others are
asking about it, I think it would be the best to simply re-sign the source
.zip archive and continuing this vote. JB, what do you think?
Regarding the release itself, I think we need to keep raising the quality
and maturity release-over-release, and test signals are an excellent way
to
demonstrate that. Due to the recent upgrades to Jenkins, usage of the DSL,
etc. (thanks INFRA and Jason Kuster), we can now, for the first time,
formally show that the release candidate clearly passes all Jenkins suites
that we have:
* All unit tests across the project, plus example ITs across all runners
[2], [3].
* All integration tests on the Apex runner [4].
* All integration tests on the Flink runner [5].
* All integration tests on the Spark runner [6].
* All integration tests on the Dataflow runner [7].
That said, I know of a few issues/regressions in the areas that are not
well tested today. I think Dan Halperin has more context, so I'll let him
speak of the details, and quote relevant JIRA issues.
With the known issues in 0.3.0-incubating, such as trouble running
examples
out-of-the-box, I think this release candidate is a clear win. Of course,
that may change if more issues are discovered.
For me, this release candidate is +1 (at this time), contingent upon no
known major issues affecting Apex, Flink and Spark runners.
Davor
[1] https://issues.apache.org/jira/browse/INFRA-13086
[2]
https://builds.apache.org/view/Beam/job/beam_PreCommit_Java_
MavenInstall/5994/
[3]
https://builds.apache.org/view/Beam/job/beam_PostCommit_Java
_MavenInstall/2116/
[4]
https://builds.apache.org/view/Beam/job/beam_PostCommit_Java
_RunnableOnService_Apex/10/
[5]
https://builds.apache.org/view/Beam/job/beam_PostCommit_Java
_RunnableOnService_Flink/1120/
[6]
https://builds.apache.org/view/Beam/job/beam_PostCommit_Java
_RunnableOnService_Spark/430/
[7]
https://builds.apache.org/view/Beam/job/beam_PostCommit_Java
_RunnableOnService_Dataflow/1830/
On Sat, Dec 17, 2016 at 4:13 PM, Kenneth Knowles <k...@google.com.invalid>
wrote:
+1, as long as it is fine for the release to be signed by a PMC member
other than the release manager. Otherwise need to replace the .asc file.
Following [Apache release checklist](
http://incubator.apache.org/guides/releasemanagement.html#check-list):
1.1 Verified checksums & signature (Davor's)
2.1 Ran unit tests and integration tests
3.1 DISCLAIMER is correct
3.2 LICENSE & NOTICE are correct
3.3 Files have license headers (RAT & checkstyle)
3.4 Provenance is clear
3.5 Dependencies license are legal (RAT) [2]
3.6 Release contains source code, no binaries
Additionally:
- Went over the generated javadoc (filed tickets but no release
blockers)
- Went over the generated release notes
- Sanity checked the Maven Central artifacts
- Confirmed that the git tag matches
- Checked the website PR
I heartily agree that the components would give much better context on
tickets. Even with that, our JIRA titles could use a lot of improvement.
On Fri, Dec 16, 2016 at 5:06 AM, Jean-Baptiste Onofré <j...@nanthrax.net>
wrote:
Hi everyone,
Please review and vote on the release candidate #3 for the version
0.4.0-incubating, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org
[2],
* all artifacts to be deployed to the Maven Central Repository [3],
* source code tag "v0.4.0-incubating-RC3" [4],
* website pull request listing the release and publishing the API
reference
manual [5].
The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PPMC affirmative votes.
Thanks,
Regards
JB
[1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
ctId=12319527&version=12338590
[2] https://dist.apache.org/repos/dist/dev/incubator/beam/0.4.0-
incubating/
[3] https://repository.apache.org/content/repositories/orgapache
beam-1008/
[4] https://git-wip-us.apache.org/repos/asf?p=incubator-beam.git
;a=tag;h=112e38e4a68b07e6bf4916d1bdcc7ecaca8bbbd4
[5] https://github.com/apache/incubator-beam-site/pull/109
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
