I vetted the binary artifacts accompanying the release by running several
jobs on the Dataflow and Direct runners. At a high level, the release looks
fine -- I ran some of my favorite jobs and they all worked swimmingly.
There are some severe bugs in BigQueryIO in the release. Specifically, we
broke the ability to write to BigQuery using different tables for every
window. To a large degree, this makes BigQuery useless when working with
unbounded data (streaming pipelines). The bugs have been fixed (and
accompanying tests added) in PRs #1651 and #1400.
Conclusion: +0.8
* 0.4.0-incubating RC3 is largely an improvement over 0.3.0-incubating,
especially in the user getting started experience.
* The bugs in BigQueryIO are blockers for BigQuery users, but this is
likely a relatively small fraction of the Beam community. I would not
retract RC3 based on this alone. Unless we plan to cut an RC4 for other
reasons, we should move forward with RC3.
I'd hope that we hear from key users of the Apex, Flink, and Spark runners
before closing the vote, even though it's technically been 72+ hours. I
suggest we wait to ensure they have an opportunity to chime in.
Thanks,
Dan
Appendix: pom.xml changes to use binary releases from Apache Staging:
<repositories>
<repository>
<id>apache.staging</id>
<name>Apache Development Staging Repository</name>
<url>https://repository.apache.org/content/repositories/staging/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
On Sun, Dec 18, 2016 at 10:14 PM, Jean-Baptiste Onofré <j...@nanthrax.net>
wrote:
> Hi guys,
>
> The good thing is that my issue to access repository.apache.org Nexus is
> now fixed.
>
> To update the signature files, we have to drop the Nexus repository to
> stage a new one,
> meaning cancel the current vote to do a new RC4.
>
> I can do that, up to you.
>
> Anyway, regarding the release content, +1 (binding).
>
> Regards
> JB
>
>
> On 12/18/2016 06:56 PM, Davor Bonaci wrote:
>
>> Indeed -- I did help JB with the release ever so slightly, due to the
>> networking connectivity issue reaching repository.apache.org, which JB
>> further described and is tracked in INFRA-13086 [1]. This is not
>> Beam-specific.
>>
>> The current signature shouldn't be a problem at all, but, since others are
>> asking about it, I think it would be the best to simply re-sign the source
>> .zip archive and continuing this vote. JB, what do you think?
>>
>> Regarding the release itself, I think we need to keep raising the quality
>> and maturity release-over-release, and test signals are an excellent way
>> to
>> demonstrate that. Due to the recent upgrades to Jenkins, usage of the DSL,
>> etc. (thanks INFRA and Jason Kuster), we can now, for the first time,
>> formally show that the release candidate clearly passes all Jenkins suites
>> that we have:
>> * All unit tests across the project, plus example ITs across all runners
>> [2], [3].
>> * All integration tests on the Apex runner [4].
>> * All integration tests on the Flink runner [5].
>> * All integration tests on the Spark runner [6].
>> * All integration tests on the Dataflow runner [7].
>>
>> That said, I know of a few issues/regressions in the areas that are not
>> well tested today. I think Dan Halperin has more context, so I'll let him
>> speak of the details, and quote relevant JIRA issues.
>>
>> With the known issues in 0.3.0-incubating, such as trouble running
>> examples
>> out-of-the-box, I think this release candidate is a clear win. Of course,
>> that may change if more issues are discovered.
>>
>> For me, this release candidate is +1 (at this time), contingent upon no
>> known major issues affecting Apex, Flink and Spark runners.
>>
>> Davor
>>
>> [1] https://issues.apache.org/jira/browse/INFRA-13086
>> [2]
>> https://builds.apache.org/view/Beam/job/beam_PreCommit_Java_
>> MavenInstall/5994/
>> [3]
>> https://builds.apache.org/view/Beam/job/beam_PostCommit_Java
>> _MavenInstall/2116/
>> [4]
>> https://builds.apache.org/view/Beam/job/beam_PostCommit_Java
>> _RunnableOnService_Apex/10/
>> [5]
>> https://builds.apache.org/view/Beam/job/beam_PostCommit_Java
>> _RunnableOnService_Flink/1120/
>> [6]
>> https://builds.apache.org/view/Beam/job/beam_PostCommit_Java
>> _RunnableOnService_Spark/430/
>> [7]
>> https://builds.apache.org/view/Beam/job/beam_PostCommit_Java
>> _RunnableOnService_Dataflow/1830/
>>
>>
>> On Sat, Dec 17, 2016 at 4:13 PM, Kenneth Knowles <k...@google.com.invalid>
>> wrote:
>>
>> +1, as long as it is fine for the release to be signed by a PMC member
>>> other than the release manager. Otherwise need to replace the .asc file.
>>>
>>> Following [Apache release checklist](
>>> http://incubator.apache.org/guides/releasemanagement.html#check-list):
>>>
>>> 1.1 Verified checksums & signature (Davor's)
>>> 2.1 Ran unit tests and integration tests
>>> 3.1 DISCLAIMER is correct
>>> 3.2 LICENSE & NOTICE are correct
>>> 3.3 Files have license headers (RAT & checkstyle)
>>> 3.4 Provenance is clear
>>> 3.5 Dependencies license are legal (RAT) [2]
>>> 3.6 Release contains source code, no binaries
>>>
>>> Additionally:
>>>
>>> - Went over the generated javadoc (filed tickets but no release
>>> blockers)
>>> - Went over the generated release notes
>>> - Sanity checked the Maven Central artifacts
>>> - Confirmed that the git tag matches
>>> - Checked the website PR
>>>
>>> I heartily agree that the components would give much better context on
>>> tickets. Even with that, our JIRA titles could use a lot of improvement.
>>>
>>>
>>> On Fri, Dec 16, 2016 at 5:06 AM, Jean-Baptiste Onofré <j...@nanthrax.net>
>>> wrote:
>>>
>>> Hi everyone,
>>>>
>>>> Please review and vote on the release candidate #3 for the version
>>>> 0.4.0-incubating, as follows:
>>>> [ ] +1, Approve the release
>>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>>
>>>> The complete staging area is available for your review, which includes:
>>>> * JIRA release notes [1],
>>>> * the official Apache source release to be deployed to dist.apache.org
>>>> [2],
>>>> * all artifacts to be deployed to the Maven Central Repository [3],
>>>> * source code tag "v0.4.0-incubating-RC3" [4],
>>>> * website pull request listing the release and publishing the API
>>>>
>>> reference
>>>
>>>> manual [5].
>>>>
>>>> The vote will be open for at least 72 hours. It is adopted by majority
>>>> approval, with at least 3 PPMC affirmative votes.
>>>>
>>>> Thanks,
>>>> Regards
>>>> JB
>>>>
>>>> [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje
>>>> ctId=12319527&version=12338590
>>>> [2] https://dist.apache.org/repos/dist/dev/incubator/beam/0.4.0-
>>>> incubating/
>>>> [3] https://repository.apache.org/content/repositories/orgapache
>>>>
>>> beam-1008/
>>>
>>>> [4] https://git-wip-us.apache.org/repos/asf?p=incubator-beam.git
>>>> ;a=tag;h=112e38e4a68b07e6bf4916d1bdcc7ecaca8bbbd4
>>>> [5] https://github.com/apache/incubator-beam-site/pull/109
>>>>
>>>>
>>>
>>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
