I definitely appreciate the frustration about how long this release is taking. It's verging on the point of ridiculous at this point, and we need to fix some of the things that caused us to get to this state (for one thing our infrastructure was so busted at one point, that Valentyn spent 2 weeks trying to get on PR merged into the release branch).
At this point, let's try and fix this Monday. Unfortunately this is not the sole issue requiring RC4. Python verification failed as well, and we need an RC4 regardless to merge those PRs. I'm hoping that RC4 is our final RC, and we can finish voting next week. Reuven On Sat, Nov 11, 2017 at 6:24 AM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Le 11 nov. 2017 09:52, "Jean-Baptiste Onofré" <j...@nanthrax.net> a écrit : > > If the purpose is to release 2.2.1 in one week, why not just to a RC4 ? > > It's not a regression because WriteFiles is new and extend the previous > FileSource. So it could consider as a severe bug, especially on WriteFiles > which is important. > > > Fair enough. > > > The core issue is the time we spent already on this release: roughly 1 > month !!! It's clearly too long due to different causes. > When I did the previous releases, it took 3 or 4 days. It's clearly the > target as, as said, I would like to have a release pace of a release every > 6 weeks. > > > > Agree and this is why 2.2.0 must be out now IMHO. If you are confident next > week is sufficient just go ahead and ignore my comment but my point was the > same: it shouldnt last so long if there is no regression :(. > > > > Regards > JB > > > On 11/11/2017 08:41 AM, Romain Manni-Bucau wrote: > > > You can see it differently: is there a critical bug? Yes! Is there a > > regression? No! > > > > So no need to wait another week (keep in mind 2 days + 3 days of vote > > makes easily 1 working week). This vote could be closed already and next > > week 2.2.1 could fix this bug, no? Overall idea is to not hold the > > community more than needed if there is no regression compared to last few > > releases. > > > > Le 11 nov. 2017 07:46, "Jean-Baptiste Onofré" <j...@nanthrax.net> a écrit > : > > > > -1 (binding) > >> > >> I agree with Eugene, data loss is severe. > >> > >> As Eugene seems confident to fix that quickly, I think it's worth to > cut a > >> RC4. > >> > >> However, I would introduce a deadline. As I would like to propose a > >> release cycle of a release every 6 weeks (whatever it contains, but it > >> really important to keep a regular pace in releases), a release should > be > >> cut in couple of days. So, maybe we can give us 2 business days to fix > >> that > >> and propose a RC4. Basically, if this issue is not fix on Tuesday night, > >> then, we move forward anyway. > >> > >> Regards > >> JB > >> > >> On 11/10/2017 07:42 PM, Eugene Kirpichov wrote: > >> > >> Unfortunately I think I found a data loss bug - it was there since 2.0.0 > >>> but I think it's serious enough that delaying a fix until the next > >>> release > >>> would be irresponsible. > >>> See https://issues.apache.org/jira/browse/BEAM-3169 > >>> > >>> On Thu, Nov 9, 2017 at 3:57 PM Robert Bradshaw > >>> <rober...@google.com.invalid> > >>> wrote: > >>> > >>> Our release notes look like nothing more than a query for the closed > >>> > >>>> jira issues. Do we have a top-level summary to highlight the big > >>>> ticket items in the release? And in particular somewhere to mention > >>>> that this is likely the last release to support Java 7 that'll get > >>>> widely read? > >>>> > >>>> On Thu, Nov 9, 2017 at 3:39 PM, Reuven Lax <re...@google.com.invalid> > >>>> wrote: > >>>> > >>>> Thanks, > >>>>> > >>>>> This RC is currently failing on a number of validation steps, so we > >>>>> need > >>>>> > >>>>> to > >>>> > >>>> cut at least one more RC. Fingers crossed that it will be the last > one. > >>>>> > >>>>> Reuven > >>>>> > >>>>> On Thu, Nov 9, 2017 at 3:36 PM, Konstantinos Katsiapis < > >>>>> katsia...@google.com.invalid> wrote: > >>>>> > >>>>> Just a remark: Release of Tensorflow Transform > >>>>> > >>>>>> <https://github.com/tensorflow/transform> 0.4.0 depends on release > of > >>>>>> Apache Beam 2.2.0 so upvoting for a release (the sooner the better). > >>>>>> > >>>>>> On Thu, Nov 9, 2017 at 3:33 PM, Reuven Lax <re...@google.com.invalid > > > >>>>>> wrote: > >>>>>> > >>>>>> Are we waiting for any more validation of this candidate? If people > >>>>>> > >>>>>>> > >>>>>>> are > >>>>>> > >>>>> > >>>> still running tests I'll hold off on RC4 (to reduce the chance of an > >>>>> > >>>>>> > >>>>>>> RC5), > >>>>>> > >>>>>> otherwise I'll cut RC4 once Valentyn's PR is merged. > >>>>>>> > >>>>>>> Reuven > >>>>>>> > >>>>>>> On Thu, Nov 9, 2017 at 2:26 PM, Valentyn Tymofieiev < > >>>>>>> valen...@google.com.invalid> wrote: > >>>>>>> > >>>>>>> https://github.com/apache/beam/pull/4109 is out to address both > >>>>>>> > >>>>>>>> > >>>>>>>> findings I > >>>>>>> > >>>>>>> reported earlier. > >>>>>>>> > >>>>>>>> On Thu, Nov 9, 2017 at 8:54 AM, Etienne Chauchot < > >>>>>>>> > >>>>>>>> echauc...@gmail.com> > >>>>>>> > >>>>>> > >>>> wrote: > >>>>> > >>>>>> > >>>>>>>> Just as a remark, I compared (on my laptop though) queries > >>>>>>>> > >>>>>>>>> > >>>>>>>>> execution > >>>>>>>> > >>>>>>> > >>>> times > >>>>> > >>>>>> > >>>>>>>> on my previous run of 2.2.0-RC3 with release 2.1.0 and I did not > >>>>>>>>> > >>>>>>>>> see > >>>>>>>> > >>>>>>> > >>>> any > >>>>> > >>>>>> > >>>>>>> performance regression. > >>>>>>>> > >>>>>>>>> > >>>>>>>>> Best > >>>>>>>>> > >>>>>>>>> Etienne > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Le 09/11/2017 à 03:13, Valentyn Tymofieiev a écrit : > >>>>>>>>> > >>>>>>>>> I looked at Python side of Dataflow & Direct runners on Linux. > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> There > >>>>>>>>> > >>>>>>>> > >>>> are > >>>>> > >>>>>> > >>>>>>> two findings: > >>>>>>>> > >>>>>>>>> > >>>>>>>>>> 1. One of the mobile gaming examples did not pass for Dataflow > >>>>>>>>>> > >>>>>>>>>> runner, > >>>>>>>>> > >>>>>>>> > >>>>>> addressed in: https://github.com/apache/beam/pull/4102 > >>>>>>> > >>>>>>>> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fapa > >>>>>>>>>> che%2Fbeam%2Fpull%2F4102&sa=D&sntz=1&usg=AFQjCNF3OS6Oo-MeNET > >>>>>>>>>> CCmOxJj5Gm2uH6g> > >>>>>>>>>> > >>>>>>>>>> . > >>>>>>>>>> > >>>>>>>>>> 2. Python streaming did not work for Dataflow runner, one PR is > >>>>>>>>>> > >>>>>>>>>> out > >>>>>>>>> > >>>>>>>> > >>>> https://github.com/apache/beam/pull/4106, but follow up PRs may > >>>>> > >>>>>> > >>>>>>>>>> be > >>>>>>>>> > >>>>>>>> > >>>> required > >>>>> > >>>>>> as we continue to investigate. If we had a PostCommit tests suite > >>>>>>>>>> > >>>>>>>>>> running > >>>>>>>>> > >>>>>>>> > >>>>>>>> against a release branch, this could have been caught earlier. > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Filed > >>>>>>>>> > >>>>>>>> > >>>> https://issues.apache.org/jira/browse/BEAM-3163. > >>>>> > >>>>>> > >>>>>>>>>> On Wed, Nov 8, 2017 at 2:39 PM, Reuven Lax > >>>>>>>>>> > >>>>>>>>>> <re...@google.com.invalid > >>>>>>>>> > >>>>>>>> > >>>> > >>>>> wrote: > >>>>>>> > >>>>>>>> > >>>>>>>>>> Hi everyone, > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> Please review and vote on the release candidate #3 for the > >>>>>>>>>>> > >>>>>>>>>>> version > >>>>>>>>>> > >>>>>>>>> > >>>> 2.2.0, > >>>>> > >>>>>> > >>>>>>>> as follows: > >>>>>>>>> > >>>>>>>>>> [ ] +1, Approve the release > >>>>>>>>>>> [ ] -1, Do not approve the release (please provide > specific > >>>>>>>>>>> > >>>>>>>>>>> comments) > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>>> > >>>>>>>>>>> The complete staging area is available for your review, which > >>>>>>>>>>> > >>>>>>>>>>> includes: > >>>>>>>>>> > >>>>>>>>> > >>>>>>> * JIRA release notes [1], > >>>>>>>> > >>>>>>>>> * the official Apache source release to be deployed to > >>>>>>>>>>> dist.apache.org > >>>>>>>>>>> [2], > >>>>>>>>>>> which is signed with the key with fingerprint B98B7708 [3], > >>>>>>>>>>> * all artifacts to be deployed to the Maven Central > >>>>>>>>>>> > >>>>>>>>>>> Repository > >>>>>>>>>> > >>>>>>>>> > >>>> [4], > >>>>> > >>>>>> > >>>>>>> * source code tag "v2.2.0-RC3" [5], > >>>>>>>> > >>>>>>>>> * website pull request listing the release and publishing > the > >>>>>>>>>>> > >>>>>>>>>>> API > >>>>>>>>>> > >>>>>>>>> > >>>>>> reference manual [6]. > >>>>>>> > >>>>>>>> * Java artifacts were built with Maven 3.5.0 and > >>>>>>>>>>> > >>>>>>>>>>> OpenJDK/Oracle > >>>>>>>>>> > >>>>>>>>> > >>>> JDK > >>>>> > >>>>>> > >>>>>>> 1.8.0_144. > >>>>>>>> > >>>>>>>>> * Python artifacts are deployed along with the source > >>>>>>>>>>> > >>>>>>>>>>> release to > >>>>>>>>>> > >>>>>>>>> > >>>> the > >>>>> > >>>>>> > >>>>>>> dist.apache.org [2]. > >>>>>>>> > >>>>>>>>> > >>>>>>>>>>> The vote will be open for at least 72 hours. It is adopted by > >>>>>>>>>>> > >>>>>>>>>>> majority > >>>>>>>>>> > >>>>>>>>> > >>>>>>> approval, with at least 3 PMC affirmative votes. > >>>>>>>> > >>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> Reuven > >>>>>>>>>>> > >>>>>>>>>>> [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?p > >>>>>>>>>>> rojectId=12319527&version=12341044 > >>>>>>>>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.2.0/ > >>>>>>>>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS > >>>>>>>>>>> [4] > >>>>>>>>>>> > >>>>>>>>>>> https://repository.apache.org/content/repositories/orgapache > >>>>>>>>>> > >>>>>>>>> > >>>> beam-1023/ > >>>>> > >>>>>> [5] https://github.com/apache/beam/tree/v2.2.0-RC3 > >>>>>>>>>>> <https://github.com/apache/beam/tree/v2.2.0-RC3> > >>>>>>>>>>> [6] https://github.com/apache/beam-site/pull/337 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> -- > >>>>>> Gus Katsiapis | Software Engineer | katsia...@google.com | > >>>>>> 650-918-7487 > >>>>>> > >>>>>> <(650)%20918-7487> > >>>>> > >>>> > >>>> > >>>>> > >>>>>> > >>>> > >>> -- > >> Jean-Baptiste Onofré > >> jbono...@apache.org > >> http://blog.nanthrax.net > >> Talend - http://www.talend.com > >> > >> > > > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com >
