Thanks @JB, add you and Tyler as reviewer and is waiting for jenkins job. On Thu, Sep 7, 2017 at 12:25 PM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote:
> It sounds good to me. > > By the way, you will need my help to complete the release process (as you > need some permissions that you don't have). > > Regards > JB > > > On 09/07/2017 01:00 AM, Reuven Lax wrote: > >> It sounds like SQL is still not in, and there are a couple of other PRs >> that people have requested in 2.2.0. I am mostly out next week, so let's >> set September 18 as a target date for cutting the first RC. That should >> hopefully give plenty of time to get SQL and the remaining PRs merged into >> master. >> >> Reuven >> >> On Thu, Aug 31, 2017 at 3:04 PM, Mingmin Xu <mingm...@gmail.com> wrote: >> >> Add https://issues.apache.org/jira/browse/BEAM-2833 which is a blocker to >>> merge DSL_SQL. There may be something wrong in the back-end(maybe >>> RunnerApi) to handle parametered CustomCoder in TestPipeline. >>> >>> On Thu, Aug 31, 2017 at 10:38 AM, Jean-Baptiste Onofré <j...@nanthrax.net> >>> wrote: >>> >>> Fair enough. >>>> >>>> That's fine for me. >>>> >>>> Regards >>>> JB >>>> >>>> On Aug 31, 2017, 19:03, at 19:03, Steve Niemitz <sniem...@apache.org> >>>> wrote: >>>> >>>>> I'll chime in as a user who would love to see 2.2.0 sooner than later, >>>>> specifically for the file IO Eugene mentioned. We're using the AvroIO >>>>> enhancements extensively, but I am hesitant to run from HEAD in master >>>>> in >>>>> production. >>>>> >>>>> On Thu, Aug 31, 2017 at 12:42 PM, Eugene Kirpichov < >>>>> kirpic...@google.com.invalid> wrote: >>>>> >>>>> There are a lot of users including very large production customers >>>>>> >>>>> who have >>>>> >>>>>> been asking specifically for the features that are in 2.2.0 (most of >>>>>> >>>>> them >>>>> >>>>>> accumulated while 2.1.0 was being iterated on) - mostly I'm referring >>>>>> >>>>> to >>>>> >>>>>> the vastly improved file IO - and they have been hesitant to use Beam >>>>>> >>>>> at >>>>> >>>>>> HEAD in production. I think the slight unusualness of having a >>>>>> >>>>> release >>>>> >>>>>> published soon after the previous release is a small price to pay for >>>>>> helping those users :) >>>>>> >>>>>> On Wed, Aug 30, 2017, 11:30 PM Jean-Baptiste Onofré <j...@nanthrax.net> >>>>>> wrote: >>>>>> >>>>>> As we released 2.1.0 couple of weeks ago, it could sound weird to >>>>>>> >>>>>> the >>>>> >>>>>> users to >>>>>>> do a 2.2.0 so fast. If we have a blocking issue, we can do a 2.1.1 >>>>>>> >>>>>> If >>>>> >>>>>> it's >>>>>> >>>>>>> new >>>>>>> features, why not having a release pace in October (2.2.0) ? >>>>>>> >>>>>>> Thoughts ? >>>>>>> >>>>>>> Regards >>>>>>> JB >>>>>>> >>>>>>> On 08/31/2017 08:27 AM, Eugene Kirpichov wrote: >>>>>>> >>>>>>>> I'd suggest to do 2.2.0 as quickly as possible, and target 2.3.0 >>>>>>>> >>>>>>> for >>>>> >>>>>> October. I don't see a reason to delay 2.2.0 until October: >>>>>>>> >>>>>>> there's a >>>>> >>>>>> huge >>>>>>> >>>>>>>> amount of features worth releasing between when 2.1.0 was cut and >>>>>>>> >>>>>>> the >>>>> >>>>>> current HEAD. >>>>>>>> >>>>>>>> On Wed, Aug 30, 2017 at 10:18 PM Jean-Baptiste Onofré >>>>>>>> >>>>>>> <j...@nanthrax.net >>>>> >>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> With a 2.2.0 in October, I think we can try to move forward on >>>>>>>>> >>>>>>>> RedisIO. >>>>>> >>>>>>> >>>>>>>>> I'm now back from vacation and I will resume the work on this >>>>>>>>> >>>>>>>> IO. >>>>> >>>>>> >>>>>>>>> Regards >>>>>>>>> JB >>>>>>>>> >>>>>>>>> On 08/30/2017 11:27 PM, Eugene Kirpichov wrote: >>>>>>>>> >>>>>>>>>> RedisIO in 2.2.0 is very unlikely. There's still a lot of >>>>>>>>>> >>>>>>>>> review >>>>> >>>>>> remaining >>>>>>>>> >>>>>>>>>> last time I checked on the PR. >>>>>>>>>> >>>>>>>>>> On Wed, Aug 30, 2017 at 2:24 PM Vilhelm von Ehrenheim < >>>>>>>>>> vonehrenh...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> Any chance to get the RedisIO in this release? >>>>>>>>>>> [BEAM-1017] Add RedisIO #1687 >>>>>>>>>>> >>>>>>>>>>> Its not my PR but ll be happy to assist if there is anything I >>>>>>>>>>> >>>>>>>>>> can >>>>> >>>>>> do >>>>>> >>>>>>> to >>>>>>> >>>>>>>> help. >>>>>>>>>>> >>>>>>>>>>> On 30 Aug 2017 22:46, "Daniel Ribeiro" >>>>>>>>>>> >>>>>>>>>> <danie...@squareup.com.invalid >>>>>> >>>>>>> >>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> It would be great to get a bump on pubsub >>>>>>>>>>>> <https://github.com/apache/beam/blob/master/pom.xml#L145> >>>>>>>>>>>> >>>>>>>>>>> dependency. >>>>>>> >>>>>>>> It >>>>>>>>> >>>>>>>>>> is >>>>>>>>>>>> currently very outdated (v1-rev10-1.22.0, which was released >>>>>>>>>>>> >>>>>>>>>>> over a >>>>> >>>>>> year >>>>>>>>> >>>>>>>>>> ago >>>>>>>>>>>> <http://repo1.maven.org/maven2/com/google/apis/google- >>>>>>>>>>>> api-services-pubsub/v1-rev10-1.22.0/> >>>>>>>>>>>> ). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Aug 30, 2017 at 1:27 PM, Eugene Kirpichov < >>>>>>>>>>>> kirpic...@google.com.invalid> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Thanks Ismael. I've marked these two issues for fix in >>>>>>>>>>>>> >>>>>>>>>>>> 2.2.0. >>>>> >>>>>> Definitely >>>>>>>>>>> >>>>>>>>>>>> agree that at least the first one must be fixed. >>>>>>>>>>>>> >>>>>>>>>>>>> Here's the current burndown list >>>>>>>>>>>>> >>>>>>>>>>>>> https://issues.apache.org/jira/projects/BEAM/versions/12341044 >>>>> - >>>>> >>>>>> we >>>>>> >>>>>>> should >>>>>>>>>>>> >>>>>>>>>>>>> clean it up. >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Aug 30, 2017 at 1:20 PM Ismaël Mejía >>>>>>>>>>>>> >>>>>>>>>>>> <ieme...@gmail.com> >>>>> >>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> The current master has accumulated a good amount of nice >>>>>>>>>>>>>> >>>>>>>>>>>>> features >>>>> >>>>>> since 2.1.0 so a new release is welcomed. I have two >>>>>>>>>>>>>> >>>>>>>>>>>>> JIRAs/PR >>>>> >>>>>> that >>>>>> >>>>>>> I >>>>>>> >>>>>>>> think are important to check/solve before the cut: >>>>>>>>>>>>>> >>>>>>>>>>>>>> BEAM-2516 (this is a regression on the performance of >>>>>>>>>>>>>> >>>>>>>>>>>>> Direct >>>>> >>>>>> runner >>>>>> >>>>>>> on >>>>>>>>>>> >>>>>>>>>>>> Java). We had never really defined if a performance >>>>>>>>>>>>>> >>>>>>>>>>>>> regression is >>>>> >>>>>> critical to be a blocker. I executed WordCount with the >>>>>>>>>>>>>> >>>>>>>>>>>>> kinglear.txt >>>>>>> >>>>>>>> (170KB) file in version 2.1.0 vs the current 2.2.0-SNAPSHOT >>>>>>>>>>>>>> >>>>>>>>>>>>> and I >>>>> >>>>>> found that the execution time passed from 5s to 126s. So >>>>>>>>>>>>>> >>>>>>>>>>>>> maybe we >>>>> >>>>>> need >>>>>>>>>>> >>>>>>>>>>>> to review this one before the release. I can understand if >>>>>>>>>>>>>> >>>>>>>>>>>>> others >>>>> >>>>>> consider this a minor issue because the Direct runner is >>>>>>>>>>>>>> >>>>>>>>>>>>> not >>>>> >>>>>> supposed >>>>>>> >>>>>>>> to be used for production, but this performance regression >>>>>>>>>>>>>> >>>>>>>>>>>>> can >>>>> >>>>>> cause >>>>>>> >>>>>>>> a >>>>>>>>>>> >>>>>>>>>>>> bad impression for a casual user starting with Beam. >>>>>>>>>>>>>> >>>>>>>>>>>>>> BEAM-2790 (fix reading from Amazon S3 via >>>>>>>>>>>>>> >>>>>>>>>>>>> HadoopFileSystem). I >>>>> >>>>>> think >>>>>>> >>>>>>>> this one is a nice to have. I am not sure that I can tackle >>>>>>>>>>>>>> >>>>>>>>>>>>> it >>>>> >>>>>> for >>>>>> >>>>>>> the >>>>>>>>>>> >>>>>>>>>>>> wednesday cut. I’m OOO until the beginning of next week, >>>>>>>>>>>>>> >>>>>>>>>>>>> but >>>>> >>>>>> maybe >>>>>> >>>>>>> someone else can take a look. In the worst case this is not >>>>>>>>>>>>>> >>>>>>>>>>>>> a >>>>> >>>>>> release >>>>>>> >>>>>>>> blocker but definitely a really nice fix to include. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Aug 30, 2017 at 8:49 PM, Eugene Kirpichov >>>>>>>>>>>>>> <kirpic...@google.com.invalid> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'd like to get the following PRs into 2.2.0: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> #3765 <https://github.com/apache/beam/pull/3765> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> [BEAM-2753 >>>>> >>>>>> <https://issues.apache.org/jira/browse/BEAM-2753>] Fixes >>>>>>>>>>>>>>> >>>>>>>>>>>>>> translation >>>>>>>>>>> >>>>>>>>>>>> of >>>>>>>>>>>>> >>>>>>>>>>>>>> WriteFiles side inputs (important bugfix for >>>>>>>>>>>>>>> >>>>>>>>>>>>>> DynamicDestinations >>>>> >>>>>> in >>>>>>> >>>>>>>> files) >>>>>>>>>>>>>> >>>>>>>>>>>>>>> #3725 <https://github.com/apache/beam/pull/3725> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> [BEAM-2827 >>>>> >>>>>> <https://issues.apache.org/jira/browse/BEAM-2827>] >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Introduces >>>>> >>>>>> AvroIO.watchForNewFiles (parity for AvroIO with TextIO in >>>>>>>>>>>>>>> >>>>>>>>>>>>>> a few >>>>> >>>>>> important >>>>>>>>>>>>> >>>>>>>>>>>>>> features) >>>>>>>>>>>>>>> #3759 <https://github.com/apache/beam/pull/3759> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> [BEAM-2828 >>>>> >>>>>> <https://issues.apache.org/jira/browse/BEAM-2828>] Moves >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Match >>>>> >>>>>> into >>>>>>>>>>> >>>>>>>>>>>> FileIO.match()/matchAll() (to prevent releasing current >>>>>>>>>>>>>>> Match.filepatterns() into 2.2.0 and then having to keep it >>>>>>>>>>>>>>> >>>>>>>>>>>>>> under >>>>> >>>>>> that >>>>>>>>>>> >>>>>>>>>>>> name) >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Aug 30, 2017 at 11:31 AM Mingmin Xu >>>>>>>>>>>>>>> >>>>>>>>>>>>>> <mingm...@gmail.com >>>>> >>>>>> >>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Glad to see that 2.2.0 is coming. Can we include SQL >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> feature in >>>>> >>>>>> next >>>>>>>>>>> >>>>>>>>>>>> release? We're in the final stage and expect to merge >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> back to >>>>> >>>>>> master >>>>>>>>>>> >>>>>>>>>>>> this >>>>>>>>>>>>>> >>>>>>>>>>>>>>> week. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Wed, Aug 30, 2017 at 11:27 AM, Reuven Lax >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> <re...@google.com.invalid >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Now that Beam 2.1.0 has finally completed, I think we >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> should >>>>> >>>>>> cut >>>>>> >>>>>>> Beam >>>>>>>>>>>>> >>>>>>>>>>>>>> 2.2.0 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> soon. I volunteer to coordinate this release. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Are there any pending pull requests that people think >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> should >>>>> >>>>>> be >>>>>> >>>>>>> merged >>>>>>>>>>>>> >>>>>>>>>>>>>> before we cut 2.2.0? If so, please let me know soon, as >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I >>>>> >>>>>> would >>>>>> >>>>>>> like >>>>>>>>>>>> >>>>>>>>>>>>> to >>>>>>>>>>>>>> >>>>>>>>>>>>>>> cut >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> by Wednesday of next week. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Reuven >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> ---- >>>>>>>>>>>>>>>> Mingmin >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Jean-Baptiste Onofré >>>>>>>>> jbono...@apache.org >>>>>>>>> http://blog.nanthrax.net >>>>>>>>> Talend - http://www.talend.com >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> Jean-Baptiste Onofré >>>>>>> jbono...@apache.org >>>>>>> http://blog.nanthrax.net >>>>>>> Talend - http://www.talend.com >>>>>>> >>>>>>> >>>>>> >>>> >>> >>> >>> -- >>> ---- >>> Mingmin >>> >>> >> > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com > -- ---- Mingmin
