Regarding to BEAM-2833, a PR is there for review( https://github.com/apache/beam/pull/3803). Once it's completed, I'll move forward for SQL merge.
Mingmin On Wed, Sep 6, 2017 at 4:00 PM, Reuven Lax <re...@google.com.invalid> wrote: > It sounds like SQL is still not in, and there are a couple of other PRs > that people have requested in 2.2.0. I am mostly out next week, so let's > set September 18 as a target date for cutting the first RC. That should > hopefully give plenty of time to get SQL and the remaining PRs merged into > master. > > Reuven > > On Thu, Aug 31, 2017 at 3:04 PM, Mingmin Xu <mingm...@gmail.com> wrote: > > > Add https://issues.apache.org/jira/browse/BEAM-2833 which is a blocker > to > > merge DSL_SQL. There may be something wrong in the back-end(maybe > > RunnerApi) to handle parametered CustomCoder in TestPipeline. > > > > On Thu, Aug 31, 2017 at 10:38 AM, Jean-Baptiste Onofré <j...@nanthrax.net> > > wrote: > > > > > Fair enough. > > > > > > That's fine for me. > > > > > > Regards > > > JB > > > > > > On Aug 31, 2017, 19:03, at 19:03, Steve Niemitz <sniem...@apache.org> > > > wrote: > > > >I'll chime in as a user who would love to see 2.2.0 sooner than later, > > > >specifically for the file IO Eugene mentioned. We're using the AvroIO > > > >enhancements extensively, but I am hesitant to run from HEAD in master > > > >in > > > >production. > > > > > > > >On Thu, Aug 31, 2017 at 12:42 PM, Eugene Kirpichov < > > > >kirpic...@google.com.invalid> wrote: > > > > > > > >> There are a lot of users including very large production customers > > > >who have > > > >> been asking specifically for the features that are in 2.2.0 (most of > > > >them > > > >> accumulated while 2.1.0 was being iterated on) - mostly I'm > referring > > > >to > > > >> the vastly improved file IO - and they have been hesitant to use > Beam > > > >at > > > >> HEAD in production. I think the slight unusualness of having a > > > >release > > > >> published soon after the previous release is a small price to pay > for > > > >> helping those users :) > > > >> > > > >> On Wed, Aug 30, 2017, 11:30 PM Jean-Baptiste Onofré < > j...@nanthrax.net> > > > >> wrote: > > > >> > > > >> > As we released 2.1.0 couple of weeks ago, it could sound weird to > > > >the > > > >> > users to > > > >> > do a 2.2.0 so fast. If we have a blocking issue, we can do a 2.1.1 > > > >If > > > >> it's > > > >> > new > > > >> > features, why not having a release pace in October (2.2.0) ? > > > >> > > > > >> > Thoughts ? > > > >> > > > > >> > Regards > > > >> > JB > > > >> > > > > >> > On 08/31/2017 08:27 AM, Eugene Kirpichov wrote: > > > >> > > I'd suggest to do 2.2.0 as quickly as possible, and target 2.3.0 > > > >for > > > >> > > October. I don't see a reason to delay 2.2.0 until October: > > > >there's a > > > >> > huge > > > >> > > amount of features worth releasing between when 2.1.0 was cut > and > > > >the > > > >> > > current HEAD. > > > >> > > > > > >> > > On Wed, Aug 30, 2017 at 10:18 PM Jean-Baptiste Onofré > > > ><j...@nanthrax.net > > > >> > > > > >> > > wrote: > > > >> > > > > > >> > >> With a 2.2.0 in October, I think we can try to move forward on > > > >> RedisIO. > > > >> > >> > > > >> > >> I'm now back from vacation and I will resume the work on this > > > >IO. > > > >> > >> > > > >> > >> Regards > > > >> > >> JB > > > >> > >> > > > >> > >> On 08/30/2017 11:27 PM, Eugene Kirpichov wrote: > > > >> > >>> RedisIO in 2.2.0 is very unlikely. There's still a lot of > > > >review > > > >> > >> remaining > > > >> > >>> last time I checked on the PR. > > > >> > >>> > > > >> > >>> On Wed, Aug 30, 2017 at 2:24 PM Vilhelm von Ehrenheim < > > > >> > >>> vonehrenh...@gmail.com> wrote: > > > >> > >>> > > > >> > >>>> Any chance to get the RedisIO in this release? > > > >> > >>>> [BEAM-1017] Add RedisIO #1687 > > > >> > >>>> > > > >> > >>>> Its not my PR but ll be happy to assist if there is anything > I > > > >can > > > >> do > > > >> > to > > > >> > >>>> help. > > > >> > >>>> > > > >> > >>>> On 30 Aug 2017 22:46, "Daniel Ribeiro" > > > >> <danie...@squareup.com.invalid > > > >> > > > > > >> > >>>> wrote: > > > >> > >>>> > > > >> > >>>>> It would be great to get a bump on pubsub > > > >> > >>>>> <https://github.com/apache/beam/blob/master/pom.xml#L145> > > > >> > dependency. > > > >> > >> It > > > >> > >>>>> is > > > >> > >>>>> currently very outdated (v1-rev10-1.22.0, which was released > > > >over a > > > >> > >> year > > > >> > >>>>> ago > > > >> > >>>>> <http://repo1.maven.org/maven2/com/google/apis/google- > > > >> > >>>>> api-services-pubsub/v1-rev10-1.22.0/> > > > >> > >>>>> ). > > > >> > >>>>> > > > >> > >>>>> > > > >> > >>>>> On Wed, Aug 30, 2017 at 1:27 PM, Eugene Kirpichov < > > > >> > >>>>> kirpic...@google.com.invalid> wrote: > > > >> > >>>>> > > > >> > >>>>>> Thanks Ismael. I've marked these two issues for fix in > > > >2.2.0. > > > >> > >>>> Definitely > > > >> > >>>>>> agree that at least the first one must be fixed. > > > >> > >>>>>> > > > >> > >>>>>> Here's the current burndown list > > > >> > >>>>>> > > > >https://issues.apache.org/jira/projects/BEAM/versions/12341044 - > > > >> we > > > >> > >>>>> should > > > >> > >>>>>> clean it up. > > > >> > >>>>>> > > > >> > >>>>>> On Wed, Aug 30, 2017 at 1:20 PM Ismaël Mejía > > > ><ieme...@gmail.com> > > > >> > >>>> wrote: > > > >> > >>>>>> > > > >> > >>>>>>> The current master has accumulated a good amount of nice > > > >features > > > >> > >>>>>>> since 2.1.0 so a new release is welcomed. I have two > > > >JIRAs/PR > > > >> that > > > >> > I > > > >> > >>>>>>> think are important to check/solve before the cut: > > > >> > >>>>>>> > > > >> > >>>>>>> BEAM-2516 (this is a regression on the performance of > > > >Direct > > > >> runner > > > >> > >>>> on > > > >> > >>>>>>> Java). We had never really defined if a performance > > > >regression is > > > >> > >>>>>>> critical to be a blocker. I executed WordCount with the > > > >> > kinglear.txt > > > >> > >>>>>>> (170KB) file in version 2.1.0 vs the current > 2.2.0-SNAPSHOT > > > >and I > > > >> > >>>>>>> found that the execution time passed from 5s to 126s. So > > > >maybe we > > > >> > >>>> need > > > >> > >>>>>>> to review this one before the release. I can understand if > > > >others > > > >> > >>>>>>> consider this a minor issue because the Direct runner is > > > >not > > > >> > supposed > > > >> > >>>>>>> to be used for production, but this performance regression > > > >can > > > >> > cause > > > >> > >>>> a > > > >> > >>>>>>> bad impression for a casual user starting with Beam. > > > >> > >>>>>>> > > > >> > >>>>>>> BEAM-2790 (fix reading from Amazon S3 via > > > >HadoopFileSystem). I > > > >> > think > > > >> > >>>>>>> this one is a nice to have. I am not sure that I can > tackle > > > >it > > > >> for > > > >> > >>>> the > > > >> > >>>>>>> wednesday cut. I’m OOO until the beginning of next week, > > > >but > > > >> maybe > > > >> > >>>>>>> someone else can take a look. In the worst case this is > not > > > >a > > > >> > release > > > >> > >>>>>>> blocker but definitely a really nice fix to include. > > > >> > >>>>>>> > > > >> > >>>>>>> > > > >> > >>>>>>> On Wed, Aug 30, 2017 at 8:49 PM, Eugene Kirpichov > > > >> > >>>>>>> <kirpic...@google.com.invalid> wrote: > > > >> > >>>>>>>> I'd like to get the following PRs into 2.2.0: > > > >> > >>>>>>>> > > > >> > >>>>>>>> #3765 <https://github.com/apache/beam/pull/3765> > > > >[BEAM-2753 > > > >> > >>>>>>>> <https://issues.apache.org/jira/browse/BEAM-2753>] Fixes > > > >> > >>>> translation > > > >> > >>>>>> of > > > >> > >>>>>>>> WriteFiles side inputs (important bugfix for > > > >DynamicDestinations > > > >> > in > > > >> > >>>>>>> files) > > > >> > >>>>>>>> #3725 <https://github.com/apache/beam/pull/3725> > > > >[BEAM-2827 > > > >> > >>>>>>>> <https://issues.apache.org/jira/browse/BEAM-2827>] > > > >Introduces > > > >> > >>>>>>>> AvroIO.watchForNewFiles (parity for AvroIO with TextIO in > > > >a few > > > >> > >>>>>> important > > > >> > >>>>>>>> features) > > > >> > >>>>>>>> #3759 <https://github.com/apache/beam/pull/3759> > > > >[BEAM-2828 > > > >> > >>>>>>>> <https://issues.apache.org/jira/browse/BEAM-2828>] Moves > > > >Match > > > >> > >>>> into > > > >> > >>>>>>>> FileIO.match()/matchAll() (to prevent releasing current > > > >> > >>>>>>>> Match.filepatterns() into 2.2.0 and then having to keep > it > > > >under > > > >> > >>>> that > > > >> > >>>>>>> name) > > > >> > >>>>>>>> > > > >> > >>>>>>>> On Wed, Aug 30, 2017 at 11:31 AM Mingmin Xu > > > ><mingm...@gmail.com > > > >> > > > > >> > >>>>>> wrote: > > > >> > >>>>>>>> > > > >> > >>>>>>>>> Glad to see that 2.2.0 is coming. Can we include SQL > > > >feature in > > > >> > >>>> next > > > >> > >>>>>>>>> release? We're in the final stage and expect to merge > > > >back to > > > >> > >>>> master > > > >> > >>>>>>> this > > > >> > >>>>>>>>> week. > > > >> > >>>>>>>>> > > > >> > >>>>>>>>> On Wed, Aug 30, 2017 at 11:27 AM, Reuven Lax > > > >> > >>>>> <re...@google.com.invalid > > > >> > >>>>>>> > > > >> > >>>>>>>>> wrote: > > > >> > >>>>>>>>> > > > >> > >>>>>>>>>> Now that Beam 2.1.0 has finally completed, I think we > > > >should > > > >> cut > > > >> > >>>>>> Beam > > > >> > >>>>>>>>> 2.2.0 > > > >> > >>>>>>>>>> soon. I volunteer to coordinate this release. > > > >> > >>>>>>>>>> > > > >> > >>>>>>>>>> Are there any pending pull requests that people think > > > >should > > > >> be > > > >> > >>>>>> merged > > > >> > >>>>>>>>>> before we cut 2.2.0? If so, please let me know soon, as > > > >I > > > >> would > > > >> > >>>>> like > > > >> > >>>>>>> to > > > >> > >>>>>>>>> cut > > > >> > >>>>>>>>>> by Wednesday of next week. > > > >> > >>>>>>>>>> > > > >> > >>>>>>>>>> Thanks, > > > >> > >>>>>>>>>> > > > >> > >>>>>>>>>> Reuven > > > >> > >>>>>>>>>> > > > >> > >>>>>>>>> > > > >> > >>>>>>>>> > > > >> > >>>>>>>>> > > > >> > >>>>>>>>> -- > > > >> > >>>>>>>>> ---- > > > >> > >>>>>>>>> Mingmin > > > >> > >>>>>>>>> > > > >> > >>>>>>> > > > >> > >>>>>> > > > >> > >>>>> > > > >> > >>>> > > > >> > >>> > > > >> > >> > > > >> > >> -- > > > >> > >> Jean-Baptiste Onofré > > > >> > >> jbono...@apache.org > > > >> > >> http://blog.nanthrax.net > > > >> > >> Talend - http://www.talend.com > > > >> > >> > > > >> > > > > > >> > > > > >> > -- > > > >> > Jean-Baptiste Onofré > > > >> > jbono...@apache.org > > > >> > http://blog.nanthrax.net > > > >> > Talend - http://www.talend.com > > > >> > > > > >> > > > > > > > > > > > -- > > ---- > > Mingmin > > > -- ---- Mingmin
