I don't think there're some particular permissions, #3782 is merged to master. So from SQL perspective I'm good to the 2.2.0 release, will do a quick POC job for verification purpose.
Mingmin On Mon, Sep 11, 2017 at 9:46 AM, Reuven Lax <re...@google.com.invalid> wrote: > Are these permissions that only you have, or does anyone on the PMC have > these permissions? I'm asking so that in the future if you are unavailable, > we know who has these permissions. We should also make sure this is all > documented on the Beam release guide. > > Reuven > > On Wed, Sep 6, 2017 at 9:25 PM, Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > > > It sounds good to me. > > > > By the way, you will need my help to complete the release process (as you > > need some permissions that you don't have). > > > > Regards > > JB > > > > > > On 09/07/2017 01:00 AM, Reuven Lax wrote: > > > >> It sounds like SQL is still not in, and there are a couple of other PRs > >> that people have requested in 2.2.0. I am mostly out next week, so let's > >> set September 18 as a target date for cutting the first RC. That should > >> hopefully give plenty of time to get SQL and the remaining PRs merged > into > >> master. > >> > >> Reuven > >> > >> On Thu, Aug 31, 2017 at 3:04 PM, Mingmin Xu <mingm...@gmail.com> wrote: > >> > >> Add https://issues.apache.org/jira/browse/BEAM-2833 which is a blocker > to > >>> merge DSL_SQL. There may be something wrong in the back-end(maybe > >>> RunnerApi) to handle parametered CustomCoder in TestPipeline. > >>> > >>> On Thu, Aug 31, 2017 at 10:38 AM, Jean-Baptiste Onofré < > j...@nanthrax.net> > >>> wrote: > >>> > >>> Fair enough. > >>>> > >>>> That's fine for me. > >>>> > >>>> Regards > >>>> JB > >>>> > >>>> On Aug 31, 2017, 19:03, at 19:03, Steve Niemitz <sniem...@apache.org> > >>>> wrote: > >>>> > >>>>> I'll chime in as a user who would love to see 2.2.0 sooner than > later, > >>>>> specifically for the file IO Eugene mentioned. We're using the > AvroIO > >>>>> enhancements extensively, but I am hesitant to run from HEAD in > master > >>>>> in > >>>>> production. > >>>>> > >>>>> On Thu, Aug 31, 2017 at 12:42 PM, Eugene Kirpichov < > >>>>> kirpic...@google.com.invalid> wrote: > >>>>> > >>>>> There are a lot of users including very large production customers > >>>>>> > >>>>> who have > >>>>> > >>>>>> been asking specifically for the features that are in 2.2.0 (most of > >>>>>> > >>>>> them > >>>>> > >>>>>> accumulated while 2.1.0 was being iterated on) - mostly I'm > referring > >>>>>> > >>>>> to > >>>>> > >>>>>> the vastly improved file IO - and they have been hesitant to use > Beam > >>>>>> > >>>>> at > >>>>> > >>>>>> HEAD in production. I think the slight unusualness of having a > >>>>>> > >>>>> release > >>>>> > >>>>>> published soon after the previous release is a small price to pay > for > >>>>>> helping those users :) > >>>>>> > >>>>>> On Wed, Aug 30, 2017, 11:30 PM Jean-Baptiste Onofré < > j...@nanthrax.net> > >>>>>> wrote: > >>>>>> > >>>>>> As we released 2.1.0 couple of weeks ago, it could sound weird to > >>>>>>> > >>>>>> the > >>>>> > >>>>>> users to > >>>>>>> do a 2.2.0 so fast. If we have a blocking issue, we can do a 2.1.1 > >>>>>>> > >>>>>> If > >>>>> > >>>>>> it's > >>>>>> > >>>>>>> new > >>>>>>> features, why not having a release pace in October (2.2.0) ? > >>>>>>> > >>>>>>> Thoughts ? > >>>>>>> > >>>>>>> Regards > >>>>>>> JB > >>>>>>> > >>>>>>> On 08/31/2017 08:27 AM, Eugene Kirpichov wrote: > >>>>>>> > >>>>>>>> I'd suggest to do 2.2.0 as quickly as possible, and target 2.3.0 > >>>>>>>> > >>>>>>> for > >>>>> > >>>>>> October. I don't see a reason to delay 2.2.0 until October: > >>>>>>>> > >>>>>>> there's a > >>>>> > >>>>>> huge > >>>>>>> > >>>>>>>> amount of features worth releasing between when 2.1.0 was cut and > >>>>>>>> > >>>>>>> the > >>>>> > >>>>>> current HEAD. > >>>>>>>> > >>>>>>>> On Wed, Aug 30, 2017 at 10:18 PM Jean-Baptiste Onofré > >>>>>>>> > >>>>>>> <j...@nanthrax.net > >>>>> > >>>>>> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> With a 2.2.0 in October, I think we can try to move forward on > >>>>>>>>> > >>>>>>>> RedisIO. > >>>>>> > >>>>>>> > >>>>>>>>> I'm now back from vacation and I will resume the work on this > >>>>>>>>> > >>>>>>>> IO. > >>>>> > >>>>>> > >>>>>>>>> Regards > >>>>>>>>> JB > >>>>>>>>> > >>>>>>>>> On 08/30/2017 11:27 PM, Eugene Kirpichov wrote: > >>>>>>>>> > >>>>>>>>>> RedisIO in 2.2.0 is very unlikely. There's still a lot of > >>>>>>>>>> > >>>>>>>>> review > >>>>> > >>>>>> remaining > >>>>>>>>> > >>>>>>>>>> last time I checked on the PR. > >>>>>>>>>> > >>>>>>>>>> On Wed, Aug 30, 2017 at 2:24 PM Vilhelm von Ehrenheim < > >>>>>>>>>> vonehrenh...@gmail.com> wrote: > >>>>>>>>>> > >>>>>>>>>> Any chance to get the RedisIO in this release? > >>>>>>>>>>> [BEAM-1017] Add RedisIO #1687 > >>>>>>>>>>> > >>>>>>>>>>> Its not my PR but ll be happy to assist if there is anything I > >>>>>>>>>>> > >>>>>>>>>> can > >>>>> > >>>>>> do > >>>>>> > >>>>>>> to > >>>>>>> > >>>>>>>> help. > >>>>>>>>>>> > >>>>>>>>>>> On 30 Aug 2017 22:46, "Daniel Ribeiro" > >>>>>>>>>>> > >>>>>>>>>> <danie...@squareup.com.invalid > >>>>>> > >>>>>>> > >>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> It would be great to get a bump on pubsub > >>>>>>>>>>>> <https://github.com/apache/beam/blob/master/pom.xml#L145> > >>>>>>>>>>>> > >>>>>>>>>>> dependency. > >>>>>>> > >>>>>>>> It > >>>>>>>>> > >>>>>>>>>> is > >>>>>>>>>>>> currently very outdated (v1-rev10-1.22.0, which was released > >>>>>>>>>>>> > >>>>>>>>>>> over a > >>>>> > >>>>>> year > >>>>>>>>> > >>>>>>>>>> ago > >>>>>>>>>>>> <http://repo1.maven.org/maven2/com/google/apis/google- > >>>>>>>>>>>> api-services-pubsub/v1-rev10-1.22.0/> > >>>>>>>>>>>> ). > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Wed, Aug 30, 2017 at 1:27 PM, Eugene Kirpichov < > >>>>>>>>>>>> kirpic...@google.com.invalid> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks Ismael. I've marked these two issues for fix in > >>>>>>>>>>>>> > >>>>>>>>>>>> 2.2.0. > >>>>> > >>>>>> Definitely > >>>>>>>>>>> > >>>>>>>>>>>> agree that at least the first one must be fixed. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Here's the current burndown list > >>>>>>>>>>>>> > >>>>>>>>>>>>> https://issues.apache.org/jira/projects/BEAM/versions/ > 12341044 > >>>>> - > >>>>> > >>>>>> we > >>>>>> > >>>>>>> should > >>>>>>>>>>>> > >>>>>>>>>>>>> clean it up. > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Wed, Aug 30, 2017 at 1:20 PM Ismaël Mejía > >>>>>>>>>>>>> > >>>>>>>>>>>> <ieme...@gmail.com> > >>>>> > >>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>> The current master has accumulated a good amount of nice > >>>>>>>>>>>>>> > >>>>>>>>>>>>> features > >>>>> > >>>>>> since 2.1.0 so a new release is welcomed. I have two > >>>>>>>>>>>>>> > >>>>>>>>>>>>> JIRAs/PR > >>>>> > >>>>>> that > >>>>>> > >>>>>>> I > >>>>>>> > >>>>>>>> think are important to check/solve before the cut: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> BEAM-2516 (this is a regression on the performance of > >>>>>>>>>>>>>> > >>>>>>>>>>>>> Direct > >>>>> > >>>>>> runner > >>>>>> > >>>>>>> on > >>>>>>>>>>> > >>>>>>>>>>>> Java). We had never really defined if a performance > >>>>>>>>>>>>>> > >>>>>>>>>>>>> regression is > >>>>> > >>>>>> critical to be a blocker. I executed WordCount with the > >>>>>>>>>>>>>> > >>>>>>>>>>>>> kinglear.txt > >>>>>>> > >>>>>>>> (170KB) file in version 2.1.0 vs the current 2.2.0-SNAPSHOT > >>>>>>>>>>>>>> > >>>>>>>>>>>>> and I > >>>>> > >>>>>> found that the execution time passed from 5s to 126s. So > >>>>>>>>>>>>>> > >>>>>>>>>>>>> maybe we > >>>>> > >>>>>> need > >>>>>>>>>>> > >>>>>>>>>>>> to review this one before the release. I can understand if > >>>>>>>>>>>>>> > >>>>>>>>>>>>> others > >>>>> > >>>>>> consider this a minor issue because the Direct runner is > >>>>>>>>>>>>>> > >>>>>>>>>>>>> not > >>>>> > >>>>>> supposed > >>>>>>> > >>>>>>>> to be used for production, but this performance regression > >>>>>>>>>>>>>> > >>>>>>>>>>>>> can > >>>>> > >>>>>> cause > >>>>>>> > >>>>>>>> a > >>>>>>>>>>> > >>>>>>>>>>>> bad impression for a casual user starting with Beam. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> BEAM-2790 (fix reading from Amazon S3 via > >>>>>>>>>>>>>> > >>>>>>>>>>>>> HadoopFileSystem). I > >>>>> > >>>>>> think > >>>>>>> > >>>>>>>> this one is a nice to have. I am not sure that I can tackle > >>>>>>>>>>>>>> > >>>>>>>>>>>>> it > >>>>> > >>>>>> for > >>>>>> > >>>>>>> the > >>>>>>>>>>> > >>>>>>>>>>>> wednesday cut. I’m OOO until the beginning of next week, > >>>>>>>>>>>>>> > >>>>>>>>>>>>> but > >>>>> > >>>>>> maybe > >>>>>> > >>>>>>> someone else can take a look. In the worst case this is not > >>>>>>>>>>>>>> > >>>>>>>>>>>>> a > >>>>> > >>>>>> release > >>>>>>> > >>>>>>>> blocker but definitely a really nice fix to include. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Wed, Aug 30, 2017 at 8:49 PM, Eugene Kirpichov > >>>>>>>>>>>>>> <kirpic...@google.com.invalid> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I'd like to get the following PRs into 2.2.0: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> #3765 <https://github.com/apache/beam/pull/3765> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> [BEAM-2753 > >>>>> > >>>>>> <https://issues.apache.org/jira/browse/BEAM-2753>] Fixes > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> translation > >>>>>>>>>>> > >>>>>>>>>>>> of > >>>>>>>>>>>>> > >>>>>>>>>>>>>> WriteFiles side inputs (important bugfix for > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> DynamicDestinations > >>>>> > >>>>>> in > >>>>>>> > >>>>>>>> files) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> #3725 <https://github.com/apache/beam/pull/3725> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> [BEAM-2827 > >>>>> > >>>>>> <https://issues.apache.org/jira/browse/BEAM-2827>] > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> Introduces > >>>>> > >>>>>> AvroIO.watchForNewFiles (parity for AvroIO with TextIO in > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> a few > >>>>> > >>>>>> important > >>>>>>>>>>>>> > >>>>>>>>>>>>>> features) > >>>>>>>>>>>>>>> #3759 <https://github.com/apache/beam/pull/3759> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> [BEAM-2828 > >>>>> > >>>>>> <https://issues.apache.org/jira/browse/BEAM-2828>] Moves > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> Match > >>>>> > >>>>>> into > >>>>>>>>>>> > >>>>>>>>>>>> FileIO.match()/matchAll() (to prevent releasing current > >>>>>>>>>>>>>>> Match.filepatterns() into 2.2.0 and then having to keep it > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> under > >>>>> > >>>>>> that > >>>>>>>>>>> > >>>>>>>>>>>> name) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Wed, Aug 30, 2017 at 11:31 AM Mingmin Xu > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> <mingm...@gmail.com > >>>>> > >>>>>> > >>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Glad to see that 2.2.0 is coming. Can we include SQL > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> feature in > >>>>> > >>>>>> next > >>>>>>>>>>> > >>>>>>>>>>>> release? We're in the final stage and expect to merge > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> back to > >>>>> > >>>>>> master > >>>>>>>>>>> > >>>>>>>>>>>> this > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> week. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Wed, Aug 30, 2017 at 11:27 AM, Reuven Lax > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> <re...@google.com.invalid > >>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Now that Beam 2.1.0 has finally completed, I think we > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> should > >>>>> > >>>>>> cut > >>>>>> > >>>>>>> Beam > >>>>>>>>>>>>> > >>>>>>>>>>>>>> 2.2.0 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> soon. I volunteer to coordinate this release. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Are there any pending pull requests that people think > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> should > >>>>> > >>>>>> be > >>>>>> > >>>>>>> merged > >>>>>>>>>>>>> > >>>>>>>>>>>>>> before we cut 2.2.0? If so, please let me know soon, as > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> I > >>>>> > >>>>>> would > >>>>>> > >>>>>>> like > >>>>>>>>>>>> > >>>>>>>>>>>>> to > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> cut > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> by Wednesday of next week. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Reuven > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>> ---- > >>>>>>>>>>>>>>>> Mingmin > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Jean-Baptiste Onofré > >>>>>>>>> jbono...@apache.org > >>>>>>>>> http://blog.nanthrax.net > >>>>>>>>> Talend - http://www.talend.com > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> -- > >>>>>>> Jean-Baptiste Onofré > >>>>>>> jbono...@apache.org > >>>>>>> http://blog.nanthrax.net > >>>>>>> Talend - http://www.talend.com > >>>>>>> > >>>>>>> > >>>>>> > >>>> > >>> > >>> > >>> -- > >>> ---- > >>> Mingmin > >>> > >>> > >> > > -- > > Jean-Baptiste Onofré > > jbono...@apache.org > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > > -- ---- Mingmin
