*Blockers* DRILL-5659 - problem is identified, Parth is working on the fix.
*Not blockers* DRILL-5660 - Vitalii DRILL-5468 - Jinfeng *Possible blockers* DRILL-5669 - Boaz Ben-Zvi / Paul NPE on CSV - Paul Kind regards Arina On Thu, Jul 13, 2017 at 1:28 AM, Paul Rogers <[email protected]> wrote: > I withdraw my suggestion to fix DRILL-5660. Was a big hassle for us to > track down the incompatibility the issue internally, but production users > probably don’t attempt to use one build of Drill with metadata files > created from another. We can worry about the issue later only if a user > actually runs into it. > > As suggested earlier, let’s just document this obscure case in a release > note. > > - Paul > > > On Jul 12, 2017, at 3:04 PM, Jinfeng Ni <[email protected]> wrote: > > > > Upgrade from 1.10 to 1.11 is irreversible only when parquet metadata > cache > > file is involved. > > > > I might be wrong, but I thought that if someone is running on 1.10.0, and > > he wants to go back to 1.0.0, where the first version of parquet > meatadata > > cache was introduced, can we guarantee that Drillbit would work > correctly? > > I'm trying to understand what DRILL-5660 asks for is a new requirement, > or > > an requirement being there starting from Drill 1.0.0? > > > > Regression failures are normally regarded as release blocker. > > > > > > > > On Wed, Jul 12, 2017 at 2:54 PM, Paul Rogers <[email protected]> wrote: > > > >> If we don’t do DRILL-5660 for 1.11 then we don’t need to do it at all, > so > >> that is a time savings. We could handle this via a release note item > that > >> simply states that an upgrade from 1.10 to 1.11 is irreversible: we do > not > >> support the ability to fall back to the old version. Everyone OK with > that? > >> > >> We are seeing some stress test regression failures that, I’m told, will > >> show up as JIRA entries shortly. > >> > >> - Paul > >> > >>> On Jul 12, 2017, at 11:05 AM, Jinfeng Ni <[email protected]> wrote: > >>> > >>> I put some analysis in DRILL-5468. > >>> > >>> Also, IMHO, DRILL-5660 is not a release blocker. > >>> > >>> > >>> > >>> On Tue, Jul 11, 2017 at 11:27 AM, Parth Chandra <[email protected]> > >> wrote: > >>> > >>>> Re DRILL-5634: > >>>> From the Apache commons web page [1] - > >>>> > >>>> Please note that Apache Commons Crypto doesn't implement the > >> cryptographic > >>>> algorithm such as AES directly.* It wraps to Openssl or JCE which > >> implement > >>>> the algorithms.* > >>>> > >>>> > >>>> Since we are cleared for using Openssl and JCE, we are OK to include > the > >>>> AES code in the UDFs. > >>>> > >>>> > >>>> Parth > >>>> > >>>> [1] https://commons.apache.org/proper/commons-crypto/ > >>>> > >>>> On Tue, Jul 11, 2017 at 10:56 AM, Arina Yelchiyeva < > >>>> [email protected]> wrote: > >>>> > >>>>> During today hangouts we agreed on approximate cut-off date - 14 > July, > >>>> 2017 > >>>>> (date may be shifted due to the blockers). > >>>>> I'll start preparing test RC on Thursday to solve any release > >> preparation > >>>>> issues beforehand. > >>>>> > >>>>> *Blockers* > >>>>> DRILL-5660 - Vitalii > >>>>> > >>>>> *Possible blockers* > >>>>> DRILL-5468 - Jinfeng > >>>>> NPE on CSV source - Paul > >>>>> DRILL-5659 - Parth > >>>>> > >>>>> *Will be included if done before the cut-off date* > >>>>> DRILL-5616 / DRILL-5665 - Boaz > >>>>> DRILL-5634 - Parth will take a look at Apache commons implementation > >> and > >>>>> update. > >>>>> > >>>>> Kind regards > >>>>> Arina > >>>>> > >>>>> On Tue, Jul 11, 2017 at 3:14 PM, Charles Givre <[email protected]> > >> wrote: > >>>>> > >>>>>> Hi Arina, > >>>>>> I can’t make the hangout, but we have a few options. Basically, > since > >>>>> all > >>>>>> the hashing algorithms have no export restrictions, we could: > >>>>>> 1. Remove the AES encrypt/decrypt until I can figure out what we > have > >>>> to > >>>>>> do (and do it) > >>>>>> 2. I can try to research it today or tomorrow to figure out exactly > >>>> what > >>>>>> we have to do to include AES. > >>>>>> 3. Remove the entire package > >>>>>> > >>>>>> I’d like to see the hashing functions at least included because > we’ve > >>>> had > >>>>>> a lot of requests for that functionality. I also don’t want this > to > >>>>> hold > >>>>>> things up, so please let me know your preference. Regardless, I’m > >>>> going > >>>>> to > >>>>>> start trying to figure out the restrictions so we’ll know for the > >>>> future. > >>>>>> > >>>>>> Best, > >>>>>> — C > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On Jul 11, 2017, at 05:58, Arina Yelchiyeva < > >>>>> [email protected]> > >>>>>> wrote: > >>>>>>> > >>>>>>> @Jinfeng, > >>>>>>> Let's discuss today on Drill hangout final cut-off date. > >>>>>>> > >>>>>>> @Charles > >>>>>>> Since there were questions about AES in DRILL-5634, should we > exclude > >>>>> it > >>>>>>> from the candidates for 1.11.0 release? > >>>>>>> > >>>>>>> @Boaz > >>>>>>> Do we have any ETA when DRILL-5616 and DRILL-5665 will pass code > >>>>> review? > >>>>>>> > >>>>>>> Also I believe we should include DRILL-5660 where we need to bump > up > >>>>>>> parquet metadata version. > >>>>>>> @Vitalii > >>>>>>> Do you have any ETA for the fix? > >>>>>>> > >>>>>>> Kind regards > >>>>>>> Arina > >>>>>>> > >>>>>>> On Tue, Jul 11, 2017 at 6:02 AM, Boaz Ben-Zvi <[email protected]> > >>>>> wrote: > >>>>>>> > >>>>>>>> Hi Arina, > >>>>>>>> > >>>>>>>> Two PR have just been submitted, relating to the Hash Aggr > >>>> Spill: > >>>>>>>> DRILL-5616 and DRILL-5665 . > >>>>>>>> Can these be considered for 1.11 as well ? > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>> Boaz > >>>>>>>> > >>>>>>>> On 7/10/17, 1:49 PM, "Paul Rogers" <[email protected]> wrote: > >>>>>>>> > >>>>>>>> DRILL-5601 is no longer a blocker for 1.11; we’ll defer that > >>>> until > >>>>>>>> after the release to allow others more time to work on other > >>>> features > >>>>>>>> rather than doing code reviews. > >>>>>>>> > >>>>>>>> - Paul > >>>>>>>> > >>>>>>>>> On Jul 10, 2017, at 1:43 PM, Jinfeng Ni <[email protected]> wrote: > >>>>>>>>> > >>>>>>>>> Hi Arina, > >>>>>>>>> > >>>>>>>>> It has been a while since we discussed 1.11.0 release. Any > update > >>>>>>>> on the > >>>>>>>>> new release plan? Do we have a cut-off date ? > >>>>>>>>> > >>>>>>>>> In the past, I believe Drill release is a time-bounded, not > feature > >>>>>>>>> bounded. If certain fixes/features could not make it within a > >>>> certain > >>>>>>>>> time-frame, we had better defer them into future release, unless > >>>>>>>> those > >>>>>>>>> fixes/features are regarded as blocking issues for a new release. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Tue, Jul 4, 2017 at 5:14 AM, Arina Yelchiyeva < > >>>>>>>> [email protected] > >>>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Latest update: > >>>>>>>>>> > >>>>>>>>>> We wait for the following Jiras before final cut off. > >>>>>>>>>> 1. DRILL-5601 - Paul > >>>>>>>>>> 2. DRILL-5420 - Kunal > >>>>>>>>>> 3. DRILL-5634 - Charles > >>>>>>>>>> > >>>>>>>>>> Kind regards > >>>>>>>>>> Arina > >>>>>>>>>> > >>>>>>>>>> On Fri, Jun 30, 2017 at 9:29 PM, Kunal Khatua <[email protected] > > > >>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Parth's already +1'd the process with a minor update. So, we're > >>>>>>>> good to > >>>>>>>>>> go > >>>>>>>>>>> on DRILL-5420. I'll change the label on that to > ready-to-commit. > >>>>>>>>>>> > >>>>>>>>>>> <http://www.mapr.com/> > >>>>>>>>>>> > >>>>>>>>>>> ________________________________ > >>>>>>>>>>> From: Parth Chandra <[email protected]> > >>>>>>>>>>> Sent: Thursday, June 29, 2017 9:44:39 AM > >>>>>>>>>>> To: [email protected] > >>>>>>>>>>> Subject: Re: [DISCUSS] Drill 1.11.0 release > >>>>>>>>>>> > >>>>>>>>>>> I'm Ok with the pcap format plugin if Pauls comments are also > >>>>>>>> addressed. > >>>>>>>>>> If > >>>>>>>>>>> the changes are done before your cutoff deadline,we can merge > it > >>>>>>>> in. > >>>>>>>>>>> > >>>>>>>>>>> On Thu, Jun 29, 2017 at 12:26 AM, Arina Yelchiyeva < > >>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Hi all, > >>>>>>>>>>>> > >>>>>>>>>>>> here is the list of Jira we are waiting before making final > cut > >>>>>>>> off: > >>>>>>>>>>>> 1. DRILL-5601 - Paul > >>>>>>>>>>>> 2. DRILL-3640 - Kunal > >>>>>>>>>>>> 3. DRILL-5420 - Kunal > >>>>>>>>>>>> > >>>>>>>>>>>> Please let me know if there are any others Jiras we should > wait > >>>>>>>> for. > >>>>>>>>>>>> > >>>>>>>>>>>> @Charles, > >>>>>>>>>>>> you have mentioned that you want to include some UDFs in Drill > >>>>>>>> 1.11, do > >>>>>>>>>>> you > >>>>>>>>>>>> have any ETA for the PR? > >>>>>>>>>>>> > >>>>>>>>>>>> @Paul / Parth > >>>>>>>>>>>> I see there is active CR for pcap format PR, do we want to > >>>>>>>> include it > >>>>>>>>>> in > >>>>>>>>>>>> this release as well? > >>>>>>>>>>>> > >>>>>>>>>>>> Kind regards > >>>>>>>>>>>> Arina > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Thu, Jun 29, 2017 at 10:05 AM, Arina Yelchiyeva < > >>>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Parth, thanks a lot the instructions. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Wed, Jun 28, 2017 at 8:24 PM, Parth Chandra < > >>>>>>>> [email protected]> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> FWIW, I put release instructions culled from gists written > by > >>>>>>>>>> previous > >>>>>>>>>>>>>> release managers into an updated gist here [1] > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> HTH > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Parth > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> [1] https://github.com/parthchandra/drill/wiki/Drill- > >>>>>>>>>> Release-Process > >>>>>>>>>>>>>> <https://github.com/parthchandra/drill/wiki/Drill- > >>>>>>>> Release-Process> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Tue, Jun 27, 2017 at 5:24 PM, Paul Rogers < > >>>> [email protected]> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi Arina, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I have two projects in progress: one is a candidate for > 1.11, > >>>>>>>> the > >>>>>>>>>>>> other > >>>>>>>>>>>>>> is > >>>>>>>>>>>>>>> not. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The “managed external sort” is the good candidate for 1.11: > >>>>>>>> The PR > >>>>>>>>>>> for > >>>>>>>>>>>>>>> DRILL-5601 is being reviewed. Getting that in will allow us > >>>> to > >>>>>>>>>>> enable > >>>>>>>>>>>>>> the > >>>>>>>>>>>>>>> “new” external sort by default. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> The other project is to address Drill’s memory > fragmentation > >>>>>>>> issue > >>>>>>>>>>>> with > >>>>>>>>>>>>>>> vectors over 16 MB in size. Two PRs have been open for a > >>>> while. > >>>>>>>>>> They > >>>>>>>>>>>> are > >>>>>>>>>>>>>>> the first of several. Each takes a while to review. So, the > >>>>>>>>>> project > >>>>>>>>>>>> does > >>>>>>>>>>>>>>> not fit 1.11. The work can go into the next release > instead. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> - Paul > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>> > >>>>>> > >>>>> > >>>> > >> > >> > >
