Hi Arina, I just realized that I did not push the C++ client fix ( DRILL-5659) to master. I have just done so.
Sorry if I messed things up for the release, but we should get this fix in. Parth On Fri, Jul 14, 2017 at 8:15 AM, Paul Rogers <[email protected]> wrote: > I looked into one NPE on CSV. Turns out it is already fixed by previous > 1.11 PRs, so we’re good. > > There is another reported by a user on the user list, but I’ve not heard > back about getting sample data. Let’s not hold the release for that. > > - Paul > > > On Jul 14, 2017, at 2:38 AM, Arina Yelchiyeva < > [email protected]> wrote: > > > > *Blockers* > > DRILL-5659 - Parth (done) > > DRILL-5669 - Boaz Ben-Zvi (fix will be ready in a couple of days) > > > > *Possible blockers* > > NPE on CSV - Paul can you please provide an update? > > > > Kind regards > > Arina > > > > On Thu, Jul 13, 2017 at 2:46 PM, Arina Yelchiyeva < > > [email protected]> wrote: > > > >> *Blockers* > >> DRILL-5659 - problem is identified, Parth is working on the fix. > >> > >> *Not blockers* > >> DRILL-5660 - Vitalii > >> DRILL-5468 - Jinfeng > >> > >> *Possible blockers* > >> DRILL-5669 - Boaz Ben-Zvi / Paul > >> NPE on CSV - Paul > >> > >> Kind regards > >> Arina > >> > >> > >> On Thu, Jul 13, 2017 at 1:28 AM, Paul Rogers <[email protected]> wrote: > >> > >>> I withdraw my suggestion to fix DRILL-5660. Was a big hassle for us to > >>> track down the incompatibility the issue internally, but production > users > >>> probably don’t attempt to use one build of Drill with metadata files > >>> created from another. We can worry about the issue later only if a user > >>> actually runs into it. > >>> > >>> As suggested earlier, let’s just document this obscure case in a > release > >>> note. > >>> > >>> - Paul > >>> > >>>> On Jul 12, 2017, at 3:04 PM, Jinfeng Ni <[email protected]> wrote: > >>>> > >>>> Upgrade from 1.10 to 1.11 is irreversible only when parquet metadata > >>> cache > >>>> file is involved. > >>>> > >>>> I might be wrong, but I thought that if someone is running on 1.10.0, > >>> and > >>>> he wants to go back to 1.0.0, where the first version of parquet > >>> meatadata > >>>> cache was introduced, can we guarantee that Drillbit would work > >>> correctly? > >>>> I'm trying to understand what DRILL-5660 asks for is a new > requirement, > >>> or > >>>> an requirement being there starting from Drill 1.0.0? > >>>> > >>>> Regression failures are normally regarded as release blocker. > >>>> > >>>> > >>>> > >>>> On Wed, Jul 12, 2017 at 2:54 PM, Paul Rogers <[email protected]> > wrote: > >>>> > >>>>> If we don’t do DRILL-5660 for 1.11 then we don’t need to do it at > all, > >>> so > >>>>> that is a time savings. We could handle this via a release note item > >>> that > >>>>> simply states that an upgrade from 1.10 to 1.11 is irreversible: we > do > >>> not > >>>>> support the ability to fall back to the old version. Everyone OK with > >>> that? > >>>>> > >>>>> We are seeing some stress test regression failures that, I’m told, > will > >>>>> show up as JIRA entries shortly. > >>>>> > >>>>> - Paul > >>>>> > >>>>>> On Jul 12, 2017, at 11:05 AM, Jinfeng Ni <[email protected]> wrote: > >>>>>> > >>>>>> I put some analysis in DRILL-5468. > >>>>>> > >>>>>> Also, IMHO, DRILL-5660 is not a release blocker. > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Tue, Jul 11, 2017 at 11:27 AM, Parth Chandra <[email protected]> > >>>>> wrote: > >>>>>> > >>>>>>> Re DRILL-5634: > >>>>>>> From the Apache commons web page [1] - > >>>>>>> > >>>>>>> Please note that Apache Commons Crypto doesn't implement the > >>>>> cryptographic > >>>>>>> algorithm such as AES directly.* It wraps to Openssl or JCE which > >>>>> implement > >>>>>>> the algorithms.* > >>>>>>> > >>>>>>> > >>>>>>> Since we are cleared for using Openssl and JCE, we are OK to > include > >>> the > >>>>>>> AES code in the UDFs. > >>>>>>> > >>>>>>> > >>>>>>> Parth > >>>>>>> > >>>>>>> [1] https://commons.apache.org/proper/commons-crypto/ > >>>>>>> > >>>>>>> On Tue, Jul 11, 2017 at 10:56 AM, Arina Yelchiyeva < > >>>>>>> [email protected]> wrote: > >>>>>>> > >>>>>>>> During today hangouts we agreed on approximate cut-off date - 14 > >>> July, > >>>>>>> 2017 > >>>>>>>> (date may be shifted due to the blockers). > >>>>>>>> I'll start preparing test RC on Thursday to solve any release > >>>>> preparation > >>>>>>>> issues beforehand. > >>>>>>>> > >>>>>>>> *Blockers* > >>>>>>>> DRILL-5660 - Vitalii > >>>>>>>> > >>>>>>>> *Possible blockers* > >>>>>>>> DRILL-5468 - Jinfeng > >>>>>>>> NPE on CSV source - Paul > >>>>>>>> DRILL-5659 - Parth > >>>>>>>> > >>>>>>>> *Will be included if done before the cut-off date* > >>>>>>>> DRILL-5616 / DRILL-5665 - Boaz > >>>>>>>> DRILL-5634 - Parth will take a look at Apache commons > implementation > >>>>> and > >>>>>>>> update. > >>>>>>>> > >>>>>>>> Kind regards > >>>>>>>> Arina > >>>>>>>> > >>>>>>>> On Tue, Jul 11, 2017 at 3:14 PM, Charles Givre <[email protected]> > >>>>> wrote: > >>>>>>>> > >>>>>>>>> Hi Arina, > >>>>>>>>> I can’t make the hangout, but we have a few options. Basically, > >>> since > >>>>>>>> all > >>>>>>>>> the hashing algorithms have no export restrictions, we could: > >>>>>>>>> 1. Remove the AES encrypt/decrypt until I can figure out what we > >>> have > >>>>>>> to > >>>>>>>>> do (and do it) > >>>>>>>>> 2. I can try to research it today or tomorrow to figure out > >>> exactly > >>>>>>> what > >>>>>>>>> we have to do to include AES. > >>>>>>>>> 3. Remove the entire package > >>>>>>>>> > >>>>>>>>> I’d like to see the hashing functions at least included because > >>> we’ve > >>>>>>> had > >>>>>>>>> a lot of requests for that functionality. I also don’t want > this > >>> to > >>>>>>>> hold > >>>>>>>>> things up, so please let me know your preference. Regardless, > I’m > >>>>>>> going > >>>>>>>> to > >>>>>>>>> start trying to figure out the restrictions so we’ll know for the > >>>>>>> future. > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> — C > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> On Jul 11, 2017, at 05:58, Arina Yelchiyeva < > >>>>>>>> [email protected]> > >>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> @Jinfeng, > >>>>>>>>>> Let's discuss today on Drill hangout final cut-off date. > >>>>>>>>>> > >>>>>>>>>> @Charles > >>>>>>>>>> Since there were questions about AES in DRILL-5634, should we > >>> exclude > >>>>>>>> it > >>>>>>>>>> from the candidates for 1.11.0 release? > >>>>>>>>>> > >>>>>>>>>> @Boaz > >>>>>>>>>> Do we have any ETA when DRILL-5616 and DRILL-5665 will pass code > >>>>>>>> review? > >>>>>>>>>> > >>>>>>>>>> Also I believe we should include DRILL-5660 where we need to > bump > >>> up > >>>>>>>>>> parquet metadata version. > >>>>>>>>>> @Vitalii > >>>>>>>>>> Do you have any ETA for the fix? > >>>>>>>>>> > >>>>>>>>>> Kind regards > >>>>>>>>>> Arina > >>>>>>>>>> > >>>>>>>>>> On Tue, Jul 11, 2017 at 6:02 AM, Boaz Ben-Zvi < > [email protected]> > >>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hi Arina, > >>>>>>>>>>> > >>>>>>>>>>> Two PR have just been submitted, relating to the Hash Aggr > >>>>>>> Spill: > >>>>>>>>>>> DRILL-5616 and DRILL-5665 . > >>>>>>>>>>> Can these be considered for 1.11 as well ? > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> > >>>>>>>>>>> Boaz > >>>>>>>>>>> > >>>>>>>>>>> On 7/10/17, 1:49 PM, "Paul Rogers" <[email protected]> wrote: > >>>>>>>>>>> > >>>>>>>>>>> DRILL-5601 is no longer a blocker for 1.11; we’ll defer that > >>>>>>> until > >>>>>>>>>>> after the release to allow others more time to work on other > >>>>>>> features > >>>>>>>>>>> rather than doing code reviews. > >>>>>>>>>>> > >>>>>>>>>>> - Paul > >>>>>>>>>>> > >>>>>>>>>>>> On Jul 10, 2017, at 1:43 PM, Jinfeng Ni <[email protected]> > wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> Hi Arina, > >>>>>>>>>>>> > >>>>>>>>>>>> It has been a while since we discussed 1.11.0 release. Any > >>> update > >>>>>>>>>>> on the > >>>>>>>>>>>> new release plan? Do we have a cut-off date ? > >>>>>>>>>>>> > >>>>>>>>>>>> In the past, I believe Drill release is a time-bounded, not > >>> feature > >>>>>>>>>>>> bounded. If certain fixes/features could not make it within a > >>>>>>> certain > >>>>>>>>>>>> time-frame, we had better defer them into future release, > unless > >>>>>>>>>>> those > >>>>>>>>>>>> fixes/features are regarded as blocking issues for a new > >>> release. > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Tue, Jul 4, 2017 at 5:14 AM, Arina Yelchiyeva < > >>>>>>>>>>> [email protected] > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Latest update: > >>>>>>>>>>>>> > >>>>>>>>>>>>> We wait for the following Jiras before final cut off. > >>>>>>>>>>>>> 1. DRILL-5601 - Paul > >>>>>>>>>>>>> 2. DRILL-5420 - Kunal > >>>>>>>>>>>>> 3. DRILL-5634 - Charles > >>>>>>>>>>>>> > >>>>>>>>>>>>> Kind regards > >>>>>>>>>>>>> Arina > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Fri, Jun 30, 2017 at 9:29 PM, Kunal Khatua < > >>> [email protected]> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Parth's already +1'd the process with a minor update. So, > >>> we're > >>>>>>>>>>> good to > >>>>>>>>>>>>> go > >>>>>>>>>>>>>> on DRILL-5420. I'll change the label on that to > >>> ready-to-commit. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> <http://www.mapr.com/> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> ________________________________ > >>>>>>>>>>>>>> From: Parth Chandra <[email protected]> > >>>>>>>>>>>>>> Sent: Thursday, June 29, 2017 9:44:39 AM > >>>>>>>>>>>>>> To: [email protected] > >>>>>>>>>>>>>> Subject: Re: [DISCUSS] Drill 1.11.0 release > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I'm Ok with the pcap format plugin if Pauls comments are > also > >>>>>>>>>>> addressed. > >>>>>>>>>>>>> If > >>>>>>>>>>>>>> the changes are done before your cutoff deadline,we can > merge > >>> it > >>>>>>>>>>> in. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Thu, Jun 29, 2017 at 12:26 AM, Arina Yelchiyeva < > >>>>>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> here is the list of Jira we are waiting before making final > >>> cut > >>>>>>>>>>> off: > >>>>>>>>>>>>>>> 1. DRILL-5601 - Paul > >>>>>>>>>>>>>>> 2. DRILL-3640 - Kunal > >>>>>>>>>>>>>>> 3. DRILL-5420 - Kunal > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Please let me know if there are any others Jiras we should > >>> wait > >>>>>>>>>>> for. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> @Charles, > >>>>>>>>>>>>>>> you have mentioned that you want to include some UDFs in > >>> Drill > >>>>>>>>>>> 1.11, do > >>>>>>>>>>>>>> you > >>>>>>>>>>>>>>> have any ETA for the PR? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> @Paul / Parth > >>>>>>>>>>>>>>> I see there is active CR for pcap format PR, do we want to > >>>>>>>>>>> include it > >>>>>>>>>>>>> in > >>>>>>>>>>>>>>> this release as well? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Kind regards > >>>>>>>>>>>>>>> Arina > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Thu, Jun 29, 2017 at 10:05 AM, Arina Yelchiyeva < > >>>>>>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Parth, thanks a lot the instructions. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Wed, Jun 28, 2017 at 8:24 PM, Parth Chandra < > >>>>>>>>>>> [email protected]> > >>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> FWIW, I put release instructions culled from gists > written > >>> by > >>>>>>>>>>>>> previous > >>>>>>>>>>>>>>>>> release managers into an updated gist here [1] > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> HTH > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Parth > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> [1] https://github.com/parthchandra/drill/wiki/Drill- > >>>>>>>>>>>>> Release-Process > >>>>>>>>>>>>>>>>> <https://github.com/parthchandra/drill/wiki/Drill- > >>>>>>>>>>> Release-Process> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> On Tue, Jun 27, 2017 at 5:24 PM, Paul Rogers < > >>>>>>> [email protected]> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Hi Arina, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> I have two projects in progress: one is a candidate for > >>> 1.11, > >>>>>>>>>>> the > >>>>>>>>>>>>>>> other > >>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>> not. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> The “managed external sort” is the good candidate for > >>> 1.11: > >>>>>>>>>>> The PR > >>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>> DRILL-5601 is being reviewed. Getting that in will allow > >>> us > >>>>>>> to > >>>>>>>>>>>>>> enable > >>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>> “new” external sort by default. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> The other project is to address Drill’s memory > >>> fragmentation > >>>>>>>>>>> issue > >>>>>>>>>>>>>>> with > >>>>>>>>>>>>>>>>>> vectors over 16 MB in size. Two PRs have been open for a > >>>>>>> while. > >>>>>>>>>>>>> They > >>>>>>>>>>>>>>> are > >>>>>>>>>>>>>>>>>> the first of several. Each takes a while to review. So, > >>> the > >>>>>>>>>>>>> project > >>>>>>>>>>>>>>> does > >>>>>>>>>>>>>>>>>> not fit 1.11. The work can go into the next release > >>> instead. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> - Paul > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>>>> > >>> > >>> > >> > >
