Hi Jason, I'm fine that release 1.5.0 starts from 03197d0f2. One thing I need point out is that the pre-commit test suite has some testcase's plan output modified because of the partition pruning change. If we run pre-commit against the candidate release for 1.5.0, we have to use a prior version of pre-commit suite.
On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse <[email protected]> wrote: > That sounds reasonable to me. I'll start preparing a release candidate. > > Jinfeng, are you okay waiting for the next release to include this change? > > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <[email protected]> wrote: > >> Hey Jason, >> >> Can you start the release vote? We've let this drag on too long. >> >> I suggest we start from 03197d0f2. It seems like the partition pruning >> changes are pretty complicated and on a pretty main path to add right >> before the vote. Since Hsuan's fix (pr 349) hasn't been reviewed, I think >> we should start the vote without it. >> >> Jacques >> >> >> -- >> Jacques Nadeau >> CTO and Co-Founder, Dremio >> >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <[email protected]> wrote: >> >> > Yes, just send a pull request. >> > https://github.com/apache/drill/pull/349 >> > >> > Can Jinfeng review it? >> > >> > Thanks, >> > >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <[email protected]> >> wrote: >> > >> > > Sean probably is quite close to find a fix for this issue. Let's see >> > > whether he can post a patch soon. >> > > >> > > >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli >> > > <[email protected]> wrote: >> > > > DRILL-4323 only happens when assertions are turned on (my install >> > script >> > > > automatically enables assertions). >> > > > >> > > > - Rahul >> > > > >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <[email protected]> >> > > wrote: >> > > > >> > > >> Rahul, >> > > >> >> > > >> Let us know if you can reproduce the issue with assertions turned >> OFF. >> > > If >> > > >> you cannot, then I think this should not be a blocker for 1.5, and >> > Hsuan >> > > >> will continue to look into fixing this. >> > > >> >> > > >> -- Zelaine >> > > >> >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen < >> > [email protected]> >> > > >> wrote: >> > > >> >> > > >> > I do see the same error with assertion turned on. >> > > >> > >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni < >> [email protected]> >> > > >> wrote: >> > > >> > >> > > >> > > Krystal, by any chance, did you turn on assertion when you start >> > > >> > drillbit? >> > > >> > > >> > > >> > > I think the error in drill-4323 will be raised when assertion = >> > on. >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen < >> > > [email protected]> >> > > >> > > wrote: >> > > >> > > > I am running drill-1.4 and is able to run count(*) >> successfully. >> > > >> > > > >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session >> > set >> > > >> > > > `store.hive.optimize_scan_with_native_readers` = true; >> > > >> > > > >> > +-------+--------------------------------------------------------+ >> > > >> > > > | ok | summary >> > | >> > > >> > > > >> > +-------+--------------------------------------------------------+ >> > > >> > > > | true | store.hive.optimize_scan_with_native_readers >> > updated. | >> > > >> > > > >> > +-------+--------------------------------------------------------+ >> > > >> > > > 1 row selected (0.374 seconds) >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select >> count(*) >> > > from >> > > >> > > > lineitem_parquet_partitioned_hive_string; >> > > >> > > > +---------+ >> > > >> > > > | EXPR$0 | >> > > >> > > > +---------+ >> > > >> > > > | 60175 | >> > > >> > > > +---------+ >> > > >> > > > 1 row selected (2.986 seconds) >> > > >> > > > >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni < >> > > [email protected]> >> > > >> > > wrote: >> > > >> > > > >> > > >> > > >> Venki and I did some investigation for DRILL-4323. The issue >> > > >> reported >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems >> > to >> > > us >> > > >> > > >> this is not a regression from 1.4.0; it's a regression from >> > 1.3.0 >> > > >> > > >> probably. >> > > >> > > >> >> > > >> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader in >> > > stead >> > > >> of >> > > >> > > >> HiveReader for "select count(*) from hive_table" query. >> > However, >> > > >> the >> > > >> > > >> Project after the scan produces empty schema. Before >> > DRILL-4083, >> > > >> > > >> Drill uses HiveScan, which works fine. >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse >> > > >> > > >> <[email protected]> wrote: >> > > >> > > >> > We could revert the change to the StoragePlugin interface >> to >> > > add >> > > >> the >> > > >> > > >> > getLogicalOptimizerRules >> > > >> > > >> > and getPhysicalOptimizerRules, and only put those in >> > > >> > > >> AbstractStoragePlugin >> > > >> > > >> > with default implementations. >> > > >> > > >> > >> > > >> > > >> > We could then update the docs to tell users to not >> implement >> > > the >> > > >> > > >> interface >> > > >> > > >> > directly, and instead extend the abstract class, which I >> > > believe >> > > >> > would >> > > >> > > >> > allow us to make changes like this in the future without >> > > breaking >> > > >> > > >> plugins. >> > > >> > > >> > At a major release version we could then just pull up all >> of >> > > the >> > > >> new >> > > >> > > >> > methods we added to the abstract class into the interface, >> or >> > > just >> > > >> > get >> > > >> > > >> rid >> > > >> > > >> > of it all-together. >> > > >> > > >> > >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti < >> > > >> > > >> [email protected]> >> > > >> > > >> > wrote: >> > > >> > > >> > >> > > >> > > >> >> Here is the partial fix: >> > > >> > > >> >> >> > > >> > > >> >> >> > > >> > > >> >> > > >> > > >> > > >> > >> > > >> >> > > >> > >> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b >> > > >> > > >> >> >> > > >> > > >> >> If the existing StragePlugins are implementing >> > > >> > AbstractStoragePlugin >> > > >> > > >> class >> > > >> > > >> >> this fix should resolve the backward compat issue. If they >> > are >> > > >> > > >> implementing >> > > >> > > >> >> StoragePlugin interface directly then it won't resolve the >> > > issue. >> > > >> > Not >> > > >> > > >> sure >> > > >> > > >> >> how to handle it in that case. Let me know if there any >> ways >> > > to >> > > >> > > resolve >> > > >> > > >> >> this issue completely. >> > > >> > > >> >> >> > > >> > > >> >> Thanks >> > > >> > > >> >> Venki >> > > >> > > >> >> >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha < >> > > [email protected] >> > > >> > >> > > >> > > >> wrote: >> > > >> > > >> >> >> > > >> > > >> >> > I was able to do the build with maven 3.3.9 (after doing >> > > 'brew >> > > >> > > install >> > > >> > > >> >> > maven'). Here are my env variables related to maven: >> > > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec >> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m >> > > >> > > >> >> > >> > > >> > > >> >> > >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau < >> > > >> > > [email protected]> >> > > >> > > >> >> > wrote: >> > > >> > > >> >> > >> > > >> > > >> >> > > Aman, for reference on the Maven version issue. You >> can >> > > see >> > > >> the >> > > >> > > >> output >> > > >> > > >> >> of >> > > >> > > >> >> > > the shade plugin between the two Apache Builds. If you >> > > search >> > > >> > for >> > > >> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven >> > 3.0.5) >> > > >> > fails >> > > >> > > >> >> because >> > > >> > > >> >> > it >> > > >> > > >> >> > > includes a bunch of hadoop (and other libraries that >> are >> > > >> listed >> > > >> > > as >> > > >> > > >> >> being >> > > >> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3), >> those >> > > >> items >> > > >> > > are >> > > >> > > >> >> > included >> > > >> > > >> >> > > so that the build completes successfully. >> > > >> > > >> >> > > >> > > >> > > >> >> > > >> https://builds.apache.org/job/drill-scm/647/consoleText >> > > >> > > >> >> > > >> https://builds.apache.org/job/drill-scm/648/consoleText >> > > >> > > >> >> > > >> > > >> > > >> >> > > -- >> > > >> > > >> >> > > Jacques Nadeau >> > > >> > > >> >> > > CTO and Co-Founder, Dremio >> > > >> > > >> >> > > >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli < >> > > >> > > >> >> > > [email protected]> wrote: >> > > >> > > >> >> > > >> > > >> > > >> >> > > > Jason, >> > > >> > > >> >> > > > >> > > >> > > >> >> > > > As of now I haven't narrowed down the exact commit >> > which >> > > >> > caused >> > > >> > > >> >> > > DRILL-4323 >> > > >> > > >> >> > > > and I do not know the scope of the fix. >> > > >> > > >> >> > > > >> > > >> > > >> >> > > > - Rahul >> > > >> > > >> >> > > > >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse < >> > > >> > > >> >> > > [email protected] >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > wrote: >> > > >> > > >> >> > > > >> > > >> > > >> >> > > > > I think it makes sense to wait for Venki's fix of >> > the >> > > >> API, >> > > >> > as >> > > >> > > >> >> > > releasing a >> > > >> > > >> >> > > > > version with the change would make >> > > reverting/refactoring >> > > >> > the >> > > >> > > API >> > > >> > > >> >> > > breaking >> > > >> > > >> >> > > > > change less useful. >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > > Do you have any idea about the scope of fixing the >> > > hive >> > > >> > > native >> > > >> > > >> >> > reader, >> > > >> > > >> >> > > or >> > > >> > > >> >> > > > > which change since the 1.4 release broke it? Is >> > > someone >> > > >> > > >> available >> > > >> > > >> >> to >> > > >> > > >> >> > > look >> > > >> > > >> >> > > > > at it today? >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul >> challapalli < >> > > >> > > >> >> > > > > [email protected]> wrote: >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > > > Just found an issue with hive native parquet >> > reader >> > > >> > > >> (DRILL-4323). >> > > >> > > >> >> > > This >> > > >> > > >> >> > > > > is a >> > > >> > > >> >> > > > > > regression from 1.4 and in my opinion should be >> > > treated >> > > >> > as >> > > >> > > a >> > > >> > > >> >> > blocker. >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > > - Rahul >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki >> Korukanti < >> > > >> > > >> >> > > > > > [email protected]> >> > > >> > > >> >> > > > > > wrote: >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was breaking the >> public >> > > >> > > interface >> > > >> > > >> API. >> > > >> > > >> >> > Let >> > > >> > > >> >> > > > me >> > > >> > > >> >> > > > > > work >> > > >> > > >> >> > > > > > > on a patch to make it backward compatible. >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > > Thanks >> > > >> > > >> >> > > > > > > Venki >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven >> > Phillips >> > > < >> > > >> > > >> >> > > [email protected] >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > > > > wrote: >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > > > I just wanted to bring up an issue that I >> just >> > > now >> > > >> > > >> >> discovered, >> > > >> > > >> >> > > that >> > > >> > > >> >> > > > > has >> > > >> > > >> >> > > > > > > > caused me a fair amount of grief. >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > > >> > > https://github.com/apache/drill/pull/300/commits >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and >> > causes >> > > >> > > >> >> StoragePlugins >> > > >> > > >> >> > > > that >> > > >> > > >> >> > > > > > were >> > > >> > > >> >> > > > > > > > compiled against currently released versions >> > of >> > > >> Drill >> > > >> > > to >> > > >> > > >> no >> > > >> > > >> >> > > longer >> > > >> > > >> >> > > > > > > > functional properly. I would prefer that >> this >> > > >> > breaking >> > > >> > > >> change >> > > >> > > >> >> > be >> > > >> > > >> >> > > > > > modified >> > > >> > > >> >> > > > > > > > to be backward compatible if possible. >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason >> > > Altekruse < >> > > >> > > >> >> > > > > > > > [email protected]> >> > > >> > > >> >> > > > > > > > wrote: >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > > > Hi Aman, >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > > This is the failure that he was seeing. He >> > > >> figured >> > > >> > > out >> > > >> > > >> that >> > > >> > > >> >> > the >> > > >> > > >> >> > > > new >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not being >> > > respected >> > > >> > when >> > > >> > > the >> > > >> > > >> >> > build >> > > >> > > >> >> > > > was >> > > >> > > >> >> > > > > > run >> > > >> > > >> >> > > > > > > > with >> > > >> > > >> >> > > > > > > > > an older Maven version, causing the jar >> size >> > > to >> > > >> > > increase >> > > >> > > >> >> > > > > > significantly. >> > > >> > > >> >> > > > > > > > He >> > > >> > > >> >> > > > > > > > > added an enforcer to make sure the JAR >> > didn't >> > > >> grow >> > > >> > > >> >> > > unexpectedly. >> > > >> > > >> >> > > > > Can >> > > >> > > >> >> > > > > > > you >> > > >> > > >> >> > > > > > > > > try to update your maven version and >> re-run >> > > the >> > > >> > > build? >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > > - Jason >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman >> > Sinha < >> > > >> > > >> >> > > > [email protected]> >> > > >> > > >> >> > > > > > > > wrote: >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the following >> build >> > > >> failure >> > > >> > > on >> > > >> > > >> the >> > > >> > > >> >> > > latest >> > > >> > > >> >> > > > > > > master >> > > >> > > >> >> > > > > > > > > > branch...is this what you saw for the >> > Apache >> > > >> > build >> > > >> > > ? >> > > >> > > >> My >> > > >> > > >> >> > mvn >> > > >> > > >> >> > > > > > version >> > > >> > > >> >> > > > > > > > > output >> > > >> > > >> >> > > > > > > > > > is shown below. Should we all be >> > upgrading >> > > to >> > > >> a >> > > >> > > newer >> > > >> > > >> >> mvn >> > > >> > > >> >> > ? >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > [INFO] --- >> > > maven-enforcer-plugin:1.3.1:enforce >> > > >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @ >> > > drill-jdbc-all >> > > >> > --- >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0: >> > > >> > > >> >> > > > > > >> org.apache.maven.plugins.enforcer.RequireFilesSize >> > > >> > > >> >> > > > > > > > > failed >> > > >> > > >> >> > > > > > > > > > with message: >> > > >> > > >> >> > > > > > > > > > The file >> drill-jdbc-all-1.5.0-SNAPSHOT.jar >> > > is >> > > >> > > outside >> > > >> > > >> the >> > > >> > > >> >> > > > > expected >> > > >> > > >> >> > > > > > > size >> > > >> > > >> >> > > > > > > > > > range. >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > This is likely due to >> > you >> > > >> > adding >> > > >> > > new >> > > >> > > >> >> > > > > dependencies >> > > >> > > >> >> > > > > > > to >> > > >> > > >> >> > > > > > > > a >> > > >> > > >> >> > > > > > > > > > java-exec and not updating the excludes >> in >> > > this >> > > >> > > >> module. >> > > >> > > >> >> > This >> > > >> > > >> >> > > is >> > > >> > > >> >> > > > > > > > important >> > > >> > > >> >> > > > > > > > > > as it minimizes the size of the >> dependency >> > > of >> > > >> > Drill >> > > >> > > >> >> > > application >> > > >> > > >> >> > > > > > > users. >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > >> > > >> > > >> >> > > >> > > >> > > >> >> > >> > > >> > > >> >> >> > > >> > > >> >> > > >> > > >> > > >> > >> > > >> >> > > >> > >> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar >> > > >> > > >> >> > > > > > > > > > size (44664121) too large. Max. is >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > >> > > >> > > >> >> > > >> > > >> > > >> >> > >> > > >> > > >> >> >> > > >> > > >> >> > > >> > > >> > > >> > >> > > >> >> > > >> > >> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > >> > > Administrators-MacBook-Pro-144:incubator-drill >> > > >> > > asinha$ >> > > >> > > >> >> mvn >> > > >> > > >> >> > > > > > --version >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5 >> > > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da; >> > > >> > > >> >> > > > > > > > 2013-02-19 >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800) >> > > >> > > >> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3 >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle >> > > >> > Corporation >> > > >> > > >> >> > > > > > > > > > Java home: >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre >> > > >> > > >> >> > > > > > > > > > Default locale: en_US, platform >> encoding: >> > > UTF-8 >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5", >> > > arch: >> > > >> > > >> "x86_64", >> > > >> > > >> >> > > family: >> > > >> > > >> >> > > > > > "mac" >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques >> > > >> Nadeau < >> > > >> > > >> >> > > > > > [email protected]> >> > > >> > > >> >> > > > > > > > > > wrote: >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > Build back to normal. It looks like >> the >> > > >> Apache >> > > >> > > >> server >> > > >> > > >> >> was >> > > >> > > >> >> > > > using >> > > >> > > >> >> > > > > > an >> > > >> > > >> >> > > > > > > > old >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I switched to >> > > >> something >> > > >> > > more >> > > >> > > >> >> > recent, >> > > >> > > >> >> > > > the >> > > >> > > >> >> > > > > > > build >> > > >> > > >> >> > > > > > > > > > > passed. >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > -- >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, >> Jacques >> > > >> > Nadeau < >> > > >> > > >> >> > > > > > > [email protected]> >> > > >> > > >> >> > > > > > > > > > > wrote: >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the Apache >> > > build >> > > >> to >> > > >> > > fail. >> > > >> > > >> >> > > > > > > Investigating... >> > > >> > > >> >> > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > > -- >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio >> > > >> > > >> >> > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, >> > Jacques >> > > >> > > Nadeau < >> > > >> > > >> >> > > > > > > > [email protected]> >> > > >> > > >> >> > > > > > > > > > > > wrote: >> > > >> > > >> >> > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> I got clean regression runs as >> well. >> > > I've >> > > >> > > merged >> > > >> > > >> the >> > > >> > > >> >> > > > patch. >> > > >> > > >> >> > > > > > > > > > > >> >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to start the >> release >> > > >> > process? >> > > >> > > >> >> > > > > > > > > > > >> >> > > >> > > >> >> > > > > > > > > > > >> -- >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio >> > > >> > > >> >> > > > > > > > > > > >> >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, >> > > Abhishek >> > > >> > > >> Girish < >> > > >> > > >> >> > > > > > > > [email protected] >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> wrote: >> > > >> > > >> >> > > > > > > > > > > >> >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs. >> TPC-H >> > > >> SF100 >> > > >> > > was >> > > >> > > >> also >> > > >> > > >> >> > > > > > successful. >> > > >> > > >> >> > > > > > > > > > > >>> >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, >> > > rahul >> > > >> > > >> >> challapalli < >> > > >> > > >> >> > > > > > > > > > > >>> [email protected]> >> wrote: >> > > >> > > >> >> > > > > > > > > > > >>> >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run with >> > > your >> > > >> > > branch. >> > > >> > > >> >> Will >> > > >> > > >> >> > > let >> > > >> > > >> >> > > > > you >> > > >> > > >> >> > > > > > > know >> > > >> > > >> >> > > > > > > > > > once >> > > >> > > >> >> > > > > > > > > > > it >> > > >> > > >> >> > > > > > > > > > > >>> > finishes >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, >> > > >> Jacques >> > > >> > > >> Nadeau < >> > > >> > > >> >> > > > > > > > > > [email protected]> >> > > >> > > >> >> > > > > > > > > > > >>> > wrote: >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I have >> an >> > > >> > updated >> > > >> > > >> patch >> > > >> > > >> >> > for >> > > >> > > >> >> > > > 4291 >> > > >> > > >> >> > > > > > > that >> > > >> > > >> >> > > > > > > > is >> > > >> > > >> >> > > > > > > > > > > >>> ready. >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems that >> > > >> something >> > > >> > > isn't >> > > >> > > >> >> > > working >> > > >> > > >> >> > > > > with >> > > >> > > >> >> > > > > > > our >> > > >> > > >> >> > > > > > > > > > > >>> extended >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been able >> to >> > > run >> > > >> an >> > > >> > > >> extended >> > > >> > > >> >> > > > > > regression. >> > > >> > > >> >> > > > > > > > > Unit >> > > >> > > >> >> > > > > > > > > > > >>> tests >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else possibly >> > > able >> > > >> to >> > > >> > > run a >> > > >> > > >> >> > > > regression >> > > >> > > >> >> > > > > > > suite >> > > >> > > >> >> > > > > > > > > > > against >> > > >> > > >> >> > > > > > > > > > > >>> > this >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can confirm >> > > things >> > > >> > look >> > > >> > > >> good >> > > >> > > >> >> and >> > > >> > > >> >> > > > start >> > > >> > > >> >> > > > > > the >> > > >> > > >> >> > > > > > > > > > release >> > > >> > > >> >> > > > > > > > > > > >>> > > process? >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks, >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > [1] >> > > >> > > >> >> > > > > >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > -- >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 >> > AM, >> > > >> > Jacques >> > > >> > > >> >> Nadeau >> > > >> > > >> >> > < >> > > >> > > >> >> > > > > > > > > > > [email protected] >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote: >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main things are >> > 4196 >> > > >> and >> > > >> > > 4291 >> > > >> > > >> >> > should >> > > >> > > >> >> > > be >> > > >> > > >> >> > > > > > > > > completed. >> > > >> > > >> >> > > > > > > > > > I >> > > >> > > >> >> > > > > > > > > > > >>> know >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce 4196 >> > > locally >> > > >> > this >> > > >> > > >> >> weekend >> > > >> > > >> >> > > so >> > > >> > > >> >> > > > I >> > > >> > > >> >> > > > > > > think >> > > >> > > >> >> > > > > > > > > > we're >> > > >> > > >> >> > > > > > > > > > > >>> close >> > > >> > > >> >> > > > > > > > > > > >>> > on >> > > >> > > >> >> > > > > > > > > > > >>> > > > that. >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > > -- >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at >> 10:14 >> > > AM, >> > > >> > > Zelaine >> > > >> > > >> >> Fong >> > > >> > > >> >> > < >> > > >> > > >> >> > > > > > > > > > > [email protected] >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote: >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this? >> What's >> > > >> > blocking >> > > >> > > us >> > > >> > > >> >> from >> > > >> > > >> >> > > > taking >> > > >> > > >> >> > > > > > > this >> > > >> > > >> >> > > > > > > > > to a >> > > >> > > >> >> > > > > > > > > > > >>> vote -- >> > > >> > > >> >> > > > > > > > > > > >>> > > the >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues? >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at >> 4:18 >> > > PM, >> > > >> > > Jacques >> > > >> > > >> >> > Nadeau >> > > >> > > >> >> > > < >> > > >> > > >> >> > > > > > > > > > > >>> [email protected]> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote: >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread... >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues that >> > were >> > > >> > > >> mentioned in >> > > >> > > >> >> > > this >> > > >> > > >> >> > > > > > thread >> > > >> > > >> >> > > > > > > > > along >> > > >> > > >> >> > > > > > > > > > > >>> with a >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed categorization: >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit >> > > >> > > >> >> > > > > > > > > > > >>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit >> > > >> > > >> >> > > > > > > > > > > >>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng >> > > >> > > >> >> > > > > > > > > > > >>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim >> > > >> > > >> >> > > > > > > > > > > >>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason >> > > >> > > >> >> > > > > > > > > > > >>> >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques >> > > >> > > >> >> > > > > > > > > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4266 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others think? >> Let's >> > > try >> > > >> to >> > > >> > > get >> > > >> > > >> the >> > > >> > > >> >> > > > > blockers >> > > >> > > >> >> > > > > > > > > wrapped >> > > >> > > >> >> > > > > > > > > > up >> > > >> > > >> >> > > > > > > > > > > >>> in >> > > >> > > >> >> > > > > > > > > > > >>> > the >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a >> > > release >> > > >> > > vote... >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > -- >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, >> Dremio >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at >> 1:48 >> > > PM, >> > > >> > > Jason >> > > >> > > >> >> > > Altekruse >> > > >> > > >> >> > > > < >> > > >> > > >> >> > > > > > > > > > > >>> > > >> [email protected]> >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote: >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All, >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator >> > changes >> > > >> > merged >> > > >> > > and >> > > >> > > >> >> > about >> > > >> > > >> >> > > a >> > > >> > > >> >> > > > > > month >> > > >> > > >> >> > > > > > > > > since >> > > >> > > >> >> > > > > > > > > > > the >> > > >> > > >> >> > > > > > > > > > > >>> last >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be >> good >> > to >> > > >> > start >> > > >> > > a >> > > >> > > >> vote >> > > >> > > >> >> > > > soon. I >> > > >> > > >> >> > > > > > > would >> > > >> > > >> >> > > > > > > > > > like >> > > >> > > >> >> > > > > > > > > > > to >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager. >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there were >> > some >> > > >> > issues >> > > >> > > >> that >> > > >> > > >> >> > were >> > > >> > > >> >> > > > > > > > identified >> > > >> > > >> >> > > > > > > > > > > after >> > > >> > > >> >> > > > > > > > > > > >>> the >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I >> think >> > > that >> > > >> > > these >> > > >> > > >> >> > issues >> > > >> > > >> >> > > > > should >> > > >> > > >> >> > > > > > > be >> > > >> > > >> >> > > > > > > > > > fixed >> > > >> > > >> >> > > > > > > > > > > >>> before >> > > >> > > >> >> > > > > > > > > > > >>> > > we >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate. >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the >> > > associated >> > > >> > > JIRAs >> > > >> > > >> it >> > > >> > > >> >> > > looked >> > > >> > > >> >> > > > > > like >> > > >> > > >> >> > > > > > > > > there >> > > >> > > >> >> > > > > > > > > > > was >> > > >> > > >> >> > > > > > > > > > > >>> a >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just >> > > adjusting >> > > >> > the >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node >> > > >> > > >> >> > > > > > > > > > > >>> > option, >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved work >> > to >> > > >> > change >> > > >> > > >> how we >> > > >> > > >> >> > > > > determine >> > > >> > > >> >> > > > > > > the >> > > >> > > >> >> > > > > > > > > > > correct >> > > >> > > >> >> > > > > > > > > > > >>> > time >> > > >> > > >> >> > > > > > > > > > > >>> > > to >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during external >> > > sort. I >> > > >> > > >> believe it >> > > >> > > >> >> > > makes >> > > >> > > >> >> > > > > > sense >> > > >> > > >> >> > > > > > > > to >> > > >> > > >> >> > > > > > > > > > make >> > > >> > > >> >> > > > > > > > > > > >>> > > external >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the >> newly >> > > >> > improved >> > > >> > > >> memory >> > > >> > > >> >> > > > > > accounting >> > > >> > > >> >> > > > > > > > > before >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not >> sure >> > > how >> > > >> > much >> > > >> > > >> work >> > > >> > > >> >> is >> > > >> > > >> >> > > > left >> > > >> > > >> >> > > > > to >> > > >> > > >> >> > > > > > > be >> > > >> > > >> >> > > > > > > > > done >> > > >> > > >> >> > > > > > > > > > > >>> there. >> > > >> > > >> >> > > > > > > > > > > >>> > > [1] >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with >> your >> > > >> > thoughts >> > > >> > > on >> > > >> > > >> a >> > > >> > > >> >> > > release >> > > >> > > >> >> > > > > > soon >> > > >> > > >> >> > > > > > > > and >> > > >> > > >> >> > > > > > > > > > any >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs >> > > >> > > >> >> > > > > > > > > > > >>> > > you >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in the >> > > >> release. >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] - >> > > >> > > >> >> > > > > > > > >> > > https://issues.apache.org/jira/browse/DRILL-4243 >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks, >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> >> > > >> > > >> >> > > > > > > > > > > >> >> > > >> > > >> >> > > > > > > > > > > >> >> > > >> > > >> >> > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > >> > > >> > > >> >> > > >> > > >> > > >> >> > >> > > >> > > >> >> >> > > >> > > >> >> > > >> > > >> > > >> > >> > > >> >> > > >> > >>
