Jinfeng, do you agree that the changes are risky? I was going mostly by commit messages.
Also just realized that the backwards compatibility fix to the storage plugin interface is after these. Could create release branch from 03197d... and then cherry pick the Venki change. Or could keep them all. Or go without the compatibility fix.... -- Jacques Nadeau CTO and Co-Founder, Dremio On Sun, Jan 31, 2016 at 7:50 PM, Jinfeng Ni <[email protected]> wrote: > Hi Jason, > > I'm fine that release 1.5.0 starts from 03197d0f2. One thing I need > point out is that the pre-commit test suite has some testcase's plan > output modified because of the partition pruning change. If we run > pre-commit against the candidate release for 1.5.0, we have to use a > prior version of pre-commit suite. > > > > On Sun, Jan 31, 2016 at 5:40 PM, Jason Altekruse > <[email protected]> wrote: > > That sounds reasonable to me. I'll start preparing a release candidate. > > > > Jinfeng, are you okay waiting for the next release to include this > change? > > > > On Sun, Jan 31, 2016 at 4:49 PM, Jacques Nadeau <[email protected]> > wrote: > > > >> Hey Jason, > >> > >> Can you start the release vote? We've let this drag on too long. > >> > >> I suggest we start from 03197d0f2. It seems like the partition pruning > >> changes are pretty complicated and on a pretty main path to add right > >> before the vote. Since Hsuan's fix (pr 349) hasn't been reviewed, I > think > >> we should start the vote without it. > >> > >> Jacques > >> > >> > >> -- > >> Jacques Nadeau > >> CTO and Co-Founder, Dremio > >> > >> On Fri, Jan 29, 2016 at 4:39 PM, Hsuan Yi Chu <[email protected]> > wrote: > >> > >> > Yes, just send a pull request. > >> > https://github.com/apache/drill/pull/349 > >> > > >> > Can Jinfeng review it? > >> > > >> > Thanks, > >> > > >> > On Fri, Jan 29, 2016 at 1:53 PM, Jinfeng Ni <[email protected]> > >> wrote: > >> > > >> > > Sean probably is quite close to find a fix for this issue. Let's see > >> > > whether he can post a patch soon. > >> > > > >> > > > >> > > On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli > >> > > <[email protected]> wrote: > >> > > > DRILL-4323 only happens when assertions are turned on (my install > >> > script > >> > > > automatically enables assertions). > >> > > > > >> > > > - Rahul > >> > > > > >> > > > On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong < > [email protected]> > >> > > wrote: > >> > > > > >> > > >> Rahul, > >> > > >> > >> > > >> Let us know if you can reproduce the issue with assertions turned > >> OFF. > >> > > If > >> > > >> you cannot, then I think this should not be a blocker for 1.5, > and > >> > Hsuan > >> > > >> will continue to look into fixing this. > >> > > >> > >> > > >> -- Zelaine > >> > > >> > >> > > >> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen < > >> > [email protected]> > >> > > >> wrote: > >> > > >> > >> > > >> > I do see the same error with assertion turned on. > >> > > >> > > >> > > >> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni < > >> [email protected]> > >> > > >> wrote: > >> > > >> > > >> > > >> > > Krystal, by any chance, did you turn on assertion when you > start > >> > > >> > drillbit? > >> > > >> > > > >> > > >> > > I think the error in drill-4323 will be raised when > assertion = > >> > on. > >> > > >> > > > >> > > >> > > > >> > > >> > > > >> > > >> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen < > >> > > [email protected]> > >> > > >> > > wrote: > >> > > >> > > > I am running drill-1.4 and is able to run count(*) > >> successfully. > >> > > >> > > > > >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter > session > >> > set > >> > > >> > > > `store.hive.optimize_scan_with_native_readers` = true; > >> > > >> > > > > >> > +-------+--------------------------------------------------------+ > >> > > >> > > > | ok | summary > >> > | > >> > > >> > > > > >> > +-------+--------------------------------------------------------+ > >> > > >> > > > | true | store.hive.optimize_scan_with_native_readers > >> > updated. | > >> > > >> > > > > >> > +-------+--------------------------------------------------------+ > >> > > >> > > > 1 row selected (0.374 seconds) > >> > > >> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select > >> count(*) > >> > > from > >> > > >> > > > lineitem_parquet_partitioned_hive_string; > >> > > >> > > > +---------+ > >> > > >> > > > | EXPR$0 | > >> > > >> > > > +---------+ > >> > > >> > > > | 60175 | > >> > > >> > > > +---------+ > >> > > >> > > > 1 row selected (2.986 seconds) > >> > > >> > > > > >> > > >> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni < > >> > > [email protected]> > >> > > >> > > wrote: > >> > > >> > > > > >> > > >> > > >> Venki and I did some investigation for DRILL-4323. The > issue > >> > > >> reported > >> > > >> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well. > Seems > >> > to > >> > > us > >> > > >> > > >> this is not a regression from 1.4.0; it's a regression > from > >> > 1.3.0 > >> > > >> > > >> probably. > >> > > >> > > >> > >> > > >> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader > in > >> > > stead > >> > > >> of > >> > > >> > > >> HiveReader for "select count(*) from hive_table" query. > >> > However, > >> > > >> the > >> > > >> > > >> Project after the scan produces empty schema. Before > >> > DRILL-4083, > >> > > >> > > >> Drill uses HiveScan, which works fine. > >> > > >> > > >> > >> > > >> > > >> > >> > > >> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse > >> > > >> > > >> <[email protected]> wrote: > >> > > >> > > >> > We could revert the change to the StoragePlugin > interface > >> to > >> > > add > >> > > >> the > >> > > >> > > >> > getLogicalOptimizerRules > >> > > >> > > >> > and getPhysicalOptimizerRules, and only put those in > >> > > >> > > >> AbstractStoragePlugin > >> > > >> > > >> > with default implementations. > >> > > >> > > >> > > >> > > >> > > >> > We could then update the docs to tell users to not > >> implement > >> > > the > >> > > >> > > >> interface > >> > > >> > > >> > directly, and instead extend the abstract class, which I > >> > > believe > >> > > >> > would > >> > > >> > > >> > allow us to make changes like this in the future without > >> > > breaking > >> > > >> > > >> plugins. > >> > > >> > > >> > At a major release version we could then just pull up > all > >> of > >> > > the > >> > > >> new > >> > > >> > > >> > methods we added to the abstract class into the > interface, > >> or > >> > > just > >> > > >> > get > >> > > >> > > >> rid > >> > > >> > > >> > of it all-together. > >> > > >> > > >> > > >> > > >> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti < > >> > > >> > > >> [email protected]> > >> > > >> > > >> > wrote: > >> > > >> > > >> > > >> > > >> > > >> >> Here is the partial fix: > >> > > >> > > >> >> > >> > > >> > > >> >> > >> > > >> > > >> > >> > > >> > > > >> > > >> > > >> > > >> > >> > > > >> > > >> > https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b > >> > > >> > > >> >> > >> > > >> > > >> >> If the existing StragePlugins are implementing > >> > > >> > AbstractStoragePlugin > >> > > >> > > >> class > >> > > >> > > >> >> this fix should resolve the backward compat issue. If > they > >> > are > >> > > >> > > >> implementing > >> > > >> > > >> >> StoragePlugin interface directly then it won't resolve > the > >> > > issue. > >> > > >> > Not > >> > > >> > > >> sure > >> > > >> > > >> >> how to handle it in that case. Let me know if there any > >> ways > >> > > to > >> > > >> > > resolve > >> > > >> > > >> >> this issue completely. > >> > > >> > > >> >> > >> > > >> > > >> >> Thanks > >> > > >> > > >> >> Venki > >> > > >> > > >> >> > >> > > >> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha < > >> > > [email protected] > >> > > >> > > >> > > >> > > >> wrote: > >> > > >> > > >> >> > >> > > >> > > >> >> > I was able to do the build with maven 3.3.9 (after > doing > >> > > 'brew > >> > > >> > > install > >> > > >> > > >> >> > maven'). Here are my env variables related to maven: > >> > > >> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec > >> > > >> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin > >> > > >> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m > >> > > >> > > >> >> > > >> > > >> > > >> >> > > >> > > >> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau < > >> > > >> > > [email protected]> > >> > > >> > > >> >> > wrote: > >> > > >> > > >> >> > > >> > > >> > > >> >> > > Aman, for reference on the Maven version issue. You > >> can > >> > > see > >> > > >> the > >> > > >> > > >> output > >> > > >> > > >> >> of > >> > > >> > > >> >> > > the shade plugin between the two Apache Builds. If > you > >> > > search > >> > > >> > for > >> > > >> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven > >> > 3.0.5) > >> > > >> > fails > >> > > >> > > >> >> because > >> > > >> > > >> >> > it > >> > > >> > > >> >> > > includes a bunch of hadoop (and other libraries > that > >> are > >> > > >> listed > >> > > >> > > as > >> > > >> > > >> >> being > >> > > >> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3), > >> those > >> > > >> items > >> > > >> > > are > >> > > >> > > >> >> > included > >> > > >> > > >> >> > > so that the build completes successfully. > >> > > >> > > >> >> > > > >> > > >> > > >> >> > > > >> https://builds.apache.org/job/drill-scm/647/consoleText > >> > > >> > > >> >> > > > >> https://builds.apache.org/job/drill-scm/648/consoleText > >> > > >> > > >> >> > > > >> > > >> > > >> >> > > -- > >> > > >> > > >> >> > > Jacques Nadeau > >> > > >> > > >> >> > > CTO and Co-Founder, Dremio > >> > > >> > > >> >> > > > >> > > >> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli > < > >> > > >> > > >> >> > > [email protected]> wrote: > >> > > >> > > >> >> > > > >> > > >> > > >> >> > > > Jason, > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > As of now I haven't narrowed down the exact > commit > >> > which > >> > > >> > caused > >> > > >> > > >> >> > > DRILL-4323 > >> > > >> > > >> >> > > > and I do not know the scope of the fix. > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > - Rahul > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse > < > >> > > >> > > >> >> > > [email protected] > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > wrote: > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > > I think it makes sense to wait for Venki's fix > of > >> > the > >> > > >> API, > >> > > >> > as > >> > > >> > > >> >> > > releasing a > >> > > >> > > >> >> > > > > version with the change would make > >> > > reverting/refactoring > >> > > >> > the > >> > > >> > > API > >> > > >> > > >> >> > > breaking > >> > > >> > > >> >> > > > > change less useful. > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > Do you have any idea about the scope of fixing > the > >> > > hive > >> > > >> > > native > >> > > >> > > >> >> > reader, > >> > > >> > > >> >> > > or > >> > > >> > > >> >> > > > > which change since the 1.4 release broke it? Is > >> > > someone > >> > > >> > > >> available > >> > > >> > > >> >> to > >> > > >> > > >> >> > > look > >> > > >> > > >> >> > > > > at it today? > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul > >> challapalli < > >> > > >> > > >> >> > > > > [email protected]> wrote: > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > > Just found an issue with hive native parquet > >> > reader > >> > > >> > > >> (DRILL-4323). > >> > > >> > > >> >> > > This > >> > > >> > > >> >> > > > > is a > >> > > >> > > >> >> > > > > > regression from 1.4 and in my opinion should > be > >> > > treated > >> > > >> > as > >> > > >> > > a > >> > > >> > > >> >> > blocker. > >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > - Rahul > >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki > >> Korukanti < > >> > > >> > > >> >> > > > > > [email protected]> > >> > > >> > > >> >> > > > > > wrote: > >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > > Sorry I didn't realize I was breaking the > >> public > >> > > >> > > interface > >> > > >> > > >> API. > >> > > >> > > >> >> > Let > >> > > >> > > >> >> > > > me > >> > > >> > > >> >> > > > > > work > >> > > >> > > >> >> > > > > > > on a patch to make it backward compatible. > >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > Thanks > >> > > >> > > >> >> > > > > > > Venki > >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven > >> > Phillips > >> > > < > >> > > >> > > >> >> > > [email protected] > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > > > wrote: > >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > > I just wanted to bring up an issue that I > >> just > >> > > now > >> > > >> > > >> >> discovered, > >> > > >> > > >> >> > > that > >> > > >> > > >> >> > > > > has > >> > > >> > > >> >> > > > > > > > caused me a fair amount of grief. > >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > > >> > > https://github.com/apache/drill/pull/300/commits > >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and > >> > causes > >> > > >> > > >> >> StoragePlugins > >> > > >> > > >> >> > > > that > >> > > >> > > >> >> > > > > > were > >> > > >> > > >> >> > > > > > > > compiled against currently released > versions > >> > of > >> > > >> Drill > >> > > >> > > to > >> > > >> > > >> no > >> > > >> > > >> >> > > longer > >> > > >> > > >> >> > > > > > > > functional properly. I would prefer that > >> this > >> > > >> > breaking > >> > > >> > > >> change > >> > > >> > > >> >> > be > >> > > >> > > >> >> > > > > > modified > >> > > >> > > >> >> > > > > > > > to be backward compatible if possible. > >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason > >> > > Altekruse < > >> > > >> > > >> >> > > > > > > > [email protected]> > >> > > >> > > >> >> > > > > > > > wrote: > >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > > Hi Aman, > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > This is the failure that he was > seeing. He > >> > > >> figured > >> > > >> > > out > >> > > >> > > >> that > >> > > >> > > >> >> > the > >> > > >> > > >> >> > > > new > >> > > >> > > >> >> > > > > > > > > exclusions in jdbc-all were not being > >> > > respected > >> > > >> > when > >> > > >> > > the > >> > > >> > > >> >> > build > >> > > >> > > >> >> > > > was > >> > > >> > > >> >> > > > > > run > >> > > >> > > >> >> > > > > > > > with > >> > > >> > > >> >> > > > > > > > > an older Maven version, causing the jar > >> size > >> > > to > >> > > >> > > increase > >> > > >> > > >> >> > > > > > significantly. > >> > > >> > > >> >> > > > > > > > He > >> > > >> > > >> >> > > > > > > > > added an enforcer to make sure the JAR > >> > didn't > >> > > >> grow > >> > > >> > > >> >> > > unexpectedly. > >> > > >> > > >> >> > > > > Can > >> > > >> > > >> >> > > > > > > you > >> > > >> > > >> >> > > > > > > > > try to update your maven version and > >> re-run > >> > > the > >> > > >> > > build? > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > - Jason > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman > >> > Sinha < > >> > > >> > > >> >> > > > [email protected]> > >> > > >> > > >> >> > > > > > > > wrote: > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > Jacques, I am getting the following > >> build > >> > > >> failure > >> > > >> > > on > >> > > >> > > >> the > >> > > >> > > >> >> > > latest > >> > > >> > > >> >> > > > > > > master > >> > > >> > > >> >> > > > > > > > > > branch...is this what you saw for the > >> > Apache > >> > > >> > build > >> > > >> > > ? > >> > > >> > > >> My > >> > > >> > > >> >> > mvn > >> > > >> > > >> >> > > > > > version > >> > > >> > > >> >> > > > > > > > > output > >> > > >> > > >> >> > > > > > > > > > is shown below. Should we all be > >> > upgrading > >> > > to > >> > > >> a > >> > > >> > > newer > >> > > >> > > >> >> mvn > >> > > >> > > >> >> > ? > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > [INFO] --- > >> > > maven-enforcer-plugin:1.3.1:enforce > >> > > >> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @ > >> > > drill-jdbc-all > >> > > >> > --- > >> > > >> > > >> >> > > > > > > > > > [WARNING] Rule 0: > >> > > >> > > >> >> > > > > > > >> org.apache.maven.plugins.enforcer.RequireFilesSize > >> > > >> > > >> >> > > > > > > > > failed > >> > > >> > > >> >> > > > > > > > > > with message: > >> > > >> > > >> >> > > > > > > > > > The file > >> drill-jdbc-all-1.5.0-SNAPSHOT.jar > >> > > is > >> > > >> > > outside > >> > > >> > > >> the > >> > > >> > > >> >> > > > > expected > >> > > >> > > >> >> > > > > > > size > >> > > >> > > >> >> > > > > > > > > > range. > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > This is likely due > to > >> > you > >> > > >> > adding > >> > > >> > > new > >> > > >> > > >> >> > > > > dependencies > >> > > >> > > >> >> > > > > > > to > >> > > >> > > >> >> > > > > > > > a > >> > > >> > > >> >> > > > > > > > > > java-exec and not updating the > excludes > >> in > >> > > this > >> > > >> > > >> module. > >> > > >> > > >> >> > This > >> > > >> > > >> >> > > is > >> > > >> > > >> >> > > > > > > > important > >> > > >> > > >> >> > > > > > > > > > as it minimizes the size of the > >> dependency > >> > > of > >> > > >> > Drill > >> > > >> > > >> >> > > application > >> > > >> > > >> >> > > > > > > users. > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > >> > > >> > > >> >> > > >> > > >> > > >> >> > >> > > >> > > >> > >> > > >> > > > >> > > >> > > >> > > >> > >> > > > >> > > >> > /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar > >> > > >> > > >> >> > > > > > > > > > size (44664121) too large. Max. is > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > >> > > >> > > >> >> > > >> > > >> > > >> >> > >> > > >> > > >> > >> > > >> > > > >> > > >> > > >> > > >> > >> > > > >> > > >> > 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> > > Administrators-MacBook-Pro-144:incubator-drill > >> > > >> > > asinha$ > >> > > >> > > >> >> mvn > >> > > >> > > >> >> > > > > > --version > >> > > >> > > >> >> > > > > > > > > > Apache Maven 3.0.5 > >> > > >> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da; > >> > > >> > > >> >> > > > > > > > 2013-02-19 > >> > > >> > > >> >> > > > > > > > > > 05:51:28-0800) > >> > > >> > > >> >> > > > > > > > > > Maven home: > /opt/local/share/java/maven3 > >> > > >> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: > Oracle > >> > > >> > Corporation > >> > > >> > > >> >> > > > > > > > > > Java home: > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > >> > > >> > > >> > >> > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre > >> > > >> > > >> >> > > > > > > > > > Default locale: en_US, platform > >> encoding: > >> > > UTF-8 > >> > > >> > > >> >> > > > > > > > > > OS name: "mac os x", version: > "10.9.5", > >> > > arch: > >> > > >> > > >> "x86_64", > >> > > >> > > >> >> > > family: > >> > > >> > > >> >> > > > > > "mac" > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, > Jacques > >> > > >> Nadeau < > >> > > >> > > >> >> > > > > > [email protected]> > >> > > >> > > >> >> > > > > > > > > > wrote: > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > Build back to normal. It looks like > >> the > >> > > >> Apache > >> > > >> > > >> server > >> > > >> > > >> >> was > >> > > >> > > >> >> > > > using > >> > > >> > > >> >> > > > > > an > >> > > >> > > >> >> > > > > > > > old > >> > > >> > > >> >> > > > > > > > > > > version of Maven. Once I switched > to > >> > > >> something > >> > > >> > > more > >> > > >> > > >> >> > recent, > >> > > >> > > >> >> > > > the > >> > > >> > > >> >> > > > > > > build > >> > > >> > > >> >> > > > > > > > > > > passed. > >> > > >> > > >> >> > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > -- > >> > > >> > > >> >> > > > > > > > > > > Jacques Nadeau > >> > > >> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio > >> > > >> > > >> >> > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, > >> Jacques > >> > > >> > Nadeau < > >> > > >> > > >> >> > > > > > > [email protected]> > >> > > >> > > >> >> > > > > > > > > > > wrote: > >> > > >> > > >> >> > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > > Hmm... this merge caused the > Apache > >> > > build > >> > > >> to > >> > > >> > > fail. > >> > > >> > > >> >> > > > > > > Investigating... > >> > > >> > > >> >> > > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > > -- > >> > > >> > > >> >> > > > > > > > > > > > Jacques Nadeau > >> > > >> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio > >> > > >> > > >> >> > > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, > >> > Jacques > >> > > >> > > Nadeau < > >> > > >> > > >> >> > > > > > > > [email protected]> > >> > > >> > > >> >> > > > > > > > > > > > wrote: > >> > > >> > > >> >> > > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> I got clean regression runs as > >> well. > >> > > I've > >> > > >> > > merged > >> > > >> > > >> the > >> > > >> > > >> >> > > > patch. > >> > > >> > > >> >> > > > > > > > > > > >> > >> > > >> > > >> >> > > > > > > > > > > >> Jason, you want to start the > >> release > >> > > >> > process? > >> > > >> > > >> >> > > > > > > > > > > >> > >> > > >> > > >> >> > > > > > > > > > > >> -- > >> > > >> > > >> >> > > > > > > > > > > >> Jacques Nadeau > >> > > >> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio > >> > > >> > > >> >> > > > > > > > > > > >> > >> > > >> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 > PM, > >> > > Abhishek > >> > > >> > > >> Girish < > >> > > >> > > >> >> > > > > > > > [email protected] > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> wrote: > >> > > >> > > >> >> > > > > > > > > > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs. > >> TPC-H > >> > > >> SF100 > >> > > >> > > was > >> > > >> > > >> also > >> > > >> > > >> >> > > > > > successful. > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 > PM, > >> > > rahul > >> > > >> > > >> >> challapalli < > >> > > >> > > >> >> > > > > > > > > > > >>> [email protected]> > >> wrote: > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run > with > >> > > your > >> > > >> > > branch. > >> > > >> > > >> >> Will > >> > > >> > > >> >> > > let > >> > > >> > > >> >> > > > > you > >> > > >> > > >> >> > > > > > > know > >> > > >> > > >> >> > > > > > > > > > once > >> > > >> > > >> >> > > > > > > > > > > it > >> > > >> > > >> >> > > > > > > > > > > >>> > finishes > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > - Rahul > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 > PM, > >> > > >> Jacques > >> > > >> > > >> Nadeau < > >> > > >> > > >> >> > > > > > > > > > [email protected]> > >> > > >> > > >> >> > > > > > > > > > > >>> > wrote: > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I > have > >> an > >> > > >> > updated > >> > > >> > > >> patch > >> > > >> > > >> >> > for > >> > > >> > > >> >> > > > 4291 > >> > > >> > > >> >> > > > > > > that > >> > > >> > > >> >> > > > > > > > is > >> > > >> > > >> >> > > > > > > > > > > >>> ready. > >> > > >> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems > that > >> > > >> something > >> > > >> > > isn't > >> > > >> > > >> >> > > working > >> > > >> > > >> >> > > > > with > >> > > >> > > >> >> > > > > > > our > >> > > >> > > >> >> > > > > > > > > > > >>> extended > >> > > >> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been > able > >> to > >> > > run > >> > > >> an > >> > > >> > > >> extended > >> > > >> > > >> >> > > > > > regression. > >> > > >> > > >> >> > > > > > > > > Unit > >> > > >> > > >> >> > > > > > > > > > > >>> tests > >> > > >> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else > possibly > >> > > able > >> > > >> to > >> > > >> > > run a > >> > > >> > > >> >> > > > regression > >> > > >> > > >> >> > > > > > > suite > >> > > >> > > >> >> > > > > > > > > > > against > >> > > >> > > >> >> > > > > > > > > > > >>> > this > >> > > >> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can > confirm > >> > > things > >> > > >> > look > >> > > >> > > >> good > >> > > >> > > >> >> and > >> > > >> > > >> >> > > > start > >> > > >> > > >> >> > > > > > the > >> > > >> > > >> >> > > > > > > > > > release > >> > > >> > > >> >> > > > > > > > > > > >>> > > process? > >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > thanks, > >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques > >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > [1] > >> > > >> > > >> >> > > > > > >> > https://github.com/jacques-n/drill/tree/DRILL-4291v2 > >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > -- > >> > > >> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau > >> > > >> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio > >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at > 11:20 > >> > AM, > >> > > >> > Jacques > >> > > >> > > >> >> Nadeau > >> > > >> > > >> >> > < > >> > > >> > > >> >> > > > > > > > > > > [email protected] > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote: > >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > > I think the main things > are > >> > 4196 > >> > > >> and > >> > > >> > > 4291 > >> > > >> > > >> >> > should > >> > > >> > > >> >> > > be > >> > > >> > > >> >> > > > > > > > > completed. > >> > > >> > > >> >> > > > > > > > > > I > >> > > >> > > >> >> > > > > > > > > > > >>> know > >> > > >> > > >> >> > > > > > > > > > > >>> > > Amit > >> > > >> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce > 4196 > >> > > locally > >> > > >> > this > >> > > >> > > >> >> weekend > >> > > >> > > >> >> > > so > >> > > >> > > >> >> > > > I > >> > > >> > > >> >> > > > > > > think > >> > > >> > > >> >> > > > > > > > > > we're > >> > > >> > > >> >> > > > > > > > > > > >>> close > >> > > >> > > >> >> > > > > > > > > > > >>> > on > >> > > >> > > >> >> > > > > > > > > > > >>> > > > that. > >> > > >> > > >> >> > > > > > > > > > > >>> > > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > > -- > >> > > >> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau > >> > > >> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, > Dremio > >> > > >> > > >> >> > > > > > > > > > > >>> > > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at > >> 10:14 > >> > > AM, > >> > > >> > > Zelaine > >> > > >> > > >> >> Fong > >> > > >> > > >> >> > < > >> > > >> > > >> >> > > > > > > > > > > [email protected] > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > wrote: > >> > > >> > > >> >> > > > > > > > > > > >>> > > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this? > >> What's > >> > > >> > blocking > >> > > >> > > us > >> > > >> > > >> >> from > >> > > >> > > >> >> > > > taking > >> > > >> > > >> >> > > > > > > this > >> > > >> > > >> >> > > > > > > > > to a > >> > > >> > > >> >> > > > > > > > > > > >>> vote -- > >> > > >> > > >> >> > > > > > > > > > > >>> > > the > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues? > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at > >> 4:18 > >> > > PM, > >> > > >> > > Jacques > >> > > >> > > >> >> > Nadeau > >> > > >> > > >> >> > > < > >> > > >> > > >> >> > > > > > > > > > > >>> [email protected]> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> wrote: > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread... > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues > that > >> > were > >> > > >> > > >> mentioned in > >> > > >> > > >> >> > > this > >> > > >> > > >> >> > > > > > thread > >> > > >> > > >> >> > > > > > > > > along > >> > > >> > > >> >> > > > > > > > > > > >>> with a > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > proposed > categorization: > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4190 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4196 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques > >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4256 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4278 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent > >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4247 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4203 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques > >> > > >> > > >> >> > > > > > > > > > > >> > > >> https://issues.apache.org/jira/browse/DRILL-4266 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques > >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > What do others think? > >> Let's > >> > > try > >> > > >> to > >> > > >> > > get > >> > > >> > > >> the > >> > > >> > > >> >> > > > > blockers > >> > > >> > > >> >> > > > > > > > > wrapped > >> > > >> > > >> >> > > > > > > > > > up > >> > > >> > > >> >> > > > > > > > > > > >>> in > >> > > >> > > >> >> > > > > > > > > > > >>> > the > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> next > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a > >> > > release > >> > > >> > > vote... > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > -- > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, > >> Dremio > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at > >> 1:48 > >> > > PM, > >> > > >> > > Jason > >> > > >> > > >> >> > > Altekruse > >> > > >> > > >> >> > > > < > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > [email protected]> > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > wrote: > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All, > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator > >> > changes > >> > > >> > merged > >> > > >> > > and > >> > > >> > > >> >> > about > >> > > >> > > >> >> > > a > >> > > >> > > >> >> > > > > > month > >> > > >> > > >> >> > > > > > > > > since > >> > > >> > > >> >> > > > > > > > > > > the > >> > > >> > > >> >> > > > > > > > > > > >>> last > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > release > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be > >> good > >> > to > >> > > >> > start > >> > > >> > > a > >> > > >> > > >> vote > >> > > >> > > >> >> > > > soon. I > >> > > >> > > >> >> > > > > > > would > >> > > >> > > >> >> > > > > > > > > > like > >> > > >> > > >> >> > > > > > > > > > > to > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> volunteer > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > to > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager. > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there > were > >> > some > >> > > >> > issues > >> > > >> > > >> that > >> > > >> > > >> >> > were > >> > > >> > > >> >> > > > > > > > identified > >> > > >> > > >> >> > > > > > > > > > > after > >> > > >> > > >> >> > > > > > > > > > > >>> the > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > transfer > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I > >> think > >> > > that > >> > > >> > > these > >> > > >> > > >> >> > issues > >> > > >> > > >> >> > > > > should > >> > > >> > > >> >> > > > > > > be > >> > > >> > > >> >> > > > > > > > > > fixed > >> > > >> > > >> >> > > > > > > > > > > >>> before > >> > > >> > > >> >> > > > > > > > > > > >>> > > we > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > cut a > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate. > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the > >> > > associated > >> > > >> > > JIRAs > >> > > >> > > >> it > >> > > >> > > >> >> > > looked > >> > > >> > > >> >> > > > > > like > >> > > >> > > >> >> > > > > > > > > there > >> > > >> > > >> >> > > > > > > > > > > was > >> > > >> > > >> >> > > > > > > > > > > >>> a > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> possible > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just > >> > > adjusting > >> > > >> > the > >> > > >> > > >> >> > > > > > > > > > max_query_memory_per_node > >> > > >> > > >> >> > > > > > > > > > > >>> > option, > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> and > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved > work > >> > to > >> > > >> > change > >> > > >> > > >> how we > >> > > >> > > >> >> > > > > determine > >> > > >> > > >> >> > > > > > > the > >> > > >> > > >> >> > > > > > > > > > > correct > >> > > >> > > >> >> > > > > > > > > > > >>> > time > >> > > >> > > >> >> > > > > > > > > > > >>> > > to > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > spill during > external > >> > > sort. I > >> > > >> > > >> believe it > >> > > >> > > >> >> > > makes > >> > > >> > > >> >> > > > > > sense > >> > > >> > > >> >> > > > > > > > to > >> > > >> > > >> >> > > > > > > > > > make > >> > > >> > > >> >> > > > > > > > > > > >>> > > external > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > sort > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the > >> newly > >> > > >> > improved > >> > > >> > > >> memory > >> > > >> > > >> >> > > > > > accounting > >> > > >> > > >> >> > > > > > > > > before > >> > > >> > > >> >> > > > > > > > > > > >>> > cutting a > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not > >> sure > >> > > how > >> > > >> > much > >> > > >> > > >> work > >> > > >> > > >> >> is > >> > > >> > > >> >> > > > left > >> > > >> > > >> >> > > > > to > >> > > >> > > >> >> > > > > > > be > >> > > >> > > >> >> > > > > > > > > done > >> > > >> > > >> >> > > > > > > > > > > >>> there. > >> > > >> > > >> >> > > > > > > > > > > >>> > > [1] > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with > >> your > >> > > >> > thoughts > >> > > >> > > on > >> > > >> > > >> a > >> > > >> > > >> >> > > release > >> > > >> > > >> >> > > > > > soon > >> > > >> > > >> >> > > > > > > > and > >> > > >> > > >> >> > > > > > > > > > any > >> > > >> > > >> >> > > > > > > > > > > >>> JIRAs > >> > > >> > > >> >> > > > > > > > > > > >>> > > you > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > would > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in > the > >> > > >> release. > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > [1] - > >> > > >> > > >> >> > > > > > > > > >> > > https://issues.apache.org/jira/browse/DRILL-4243 > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks, > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > Jason > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > >> > > >> > > >> >> > > > > > > > > > > >>> > > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > > >> > > >> > > >> >> > > > > > > > > > > >>> > > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >> > >> > > >> > > >> >> > > > > > > > > > > >> > >> > > >> > > >> >> > > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > > >> > > >> > > >> >> > > > > > > > > > >> > > >> > > >> >> > > > > > > > > >> > > >> > > >> >> > > > > > > > >> > > >> > > >> >> > > > > > > >> > > >> > > >> >> > > > > > >> > > >> > > >> >> > > > > >> > > >> > > >> >> > > > >> > > >> > > >> >> > > >> > > >> > > >> >> > >> > > >> > > >> > >> > > >> > > > >> > > >> > > >> > > >> > >> > > > >> > > >> > >
