I was able to do the build with maven 3.3.9 (after doing 'brew install maven'). Here are my env variables related to maven: M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec M2=/usr/local/Cellar/maven/3.3.9/libexec/bin MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <jacq...@dremio.com> wrote: > Aman, for reference on the Maven version issue. You can see the output of > the shade plugin between the two Apache Builds. If you search for > maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails because it > includes a bunch of hadoop (and other libraries that are listed as being > excluded). If you look at build 648 (maven 3.3.3), those items are included > so that the build completes successfully. > > https://builds.apache.org/job/drill-scm/647/consoleText > https://builds.apache.org/job/drill-scm/648/consoleText > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli < > challapallira...@gmail.com> wrote: > > > Jason, > > > > As of now I haven't narrowed down the exact commit which caused > DRILL-4323 > > and I do not know the scope of the fix. > > > > - Rahul > > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse < > altekruseja...@gmail.com > > > > > wrote: > > > > > I think it makes sense to wait for Venki's fix of the API, as > releasing a > > > version with the change would make reverting/refactoring the API > breaking > > > change less useful. > > > > > > Do you have any idea about the scope of fixing the hive native reader, > or > > > which change since the 1.4 release broke it? Is someone available to > look > > > at it today? > > > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli < > > > challapallira...@gmail.com> wrote: > > > > > > > Just found an issue with hive native parquet reader (DRILL-4323). > This > > > is a > > > > regression from 1.4 and in my opinion should be treated as a blocker. > > > > > > > > - Rahul > > > > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti < > > > > venki.koruka...@gmail.com> > > > > wrote: > > > > > > > > > Sorry I didn't realize I was breaking the public interface API. Let > > me > > > > work > > > > > on a patch to make it backward compatible. > > > > > > > > > > Thanks > > > > > Venki > > > > > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips < > ste...@dremio.com > > > > > > > > wrote: > > > > > > > > > > > I just wanted to bring up an issue that I just now discovered, > that > > > has > > > > > > caused me a fair amount of grief. > > > > > > > > > > > > https://github.com/apache/drill/pull/300/commits > > > > > > > > > > > > DRILL-4198 changes a user-facing API, and causes StoragePlugins > > that > > > > were > > > > > > compiled against currently released versions of Drill to no > longer > > > > > > functional properly. I would prefer that this breaking change be > > > > modified > > > > > > to be backward compatible if possible. > > > > > > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse < > > > > > > altekruseja...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hi Aman, > > > > > > > > > > > > > > This is the failure that he was seeing. He figured out that the > > new > > > > > > > exclusions in jdbc-all were not being respected when the build > > was > > > > run > > > > > > with > > > > > > > an older Maven version, causing the jar size to increase > > > > significantly. > > > > > > He > > > > > > > added an enforcer to make sure the JAR didn't grow > unexpectedly. > > > Can > > > > > you > > > > > > > try to update your maven version and re-run the build? > > > > > > > > > > > > > > - Jason > > > > > > > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha < > > asi...@maprtech.com> > > > > > > wrote: > > > > > > > > > > > > > > > Jacques, I am getting the following build failure on the > latest > > > > > master > > > > > > > > branch...is this what you saw for the Apache build ? My mvn > > > > version > > > > > > > output > > > > > > > > is shown below. Should we all be upgrading to a newer mvn ? > > > > > > > > > > > > > > > > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all --- > > > > > > > > [WARNING] Rule 0: > > > > org.apache.maven.plugins.enforcer.RequireFilesSize > > > > > > > failed > > > > > > > > with message: > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is outside the > > > expected > > > > > size > > > > > > > > range. > > > > > > > > > > > > > > > > This is likely due to you adding new > > > dependencies > > > > > to > > > > > > a > > > > > > > > java-exec and not updating the excludes in this module. This > is > > > > > > important > > > > > > > > as it minimizes the size of the dependency of Drill > application > > > > > users. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar > > > > > > > > size (44664121) too large. Max. is > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar > > > > > > > > > > > > > > > > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill asinha$ mvn > > > > --version > > > > > > > > Apache Maven 3.0.5 > (r01de14724cdef164cd33c7c8c2fe155faf9602da; > > > > > > 2013-02-19 > > > > > > > > 05:51:28-0800) > > > > > > > > Maven home: /opt/local/share/java/maven3 > > > > > > > > Java version: 1.7.0_45, vendor: Oracle Corporation > > > > > > > > Java home: > > > > > > > > > > > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre > > > > > > > > Default locale: en_US, platform encoding: UTF-8 > > > > > > > > OS name: "mac os x", version: "10.9.5", arch: "x86_64", > family: > > > > "mac" > > > > > > > > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques Nadeau < > > > > jacq...@dremio.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Build back to normal. It looks like the Apache server was > > using > > > > an > > > > > > old > > > > > > > > > version of Maven. Once I switched to something more recent, > > the > > > > > build > > > > > > > > > passed. > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Jacques Nadeau > > > > > > > > > CTO and Co-Founder, Dremio > > > > > > > > > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques Nadeau < > > > > > jacq...@dremio.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hmm... this merge caused the Apache build to fail. > > > > > Investigating... > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Jacques Nadeau > > > > > > > > > > CTO and Co-Founder, Dremio > > > > > > > > > > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques Nadeau < > > > > > > jacq...@dremio.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > >> I got clean regression runs as well. I've merged the > > patch. > > > > > > > > > >> > > > > > > > > > >> Jason, you want to start the release process? > > > > > > > > > >> > > > > > > > > > >> -- > > > > > > > > > >> Jacques Nadeau > > > > > > > > > >> CTO and Co-Founder, Dremio > > > > > > > > > >> > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek Girish < > > > > > > agir...@mapr.com > > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > >> > > > > > > > > > >>> Had two clean Functional runs. TPC-H SF100 was also > > > > successful. > > > > > > > > > >>> > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul challapalli < > > > > > > > > > >>> challapallira...@gmail.com> wrote: > > > > > > > > > >>> > > > > > > > > > >>> > Kicked off a functional run with your branch. Will > let > > > you > > > > > know > > > > > > > > once > > > > > > > > > it > > > > > > > > > >>> > finishes > > > > > > > > > >>> > > > > > > > > > > >>> > - Rahul > > > > > > > > > >>> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM, Jacques Nadeau < > > > > > > > > jacq...@dremio.com> > > > > > > > > > >>> > wrote: > > > > > > > > > >>> > > > > > > > > > > >>> > > 4196 was merged today. I have an updated patch for > > 4291 > > > > > that > > > > > > is > > > > > > > > > >>> ready. > > > > > > > > > >>> > > Unfortunately, it seems that something isn't > working > > > with > > > > > our > > > > > > > > > >>> extended > > > > > > > > > >>> > > tests so I haven't been able to run an extended > > > > regression. > > > > > > > Unit > > > > > > > > > >>> tests > > > > > > > > > >>> > > pass. Is someone else possibly able to run a > > regression > > > > > suite > > > > > > > > > against > > > > > > > > > >>> > this > > > > > > > > > >>> > > branch [1] so we can confirm things look good and > > start > > > > the > > > > > > > > release > > > > > > > > > >>> > > process? > > > > > > > > > >>> > > > > > > > > > > > >>> > > thanks, > > > > > > > > > >>> > > Jacques > > > > > > > > > >>> > > > > > > > > > > > >>> > > [1] > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2 > > > > > > > > > >>> > > > > > > > > > > > >>> > > -- > > > > > > > > > >>> > > Jacques Nadeau > > > > > > > > > >>> > > CTO and Co-Founder, Dremio > > > > > > > > > >>> > > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM, Jacques Nadeau < > > > > > > > > > jacq...@dremio.com > > > > > > > > > >>> > > > > > > > > > > >>> > > wrote: > > > > > > > > > >>> > > > > > > > > > > > >>> > > > I think the main things are 4196 and 4291 should > be > > > > > > > completed. > > > > > > > > I > > > > > > > > > >>> know > > > > > > > > > >>> > > Amit > > > > > > > > > >>> > > > was able to reproduce 4196 locally this weekend > so > > I > > > > > think > > > > > > > > we're > > > > > > > > > >>> close > > > > > > > > > >>> > on > > > > > > > > > >>> > > > that. > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > -- > > > > > > > > > >>> > > > Jacques Nadeau > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM, Zelaine Fong < > > > > > > > > > zf...@maprtech.com > > > > > > > > > >>> > > > > > > > > > > >>> > > wrote: > > > > > > > > > >>> > > > > > > > > > > > > >>> > > >> Any updates on this? What's blocking us from > > taking > > > > > this > > > > > > > to a > > > > > > > > > >>> vote -- > > > > > > > > > >>> > > the > > > > > > > > > >>> > > >> sort merge join issues? > > > > > > > > > >>> > > >> > > > > > > > > > >>> > > >> -- Zelaine > > > > > > > > > >>> > > >> > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM, Jacques Nadeau > < > > > > > > > > > >>> jacq...@dremio.com> > > > > > > > > > >>> > > >> wrote: > > > > > > > > > >>> > > >> > > > > > > > > > >>> > > >> > Bumping this thread... > > > > > > > > > >>> > > >> > > > > > > > > > > >>> > > >> > Here are the issues that were mentioned in > this > > > > thread > > > > > > > along > > > > > > > > > >>> with a > > > > > > > > > >>> > > >> > proposed categorization: > > > > > > > > > >>> > > >> > > > > > > > > > > >>> > > >> > Release Blockers > > > > > > > > > >>> > > >> > In-progress Amit > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4190 > > > > > > > > > >>> > > >> > In-progress Amit > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4196 > > > > > > > > > >>> > > >> > Ready to merge Jacques > > > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4246 > > > > > > > > > >>> > > >> > In-review Jinfeng > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4256 > > > > > > > > > >>> > > >> > In-progress Jacques > > > > > > > > > >>> > https://issues.apache.org/jira/browse/DRILL-4278 > > > > > > > > > >>> > > >> > Ready to merge Laurent > > > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4285 > > > > > > > > > >>> > > >> > Nice to Have > > > > > > > > > >>> > > >> > Open Jason/Hakim > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4247 > > > > > > > > > >>> > > >> > In-progress Jason > > > > > > > > > >>> https://issues.apache.org/jira/browse/DRILL-4203 > > > > > > > > > >>> > > >> > Open Jacques > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266 > > > > > > > > > >>> > > >> > Ready to merge Jacques > > > > > > > > > >>> > > https://issues.apache.org/jira/browse/DRILL-4131 > > > > > > > > > >>> > > >> > > > > > > > > > > >>> > > >> > What do others think? Let's try to get the > > > blockers > > > > > > > wrapped > > > > > > > > up > > > > > > > > > >>> in > > > > > > > > > >>> > the > > > > > > > > > >>> > > >> next > > > > > > > > > >>> > > >> > day or two and start a release vote... > > > > > > > > > >>> > > >> > > > > > > > > > > >>> > > >> > > > > > > > > > > >>> > > >> > > > > > > > > > > >>> > > >> > -- > > > > > > > > > >>> > > >> > Jacques Nadeau > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio > > > > > > > > > >>> > > >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM, Jason > Altekruse > > < > > > > > > > > > >>> > > >> altekruseja...@gmail.com> > > > > > > > > > >>> > > >> > wrote: > > > > > > > > > >>> > > >> > > > > > > > > > > >>> > > >> > > Hello All, > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > > >> > > With the allocator changes merged and about > a > > > > month > > > > > > > since > > > > > > > > > the > > > > > > > > > >>> last > > > > > > > > > >>> > > >> > release > > > > > > > > > >>> > > >> > > I think it would be good to start a vote > > soon. I > > > > > would > > > > > > > > like > > > > > > > > > to > > > > > > > > > >>> > > >> volunteer > > > > > > > > > >>> > > >> > to > > > > > > > > > >>> > > >> > > be release manager. > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > > >> > > I know that there were some issues that were > > > > > > identified > > > > > > > > > after > > > > > > > > > >>> the > > > > > > > > > >>> > > >> > transfer > > > > > > > > > >>> > > >> > > patch was merged. I think that these issues > > > should > > > > > be > > > > > > > > fixed > > > > > > > > > >>> before > > > > > > > > > >>> > > we > > > > > > > > > >>> > > >> > cut a > > > > > > > > > >>> > > >> > > release candidate. > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > > >> > > From looking at the associated JIRAs it > looked > > > > like > > > > > > > there > > > > > > > > > was > > > > > > > > > >>> a > > > > > > > > > >>> > > >> possible > > > > > > > > > >>> > > >> > > short term fix just adjusting the > > > > > > > > max_query_memory_per_node > > > > > > > > > >>> > option, > > > > > > > > > >>> > > >> and > > > > > > > > > >>> > > >> > > some more involved work to change how we > > > determine > > > > > the > > > > > > > > > correct > > > > > > > > > >>> > time > > > > > > > > > >>> > > to > > > > > > > > > >>> > > >> > > spill during external sort. I believe it > makes > > > > sense > > > > > > to > > > > > > > > make > > > > > > > > > >>> > > external > > > > > > > > > >>> > > >> > sort > > > > > > > > > >>> > > >> > > work well with the newly improved memory > > > > accounting > > > > > > > before > > > > > > > > > >>> > cutting a > > > > > > > > > >>> > > >> > > release, but I'm not sure how much work is > > left > > > to > > > > > be > > > > > > > done > > > > > > > > > >>> there. > > > > > > > > > >>> > > [1] > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > > >> > > Please respond with your thoughts on a > release > > > > soon > > > > > > and > > > > > > > > any > > > > > > > > > >>> JIRAs > > > > > > > > > >>> > > you > > > > > > > > > >>> > > >> > would > > > > > > > > > >>> > > >> > > like to include in the release. > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > > >> > > [1] - > > > > > > https://issues.apache.org/jira/browse/DRILL-4243 > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > > >> > > Thanks, > > > > > > > > > >>> > > >> > > Jason > > > > > > > > > >>> > > >> > > > > > > > > > > > >>> > > >> > > > > > > > > > > >>> > > >> > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
