1024/4096/2032195 <==> soft/hard/system On Mon, Nov 9, 2015 at 4:50 PM, Jacques Nadeau <jacq...@dremio.com> wrote:
> What about open files? > On Nov 9, 2015 4:20 PM, "Hanifi Gunes" <hgu...@maprtech.com> wrote: > > > Using the default parallelism, I get the same set of exceptions on > CentOS. > > > > Also max # of threads per process is the system default, 319548. > > > > On Mon, Nov 9, 2015 at 4:13 PM, Amit Hadke <amit.ha...@gmail.com> wrote: > > > > > Unfortunately, yes I changed them at OS level. > > > > > > I agree that making OS level changes to run tests is not something we > > > should advocate. > > > It was just an observation that tests seem to pass on linux with > default > > > settings but on mac behavior is irregular. > > > > > > ~ Amit. > > > > > > > > > On Mon, Nov 9, 2015 at 3:48 PM, Jason Altekruse < > > altekruseja...@gmail.com> > > > wrote: > > > > > > > Amit, > > > > > > > > Did you change these at the OS level? I would hope that we wouldn't > > have > > > to > > > > make people jump through such hoops to run the build, was there a > > thread > > > I > > > > missed where we discussed this something that we should be expecting > > > > everyone to do? I did see the threads about increasing the maven > > permgen > > > > and did so accordingly, but I think adjusting something like the > system > > > > ulimit is something we really should try to avoid. > > > > > > > > On Mon, Nov 9, 2015 at 3:44 PM, Abdel Hakim Deneche < > > > adene...@maprtech.com > > > > > > > > > wrote: > > > > > > > > > I just tried with the default forkCount and got similar failures. > I'm > > > > > running the tests on a linux machine with the default maven options > > > > > > > > > > Thanks > > > > > > > > > > On Mon, Nov 9, 2015 at 3:21 PM, Jacques Nadeau <jacq...@dremio.com > > > > > > wrote: > > > > > > > > > > > So you're changing the forkCount from the default? What happens > > when > > > > you > > > > > > run with the default? > > > > > > > > > > > > Also, did you guys run with extra permgen or the default? Also, > can > > > you > > > > > > confirm what type of machine you are running on? > > > > > > > > > > > > Want to figure out why things are different. > > > > > > > > > > > > thanks, > > > > > > Jacques > > > > > > > > > > > > -- > > > > > > Jacques Nadeau > > > > > > CTO and Co-Founder, Dremio > > > > > > > > > > > > On Mon, Nov 9, 2015 at 2:23 PM, Abdel Hakim Deneche < > > > > > adene...@maprtech.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Saw similar failures when running unit tests with forkCount = > 1: > > > > > > > > > > > > > > Tests in error: > > > > > > > > > > > > TestImpersonationQueries.sequenceFileChainedImpersonationWithView » > > > > > > > > UserRemote > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationJoinEachSideReachesMaxUserHops:233->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > > > > > > » Rpc > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationExceedsMaxUserHops:219->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > > > > > > » IllegalState > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.avroChainedImpersonationWithView:280->BaseTestImpersonation.createView:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > > > > > > » IllegalState > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.testDirectImpersonation_HasGroupReadPermissions:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > > > > > > » IllegalState > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.testDirectImpersonation_NoReadPermissions:196->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > > > > > > » IllegalState > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationEqualToMaxUserHops:210->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > > > > > > » IllegalState > > > > > > > > TestCountDownLatchInjection.latchInjected:146 NullPointer > > > > > > > > TestMergingReceiver.testMultipleProvidersMixedSizes:115 » > > > > > > > > IllegalReferenceCount > > > > > > > > Tests run: 1482, Failures: 0, Errors: 9, Skipped: 118 > > > > > > > > > > > > > > > > > > > > > Unfortunately I'm changing my vote to -1 > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 9, 2015 at 2:21 PM, Hanifi Gunes < > > hgu...@maprtech.com> > > > > > > wrote: > > > > > > > > > > > > > > > -1 (binding) > > > > > > > > > > > > > > > > 5/5 consecutive runs I get the following consistently with > > > > forkCount > > > > > > set > > > > > > > to > > > > > > > > 1 on build. > > > > > > > > > > > > > > > > Tests in error: > > > > > > > > > > > > TestImpersonationQueries.sequenceFileChainedImpersonationWithView » > > > > > > > > UserRemote > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationJoinEachSideReachesMaxUserHops:233->BaseTestQuery.updateClient:222->BaseTestQuery. > > > > > > > > updateClient:236->BaseTestQuery.updateClient:213 » > > > IllegalState > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationExceedsMaxUserHops:219->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient: > > > > > > > > 236->BaseTestQuery.updateClient:213 » IllegalState > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.avroChainedImpersonationWithView:280->BaseTestImpersonation.createView:186->BaseTestQuery.updateClient:222- > > > > > > > > > > > > >BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213 > > > > > » > > > > > > > > IllegalState > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.testDirectImpersonation_HasGroupReadPermissions:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient: > > > > > > > > 236->BaseTestQuery.updateClient:213 » IllegalState > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.testDirectImpersonation_NoReadPermissions:196->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236- > > > > > > > > >BaseTestQuery.updateClient:213 » IllegalState > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TestImpersonationQueries.testMultiLevelImpersonationEqualToMaxUserHops:210->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient: > > > > > > > > 236->BaseTestQuery.updateClient:213 » IllegalState > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 9, 2015 at 2:19 PM, Stefán Baxter < > > > > > > ste...@activitystream.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > I'm not a committer and not voting but I want to point out > > that > > > > the > > > > > > > UDFs > > > > > > > > > that we are using and run with 1.1 and 1.2 do not, for some > > > > reason, > > > > > > run > > > > > > > > > with 1.3. > > > > > > > > > > > > > > > > > > I'm inclined to insist that it's something we are doing > wrong > > > but > > > > > if > > > > > > > this > > > > > > > > > counts as a test then it's failing. > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > -Stefan > > > > > > > > > > > > > > > > > > On Mon, Nov 9, 2015 at 10:09 PM, Norris Lee < > > norr...@simba.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 Non-binding. > > > > > > > > > > > > > > > > > > > > Built from source on Linux, tested ODBC against various > > data > > > > > > sources > > > > > > > > > > (Hive, DFS-csv, tsv, parquet, json) > > > > > > > > > > > > > > > > > > > > Norris > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: Abdel Hakim Deneche [mailto:adene...@maprtech.com] > > > > > > > > > > Sent: Monday, November 09, 2015 10:16 AM > > > > > > > > > > To: dev@drill.apache.org > > > > > > > > > > Subject: Re: [VOTE] Release Apache Drill 1.3.0 (rc1) > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > Checked checksum files and gpg signature for all 3 files. > > > Built > > > > > > from > > > > > > > > > > source, deployed on a 4 nodes cluster then run several > > window > > > > > > > function > > > > > > > > > > queries on TPCDS SF100. Looks great. > > > > > > > > > > > > > > > > > > > > On Mon, Nov 9, 2015 at 8:14 AM, Jacques Nadeau < > > > > > jacq...@dremio.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > I've seen another instance of DRILL-4041. Since it is > > > > > exceedingly > > > > > > > > > > > rare, I don't think it is a release stopper. I will > spend > > > > some > > > > > > more > > > > > > > > > > > time reviewing it. > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > Jacques Nadeau > > > > > > > > > > > CTO and Co-Founder, Dremio > > > > > > > > > > > > > > > > > > > > > > On Sun, Nov 8, 2015 at 7:34 PM, Amit Hadke < > > > > > amit.ha...@gmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > +1, non-binding > > > > > > > > > > > > Downloaded tarball, mvn clean install Ran join > > queries,, > > > > was > > > > > > able > > > > > > > > to > > > > > > > > > > > > run queries on json, parquet and sequence files. > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Nov 8, 2015 at 10:01 AM, Abhijit Pol < > > > > > > abhi...@dremio.com > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > resending the vote with details > > > > > > > > > > > > > > > > > > > > > > > > > > +1, non-binding > > > > > > > > > > > > > > > > > > > > > > > > > > ran maven tests, installed on Mac, ran drill in > > > embedded > > > > > > mode, > > > > > > > > > > > > > verified > > > > > > > > > > > > Web > > > > > > > > > > > > > UI for CSV header config changes, ran few queries > on > > > CSV > > > > > > files > > > > > > > > > > > > > with and without header, ran few queries against > > large > > > > CSV > > > > > > > files > > > > > > > > > > > > > with header in > > > > > > > > > > > > HDFS > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Nov 8, 2015 at 9:07 AM, Hsuan Yi Chu < > > > > > > > > hyi...@maprtech.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > mvn clean install on mac and linux > > > > > > > > > > > > > > > > > > > > > > > > > > > > tried a few queries on distributed mode. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Things looked fine > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Nov 8, 2015 at 8:16 AM, Aman Sinha < > > > > > > > > asi...@maprtech.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1. > > > > > > > > > > > > > > > Downloaded the binary tar ball and installed on > > > Mac. > > > > > Ran > > > > > > > > > > > > > > > Drill in > > > > > > > > > > > > > > embedded > > > > > > > > > > > > > > > mode. Tested a few join, aggregation and limit > > > > queries > > > > > > > > > > > > > > > against > > > > > > > > > > > > parquet > > > > > > > > > > > > > > > data with and without metadata cache. Tested > > > > > > cancellation > > > > > > > > for > > > > > > > > > > > couple > > > > > > > > > > > > > of > > > > > > > > > > > > > > > queries. Looked at query profiles on the > WebUI. > > > > > Looks > > > > > > > > good. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Nov 7, 2015 at 11:53 PM, Abhijit Pol > > > > > > > > > > > > > > > <abhi...@dremio.com> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Nov 7, 2015 at 9:41 PM, andrew < > > > > > > > and...@primer.org> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 (non-binding, non-comitter) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Nov 6, 2015, at 10:15 PM, Jacques > > Nadeau < > > > > > > > > > > > > jacq...@dremio.com> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hey Everybody, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to propose a new release > candidate > > > of > > > > > > Apache > > > > > > > > > > > > > > > > > > Drill, > > > > > > > > > > > > > > version > > > > > > > > > > > > > > > > > > 1.3.0. This is the second release > > candidate > > > > > (rc1). > > > > > > > > > > > > > > > > > > This > > > > > > > > > > > > > addresses > > > > > > > > > > > > > > > > some > > > > > > > > > > > > > > > > > > issues identified in the first release > > > > candidate > > > > > > > > > > > > > > > > > > including > > > > > > > > > > > some > > > > > > > > > > > > > > test > > > > > > > > > > > > > > > > > > threading issues, Parquet dictionary read > > > issue > > > > > > with > > > > > > > > > > > > > > > > > > binary > > > > > > > > > > > > > values, > > > > > > > > > > > > > > > > Joda > > > > > > > > > > > > > > > > > > upgrade to avoid a concurrency bug and a > > > couple > > > > > > other > > > > > > > > > > > > > > > > > > small > > > > > > > > > > > > > things. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The tarball artifacts are hosted at [2] > and > > > the > > > > > > maven > > > > > > > > > > > artifacts > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > > hosted > > > > > > > > > > > > > > > > > > at [3]. This release candidate is based > on > > > > commit > > > > > > > > > > > > > > > > > > f17ebd2fbf2bf5ba201a635f5b6ec3615afe2305 > > > > located > > > > > at > > > > > > > > [4]. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The vote will be open for 72 hours ending > > at > > > > 10PM > > > > > > > > > > > > > > > > > > Pacific, > > > > > > > > > > > > > November > > > > > > > > > > > > > > > 9, > > > > > > > > > > > > > > > > > 2015. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ ] +1 > > > > > > > > > > > > > > > > > > [ ] +0 > > > > > > > > > > > > > > > > > > [ ] -1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > thanks, > > > > > > > > > > > > > > > > > > Jacques > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313 > > > > > > > > > > > 820&version=12332946[2] > > > > > > > > > > > > > > > > > > > > > > > > > > http://people.apache.org/~jacques/apache-drill-1.3.0.rc1 > > > > > > > > > > > > > > > > > > / > > > > > > > > > > > > > > > > > > [3] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachedrill-1014 > > > > > > > > > > > / > > > > > > > > > > > > > > > > > > [4] > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/jacques-n/drill/tree/drill-1.3.0-rc1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > Jacques Nadeau > > > > > > > > > > > > > > > > > > CTO and Co-Founder, Dremio > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > Abdelhakim Deneche > > > > > > > > > > > > > > > > > > > > Software Engineer > > > > > > > > > > > > > > > > > > > > <http://www.mapr.com/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Now Available - Free Hadoop On-Demand Training < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Abdelhakim Deneche > > > > > > > > > > > > > > Software Engineer > > > > > > > > > > > > > > <http://www.mapr.com/> > > > > > > > > > > > > > > > > > > > > > Now Available - Free Hadoop On-Demand Training > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Abdelhakim Deneche > > > > > > > > > > Software Engineer > > > > > > > > > > <http://www.mapr.com/> > > > > > > > > > > > > > > > Now Available - Free Hadoop On-Demand Training > > > > > < > > > > > > > > > > > > > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > > > > > > > > > > > > > >
