Re: Minimise query plan time for dfs plugin for local file system on tsv file

I did not get a chance to review the log file. However the next thing I would try is to make your cluster a single node cluster first and then run the same explain plan query separately on each individual file. On Mar 7, 2017 5:09 AM, "PROJJWAL SAHA" wrote: > Hi Rahul, > > thanks for your sug

Re: [HANGOUT] Topics for 03/07/2017

Hangout starting now. Please join the hangout at: https://hangouts.google.com/hangouts/_/a37hhmg6ejge3ey5sp5eu6scnue Gautam From: Gautam Parai Sent: Monday, March 6, 2017 9:41:07 AM To: user@drill.apache.org; d...@drill.apache.org Subject: [HANGOUT] Topics fo

Re: Configuring Drill Memory Usage under Windows

I've not tried running Drill embedded on Windows, but you can try checking the parameters passed to the running Drill JVM to validate that the settings were picked up ? Kunal Khatua Engineering [MapR] www.mapr.com

Re: [HANGOUT] Topics for 03/07/2017

Hi all, Here are the notes for today's meeting: Attendees: Aman, Arina, Gautam, Karthikeyan, Laurent, Paul, Serhii, Vitalii Arina - Arina is working on a JIRA to fix wrong results when Left outer join is done using NLJ. Arina 'Why did Drill did not do non-equi joins to begin with?' to

Re: Configuring Drill Memory Usage under Windows

I've not tried running Drill embedded on Windows, but you can try checking the parameters passed to the running Drill JVM to validate that the settings were picked up ? Kunal Khatua Engineering [MapR] www.mapr.com

Re: Minimise query plan time for dfs plugin for local file system on tsv file

Looking at the 1st two lines of the log shows that the bulk of time was lost before the query even went into the real planning stage of the query: 2017-03-07 06:27:28,074 [274166de-f543-3fa7-ef9e-8e9e87d5d6a0:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id 274166d

Re: Configuring Drill Memory Usage under Windows

Hi, Kunal. Part of my problem is that I'm unsure what JVM parameters control memory allocation. https://drill.apache.org/docs/configuring-drill-memory/ seems to focus on configuring shell environment variables. I can confirm that uncommenting and setting the appropriate exports in drill-env.sh do

Re: Configuring Drill Memory Usage under Windows

Success! I modified sqlline.bat to the following: set DRILL_SHELL_JAVA_OPTS=%DRILL_SHELL_JAVA_OPTS% -Xms%DRILL_HEAP% -Xmx%DRILL_HEAP% -XX:MaxDirectMemorySize=%DRILL_MAX_DIRECT_MEMORY% -Dlog.path="%DRILL_LOG_DIR%\sqlline.log" -Dlog.query.path="%DRILL_LOG_DIR%\sqlline_queries.log" -DDRILL_HEAP=%DRIL

Re: Minimise query plan time for dfs plugin for local file system on tsv file

Hi Kunal, Good catch ! Thanks for this pointer. I enabled logging at org.apache.drill level and I found - 2017-03-08 01:39:01,822 [274058fa-79df-9c74-3219-3fcb83a04605:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id 274058fa-79df-9c74-3219-3fcb83a04605: explain plan f