Success! I modified sqlline.bat to the following: set DRILL_SHELL_JAVA_OPTS=%DRILL_SHELL_JAVA_OPTS% -Xms%DRILL_HEAP% -Xmx%DRILL_HEAP% -XX:MaxDirectMemorySize=%DRILL_MAX_DIRECT_MEMORY% -Dlog.path="%DRILL_LOG_DIR%\sqlline.log" -Dlog.query.path="%DRILL_LOG_DIR%\sqlline_queries.log" -DDRILL_HEAP=%DRILL_HEAP% -DDRILL_MAX_DIRECT_MEMORY=%DRILL_MAX_DIRECT_MEMORY%
This allows me to specify DRILL_HEAP and DRILL_MAX_DIRECT_MEMORY via environment variables similar to drill-env.sh and have them pass through to the JVM. Manually specifying the -Xms -Xmx and -XX:MaxDirectMemorySize options still strikes me as odd. I don't see anywhere these are set in the shell version of sqlline, but as this definitely seems to work ¯\_(ツ)_/¯ David On Tue, Mar 7, 2017 at 12:05 PM, David F. Severski <da...@severski.net> wrote: > Hi, Kunal. > > Part of my problem is that I'm unsure what JVM parameters control memory > allocation. https://drill.apache.org/docs/configuring-drill-memory/ seems > to focus on configuring shell environment variables. I can confirm that > uncommenting and setting the appropriate exports in drill-env.sh do not > result in changes to the environment when running sqlline.bat on Windows. > > By modifying the DRILL_JAVA_OPTS and DRILL_SHELL_JAVA_OPTS in sqlline.bat, > I can get a JVM launch command like this: > > "C:\Program Files\Java\jdk1.8.0_121\bin\java.exe" > -Dlog.path="C:\localbin\apache-drill-1.9.0\log\sqlline.log" > -Dlog.query.path="C:\localbin\apache-drill-1.9.0\log\sqlline_queries.log" > -DDRILL_HEAP=16G -DDRILL_MAX_DIRECT_MEMORY=16G -Dbounds=false -cp > "C:\localbin\apache-drill-1.9.0\conf;C:\localbin\apache- > drill-1.9.0\jars\*;C:\localbin\apache-drill-1.9.0\ > jars\ext\*;C:\localbin\apache-drill-1.9.0\jars\3rdparty\*;C: > \localbin\apache-drill-1.9.0\jars\classb\*" sqlline.SqlLine -d > org.apache.drill.jdbc.Driver -u jdbc:drill:zk=local > > Unfortunately, that still results in only 7GB of memory associated with > drill. > > Independent of the Windows question, can you confirm what ultimate options > configure the memory allocations? Should I be passing a particular -D > option in my java launch command? Is something read from an environment > variable on JVM creation? There's a fair amount of user tweaking required > beyond the published docs to get drill-embeded to work on Windows, so if I > knew what to drill was looking for I could probably get the right launching > magic to occur. :) > > David > > On Tue, Mar 7, 2017 at 10:32 AM, Kunal Khatua <kkha...@mapr.com> wrote: > >> I've not tried running Drill embedded on Windows, but you can try >> checking the parameters passed to the running Drill JVM to validate that >> the settings were picked up ? >> >> >> Kunal Khatua >> >> Engineering >> >> [MapR]<http://www.mapr.com/> >> >> www.mapr.com<http://www.mapr.com/> >> >> ________________________________ >> From: David F. Severski <da...@severski.net> >> Sent: Monday, March 6, 2017 10:10:44 AM >> To: user@drill.apache.org >> Subject: Configuring Drill Memory Usage under Windows >> >> Greetings! >> >> I'm a new user of Drill 1.9.0 under Windows 10 w/Java 1.8.0_121 (x64). I >> am >> trying to configure drill-embedded to have more direct memory available to >> it than the default 7GB I see when starting on my 32GB equipped >> workstation. Uncommenting the DRILL_HEAP and DRILL_MAX_DIRECT_MEMORY >> settings from `conf/drill-env.sh` and setting them to 16G has no effect >> (value of direct_max via "select * from sys.memory;" is unchanged [7Gig] >> after a restart). >> >> General web searches and specific searches on Stack Overflow haven't >> turned >> up any similar issues. What is the correct way to increase memory >> available >> to drill when launching under Windows? >> >> David >> > >
