[
https://issues.apache.org/jira/browse/DRILL-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Girish closed DRILL-6813.
----------------------------------
> Inner-Join between two tar.gz files throwing error- One or more nodes ran out
> of memory while executing the query
> -----------------------------------------------------------------------------------------------------------------
>
> Key: DRILL-6813
> URL: https://issues.apache.org/jira/browse/DRILL-6813
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.14.0
> Reporter: Ashish Pancholi
> Assignee: Abhishek Girish
> Priority: Major
>
> I am using `Apache Drill` version `1.14` on `windows` system having 8 GB RAM
> and running Drill using the command:
> {code:java}
> sqlline.bat -u "jdbc:drill:zk=local"{code}
> I am trying to `execute` a `join query` on two `compressed` and `archived`
> `CSV` files.
> Query:
> {code:java}
> SELECT * FROM
> dfs.`C:\Users\admin\Desktop\DRILL_FILES\csvFileParquet\TBL_MOREDATA-20180924181406.tar.gz`
> AS Table0 INNER JOIN
> dfs.`C:\Users\admin\Desktop\DRILL_FILES\csvFileParquet\TBL_MOREDATA1-20180924181406.tar.gz`
> AS Table1 ON Table0.columns[0]=Table1.columns[0]{code}
> But an out of memory, error occurred:
> {code:java}
> org.apache.drill.common.exceptions.UserRemoteException: RESOURCE ERROR: One
> or more nodes ran out of memory while executing the query. Unable to allocate
> buffer of size 131072 (rounded from 86104) due to memory limit (630194176).
> Current allocation: 630108434 Fragment 0:0 [Error Id:
> 585c0644-5fd5-446e-b9b3-d48e0771eb2a on DESKTOP-SM3E3KM:31010]{code}
> To resolve the issue, I tried to update `config\drill-env.sh` file but the
> issue remains the same and it looks like updating the script file does not
> reflect the changes because I am trying to increase the DIRECT MEMORY beyond
> the system memory (RAM), every time drill starts up peacefully. Not even
> complaining that you have exceeded the memory, therefore, it looks like the
> changes are not reflecting.
> {code:java}
> export DRILLBIT_MAX_PROC_MEM=12G
> export DRILL_HEAP=2G
> export DRILL_MAX_DIRECT_MEMORY=10G{code}
> whereas my system's main memory is only 8 GB.
> Please help me to resolve the out of memory error. I had even run the below
> queries, in order to follow the troubleshooting instructions but the issue
> remains the same.
> {code:java}
> alter session set `planner.enable_hashagg` = false;
> alter session set `planner.enable_hashjoin` = false;
> alter session set planner.width.max_per_node=3;
> alter system set planner.width.max_per_query = 100;{code}
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
