[jira] [Created] (HIVE-6448) Java heap space pasring a query with many union all

[dima machlin (JIRA)]

dima machlin created HIVE-6448:
----------------------------------

             Summary: Java heap space pasring a query with many union all
                 Key: HIVE-6448
                 URL: https://issues.apache.org/jira/browse/HIVE-6448
             Project: Hive
          Issue Type: Bug
          Components: HiveServer2, Query Processor
    Affects Versions: 0.11.0
         Environment:  
            Reporter: dima machlin


too many union all statements in a single query require massive amount of 
memory to process.
Under default HiveServer Xmx (256) we can run ~50 union alls.
It seems to be rather linear as increasing the Xmx to 1024 we can run ~200.
The error is :

java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:2882)
        at 
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
        at 
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
        at java.lang.StringBuilder.append(StringBuilder.java:119)
        at 
org.apache.hadoop.hive.ql.QueryPlan.getJSONKeyValue(QueryPlan.java:453)
        at org.apache.hadoop.hive.ql.QueryPlan.getJSONQuery(QueryPlan.java:598)
        at org.apache.hadoop.hive.ql.QueryPlan.toString(QueryPlan.java:609)
        at 
org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress(HiveHistory.java:499)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:136)
        at 
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)