Rahul Challapalli created DRILL-3794:
----------------------------------------
Summary: Fragments outlive query execution in certain out of
memory cases
Key: DRILL-3794
URL: https://issues.apache.org/jira/browse/DRILL-3794
Project: Apache Drill
Issue Type: Bug
Components: Execution - Flow
Reporter: Rahul Challapalli
Assignee: Chris Westin
Fix For: 1.2.0
git.commit.id.abbrev=240a455
I created the data set by duplicating the attached lineitem.parquet (tpch0.01)
file 5000 times into the lineitem folder.
The below query fails with an out of memory error
{code}
0: jdbc:drill:zk=10.10.100.190:5181> select
. . . . . . . . . . . . . . . . . .> n.n_name,
. . . . . . . . . . . . . . . . . .> sum(l.l_extendedprice * (1 -
l.l_discount)) as revenue
. . . . . . . . . . . . . . . . . .> from
. . . . . . . . . . . . . . . . . .> customer c,
. . . . . . . . . . . . . . . . . .> orders o,
. . . . . . . . . . . . . . . . . .> lineitem l,
. . . . . . . . . . . . . . . . . .> supplier s,
. . . . . . . . . . . . . . . . . .> nation n,
. . . . . . . . . . . . . . . . . .> region r
. . . . . . . . . . . . . . . . . .> where
. . . . . . . . . . . . . . . . . .> c.c_custkey = o.o_custkey
. . . . . . . . . . . . . . . . . .> and l.l_orderkey = o.o_orderkey
. . . . . . . . . . . . . . . . . .> and l.l_suppkey = s.s_suppkey
. . . . . . . . . . . . . . . . . .> and c.c_nationkey = s.s_nationkey
. . . . . . . . . . . . . . . . . .> and s.s_nationkey = n.n_nationkey
. . . . . . . . . . . . . . . . . .> and n.n_regionkey = r.r_regionkey
. . . . . . . . . . . . . . . . . .> and r.r_name = 'EUROPE'
. . . . . . . . . . . . . . . . . .> and o.o_orderdate >= date '1997-01-01'
. . . . . . . . . . . . . . . . . .> and o.o_orderdate < date '1997-01-01' +
interval '1' year
. . . . . . . . . . . . . . . . . .> group by
. . . . . . . . . . . . . . . . . .> n.n_name
. . . . . . . . . . . . . . . . . .> order by
. . . . . . . . . . . . . . . . . .> revenue desc;
java.lang.RuntimeException: java.sql.SQLException: RESOURCE ERROR: One or more
nodes ran out of memory while executing the query.
Fragment 5:16
[Error Id: 73a0cd7e-507c-488d-8126-3caa1849a633 on qa-node191.qa.lab:31010]
at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
at
sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
at sqlline.SqlLine.print(SqlLine.java:1583)
at sqlline.Commands.execute(Commands.java:852)
at sqlline.Commands.sql(Commands.java:751)
at sqlline.SqlLine.dispatch(SqlLine.java:738)
at sqlline.SqlLine.begin(SqlLine.java:612)
at sqlline.SqlLine.start(SqlLine.java:366)
at sqlline.SqlLine.main(SqlLine.java:259)
{code}
Now when I ran a jstack on the drillbit I still see that there are fragments in
the waiting state. I attached the jstack output as well.
My cluster config
{code}
No Of Nodes : 2
DRILL_MAX_DIRECT_MEMORY="32G"
DRILL_MAX_HEAP="4G"
{code}
Let me know if you need anything
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
