[jira] [Updated] (DRILL-3794) Fragments outlive query execution in certain out of memory cases
[ https://issues.apache.org/jira/browse/DRILL-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khatua updated DRILL-3794: Reviewer: Khurram Faraaz [~khfaraaz] Can you verify if this issue is resolved with DRILL-5599 (Drill 1.11.0)? > Fragments outlive query execution in certain out of memory cases > > > Key: DRILL-3794 > URL: https://issues.apache.org/jira/browse/DRILL-3794 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Reporter: Rahul Challapalli > Fix For: Future > > Attachments: error.log, jstack.log, lineitem.parquet > > > git.commit.id.abbrev=240a455 > I created the data set by duplicating the attached lineitem.parquet > (tpch0.01) file 5000 times into the lineitem folder. > The below query fails with an out of memory error > {code} > 0: jdbc:drill:zk=10.10.100.190:5181> select > . . . . . . . . . . . . . . . . . .> n.n_name, > . . . . . . . . . . . . . . . . . .> sum(l.l_extendedprice * (1 - > l.l_discount)) as revenue > . . . . . . . . . . . . . . . . . .> from > . . . . . . . . . . . . . . . . . .> customer c, > . . . . . . . . . . . . . . . . . .> orders o, > . . . . . . . . . . . . . . . . . .> lineitem l, > . . . . . . . . . . . . . . . . . .> supplier s, > . . . . . . . . . . . . . . . . . .> nation n, > . . . . . . . . . . . . . . . . . .> region r > . . . . . . . . . . . . . . . . . .> where > . . . . . . . . . . . . . . . . . .> c.c_custkey = o.o_custkey > . . . . . . . . . . . . . . . . . .> and l.l_orderkey = o.o_orderkey > . . . . . . . . . . . . . . . . . .> and l.l_suppkey = s.s_suppkey > . . . . . . . . . . . . . . . . . .> and c.c_nationkey = s.s_nationkey > . . . . . . . . . . . . . . . . . .> and s.s_nationkey = n.n_nationkey > . . . . . . . . . . . . . . . . . .> and n.n_regionkey = r.r_regionkey > . . . . . . . . . . . . . . . . . .> and r.r_name = 'EUROPE' > . . . . . . . . . . . . . . . . . .> and o.o_orderdate >= date '1997-01-01' > . . . . . . . . . . . . . . . . . .> and o.o_orderdate < date '1997-01-01' > + interval '1' year > . . . . . . . . . . . . . . . . . .> group by > . . . . . . . . . . . . . . . . . .> n.n_name > . . . . . . . . . . . . . . . . . .> order by > . . . . . . . . . . . . . . . . . .> revenue desc; > java.lang.RuntimeException: java.sql.SQLException: RESOURCE ERROR: One or > more nodes ran out of memory while executing the query. > Fragment 5:16 > [Error Id: 73a0cd7e-507c-488d-8126-3caa1849a633 on qa-node191.qa.lab:31010] > at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73) > at > sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87) > at sqlline.TableOutputFormat.print(TableOutputFormat.java:118) > at sqlline.SqlLine.print(SqlLine.java:1583) > at sqlline.Commands.execute(Commands.java:852) > at sqlline.Commands.sql(Commands.java:751) > at sqlline.SqlLine.dispatch(SqlLine.java:738) > at sqlline.SqlLine.begin(SqlLine.java:612) > at sqlline.SqlLine.start(SqlLine.java:366) > at sqlline.SqlLine.main(SqlLine.java:259) > {code} > Now when I ran a jstack on the drillbit I still see that there are fragments > in the waiting state. I attached the jstack output as well. > My cluster config > {code} > No Of Nodes : 2 > DRILL_MAX_DIRECT_MEMORY="32G" > DRILL_MAX_HEAP="4G" > {code} > Let me know if you need anything -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (DRILL-3794) Fragments outlive query execution in certain out of memory cases
[ https://issues.apache.org/jira/browse/DRILL-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Challapalli updated DRILL-3794: - Attachment: lineitem.parquet error.log jstack.log > Fragments outlive query execution in certain out of memory cases > > > Key: DRILL-3794 > URL: https://issues.apache.org/jira/browse/DRILL-3794 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Reporter: Rahul Challapalli >Assignee: Chris Westin > Fix For: 1.2.0 > > Attachments: error.log, jstack.log, lineitem.parquet > > > git.commit.id.abbrev=240a455 > I created the data set by duplicating the attached lineitem.parquet > (tpch0.01) file 5000 times into the lineitem folder. > The below query fails with an out of memory error > {code} > 0: jdbc:drill:zk=10.10.100.190:5181> select > . . . . . . . . . . . . . . . . . .> n.n_name, > . . . . . . . . . . . . . . . . . .> sum(l.l_extendedprice * (1 - > l.l_discount)) as revenue > . . . . . . . . . . . . . . . . . .> from > . . . . . . . . . . . . . . . . . .> customer c, > . . . . . . . . . . . . . . . . . .> orders o, > . . . . . . . . . . . . . . . . . .> lineitem l, > . . . . . . . . . . . . . . . . . .> supplier s, > . . . . . . . . . . . . . . . . . .> nation n, > . . . . . . . . . . . . . . . . . .> region r > . . . . . . . . . . . . . . . . . .> where > . . . . . . . . . . . . . . . . . .> c.c_custkey = o.o_custkey > . . . . . . . . . . . . . . . . . .> and l.l_orderkey = o.o_orderkey > . . . . . . . . . . . . . . . . . .> and l.l_suppkey = s.s_suppkey > . . . . . . . . . . . . . . . . . .> and c.c_nationkey = s.s_nationkey > . . . . . . . . . . . . . . . . . .> and s.s_nationkey = n.n_nationkey > . . . . . . . . . . . . . . . . . .> and n.n_regionkey = r.r_regionkey > . . . . . . . . . . . . . . . . . .> and r.r_name = 'EUROPE' > . . . . . . . . . . . . . . . . . .> and o.o_orderdate >= date '1997-01-01' > . . . . . . . . . . . . . . . . . .> and o.o_orderdate < date '1997-01-01' > + interval '1' year > . . . . . . . . . . . . . . . . . .> group by > . . . . . . . . . . . . . . . . . .> n.n_name > . . . . . . . . . . . . . . . . . .> order by > . . . . . . . . . . . . . . . . . .> revenue desc; > java.lang.RuntimeException: java.sql.SQLException: RESOURCE ERROR: One or > more nodes ran out of memory while executing the query. > Fragment 5:16 > [Error Id: 73a0cd7e-507c-488d-8126-3caa1849a633 on qa-node191.qa.lab:31010] > at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73) > at > sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87) > at sqlline.TableOutputFormat.print(TableOutputFormat.java:118) > at sqlline.SqlLine.print(SqlLine.java:1583) > at sqlline.Commands.execute(Commands.java:852) > at sqlline.Commands.sql(Commands.java:751) > at sqlline.SqlLine.dispatch(SqlLine.java:738) > at sqlline.SqlLine.begin(SqlLine.java:612) > at sqlline.SqlLine.start(SqlLine.java:366) > at sqlline.SqlLine.main(SqlLine.java:259) > {code} > Now when I ran a jstack on the drillbit I still see that there are fragments > in the waiting state. I attached the jstack output as well. > My cluster config > {code} > No Of Nodes : 2 > DRILL_MAX_DIRECT_MEMORY="32G" > DRILL_MAX_HEAP="4G" > {code} > Let me know if you need anything -- This message was sent by Atlassian JIRA (v6.3.4#6332)