[
https://issues.apache.org/jira/browse/DRILL-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170418#comment-16170418
]
Khurram Faraaz commented on DRILL-3794:
---------------------------------------
Verified on same test setup, query completed execution successfully and returns
results on Drill 1.12.0 commit id: aaff1b35b7339fb4e6ab480dd517994ff9f0a5c5
{noformat}
No of nodes (drillbits): 3
DRILL_MAX_DIRECT_MEMORY="32G"
DRILL_MAX_HEAP="4G"
{noformat}
{noformat}
apache drill 1.12.0-SNAPSHOT
"a drill is a terrible thing to waste"
0: jdbc:drill:schema=dfs.tmp> use dfs.drillTestDirTpch01Parquet;
+-------+------------------------------------------------------------+
| ok | summary |
+-------+------------------------------------------------------------+
| true | Default schema changed to [dfs.drillTestDirTpch01Parquet] |
+-------+------------------------------------------------------------+
1 row selected (0.407 seconds)
0: jdbc:drill:schema=dfs.tmp> select
. . . . . . . . . . . . . . > n.n_name,
. . . . . . . . . . . . . . > sum(l.l_extendedprice * (1 - l.l_discount)) as
revenue
. . . . . . . . . . . . . . > from
. . . . . . . . . . . . . . > customer c,
. . . . . . . . . . . . . . > orders o,
. . . . . . . . . . . . . . > lineitem l,
. . . . . . . . . . . . . . > supplier s,
. . . . . . . . . . . . . . > nation n,
. . . . . . . . . . . . . . > region r
. . . . . . . . . . . . . . > where
. . . . . . . . . . . . . . > c.c_custkey = o.o_custkey
. . . . . . . . . . . . . . > and l.l_orderkey = o.o_orderkey
. . . . . . . . . . . . . . > and l.l_suppkey = s.s_suppkey
. . . . . . . . . . . . . . > and c.c_nationkey = s.s_nationkey
. . . . . . . . . . . . . . > and s.s_nationkey = n.n_nationkey
. . . . . . . . . . . . . . > and n.n_regionkey = r.r_regionkey
. . . . . . . . . . . . . . > and r.r_name = 'EUROPE'
. . . . . . . . . . . . . . > and o.o_orderdate >= date '1997-01-01'
. . . . . . . . . . . . . . > and o.o_orderdate < date '1997-01-01' + interval
'1' year
. . . . . . . . . . . . . . > group by
. . . . . . . . . . . . . . > n.n_name
. . . . . . . . . . . . . . > order by
. . . . . . . . . . . . . . > revenue desc;
+-----------------+-----------------------+
| n_name | revenue |
+-----------------+-----------------------+
| ROMANIA | 3.7675766801630807E9 |
| UNITED KINGDOM | 2.0677597962654018E9 |
| GERMANY | 1.9891449989498963E9 |
| RUSSIA | 1.854783967116909E9 |
| FRANCE | 1.1127486537297122E9 |
+-----------------+-----------------------+
5 rows selected (44.673 seconds)
{noformat}
> Fragments outlive query execution in certain out of memory cases
> ----------------------------------------------------------------
>
> Key: DRILL-3794
> URL: https://issues.apache.org/jira/browse/DRILL-3794
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Reporter: Rahul Challapalli
> Fix For: Future
>
> Attachments: error.log, jstack.log, lineitem.parquet
>
>
> git.commit.id.abbrev=240a455
> I created the data set by duplicating the attached lineitem.parquet
> (tpch0.01) file 5000 times into the lineitem folder.
> The below query fails with an out of memory error
> {code}
> 0: jdbc:drill:zk=10.10.100.190:5181> select
> . . . . . . . . . . . . . . . . . .> n.n_name,
> . . . . . . . . . . . . . . . . . .> sum(l.l_extendedprice * (1 -
> l.l_discount)) as revenue
> . . . . . . . . . . . . . . . . . .> from
> . . . . . . . . . . . . . . . . . .> customer c,
> . . . . . . . . . . . . . . . . . .> orders o,
> . . . . . . . . . . . . . . . . . .> lineitem l,
> . . . . . . . . . . . . . . . . . .> supplier s,
> . . . . . . . . . . . . . . . . . .> nation n,
> . . . . . . . . . . . . . . . . . .> region r
> . . . . . . . . . . . . . . . . . .> where
> . . . . . . . . . . . . . . . . . .> c.c_custkey = o.o_custkey
> . . . . . . . . . . . . . . . . . .> and l.l_orderkey = o.o_orderkey
> . . . . . . . . . . . . . . . . . .> and l.l_suppkey = s.s_suppkey
> . . . . . . . . . . . . . . . . . .> and c.c_nationkey = s.s_nationkey
> . . . . . . . . . . . . . . . . . .> and s.s_nationkey = n.n_nationkey
> . . . . . . . . . . . . . . . . . .> and n.n_regionkey = r.r_regionkey
> . . . . . . . . . . . . . . . . . .> and r.r_name = 'EUROPE'
> . . . . . . . . . . . . . . . . . .> and o.o_orderdate >= date '1997-01-01'
> . . . . . . . . . . . . . . . . . .> and o.o_orderdate < date '1997-01-01'
> + interval '1' year
> . . . . . . . . . . . . . . . . . .> group by
> . . . . . . . . . . . . . . . . . .> n.n_name
> . . . . . . . . . . . . . . . . . .> order by
> . . . . . . . . . . . . . . . . . .> revenue desc;
> java.lang.RuntimeException: java.sql.SQLException: RESOURCE ERROR: One or
> more nodes ran out of memory while executing the query.
> Fragment 5:16
> [Error Id: 73a0cd7e-507c-488d-8126-3caa1849a633 on qa-node191.qa.lab:31010]
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {code}
> Now when I ran a jstack on the drillbit I still see that there are fragments
> in the waiting state. I attached the jstack output as well.
> My cluster config
> {code}
> No Of Nodes : 2
> DRILL_MAX_DIRECT_MEMORY="32G"
> DRILL_MAX_HEAP="4G"
> {code}
> Let me know if you need anything
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
