[
https://issues.apache.org/jira/browse/DRILL-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505224#comment-16505224
]
ASF GitHub Bot commented on DRILL-6477:
---------------------------------------
kkhatua opened a new pull request #1309: DRILL-6477: Drillbit crashes with OOME
(Heap) for a large WebUI query
URL: https://github.com/apache/drill/pull/1309
For queries submitted through the WebUI and retrieving a large result-set,
the Drillbit often hangs or crashes due to the (foreman) Drillbit running out
of Heap memory.
This is because the Web client translates the result set into a massive
object in the heap-space and tries to send that back to the browser. This
results in the VM thread actively trying to perform GC if the memory is not
sufficient.
The workaround is to have the active webConnection of the query periodically
timeout to allow for checking the consumed heap-space. A level of 0.85 (i.e.
85%) is set as default threshold, crossing which, a query submitted through the
REST API is marked and failed.
In addition, a user exception is thrown, indicting the cause of the query
failing, along with alternative suggestions for re-executing the query.
This is the example of a query that tried to scan an entire 60M row table
through the browser. The query failed with the following error message:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Drillbit hangs/crashes with OOME Java Heap Space for a large query through
> WebUI
> --------------------------------------------------------------------------------
>
> Key: DRILL-6477
> URL: https://issues.apache.org/jira/browse/DRILL-6477
> Project: Apache Drill
> Issue Type: Bug
> Components: Web Server
> Affects Versions: 1.13.0
> Reporter: Kunal Khatua
> Assignee: Kunal Khatua
> Priority: Major
> Fix For: 1.14.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> For queries submitted through the WebUI and retrieving a large resultset, the
> Drillbit often hangs or crashes due to the (foreman) Drillbit running out of
> Heap memory.
> This is because the Web client translates the resultset into a massive object
> in the heap-space and tries to send that back to the browser. This results in
> the VM thread actively trying to perform GC if the memory is not sufficient.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
