[
https://issues.apache.org/jira/browse/DRILL-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570295#comment-17570295
]
ASF GitHub Bot commented on DRILL-8268:
---------------------------------------
jnturton commented on PR #2610:
URL: https://github.com/apache/drill/pull/2610#issuecomment-1193118355
@luocooong the main goal of this PR is to solve issues in RC0 of Drill
1.20.2 and produce RC1.
Unrelated to that, I can expand a bit more on my reasoning for the commit
disabling the heap memory usage limiter in the REST server by default. Since we
stream rather than buffer REST API query results now, we no longer have an
excuse for runaway heap usage. Further, while preventing the next REST query
from running can save the Drillbit from an OOM death, when the user complains
that they cannot run their query an admin is very likely to restart that
Drillbit _anyway_. So we probably only delayed the inevitable even though the
landing was made a bit softer and other processing on the Drillbit may have
been able to continue.
Since now the only things that can cause runaway heap usage (that I can see)
under a constant load are leaks, unecessary object references in Java's case, I
think we should bite the bullet and fail hard with OOM when we exhaust the
heap. The Drillbit can easily be reincarnated upon its death by systemd or
another supervisor, which is preferable to having it limp on declining queries
arriving over REST. We also get to stop applying our own heap management rule
which may interact badly with the GC in use. E.g. if we start turning REST
queries away at 85% heap usage but 90% heap usage would have stirred the user's
GC to do a deep clean (in reality I think they all kick in considerably earlier
but I'm really making a design argument here) then we are working against our
own JVM. Lastly any ensuing bug reports overtly containing "OutOfMemoryError"
are more likely, I believe, to send us hunting heap memory leaks.
cc @paul-rogers.
> Fix Hadoop 2 and Netty lib exclusions, REST mem limiter disabled by default
> ---------------------------------------------------------------------------
>
> Key: DRILL-8268
> URL: https://issues.apache.org/jira/browse/DRILL-8268
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.20.1
> Reporter: James Turton
> Assignee: James Turton
> Priority: Major
> Fix For: 1.20.2
>
>
> # New exclusions of reload4j, slf4j-reload4j are required in the Hadoop 2
> profile, probably due to the upgrade of Hadoop from 2.10.1 to 2.10.2.
> # We remove the netty-all metapackage which entered the dependency tree with
> the change introducing the Netty bom bringing many uneeded libs with it.
> # The heap memory usage limiting logic in the REST server becomes disabled
> by default since REST query results are streamed these days. This change aims
> to let the Java GC now do its job without interference and if that results in
> OOM under a constant load then there is good evidence for a heap leak which
> must be tracked down and completely resolved anyway, not mitigated or "swept
> under a rug".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
