[
https://issues.apache.org/jira/browse/DRILL-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570821#comment-17570821
]
ASF GitHub Bot commented on DRILL-8268:
---------------------------------------
jnturton commented on PR #2610:
URL: https://github.com/apache/drill/pull/2610#issuecomment-1193860317
@luocooong @paul-rogers a bit of data related to the heap memory usage
limiting from some experiments run by a helpful user.
I did test the recommendation given by you and it worked well. Below are
steps of testing
> drill.exec.http.memory.heap.failure.threshold = 2
>
> Before making changes in the drill-override.conf
> 1. Opened 4 sessions of drill web UI query
> 2. Fired same query in all 4 sessions. This query will scan through 8000
parquet compress files
>
> Results - all 4 sessions gave heap error.
> Observations
> 1. The heap memory displayed on "metrics" tab varies between 30-80% while
query is running
> 2. The heap memory is constant at 87.89% after queries are failed
>
> After making the changes in the drill-overide.conf as recommended by you
> 1. Opened 4 sessions of drill web UI query
> 2. Fired same query in all 4 sessions. This query will scan through 8000
parquet compress files
> Results - All 4 sessions returned the results. It took average 6+ minutes
to return the results
> Observations
> 1. Could not observe the heap memory displayed on "metrics" as display was
just in running state and then gave error "page cannot be displayed" during
query run
> 2. The heap memory is varying between 69.71% after queries are
successfully completed
>
> I ran the same queries again and again and below is the heap memory after
every run
> 2nd run - Between 61-62%
> 3rd run - . Between 82-83%
> 4th Run - Between 69-70%
> 5th Run - Between 65-66%
> 6th run - Between 69-70%
> 7th run - Between 70-71%
> 8th run - increased one session, so now total are 5 web sessions - 1
session gave "Page cannot be displayed error. 4 returned the results - Memory
79-80%
> 9th run (5 sessions) - - 1 session gave "Page cannot be displayed error. 4
returned the results - Memory 66-67%
>
> Fix Hadoop 2 and Netty lib exclusions, REST mem limiter disabled by default
> ---------------------------------------------------------------------------
>
> Key: DRILL-8268
> URL: https://issues.apache.org/jira/browse/DRILL-8268
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.20.1
> Reporter: James Turton
> Assignee: James Turton
> Priority: Major
> Fix For: 1.20.2
>
>
> # New exclusions of reload4j, slf4j-reload4j are required in the Hadoop 2
> profile, probably due to the upgrade of Hadoop from 2.10.1 to 2.10.2.
> # We remove the netty-all metapackage which entered the dependency tree with
> the change introducing the Netty bom bringing many uneeded libs with it.
> # The heap memory usage limiting logic in the REST server becomes disabled
> by default since REST query results are streamed these days. This change aims
> to let the Java GC now do its job without interference and if that results in
> OOM under a constant load then there is good evidence for a heap leak which
> must be tracked down and completely resolved anyway, not mitigated or "swept
> under a rug".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
