[
https://issues.apache.org/jira/browse/PARQUET-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16741008#comment-16741008
]
Artavazd Balaian commented on PARQUET-1491:
-------------------------------------------
It was problem with my environment. I put {{log4j.properties}} configuration in
wrong place that's why it was in {{DEBUG}} mode every time, that's why I saw a
lot of allocations. We can close this issue.
> Conditional debug logging in InternalParquetRecordReader to reduce GC
> ---------------------------------------------------------------------
>
> Key: PARQUET-1491
> URL: https://issues.apache.org/jira/browse/PARQUET-1491
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-avro
> Reporter: Artavazd Balaian
> Priority: Minor
> Labels: pull-request-available
> Attachments: image-2019-01-12-04-03-48-005.png,
> image-2019-01-12-04-09-18-359.png, image-2019-01-12-04-10-49-230.png
>
>
> Currently there is no check for the log level in
> [https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/InternalParquetRecordReader.javaL249|https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/InternalParquetRecordReader.java#L230;L249.]
> which causes a lot of memory allocation and performance degradation.
> Link to parquet file which was used:
> [https://drive.google.com/open?id=1xCMZrUPWvlS4KOFO8m9EmtkvDy-SiRHq]
> Screenshot of Java Mission Control comparison with fix and without (link to
> the JFR files
> [https://drive.google.com/open?id=1blSeF-AyAhQyRYaqVsihyzy7pJCJt7U3):]
> !image-2019-01-12-04-03-48-005.png|width=956,height=538!
> !image-2019-01-12-04-10-49-230.png|width=1403,height=760!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
