[
https://issues.apache.org/jira/browse/PARQUET-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artavazd Balaian updated PARQUET-1491:
--------------------------------------
Description:
Currently there is no check for the log level in
[https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/InternalParquetRecordReader.javaL249|https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/InternalParquetRecordReader.java#L230;L249.]
which causes a lot of memory allocation and performance degradation.
Screenshot of Java Mission Control comparison with fix and without (link to the
JFR files [https://drive.google.com/open?id=1blSeF-AyAhQyRYaqVsihyzy7pJCJt7U3):]
!image-2019-01-12-04-03-48-005.png|width=956,height=538!
was:
Currently there is no check for the log level in
[https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/InternalParquetRecordReader.javaL249|https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/InternalParquetRecordReader.java#L230;L249.]
which causes a lot of memory allocation and performance degradation.
Screenshot of Java Mission Control comparison with fix and without (link to the
JFR files https://drive.google.com/open?id=1blSeF-AyAhQyRYaqVsihyzy7pJCJt7U3):
!image-2019-01-12-04-03-48-005.png!
> Conditional debug logging in InternalParquetRecordReader to reduce GC
> ---------------------------------------------------------------------
>
> Key: PARQUET-1491
> URL: https://issues.apache.org/jira/browse/PARQUET-1491
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-avro
> Reporter: Artavazd Balaian
> Priority: Minor
> Attachments: image-2019-01-12-04-03-48-005.png
>
>
> Currently there is no check for the log level in
> [https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/InternalParquetRecordReader.javaL249|https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/InternalParquetRecordReader.java#L230;L249.]
> which causes a lot of memory allocation and performance degradation.
> Screenshot of Java Mission Control comparison with fix and without (link to
> the JFR files
> [https://drive.google.com/open?id=1blSeF-AyAhQyRYaqVsihyzy7pJCJt7U3):]
> !image-2019-01-12-04-03-48-005.png|width=956,height=538!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
