[
https://issues.apache.org/jira/browse/DRILL-8366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8366.
-------------------------------
Resolution: Fixed
> Late release of compressor memory in the Parquet writer
> -------------------------------------------------------
>
> Key: DRILL-8366
> URL: https://issues.apache.org/jira/browse/DRILL-8366
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
> Affects Versions: 1.20.2
> Reporter: James Turton
> Assignee: James Turton
> Priority: Minor
> Fix For: 1.20.3
>
>
> The Parquet writer waits until the end of the entire write before releasing
> its compression codec factory. The factory in turn releases compressors which
> release direct memory buffers used during compression. This deferred release
> leads a build up of direct memory use and can cause large write jobs to fail.
> The Parquet writer can instead release the abovementioned each time that a
> file/row group is flushed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
