[
https://issues.apache.org/jira/browse/ARROW-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weston Pace updated ARROW-8599:
-------------------------------
Fix Version/s: (was: 4.0.0)
5.0.0
> [C++][Parquet] Optional parallel processing when writing Parquet files
> ----------------------------------------------------------------------
>
> Key: ARROW-8599
> URL: https://issues.apache.org/jira/browse/ARROW-8599
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Wes McKinney
> Assignee: Weston Pace
> Priority: Major
> Fix For: 5.0.0
>
>
> If we permit encoded columns in row groups to be buffered in memory rather
> than immediately written out to the {{OutputStream}}, then we can use
> multiple threads for the encoding / compression of columns. Combined with a
> separate thread to take the encoded columns and write them out to disk, this
> should yield substantially improved file write times.
> This could be enabled through an option since it would increase memory use
> when writing. The memory use can be somewhat constrained by limiting the size
> of row groups
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
