[
https://issues.apache.org/jira/browse/ORC-21?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14996980#comment-14996980
]
ASF GitHub Bot commented on ORC-21:
-----------------------------------
Github user omalley commented on a diff in the pull request:
https://github.com/apache/orc/pull/12#discussion_r44304442
--- Diff: c++/src/Reader.cc ---
@@ -1433,6 +1538,7 @@ namespace orc {
}
void ReaderImpl::startNextStripe() {
+ reader.reset(); // ColumnReaders use lots of memory; free old memory
first
--- End diff --
Sorry, I missed that it was the reset on the unique_ptr rather than the
ColumnReader.
In my profiling, one of the major costs for files with little stripe sizes
is rebuilding the ColumnReader and the associated buffer. At some point, we
need to clean it up so that it reuses the current ColumnReader and buffers.
> Add functionality to estimate memory footprint
> ----------------------------------------------
>
> Key: ORC-21
> URL: https://issues.apache.org/jira/browse/ORC-21
> Project: Orc
> Issue Type: Task
> Reporter: Aliaksei Sandryhaila
> Assignee: Aliaksei Sandryhaila
>
> ORC library allocates multiple large buffers to read and materialize ORC
> files. For stability of applications that use the library, it may be
> desirable to have an estimate (preferably, a tight upper bound) of a memory
> footprint.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
