[
https://issues.apache.org/jira/browse/ARROW-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16849178#comment-16849178
]
Antoine Pitrou commented on ARROW-5428:
---------------------------------------
So this is setting bounds on the stream, right? It sounds fine on the
principle. Is it sufficient to set it at BufferedInputStream construct time, or
does it need to change afterwards?
> [C++] Add option to set "read extent" in arrow::io::BufferedInputStream
> -----------------------------------------------------------------------
>
> Key: ARROW-5428
> URL: https://issues.apache.org/jira/browse/ARROW-5428
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Wes McKinney
> Assignee: Wes McKinney
> Priority: Major
> Fix For: 0.14.0
>
>
> I'm looking at simplifying libparquet to use common IO interfaces rather than
> its own custom ones
> The {{parquet::BufferedInputStream}} interface has an option to not read
> beyond a particular number of bytes. For example, if we were reading a 32MB
> block with 1MB buffering, then we will not consume more than 32MB from the
> raw InputStream.
> This seems like a fairly trivial addition to
> {{arrow::io::BufferedInputStream}} to track total read bytes and do not read
> beyond the configured extent. We'll have to add a method like
> {{set_read_extent}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
