[ https://issues.apache.org/jira/browse/ARROW-500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17268799#comment-17268799 ]
Weston Pace commented on ARROW-500: ----------------------------------- The asynchronous iterator utilities now have `AddReadahead(N)` which I think adds queuing in the manner described here. That plus adding parallelism to Visit plus adding some kind of FlatMap operator will I think cover most of the cases described here. > [C++] Implement concurrent IO read queue for file-like sources > -------------------------------------------------------------- > > Key: ARROW-500 > URL: https://issues.apache.org/jira/browse/ARROW-500 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ > Reporter: Wes McKinney > Priority: Major > Labels: filesystem > > In a multithreaded setting, we may spawn many threads which will have access > to a shared IO resources. It may be useful to create a thread-safe IO queue > implementing the {{arrow::io::ReadableFileInterface}}, limiting the number of > concurrent requests to the desired number (which may be 1, for services not > permitting concurrent access). -- This message was sent by Atlassian Jira (v8.3.4#803005)