[
https://issues.apache.org/jira/browse/ARROW-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623386#comment-17623386
]
Weston Pace commented on ARROW-18113:
-------------------------------------
{quote}
Linking io_uring to the Future API could be a bit challenging because Linux
doesn't have a builtin thread pool API, so you still end up having to poll the
completion queue. I was thinking this could probably be integrated into the
Executor implementation we have. In between tasks it would poll the completion
queue and schedule the individual continuation futures that get finished.
{quote}
Does io_uring have any kind of select-style API method like "wait until at
least one pending read has finished"
{quote}
Using preadv would map pretty well but it would only ever return one future, I
guess something like `Future<std::vector<Buffer>>`.
{quote}
Maybe this would be a better interface. I suppose it would boil down on
whether or not the code can proceed with partial results. I think there might
be cases where this is true but I'd really have to dive into the readers to
recall exactly.
> Implement a read range process without caching
> ----------------------------------------------
>
> Key: ARROW-18113
> URL: https://issues.apache.org/jira/browse/ARROW-18113
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Percy Camilo Triveño Aucahuasi
> Assignee: Percy Camilo Triveño Aucahuasi
> Priority: Major
>
> The current
> [ReadRangeCache|https://github.com/apache/arrow/blob/e06e98db356e602212019cfbae83fd3d5347292d/cpp/src/arrow/io/caching.h#L100]
> is mixing caching with coalescing and making difficult to implement readers
> capable to really perform concurrent reads on coalesced data (see this
> [github
> comment|https://github.com/apache/arrow/pull/14226#discussion_r999334979] for
> additional context); for instance, right now the prebuffering feature of
> those readers cannot handle concurrent invocations.
> The goal for this ticket is to implement a similar component to
> ReadRangeCache for performing non-cache reads (doing only the coalescing part
> instead). So, once we have that new capability, we can port the parquet and
> IPC readers to this new component and keep improving the reading process
> (that would be part of other set of follow-up tickets). Similar ideas were
> mentioned here https://issues.apache.org/jira/browse/ARROW-17599
> Maybe a good place to implement this new capability is inside the file system
> abstraction (as part of a dedicated method to read coalesced data) and where
> the abstract file system can provide a default implementation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
