[
https://issues.apache.org/jira/browse/ARROW-14191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-14191:
-----------------------------------
Labels: kernel pull-request-available query-engine (was: kernel
query-engine)
> [C++][Dataset] Dataset writes should respect backpressure
> ---------------------------------------------------------
>
> Key: ARROW-14191
> URL: https://issues.apache.org/jira/browse/ARROW-14191
> Project: Apache Arrow
> Issue Type: Improvement
> Reporter: Weston Pace
> Assignee: Weston Pace
> Priority: Major
> Labels: kernel, pull-request-available, query-engine
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If the write destination is slow then the dataset writer should back off and
> apply backpressure to pause the reader. This will allow simple dataset API
> scans to operate on large out of core datasets.
> This is dependent on ARROW-13611 which adds a backpressure feature for
> regular scanning (but not writing data) and on ARROW-13542 which moves the
> dataset write to a node in the exec plan.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
