[
https://issues.apache.org/jira/browse/ARROW-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17290873#comment-17290873
]
Anton Malakhov commented on ARROW-7001:
---------------------------------------
Please consider ideas from this blog:
https://habr.com/en/company/intel/blog/542908/
> [C++] Develop threading APIs to accommodate nested parallelism
> ---------------------------------------------------------------
>
> Key: ARROW-7001
> URL: https://issues.apache.org/jira/browse/ARROW-7001
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
> Reporter: Wes McKinney
> Assignee: Weston Pace
> Priority: Major
> Fix For: 4.0.0
>
>
> Tasks invoked in parallel may be able to submit their own subtasks, which in
> OpenMP and TBB documentation is often called "nested parallelism".
> If a task blocks on the completion of subtasks, then outright deadlocks are
> possible -- running tasks are all blocking on their subtasks, but the thread
> pool will not schedule any further tasks.
> I suggest that such code have a way to indicate to the thread pool (if one is
> passed in) that it is blocking on the completion of other tasks so that
> further tasks can be run while the task waits for its child tasks to
> complete. One possible way to do this is to have a floating "soft limit" for
> concurrent tasks that can be incremented when tasks are waiting.
> So if we normally allow 8 concurrent tasks, then this can be temporarily
> increased for each "suspended" task. Preferably we would provide some way for
> the dependent task group to "awaken" the suspended task so that it does not
> have to do any work while waiting for the task group to finish
> Note this feature can also be used in tasks that are waiting for IO calls
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
