Hi.
As I may have mentioned earlier, I'm implementing OpenMP 4.1's task
priorities for GCC's libgomp. This has had me looking at how we
currently implement scheduling, and I believe we have a problem in how
we rearrange dependencies.
Currently in gomp_task_maybe_wait_for_dependencies(), we have the
following code for rearranging the first dependency (tsk) from a list of
dependencies.
else if (tsk != task->children)
{
/* Remove tsk from the sibling list... */
tsk->prev_child->next_child = tsk->next_child;
tsk->next_child->prev_child = tsk->prev_child;
/* ...and insert it into the running task's
children. */
-->BAD tsk->prev_child = task->children;
-->BAD tsk->next_child = task->children->next_child;
task->children = tsk;
tsk->prev_child->next_child = tsk;
tsk->next_child->prev_child = tsk;
}
If say, you have a parent_depends_on task PD1 that is in the following
children queue:
C1 -> C2 -> C3 -> PD1 -> C4
(Where parent->children points to C1 and C4 wraps around to C1 as per
the circular list.)
The above code will transform the children queue into:
PD1 -> C2 -> C3 -> C4 -> C1
The surrounding code looks sane, but this particular snippet quoted
above has us moving the first child to the end of the queue, which is
probably not what we want. However, at least we're still keeping the
parent_depends_on tasks first, just that we moved other previously
unaffected children to the end of the queue.
What we really want is:
PD1 -> C1 -> C2 -> C3 -> C4
The attached patch does just that. I have also rewritten the comments
now that I actually understand what's going on :). Eventually this will
all become clearer with my upcoming/proposed API for dealing with all
these queues.
As discussed on IRC, there is another issue you pointed out in
gomp_task_run_pre(), which I will address in a followup patch.
Tested on x86-64 Linux by running "make check-target-libgomp" with
OMP_NUM_THREADS={1,2,55}.
OK for branch?
commit 6d8f6db0583326d803c7c7abd8ea26cc842643fc
Author: Aldy Hernandez <al...@redhat.com>
Date: Fri Oct 2 15:40:30 2015 -0700
* task.c (gomp_task_maybe_wait_for_dependencies): Fix scheduling
problem such that the first non parent_depends_on task does not
end up at the end of the children queue.
diff --git a/libgomp/task.c b/libgomp/task.c
index f6a67eb..5c412fc 100644
--- a/libgomp/task.c
+++ b/libgomp/task.c
@@ -1140,17 +1140,37 @@ gomp_task_maybe_wait_for_dependencies (void **depend)
{
tsk->parent_depends_on = true;
++num_awaited;
+ /* If a task we need to wait for is not already
+ running and is ready to be scheduled, move it to
+ front, so that we run it as soon as possible.
+
+ We rearrange the children queue such that all
+ parent_depends_on tasks are first, and
+ last_parent_depends_on points to the last such task
+ we rearranged. For example, given the following
+ children where PD[123] are the parent_depends_on
+ tasks:
+
+ task->children
+ |
+ V
+ C1 -> C2 -> C3 -> PD1 -> PD2 -> PD3 -> C4
+
+ We rearrange such that:
+
+ task->children
+ | +--- last_parent_depends_on
+ | |
+ V V
+ PD1 -> PD2 -> PD3 -> C1 -> C2 -> C3 -> C4
+ */
+
if (tsk->num_dependees == 0 && tsk->kind == GOMP_TASK_WAITING)
{
- /* If a task we need to wait for is not already
- running and is ready to be scheduled, move it
- to front, so that we run it as soon as possible. */
if (last_parent_depends_on)
{
- /* Remove tsk from the sibling list... */
tsk->prev_child->next_child = tsk->next_child;
tsk->next_child->prev_child = tsk->prev_child;
- /* ...and insert it into last_parent_depends_on. */
tsk->prev_child = last_parent_depends_on;
tsk->next_child = last_parent_depends_on->next_child;
tsk->prev_child->next_child = tsk;
@@ -1158,21 +1178,14 @@ gomp_task_maybe_wait_for_dependencies (void **depend)
}
else if (tsk != task->children)
{
- /* Remove tsk from the sibling list... */
tsk->prev_child->next_child = tsk->next_child;
tsk->next_child->prev_child = tsk->prev_child;
- /* ...and insert it into the running task's
- children. */
- tsk->prev_child = task->children;
- tsk->next_child = task->children->next_child;
+ tsk->prev_child = task->children->prev_child;
+ tsk->next_child = task->children;
task->children = tsk;
tsk->prev_child->next_child = tsk;
tsk->next_child->prev_child = tsk;
}
- else
- {
- /* It's already in task->children. Nothing to do. */;
- }
last_parent_depends_on = tsk;
}
}
