[PATCH 2/2] [RFC] reservation: add suppport for read-only access using rcu

[Thomas Hellstrom]

On 04/10/2014 01:08 PM, Thomas Hellstrom wrote:
> On 04/10/2014 12:07 PM, Maarten Lankhorst wrote:
>> Hey,
>>
>> op 10-04-14 10:46, Thomas Hellstrom schreef:
>>> Hi!
>>>
>>> Ugh. This became more complicated than I thought, but I'm OK with moving
>>> TTM over to fence while we sort out
>>> how / if we're going to use this.
>>>
>>> While reviewing, it struck me that this is kind of error-prone, and hard
>>> to follow since we're operating on a structure that may be
>>> continually updated under us, needing a lot of RCU-specific macros and
>>> barriers.
>> Yeah, but with the exception of dma_buf_poll I don't think there is
>> anything else
>> outside drivers/base/reservation.c has to deal with rcu.
>>
>>> Also the rcu wait appears to not complete until there are no busy fences
>>> left (new ones can be added while we wait) rather than
>>> waiting on a snapshot of busy fences.
>> This has been by design, because 'wait for bo idle' type of functions
>> only care
>> if the bo is completely idle or not.
> No, not when using RCU, because the bo may be busy again before the
> function returns :)
> Complete idleness can only be guaranteed if holding the reservation, or
> otherwise making sure
> that no new rendering is submitted to the buffer, so it's an overkill to
> wait for complete idleness here.
>
Although, if we fail to get a refcount for a fence, and it's still busy
we need to do  a seq retry,
because the fence might have been replaced by another fence from the
same context, without being idle. That check is not present in the
snapshot code I sent.

/Thomas

[PATCH 2/2] [RFC] reservation: add suppport for read-only access using rcu

[Thomas Hellstrom]

On 04/10/2014 12:07 PM, Maarten Lankhorst wrote:
> Hey,
>
> op 10-04-14 10:46, Thomas Hellstrom schreef:
>> Hi!
>>
>> Ugh. This became more complicated than I thought, but I'm OK with moving
>> TTM over to fence while we sort out
>> how / if we're going to use this.
>>
>> While reviewing, it struck me that this is kind of error-prone, and hard
>> to follow since we're operating on a structure that may be
>> continually updated under us, needing a lot of RCU-specific macros and
>> barriers.
> Yeah, but with the exception of dma_buf_poll I don't think there is
> anything else
> outside drivers/base/reservation.c has to deal with rcu.
>
>> Also the rcu wait appears to not complete until there are no busy fences
>> left (new ones can be added while we wait) rather than
>> waiting on a snapshot of busy fences.
> This has been by design, because 'wait for bo idle' type of functions
> only care
> if the bo is completely idle or not.

No, not when using RCU, because the bo may be busy again before the
function returns :)
Complete idleness can only be guaranteed if holding the reservation, or
otherwise making sure
that no new rendering is submitted to the buffer, so it's an overkill to
wait for complete idleness here.

>
> It would be easy to make a snapshot even without seqlocks, just copy
> reservation_object_test_signaled_rcu to return a shared list if
> test_all is set, or return pointer to exclusive otherwise.
>
>> I wonder if these issues can be addressed by having a function that
>> provides a snapshot of all busy fences: This can be accomplished
>> either by including the exclusive fence in the fence_list structure and
>> allocate a new such structure each time it is updated. The RCU reader
>> could then just make a copy of the current fence_list structure pointed
>> to by &obj->fence, but I'm not sure we want to reallocate *each* time we
>> update the fence pointer.
> No, the most common operation is updating fence pointers, which is why
> the current design makes that cheap. It's also why doing rcu reads is
> more expensive.
>> The other approach uses a seqlock to obtain a consistent snapshot, and
>> I've attached an incomplete outline, and I'm not 100% whether it's OK to
>> combine RCU and seqlocks in this way...
>>
>> Both these approaches have the benefit of hiding the RCU snapshotting in
>> a single function, that can then be used by any waiting
>> or polling function.
>>
>
> I think the middle way with using seqlocks to protect the fence_excl
> pointer and shared list combination,
> and using RCU to protect the refcounts for fences and the availability
> of the list could work for our usecase
> and might remove a bunch of memory barriers. But yeah that depends on
> layering rcu and seqlocks.
> No idea if that is allowed. But I suppose it is.
>
> Also, you're being overly paranoid with seqlock reading, we would only
> need something like this:
>
> rcu_read_lock()
> preempt_disable()
> seq = read_seqcount_begin()
> read fence_excl, shared_count = ACCESS_ONCE(fence->shared_count)
> copy shared to a struct.
> if (read_seqcount_retry()) { unlock and retry }
>   preempt_enable();
>   use fence_get_rcu() to bump refcount on everything, if that fails
> unlock, put, and retry
> rcu_read_unlock()
>
> But the shared list would still need to be RCU'd, to make sure we're
> not reading freed garbage.

Ah. OK,
But I think we should use rcu inside seqcount, because
read_seqcount_begin() may spin for a long time if there are
many writers. Also I don't think the preempt_disable() is needed for
read_seq critical sections other than they might
decrease the risc of retries..

Thanks,
Thomas


>
> ~Maarten

[PATCH 2/2] [RFC] reservation: add suppport for read-only access using rcu

[Maarten Lankhorst]

Hey,

op 10-04-14 10:46, Thomas Hellstrom schreef:
> Hi!
>
> Ugh. This became more complicated than I thought, but I'm OK with moving
> TTM over to fence while we sort out
> how / if we're going to use this.
>
> While reviewing, it struck me that this is kind of error-prone, and hard
> to follow since we're operating on a structure that may be
> continually updated under us, needing a lot of RCU-specific macros and
> barriers.
Yeah, but with the exception of dma_buf_poll I don't think there is anything 
else
outside drivers/base/reservation.c has to deal with rcu.

> Also the rcu wait appears to not complete until there are no busy fences
> left (new ones can be added while we wait) rather than
> waiting on a snapshot of busy fences.
This has been by design, because 'wait for bo idle' type of functions only care
if the bo is completely idle or not.

It would be easy to make a snapshot even without seqlocks, just copy
reservation_object_test_signaled_rcu to return a shared list if test_all is 
set, or return pointer to exclusive otherwise.

> I wonder if these issues can be addressed by having a function that
> provides a snapshot of all busy fences: This can be accomplished
> either by including the exclusive fence in the fence_list structure and
> allocate a new such structure each time it is updated. The RCU reader
> could then just make a copy of the current fence_list structure pointed
> to by &obj->fence, but I'm not sure we want to reallocate *each* time we
> update the fence pointer.
No, the most common operation is updating fence pointers, which is why
the current design makes that cheap. It's also why doing rcu reads is more 
expensive.
> The other approach uses a seqlock to obtain a consistent snapshot, and
> I've attached an incomplete outline, and I'm not 100% whether it's OK to
> combine RCU and seqlocks in this way...
>
> Both these approaches have the benefit of hiding the RCU snapshotting in
> a single function, that can then be used by any waiting
> or polling function.
>

I think the middle way with using seqlocks to protect the fence_excl pointer 
and shared list combination,
and using RCU to protect the refcounts for fences and the availability of the 
list could work for our usecase
and might remove a bunch of memory barriers. But yeah that depends on layering 
rcu and seqlocks.
No idea if that is allowed. But I suppose it is.

Also, you're being overly paranoid with seqlock reading, we would only need 
something like this:

rcu_read_lock()
 preempt_disable()
 seq = read_seqcount_begin();
 read fence_excl, shared_count = ACCESS_ONCE(fence->shared_count)
 copy shared to a struct.
 if (read_seqcount_retry()) { unlock and retry }
   preempt_enable();
   use fence_get_rcu() to bump refcount on everything, if that fails unlock, 
put, and retry
rcu_read_unlock()

But the shared list would still need to be RCU'd, to make sure we're not 
reading freed garbage.

~Maarten

[PATCH 2/2] [RFC] reservation: add suppport for read-only access using rcu

[Thomas Hellstrom]

Hi!

Ugh. This became more complicated than I thought, but I'm OK with moving
TTM over to fence while we sort out
how / if we're going to use this.

While reviewing, it struck me that this is kind of error-prone, and hard
to follow since we're operating on a structure that may be
continually updated under us, needing a lot of RCU-specific macros and
barriers.

Also the rcu wait appears to not complete until there are no busy fences
left (new ones can be added while we wait) rather than
waiting on a snapshot of busy fences.

I wonder if these issues can be addressed by having a function that
provides a snapshot of all busy fences: This can be accomplished
either by including the exclusive fence in the fence_list structure and
allocate a new such structure each time it is updated. The RCU reader
could then just make a copy of the current fence_list structure pointed
to by &obj->fence, but I'm not sure we want to reallocate *each* time we
update the fence pointer.

The other approach uses a seqlock to obtain a consistent snapshot, and
I've attached an incomplete outline, and I'm not 100% whether it's OK to
combine RCU and seqlocks in this way...

Both these approaches have the benefit of hiding the RCU snapshotting in
a single function, that can then be used by any waiting
or polling function.

/Thomas



On 04/09/2014 04:49 PM, Maarten Lankhorst wrote:
> This adds 3 more functions to deal with rcu.
>
> reservation_object_wait_timeout_rcu() will wait on all fences of the
> reservation_object, without obtaining the ww_mutex.
>
> reservation_object_test_signaled_rcu() will test if all fences of the
> reservation_object are signaled without using the ww_mutex.
>
> reservation_object_get_excl() is added because touching the fence_excl
> member directly will trigger a sparse warning.
>
> Signed-off-by: Maarten Lankhorst 
> ---
>  drivers/base/dma-buf.c  |   46 +++--
>  drivers/base/reservation.c  |  147 
> +--
>  include/linux/fence.h   |   22 ++
>  include/linux/reservation.h |   40 
>  4 files changed, 224 insertions(+), 31 deletions(-)
>

-- next part --
A non-text attachment was scrubbed...
Name: rcu_fence.diff
Type: text/x-patch
Size: 4139 bytes
Desc: not available
URL: 

[PATCH 2/2] [RFC] reservation: add suppport for read-only access using rcu

[Maarten Lankhorst]

This adds 3 more functions to deal with rcu.

reservation_object_wait_timeout_rcu() will wait on all fences of the
reservation_object, without obtaining the ww_mutex.

reservation_object_test_signaled_rcu() will test if all fences of the
reservation_object are signaled without using the ww_mutex.

reservation_object_get_excl() is added because touching the fence_excl
member directly will trigger a sparse warning.

Signed-off-by: Maarten Lankhorst 
---
 drivers/base/dma-buf.c  |   46 +++--
 drivers/base/reservation.c  |  147 +--
 include/linux/fence.h   |   22 ++
 include/linux/reservation.h |   40 
 4 files changed, 224 insertions(+), 31 deletions(-)

diff --git a/drivers/base/dma-buf.c b/drivers/base/dma-buf.c
index d89a98d2c37b..fc2d7546b8b0 100644
--- a/drivers/base/dma-buf.c
+++ b/drivers/base/dma-buf.c
@@ -151,14 +151,22 @@ static unsigned int dma_buf_poll(struct file *file, 
poll_table *poll)
if (!events)
return 0;

-   ww_mutex_lock(&resv->lock, NULL);
+   rcu_read_lock();

-   fobj = resv->fence;
-   if (!fobj)
-   goto out;
+   fobj = rcu_dereference(resv->fence);
+   if (fobj) {
+   shared_count = ACCESS_ONCE(fobj->shared_count);
+   smp_mb(); /* shared_count needs transitivity wrt fence_excl */
+   } else
+   shared_count = 0;
+   fence_excl = rcu_dereference(resv->fence_excl);

-   shared_count = fobj->shared_count;
-   fence_excl = resv->fence_excl;
+   /*
+* This would have needed a smp_read_barrier_depends()
+* because shared_count needs to be read before shared[i], but
+* spin_lock_irq and spin_unlock_irq provide even stronger
+* guarantees.
+*/

if (fence_excl && (!(events & POLLOUT) || shared_count == 0)) {
struct dma_buf_poll_cb_t *dcb = &dmabuf->cb_excl;
@@ -176,14 +184,20 @@ static unsigned int dma_buf_poll(struct file *file, 
poll_table *poll)
spin_unlock_irq(&dmabuf->poll.lock);

if (events & pevents) {
-   if (!fence_add_callback(fence_excl, &dcb->cb,
+   if (!fence_get_rcu(fence_excl)) {
+   /* force a recheck */
+   events &= ~pevents;
+   dma_buf_poll_cb(NULL, &dcb->cb);
+   } else if (!fence_add_callback(fence_excl, &dcb->cb,
   dma_buf_poll_cb)) {
events &= ~pevents;
+   fence_put(fence_excl);
} else {
/*
 * No callback queued, wake up any additional
 * waiters.
 */
+   fence_put(fence_excl);
dma_buf_poll_cb(NULL, &dcb->cb);
}
}
@@ -205,13 +219,25 @@ static unsigned int dma_buf_poll(struct file *file, 
poll_table *poll)
goto out;

for (i = 0; i < shared_count; ++i) {
-   struct fence *fence = fobj->shared[i];
-
+   struct fence *fence = fence_get_rcu(fobj->shared[i]);
+   if (!fence) {
+   /*
+* fence refcount dropped to zero, this means
+* that fobj has been freed
+*
+* call dma_buf_poll_cb and force a recheck!
+*/
+   events &= ~POLLOUT;
+   dma_buf_poll_cb(NULL, &dcb->cb);
+   break;
+   }
if (!fence_add_callback(fence, &dcb->cb,
dma_buf_poll_cb)) {
+   fence_put(fence);
events &= ~POLLOUT;
break;
}
+   fence_put(fence);
}

/* No callback queued, wake up any additional waiters. */
@@ -220,7 +246,7 @@ static unsigned int dma_buf_poll(struct file *file, 
poll_table *poll)
}

 out:
-   ww_mutex_unlock(&resv->lock);
+   rcu_read_unlock();
return events;
 }

diff --git a/drivers/base/reservation.c b/drivers/base/reservation.c
index b82a5b630a8e..4cdce63140b8 100644
--- a/drivers/base/reservation.c
+++ b/drivers/base/reservation.c
@@ -87,9 +87,13 @@ reservation_object_add_shared_inplace(struct 
reservation_object *obj,
struct fence *old_fence = fobj->shared[i];

fence_get(fence);
+   /* for the