Re: [Mesa-dev] [RFC 0/1] Port dri2GetBuffersWithFormat/dri2GetBuffers to XCB

[gregory hainaut]

On Thu, 27 Apr 2017 17:11:30 +0900
Michel Dänzer  wrote:

> On 26/04/17 07:06 PM, Gregory Hainaut wrote:
> > On 4/26/17, Michel Dänzer  wrote:  
>  [...]  
>  [...]  
>  [...]  
> > 
> > I didn't test it (yet). But I think it is safe to call XCB from
> > various threads. However if one thread use XLIB, you're screwed (to
> > access the same display).
> >   
>  [...]  
>  [...]  
>  [...]  
> > I far from an expert so maybe I misunderstand some stuffs. So, as far
> > as I understand XLIB and XCB are mixed together. They likely share the
> > same queues.
> > 
> > Let's say you have an app that does Xlib operations on display (for
> > example XGetGeometry). As XInitThread wasn't called, you're in YOLO
> > mode.
> > 
> > glthread (gallium->DRI2) will do operation (GetBuffer) on the same
> > display through an XCB connection but it is still the same display.
> > XCB might lock it properly but Xlib call is lock-free.  
> 
> I was hoping the lower XCB layer could be used in a thread-safe way even
> if the higher libX11 layer isn't. But maybe that's just wishful thinking.

Hello Michel,

Yeah me too. Besides libX11 relies on (the thread-safe) XCB too.
 
> > What happen in my case is that XCB reply was corrupted (count was huge
> > which lead to memory bad access). My guess was that Xlib calls from
> > app were the culprit.  
> 
> It would be nice to get some certainty, e.g. via the valgrind
> helgrind/drd tools.
So I tried drd. Unfortunately I'm afraid that my testcase (PCSX2) is way 
too complex for this kind of tool.

However, I always got a huge value on the reply->count in GetBuffer. I
investigated it further. Reply->count was 94373760 which can be read
as 1440,1920 so my surface resolution. I think it is enough to prove that
xcb_dri2_get_buffers_with_format_reply is stealing the reply of XGetGeometry.

Note my 2nd kind of crash is XIOError due to a null reply on XGetGeometry.
which make sense based on the above behavior.

FWIW, the crash seem to be gone with this patch + my PCSX2-XCB patches.

However, I need to check DRI3 behavior when I resize the window. As you said
it could trigger buffer invalidation too.
 
> 
>  [...]  
>  [...]  
> > On one hand, I don't like crash neither but in other hand, it would be
> > nice to keep glthread for application that call XInitThread properly.  
> 
> We could check dpy->lock_fns and only enable glthread with DRI2 if it's
> non-NULL. If it's NULL and the environment variable mesa_glthread=true
> is set, we could print a warning to stderr explaining that glthread
> can't be enabled because the application didn't call XInitThreads.
> 
It is good idea. I will check, if I can manage to implement such a check.

Cheers,
Gregory
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [RFC 0/1] Port dri2GetBuffersWithFormat/dri2GetBuffers to XCB

[Michel Dänzer]

On 26/04/17 07:06 PM, Gregory Hainaut wrote:
> On 4/26/17, Michel Dänzer  wrote:
>> On 26/04/17 05:07 PM, Gregory Hainaut wrote:
>>> Following the discussion in "[PATCH v4 0/3] asynchronous pbo transfer with
>>> glthread"
>>>
>>> It will help apps that are ported to XCB.
>>
>> How so?
> 
> I didn't test it (yet). But I think it is safe to call XCB from
> various threads. However if one thread use XLIB, you're screwed (to
> access the same display).
> 
>>
>>> But Xlib (without XInitThread) apps will still crash when glthread is
>>> enabled on DRI2.
>>
>> Do the crashes provide information about where glthread is still calling
>> libX11 APIs?
> I far from an expert so maybe I misunderstand some stuffs. So, as far
> as I understand XLIB and XCB are mixed together. They likely share the
> same queues.
> 
> Let's say you have an app that does Xlib operations on display (for
> example XGetGeometry). As XInitThread wasn't called, you're in YOLO
> mode.
> 
> glthread (gallium->DRI2) will do operation (GetBuffer) on the same
> display through an XCB connection but it is still the same display.
> XCB might lock it properly but Xlib call is lock-free.

I was hoping the lower XCB layer could be used in a thread-safe way even
if the higher libX11 layer isn't. But maybe that's just wishful thinking.


> What happen in my case is that XCB reply was corrupted (count was huge
> which lead to memory bad access). My guess was that Xlib calls from
> app were the culprit.

It would be nice to get some certainty, e.g. via the valgrind
helgrind/drd tools.


>>> There are 3 remaining possibilities
>>> * Won't fix. DRI3/XCB is the future
>>> * Enable thread safe Xlib by default (which would make sense in multicore
>>> CPU era)
>>> * Track gl call that will use X11 API to force a sync
>>
>> Until either of the latter two happens, glthread should only be enabled
>> with DRI3.
> On one hand, I don't like crash neither but in other hand, it would be
> nice to keep glthread for application that call XInitThread properly.

We could check dpy->lock_fns and only enable glthread with DRI2 if it's
non-NULL. If it's NULL and the environment variable mesa_glthread=true
is set, we could print a warning to stderr explaining that glthread
can't be enabled because the application didn't call XInitThreads.


-- 
Earthling Michel Dänzer   |   http://www.amd.com
Libre software enthusiast | Mesa and X developer
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [RFC 0/1] Port dri2GetBuffersWithFormat/dri2GetBuffers to XCB

[Gregory Hainaut]

On 4/26/17, Michel Dänzer  wrote:
> On 26/04/17 05:07 PM, Gregory Hainaut wrote:
>> Following the discussion in "[PATCH v4 0/3] asynchronous pbo transfer with
>> glthread"
>>
>> It will help apps that are ported to XCB.
>
> How so?

I didn't test it (yet). But I think it is safe to call XCB from
various threads. However if one thread use XLIB, you're screwed (to
access the same display).

>
>> But Xlib (without XInitThread) apps will still crash when glthread is
>> enabled on DRI2.
>
> Do the crashes provide information about where glthread is still calling
> libX11 APIs?
I far from an expert so maybe I misunderstand some stuffs. So, as far
as I understand XLIB and XCB are mixed together. They likely share the
same queues.

Let's say you have an app that does Xlib operations on display (for
example XGetGeometry). As XInitThread wasn't called, you're in YOLO
mode.

glthread (gallium->DRI2) will do operation (GetBuffer) on the same
display through an XCB connection but it is still the same display.
XCB might lock it properly but Xlib call is lock-free.

What happen in my case is that XCB reply was corrupted (count was huge
which lead to memory bad access). My guess was that Xlib calls from
app were the culprit. I will test my XCB version of my app + this
patch. Hopefully, it would be crash free.

>> I only tested the patch on Nouveau
>>
>> There are 3 remaining possibilities
>> * Won't fix. DRI3/XCB is the future
>> * Enable thread safe Xlib by default (which would make sense in multicore
>> CPU era)
>> * Track gl call that will use X11 API to force a sync
>
> Until either of the latter two happens, glthread should only be enabled
> with DRI3.
On one hand, I don't like crash neither but in other hand, it would be
nice to keep glthread for application that call XInitThread properly.
At least, glthread isn't enabled by default.

>
> --
> Earthling Michel Dänzer   |   http://www.amd.com
> Libre software enthusiast | Mesa and X developer
>
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [RFC 0/1] Port dri2GetBuffersWithFormat/dri2GetBuffers to XCB

[Michel Dänzer]

On 26/04/17 05:07 PM, Gregory Hainaut wrote:
> Following the discussion in "[PATCH v4 0/3] asynchronous pbo transfer with 
> glthread"
> 
> It will help apps that are ported to XCB.

How so?


> But Xlib (without XInitThread) apps will still crash when glthread is
> enabled on DRI2.

Do the crashes provide information about where glthread is still calling
libX11 APIs?


> I only tested the patch on Nouveau
> 
> There are 3 remaining possibilities
> * Won't fix. DRI3/XCB is the future
> * Enable thread safe Xlib by default (which would make sense in multicore CPU 
> era)
> * Track gl call that will use X11 API to force a sync

Until either of the latter two happens, glthread should only be enabled
with DRI3.


-- 
Earthling Michel Dänzer   |   http://www.amd.com
Libre software enthusiast | Mesa and X developer
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

[Mesa-dev] [RFC 0/1] Port dri2GetBuffersWithFormat/dri2GetBuffers to XCB

[Gregory Hainaut]

Following the discussion in "[PATCH v4 0/3] asynchronous pbo transfer with 
glthread"

It will help apps that are ported to XCB. But Xlib (without XInitThread) apps
will still crash when glthread is enabled on DRI2.

I only tested the patch on Nouveau

There are 3 remaining possibilities
* Won't fix. DRI3/XCB is the future
* Enable thread safe Xlib by default (which would make sense in multicore CPU 
era)
* Track gl call that will use X11 API to force a sync

Note: I don't know if xcb_dri2_get_buffers_buffers can return a NULL
so I added a check in doubt.

Best Regards,

Gregory Hainaut (1):
  glx: port dri2GetBuffers/dri2GetBuffersWithFormat to XCB

 src/glx/dri2.c | 118 -
 src/glx/dri2.h |  25 
 src/glx/dri2_glx.c |  64 +
 3 files changed, 47 insertions(+), 160 deletions(-)

-- 
2.11.0

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev