Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Dave Airlie]

On 13 February 2017 at 19:07, Michel Dänzer  wrote:
> On 11/02/17 06:36 AM, gregory hainaut wrote:
>>
>> On GLX side, I can explain some crashes (didn't debug all neither fail)
>> and I suspect there are related to the previous report.
>>
>> At the context creation a glthread dispatcher is created and the *TLS*
>> variable u_current_table is set with the new gl marshal function.
>> If deprecated non-VBO functions are detected
>> (glBegin/glVertexPointer/... and others horror from the previous
>> century), the code will disable glthread.
>>
>> 1/ thread will be deleted
>> 2/ u_current_table will be set to the original value
>>
>> However u_current_table is local to a thread. So a single thread will
>> get the correct dispatcher table. Remaining threads will keep the
>> previous marshal thread. If a glthread function is called without the
>> thread behind, then BOOM.
>
> If I understand correctly, this scenario can only happen if the
> application takes advantage of the GLX_MESA_multithread_makecurrent
> extension. I wonder if we shouldn't just stop advertising that extension
> with glthread enabled.
>
> Actually, I wonder how well GLX_MESA_multithread_makecurrent works
> regardless of glthread. I suspect the assumption of a 1:1 correspondence
> between threads and contexts is quite widespread in the Mesa code. Case
> in point, the piglit test glx@glx-multithread-texture has been failing
> forever with radeonsi, though it passes with llvmpipe. Are there any
> real world applications which require GLX_MESA_multithread_makecurrent,
> or at least get a significant benefit from it?

I think glvnd also wants to disable this extension, cairo was the
thing that used it
but who knows what hacks have gone into other stuff.

Dave.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[gregory hainaut]

On Mon, 13 Feb 2017 17:04:01 +0100
Marek Olšák  wrote:

> On Mon, Feb 13, 2017 at 4:14 PM, Gregory Hainaut
>  wrote:
> > If I remember correctly I got something like 4K-8K fails (total is 40K
> > tests)  when I broke the GL1 restore dispatch function. So it is
> > around 10%-20%. Maybe it was only the test below 3.1. I was afraid
> > that various modern extension could still be tested with some GL1
> > drawing.
> 
> The number of tests run doesn't necessarily correspond to the amount
> of test coverage. 10 tests doing different things can be more useful
> than 1 tests doing the same thing.
> 
> Marek

Fair point.

As a side note, I tested both glxgear and glmark2 which are now crash-free :)
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Mon, Feb 13, 2017 at 4:14 PM, Gregory Hainaut
 wrote:
> If I remember correctly I got something like 4K-8K fails (total is 40K
> tests)  when I broke the GL1 restore dispatch function. So it is
> around 10%-20%. Maybe it was only the test below 3.1. I was afraid
> that various modern extension could still be tested with some GL1
> drawing.

The number of tests run doesn't necessarily correspond to the amount
of test coverage. 10 tests doing different things can be more useful
than 1 tests doing the same thing.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Gregory Hainaut]

If I remember correctly I got something like 4K-8K fails (total is 40K
tests)  when I broke the GL1 restore dispatch function. So it is
around 10%-20%. Maybe it was only the test below 3.1. I was afraid
that various modern extension could still be tested with some GL1
drawing.

On 2/13/17, Marek Olšák  wrote:
> On Mon, Feb 13, 2017 at 3:56 PM, Gregory Hainaut
>  wrote:
>> Hum, I need to check it. I think I disabled EGL because I didn't have
>> latest waffle on my Debian (and Nvidia didn't support fully EGL back
>> then).
>>
>> However, I forgot to say that a fair number of piglit tests are based
>> on GL1 direct call. (typically to draw a rectangle without a VBO). All
>
> I'd disagree. All GL 3.1 tests and above have to use VBOs. GL core requires
> it.
>
> Marek
>
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Mon, Feb 13, 2017 at 3:56 PM, Gregory Hainaut
 wrote:
> Hum, I need to check it. I think I disabled EGL because I didn't have
> latest waffle on my Debian (and Nvidia didn't support fully EGL back
> then).
>
> However, I forgot to say that a fair number of piglit tests are based
> on GL1 direct call. (typically to draw a rectangle without a VBO). All

I'd disagree. All GL 3.1 tests and above have to use VBOs. GL core requires it.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Gregory Hainaut]

Hum, I need to check it. I think I disabled EGL because I didn't have
latest waffle on my Debian (and Nvidia didn't support fully EGL back
then).

However, I forgot to say that a fair number of piglit tests are based
on GL1 direct call. (typically to draw a rectangle without a VBO). All
those tests will disable glthread in the middle. In the current state,
piglit is far from ideal to test glthread. Maybe CTS could help to
increase the coverage. I will give it a shot.

Gregory

On 2/13/17, Marek Olšák  wrote:
> On Mon, Feb 13, 2017 at 11:10 AM, Gregory Hainaut
>  wrote:
>>> On 11/02/17 06:36 AM, gregory hainaut wrote:
>>> >
>>> > On GLX side, I can explain some crashes (didn't debug all neither
>>> > fail)
>>> > and I suspect there are related to the previous report.
>>> >
>>> > At the context creation a glthread dispatcher is created and the *TLS*
>>> > variable u_current_table is set with the new gl marshal function.
>>> > If deprecated non-VBO functions are detected
>>> > (glBegin/glVertexPointer/... and others horror from the previous
>>> > century), the code will disable glthread.
>>> >
>>> > 1/ thread will be deleted
>>> > 2/ u_current_table will be set to the original value
>>> >
>>> > However u_current_table is local to a thread. So a single thread will
>>> > get the correct dispatcher table. Remaining threads will keep the
>>> > previous marshal thread. If a glthread function is called without the
>>> > thread behind, then BOOM.
>>>
>>> If I understand correctly, this scenario can only happen if the
>>> application takes advantage of the GLX_MESA_multithread_makecurrent
>>> extension. I wonder if we shouldn't just stop advertising that extension
>>> with glthread enabled.
>>>
>>> Actually, I wonder how well GLX_MESA_multithread_makecurrent works
>>> regardless of glthread. I suspect the assumption of a 1:1 correspondence
>>> between threads and contexts is quite widespread in the Mesa code. Case
>>> in point, the piglit test glx at glx-multithread-texture has been
>>> failing
>>> forever with radeonsi, though it passes with llvmpipe. Are there any
>>> real world applications which require GLX_MESA_multithread_makecurrent,
>>> or at least get a significant benefit from it?
>>
>>
>> Hello,
>>
>>
>> Yes you're right. I just found out yesterday that it was related to
>> GLX_MESA_multithread_makecurrent.
>> I did a small patch to not kill glthread when direct GL1 code is used. It
>> avoid the crash but it is far
>> from robust. You could imagine situation where 1 thread uses dispatcher
>> function and another
>> use std serial call (of course without any sync of glthread). Now if you
>> use
>> only modern GL
>> (that will always use glthread), it might work. Note: Piglit is working
>> as
>> both thread will restore
>> the standard dispatch function.
>>
>> Feral said that they tried the extension but a gl dispatcher was faster.
>> Googling a bit I found
>> Eric's initial mail. So it seems to be used. Extension isn't published on
>> Kronos, so usage likely
>> remain confidential. Potentially glthread will be as fast as GLX_MT on
>> cairo. But you're right,
>> I don't see the point to support both at the same time.
>>>
>>> I'm happy now with the implementation of this extension, I've got
>>> tests for most of the issues mentioned (the one that's really missing
>>> is the bind-to-new-drawable one, but I don't anticipate issues with
>>> it), and cairo-gl is getting a ~35% performance improvement from it
>>> and is passing cairo's threading tests other than the one that fails
>>> due to non-threading-related issuse.
>>
>>
>> Actually glx-multithread-texture is pass with glthread (nouveau)
>>
>>
>> As a quick summary:
>> * there are now only 2 minors fail on piglit with my latest patches
>> (sent
>> to Marek)
>> * I have a pending patch to allow asynchronous PBO transfer
>> * Now that piglit is crash free I will give a try to both glxgear and
>> glmark. Hopefully they will be both good.
>
> Note that if you run piglit with egl/drm, it might not have glthread
> enabled.
>
> Marek
>
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Mon, Feb 13, 2017 at 11:10 AM, Gregory Hainaut
 wrote:
>> On 11/02/17 06:36 AM, gregory hainaut wrote:
>> >
>> > On GLX side, I can explain some crashes (didn't debug all neither fail)
>> > and I suspect there are related to the previous report.
>> >
>> > At the context creation a glthread dispatcher is created and the *TLS*
>> > variable u_current_table is set with the new gl marshal function.
>> > If deprecated non-VBO functions are detected
>> > (glBegin/glVertexPointer/... and others horror from the previous
>> > century), the code will disable glthread.
>> >
>> > 1/ thread will be deleted
>> > 2/ u_current_table will be set to the original value
>> >
>> > However u_current_table is local to a thread. So a single thread will
>> > get the correct dispatcher table. Remaining threads will keep the
>> > previous marshal thread. If a glthread function is called without the
>> > thread behind, then BOOM.
>>
>> If I understand correctly, this scenario can only happen if the
>> application takes advantage of the GLX_MESA_multithread_makecurrent
>> extension. I wonder if we shouldn't just stop advertising that extension
>> with glthread enabled.
>>
>> Actually, I wonder how well GLX_MESA_multithread_makecurrent works
>> regardless of glthread. I suspect the assumption of a 1:1 correspondence
>> between threads and contexts is quite widespread in the Mesa code. Case
>> in point, the piglit test glx at glx-multithread-texture has been failing
>> forever with radeonsi, though it passes with llvmpipe. Are there any
>> real world applications which require GLX_MESA_multithread_makecurrent,
>> or at least get a significant benefit from it?
>
>
> Hello,
>
>
> Yes you're right. I just found out yesterday that it was related to
> GLX_MESA_multithread_makecurrent.
> I did a small patch to not kill glthread when direct GL1 code is used. It
> avoid the crash but it is far
> from robust. You could imagine situation where 1 thread uses dispatcher
> function and another
> use std serial call (of course without any sync of glthread). Now if you use
> only modern GL
> (that will always use glthread), it might work. Note: Piglit is working as
> both thread will restore
> the standard dispatch function.
>
> Feral said that they tried the extension but a gl dispatcher was faster.
> Googling a bit I found
> Eric's initial mail. So it seems to be used. Extension isn't published on
> Kronos, so usage likely
> remain confidential. Potentially glthread will be as fast as GLX_MT on
> cairo. But you're right,
> I don't see the point to support both at the same time.
>>
>> I'm happy now with the implementation of this extension, I've got
>> tests for most of the issues mentioned (the one that's really missing
>> is the bind-to-new-drawable one, but I don't anticipate issues with
>> it), and cairo-gl is getting a ~35% performance improvement from it
>> and is passing cairo's threading tests other than the one that fails
>> due to non-threading-related issuse.
>
>
> Actually glx-multithread-texture is pass with glthread (nouveau)
>
>
> As a quick summary:
> * there are now only 2 minors fail on piglit with my latest patches  (sent
> to Marek)
> * I have a pending patch to allow asynchronous PBO transfer
> * Now that piglit is crash free I will give a try to both glxgear and
> glmark. Hopefully they will be both good.

Note that if you run piglit with egl/drm, it might not have glthread enabled.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Gregory Hainaut]

>
> On 11/02/17 06:36 AM, gregory hainaut wrote:
> > >* On GLX side, I can explain some crashes (didn't debug all neither fail)
> *>* and I suspect there are related to the previous report.
> *> >* At the context creation a glthread dispatcher is created and the *TLS*
> *>* variable u_current_table is set with the new gl marshal function.
> *>* If deprecated non-VBO functions are detected
> *>* (glBegin/glVertexPointer/... and others horror from the previous
> *>* century), the code will disable glthread.
> *> >* 1/ thread will be deleted
> *>* 2/ u_current_table will be set to the original value
> *> >* However u_current_table is local to a thread. So a single thread will
> *>* get the correct dispatcher table. Remaining threads will keep the
> *>* previous marshal thread. If a glthread function is called without the
> *>* thread behind, then BOOM.
> *
> If I understand correctly, this scenario can only happen if the
> application takes advantage of the GLX_MESA_multithread_makecurrent
> extension. I wonder if we shouldn't just stop advertising that extension
> with glthread enabled.
>
> Actually, I wonder how well GLX_MESA_multithread_makecurrent works
> regardless of glthread. I suspect the assumption of a 1:1 correspondence
> between threads and contexts is quite widespread in the Mesa code. Case
> in point, the piglit test glx at glx-multithread-texture 
>  has been failing
> forever with radeonsi, though it passes with llvmpipe. Are there any
> real world applications which require GLX_MESA_multithread_makecurrent,
> or at least get a significant benefit from it?
>
>
Hello,


Yes you're right. I just found out yesterday that it was related to
GLX_MESA_multithread_makecurrent.
I did a small patch to not kill glthread when direct GL1 code is used. It
avoid the crash but it is far
from robust. You could imagine situation where 1 thread uses dispatcher
function and another
use std serial call (of course without any sync of glthread). Now if you
use only modern GL
(that will always use glthread), it might work. Note: Piglit is working as
both thread will restore
the standard dispatch function.

Feral said that they tried the extension but a gl dispatcher was faster.
Googling a bit I found
Eric's initial mail. So it seems to be used. Extension isn't published on
Kronos, so usage likely
remain confidential. Potentially glthread will be as fast as GLX_MT on
cairo. But you're right,
I don't see the point to support both at the same time.

> I'm happy now with the implementation of this extension, I've got
> tests for most of the issues mentioned (the one that's really missing
> is the bind-to-new-drawable one, but I don't anticipate issues with
> it), and cairo-gl is getting a ~35% performance improvement from it
> and is passing cairo's threading tests other than the one that fails
> due to non-threading-related issuse.
>
>
Actually glx-multithread-texture is pass with glthread (nouveau)


As a quick summary:
* there are now only 2 minors fail on piglit with my latest patches  (sent
to Marek)
* I have a pending patch to allow asynchronous PBO transfer
* Now that piglit is crash free I will give a try to both glxgear and
glmark. Hopefully they will be both good.

Gregory
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Michel Dänzer]

On 11/02/17 06:36 AM, gregory hainaut wrote:
> 
> On GLX side, I can explain some crashes (didn't debug all neither fail)
> and I suspect there are related to the previous report.
> 
> At the context creation a glthread dispatcher is created and the *TLS*
> variable u_current_table is set with the new gl marshal function.
> If deprecated non-VBO functions are detected
> (glBegin/glVertexPointer/... and others horror from the previous
> century), the code will disable glthread.
> 
> 1/ thread will be deleted
> 2/ u_current_table will be set to the original value
> 
> However u_current_table is local to a thread. So a single thread will
> get the correct dispatcher table. Remaining threads will keep the
> previous marshal thread. If a glthread function is called without the
> thread behind, then BOOM.

If I understand correctly, this scenario can only happen if the
application takes advantage of the GLX_MESA_multithread_makecurrent
extension. I wonder if we shouldn't just stop advertising that extension
with glthread enabled.

Actually, I wonder how well GLX_MESA_multithread_makecurrent works
regardless of glthread. I suspect the assumption of a 1:1 correspondence
between threads and contexts is quite widespread in the Mesa code. Case
in point, the piglit test glx@glx-multithread-texture has been failing
forever with radeonsi, though it passes with llvmpipe. Are there any
real world applications which require GLX_MESA_multithread_makecurrent,
or at least get a significant benefit from it?


-- 
Earthling Michel Dänzer   |   http://www.amd.com
Libre software enthusiast | Mesa and X developer
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Timothy Arceri]

Just to clarify my email, I wasn't saying shader-cache should land 
before this, or that we should prioritize one over the other. I was 
simply venting frustration that I had to keep it rebased out of tree and 
keep it passing piglit when it seems I didn't need too. This is nothing 
against this series or Marek.


Shader-cache has been in a useful state for at least the past 6 months 
it would have been nice to get it in and have real users testing. I 
agree that this is probably the only way to get this series tested and 
up to scratch also. I do think we should try to at least get piglit 
mostly working before merging (looks like Gregory has made some progress 
with this) even if piglit is unlikely to be the best test case for this, 
it seems like it a low bar to have to clear.


Tim

On 11/02/17 00:29, Marek Olšák wrote:
I never disagreed with you about shader cache and I never said that 
glthread would be merged first. I'm just describing the situation and 
recent news.


Marek


On Feb 10, 2017 2:15 PM, "Edward O'Callaghan" 
> wrote:


Wait what? You just side stepped everything I just said with
regards to
prioritizing the volume of work that went into shader caching over gl
dispatch with essentially `gl dispatch is really great and our users
friend it super useful now`.

That's fine, I'm not ageist it, maybe you misinterpreted me? However
that is nothing to do with what I was saying, as to reiterate my point
was precise and finely scoped; We should get shader caching i's
and t's
dotted and crossed and help Timothy get though the last 5% there.

Kind Regards,
Edward.

On 02/11/2017 12:01 AM, Marek Olšák wrote:
> FYI. Civilization VI is another game that works with and
benefits from
> glthread. The game was just released. Even Nvidia is CPU-bound on
> highest details and can't reach 45 fps with full hd. People
wanting to
> play Civ VI with decent frame rate will want glthread. I don't think
> they care too much about our the community processes, so I
expect there
> will be quite a few users using out-of-tree builds of Mesa.
>
> Also, Pierre-Loup from Valve said on IRC yesterday that they are
> probably gonna ship glthread and make their own whitelist,
regardless of
> the outcome of this discussion. It would be preferable to have that
> whitelist in master too, but that may be difficult if we can't
merge it.
>
> If distributions and vendors start shipping glthread, we might
as well
> merge it, because at that point there is no advantage in keeping
this
> out of tree if it forces users to use out of tree builds. We'll
get bug
> reports regardless.
>
> Also Eero, I don't care about glmark at this very moment. It's
not like
> I'm merging this today, so it doesn't matter. Maybe it will
matter next
> week or next month. We'll likely not support glmark anyway, so
the fix
> will most likely be disabling multithreading on the fly than
trying to
> fix the crash.
>
> Marek
>
>
> On Feb 10, 2017 12:58 PM, "Edward O'Callaghan"
> 
>> wrote:
>
>
>
> On 02/10/2017 10:50 PM, Marek Olšák wrote:
> > On Fri, Feb 10, 2017 at 12:48 PM, Edward O'Callaghan
> > 
>>
> wrote:
> >>
> >>
> >> On 02/10/2017 10:36 PM, Marek Olšák wrote:
> >>> On Fri, Feb 10, 2017 at 12:26 PM, Edward O'Callaghan
> >>> 
> >> wrote:
> 
> 
>  On 02/08/2017 09:13 AM, Timothy Arceri wrote:
> > On Tue, 2017-02-07 at 10:56 +0100, Marek Olšák wrote:
> >> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>  >> g> wrote:
> >>> On Monday, February 6, 2017 8:54:40 PM PST Marek
Olšák wrote:
>  On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand
> 
>
>  > wrote:
> > FYI glmark2 segfaults with mesa_glthread=true.
Expected that
> > some programs
> > will segfault?
> 
>  Yes, even segfaults are expected with
mesa_glthread=true.
> 
>  Marek
> >>>
> >>> Would it make sense to be crash-free 

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[gregory hainaut]

Hello,

I ran piglit (glthread false vs true) on the Marek branch and Nouveau driver.
Of course we can still debate, if piglit is good enough to detect multi 
threading issue.

I spot a small typo that generate ~400 fails (*mat2x4* uniform got 6 scalar 
instead of 8).

piglit status with the typo patch : warn +2/ fail + 4/ crash +5 

All regressions are on GLX tests but glsl-uniform-out-of-bounds. The
latter reports a wrong error INVALID_VALUE (there is an extra glthread check
because count is too big) instead of INVALID_OPERATION (not an array
uniform must have a count of 1). I'm not sure it worth a fix.


On GLX side, I can explain some crashes (didn't debug all neither fail)
and I suspect there are related to the previous report.

At the context creation a glthread dispatcher is created and the *TLS*
variable u_current_table is set with the new gl marshal function.
If deprecated non-VBO functions are detected
(glBegin/glVertexPointer/... and others horror from the previous
century), the code will disable glthread.

1/ thread will be deleted
2/ u_current_table will be set to the original value

However u_current_table is local to a thread. So a single thread will
get the correct dispatcher table. Remaining threads will keep the
previous marshal thread. If a glthread function is called without the
thread behind, then BOOM. I don't know how to fix it. However, is it
really a necessity to care for such "old" application.

Note: display list update also the table, I'm not sure of the impact. I
don't know if piglit test display list


Unfortunately it is way too late (at least for closed application, open
source can easily be updated), but one could imagine a context flag to
select a gl dispatcher. IMHO, the application is the best judge to
enable it. Automatic detection on the driver is quite complicated. It is way 
cheaper
to add an env value in the .desktop file.


IMHO, it would be better to just drop the "automatic removal" of
glthread. Old application will be slower if an user set wrongly an env
variable which is more reasonable than a crash.


Cheers,
Gregory
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

I never disagreed with you about shader cache and I never said that
glthread would be merged first. I'm just describing the situation and
recent news.

Marek


On Feb 10, 2017 2:15 PM, "Edward O'Callaghan" 
wrote:

Wait what? You just side stepped everything I just said with regards to
prioritizing the volume of work that went into shader caching over gl
dispatch with essentially `gl dispatch is really great and our users
friend it super useful now`.

That's fine, I'm not ageist it, maybe you misinterpreted me? However
that is nothing to do with what I was saying, as to reiterate my point
was precise and finely scoped; We should get shader caching i's and t's
dotted and crossed and help Timothy get though the last 5% there.

Kind Regards,
Edward.

On 02/11/2017 12:01 AM, Marek Olšák wrote:
> FYI. Civilization VI is another game that works with and benefits from
> glthread. The game was just released. Even Nvidia is CPU-bound on
> highest details and can't reach 45 fps with full hd. People wanting to
> play Civ VI with decent frame rate will want glthread. I don't think
> they care too much about our the community processes, so I expect there
> will be quite a few users using out-of-tree builds of Mesa.
>
> Also, Pierre-Loup from Valve said on IRC yesterday that they are
> probably gonna ship glthread and make their own whitelist, regardless of
> the outcome of this discussion. It would be preferable to have that
> whitelist in master too, but that may be difficult if we can't merge it.
>
> If distributions and vendors start shipping glthread, we might as well
> merge it, because at that point there is no advantage in keeping this
> out of tree if it forces users to use out of tree builds. We'll get bug
> reports regardless.
>
> Also Eero, I don't care about glmark at this very moment. It's not like
> I'm merging this today, so it doesn't matter. Maybe it will matter next
> week or next month. We'll likely not support glmark anyway, so the fix
> will most likely be disabling multithreading on the fly than trying to
> fix the crash.
>
> Marek
>
>
> On Feb 10, 2017 12:58 PM, "Edward O'Callaghan"
> > wrote:
>
>
>
> On 02/10/2017 10:50 PM, Marek Olšák wrote:
> > On Fri, Feb 10, 2017 at 12:48 PM, Edward O'Callaghan
> > >
> wrote:
> >>
> >>
> >> On 02/10/2017 10:36 PM, Marek Olšák wrote:
> >>> On Fri, Feb 10, 2017 at 12:26 PM, Edward O'Callaghan
> >>>  > wrote:
> 
> 
>  On 02/08/2017 09:13 AM, Timothy Arceri wrote:
> > On Tue, 2017-02-07 at 10:56 +0100, Marek Olšák wrote:
> >> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>  >> g> wrote:
> >>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>  On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand
> 
> > wrote:
> > FYI glmark2 segfaults with mesa_glthread=true. Expected that
> > some programs
> > will segfault?
> 
>  Yes, even segfaults are expected with mesa_glthread=true.
> 
>  Marek
> >>>
> >>> Would it make sense to be crash-free or even regression-free
> on at
> >>> least Piglit, before merging?  (Or are we there already?)
> >>
> >> It's not necessary. glthread is disabled by default. Nobody has
> >> tested
> >> piglit with glthread. That will follow after it's been merged,
or
> >> never if it's never merged.
> >
> > I've been trying to land shader-cache patches that actually do
> pass
> > piglit for over a year with the same reasoning that it will be
> disable
> > by default and can only be improved with testing I can't
> possibly do on
> > my own.
> >
> > Although I have no objections to this being merged I'll be
> extremely
> > frustrated if this is allowed to be merged known to not even
pass
> > piglit while I've wasted countless hours rebasing shader cache
> over
> > many months.
> >
> 
>  Regardless of all the chatter on this thread in and around GL
> dispatch,
>  which I agree poses significant challenges to get into
> something 'ideal'
>  - which is hard to define for something like this..
> 
>  I think Timothy makes a really fair and just case here; in
> that, could
>  we perhaps prioritize getting the shader cache stuff in before
>  attempting GL dispatch? I think this both morally and
> technically the
>  right thing to do in my humble opinion.
> >>>
> >>> There is a small 

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Edward O'Callaghan]

Wait what? You just side stepped everything I just said with regards to
prioritizing the volume of work that went into shader caching over gl
dispatch with essentially `gl dispatch is really great and our users
friend it super useful now`.

That's fine, I'm not ageist it, maybe you misinterpreted me? However
that is nothing to do with what I was saying, as to reiterate my point
was precise and finely scoped; We should get shader caching i's and t's
dotted and crossed and help Timothy get though the last 5% there.

Kind Regards,
Edward.

On 02/11/2017 12:01 AM, Marek Olšák wrote:
> FYI. Civilization VI is another game that works with and benefits from
> glthread. The game was just released. Even Nvidia is CPU-bound on
> highest details and can't reach 45 fps with full hd. People wanting to
> play Civ VI with decent frame rate will want glthread. I don't think
> they care too much about our the community processes, so I expect there
> will be quite a few users using out-of-tree builds of Mesa.
> 
> Also, Pierre-Loup from Valve said on IRC yesterday that they are
> probably gonna ship glthread and make their own whitelist, regardless of
> the outcome of this discussion. It would be preferable to have that
> whitelist in master too, but that may be difficult if we can't merge it.
> 
> If distributions and vendors start shipping glthread, we might as well
> merge it, because at that point there is no advantage in keeping this
> out of tree if it forces users to use out of tree builds. We'll get bug
> reports regardless.
> 
> Also Eero, I don't care about glmark at this very moment. It's not like
> I'm merging this today, so it doesn't matter. Maybe it will matter next
> week or next month. We'll likely not support glmark anyway, so the fix
> will most likely be disabling multithreading on the fly than trying to
> fix the crash.
> 
> Marek
> 
> 
> On Feb 10, 2017 12:58 PM, "Edward O'Callaghan"
> > wrote:
> 
> 
> 
> On 02/10/2017 10:50 PM, Marek Olšák wrote:
> > On Fri, Feb 10, 2017 at 12:48 PM, Edward O'Callaghan
> > >
> wrote:
> >>
> >>
> >> On 02/10/2017 10:36 PM, Marek Olšák wrote:
> >>> On Fri, Feb 10, 2017 at 12:26 PM, Edward O'Callaghan
> >>>  > wrote:
> 
> 
>  On 02/08/2017 09:13 AM, Timothy Arceri wrote:
> > On Tue, 2017-02-07 at 10:56 +0100, Marek Olšák wrote:
> >> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>  >> g> wrote:
> >>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>  On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand
> 
> > wrote:
> > FYI glmark2 segfaults with mesa_glthread=true. Expected that
> > some programs
> > will segfault?
> 
>  Yes, even segfaults are expected with mesa_glthread=true.
> 
>  Marek
> >>>
> >>> Would it make sense to be crash-free or even regression-free
> on at
> >>> least Piglit, before merging?  (Or are we there already?)
> >>
> >> It's not necessary. glthread is disabled by default. Nobody has
> >> tested
> >> piglit with glthread. That will follow after it's been merged, or
> >> never if it's never merged.
> >
> > I've been trying to land shader-cache patches that actually do
> pass
> > piglit for over a year with the same reasoning that it will be
> disable
> > by default and can only be improved with testing I can't
> possibly do on
> > my own.
> >
> > Although I have no objections to this being merged I'll be
> extremely
> > frustrated if this is allowed to be merged known to not even pass
> > piglit while I've wasted countless hours rebasing shader cache
> over
> > many months.
> >
> 
>  Regardless of all the chatter on this thread in and around GL
> dispatch,
>  which I agree poses significant challenges to get into
> something 'ideal'
>  - which is hard to define for something like this..
> 
>  I think Timothy makes a really fair and just case here; in
> that, could
>  we perhaps prioritize getting the shader cache stuff in before
>  attempting GL dispatch? I think this both morally and
> technically the
>  right thing to do in my humble opinion.
> >>>
> >>> There is a small difference. The shader cache is expected to be
> >>> enabled by default, so there is a certain level of quality required.
> >>
> >> Hey Marek,
> >>
> >> I am under the impression that it being enabled by default isn't
> 

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

FYI. Civilization VI is another game that works with and benefits from
glthread. The game was just released. Even Nvidia is CPU-bound on highest
details and can't reach 45 fps with full hd. People wanting to play Civ VI
with decent frame rate will want glthread. I don't think they care too much
about our the community processes, so I expect there will be quite a few
users using out-of-tree builds of Mesa.

Also, Pierre-Loup from Valve said on IRC yesterday that they are probably
gonna ship glthread and make their own whitelist, regardless of the outcome
of this discussion. It would be preferable to have that whitelist in master
too, but that may be difficult if we can't merge it.

If distributions and vendors start shipping glthread, we might as well
merge it, because at that point there is no advantage in keeping this out
of tree if it forces users to use out of tree builds. We'll get bug reports
regardless.

Also Eero, I don't care about glmark at this very moment. It's not like I'm
merging this today, so it doesn't matter. Maybe it will matter next week or
next month. We'll likely not support glmark anyway, so the fix will most
likely be disabling multithreading on the fly than trying to fix the crash.

Marek


On Feb 10, 2017 12:58 PM, "Edward O'Callaghan" 
wrote:

>
>
> On 02/10/2017 10:50 PM, Marek Olšák wrote:
> > On Fri, Feb 10, 2017 at 12:48 PM, Edward O'Callaghan
> >  wrote:
> >>
> >>
> >> On 02/10/2017 10:36 PM, Marek Olšák wrote:
> >>> On Fri, Feb 10, 2017 at 12:26 PM, Edward O'Callaghan
> >>>  wrote:
> 
> 
>  On 02/08/2017 09:13 AM, Timothy Arceri wrote:
> > On Tue, 2017-02-07 at 10:56 +0100, Marek Olšák wrote:
> >> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>  >> g> wrote:
> >>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>  On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand  > wrote:
> > FYI glmark2 segfaults with mesa_glthread=true. Expected that
> > some programs
> > will segfault?
> 
>  Yes, even segfaults are expected with mesa_glthread=true.
> 
>  Marek
> >>>
> >>> Would it make sense to be crash-free or even regression-free on at
> >>> least Piglit, before merging?  (Or are we there already?)
> >>
> >> It's not necessary. glthread is disabled by default. Nobody has
> >> tested
> >> piglit with glthread. That will follow after it's been merged, or
> >> never if it's never merged.
> >
> > I've been trying to land shader-cache patches that actually do pass
> > piglit for over a year with the same reasoning that it will be
> disable
> > by default and can only be improved with testing I can't possibly do
> on
> > my own.
> >
> > Although I have no objections to this being merged I'll be extremely
> > frustrated if this is allowed to be merged known to not even pass
> > piglit while I've wasted countless hours rebasing shader cache over
> > many months.
> >
> 
>  Regardless of all the chatter on this thread in and around GL
> dispatch,
>  which I agree poses significant challenges to get into something
> 'ideal'
>  - which is hard to define for something like this..
> 
>  I think Timothy makes a really fair and just case here; in that, could
>  we perhaps prioritize getting the shader cache stuff in before
>  attempting GL dispatch? I think this both morally and technically the
>  right thing to do in my humble opinion.
> >>>
> >>> There is a small difference. The shader cache is expected to be
> >>> enabled by default, so there is a certain level of quality required.
> >>
> >> Hey Marek,
> >>
> >> I am under the impression that it being enabled by default isn't a hard
> >> requirement for it to be merged. Maybe Timothy can weigh in on it when
> >> he is online?
> >
> > There is no official hard requirement. Everything is a judgement call
> > based on circumstances.
>
> Yes, OK, I agree; So why assert the above response then? Who is
> expecting it to be enabled by default? To reiterate I believe Timothy
> would like it merged first and foremost, then perhaps enable it by
> default if that is OK with everyone. I didn't see anywhere he expected
> it to be on by default. However we should wait for his response on that.
>
> Regards,
> Edward.
>
> >
> > Marek
> >
>
>
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Eero Tamminen]

Hi,

On 10.02.2017 13:00, Marek Olšák wrote:

The glmark2 issue is not important, because glthread will never
support glmark2. (because glmark2 is a SwapBuffers benchmark like
glxgears)


Whether some test is SwapBuffers benchmark or not, or whether it 
benefits from glthread, is irrelevant.  What is relevant is:

* Does glthread cause test to crash
* Is the crash fault of glthread

I.e. is glthread broken.  If glthread is broken, performance numbers 
you've provided for other use-cases cannot be trusted, regardless of how 
promising they look.


(Without performance numbers that can be trusted to prove (current / 
eventual) working version of glthread to improve performance, there's no 
reason why the code would get merged.  I've seen a lot of examples of 
initially promising optimizations that weren't anymore when they had 
been fixed to work correctly.)



Note that while glmark2 test-suite tests with their default settings are 
indeed SwapBuffers tests (except for "terrain" test on weaker Intel 
GPUs), you're free to use some other setting [1].


For example these should be GPU (FS) limited:
glmark2 -b loop:fragment-steps=4096 --fullscreen
glmark2 -b conditionals:fragment-steps=1024:vertex-steps=4096
glmark2 -b function:fragment-steps=4096

Last one gets stack overflow in Mesa, at least on Intel, maybe gallium 
fares better.


(Moving things away from main thread like glthread does could be an 
issue, as on linux only main thread stack grows on-demand, other threads 
have fixed stack sizes, nowadays by default 8MB.)



Btw.  When Mesa gets glvnd support, will glthread need also updates?


- Eero

[1] Or tell it to use your own, more complex 3DS models and larger 
textures, or even replace the included shaders with something more 
demanding.  If somebody would add an option with which you can specify 
viewport matrix into which tests are rendered before doing buffer swap, 
even with the default options you could get tests GPU limited.  I'm sure 
upstream glmark2 would be happen to receive such changes.



Marek

On Fri, Feb 10, 2017 at 11:32 AM, Eero Tamminen
 wrote:

Hi,

On 09.02.2017 20:27, Ernst Sjöstrand wrote:


2017-02-09 18:06 GMT+01:00 Eero Tamminen 

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Edward O'Callaghan]

On 02/10/2017 10:50 PM, Marek Olšák wrote:
> On Fri, Feb 10, 2017 at 12:48 PM, Edward O'Callaghan
>  wrote:
>>
>>
>> On 02/10/2017 10:36 PM, Marek Olšák wrote:
>>> On Fri, Feb 10, 2017 at 12:26 PM, Edward O'Callaghan
>>>  wrote:


 On 02/08/2017 09:13 AM, Timothy Arceri wrote:
> On Tue, 2017-02-07 at 10:56 +0100, Marek Olšák wrote:
>> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke > g> wrote:
>>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
 On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand  wrote:
> FYI glmark2 segfaults with mesa_glthread=true. Expected that
> some programs
> will segfault?

 Yes, even segfaults are expected with mesa_glthread=true.

 Marek
>>>
>>> Would it make sense to be crash-free or even regression-free on at
>>> least Piglit, before merging?  (Or are we there already?)
>>
>> It's not necessary. glthread is disabled by default. Nobody has
>> tested
>> piglit with glthread. That will follow after it's been merged, or
>> never if it's never merged.
>
> I've been trying to land shader-cache patches that actually do pass
> piglit for over a year with the same reasoning that it will be disable
> by default and can only be improved with testing I can't possibly do on
> my own.
>
> Although I have no objections to this being merged I'll be extremely
> frustrated if this is allowed to be merged known to not even pass
> piglit while I've wasted countless hours rebasing shader cache over
> many months.
>

 Regardless of all the chatter on this thread in and around GL dispatch,
 which I agree poses significant challenges to get into something 'ideal'
 - which is hard to define for something like this..

 I think Timothy makes a really fair and just case here; in that, could
 we perhaps prioritize getting the shader cache stuff in before
 attempting GL dispatch? I think this both morally and technically the
 right thing to do in my humble opinion.
>>>
>>> There is a small difference. The shader cache is expected to be
>>> enabled by default, so there is a certain level of quality required.
>>
>> Hey Marek,
>>
>> I am under the impression that it being enabled by default isn't a hard
>> requirement for it to be merged. Maybe Timothy can weigh in on it when
>> he is online?
> 
> There is no official hard requirement. Everything is a judgement call
> based on circumstances.

Yes, OK, I agree; So why assert the above response then? Who is
expecting it to be enabled by default? To reiterate I believe Timothy
would like it merged first and foremost, then perhaps enable it by
default if that is OK with everyone. I didn't see anywhere he expected
it to be on by default. However we should wait for his response on that.

Regards,
Edward.

> 
> Marek
> 



signature.asc
Description: OpenPGP digital signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Fri, Feb 10, 2017 at 12:48 PM, Edward O'Callaghan
 wrote:
>
>
> On 02/10/2017 10:36 PM, Marek Olšák wrote:
>> On Fri, Feb 10, 2017 at 12:26 PM, Edward O'Callaghan
>>  wrote:
>>>
>>>
>>> On 02/08/2017 09:13 AM, Timothy Arceri wrote:
 On Tue, 2017-02-07 at 10:56 +0100, Marek Olšák wrote:
> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke  g> wrote:
>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand >>
>>> Yes, even segfaults are expected with mesa_glthread=true.
>>>
>>> Marek
>>
>> Would it make sense to be crash-free or even regression-free on at
>> least Piglit, before merging?  (Or are we there already?)
>
> It's not necessary. glthread is disabled by default. Nobody has
> tested
> piglit with glthread. That will follow after it's been merged, or
> never if it's never merged.

 I've been trying to land shader-cache patches that actually do pass
 piglit for over a year with the same reasoning that it will be disable
 by default and can only be improved with testing I can't possibly do on
 my own.

 Although I have no objections to this being merged I'll be extremely
 frustrated if this is allowed to be merged known to not even pass
 piglit while I've wasted countless hours rebasing shader cache over
 many months.

>>>
>>> Regardless of all the chatter on this thread in and around GL dispatch,
>>> which I agree poses significant challenges to get into something 'ideal'
>>> - which is hard to define for something like this..
>>>
>>> I think Timothy makes a really fair and just case here; in that, could
>>> we perhaps prioritize getting the shader cache stuff in before
>>> attempting GL dispatch? I think this both morally and technically the
>>> right thing to do in my humble opinion.
>>
>> There is a small difference. The shader cache is expected to be
>> enabled by default, so there is a certain level of quality required.
>
> Hey Marek,
>
> I am under the impression that it being enabled by default isn't a hard
> requirement for it to be merged. Maybe Timothy can weigh in on it when
> he is online?

There is no official hard requirement. Everything is a judgement call
based on circumstances.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Edward O'Callaghan]

On 02/10/2017 10:36 PM, Marek Olšák wrote:
> On Fri, Feb 10, 2017 at 12:26 PM, Edward O'Callaghan
>  wrote:
>>
>>
>> On 02/08/2017 09:13 AM, Timothy Arceri wrote:
>>> On Tue, 2017-02-07 at 10:56 +0100, Marek Olšák wrote:
 On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke  wrote:
> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand >> wrote:
>>> FYI glmark2 segfaults with mesa_glthread=true. Expected that
>>> some programs
>>> will segfault?
>>
>> Yes, even segfaults are expected with mesa_glthread=true.
>>
>> Marek
>
> Would it make sense to be crash-free or even regression-free on at
> least Piglit, before merging?  (Or are we there already?)

 It's not necessary. glthread is disabled by default. Nobody has
 tested
 piglit with glthread. That will follow after it's been merged, or
 never if it's never merged.
>>>
>>> I've been trying to land shader-cache patches that actually do pass
>>> piglit for over a year with the same reasoning that it will be disable
>>> by default and can only be improved with testing I can't possibly do on
>>> my own.
>>>
>>> Although I have no objections to this being merged I'll be extremely
>>> frustrated if this is allowed to be merged known to not even pass
>>> piglit while I've wasted countless hours rebasing shader cache over
>>> many months.
>>>
>>
>> Regardless of all the chatter on this thread in and around GL dispatch,
>> which I agree poses significant challenges to get into something 'ideal'
>> - which is hard to define for something like this..
>>
>> I think Timothy makes a really fair and just case here; in that, could
>> we perhaps prioritize getting the shader cache stuff in before
>> attempting GL dispatch? I think this both morally and technically the
>> right thing to do in my humble opinion.
> 
> There is a small difference. The shader cache is expected to be
> enabled by default, so there is a certain level of quality required.

Hey Marek,

I am under the impression that it being enabled by default isn't a hard
requirement for it to be merged. Maybe Timothy can weigh in on it when
he is online?

Cheers,
Edward.

> glthread won't likely be enabled by default, because we don't even
> know whether that's possible given the difficulty of the task.
> 
> The shader cache will likely be merged first anyway.
> 
> Marek
> 



signature.asc
Description: OpenPGP digital signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Fri, Feb 10, 2017 at 12:26 PM, Edward O'Callaghan
 wrote:
>
>
> On 02/08/2017 09:13 AM, Timothy Arceri wrote:
>> On Tue, 2017-02-07 at 10:56 +0100, Marek Olšák wrote:
>>> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke >> g> wrote:
 On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand > wrote:
>> FYI glmark2 segfaults with mesa_glthread=true. Expected that
>> some programs
>> will segfault?
>
> Yes, even segfaults are expected with mesa_glthread=true.
>
> Marek

 Would it make sense to be crash-free or even regression-free on at
 least Piglit, before merging?  (Or are we there already?)
>>>
>>> It's not necessary. glthread is disabled by default. Nobody has
>>> tested
>>> piglit with glthread. That will follow after it's been merged, or
>>> never if it's never merged.
>>
>> I've been trying to land shader-cache patches that actually do pass
>> piglit for over a year with the same reasoning that it will be disable
>> by default and can only be improved with testing I can't possibly do on
>> my own.
>>
>> Although I have no objections to this being merged I'll be extremely
>> frustrated if this is allowed to be merged known to not even pass
>> piglit while I've wasted countless hours rebasing shader cache over
>> many months.
>>
>
> Regardless of all the chatter on this thread in and around GL dispatch,
> which I agree poses significant challenges to get into something 'ideal'
> - which is hard to define for something like this..
>
> I think Timothy makes a really fair and just case here; in that, could
> we perhaps prioritize getting the shader cache stuff in before
> attempting GL dispatch? I think this both morally and technically the
> right thing to do in my humble opinion.

There is a small difference. The shader cache is expected to be
enabled by default, so there is a certain level of quality required.
glthread won't likely be enabled by default, because we don't even
know whether that's possible given the difficulty of the task.

The shader cache will likely be merged first anyway.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Edward O'Callaghan]

On 02/08/2017 09:13 AM, Timothy Arceri wrote:
> On Tue, 2017-02-07 at 10:56 +0100, Marek Olšák wrote:
>> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke > g> wrote:
>>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
 On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand  wrote:
> FYI glmark2 segfaults with mesa_glthread=true. Expected that
> some programs
> will segfault?

 Yes, even segfaults are expected with mesa_glthread=true.

 Marek
>>>
>>> Would it make sense to be crash-free or even regression-free on at
>>> least Piglit, before merging?  (Or are we there already?)
>>
>> It's not necessary. glthread is disabled by default. Nobody has
>> tested
>> piglit with glthread. That will follow after it's been merged, or
>> never if it's never merged.
> 
> I've been trying to land shader-cache patches that actually do pass
> piglit for over a year with the same reasoning that it will be disable
> by default and can only be improved with testing I can't possibly do on
> my own.
> 
> Although I have no objections to this being merged I'll be extremely
> frustrated if this is allowed to be merged known to not even pass
> piglit while I've wasted countless hours rebasing shader cache over
> many months.
> 

Regardless of all the chatter on this thread in and around GL dispatch,
which I agree poses significant challenges to get into something 'ideal'
- which is hard to define for something like this..

I think Timothy makes a really fair and just case here; in that, could
we perhaps prioritize getting the shader cache stuff in before
attempting GL dispatch? I think this both morally and technically the
right thing to do in my humble opinion.

Kindest Regards,
Edward.

> 
>>
>> Marek
>> ___
>> mesa-dev mailing list
>> mesa-dev@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
> 



signature.asc
Description: OpenPGP digital signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

The glmark2 issue is not important, because glthread will never
support glmark2. (because glmark2 is a SwapBuffers benchmark like
glxgears)

Marek

On Fri, Feb 10, 2017 at 11:32 AM, Eero Tamminen
 wrote:
> Hi,
>
> On 09.02.2017 20:27, Ernst Sjöstrand wrote:
>>
>> 2017-02-09 18:06 GMT+01:00 Eero Tamminen > My personal feeling is that minimal merging criteria should be:
>> * no known segfaults
>>
>> I bet that glmark2 just does something that's not allowed. Mesa can't
>> prevent all application bugs? Seems like it has a very buggy history
>> with lots of segfaults in other situations also.
>
>
> You don't know which one is at fault without investigating it.
>
> glmark2 is open source, so it's easy to debug, and based on my experience
> has active maintenance:
> https://github.com/glmark2/glmark2/issues/25
>
> If bug turns out to be in glmark2 instead of glthread code, file a bug:
> https://github.com/glmark2/glmark2/issues/
>
> If developers agree that it's bug on glmark2 side (or at least don't point
> out an issue at Mesa side), segfault is OK.
>
> (It would be best to test latest glmark2 git version, but IMHO waf usage
> makes it a bit pain to build.)
>
>
> glxgears uses old GL stuff:
> https://cgit.freedesktop.org/mesa/demos/tree/src/xdemos/glxgears.c
>
> If glthread doesn't work with that (e.g. glBegin() / glEnd()), it should
> either disable threading, or abort with warning.  Silently corrupting things
> is not OK even for optional code paths.
>
>
> I forgot one more thing from the minimum requirements:
> * Once the segfaults and piglit failures have been investigated & fixed, or
> explained not to be glthread issues, glthread still shouldn't slow Mesa down
> when it's _disabled_
>
> (Marginal slowdown at startup might be OK. At run-time slowdowns, when
> glmark is not enabled, are not going to fly as most programs will be run
> without enabling glthread.)
>
>
>
> - Eero
>
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Eero Tamminen]

Hi,

On 09.02.2017 20:27, Ernst Sjöstrand wrote:

2017-02-09 18:06 GMT+01:00 Eero Tamminen 

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Thu, Feb 9, 2017 at 8:18 PM, Marek Olšák  wrote:
> On Thu, Feb 9, 2017 at 6:23 PM, Ian Romanick  wrote:
>> On 02/09/2017 03:03 PM, Marek Olšák wrote:
>>> On Thu, Feb 9, 2017 at 1:52 PM, Ian Romanick  wrote:
 On 02/08/2017 10:26 AM, Nicolai Hähnle wrote:
> On 07.02.2017 23:54, Matt Turner wrote:
>> On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
>>> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>>>  wrote:
 On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
> wrote:
>> FYI glmark2 segfaults with mesa_glthread=true. Expected that some
>> programs
>> will segfault?
>
> Yes, even segfaults are expected with mesa_glthread=true.
>
> Marek

 Would it make sense to be crash-free or even regression-free on at
 least Piglit, before merging?  (Or are we there already?)
>>>
>>> It's not necessary. glthread is disabled by default. Nobody has tested
>>> piglit with glthread. That will follow after it's been merged, or
>>> never if it's never merged.
>>
>> I don't understand why you're so concerned about merging untested
>> code. That violates some pretty fundamental development practices of
>> the project.
>>
>> It's exactly unfinished projects like this that cause problems and
>> inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
>> think it's a burden to develop something out of the master branch
>> until it's somewhat useful.
>
> The code is already somewhat useful. Actually, make that _very_ useful
> (big performance improvement) for _some_ cases.
>
> I suspect most of the people in this discussion haven't really looked at
> the code in detail (myself included). We should probably do some of that
> before it is merged, since the code isn't just a new driver that is
> isolated in its own directory. So I agree with Emil that it makes sense
> to let the patches go over the mailing list once.
>
> However, it's fine to merge this without passing piglit.

 No, it absolutely is not fine to merge.  We have never allowed such a
 thing, and I'll be damned if I'll allow this project to start.  Things
 that land that are known to be broken never actually get fixed.  Then we
 have to waste time fielding bug reports and Phoronix threads because
 users turn on the performance features and everything breaks.  It's just
 a terrible idea.
>>>
>>> It does pass piglit, but only when it's disabled.
>>>
>>> We have to ask the question of how long it will take to reach the
>>> level of perfection that some people here demand. 1 year? 2? 4 years
>>> even? Are we willing to wait that long? Is there a sufficient minimum
>>> requirement on merging this project that's possible to reach within 2
>>> weeks? Instead of saying "absolutely not" and "terrible idea", why not
>>> just say "yes if X gets done"?
>>
>> Nobody has touched this code in years, yet you're speaking as if it has
>> been in active development for the whole time.  Eric and Paul
>> implemented, found that it didn't help any applications that existed at
>> the time, and abandoned it.
>
> There are some good arguments in this thread, but it's not this one. I
> did touch the code. I made Borderlands 2 work by adding Gallium
> support, DRI3 support, and fixing race conditions and other bugs in
> the core glthread code and glapi XML files. Even though the commits
> have Eric and Paul as authors, I edited most of them in some way.
>
> So if I understand correctly, the requirement for merging is to pass
> piglit. If we drop support for compat profiles (do we need to drop
> support for GLES?), it shouldn't be too hard.

Talos Principle uses the compat profile, so we can't drop it. :(

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Eric Anholt]

Ian Romanick  writes:

> [ Unknown signature status ]
> On 02/09/2017 05:14 PM, Eric Anholt wrote:
>> Ian Romanick  writes:
>> 
>>> On 02/08/2017 10:26 AM, Nicolai Hähnle wrote:
 On 07.02.2017 23:54, Matt Turner wrote:
> On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
>> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>>  wrote:
>>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
 On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
 wrote:
> FYI glmark2 segfaults with mesa_glthread=true. Expected that some
> programs
> will segfault?

 Yes, even segfaults are expected with mesa_glthread=true.

 Marek
>>>
>>> Would it make sense to be crash-free or even regression-free on at
>>> least Piglit, before merging?  (Or are we there already?)
>>
>> It's not necessary. glthread is disabled by default. Nobody has tested
>> piglit with glthread. That will follow after it's been merged, or
>> never if it's never merged.
>
> I don't understand why you're so concerned about merging untested
> code. That violates some pretty fundamental development practices of
> the project.
>
> It's exactly unfinished projects like this that cause problems and
> inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
> think it's a burden to develop something out of the master branch
> until it's somewhat useful.

 The code is already somewhat useful. Actually, make that _very_ useful
 (big performance improvement) for _some_ cases.

 I suspect most of the people in this discussion haven't really looked at
 the code in detail (myself included). We should probably do some of that
 before it is merged, since the code isn't just a new driver that is
 isolated in its own directory. So I agree with Emil that it makes sense
 to let the patches go over the mailing list once.

 However, it's fine to merge this without passing piglit.
>>>
>>> No, it absolutely is not fine to merge.  We have never allowed such a
>>> thing, and I'll be damned if I'll allow this project to start.  Things
>>> that land that are known to be broken never actually get fixed.  Then we
>>> have to waste time fielding bug reports and Phoronix threads because
>>> users turn on the performance features and everything breaks.  It's just
>>> a terrible idea.
>> 
>> Yeah, just like how we gated the GLSL compiler until it was completely
>> done (we didn't) and NIR until it was completely done (we didn't) and
>> Vulkan until it was completely done (we didn't) and...
>
> But we did require that it pass the test suites.  All of that lived in
> branches for a really long time with active development.  The GLSL
> compiler and NIR are two examples that support my claim.  Those were
> massive branch merges that waited much longer to merge than the people
> working on them wanted.

We also didn't port fragment programs and fixed function over to the new
backend for a long time because it was thoroughly incomplete, and that
kept our tests running.  We hid the i965 GLSL IR VS under an environment
variable while we were finishing it, too.

git log -p | grep INTEL_USE_NIR will give you a whole lot of more recent
commits about how in the Intel driver it's normal to land code that's in
progress before it's completely done, hidden under an env var.


signature.asc
Description: PGP signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Thu, Feb 9, 2017 at 6:23 PM, Ian Romanick  wrote:
> On 02/09/2017 03:03 PM, Marek Olšák wrote:
>> On Thu, Feb 9, 2017 at 1:52 PM, Ian Romanick  wrote:
>>> On 02/08/2017 10:26 AM, Nicolai Hähnle wrote:
 On 07.02.2017 23:54, Matt Turner wrote:
> On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
>> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>>  wrote:
>>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
 On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
 wrote:
> FYI glmark2 segfaults with mesa_glthread=true. Expected that some
> programs
> will segfault?

 Yes, even segfaults are expected with mesa_glthread=true.

 Marek
>>>
>>> Would it make sense to be crash-free or even regression-free on at
>>> least Piglit, before merging?  (Or are we there already?)
>>
>> It's not necessary. glthread is disabled by default. Nobody has tested
>> piglit with glthread. That will follow after it's been merged, or
>> never if it's never merged.
>
> I don't understand why you're so concerned about merging untested
> code. That violates some pretty fundamental development practices of
> the project.
>
> It's exactly unfinished projects like this that cause problems and
> inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
> think it's a burden to develop something out of the master branch
> until it's somewhat useful.

 The code is already somewhat useful. Actually, make that _very_ useful
 (big performance improvement) for _some_ cases.

 I suspect most of the people in this discussion haven't really looked at
 the code in detail (myself included). We should probably do some of that
 before it is merged, since the code isn't just a new driver that is
 isolated in its own directory. So I agree with Emil that it makes sense
 to let the patches go over the mailing list once.

 However, it's fine to merge this without passing piglit.
>>>
>>> No, it absolutely is not fine to merge.  We have never allowed such a
>>> thing, and I'll be damned if I'll allow this project to start.  Things
>>> that land that are known to be broken never actually get fixed.  Then we
>>> have to waste time fielding bug reports and Phoronix threads because
>>> users turn on the performance features and everything breaks.  It's just
>>> a terrible idea.
>>
>> It does pass piglit, but only when it's disabled.
>>
>> We have to ask the question of how long it will take to reach the
>> level of perfection that some people here demand. 1 year? 2? 4 years
>> even? Are we willing to wait that long? Is there a sufficient minimum
>> requirement on merging this project that's possible to reach within 2
>> weeks? Instead of saying "absolutely not" and "terrible idea", why not
>> just say "yes if X gets done"?
>
> Nobody has touched this code in years, yet you're speaking as if it has
> been in active development for the whole time.  Eric and Paul
> implemented, found that it didn't help any applications that existed at
> the time, and abandoned it.

There are some good arguments in this thread, but it's not this one. I
did touch the code. I made Borderlands 2 work by adding Gallium
support, DRI3 support, and fixing race conditions and other bugs in
the core glthread code and glapi XML files. Even though the commits
have Eric and Paul as authors, I edited most of them in some way.

So if I understand correctly, the requirement for merging is to pass
piglit. If we drop support for compat profiles (do we need to drop
support for GLES?), it shouldn't be too hard.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Ernst Sjöstrand]

2017-02-09 18:06 GMT+01:00 Eero Tamminen :

My personal feeling is that minimal merging criteria should be:
> * no known segfaults
>

I bet that glmark2 just does something that's not allowed. Mesa can't
prevent all application bugs? Seems like it has a very buggy history with
lots of segfaults in other situations also.

Regards
//Ernst
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Kenneth Graunke]

On Thursday, February 9, 2017 5:23:47 PM PST Ian Romanick wrote:
[snip]
> Not achieving the desired performance is vastly different from
> segfaults.  Does the threaded mode in NVIDIA's driver crash?  I'd wager
> not.

Sadly, yes it does.  A while back, I tried running some Steam games
on Linux using my NVIDIA machine, with an out of the box configuration,
and they totally crashed.  Workaround was to disable GL threaded
optimizations.  Of course, I also had crashes which I had to update the
CPU microcode to workaround, and tearing right through the middle of
every single frame (needed more workarounds), and all kinds of
problems...so...not sure that's the standard of quality I want to
aspire to...

--Ken


signature.asc
Description: This is a digitally signed message part.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Ian Romanick]

On 02/09/2017 05:26 PM, Ian Romanick wrote:
> On 02/09/2017 05:14 PM, Eric Anholt wrote:
>> Ian Romanick  writes:
>>
>>> On 02/08/2017 10:26 AM, Nicolai Hähnle wrote:
 On 07.02.2017 23:54, Matt Turner wrote:
> On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
>> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>>  wrote:
>>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
 On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
 wrote:
> FYI glmark2 segfaults with mesa_glthread=true. Expected that some
> programs
> will segfault?

 Yes, even segfaults are expected with mesa_glthread=true.

 Marek
>>>
>>> Would it make sense to be crash-free or even regression-free on at
>>> least Piglit, before merging?  (Or are we there already?)
>>
>> It's not necessary. glthread is disabled by default. Nobody has tested
>> piglit with glthread. That will follow after it's been merged, or
>> never if it's never merged.
>
> I don't understand why you're so concerned about merging untested
> code. That violates some pretty fundamental development practices of
> the project.
>
> It's exactly unfinished projects like this that cause problems and
> inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
> think it's a burden to develop something out of the master branch
> until it's somewhat useful.

 The code is already somewhat useful. Actually, make that _very_ useful
 (big performance improvement) for _some_ cases.

 I suspect most of the people in this discussion haven't really looked at
 the code in detail (myself included). We should probably do some of that
 before it is merged, since the code isn't just a new driver that is
 isolated in its own directory. So I agree with Emil that it makes sense
 to let the patches go over the mailing list once.

 However, it's fine to merge this without passing piglit.
>>>
>>> No, it absolutely is not fine to merge.  We have never allowed such a
>>> thing, and I'll be damned if I'll allow this project to start.  Things
>>> that land that are known to be broken never actually get fixed.  Then we
>>> have to waste time fielding bug reports and Phoronix threads because
>>> users turn on the performance features and everything breaks.  It's just
>>> a terrible idea.
>>
>> Yeah, just like how we gated the GLSL compiler until it was completely
>> done (we didn't) and NIR until it was completely done (we didn't) and
>> Vulkan until it was completely done (we didn't) and...
> 
> But we did require that it pass the test suites.  All of that lived in
> branches for a really long time with active development.  The GLSL
> compiler and NIR are two examples that support my claim.  Those were
> massive branch merges that waited much longer to merge than the people
> working on them wanted.

Also... the GLSL compiler was long before we did time-based releases.
We block the release for a really long time so that we could achieve the
quality that we need.

>> Software that people care about gets fixed.  I'm also concerned that
>> nobody actually cares about getting glthread working completely, given
>> Marek's attitude toward piglit conformance (and my also ignoring the
>> branch for the last however many years).  However, "we have never
>> allowed merging broken software that's only turned on under env
>> vars/configure" is totally false.  We do that regularly for big things
>> we care about.
> 
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Ian Romanick]

On 02/09/2017 05:14 PM, Eric Anholt wrote:
> Ian Romanick  writes:
> 
>> On 02/08/2017 10:26 AM, Nicolai Hähnle wrote:
>>> On 07.02.2017 23:54, Matt Turner wrote:
 On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>  wrote:
>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
>>> wrote:
 FYI glmark2 segfaults with mesa_glthread=true. Expected that some
 programs
 will segfault?
>>>
>>> Yes, even segfaults are expected with mesa_glthread=true.
>>>
>>> Marek
>>
>> Would it make sense to be crash-free or even regression-free on at
>> least Piglit, before merging?  (Or are we there already?)
>
> It's not necessary. glthread is disabled by default. Nobody has tested
> piglit with glthread. That will follow after it's been merged, or
> never if it's never merged.

 I don't understand why you're so concerned about merging untested
 code. That violates some pretty fundamental development practices of
 the project.

 It's exactly unfinished projects like this that cause problems and
 inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
 think it's a burden to develop something out of the master branch
 until it's somewhat useful.
>>>
>>> The code is already somewhat useful. Actually, make that _very_ useful
>>> (big performance improvement) for _some_ cases.
>>>
>>> I suspect most of the people in this discussion haven't really looked at
>>> the code in detail (myself included). We should probably do some of that
>>> before it is merged, since the code isn't just a new driver that is
>>> isolated in its own directory. So I agree with Emil that it makes sense
>>> to let the patches go over the mailing list once.
>>>
>>> However, it's fine to merge this without passing piglit.
>>
>> No, it absolutely is not fine to merge.  We have never allowed such a
>> thing, and I'll be damned if I'll allow this project to start.  Things
>> that land that are known to be broken never actually get fixed.  Then we
>> have to waste time fielding bug reports and Phoronix threads because
>> users turn on the performance features and everything breaks.  It's just
>> a terrible idea.
> 
> Yeah, just like how we gated the GLSL compiler until it was completely
> done (we didn't) and NIR until it was completely done (we didn't) and
> Vulkan until it was completely done (we didn't) and...

But we did require that it pass the test suites.  All of that lived in
branches for a really long time with active development.  The GLSL
compiler and NIR are two examples that support my claim.  Those were
massive branch merges that waited much longer to merge than the people
working on them wanted.

> Software that people care about gets fixed.  I'm also concerned that
> nobody actually cares about getting glthread working completely, given
> Marek's attitude toward piglit conformance (and my also ignoring the
> branch for the last however many years).  However, "we have never
> allowed merging broken software that's only turned on under env
> vars/configure" is totally false.  We do that regularly for big things
> we care about.




signature.asc
Description: OpenPGP digital signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Eero Tamminen]

Hi,

On 09.02.2017 19:14, Eric Anholt wrote:

Ian Romanick  writes:

...

No, it absolutely is not fine to merge.  We have never allowed such a
thing, and I'll be damned if I'll allow this project to start.  Things
that land that are known to be broken never actually get fixed.  Then we
have to waste time fielding bug reports and Phoronix threads because
users turn on the performance features and everything breaks.  It's just
a terrible idea.


Yeah, just like how we gated the GLSL compiler until it was completely
done (we didn't) and NIR until it was completely done (we didn't) and
Vulkan until it was completely done (we didn't) and...


None of the features you list were purely for performance.  If it's just 
supposed to [1] improve performance and complicates the existing code, 
gate should be higher.


[1] You cannot fully trust the provided performance numbers if it's not 
functioning properly.




Software that people care about gets fixed.  I'm also concerned that
nobody actually cares about getting glthread working completely, given
Marek's attitude toward piglit conformance (and my also ignoring the
branch for the last however many years).  However, "we have never
allowed merging broken software that's only turned on under env
vars/configure" is totally false.  We do that regularly for big things
we care about.



- Eero

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Ian Romanick]

On 02/09/2017 03:03 PM, Marek Olšák wrote:
> On Thu, Feb 9, 2017 at 1:52 PM, Ian Romanick  wrote:
>> On 02/08/2017 10:26 AM, Nicolai Hähnle wrote:
>>> On 07.02.2017 23:54, Matt Turner wrote:
 On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>  wrote:
>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
>>> wrote:
 FYI glmark2 segfaults with mesa_glthread=true. Expected that some
 programs
 will segfault?
>>>
>>> Yes, even segfaults are expected with mesa_glthread=true.
>>>
>>> Marek
>>
>> Would it make sense to be crash-free or even regression-free on at
>> least Piglit, before merging?  (Or are we there already?)
>
> It's not necessary. glthread is disabled by default. Nobody has tested
> piglit with glthread. That will follow after it's been merged, or
> never if it's never merged.

 I don't understand why you're so concerned about merging untested
 code. That violates some pretty fundamental development practices of
 the project.

 It's exactly unfinished projects like this that cause problems and
 inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
 think it's a burden to develop something out of the master branch
 until it's somewhat useful.
>>>
>>> The code is already somewhat useful. Actually, make that _very_ useful
>>> (big performance improvement) for _some_ cases.
>>>
>>> I suspect most of the people in this discussion haven't really looked at
>>> the code in detail (myself included). We should probably do some of that
>>> before it is merged, since the code isn't just a new driver that is
>>> isolated in its own directory. So I agree with Emil that it makes sense
>>> to let the patches go over the mailing list once.
>>>
>>> However, it's fine to merge this without passing piglit.
>>
>> No, it absolutely is not fine to merge.  We have never allowed such a
>> thing, and I'll be damned if I'll allow this project to start.  Things
>> that land that are known to be broken never actually get fixed.  Then we
>> have to waste time fielding bug reports and Phoronix threads because
>> users turn on the performance features and everything breaks.  It's just
>> a terrible idea.
> 
> It does pass piglit, but only when it's disabled.
> 
> We have to ask the question of how long it will take to reach the
> level of perfection that some people here demand. 1 year? 2? 4 years
> even? Are we willing to wait that long? Is there a sufficient minimum
> requirement on merging this project that's possible to reach within 2
> weeks? Instead of saying "absolutely not" and "terrible idea", why not
> just say "yes if X gets done"?

Nobody has touched this code in years, yet you're speaking as if it has
been in active development for the whole time.  Eric and Paul
implemented, found that it didn't help any applications that existed at
the time, and abandoned it.

> If you only expect absolutism and perfectionism, you'll be very
> disappointed with this project. Doing a threaded GL dispatch is hard.
> NVIDIA had had performance issues with it for a very long time. AMD
> doesn't even have a solution that can be enabled by default.

Not achieving the desired performance is vastly different from
segfaults.  Does the threaded mode in NVIDIA's driver crash?  I'd wager
not.  We have always had high quality standards for Mesa.  We have
always adhered to the specified OpenGL behavior (and worked to fix the
spec when it didn't match other vendors).  These are the things that
have made Mesa successful in the presence of drivers with more features
and / or better performance.  It has also made software developers like
Mesa and trust us.  I haven't heard a compelling reason to abandon those
fundamental principles.

There have been times in the past where we have landed performance
oriented features that didn't initially provide performance benefits to
all applications.  i965's transition to NIR is one such case.  However,
we did require that the test suites passed with the feature enabled.

> I'm not heavily invested in this project (yet), so it won't bother me
> much if this doesn't get merged before mid-2017. If people wanna
> contribute, they can send me pull requests. (please do it early &
> often, don't wait until you have something that's good enough)
> 
> I've been contemplating doing threaded gallium dispatch for a while
> now. It should be very easy if we copy some sync prevention tricks
> from radeonsi, and we might be able to enable it by default for all GL
> apps (core & compat) from day 1. That can be our short-term goal for
> gallium, while threaded GL can be our long-term goal and only limited
> to whitelisted apps for quite a while.
> 
> Marek

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Eric Anholt]

Ian Romanick  writes:

> On 02/08/2017 10:26 AM, Nicolai Hähnle wrote:
>> On 07.02.2017 23:54, Matt Turner wrote:
>>> On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
 On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
  wrote:
> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
>> wrote:
>>> FYI glmark2 segfaults with mesa_glthread=true. Expected that some
>>> programs
>>> will segfault?
>>
>> Yes, even segfaults are expected with mesa_glthread=true.
>>
>> Marek
>
> Would it make sense to be crash-free or even regression-free on at
> least Piglit, before merging?  (Or are we there already?)

 It's not necessary. glthread is disabled by default. Nobody has tested
 piglit with glthread. That will follow after it's been merged, or
 never if it's never merged.
>>>
>>> I don't understand why you're so concerned about merging untested
>>> code. That violates some pretty fundamental development practices of
>>> the project.
>>>
>>> It's exactly unfinished projects like this that cause problems and
>>> inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
>>> think it's a burden to develop something out of the master branch
>>> until it's somewhat useful.
>> 
>> The code is already somewhat useful. Actually, make that _very_ useful
>> (big performance improvement) for _some_ cases.
>> 
>> I suspect most of the people in this discussion haven't really looked at
>> the code in detail (myself included). We should probably do some of that
>> before it is merged, since the code isn't just a new driver that is
>> isolated in its own directory. So I agree with Emil that it makes sense
>> to let the patches go over the mailing list once.
>> 
>> However, it's fine to merge this without passing piglit.
>
> No, it absolutely is not fine to merge.  We have never allowed such a
> thing, and I'll be damned if I'll allow this project to start.  Things
> that land that are known to be broken never actually get fixed.  Then we
> have to waste time fielding bug reports and Phoronix threads because
> users turn on the performance features and everything breaks.  It's just
> a terrible idea.

Yeah, just like how we gated the GLSL compiler until it was completely
done (we didn't) and NIR until it was completely done (we didn't) and
Vulkan until it was completely done (we didn't) and...

Software that people care about gets fixed.  I'm also concerned that
nobody actually cares about getting glthread working completely, given
Marek's attitude toward piglit conformance (and my also ignoring the
branch for the last however many years).  However, "we have never
allowed merging broken software that's only turned on under env
vars/configure" is totally false.  We do that regularly for big things
we care about.


signature.asc
Description: PGP signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Matt Turner]

On Thu, Feb 9, 2017 at 3:03 PM, Marek Olšák  wrote:
> On Thu, Feb 9, 2017 at 1:52 PM, Ian Romanick  wrote:
>> On 02/08/2017 10:26 AM, Nicolai Hähnle wrote:
>>> On 07.02.2017 23:54, Matt Turner wrote:
 On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>  wrote:
>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
>>> wrote:
 FYI glmark2 segfaults with mesa_glthread=true. Expected that some
 programs
 will segfault?
>>>
>>> Yes, even segfaults are expected with mesa_glthread=true.
>>>
>>> Marek
>>
>> Would it make sense to be crash-free or even regression-free on at
>> least Piglit, before merging?  (Or are we there already?)
>
> It's not necessary. glthread is disabled by default. Nobody has tested
> piglit with glthread. That will follow after it's been merged, or
> never if it's never merged.

 I don't understand why you're so concerned about merging untested
 code. That violates some pretty fundamental development practices of
 the project.

 It's exactly unfinished projects like this that cause problems and
 inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
 think it's a burden to develop something out of the master branch
 until it's somewhat useful.
>>>
>>> The code is already somewhat useful. Actually, make that _very_ useful
>>> (big performance improvement) for _some_ cases.
>>>
>>> I suspect most of the people in this discussion haven't really looked at
>>> the code in detail (myself included). We should probably do some of that
>>> before it is merged, since the code isn't just a new driver that is
>>> isolated in its own directory. So I agree with Emil that it makes sense
>>> to let the patches go over the mailing list once.
>>>
>>> However, it's fine to merge this without passing piglit.
>>
>> No, it absolutely is not fine to merge.  We have never allowed such a
>> thing, and I'll be damned if I'll allow this project to start.  Things
>> that land that are known to be broken never actually get fixed.  Then we
>> have to waste time fielding bug reports and Phoronix threads because
>> users turn on the performance features and everything breaks.  It's just
>> a terrible idea.
>
> It does pass piglit, but only when it's disabled.

Have you actually run piglit? What are the results on radeonsi for
glthread=true vs glthread=false?
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Eero Tamminen]

Hi,

On 09.02.2017 17:03, Marek Olšák wrote:

On Thu, Feb 9, 2017 at 1:52 PM, Ian Romanick  wrote:

...

No, it absolutely is not fine to merge.  We have never allowed such a
thing, and I'll be damned if I'll allow this project to start.  Things
that land that are known to be broken never actually get fixed.  Then we
have to waste time fielding bug reports and Phoronix threads because
users turn on the performance features and everything breaks.  It's just
a terrible idea.


It does pass piglit, but only when it's disabled.

We have to ask the question of how long it will take to reach the
level of perfection that some people here demand. 1 year? 2? 4 years
even? Are we willing to wait that long? Is there a sufficient minimum
requirement on merging this project that's possible to reach within 2
weeks? Instead of saying "absolutely not" and "terrible idea", why not
just say "yes if X gets done"?




My personal feeling is that minimal merging criteria should be:
* no known segfaults
* no functional piglit failures (or if there are, they're known, 
somewhat investigated and commonly agreed to be OK for merging)


After the functional issues are fixed, performance issues are fine as 
long as it still clearly helps (>=10%?) some games.


In general, performance numbers for features that have functional 
issues, shouldn't be trusted.  If e.g. crashes are due to missing 
synchronization, does adding the required synchronization also lose the 
measured performance increase?




If you only expect absolutism and perfectionism, you'll be very
disappointed with this project. Doing a threaded GL dispatch is hard.
NVIDIA had had performance issues with it for a very long time.


Those are fine.  Have they had also crashes or rendering issues?

Is realistic approach just automatically disabling threading when 
program uses problematic GL feature/version? (Can it be disabled if it's 
already in use?)



> AMD doesn't even have a solution that can be enabled by default.
>

I'm not heavily invested in this project (yet), so it won't bother me
much if this doesn't get merged before mid-2017. If people wanna
contribute, they can send me pull requests. (please do it early &
often, don't wait until you have something that's good enough)

I've been contemplating doing threaded gallium dispatch for a while
now. It should be very easy if we copy some sync prevention tricks
from radeonsi, and we might be able to enable it by default for all GL
apps (core & compat) from day 1. That can be our short-term goal for
gallium, while threaded GL can be our long-term goal and only limited
to whitelisted apps for quite a while.



- Eero

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Thu, Feb 9, 2017 at 1:52 PM, Ian Romanick  wrote:
> On 02/08/2017 10:26 AM, Nicolai Hähnle wrote:
>> On 07.02.2017 23:54, Matt Turner wrote:
>>> On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
 On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
  wrote:
> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
>> wrote:
>>> FYI glmark2 segfaults with mesa_glthread=true. Expected that some
>>> programs
>>> will segfault?
>>
>> Yes, even segfaults are expected with mesa_glthread=true.
>>
>> Marek
>
> Would it make sense to be crash-free or even regression-free on at
> least Piglit, before merging?  (Or are we there already?)

 It's not necessary. glthread is disabled by default. Nobody has tested
 piglit with glthread. That will follow after it's been merged, or
 never if it's never merged.
>>>
>>> I don't understand why you're so concerned about merging untested
>>> code. That violates some pretty fundamental development practices of
>>> the project.
>>>
>>> It's exactly unfinished projects like this that cause problems and
>>> inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
>>> think it's a burden to develop something out of the master branch
>>> until it's somewhat useful.
>>
>> The code is already somewhat useful. Actually, make that _very_ useful
>> (big performance improvement) for _some_ cases.
>>
>> I suspect most of the people in this discussion haven't really looked at
>> the code in detail (myself included). We should probably do some of that
>> before it is merged, since the code isn't just a new driver that is
>> isolated in its own directory. So I agree with Emil that it makes sense
>> to let the patches go over the mailing list once.
>>
>> However, it's fine to merge this without passing piglit.
>
> No, it absolutely is not fine to merge.  We have never allowed such a
> thing, and I'll be damned if I'll allow this project to start.  Things
> that land that are known to be broken never actually get fixed.  Then we
> have to waste time fielding bug reports and Phoronix threads because
> users turn on the performance features and everything breaks.  It's just
> a terrible idea.

It does pass piglit, but only when it's disabled.

We have to ask the question of how long it will take to reach the
level of perfection that some people here demand. 1 year? 2? 4 years
even? Are we willing to wait that long? Is there a sufficient minimum
requirement on merging this project that's possible to reach within 2
weeks? Instead of saying "absolutely not" and "terrible idea", why not
just say "yes if X gets done"?

If you only expect absolutism and perfectionism, you'll be very
disappointed with this project. Doing a threaded GL dispatch is hard.
NVIDIA had had performance issues with it for a very long time. AMD
doesn't even have a solution that can be enabled by default.

I'm not heavily invested in this project (yet), so it won't bother me
much if this doesn't get merged before mid-2017. If people wanna
contribute, they can send me pull requests. (please do it early &
often, don't wait until you have something that's good enough)

I've been contemplating doing threaded gallium dispatch for a while
now. It should be very easy if we copy some sync prevention tricks
from radeonsi, and we might be able to enable it by default for all GL
apps (core & compat) from day 1. That can be our short-term goal for
gallium, while threaded GL can be our long-term goal and only limited
to whitelisted apps for quite a while.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Ian Romanick]

On 02/08/2017 10:26 AM, Nicolai Hähnle wrote:
> On 07.02.2017 23:54, Matt Turner wrote:
>> On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
>>> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke
>>>  wrote:
 On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
> wrote:
>> FYI glmark2 segfaults with mesa_glthread=true. Expected that some
>> programs
>> will segfault?
>
> Yes, even segfaults are expected with mesa_glthread=true.
>
> Marek

 Would it make sense to be crash-free or even regression-free on at
 least Piglit, before merging?  (Or are we there already?)
>>>
>>> It's not necessary. glthread is disabled by default. Nobody has tested
>>> piglit with glthread. That will follow after it's been merged, or
>>> never if it's never merged.
>>
>> I don't understand why you're so concerned about merging untested
>> code. That violates some pretty fundamental development practices of
>> the project.
>>
>> It's exactly unfinished projects like this that cause problems and
>> inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
>> think it's a burden to develop something out of the master branch
>> until it's somewhat useful.
> 
> The code is already somewhat useful. Actually, make that _very_ useful
> (big performance improvement) for _some_ cases.
> 
> I suspect most of the people in this discussion haven't really looked at
> the code in detail (myself included). We should probably do some of that
> before it is merged, since the code isn't just a new driver that is
> isolated in its own directory. So I agree with Emil that it makes sense
> to let the patches go over the mailing list once.
> 
> However, it's fine to merge this without passing piglit.

No, it absolutely is not fine to merge.  We have never allowed such a
thing, and I'll be damned if I'll allow this project to start.  Things
that land that are known to be broken never actually get fixed.  Then we
have to waste time fielding bug reports and Phoronix threads because
users turn on the performance features and everything breaks.  It's just
a terrible idea.

> Cheers,
> Nicolai
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Ian Romanick]

On 02/07/2017 02:57 AM, Kenneth Graunke wrote:
> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand  wrote:
>>> FYI glmark2 segfaults with mesa_glthread=true. Expected that some programs
>>> will segfault?
>>
>> Yes, even segfaults are expected with mesa_glthread=true.
>>
>> Marek
> 
> Would it make sense to be crash-free or even regression-free on at
> least Piglit, before merging?  (Or are we there already?)

s/make sense/be an absolute hard requirement full stop/.  Yes.

> --Ken
> 
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev




signature.asc
Description: OpenPGP digital signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Ernst Sjöstrand]

I benchmarked Total War: Attila and CS:GO quickly yesterday, both ran fine
but none of them seemed to gain any performance. Maybe lost a little.
I saw that Tomb Raider only had ~130% cpu usage on my system, perhaps
that's a candidate even though it's a Feral title? It's 32-bit though and I
couldn't bother to cross compile.

FYI I think glmark2 is a bit buggy: https://bugs.launchpad.net/glmark2
Tough it does run fine for me without threading right now...

Regards
//Ernst

2017-02-08 18:45 GMT+01:00 Marek Olšák :

> On Wed, Feb 8, 2017 at 6:29 PM, Eero Tamminen 
> wrote:
> > Hi,
> >
> > On 08.02.2017 15:10, Marek Olšák wrote:
> >>
> >> On Feb 8, 2017 11:15 AM, "Eero Tamminen"  >> If there are known crashes, and no piglit statistics, how you can
> >> trust that the games which the patch series seems to already help,
> >> actually do work?  That they don't crash or deadlock at some point
> >> because of it?
> >>
> >> I hate to repeat myself, but it was tested with quite a bunch of games.
> >> You know, I actually made it work for Gallium including bug fixes in
> >> glthread like bad locking and race conditions. I know Samuel Pitoiset
> >> also tested it with good results. A bunch of people on IRC tested it
> too.
> >
> >
> > Do you have some e.g. wiki page where this status info is collected?
> >
> >
> > So far the data provided in this thread is following:
> >
> > Improves performance:
> > * Borderlands 2 (with multiple game gfx settings & driver/HW?)
> >
> > Regresses performance:
> > * Shadow of Mordor (with multiple game gfx settings & driver/HW?)
> > * PCSX2 emulator
> >
> > Crashes:
> > * glmark2 (one test? all tests?  both GL & GLES version?)
> > * fgl_glxgears
> >
> >
> >> Concerning the glmark crash, I don't care. Get over it.
> >
> > Glmark & glxgears are rather simple, so it's just a bit surprising.
>
> I don't understand why any of this is important. Did you not notice
> that it's disabled by default and the implementation is not complete?
> I think I said it somewhere.
>
> Marek
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[gregory hainaut]

> Regresses performance:
> * Shadow of Mordor (with multiple game gfx settings & driver/HW?)
> * PCSX2 emulator

As Marek said the code still miss some optimization opportunities. FWIW, PCSX2 
uses persistant PBO for texture transfer so it can be done without any sync. I 
removed the sync (with a hack), I won +25% on FPS and rendering was limited by 
the GPU (Nouveau).

Gregory
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Wed, Feb 8, 2017 at 6:29 PM, Eero Tamminen  wrote:
> Hi,
>
> On 08.02.2017 15:10, Marek Olšák wrote:
>>
>> On Feb 8, 2017 11:15 AM, "Eero Tamminen" > If there are known crashes, and no piglit statistics, how you can
>> trust that the games which the patch series seems to already help,
>> actually do work?  That they don't crash or deadlock at some point
>> because of it?
>>
>> I hate to repeat myself, but it was tested with quite a bunch of games.
>> You know, I actually made it work for Gallium including bug fixes in
>> glthread like bad locking and race conditions. I know Samuel Pitoiset
>> also tested it with good results. A bunch of people on IRC tested it too.
>
>
> Do you have some e.g. wiki page where this status info is collected?
>
>
> So far the data provided in this thread is following:
>
> Improves performance:
> * Borderlands 2 (with multiple game gfx settings & driver/HW?)
>
> Regresses performance:
> * Shadow of Mordor (with multiple game gfx settings & driver/HW?)
> * PCSX2 emulator
>
> Crashes:
> * glmark2 (one test? all tests?  both GL & GLES version?)
> * fgl_glxgears
>
>
>> Concerning the glmark crash, I don't care. Get over it.
>
> Glmark & glxgears are rather simple, so it's just a bit surprising.

I don't understand why any of this is important. Did you not notice
that it's disabled by default and the implementation is not complete?
I think I said it somewhere.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Eero Tamminen]

Hi,

On 08.02.2017 15:10, Marek Olšák wrote:

On Feb 8, 2017 11:15 AM, "Eero Tamminen"  Concerning the glmark crash, I don't care. Get over it.

Glmark & glxgears are rather simple, so it's just a bit surprising.


- Eero

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Feb 8, 2017 11:15 AM, "Eero Tamminen"  wrote:

Hi,


On 08.02.2017 02:40, Marek Olšák wrote:

> I'm not concerned about anything and you missed that it had actually
> been tested to the extent that we can enable it for some existing games
> and at least one unreleased game already.
>
> I also don't have to be the guy merging it and I mean that. But if I do
> it, this is how I'd like to do it.
>
> From my point of view, we don't strictly have to make piglit work. We
> only have to make games that can benefit work.  Piglit can only help us
> get there more easily.
>

If there are known crashes, and no piglit statistics, how you can trust
that the games which the patch series seems to already help, actually do
work?  That they don't crash or deadlock at some point because of it?


I hate to repeat myself, but it was tested with quite a bunch of games. You
know, I actually made it work for Gallium including bug fixes in glthread
like bad locking and race conditions. I know Samuel Pitoiset also tested it
with good results. A bunch of people on IRC tested it too.

Concerning the glmark crash, I don't care. Get over it.

Marek





- Eero


___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Eero Tamminen]

Hi,

On 08.02.2017 02:40, Marek Olšák wrote:

I'm not concerned about anything and you missed that it had actually
been tested to the extent that we can enable it for some existing games
and at least one unreleased game already.

I also don't have to be the guy merging it and I mean that. But if I do
it, this is how I'd like to do it.

From my point of view, we don't strictly have to make piglit work. We
only have to make games that can benefit work.  Piglit can only help us
get there more easily.


If there are known crashes, and no piglit statistics, how you can trust 
that the games which the patch series seems to already help, actually do 
work?  That they don't crash or deadlock at some point because of it?



- Eero

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Gregory Hainaut]

>
> >* Hello James,
> *> > >* Did you have the opportunity to compare Feral's dispatcher threading
> *>* implementation versus Nvidia's threading option. If yes, is there any
> *>* performance difference? In other word, did you implement it because
> *>* some (most) drivers miss this optimization or because it can really
> *>* bring some performance improvements (when done carefully).
> *
> We can't directly compare to Nvidia's option, as our threaded GL
> implementation is a necessity for us. OpenGL contexts must be bound to
> a specific thread, unlike D3D devices, and we have games written for
> D3D using multiple threads to dispatch graphics work (and sometimes
> using multiple D3D devices from the same thread). We make one dispatch
> thread for each GL context to avoid having to unbind the context from a
> thread. There is some locking so that only one game thread may submit
> to a context's dispatch thread at once, but that locking is very
> lightweight compared to what would happen if we repeatedly attached and
> detached GL contexts on game threads. GL_KHR_context_flush_control was
> supposed to help with that, by allowing opting out of flushing the GL
> context when unbinding it from a thread, but last time we tried that,
> there were some synchronous X requests in SDL2 making it remain slow.
>
> Having said that, we did at one point have a path which used
> GLX_MESA_multithreaded_make_current when it was available. We found the
> GL dispatch threads was quicker on all the games we were working on at
> the time.
>
> James
>
>
Oh I see. I already got similar context threading issue when I ported my
app from Dx to GL. Thanks  for the full explanation (and for all games
ported on Linux ^^).
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Nicolai Hähnle]

On 07.02.2017 23:54, Matt Turner wrote:

On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:

On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke  wrote:

On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:

On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand  wrote:

FYI glmark2 segfaults with mesa_glthread=true. Expected that some programs
will segfault?


Yes, even segfaults are expected with mesa_glthread=true.

Marek


Would it make sense to be crash-free or even regression-free on at
least Piglit, before merging?  (Or are we there already?)


It's not necessary. glthread is disabled by default. Nobody has tested
piglit with glthread. That will follow after it's been merged, or
never if it's never merged.


I don't understand why you're so concerned about merging untested
code. That violates some pretty fundamental development practices of
the project.

It's exactly unfinished projects like this that cause problems and
inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
think it's a burden to develop something out of the master branch
until it's somewhat useful.


The code is already somewhat useful. Actually, make that _very_ useful 
(big performance improvement) for _some_ cases.


I suspect most of the people in this discussion haven't really looked at 
the code in detail (myself included). We should probably do some of that 
before it is merged, since the code isn't just a new driver that is 
isolated in its own directory. So I agree with Emil that it makes sense 
to let the patches go over the mailing list once.


However, it's fine to merge this without passing piglit.

Cheers,
Nicolai
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Feb 7, 2017 11:55 PM, "Matt Turner"  wrote:

On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke 
wrote:
>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand 
wrote:
>>> > FYI glmark2 segfaults with mesa_glthread=true. Expected that some
programs
>>> > will segfault?
>>>
>>> Yes, even segfaults are expected with mesa_glthread=true.
>>>
>>> Marek
>>
>> Would it make sense to be crash-free or even regression-free on at
>> least Piglit, before merging?  (Or are we there already?)
>
> It's not necessary. glthread is disabled by default. Nobody has tested
> piglit with glthread. That will follow after it's been merged, or
> never if it's never merged.

I don't understand why you're so concerned about merging untested
code. That violates some pretty fundamental development practices of
the project.

It's exactly unfinished projects like this that cause problems and
inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
think it's a burden to develop something out of the master branch
until it's somewhat useful.


I'm not concerned about anything and you missed that it had actually been
tested to the extent that we can enable it for some existing games and at
least one unreleased game already.

I also don't have to be the guy merging it and I mean that. But if I do it,
this is how I'd like to do it.

>From my point of view, we don't strictly have to make piglit work. We only
have to make games that can benefit work. Piglit can only help us get there
more easily.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Matt Turner]

On Tue, Feb 7, 2017 at 10:56 AM, Marek Olšák  wrote:
> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke  wrote:
>> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand  wrote:
>>> > FYI glmark2 segfaults with mesa_glthread=true. Expected that some programs
>>> > will segfault?
>>>
>>> Yes, even segfaults are expected with mesa_glthread=true.
>>>
>>> Marek
>>
>> Would it make sense to be crash-free or even regression-free on at
>> least Piglit, before merging?  (Or are we there already?)
>
> It's not necessary. glthread is disabled by default. Nobody has tested
> piglit with glthread. That will follow after it's been merged, or
> never if it's never merged.

I don't understand why you're so concerned about merging untested
code. That violates some pretty fundamental development practices of
the project.

It's exactly unfinished projects like this that cause problems and
inevitably have to be deleted later (ilo, openvg, d3d1x, etc). I don't
think it's a burden to develop something out of the master branch
until it's somewhat useful.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Timothy Arceri]

On Tue, 2017-02-07 at 10:56 +0100, Marek Olšák wrote:
> On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke  g> wrote:
> > On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
> > > On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand  > > > wrote:
> > > > FYI glmark2 segfaults with mesa_glthread=true. Expected that
> > > > some programs
> > > > will segfault?
> > > 
> > > Yes, even segfaults are expected with mesa_glthread=true.
> > > 
> > > Marek
> > 
> > Would it make sense to be crash-free or even regression-free on at
> > least Piglit, before merging?  (Or are we there already?)
> 
> It's not necessary. glthread is disabled by default. Nobody has
> tested
> piglit with glthread. That will follow after it's been merged, or
> never if it's never merged.

I've been trying to land shader-cache patches that actually do pass
piglit for over a year with the same reasoning that it will be disable
by default and can only be improved with testing I can't possibly do on
my own.

Although I have no objections to this being merged I'll be extremely
frustrated if this is allowed to be merged known to not even pass
piglit while I've wasted countless hours rebasing shader cache over
many months.


> 
> Marek
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[James Legg]

Hi Gregory,

On Tue, 2017-02-07 at 16:04 +0100, Gregory Hainaut wrote:
> > Hi,
> > 
> > On Mon, 2017-02-06 at 13:43 +0100, Jan Ziak wrote:
> > > Shadow of Mordor benchmark: 30 FPS w/o glthread -> 20 FPS with
> > > glthread
> > > 
> > 
> > For what it is worth, all the Feral games have a dispatch thread that
> > primarily calls GL functions.
> > 
> > James
> > 
> 
> Hello James,
> 
> 
> Did you have the opportunity to compare Feral's dispatcher threading
> implementation versus Nvidia's threading option. If yes, is there any
> performance difference? In other word, did you implement it because
> some (most) drivers miss this optimization or because it can really
> bring some performance improvements (when done carefully).

We can't directly compare to Nvidia's option, as our threaded GL
implementation is a necessity for us. OpenGL contexts must be bound to
a specific thread, unlike D3D devices, and we have games written for
D3D using multiple threads to dispatch graphics work (and sometimes
using multiple D3D devices from the same thread). We make one dispatch
thread for each GL context to avoid having to unbind the context from a
thread. There is some locking so that only one game thread may submit
to a context's dispatch thread at once, but that locking is very
lightweight compared to what would happen if we repeatedly attached and
detached GL contexts on game threads. GL_KHR_context_flush_control was
supposed to help with that, by allowing opting out of flushing the GL
context when unbinding it from a thread, but last time we tried that,
there were some synchronous X requests in SDL2 making it remain slow.

Having said that, we did at one point have a path which used
GLX_MESA_multithreaded_make_current when it was available. We found the
GL dispatch threads was quicker on all the games we were working on at
the time.

James
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Gregory Hainaut]

>
> Hi,
>
> On Mon, 2017-02-06 at 13:43 +0100, Jan Ziak wrote:
> >* Shadow of Mordor benchmark: 30 FPS w/o glthread -> 20 FPS with
> *>* glthread
> *>
> For what it is worth, all the Feral games have a dispatch thread that
> primarily calls GL functions.
>
> James
>
>
Hello James,


Did you have the opportunity to compare Feral's dispatcher threading
implementation versus Nvidia's threading option. If yes, is there any
performance difference? In other word, did you implement it because some
(most) drivers miss this optimization or because it can really bring some
performance improvements (when done carefully).


Thanks you,
Gregory
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Feb 7, 2017 12:54 PM, "Emil Velikov"  wrote:

On 7 February 2017 at 11:23, Jan Ziak <0xe2.0x9a.0...@gmail.com> wrote:
> On Mon, Feb 6, 2017 at 11:48 PM, Marek Olšák  wrote:
>> On Mon, Feb 6, 2017 at 9:27 PM, Jan Ziak <0xe2.0x9a.0...@gmail.com>
wrote:
>>> Hello
>>>
>>> I am against application profiles - in the form of "a
>>> community-maintained whitelist of apps" or in any other form
>>> explicitly associating the name/ID of an app with a Mesa variable
>>> which controls the behavior of Mesa.
>>>
>>> Application profiles would be a manifestation of poor algorithms in
>>> the OpenGL implementation.
>>
>> No, it's called incremental progress.
>>
>> The Mesa community doesn't have resources to develop a multithreaded
>> solution that is perfect from day 1. Incremental progress will get us
>> there eventually. Or not. But it's the only way to get somewhere with
>> our limited resources.
>>
>> Marek
>
> No. The point is: You are proposing a transient solution that isn't
> automated (i.e: a solution that isn't resembling machine learning in
> any way). https://en.wikipedia.org/wiki/Machine_learning
>
Have to agree with Marek here. An explicit opt-in solution would be
nice to start with considering our resources.

> Instead, why don't you introduce a simple C/C++ variable controlling
> whether to use glthread?
...
> The above solution would be totally simple to implement in Mesa. A
> human-maintained whitelist is clearly a subpar solution due to
> multiple factors.
>
Ideally we'll have something like this as we get to the point where no
crashes or rendering issues are present.
Until then this is not an option - it will deteriorate the user
experience very significantly.

Marek, please send the series to the list and _if_ you want guard the
lot behind a configure switch.
Alternatively a big warning message [when the variable is toggled]
might be nice idea ?


A configure switch isn't necessary. It's guarded by the drirc toggle
mesa_glthread, which is also an environment variable. The warning is
printed by the XML config code just like for any other driconf/drirc toggle.

Marek



Thanks
Emil
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Emil Velikov]

On 7 February 2017 at 11:23, Jan Ziak <0xe2.0x9a.0...@gmail.com> wrote:
> On Mon, Feb 6, 2017 at 11:48 PM, Marek Olšák  wrote:
>> On Mon, Feb 6, 2017 at 9:27 PM, Jan Ziak <0xe2.0x9a.0...@gmail.com> wrote:
>>> Hello
>>>
>>> I am against application profiles - in the form of "a
>>> community-maintained whitelist of apps" or in any other form
>>> explicitly associating the name/ID of an app with a Mesa variable
>>> which controls the behavior of Mesa.
>>>
>>> Application profiles would be a manifestation of poor algorithms in
>>> the OpenGL implementation.
>>
>> No, it's called incremental progress.
>>
>> The Mesa community doesn't have resources to develop a multithreaded
>> solution that is perfect from day 1. Incremental progress will get us
>> there eventually. Or not. But it's the only way to get somewhere with
>> our limited resources.
>>
>> Marek
>
> No. The point is: You are proposing a transient solution that isn't
> automated (i.e: a solution that isn't resembling machine learning in
> any way). https://en.wikipedia.org/wiki/Machine_learning
>
Have to agree with Marek here. An explicit opt-in solution would be
nice to start with considering our resources.

> Instead, why don't you introduce a simple C/C++ variable controlling
> whether to use glthread?
...
> The above solution would be totally simple to implement in Mesa. A
> human-maintained whitelist is clearly a subpar solution due to
> multiple factors.
>
Ideally we'll have something like this as we get to the point where no
crashes or rendering issues are present.
Until then this is not an option - it will deteriorate the user
experience very significantly.

Marek, please send the series to the list and _if_ you want guard the
lot behind a configure switch.
Alternatively a big warning message [when the variable is toggled]
might be nice idea ?

Thanks
Emil
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Eero Tamminen]

Hi,

On 07.02.2017 13:23, Jan Ziak wrote:

On Mon, Feb 6, 2017 at 11:48 PM, Marek Olšák  wrote:

On Mon, Feb 6, 2017 at 9:27 PM, Jan Ziak <0xe2.0x9a.0...@gmail.com> wrote:

Hello

I am against application profiles - in the form of "a
community-maintained whitelist of apps" or in any other form
explicitly associating the name/ID of an app with a Mesa variable
which controls the behavior of Mesa.

Application profiles would be a manifestation of poor algorithms in
the OpenGL implementation.


No, it's called incremental progress.

The Mesa community doesn't have resources to develop a multithreaded
solution that is perfect from day 1. Incremental progress will get us
there eventually. Or not. But it's the only way to get somewhere with
our limited resources.


No. The point is: You are proposing a transient solution that isn't
automated (i.e: a solution that isn't resembling machine learning in
any way). https://en.wikipedia.org/wiki/Machine_learning

Instead, why don't you introduce a simple C/C++ variable controlling
whether to use glthread? The variable needs to be initially fed with
simple statistics about whether glthread has positive or negative
effect on performance. The simple statistics can be obtained from
alternate frame rendering:


Those kind of statistics might change from run-to-run (due to background 
load and other factors).


"randomly" enabling something that is (at least currently) as likely to 
segfault the program as not, instead of it being explicitly controlled, 
sounds pretty awful.


After threaded dispatch has been extensively tested (piglit, many games, 
valgrind etc) and there are no know issues, maybe then it could be 
considered.



- Eero


// Machine learning
bool use_thread = 0;
float fps[2] = {};
for i=0; i<1000; i++ {
  if use_thread
render the frame with glthread=on
  else
render the frame with glthread=off
  fps[use_thread] += 1.0 / frame_time;
  use_thread = 1 - use_thread;
}
const bool glthread_means_higher_fps = (fps[1] > fps[0]);

// Using the learned information
while(1) {
  if glthread_means_higher_fps
render the frame with glthread=on
  else
render the frame with glthread=off
}

The above solution would be totally simple to implement in Mesa. A
human-maintained whitelist is clearly a subpar solution due to
multiple factors.

Jan
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev



___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Jan Ziak]

On Mon, Feb 6, 2017 at 11:48 PM, Marek Olšák  wrote:
> On Mon, Feb 6, 2017 at 9:27 PM, Jan Ziak <0xe2.0x9a.0...@gmail.com> wrote:
>> Hello
>>
>> I am against application profiles - in the form of "a
>> community-maintained whitelist of apps" or in any other form
>> explicitly associating the name/ID of an app with a Mesa variable
>> which controls the behavior of Mesa.
>>
>> Application profiles would be a manifestation of poor algorithms in
>> the OpenGL implementation.
>
> No, it's called incremental progress.
>
> The Mesa community doesn't have resources to develop a multithreaded
> solution that is perfect from day 1. Incremental progress will get us
> there eventually. Or not. But it's the only way to get somewhere with
> our limited resources.
>
> Marek

No. The point is: You are proposing a transient solution that isn't
automated (i.e: a solution that isn't resembling machine learning in
any way). https://en.wikipedia.org/wiki/Machine_learning

Instead, why don't you introduce a simple C/C++ variable controlling
whether to use glthread? The variable needs to be initially fed with
simple statistics about whether glthread has positive or negative
effect on performance. The simple statistics can be obtained from
alternate frame rendering:

// Machine learning
bool use_thread = 0;
float fps[2] = {};
for i=0; i<1000; i++ {
  if use_thread
render the frame with glthread=on
  else
render the frame with glthread=off
  fps[use_thread] += 1.0 / frame_time;
  use_thread = 1 - use_thread;
}
const bool glthread_means_higher_fps = (fps[1] > fps[0]);

// Using the learned information
while(1) {
  if glthread_means_higher_fps
render the frame with glthread=on
  else
render the frame with glthread=off
}

The above solution would be totally simple to implement in Mesa. A
human-maintained whitelist is clearly a subpar solution due to
multiple factors.

Jan
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Tue, Feb 7, 2017 at 2:57 AM, Kenneth Graunke  wrote:
> On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
>> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand  wrote:
>> > FYI glmark2 segfaults with mesa_glthread=true. Expected that some programs
>> > will segfault?
>>
>> Yes, even segfaults are expected with mesa_glthread=true.
>>
>> Marek
>
> Would it make sense to be crash-free or even regression-free on at
> least Piglit, before merging?  (Or are we there already?)

It's not necessary. glthread is disabled by default. Nobody has tested
piglit with glthread. That will follow after it's been merged, or
never if it's never merged.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Kenneth Graunke]

On Monday, February 6, 2017 8:54:40 PM PST Marek Olšák wrote:
> On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand  wrote:
> > FYI glmark2 segfaults with mesa_glthread=true. Expected that some programs
> > will segfault?
> 
> Yes, even segfaults are expected with mesa_glthread=true.
> 
> Marek

Would it make sense to be crash-free or even regression-free on at
least Piglit, before merging?  (Or are we there already?)

--Ken


signature.asc
Description: This is a digitally signed message part.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Mon, Feb 6, 2017 at 9:27 PM, Jan Ziak <0xe2.0x9a.0...@gmail.com> wrote:
> Hello
>
> I am against application profiles - in the form of "a
> community-maintained whitelist of apps" or in any other form
> explicitly associating the name/ID of an app with a Mesa variable
> which controls the behavior of Mesa.
>
> Application profiles would be a manifestation of poor algorithms in
> the OpenGL implementation.

No, it's called incremental progress.

The Mesa community doesn't have resources to develop a multithreaded
solution that is perfect from day 1. Incremental progress will get us
there eventually. Or not. But it's the only way to get somewhere with
our limited resources.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Jan Ziak]

Hello

I am against application profiles - in the form of "a
community-maintained whitelist of apps" or in any other form
explicitly associating the name/ID of an app with a Mesa variable
which controls the behavior of Mesa.

Application profiles would be a manifestation of poor algorithms in
the OpenGL implementation.

Application profiles would hinder entry of new players into the Linux
OpenGL application market, because their applications would be at the
disadvantage of not being in the whitelist. Application profiles would
make competition (economics) less effective.

A solution superior to application profiles is to analyze
application's OpenGL usage patterns and make optimization decisions
based on those patterns. The OpenGL implementation (Mesa) can store
the patterns and computed values in a disk cache to avoid
recomputation if the app is started again.

Jan

> Yes, I'm aware that glthread is far from perfect. However, I don't consider
> that an issue. My idea is that the actual work will take place in master. I
> have zero faith that any work on that will take place outside of master.
>
> Currently I don't expect it to work with any GL4 apps, because the threaded
> dispatch isn't aware of many GL4 functions. Initially we'll have a
> community-maintained whitelist of apps benefitting from glthread.
>
> Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

On Mon, Feb 6, 2017 at 8:20 PM, Ernst Sjöstrand  wrote:
> FYI glmark2 segfaults with mesa_glthread=true. Expected that some programs
> will segfault?

Yes, even segfaults are expected with mesa_glthread=true.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Ernst Sjöstrand]

FYI glmark2 segfaults with mesa_glthread=true. Expected that some programs
will segfault?

ATTENTION: default value of option mesa_glthread overridden by environment.
[New Thread 0x7fffed73d700 (LWP 23060)]
_mesa_glthread_init
===
glmark2 2014.03
===
OpenGL Information
GL_VENDOR: X.Org
GL_RENDERER:   Gallium 0.4 on AMD FIJI (DRM 3.9.0 / 4.10.0-rc3+, LLVM
5.0.0)
GL_VERSION:3.0 Mesa 17.1.0-devel (git-c91d721)
===
[New Thread 0x7fffecf3c700 (LWP 23061)]
_mesa_glthread_init
_mesa_glthread_destroy
[Thread 0x7fffed73d700 (LWP 23060) exited]

Thread 1 "glmark2" received signal SIGSEGV, Segmentation fault.

Here's the backtrace:
http://pastebin.com/0FrM0Q0A

Regards
//Ernst


2017-02-06 1:11 GMT+01:00 Marek Olšák :

> Hi,
>
> Back in 2012-2013, then-Intel employees Eric Anholt and Paul Berry
> wrote this threaded GL dispatch where GL calls are queued and executed
> in a different thread. It was supposed to deal with high CPU overhead
> of Mesa, but at the time most games used the compatibility profile and
> Steam didn't really exist on Linux, so it didn't help many (if any)
> apps.
>
> Things are different today. We have Steam and most games use the GL
> core profile. We know of several games that have better performance
> with glthread, namely Borderlands 2, and some people reported to me
> that some other games also benefit. It's about time we put this into
> mainline Mesa.
>
> My plan is that we merge it as-is or with minor changes, and then
> we'll clean it up and improve it while it's in master. It's disabled
> by default, so it shouldn't bother anyone who doesn't want it. There
> is a drirc option to turn it on (just use the corresponding env var).
> All Gallium drivers support it.
>
> A note on synchronizations. Borderlands 2 has 170 thread syncs per
> frame. That means the app thread has to stop and wait 170x per frame.
> Despite that, it still has 70% higher performance in some cases. My
> theory is that if you have a lot of draw calls, you can have a lot of
> syncs, because the sheer amount of draw calls will just make those
> syncs irrelevant. 200 syncs per 4k draw calls is like 1 sync per 20
> draw calls.
>
> Here it is: https://cgit.freedesktop.org/~mareko/mesa/log/?h=glthread
>
> The plan is to merge everything up to the gallium commit (without the
> Intel commits, I'll let Intel decide what to do with them). I can send
> the whole series to the list if that's preferable.
>
> Opinions?
>
> Marek
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Eric Anholt]

Marek Olšák  writes:

> Yes, I'm aware that glthread is far from perfect. However, I don't consider
> that an issue. My idea is that the actual work will take place in master. I
> have zero faith that any work on that will take place outside of master.

Yeah.  Now that there's some evidence it's useful, let's work on getting
it merged and then fix it up more once it's in tree.


signature.asc
Description: PGP signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Emil Velikov]

On 6 February 2017 at 00:11, Marek Olšák  wrote:
> Hi,
>
> Back in 2012-2013, then-Intel employees Eric Anholt and Paul Berry
> wrote this threaded GL dispatch where GL calls are queued and executed
> in a different thread. It was supposed to deal with high CPU overhead
> of Mesa, but at the time most games used the compatibility profile and
> Steam didn't really exist on Linux, so it didn't help many (if any)
> apps.
>
> Things are different today. We have Steam and most games use the GL
> core profile. We know of several games that have better performance
> with glthread, namely Borderlands 2, and some people reported to me
> that some other games also benefit. It's about time we put this into
> mainline Mesa.
>
> My plan is that we merge it as-is or with minor changes, and then
> we'll clean it up and improve it while it's in master. It's disabled
> by default, so it shouldn't bother anyone who doesn't want it. There
> is a drirc option to turn it on (just use the corresponding env var).
> All Gallium drivers support it.
>
> A note on synchronizations. Borderlands 2 has 170 thread syncs per
> frame. That means the app thread has to stop and wait 170x per frame.
> Despite that, it still has 70% higher performance in some cases. My
> theory is that if you have a lot of draw calls, you can have a lot of
> syncs, because the sheer amount of draw calls will just make those
> syncs irrelevant. 200 syncs per 4k draw calls is like 1 sync per 20
> draw calls.
>
> Here it is: https://cgit.freedesktop.org/~mareko/mesa/log/?h=glthread
>
> The plan is to merge everything up to the gallium commit (without the
> Intel commits, I'll let Intel decide what to do with them). I can send
> the whole series to the list if that's preferable.
>
Please send the lot to the list - it would be mostly an "ack, fine
with me" case.
Do CC Mauro for the patches that affect build systems, so he can
address the Android side.

Thanks
Emil
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[James Legg]

Hi,

On Mon, 2017-02-06 at 13:43 +0100, Jan Ziak wrote:
> Shadow of Mordor benchmark: 30 FPS w/o glthread -> 20 FPS with
> glthread
> 

For what it is worth, all the Feral games have a dispatch thread that
primarily calls GL functions.

James
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Steven Newbury]

On Mon, 2017-02-06 at 01:11 +0100, Marek Olšák wrote:
> Hi,
> 
> Back in 2012-2013, then-Intel employees Eric Anholt and Paul Berry
> wrote this threaded GL dispatch where GL calls are queued and
> executed
> in a different thread. It was supposed to deal with high CPU overhead
> of Mesa, but at the time most games used the compatibility profile
> and
> Steam didn't really exist on Linux, so it didn't help many (if any)
> apps.
> 
> Things are different today. We have Steam and most games use the GL
> core profile. We know of several games that have better performance
> with glthread, namely Borderlands 2, and some people reported to me
> that some other games also benefit. It's about time we put this into
> mainline Mesa.
> 
> My plan is that we merge it as-is or with minor changes, and then
> we'll clean it up and improve it while it's in master. It's disabled
> by default, so it shouldn't bother anyone who doesn't want it. There
> is a drirc option to turn it on (just use the corresponding env var).
> All Gallium drivers support it.
> 
> A note on synchronizations. Borderlands 2 has 170 thread syncs per
> frame. That means the app thread has to stop and wait 170x per frame.
> Despite that, it still has 70% higher performance in some cases. My
> theory is that if you have a lot of draw calls, you can have a lot of
> syncs, because the sheer amount of draw calls will just make those
> syncs irrelevant. 200 syncs per 4k draw calls is like 1 sync per 20
> draw calls.
> 
> Here it is: https://cgit.freedesktop.org/~mareko/mesa/log/?h=glthread
> 
> The plan is to merge everything up to the gallium commit (without the
> Intel commits, I'll let Intel decide what to do with them). I can
> send
> the whole series to the list if that's preferable.
> 
> Opinions?
Despite not requesting merge of the Intel commits, I took a look, and
they're (mis)applied to variously i915/i965 where previously there was
the "intel" driver directory.  Presumably i965 is the correct driver to
be applying the changes to and I've reworked the patches accordingly.

Will test them shortly.


signature.asc
Description: This is a digitally signed message part
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

Yes, I'm aware that glthread is far from perfect. However, I don't consider
that an issue. My idea is that the actual work will take place in master. I
have zero faith that any work on that will take place outside of master.

Currently I don't expect it to work with any GL4 apps, because the threaded
dispatch isn't aware of many GL4 functions. Initially we'll have a
community-maintained whitelist of apps benefitting from glthread.

Marek


On Feb 6, 2017 11:10 AM, "Gregory Hainaut" 
wrote:

Hello,

> A note on synchronizations. Borderlands 2 has 170 thread syncs per
> frame. That means the app thread has to stop and wait 170x per frame.
> Despite that, it still has 70% higher performance in some cases. My
> theory is that if you have a lot of draw calls, you can have a lot of
> syncs, because the sheer amount of draw calls will just make those
> syncs irrelevant. 200 syncs per 4k draw calls is like 1 sync per 20
> draw calls.

Here a feedback of my quick test

On PCSX2 (PS2 emulator), I noticed that synchronization badly impacts
the perf. In my case, there are mostly related to texture transfer
(CPU->GPU) and clear buffer functions. Strangely I didn't notice
anything related to BufferSubData* but I guess it is the same.

Those functions trigger a sync because of the pointer parameter.
However texture transfer could use a PBO so it isn't a real pointer.
And clear uses a pointer to a color hence a small payload (worst case
is likely around 16/32B). IMHO, it can surely be inlined/memcpy in the
gl dispatcher (otherwise the old GL2 clear API is sync free).

I hacked the code to remove the sync on texture transfer and I got a
major speed boost. I didn't count the number of draw call neither sync
ratio. But I suspect that perf impact could depends on the sync
repartition. Unlike me, I guess that Borderlands2 uploads/clears
buffers/textures/uniform at the start of the frame. Which mean various
small sync at the start of the frame (which might be optimized as a
spin lock). Therefore the hot rendering loop might be sync free hence
the speed boost.

To conclude, based on my single testcase, current state of the code
isn't yet optimal and it might explain why few apps see any perf
improvement so far. But the potential is here.

Cheers,
Gregory
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Jan Ziak]

Hello

Just some notes for mesa_glthread=true:

- Shadow of Mordor benchmark: 30 FPS w/o glthread -> 20 FPS with glthread

- fgl_glxgears: SIGSEGV with mesa_glthread=true
(gdb) bt
#0  dlist_alloc (ctx=ctx@entry=0x78b220,
opcode=opcode@entry=OPCODE_CLEAR, bytes=bytes@entry=4, align8=false)
   at 
/var/tmp/portage/media-libs/mesa-/work/mesa-/src/mesa/main/dlist.c:1390
#1  0x76c3fc19 in alloc_instruction (nparams=1,
opcode=OPCODE_CLEAR, ctx=0x78b220)
   at 
/var/tmp/portage/media-libs/mesa-/work/mesa-/src/mesa/main/dlist.c:1474
#2  save_Clear (mask=16640) at
/var/tmp/portage/media-libs/mesa-/work/mesa-/src/mesa/main/dlist.c:1933
#3  0x00404509 in drawGears ()
#4  0x0040304a in event_loop ()
#5  0x00402279 in main ()

Jan
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

[Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Gregory Hainaut]

Hello,

> A note on synchronizations. Borderlands 2 has 170 thread syncs per
> frame. That means the app thread has to stop and wait 170x per frame.
> Despite that, it still has 70% higher performance in some cases. My
> theory is that if you have a lot of draw calls, you can have a lot of
> syncs, because the sheer amount of draw calls will just make those
> syncs irrelevant. 200 syncs per 4k draw calls is like 1 sync per 20
> draw calls.

Here a feedback of my quick test

On PCSX2 (PS2 emulator), I noticed that synchronization badly impacts
the perf. In my case, there are mostly related to texture transfer
(CPU->GPU) and clear buffer functions. Strangely I didn't notice
anything related to BufferSubData* but I guess it is the same.

Those functions trigger a sync because of the pointer parameter.
However texture transfer could use a PBO so it isn't a real pointer.
And clear uses a pointer to a color hence a small payload (worst case
is likely around 16/32B). IMHO, it can surely be inlined/memcpy in the
gl dispatcher (otherwise the old GL2 clear API is sync free).

I hacked the code to remove the sync on texture transfer and I got a
major speed boost. I didn't count the number of draw call neither sync
ratio. But I suspect that perf impact could depends on the sync
repartition. Unlike me, I guess that Borderlands2 uploads/clears
buffers/textures/uniform at the start of the frame. Which mean various
small sync at the start of the frame (which might be optimized as a
spin lock). Therefore the hot rendering loop might be sync free hence
the speed boost.

To conclude, based on my single testcase, current state of the code
isn't yet optimal and it might explain why few apps see any perf
improvement so far. But the potential is here.

Cheers,
Gregory
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

[Mesa-dev] Time to merge threaded GL dispatch? (aka glthread)

[Marek Olšák]

Hi,

Back in 2012-2013, then-Intel employees Eric Anholt and Paul Berry
wrote this threaded GL dispatch where GL calls are queued and executed
in a different thread. It was supposed to deal with high CPU overhead
of Mesa, but at the time most games used the compatibility profile and
Steam didn't really exist on Linux, so it didn't help many (if any)
apps.

Things are different today. We have Steam and most games use the GL
core profile. We know of several games that have better performance
with glthread, namely Borderlands 2, and some people reported to me
that some other games also benefit. It's about time we put this into
mainline Mesa.

My plan is that we merge it as-is or with minor changes, and then
we'll clean it up and improve it while it's in master. It's disabled
by default, so it shouldn't bother anyone who doesn't want it. There
is a drirc option to turn it on (just use the corresponding env var).
All Gallium drivers support it.

A note on synchronizations. Borderlands 2 has 170 thread syncs per
frame. That means the app thread has to stop and wait 170x per frame.
Despite that, it still has 70% higher performance in some cases. My
theory is that if you have a lot of draw calls, you can have a lot of
syncs, because the sheer amount of draw calls will just make those
syncs irrelevant. 200 syncs per 4k draw calls is like 1 sync per 20
draw calls.

Here it is: https://cgit.freedesktop.org/~mareko/mesa/log/?h=glthread

The plan is to merge everything up to the gallium commit (without the
Intel commits, I'll let Intel decide what to do with them). I can send
the whole series to the list if that's preferable.

Opinions?

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev