Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[⚛]

On Mon, Jun 6, 2016 at 12:40 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
> I apologize for the confusion. I won't happen again.

Typo: "I won't ..." -> "It won't ..."
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[⚛]

On Wed, Jun 1, 2016 at 7:07 PM, Marek Olšák  wrote:
> On Wed, Jun 1, 2016 at 6:06 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
>> On Wed, Jun 1, 2016 at 3:53 PM, Marek Olšák  wrote:
>>> Because of external factors you can't predict, your driver suddenly
>>> receives a bunch of shaders that take 2000 ms to compile right before
>>> a draw call. Your budget is 16 ms per frame to get 30 fps, but you
>>> can't render the frame if you don't compile those shaders. The problem
>>> is how to fit the compilation that takes 2000 ms and is required
>>> render the frame into 16 ms. Can you see where I'm going?
>>
>> I don't understand why transforming parts of shader compilation into
>> asynchronous code isn't the right way to go. Async parts of
>> compilation can potentially run on other CPU cores, maybe lowering
>> those 2000ms down to 750ms.
>
> No. Shader compilation can only be asynchronous if it's far enough
> from a draw call and the app doesn't query its status. If it's next to
> a draw call, multithreading is useless. Completely useless.
>
> We need to get below 33 ms for all shaders needed to be compiled to
> render a frame. If there are 10 VS and 10 PS, one shader must be
> compiled within 1.65 ms on average. I don't see where your random
> guess meets that goal.

Ok, now I understand where the primary problem is: I am at least
several years ahead of you in this.

I apologize for the confusion. I won't happen again. I am leaving this
mailing list.

Farewell.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Matt Turner]

On Wed, Jun 1, 2016 at 8:51 AM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
> On Wed, Jun 1, 2016 at 3:50 PM, Erik Faye-Lund  wrote:
>> On Wed, Jun 1, 2016 at 3:02 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
>>> On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:
 I'll let you figure it out by yourself.
>>>
>>> Why would you withhold information if you already have it? Are you a
>>> "bad person" or something?
>>
>> The problem has been described in countless threads on this mailing
>> list, but I'll repeat it:
>
> Strange. If it has already been discussed many times then why isn't
> the problem solved yet?

By that metric world peace should have been solved long ago.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Marek Olšák]

On Wed, Jun 1, 2016 at 7:40 PM, Patrick Baggett
 wrote:
>>
>>
>> No. Shader compilation can only be asynchronous if it's far enough
>> from a draw call and the app doesn't query its status. If it's next to
>> a draw call, multithreading is useless. Completely useless.
>>

I take my statement back. Multithreading can be useful for compiling
VS and FS in parallel when it's next to a draw call.

>
> I don't know a lot about the shader compilation/linking process, so
> I'm just asking this for my own benefit.
>
> I read that the optimizations take a long time. Is it possible to
> create a sort of -O0 version of the shader while the real version is
> generated by some thread pool? Or would there be some shaders that
> would just fail to run unless optimization took place (and the
> developers count on that)?

The latter. Register allocation is usually required for being able to
run a shader.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Patrick Baggett]

>
>
> No. Shader compilation can only be asynchronous if it's far enough
> from a draw call and the app doesn't query its status. If it's next to
> a draw call, multithreading is useless. Completely useless.
>

I don't know a lot about the shader compilation/linking process, so
I'm just asking this for my own benefit.

I read that the optimizations take a long time. Is it possible to
create a sort of -O0 version of the shader while the real version is
generated by some thread pool? Or would there be some shaders that
would just fail to run unless optimization took place (and the
developers count on that)?

> We need to get below 33 ms for all shaders needed to be compiled to
> render a frame. If there are 10 VS and 10 PS, one shader must be
> compiled within 1.65 ms on average. I don't see where your random
> guess meets that goal.
>
> Marek
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Marek Olšák]

On Wed, Jun 1, 2016 at 6:06 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
> On Wed, Jun 1, 2016 at 3:53 PM, Marek Olšák  wrote:
>> Because of external factors you can't predict, your driver suddenly
>> receives a bunch of shaders that take 2000 ms to compile right before
>> a draw call. Your budget is 16 ms per frame to get 30 fps, but you
>> can't render the frame if you don't compile those shaders. The problem
>> is how to fit the compilation that takes 2000 ms and is required
>> render the frame into 16 ms. Can you see where I'm going?
>
> I don't understand why transforming parts of shader compilation into
> asynchronous code isn't the right way to go. Async parts of
> compilation can potentially run on other CPU cores, maybe lowering
> those 2000ms down to 750ms.

No. Shader compilation can only be asynchronous if it's far enough
from a draw call and the app doesn't query its status. If it's next to
a draw call, multithreading is useless. Completely useless.

We need to get below 33 ms for all shaders needed to be compiled to
render a frame. If there are 10 VS and 10 PS, one shader must be
compiled within 1.65 ms on average. I don't see where your random
guess meets that goal.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Emil Velikov]

On 1 June 2016 at 17:34, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
> On Wed, Jun 1, 2016 at 4:06 PM, Brian Paul  wrote:
>> I think the issue here is someone comes along with no history in the project
>> and asserts that he has a great solution for a problem
>
> Asynchronous computation is an essential part of the solution to the
> problem (problem == faster compiler response time in Mesa).
>
>> while developers with
>> many years of experience know it's not that simple.
>
> I never wrote it was simple.
>
True you didn't. Yet the method you presented it implied it is that
simple. Being more explicit in what you mean will do you more good
than bad.

>> It takes time to
>> explain all this and most of us are very busy with other tasks.
>
> I agree. I don't have time to explain to you in detail why it will work.
>
>> Furthermore, people sometimes interpret no response to their ideas as
>> implicit endorsement when that's not the case at all.  Better for the
>> experienced people to give a quick no/nak than to say nothing and let
>> someone assume that his idea is supported.
>
> Ahh, so you feel superior. ... I am wondering how long is that feeling
> going to last.
>
>> I suspect that what Marek was doing.
>
> I don't know.
>
>> Finally, I think some of us have a hard time taking people seriously when
>> they don't identify themselves with their real name.
>
> You will have to make an exception in my case.
Dude will kindly ask you to calm down. Similar to your account request
a while you're acting immature and provocative.

Brian is one of the most reasonable and helpful people around here. I
believe we all owe his some respect if it wasn't for his skills and
contributions then at least for starting the whole project.

If you find such replies annoying/demanding/etc. I would suggest
taking a short break and replying at a later stage. Nobody is gaining
anything in such discussions.

Thanks
Emil
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[⚛]

On Wed, Jun 1, 2016 at 4:06 PM, Brian Paul  wrote:
> I think the issue here is someone comes along with no history in the project
> and asserts that he has a great solution for a problem

Asynchronous computation is an essential part of the solution to the
problem (problem == faster compiler response time in Mesa).

> while developers with
> many years of experience know it's not that simple.

I never wrote it was simple.

> It takes time to
> explain all this and most of us are very busy with other tasks.

I agree. I don't have time to explain to you in detail why it will work.

> Furthermore, people sometimes interpret no response to their ideas as
> implicit endorsement when that's not the case at all.  Better for the
> experienced people to give a quick no/nak than to say nothing and let
> someone assume that his idea is supported.

Ahh, so you feel superior. ... I am wondering how long is that feeling
going to last.

> I suspect that what Marek was doing.

I don't know.

> Finally, I think some of us have a hard time taking people seriously when
> they don't identify themselves with their real name.

You will have to make an exception in my case.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[⚛]

On Wed, Jun 1, 2016 at 3:53 PM, Marek Olšák  wrote:
> Because of external factors you can't predict, your driver suddenly
> receives a bunch of shaders that take 2000 ms to compile right before
> a draw call. Your budget is 16 ms per frame to get 30 fps, but you
> can't render the frame if you don't compile those shaders. The problem
> is how to fit the compilation that takes 2000 ms and is required
> render the frame into 16 ms. Can you see where I'm going?

I don't understand why transforming parts of shader compilation into
asynchronous code isn't the right way to go. Async parts of
compilation can potentially run on other CPU cores, maybe lowering
those 2000ms down to 750ms.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[⚛]

On Wed, Jun 1, 2016 at 3:50 PM, Erik Faye-Lund  wrote:
> On Wed, Jun 1, 2016 at 3:02 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
>> On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:
>>> I'll let you figure it out by yourself.
>>
>> Why would you withhold information if you already have it? Are you a
>> "bad person" or something?
>
> The problem has been described in countless threads on this mailing
> list, but I'll repeat it:

Strange. If it has already been discussed many times then why isn't
the problem solved yet?

> The vast majority of applications/games does glCompileShader()
> immediately followed by glGetShaderiv(GL_COMPILE_STATUS), which would
> force stalling for the result, leading to no improvement.

With asynchronous computations you can return
glGetShaderiv(GL_COMPILE_STATUS) as soon as the compiler determines
that the source code is valid. The source code is valid if it passes
the lexer, the parser, the type checker, as well as all checkers that
have no established formal name.

Optimization is a transformation that fully preserves the original
program semantics and aside from bugs optimization has no influence on
GL_COMPILE_STATUS.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Eero Tamminen]

Hi,

On 01.06.2016 18:02, Jason Ekstrand wrote:

On Jun 1, 2016 7:43 AM, "Eero Tamminen" > wrote:
 > Startup time may not be a problem for games (some gamers may
disagree), but there are also devices which have legal / certification
requirements for startup times, where shader compilation time could be a
problem, especially as Mesa shader compilation tends to get slower over
time (as compiler does more optimizations).

Again, a problem with a different solution: Properly implementing
glGetProgramBinary which should be pretty easy once the Timothy is done
with the shader cache.

 > Those devices have pretty static content (at least for startup), so
their shaders could be pre-compiled if 3D driver supports that and has
offline compiler.  Many proprietary drivers do, Mesa doesn't, at least
not yet.

You don't need an offline compiler.  The developer of said app just
needs to regenerate their program binaries after the update the driver.

>

Not trying to be negative here.  Just pointing out that delayed
compilation isn't the best solution to that problem either.


By offline I meant something that compiles the shader for another 
architecture than what the build machine uses (which is often the case 
with embedded devices), and preferably something that can be automated 
as part of the whole device SW stack build.



- Eero

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Marek Olšák]

On Wed, Jun 1, 2016 at 4:58 PM, Eero Tamminen 
wrote:

> Hi,
>
> On 01.06.2016 16:53, Marek Olšák wrote:
>
>> On Wed, Jun 1, 2016 at 3:02 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
>>
>>> On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:
>>>
 I'll let you figure it out by yourself.

>>>
>>> Why would you withhold information if you already have it? Are you a
>>> "bad person" or something?
>>>
>>> As far as my memory goes, I have never posted the sentence "I'll let
>>> you figure it out by yourself" to the Internet because I believe it is
>>> wrong.
>>>
>>
>> :)
>>
>> Shader compilation at game loading time is never a problem, so we can
>> ignore that.
>>
>
> Startup time may not be a problem for games (some gamers may disagree),
> but there are also devices which have legal / certification requirements
> for startup times, where shader compilation time could be a problem,
> especially as Mesa shader compilation tends to get slower over time (as
> compiler does more optimizations).
>
> Those devices have pretty static content (at least for startup), so their
> shaders could be pre-compiled if 3D driver supports that and has offline
> compiler.  Many proprietary drivers do, Mesa doesn't, at least not yet.
>
>
> Shader compilation right before draw calls is what's unpleasant and
>> when people talk about it in the negative sense, they mean this.
>>
>> Because of external factors you can't predict, your driver suddenly
>> receives a bunch of shaders that take 2000 ms to compile right before
>> a draw call. Your budget is 16 ms per frame to get 30 fps, but you
>> can't render the frame if you don't compile those shaders. The problem
>> is how to fit the compilation that takes 2000 ms and is required
>> render the frame into 16 ms. Can you see where I'm going?
>>
>
> Besides application itself compiling a shader, some state change done by
> application may also trigger shader recompile in Mesa.  While for
> application requested compilation nothing can be done to squeeze it into
> 16ms, there are some chances of reducing need for state-change triggered
> recompiles (and these improvements have been done to Mesa during the years).


radeonsi doesn't do state-change triggered recompiles since LLVM 3.8. I
think nv50/nvc0 doesn't do that either, though I do understand this is a
very serious issue with other drivers.

The real problem for us (radeonsi specifically) is that some apps really
load shaders right before they wanna use them.

Marek

PS: My mistake, 16 ms is a requirement for 60 fps, not 30.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Jason Ekstrand]

On Jun 1, 2016 7:43 AM, "Eero Tamminen"  wrote:
>
> Hi,
>
>
> On 01.06.2016 16:53, Marek Olšák wrote:
>>
>> On Wed, Jun 1, 2016 at 3:02 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
>>>
>>> On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:

 I'll let you figure it out by yourself.
>>>
>>>
>>> Why would you withhold information if you already have it? Are you a
>>> "bad person" or something?
>>>
>>> As far as my memory goes, I have never posted the sentence "I'll let
>>> you figure it out by yourself" to the Internet because I believe it is
>>> wrong.
>>
>>
>> :)
>>
>> Shader compilation at game loading time is never a problem, so we can
>> ignore that.
>
>
> Startup time may not be a problem for games (some gamers may disagree),
but there are also devices which have legal / certification requirements
for startup times, where shader compilation time could be a problem,
especially as Mesa shader compilation tends to get slower over time (as
compiler does more optimizations).

Again, a problem with a different solution: Properly implementing
glGetProgramBinary which should be pretty easy once the Timothy is done
with the shader cache.

> Those devices have pretty static content (at least for startup), so their
shaders could be pre-compiled if 3D driver supports that and has offline
compiler.  Many proprietary drivers do, Mesa doesn't, at least not yet.

You don't need an offline compiler.  The developer of said app just needs
to regenerate their program binaries after the update the driver.

Not trying to be negative here.  Just pointing out that delayed compilation
isn't the best solution to that problem either.

At one point in the past, I did consider a crazy scheme in which the driver
did a quick-and-dirty compile up-front and then did a longer, better
optimized, compile in the background.  That idea may still have some merit,
bit it's going to take a *lot* more plumbing in the front and back ends
than just putting std::future on a field in gl_program.

>> Shader compilation right before draw calls is what's unpleasant and
>> when people talk about it in the negative sense, they mean this.
>>
>> Because of external factors you can't predict, your driver suddenly
>> receives a bunch of shaders that take 2000 ms to compile right before
>> a draw call. Your budget is 16 ms per frame to get 30 fps, but you
>> can't render the frame if you don't compile those shaders. The problem
>> is how to fit the compilation that takes 2000 ms and is required
>> render the frame into 16 ms. Can you see where I'm going?
>
>
> Besides application itself compiling a shader, some state change done by
application may also trigger shader recompile in Mesa.  While for
application requested compilation nothing can be done to squeeze it into
16ms, there are some chances of reducing need for state-change triggered
recompiles (and these improvements have been done to Mesa during the years).
>
>
> - Eero
>
>
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Eero Tamminen]

Hi,

On 01.06.2016 16:53, Marek Olšák wrote:

On Wed, Jun 1, 2016 at 3:02 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:

On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:

I'll let you figure it out by yourself.


Why would you withhold information if you already have it? Are you a
"bad person" or something?

As far as my memory goes, I have never posted the sentence "I'll let
you figure it out by yourself" to the Internet because I believe it is
wrong.


:)

Shader compilation at game loading time is never a problem, so we can
ignore that.


Startup time may not be a problem for games (some gamers may disagree), 
but there are also devices which have legal / certification requirements 
for startup times, where shader compilation time could be a problem, 
especially as Mesa shader compilation tends to get slower over time (as 
compiler does more optimizations).


Those devices have pretty static content (at least for startup), so 
their shaders could be pre-compiled if 3D driver supports that and has 
offline compiler.  Many proprietary drivers do, Mesa doesn't, at least 
not yet.




Shader compilation right before draw calls is what's unpleasant and
when people talk about it in the negative sense, they mean this.

Because of external factors you can't predict, your driver suddenly
receives a bunch of shaders that take 2000 ms to compile right before
a draw call. Your budget is 16 ms per frame to get 30 fps, but you
can't render the frame if you don't compile those shaders. The problem
is how to fit the compilation that takes 2000 ms and is required
render the frame into 16 ms. Can you see where I'm going?


Besides application itself compiling a shader, some state change done by 
application may also trigger shader recompile in Mesa.  While for 
application requested compilation nothing can be done to squeeze it into 
16ms, there are some chances of reducing need for state-change triggered 
recompiles (and these improvements have been done to Mesa during the years).



- Eero

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Markus Wick]

Am 2016-06-01 15:53, schrieb Marek Olšák:

On Wed, Jun 1, 2016 at 3:02 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
Shader compilation at game loading time is never a problem, so we can
ignore that.


I have to disagree here. For the dolphin emulator, we proposed a GLSL 
shader cache (because of mesa...). The idea was discarded as booting the 
emulator took up to minutes.



Shader compilation right before draw calls is what's unpleasant and
when people talk about it in the negative sense, they mean this.


I guess games with this behavior also: compile one shader, use it, 
repeat. Within a single frame. A threaded dispatcher (with shader 
compilation starting on the dispatcher thread) may be able to improve 
this behavior. I have no clue about the required effort, but I'm sure 
it's _by far_ more than simple threading primitives abstracted by 
std::async. And it's orthogonal to parallel compilation ;)



Because of external factors you can't predict, your driver suddenly
receives a bunch of shaders that take 2000 ms to compile right before
a draw call. Your budget is 16 ms per frame to get 30 fps, but you
can't render the frame if you don't compile those shaders. The problem
is how to fit the compilation that takes 2000 ms and is required
render the frame into 16 ms. Can you see where I'm going?


Of course, this will only improve this behavior. There is no way to fit 
within the 16ms.


But such an implementation will be required for 
ARB_parallel_shader_compile. For eg the dolphin emulator, we want to 
compile some slow generic shaders. While running, specialized shaders 
shall be compiled. But so we need the query to check if the compilation 
has finished. Of course, in a non-blocking way.


But I see, emulators have some special requirements. I doubt this will 
help games a lot. Hopefully, engines will start to use this features 
wisely when they'll more available.


This does not replace the requirement for an on-disk cache, or vise 
versa.



But bad luck, neither calling "use std::async" nor this mail helps at 
all. The required restructuring needs months of man power. Not to 
mention that the shader compiler needs to be thread-safe




Regards, degasus
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Brian Paul]

On 06/01/2016 07:02 AM, ⚛ wrote:

On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:

I'll let you figure it out by yourself.


Why would you withhold information if you already have it? Are you a
"bad person" or something?

As far as my memory goes, I have never posted the sentence "I'll let
you figure it out by yourself" to the Internet because I believe it is
wrong.


I think the issue here is someone comes along with no history in the 
project and asserts that he has a great solution for a problem while 
developers with many years of experience know it's not that simple.  It 
takes time to explain all this and most of us are very busy with other 
tasks.


Furthermore, people sometimes interpret no response to their ideas as 
implicit endorsement when that's not the case at all.  Better for the 
experienced people to give a quick no/nak than to say nothing and let 
someone assume that his idea is supported.  I suspect that what Marek 
was doing.


Finally, I think some of us have a hard time taking people seriously 
when they don't identify themselves with their real name.


-Brian

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Eero Tamminen]

Hi,

On 01.06.2016 16:50, Erik Faye-Lund wrote:

On Wed, Jun 1, 2016 at 3:02 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:

On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:

I'll let you figure it out by yourself.


Why would you withhold information if you already have it? Are you a
"bad person" or something?


The problem has been described in countless threads on this mailing
list, but I'll repeat it:

The vast majority of applications/games does glCompileShader()
immediately followed by glGetShaderiv(GL_COMPILE_STATUS), which would
force stalling for the result, leading to no improvement.


And the solution to the problem is shader cache, for which there's a 
(large) patch series on the list.


Most of the time in shader compiling goes to the linking stage (many of 
the optimizations are done at that stage).



- Eero

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Erik Faye-Lund]

On Wed, Jun 1, 2016 at 3:02 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
> On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:
>> I'll let you figure it out by yourself.
>
> Why would you withhold information if you already have it? Are you a
> "bad person" or something?

The problem has been described in countless threads on this mailing
list, but I'll repeat it:

The vast majority of applications/games does glCompileShader()
immediately followed by glGetShaderiv(GL_COMPILE_STATUS), which would
force stalling for the result, leading to no improvement.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Marek Olšák]

On Wed, Jun 1, 2016 at 3:02 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
> On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:
>> I'll let you figure it out by yourself.
>
> Why would you withhold information if you already have it? Are you a
> "bad person" or something?
>
> As far as my memory goes, I have never posted the sentence "I'll let
> you figure it out by yourself" to the Internet because I believe it is
> wrong.

:)

Shader compilation at game loading time is never a problem, so we can
ignore that.

Shader compilation right before draw calls is what's unpleasant and
when people talk about it in the negative sense, they mean this.

Because of external factors you can't predict, your driver suddenly
receives a bunch of shaders that take 2000 ms to compile right before
a draw call. Your budget is 16 ms per frame to get 30 fps, but you
can't render the frame if you don't compile those shaders. The problem
is how to fit the compilation that takes 2000 ms and is required
render the frame into 16 ms. Can you see where I'm going?

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[⚛]

On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:
> On Fri, May 27, 2016 at 8:49 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
>> Hello.
>>
>> http://en.cppreference.com/w/cpp/thread/future
>> http://en.cppreference.com/w/cpp/thread/async
>>
>> Assumption: Shader compilation will need run on separate thread(s).
>>
>> From a certain perspective, one of the easy ways of removing Mesa shader
>> compilation from the "main" thread would be to use std::future for some
>> fields in struct gl_program (defined in mtypes.h) and in related source
>> code.
>
> Your perspective is completely wrong. First, you need to understand
> why shader compilation speed is a problem for some apps and when you
> do, you'll realize there is only one solution and this is not it. I'll
> let you figure it out by yourself.
>
> tl;dr Understand the problem before you propose a solution.

http://en.wikipedia.org/wiki/Constructive_criticism#Description
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[⚛]

On Wed, Jun 1, 2016 at 2:19 PM, Marek Olšák  wrote:
> I'll let you figure it out by yourself.

Why would you withhold information if you already have it? Are you a
"bad person" or something?

As far as my memory goes, I have never posted the sentence "I'll let
you figure it out by yourself" to the Internet because I believe it is
wrong.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Marek Olšák]

On Fri, May 27, 2016 at 8:49 PM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
> Hello.
>
> http://en.cppreference.com/w/cpp/thread/future
> http://en.cppreference.com/w/cpp/thread/async
>
> Assumption: Shader compilation will need run on separate thread(s).
>
> From a certain perspective, one of the easy ways of removing Mesa shader
> compilation from the "main" thread would be to use std::future for some
> fields in struct gl_program (defined in mtypes.h) and in related source
> code.

Your perspective is completely wrong. First, you need to understand
why shader compilation speed is a problem for some apps and when you
do, you'll realize there is only one solution and this is not it. I'll
let you figure it out by yourself.

tl;dr Understand the problem before you propose a solution.

Marek
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Francisco Jerez]

⚛ <0xe2.0x9a.0...@gmail.com> writes:

> Hello.
>
> http://en.cppreference.com/w/cpp/thread/future
> http://en.cppreference.com/w/cpp/thread/async
>
> Assumption: Shader compilation will need run on separate thread(s).
>
> From a certain perspective, one of the easy ways of removing Mesa shader
> compilation from the "main" thread would be to use std::future for some
> fields in struct gl_program (defined in mtypes.h) and in related source
> code.
>
> Using std::future in the source code would mean that some parts of Mesa
> need to be converted from C to C++11.
>
> This post to mesa-dev is just to start the discussion and to determine how
> many devs are in favor of C++11 (and why) and how many are against C++11
> (and why) in Mesa.
>
> Looking forward to your opinions.

Yeah, IMO std::async is a minimally intrusive way to offload
computations to a separate thread which avoids many of the pitfalls of
managing concurrency and thread pools by hand.  For the moment I'm going
to stay away from the discussion of whether it's the right solution for
the GL front-end or not, but I think it would definitely make sense to
try out your idea in the CL state tracker, since it already uses C++11
extensively and currently doesn't take advantage of concurrency for
kernel compilation (even though the CL API was designed to allow
implementations to build CL kernels asynchronously).

> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev


signature.asc
Description: PGP signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Emil Velikov]

On 27 May 2016 at 21:11, Jason Ekstrand  wrote:

> On Fri, May 27, 2016 at 11:49 AM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
>
>> Hello.
>>
>> http://en.cppreference.com/w/cpp/thread/future
>> http://en.cppreference.com/w/cpp/thread/async
>>
>> Assumption: Shader compilation will need run on separate thread(s).
>>
>> From a certain perspective, one of the easy ways of removing Mesa shader
>> compilation from the "main" thread would be to use std::future for some
>> fields in struct gl_program (defined in mtypes.h) and in related source
>> code.
>>
>
> Given that mtypes.h is included (and struct gl_program) is used by a *lot*
> of C code, I think using any sort of C++ there is a non-starter, 2011 or
> otherwise.
>
> As far as threading goes, we have other ways of dealing with threading
> than introducing C++11.  Something that works with C is probably a better
> path forward.
>
>
>> Using std::future in the source code would mean that some parts of Mesa
>> need to be converted from C to C++11.
>>
>
> The "some parts" you're talking about are almost all of mesa.  I don't
> think that's going to happen.
>
>
>> This post to mesa-dev is just to start the discussion and to determine
>> how many devs are in favor of C++11 (and why) and how many are against
>> C++11 (and why) in Mesa.
>>
>> I believe that most people agree with Jason here, at least I do. In
general the less time we spend re-factoring, the more time we'll have for
bringing features (and/or perf. improvements) to new and old hardware.
Surely one can resolve the said issue, with a smaller hammer (and thus
quicker) than C++11 ?

Regards,
Emil
P.S. Don't be shy to reply on the build fix patch I've Cc'd you earlier. A
"seems to work/Tested-by: ..." or any other input is welcome.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Discussion: C++11 std::future in Mesa

[Jason Ekstrand]

On Fri, May 27, 2016 at 11:49 AM, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:

> Hello.
>
> http://en.cppreference.com/w/cpp/thread/future
> http://en.cppreference.com/w/cpp/thread/async
>
> Assumption: Shader compilation will need run on separate thread(s).
>
> From a certain perspective, one of the easy ways of removing Mesa shader
> compilation from the "main" thread would be to use std::future for some
> fields in struct gl_program (defined in mtypes.h) and in related source
> code.
>

Given that mtypes.h is included (and struct gl_program) is used by a *lot*
of C code, I think using any sort of C++ there is a non-starter, 2011 or
otherwise.

As far as threading goes, we have other ways of dealing with threading than
introducing C++11.  Something that works with C is probably a better path
forward.


> Using std::future in the source code would mean that some parts of Mesa
> need to be converted from C to C++11.
>

The "some parts" you're talking about are almost all of mesa.  I don't
think that's going to happen.


> This post to mesa-dev is just to start the discussion and to determine how
> many devs are in favor of C++11 (and why) and how many are against C++11
> (and why) in Mesa.
>
> Looking forward to your opinions.
>
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>
>
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

[Mesa-dev] Discussion: C++11 std::future in Mesa

[⚛]

Hello.

http://en.cppreference.com/w/cpp/thread/future
http://en.cppreference.com/w/cpp/thread/async

Assumption: Shader compilation will need run on separate thread(s).

>From a certain perspective, one of the easy ways of removing Mesa shader
compilation from the "main" thread would be to use std::future for some
fields in struct gl_program (defined in mtypes.h) and in related source
code.

Using std::future in the source code would mean that some parts of Mesa
need to be converted from C to C++11.

This post to mesa-dev is just to start the discussion and to determine how
many devs are in favor of C++11 (and why) and how many are against C++11
(and why) in Mesa.

Looking forward to your opinions.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev