Re: [Flightgear-devel] Cloud layer performance
So I know clouds write a z-buffer, and I know terrain reads a z-buffer - they just don't seem to use the same z-buffer. In which case the whole scheme doesn't work as advertized. Does that make any sense? Is there anything special about clouds? Seems I can't figure that one out on my own, so I abandoned the project. I guess I'll see then how the performance of overcast layers looks with a modern GPU and get a new computer then. * Thorsten -- LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
[Flightgear-devel] Cloud layer performance
I've managed to get a test implementation for the idea of filling the depth buffer early by clouds by inserting tracers into some of the cloud calls. The tracers are in essence just opaque white sheets marked with a top shade factor of 0.0 (black - no real cloud can ever have that low a value). There follow then two passes for cloud rendering - the first one does just the tracers and basically passes by all real clouds, the second pass does not consider tracers. During the first pass, depth buffer is written. Since there's one tracer per 10-50 sprites and the operations done on the tracer are bare minimum, the additional workload caused by tracer rendering is small. This has some quirks (the tracers become visible while the layer is drawn, although they are normally hidden by the real clouds, tracers at the edge of the range flicker,...), but it works... halfway. The depth buffer filled by the tracers does have a notable effect on other clouds, but not on the terrain. In numbers - in my benchmark situation I am halfway between a 2000 ft high Nimbus layer and the terrain. I get 20 fps looking straight at the horizon and 23 fps looking 30 deg up through the layer. With tracers I get 24 fps looking straight and 38 fps looking 30 degrees up, i.e. I am doing way faster to render the layer only. However, going above the cloud layer looking down, my framerate never changes. Even if I render only the tracers and zoom in so that only a while sheet is in my field of view (and no terrain pixels at all would have to be done) my framerate is still the same as without the tracer. In other words, the tracers in the depth buffer don't seem to work for the terrain, although they do work for other clouds. I've tested that I am not vertex-shader dominated by discarding half the pixels of the terrain in my field of view - that gave a noticeable framerate boost, so there's something else. I attach the effect xml for the additional first pass below. I'm not completely sure about all the aspects, but... it does the job on clouds, so it can't be that wrong. This should be done in render bin -1, i.e. before terrain. Very similar code for trees does have an impact for terrain rendering, i.e. an early tree pass does fill the depth buffer with effect on the terrain. What am I not seeing? pass lightingtrue/lighting material ambient type=vec4d0.5 0.5 0.5 1.0/ambient diffuse type=vec4d0.5 0.5 0.5 1.0/diffuse color-modeoff/color-mode /material depth functionlequal/function write-masktrue/write-mask /depth render-bin bin-number-1/bin-number bin-nameRenderBin/bin-name /render-bin texture-unit unit0/unit type2d/type imageusetexture[0]/image/use/image wrap-sclamp/wrap-s wrap-tclamp/wrap-t /texture-unit program vertex-shaderShaders/tracer.vert/vertex-shader fragment-shaderShaders/tracer.frag/fragment-shader attribute nameusrAttr1/name index10/index /attribute attribute nameusrAttr2/name index11/index /attribute /program uniform namebaseTexture/name typesampler-2d/type value type=int0/value /uniform uniform namerange/name typefloat/type valueuserange/use/value /uniform vertex-program-two-sidetrue/vertex-program-two-side /pass Some other idea: If I extend the Nimbus layer all the way to 75 km visibility distance, I eventually become vertex-shader dominated in which case tracers are no longer a good strategy. It occurred to me that there's a fairly simple scheme we could try for this situation: We assign a random number [0..1] to each of the sprites such that all four vertices of the quad get the same number and pass that as attribute. Inside the shader, we dynamically compute a threshold based on distance (or perhaps better view angle) and check the random number against the threshold, so that say for 50 km distance we render a quad only if its random number is above 0.8, for 70 km only if its number is above 0.95. This would allow to dynamically dilute the clouds as we go further out, the checks inside the shader are computationally light and the visual effect should not be pronounced as we see the clouds under a very shallow angle at large distances. For very thin clouds which we don't want to discard, we can bias the random number distribution at creation time. Stuart, is this feasible? There seems to be one free slot in the attribute vectors. Thanks in advance for any additional insights. * Thorsten -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct
Re: [Flightgear-devel] Cloud layer performance
I haven't really been following this thread closely... are the tracers basically occludes? Are they completely opaque? One possible optimization for the cloud rendering would be to use polygons that closely follow the outline of the sprite instead of quads; this saves on fill costs. Also, I'm pretty sure you don't want two-sided lighting for the tracers. Tim On Wed, Oct 24, 2012 at 8:53 AM, Renk Thorsten thorsten.i.r...@jyu.fi wrote: I've managed to get a test implementation for the idea of filling the depth buffer early by clouds by inserting tracers into some of the cloud calls. The tracers are in essence just opaque white sheets marked with a top shade factor of 0.0 (black - no real cloud can ever have that low a value). There follow then two passes for cloud rendering - the first one does just the tracers and basically passes by all real clouds, the second pass does not consider tracers. During the first pass, depth buffer is written. Since there's one tracer per 10-50 sprites and the operations done on the tracer are bare minimum, the additional workload caused by tracer rendering is small. This has some quirks (the tracers become visible while the layer is drawn, although they are normally hidden by the real clouds, tracers at the edge of the range flicker,...), but it works... halfway. The depth buffer filled by the tracers does have a notable effect on other clouds, but not on the terrain. In numbers - in my benchmark situation I am halfway between a 2000 ft high Nimbus layer and the terrain. I get 20 fps looking straight at the horizon and 23 fps looking 30 deg up through the layer. With tracers I get 24 fps looking straight and 38 fps looking 30 degrees up, i.e. I am doing way faster to render the layer only. However, going above the cloud layer looking down, my framerate never changes. Even if I render only the tracers and zoom in so that only a while sheet is in my field of view (and no terrain pixels at all would have to be done) my framerate is still the same as without the tracer. In other words, the tracers in the depth buffer don't seem to work for the terrain, although they do work for other clouds. I've tested that I am not vertex-shader dominated by discarding half the pixels of the terrain in my field of view - that gave a noticeable framerate boost, so there's something else. I attach the effect xml for the additional first pass below. I'm not completely sure about all the aspects, but... it does the job on clouds, so it can't be that wrong. This should be done in render bin -1, i.e. before terrain. Very similar code for trees does have an impact for terrain rendering, i.e. an early tree pass does fill the depth buffer with effect on the terrain. What am I not seeing? pass lightingtrue/lighting material ambient type=vec4d0.5 0.5 0.5 1.0/ambient diffuse type=vec4d0.5 0.5 0.5 1.0/diffuse color-modeoff/color-mode /material depth functionlequal/function write-masktrue/write-mask /depth render-bin bin-number-1/bin-number bin-nameRenderBin/bin-name /render-bin texture-unit unit0/unit type2d/type imageusetexture[0]/image/use/image wrap-sclamp/wrap-s wrap-tclamp/wrap-t /texture-unit program vertex-shaderShaders/tracer.vert/vertex-shader fragment-shaderShaders/tracer.frag/fragment-shader attribute nameusrAttr1/name index10/index /attribute attribute nameusrAttr2/name index11/index /attribute /program uniform namebaseTexture/name typesampler-2d/type value type=int0/value /uniform uniform namerange/name typefloat/type valueuserange/use/value /uniform vertex-program-two-sidetrue/vertex-program-two-side /pass Some other idea: If I extend the Nimbus layer all the way to 75 km visibility distance, I eventually become vertex-shader dominated in which case tracers are no longer a good strategy. It occurred to me that there's a fairly simple scheme we could try for this situation: We assign a random number [0..1] to each of the sprites such that all four vertices of the quad get the same number and pass that as attribute. Inside the shader, we dynamically compute a threshold based on distance (or perhaps better view angle) and check the random number against the threshold, so that say for 50 km distance we render a quad only if its random number is above 0.8, for 70 km only if its number is above 0.95. This would allow to dynamically dilute the clouds as we go further out, the checks inside the shader are computationally light and the visual effect should not be pronounced as we see the clouds under a very shallow angle at large
Re: [Flightgear-devel] Cloud layer performance
I haven't really been following this thread closely... are the
tracers basically occludes? Are they completely opaque?
Yes, they're completely opaque. That's tracer.frag - if it gets a white
gl_Color from the vertex shader, it's a real cloud and the fragment is dropped,
if it gets any other color the pixel is that of a tracer it produces an opaque
white pixel:
void main(void)
{
if (gl_Color == vec4(1.0, 1.0, 1.0, 1.0)) // this is a real cloud
{discard;}
gl_FragColor = vec4 (1.0, 1.0, 1.0, 1.0);
}
'occluders' would perhaps be a more intuitive name then...
One possible optimization for the cloud rendering would be to use
polygons that closely follow the outline of the sprite instead of
quads; this saves on fill costs.
The tracer/occluder is supposed to deal with the whole cloud, not any
individual sprite (individual sprites may be just a bit transparent even in the
center, but a collection of 20 of them is no longer, and in any case doing it
on a per-sprite basis means 20 times the load for the vertex shader). We don't
know the shape of the cloud before we create them, and to generate a polygon
representing in detail the opaque part of the final cloud which is a collection
of sprites with varying transparency and looks different from various positions
is something which sounds highly non-trivial to me. The tracer is just the
simplest guess occluding part of the opaque core of the cloud. It doesn't need
to be hugely efficient on an individual cloud, because there'll be hundreds of
them in a row, so even if we get only 20 % of any individual cloud, the end
result is that we get opaque everywhere after 10 clouds rather than doing a row
of 200 everywhere.
Also, I'm pretty sure you don't want two-sided lighting for the tracers.
Okay - I'll switch this off then. But that can't explain the depth buffer
issue, can it?
* Thorsten
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Cloud layer performance
On Wed, Oct 24, 2012 at 10:40 AM, Renk Thorsten thorsten.i.r...@jyu.fi wrote:
I haven't really been following this thread closely... are the
tracers basically occludes? Are they completely opaque?
Yes, they're completely opaque. That's tracer.frag - if it gets a white
gl_Color from the vertex shader, it's a real cloud and the fragment is
dropped, if it gets any other color the pixel is that of a tracer it produces
an opaque white pixel:
void main(void)
{
if (gl_Color == vec4(1.0, 1.0, 1.0, 1.0)) // this is a real cloud
{discard;}
gl_FragColor = vec4 (1.0, 1.0, 1.0, 1.0);
}
'occluders' would perhaps be a more intuitive name then...
Yes, I meant occluders; perhaps I was on occludes :)
One possible optimization for the cloud rendering would be to use
polygons that closely follow the outline of the sprite instead of
quads; this saves on fill costs.
The tracer/occluder is supposed to deal with the whole cloud, not any
individual sprite (individual sprites may be just a bit transparent even in
the center, but a collection of 20 of them is no longer, and in any case
doing it on a per-sprite basis means 20 times the load for the vertex
shader). We don't know the shape of the cloud before we create them, and to
generate a polygon representing in detail the opaque part of the final cloud
which is a collection of sprites with varying transparency and looks
different from various positions is something which sounds highly non-trivial
to me. The tracer is just the simplest guess occluding part of the opaque
core of the cloud. It doesn't need to be hugely efficient on an individual
cloud, because there'll be hundreds of them in a row, so even if we get only
20 % of any individual cloud, the end result is that we get opaque everywhere
after 10 clouds rather than doing a row of 200 everywhere.
I was thinking more of the sprite rendering, so my comment was a bit
off-topic here. How are you generating the tracer? I gather that it
doesn't use any textures.
I realize that I'm confused about whether you are working on Stuart's
clouds in C++ or clouds in Nasal. Your shader seems like an expensive
way to toggle between tracers and real sprites; it would be much
better to not render the sprite geometry at all in your tracer pass.
Also, I'm pretty sure you don't want two-sided lighting for the tracers.
Okay - I'll switch this off then. But that can't explain the depth buffer
issue, can it?
Probably not.
You should use a color-mask type=vec4d0 0 0 0/color-mask
declaration to turn off color writes when rendering the tracers, since
you don't want them to actually appear in the output. That will save
you some fill costs.
As for not seeing much improvement when the terrain is occluded, I can
think of several possibilities. You said that I've tested that I am
not vertex-shader dominated by discarding half the pixels of the
terrain in my field of view; how did you do that?
This optimization depends on being able to do an early Z test to not
rasterize big regions of the terrain triangles and avoid running the
terrain shaders on those pixels that will fail the Z test, but this
can only be done if the shader can't affect the Z output value.
Writing the fragment depth would do that (urban shader), but the
classic offenders are discard and alpha testing. It's probably worth
checking that the terrain shaders aren't doing those things. A quick
look at terrain-default.eff shows that alpha-test is set to
transparent which, IIRC, means that it is set based on whether or
not the texture is transparent; it should probably just be off.
Tim
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Cloud layer performance
I realize that I'm confused about whether you are working on Stuart's
clouds in C++ or clouds in Nasal.
They're the same now - there's an interface to generate a cloud from Nasal
which is what I'm using, whereas Basic Weather uses and xml format for layer
declaration, but for rendering purpose there is no difference.
How are you generating the tracer? I gather that it
doesn't use any textures.
If a flag is set in the cloud definition, after a cloud (consisting of a number
of sprites) has been generated, a tracer is inserted at the same position in
the center of the cloud. Currently only works if you insert clouds from Nasal.
Formally a tracer is a cloud with just a single sprite, but a cloud with a top
shade factor set to zero (i.e. a pitch-black top) - which is why we know in the
shader it can't be a real cloud.
Your shader seems like an expensive
way to toggle between tracers and real sprites; it would be much
better to not render the sprite geometry at all in your tracer pass.
I basically don't - if a vertex not belonging to a tracer arrives in
tracer.vert, all the vertex shader does is to move it out of the view frustrum
(come to think of it, the if-statement in the fragment shader shouldn't even be
necessary - sorry, this is just mess in progress). Also the effect currently
declares a texture which I'm not using because at some point I wanted to test
round tracers.
You should use a color-mask type=vec4d0 0 0 0/color-mask
declaration to turn off color writes when rendering the tracers, since
you don't want them to actually appear in the output. That will save
you some fill costs.
Thanks, that's very helpful.
As for not seeing much improvement when the terrain is occluded, I can
think of several possibilities. You said that I've tested that I am
not vertex-shader dominated by discarding half the pixels of the
terrain in my field of view; how did you do that?
By putting
if (gl_FragCoord.y 300) {discard;}
early into the terrain fragment shader.
but this
can only be done if the shader can't affect the Z output value.
Writing the fragment depth would do that (urban shader), but the
classic offenders are discard and alpha testing.
Thanks for the pointers - I'll have a look. How come it works for clouds then?
The normal cloud passes all use alpha testing...
* Thorsten
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Cloud layer performance
On Wed, Oct 24, 2012 at 1:27 PM, Renk Thorsten thorsten.i.r...@jyu.fi wrote:
I realize that I'm confused about whether you are working on Stuart's
clouds in C++ or clouds in Nasal.
How are you generating the tracer? I gather that it
doesn't use any textures.
If a flag is set in the cloud definition, after a cloud (consisting of a
number of sprites) has been generated, a tracer is inserted at the same
position in the center of the cloud. Currently only works if you insert
clouds from Nasal. Formally a tracer is a cloud with just a single sprite,
but a cloud with a top shade factor set to zero (i.e. a pitch-black top) -
which is why we know in the shader it can't be a real cloud.
Your shader seems like an expensive
way to toggle between tracers and real sprites; it would be much
better to not render the sprite geometry at all in your tracer pass.
I basically don't - if a vertex not belonging to a tracer arrives in
tracer.vert, all the vertex shader does is to move it out of the view
frustrum (come to think of it, the if-statement in the fragment shader
shouldn't even be necessary - sorry, this is just mess in progress). Also the
effect currently declares a texture which I'm not using because at some point
I wanted to test round tracers.
If I understand correctly, you have only one model of the cloud, which
has a bunch of sprites and a few tracer polygons. I don't know if this
is practical in the current system, but I'm saying that it would be
better to draw only the tracers in the tracer pass and the sprites in
the real pass as you do know which is which in advance. I do
appreciate the clipping trick in the vertex shader :)
As for not seeing much improvement when the terrain is occluded, I can
think of several possibilities. You said that I've tested that I am
not vertex-shader dominated by discarding half the pixels of the
terrain in my field of view; how did you do that?
By putting
if (gl_FragCoord.y 300) {discard;}
early into the terrain fragment shader.
OK
but this
can only be done if the shader can't affect the Z output value.
Writing the fragment depth would do that (urban shader), but the
classic offenders are discard and alpha testing.
Thanks for the pointers - I'll have a look. How come it works for clouds
then? The normal cloud passes all use alpha testing...
They don't write the Z buffer at all, do they?
Tim
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Cloud layer performance
If I understand correctly, you have only one model of the cloud, which has a bunch of sprites and a few tracer polygons. I don't know if this is practical in the current system, but I'm saying that it would be better to draw only the tracers in the tracer pass and the sprites in the real pass as you do know which is which in advance. I do appreciate the clipping trick in the vertex shader :) I think it's two models - one containing the actual cloud sprites and one containing the tracer. Both run via both shaders, but the first pass only does real work for the tracers and discards real clouds, the second pass does work for real clouds but discards tracers. In principle it would be (marginally) better to do one pass for clouds and tracers each, since as you say we know up front which is which. In practice tracers need to inherit the movement of the clouds, so it doesn't do to just put models in, they need to inherit the visibility range, and I guess all this would require changes on the C++ level, duplicating or generalizing part of Stuart's cloud generating system. The expected performance gain is marginal - I can't even measure the difference between adding tracers without z-buffer write and not adding tracers. So I think for testing purposes it's okay to go with the scheme, it's not where the performance is burnt. They don't write the Z buffer at all, do they? No, but what I mean is: I see that the first cloud pass for the tracers writes the z-buffer, because if I insert the tracers at the wrong altitude, they punch neat square holes into my real layer where the z-buffer says not to do the real cloud. This works in spite of the fact that the second pass uses alpha testing, as do all cloud effects currently declared. The tracers just don't do this for the terrain. But the second terrain pass as such is fine with the z-buffer. I have introduced a first pass for trees in render bin -1, and if I don't do the second tree pass I get neat tree-shaped holes punched into my terrain mesh and the expected performance bonus for not doing a lot of terrain pixels. So I know clouds write a z-buffer, and I know terrain reads a z-buffer - they just don't seem to use the same z-buffer. In which case the whole scheme doesn't work as advertized. Does that make any sense? Is there anything special about clouds? * Thorsten -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
