Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
On Sun, Dec 18, 2011 at 9:31 PM, Stuart Buchanan wrote: On Sun, Dec 18, 2011 at 8:18 PM, Torsten Dreyer wrote: I suggest we default the impostors to be switched off by default. That's a trivial change to line 72 of cloudfield.cxx. I don't know if that impacts the perf improvements that Matthias is seeing. I'm also happy to have this left until after the 2.6.0 release. Now the release branch has been created, I've committed these changes to simgear/next, flightgear/next and fgdata/master. The Impostors are switched off by default. Unfortunately, one of the side-effects of switching them on is that some clouds will jump into existence, caused by an impostor being generated while the cloud is out of range, and not being re-generated (or switched off) until the cloud is well within range. If you want to use them,you will need to set /sim/rendering/clouds3d-use-impostors=true _before_ generating any clouds (e.g. from the command-line). There are also a number of properties under /sim/rendering/ to set the LoD and impostor ranges. I've set them such that a typical scatter layer of cumulus under Global weather will generate ~20 Impostors, and ~200 lower level LoD nodes for some 3000 clouds. Feedback welcome as always. -Stuart -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Hi Stuart, On Saturday, December 17, 2011 20:14:22 you wrote: I've committed the code changes to my repository clones for the impostors, some configuration for LoD nodes, and having a geometry per cloud, with a bubble sort for ordering. It would be fantastic if people could take the time to cherry-pick these commits and see if it makes any performance difference for them. I anticipate the biggest difference being to 3D clouds generated by Global Weather, as those clouds tend to have more sprites than the Local Weather equivalents. Good to see! And in fact without testing too much I think we are on the right track. Just starting and comparing the frame rates with and without the changes gives a 3x performance boost on my setup! Well, I have started the default fgfs at LOWI with the world scenery tiles and switched on and off the 3d clouds. This is no reproducible benchmark but at least a very good sign. So, since you do not see that huge differences, it appears that your 3D driver draws way faster than mine :) For reference, what os/gpu/driver do you use? The rule is really to call as few gl functions as possible. That is do a scene that feeds all in huge batches doing just the same. And keep in mind that there is a cutoff point - mostly higher than thought at the first moment - where it gets more work on the GPU where it starts to make sense to split these batches in halfes... Just having a very quick look over the code: You still have a normal array that has 4 entries and that is set to BIND_OVERALL which makes the shader get in the end only the first member for every vertex it processes. This makes me believe that the normal is no longer used? If so, just omit the normal array completely and set the binding to OFF (I believe it is called OFF). I think this should work. Greetings Mathias -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
On Sat, 2011-12-17 at 19:14 +, Stuart Buchanan wrote: As I said previously, I'm not seeing massive changes in frame-rates with these changes. On my system, running the c172p from KLVK I get 30 fps with 2D clouds and 28-29fps with 3D clouds. However, the draw time presented by the on-screen OSG statistics appears to have dropped. Same here; little difference between these settings but I have fairly decent video hardware (37fps with 3D clouds and 52fps with 2D clouds) using the stormy Monday setup. Erik -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
2011/12/18 Mathias Fröhlich wrote: Good to see! And in fact without testing too much I think we are on the right track. Just starting and comparing the frame rates with and without the changes gives a 3x performance boost on my setup! Excellent. So, since you do not see that huge differences, it appears that your 3D driver draws way faster than mine :) For reference, what os/gpu/driver do you use? I've got a NVidia GT260M with 1GB of memory, on Ubuntu 10.4, using the NVidia 195.36.24 driver. Due to a NVidia PowerMizer bug it only runs at 275MHz rather than 550MHz. Just having a very quick look over the code: You still have a normal array that has 4 entries and that is set to BIND_OVERALL which makes the shader get in the end only the first member for every vertex it processes. This makes me believe that the normal is no longer used? If so, just omit the normal array completely and set the binding to OFF (I believe it is called OFF). I think this should work. Correct, we don't use the normal. I'll make that change and re-test. One question for the release managers: Do performance improvements such as these count as bug fixes or features? Should we try to get this into the 2.6.0 release, or wait until 2.8.0? -Stuart -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
On 18 Dec 2011, at 11:06, Stuart Buchanan wrote: One question for the release managers: Do performance improvements such as these count as bug fixes or features? Should we try to get this into the 2.6.0 release, or wait until 2.8.0? I think the answer is 'it depends', but given that you have been working on the code in a branch (when equivalent hacking could easily have taken place in next/ ), and are at the point of 'it's basically done, needs testing', *and* it makes a major difference for at least some people, I'd be inclined to cherry-pick it in early next week. (We can always roll it back, and it doesn't impact aircraft compatibility either way) However, that's only my opinion. James -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Hi, On Sunday, December 18, 2011 12:06:22 Stuart Buchanan wrote: One question for the release managers: Do performance improvements such as these count as bug fixes or features? Should we try to get this into the 2.6.0 release, or wait until 2.8.0? Given that you have already prepared this for some time now, I would tend to accept this '1 day too late' change. Also, looking at the changes these look pretty straight forward. So, I vould vote for an exception in this case. Mathias -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
One question for the release managers: Do performance improvements such as these count as bug fixes or features? Should we try to get this into the 2.6.0 release, or wait until 2.8.0? For me, 3d clouds are currently mostly unusable. If this cures the issue, I'd call it a fix. However, the feature-freeze is for preparing a mostly stable (not necessarily perfect) codebase, so please decide yourself: Does it need extensive testing? Does it interfere with other systems? Does it have the potential to raise additional issues? Does it have the potential to endanger the release schedule? What do you think? If you feel OK, so do I. Torsten -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
On Sun, Dec 18, 2011 at 8:18 PM, Torsten Dreyer wrote: One question for the release managers: Do performance improvements such as these count as bug fixes or features? Should we try to get this into the 2.6.0 release, or wait until 2.8.0? For me, 3d clouds are currently mostly unusable. If this cures the issue, I'd call it a fix. However, the feature-freeze is for preparing a mostly stable (not necessarily perfect) codebase, so please decide yourself: Does it need extensive testing? Possibly. In particular, the Impostor changes will increase the memory requirements and may have some graphics-card specific behaviour I'm not aware of. One way to address this might be to default the /sim/rendering/clouds3d-use-impostors to false, which will default to the previous behaviour (e.g. LoD nodes rather than Impostors), while retaining the geometry changes that are less controversial. Does it interfere with other systems? No. Does it have the potential to raise additional issues? There's a stability risk from the Impostors, as these are new OSG constructs we've not used before, and which AFAICT aren't commonly used elsewhere. Does it have the potential to endanger the release schedule? No. The changes are self-contained and could be easily backed out at any point. What do you think? I suggest we default the impostors to be switched off by default. That's a trivial change to line 72 of cloudfield.cxx. I don't know if that impacts the perf improvements that Matthias is seeing. I'm also happy to have this left until after the 2.6.0 release. -Stuart -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
2011/12/15 Mathias Fröhlich wrote: On Wednesday, December 14, 2011 15:49:22 Stuart Buchanan wrote: 2011/12/14 Mathias Fröhlich wrote: But, the question is how many cloud drawables do we have? The render Bin sorting bottleneck - if we run into this - is O(log(n)*n) with the n= number depth sorted drawables. Which means we need to have a huge amount of cloud drawables that this effect dominates. As you've seen from the code, at present, we have a cloud drawable per sprite, where there are multiple sprites per cloud, and 1000 clouds in a typical Cu layer. Than I assume 10 to 500 sprites in one cloud? Which makes about 1 draws for the clouds. That could explain the observations. For the global weather 3D clouds it is typically between 5 and 15 sprites per cloud. We perform some sprite-culling to avoid creating sprites that are too close together. For Local weather it's sometimes as low as 1 sprite per cloud. Ok, looking into the Cloud drawable implementation, I believe that your almost first response is probably the easiest. Just without point sprites, just improoving what is currently done: Try to put that multiple draws into a single draw using array objects. Make sure that you still get a 'fast drawable'. YOu can verify this by asking the geometry if osg::Geometry::areFastPathsUsed() returns true. That is mostly: do not use any index arrays. The only indices which may be worthwhile are the indexed primitives sets. Sorting inside the drawable is then done by either redoing the arrays or probably better by using an indexed primitive set and reordering the indices. I will try this tonight. Thanks again for looking into this and helping to point me in the right direction. I made the changes you suggested last night so that there is a single geometry per cloud instead of one per sprite. That should have reduced the number of draws by a factor of 10. The code isn't ready to be checked in (I've still got to sort the geometry), but so far it hasn't made a significant difference to the frame-rate on my system. However, I've got a fairly big graphics card and get fairly high frame-rates, so it may help more for other people. It certainly doesn't make things any worse, and the code changes are smaller than I had feared, so I expect to get this change checked in for 2.6.0. (BTW - I think I've managed to get Impostors working though I've still to see any performance gain) Ok, if this helps this is fine. So the granularity is one cloud per imposter? No. At that level my machine ran out of memory. I've got an Impostor for each top level LoD node, which typically contains the clouds in a 10kmx10km area, so we end up with ~18 Impostors. Still no performance gain from this either. I've made it configurable using properties, so once it's checked in people can experiment to see if varying the LoD size, and hence number of impostors, helps. Again, the code changes were remarkably small, so I expect to have this ready in time for 2.6.0 For this approach, you need to know that setting up a new fbo for draw is not exactly cheap. It's not catastropic, but again don't do too often. Also each cloud probably uses one texture of some size which will occupy memory. Having 1000 of them is probably significant. Also drawing a cloud is than not only the draw, but also the texture change state change that needs to happen on every draw. This could still be a net win. But that's what I think is good to know to balance the orders of magnitude. May be it helps to have less individual clouds where each of them is drawn with a single draw? An idea that I had yesterday. Just a dumb question: Is it possible that we have some problem so that we have more clouds in the visible scene than we really expect there? That is is there a bug that makes some of them being drawn techically but not appear on the screen where they should be? Which could make draw so expensive by drawing more than we can see? So, just because experience shows that problems have often very simple reasons compared to the first ideas what could happen ... Very good point. So we might not be culling the clouds properly? I can easily put in some code to check this. I _think_ I've got the bounding boxes set up correctly and a sensible LoD tree, but this is something I should check. I hope that I really guide into the right direction. Please keep suggesting ideas. I'm a bit time-limited right now, so it may take me some time to write the code to test them, but I'm all ears :) -Stuart -- 10 Tips for Better Server Consolidation Server virtualization is being driven by many needs. But none more important than the need to reduce IT complexity while improving strategic productivity. Learn More! http://www.accelacomm.com/jaw/sdnl/114/51507609/ ___ Flightgear-devel mailing list
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
2011/12/14 Mathias Fröhlich wrote: But, the question is how many cloud drawables do we have? The render Bin sorting bottleneck - if we run into this - is O(log(n)*n) with the n= number depth sorted drawables. Which means we need to have a huge amount of cloud drawables that this effect dominates. As you've seen from the code, at present, we have a cloud drawable per sprite, where there are multiple sprites per cloud, and 1000 clouds in a typical Cu layer. Ok, looking into the Cloud drawable implementation, I believe that your almost first response is probably the easiest. Just without point sprites, just improoving what is currently done: Try to put that multiple draws into a single draw using array objects. Make sure that you still get a 'fast drawable'. YOu can verify this by asking the geometry if osg::Geometry::areFastPathsUsed() returns true. That is mostly: do not use any index arrays. The only indices which may be worthwhile are the indexed primitives sets. Sorting inside the drawable is then done by either redoing the arrays or probably better by using an indexed primitive set and reordering the indices. I will try this tonight. Thanks again for looking into this and helping to point me in the right direction. (BTW - I think I've managed to get Impostors working though I've still to see any performance gain) -Stuart -- Cloud Computing - Latest Buzzword or a Glimpse of the Future? This paper surveys cloud computing today: What are the benefits? Why are businesses embracing it? What are its payoffs and pitfalls? http://www.accelacomm.com/jaw/sdnl/114/51425149/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
On 14 Dec 2011, at 14:49, Stuart Buchanan wrote: (BTW - I think I've managed to get Impostors working though I've still to see any performance gain) That's very interesting, because if we had truly generic impostors working, they could also be used to push out the scenery rendering substantially. (basically, tiles beyond the core rings would be rendered as impostors - there's some demos of people doing this online, in other terrain engines) James -- Cloud Computing - Latest Buzzword or a Glimpse of the Future? This paper surveys cloud computing today: What are the benefits? Why are businesses embracing it? What are its payoffs and pitfalls? http://www.accelacomm.com/jaw/sdnl/114/51425149/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Stuart, On Wednesday, December 14, 2011 15:49:22 Stuart Buchanan wrote: 2011/12/14 Mathias Fröhlich wrote: But, the question is how many cloud drawables do we have? The render Bin sorting bottleneck - if we run into this - is O(log(n)*n) with the n= number depth sorted drawables. Which means we need to have a huge amount of cloud drawables that this effect dominates. As you've seen from the code, at present, we have a cloud drawable per sprite, where there are multiple sprites per cloud, and 1000 clouds in a typical Cu layer. Than I assume 10 to 500 sprites in one cloud? Which makes about 1 draws for the clouds. That could explain the observations. Ok - thanks. Ok, looking into the Cloud drawable implementation, I believe that your almost first response is probably the easiest. Just without point sprites, just improoving what is currently done: Try to put that multiple draws into a single draw using array objects. Make sure that you still get a 'fast drawable'. YOu can verify this by asking the geometry if osg::Geometry::areFastPathsUsed() returns true. That is mostly: do not use any index arrays. The only indices which may be worthwhile are the indexed primitives sets. Sorting inside the drawable is then done by either redoing the arrays or probably better by using an indexed primitive set and reordering the indices. I will try this tonight. Thanks again for looking into this and helping to point me in the right direction. (BTW - I think I've managed to get Impostors working though I've still to see any performance gain) Ok, if this helps this is fine. So the granularity is one cloud per imposter? For this approach, you need to know that setting up a new fbo for draw is not exactly cheap. It's not catastropic, but again don't do too often. Also each cloud probably uses one texture of some size which will occupy memory. Having 1000 of them is probably significant. Also drawing a cloud is than not only the draw, but also the texture change state change that needs to happen on every draw. This could still be a net win. But that's what I think is good to know to balance the orders of magnitude. May be it helps to have less individual clouds where each of them is drawn with a single draw? An idea that I had yesterday. Just a dumb question: Is it possible that we have some problem so that we have more clouds in the visible scene than we really expect there? That is is there a bug that makes some of them being drawn techically but not appear on the screen where they should be? Which could make draw so expensive by drawing more than we can see? So, just because experience shows that problems have often very simple reasons compared to the first ideas what could happen ... I hope that I really guide into the right direction. Greetings Mathias -- 10 Tips for Better Server Consolidation Server virtualization is being driven by many needs. But none more important than the need to reduce IT complexity while improving strategic productivity. Learn More! http://www.accelacomm.com/jaw/sdnl/114/51507609/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
IIRC clouds were moved into bin 10 to improve appearance vis-à-vis particles. If we put clouds back into bin 9 and particles remain in 10 all the cooling towers, chimney efflux, aircraft contrails, exhausts etc. are drawn after the clouds i.e, in front. Rather looks as if we can have realism or framerate but not both. Are these assignments runtime-changeable? If so, one could use a simple criteria to get it (roughly) right. * particle effects associated with an aircraft are always drawn in front (they must be nearby to see them anyway) * aircraft contrails are always treated as clouds (they are clouds) * ground-based model effects are drawn in front if the current view is below some decision altitude, and they are drawn at back if the aircraft is above (with the decision altitude being the lowest cloud layer) This set of rules should get so much right that the exceptions don't really matter. Cheers, * Thorsten -- Systems Optimization Self Assessment Improve efficiency and utilization of IT resources. Drive out cost and improve service delivery. Take 5 minutes to use this Systems Optimization Self Assessment. http://www.accelacomm.com/jaw/sdnl/114/51450054/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Thorsten wrote: IIRC clouds were moved into bin 10 to improve appearance vis-à-vis particles. If we put clouds back into bin 9 and particles remain in 10 all the cooling towers, chimney efflux, aircraft contrails, exhausts etc. are drawn after the clouds i.e, in front. Rather looks as if we can have realism or framerate but not both. Are these assignments runtime-changeable? Not that I can see at the moment - right now particles are hard-coded to go into bin 10, and the others are shader based which can be changed relatively easily - but not during runtime. If so, one could use a simple criteria to get it (roughly) right. * particle effects associated with an aircraft are always drawn in front (they must be nearby to see them anyway) Sort of true - but this will generate odd effects in tower views. * aircraft contrails are always treated as clouds (they are clouds) Yes - true - but they are particle based so they will follow whatever heuristic is appropriate for aircraft above. So they will almost never be right. The persistent contrails are shader based, and can go into whichever bin the clouds go, and will work fine, except they will not necessarily match the particle based contrails. * ground-based model effects are drawn in front if the current view is below some decision altitude, and they are drawn at back if the aircraft is above (with the decision altitude being the lowest cloud layer) We make no distinction between types of particles - they're all the treated the same atm and get bunged into bin 10. This set of rules should get so much right that the exceptions don't really matter. So you can see why the decision was made to lump them all together into 10 and let God, sorry OSG, sort them out. And so it does by and large, making a pretty good job of it, but it takes its time over it. Catch 22 or what? Vivian -- Systems Optimization Self Assessment Improve efficiency and utilization of IT resources. Drive out cost and improve service delivery. Take 5 minutes to use this Systems Optimization Self Assessment. http://www.accelacomm.com/jaw/sdnl/114/51450054/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Stuart, On Monday, December 12, 2011 14:38:37 Vivian Meazza wrote: IIRC clouds were moved into bin 10 to improve appearance vis-à-vis particles. If we put clouds back into bin 9 and particles remain in 10 all the cooling towers, chimney efflux, aircraft contrails, exhausts etc. are drawn after the clouds i.e, in front. Rather looks as if we can have realism or framerate but not both. Hmm, If I change the renderbin to 9 and instantiate a non sorted one, I am still draw limited here. Also If I look at the application runtime profile I see functions dominating that happen during draw in the driver. There is no sorting function high in profiles. May be OpenGL drivers that draw faster are depth sorting limited. But being draw limited this is what I can see here. But, the question is how many cloud drawables do we have? The render Bin sorting bottleneck - if we run into this - is O(log(n)*n) with the n= number depth sorted drawables. Which means we need to have a huge amount of cloud drawables that this effect dominates. Ok, looking into the Cloud drawable implementation, I believe that your almost first response is probably the easiest. Just without point sprites, just improoving what is currently done: Try to put that multiple draws into a single draw using array objects. Make sure that you still get a 'fast drawable'. YOu can verify this by asking the geometry if osg::Geometry::areFastPathsUsed() returns true. That is mostly: do not use any index arrays. The only indices which may be worthwhile are the indexed primitives sets. Sorting inside the drawable is then done by either redoing the arrays or probably better by using an indexed primitive set and reordering the indices. Greetings Mathias -- Cloud Computing - Latest Buzzword or a Glimpse of the Future? This paper surveys cloud computing today: What are the benefits? Why are businesses embracing it? What are its payoffs and pitfalls? http://www.accelacomm.com/jaw/sdnl/114/51425149/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Emilian On Sunday 11 December 2011 22:04:02 Stuart Buchanan wrote: On Thu, Dec 8, 2011 at 10:47 AM,I wrote: 2011/12/8 Mathias Fröhlich wrote: If I do not respond to list mails when you need some response, fell free to contact me directly. I just miss some mails every now and then ... Thanks for the offer. Will do. I've had a look, and I think I can change the code to create a single PrimitiveSet for each cloud fairly easily. On thinking about this a bit more, one thing that I don't quite understand is why the behaviour for clouds should differ so much from our random vegetation. The random vegetation code we have is very similar - a small number of geometries being used again and again. Yet, the performance is far, far better, even with much higher numbers of objects. I had thought that the main difference was the use of transparency, where the clouds are larger and generally more transparent than the trees. If so, and the alpha blending of the textures has the most impact on framerate, will changing the geometry help significantly? Or is it the case that the transparency _within_ a geometry is much more effectively handled by OSG than the transparency between different geometries? -Stuart On a sidenote to this discussion, all cloud*.eff files force render bin 10, moving them back to render bin 9 (as they were before according to this: http://mapserver.flightgear.org/git/?p=fgdata;a=commitdiff;h=f94af651aecc6 3ea1989529f0114b28b4bcef48f and also to the setup in simgear/scene/util/RenderConstants.hxx ) gives me back a lot of performance (roughly from 15fps to 30 fps with fair weather, and much less framedrop in cloudheavy configs with a 8600gt). Maybe there's some performance to be gained back from that too? IIRC clouds were moved into bin 10 to improve appearance vis-à-vis particles. If we put clouds back into bin 9 and particles remain in 10 all the cooling towers, chimney efflux, aircraft contrails, exhausts etc. are drawn after the clouds i.e, in front. Rather looks as if we can have realism or framerate but not both. At the time performance was not unduly hit - but I might be remembering incorrectly. Vivian -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Do we use instancing yet? That could give some performance for DirectX 10+ / OpenGL 3.1 cards. -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
But this is definitely some food for the brain. Here is what they write: So, how do we render these clouds in x-plane 10? the answer is levels of detail, which is a display of resolutions of buckets. (...) AGAIN! NO MATTER HOW BIG THE SKY, EACH LEVEL OF DETAIL ONLY COSTS US 1,000 PUFFS! Cutting through all the excitement of the blog writer - we've been discussing this previously, and (at last I) disregarded it for the following reasons: 1) Using many small 'puffs' (or 'sprites' in Stuart's terminology, or 'cloudlets' in mine) are a nice way to render diffuse shapeless clouds, but to arrange them in such a way that you get the correct shape of a well-defined Cumulus cloud is near impossible (which is way we use not so many larger full-cloud image textures). 2) Setting up 'puffs' of different size dependent on distance in a static setup is all nice and crispy, but unfortunately the darn airplanes *move*, so you need some algorithm which replaces a collection of small puffs by a single large puff - and this transition shouldn't actually be visible. If you code it for just a few configurations, it can be done, but then your clouds lack variety. 3) The problem that follows from the above two is that you must make a transition from a Cu cloud composed of 100 small puffs to one which is a single large puff, but the cloud shouldn't change visibly while you do so - which is tricky because Cu clouds have a lot of structure. 4) The problem is worsened by the fact that Cu clouds are not exactly self-similar across too large distance scales - a single puff which works well to represent a cloud from the distance isn't represented well by 10 smaller versions of the same puff - the shape gets wrong. Instead, you'd need a mixture of different smaller puff types dependent on where in the cloud you are. 5) Talking about km-sized puffs at distance is all nice, but the cloud layer may just be 100 m thick, so you can't represent it by 1000 m scale puffs without making it thicker than it really is. Again, with shapeless clouds you can use z-rescaling to deal with the problem, but with structured clouds, that shows up in a very pronounced way. My guess is that X-plane with this approach invites trouble with representing cloud structures, but fares better with very featureless clouds. Also, shape variation of clouds might be an issue, dependent on how this is coded in detail. It's not a bad idea, but the devil is in the detail, and a dynamical way of doing a reduction of complexity at distance (= the imposters) seems a better way to me. Anyway, it's hardly like this writer invented the wheel... Cheers, * Thorsten -- Cloud Services Checklist: Pricing and Packaging Optimization This white paper is intended to serve as a reference, checklist and point of discussion for anyone considering optimizing the pricing and packaging model of a cloud services business. Read Now! http://www.accelacomm.com/jaw/sfnl/114/51491232/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Stuart wrote: On Mon, Dec 5, 2011 at 8:26 AM, Thorsten Renk wrote: Since according to the newsletter Stuart's current ongoing quest is to get better performance for 3d clouds, here are some of my observation: Thanks very much for the observations. Lots of food for thought :) As an FYI, the investigations I've been doing haven't born much fruit. In fact, I've been thinking that my quest is a bit like that of the Holy Grail - something you never actually attain :) I had come to the conclusion that the only way to get a signficant increase in performance would be move to using Impostors. That's a big change, particularly as the OSG implimentation appears to be broken/bit-rotted. I've been strenuously avoiding having to think about implementing it myself, but I may have to just bite the bullet. Either way, it isn't going to happen for 2.6.0! * I've noticed that when I use the relatively lowres Altocumulus texture sheet (3x3 on one sheet) I can basically use a ridiculous number of sprites without performance deterioration, whereas when I use the hires Cumulus sheets (1x2 plus 1x3) the number of sprites I can show before performance takes a nosedive goes down substantially. That's very interesting information indeed. I will do some like-for-like experiments One contributing factor may be differences in the amount of transparency in the different textures. * There seems still to be stuff computed in the shaders per vertex that is actually an uniform per frame - eyepos for instance. I wonder if the computations could be speeded up significantly by consequently pulling all things that are really uniforms out of the shaders. I'll take a look. Vector and matrix calculations should be very efficient in the GPU, and we're only performing these per-vertex rather than per- fragment, so there may not be much benefit. * We're likewise fond of computing stuff per frame that changes more like per minute. The orientation of faraway clouds doesn't have to be computed per frame, because it can't change much per frame. If there'd be a way to store the value used last time, then (based on a distance criterion), one could assign clouds into n task groups and recompute a task group only every nth frame and use the last stored value otherwise. Back when I rotated clouds from Nasal, this did work and improved performance by a factor 5 or 6 - not sure how much it could do with a Shader setup, not sure how to do it technically, but my guess is that it would speed things up. We already use this technique for sorting the sprites within the cloud, by using a heuristic that if the sprites were already sorted the previous time we checked, they probably still are. We could do something similar for calculating the eyepoint outside of the shader, but as pointed out above, I'm not sure this is the main perf limitation. Emilian and I noticed that the local cloud effect files are using render bin 10 - thus competing with all other transparent objects, while the global clouds use render bin 9 - the dedicated cloud bin. We have tried moving all the clouds to render-bin 9: Emilian reports a significant gain in performance but I see little change here. However, we are not aware of the reason for the use of different bins. We also note that you are back to using the .png textures with their black edges etc., rather than the .dds that we provided. Use of .dds textures should at least be no worse than .png and at best will provide a small performance advantage. Unfortunately, we are only scratching around the edges for improved performance using the existing technique, and really we need to try something a bit more radical. Vivian -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Vivian: We also note that you are back to using the .png textures with their black edges etc., rather than the .dds that we provided. Use of .dds textures should at least be no worse than .png and at best will provide a small performance advantage. That's a technical requirement of Stuart's system, no intentional step: Textures have to be arranged in regular m x n structures on the sheet. I don't think I have seen dds textures from you with this structure. What I used earlier and what you converted to dds where textures associated with a model which supported any arrangement of textures on the sheet. The dds couldn't be used, and I needed everything in m x n sheets in a hurry - Emilian at some point offered help for the transition in the forum, but that apparently got lost on the way. My original intention was that the texture sheets get completely reworked and replaced by better quality imagery where possible, then converted to dds while the current sheets are to be used for a transition period only. But evidence suggests that that period may be longer than expected... So again, there is no intention to disregard your conversion work involved here - sorry if that impression was created. * Thorsten -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
On Mon, Dec 5, 2011 at 12:32 PM, Vivian Meazza wrote: Emilian and I noticed that the local cloud effect files are using render bin 10 - thus competing with all other transparent objects, while the global clouds use render bin 9 - the dedicated cloud bin. We have tried moving all the clouds to render-bin 9: Emilian reports a significant gain in performance but I see little change here. However, we are not aware of the reason for the use of different bins. The current GIT local weather cloud generation goes through the same code path as the global clouds, and so will use render-bin 9. I suspect those effect file are used for some fo the more complex clouds that have not been moved across for various (good) reasons. The use of a separate render-bin was for performance reasons (in particular we handle ordering of sprites within the cloud system itself). On my system, changing to render bin 10 halved frame rates. Unfortunately, we are only scratching around the edges for improved performance using the existing technique, and really we need to try something a bit more radical. Agreed. -Stuart -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Unfortunately, we are only scratching around the edges for improved performance using the existing technique, and really we need to try something a bit more radical. I read that X-Plane 10 is using a complete new system for clouds (LOD based): I haven't seen this in action, and don't know how good this really is. But this is definitely some food for the brain. Here is what they write: So, how do we render these clouds in x-plane 10? the answer is levels of detail, which is a display of resolutions of buckets. we have a HIGH resolution of cloud geometry. each puff is 1 meter across. BUT, those puffs only go out, say, 10 meters. in other words, sure: you can see cloud puffs that are 1 meter across. but you can only see them up to 10 meters away. you see the small, detailed puffs up close only. that ain't many puffs. just 10x10x10, or 1,000 puffs. we can handle that. then, for puffs that are 10 meters away to 100 meters away, we draw puffs that are 10 meters across! how many puffs to fill that space? well, we have 1 puff every 10 meters, we go out 100 meters in visibility for a moment, so that is 10 puffs north/south x 10 east/west x 10 up/down, for 1,000 puffs. hhmmm... we just covered 10x the space, but without using the exact SAME number of puffs as we did for the smaller area, by making each puff bigger. sure, the bigger puffs are less detailed, but since they are farther away, you could not see the 1-meter-diameter puffs anyway... they are too far away to see when they are that small. then, from 100 meters to 1,000 meters, we draw puffs that are 100 meters across. we have 1 puff every 100 meters for 1,000-meter range. that is that is 10 puffs north/south x 10 east/west x 10 up/down, for 1,000 puffs. again, only 1,000 puffs. BUT, we are covering a full kilometer! the better part of a mile! how about we go out 100 kilometers??? each puff is, say, 10 kilometers across if drawn so far away. that is 1 puff every 10 kilometers for 100 kilometers. so that is 10 puffs north/south x 10 east/west x 10 up/down, for 1,000 puffs. AGAIN! NO MATTER HOW BIG THE SKY, EACH LEVEL OF DETAIL ONLY COSTS US 1,000 PUFFS! 7 levels of detail is plenty of detail. we call each level of detail a 'resolution'. so 7 resolutions gives 1-meter wide puffs of close, and huge puffs very far away, as seen from orbit!!! and we wind up with: 7,000 puffs. cheers, Robert -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Trying to get more performance out of the 3Dclouds!
Hi, I read that X-Plane 10 is using a complete new system for clouds (LOD based): I haven't seen this in action, and don't know how good this really is. I have tried the X-Plane 10 Demo. I have a DualCore 2.6 Ghz, 4 GB Ram, Nvidea GTX460. While graphic looks really good, and the idea behind the new clouds system seems to be a major progress for X-Plane, I must say, I'm rather dissapointed from X-Plane 10. Even with low settings and (!)clear sky I only get barely 30fps. With some clouds, it gets down to 20fps and less. I didn't had the chance to see the clouds at max settings. And this even gets worse using a aircraft with modeled 3d-cockpit. And forget about the fancy HDR-lighting- 7fps was the maximum with. Their recommendation is to have a more modern Multi-core computer for using X-Pane. So I'm not sure if this will really help us. While trying to get the best settings for my Alaska-Screenshots: http://www.flightgear.org/forums/viewtopic.php?f=19t=14482 I noticed again, that the most problem was the transparency right after the amount of cloudlets. Thorstens Local weather seems to behave sometimes better than the Global weather, sometimes worse. Difficult to tell when and why. Transparency kills all- that's the main problem in my eyes. Heiko still in work: http://www.hoerbird.net/galerie.html But already done: http://www.hoerbird.net/reisen.html -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
