Re: [osg-users] Supporting multiple frame rates
On 4/28/2011 9:39 AM, Brad Huber wrote: > We currently have an OSG based application which renders at 30-60+ fps on our > development > hardware with discrete gpus. Unfortunately we have to be able to deploy on > a wide range > of hardware including stuff that is significantly less capable (Intel GPUs, > etc). On some > of these slower platforms we are seeing framerates that are 10-15 fps. This > is actually > ok for us with one exception, live video. > We show live video in the scene graph (for now strictly in the HUD). We have > a > requirement to make the video render at native speed (~30 fps). I am looking > for a way to > make the HUD/video render at 30 fps and let the rest of the scene graph > render at a > floating rate (whatever the machine can handle). > How might this be accomplished? Since this is a HUD, obviously it must > ultimately render > to the same graphics context, however it doesn’t really need to respect the > depth buffer, > etc of the rest of the scene (it should always render on top). > PS Should I be thinking about render to texture? I would. This is a long shot, but what if you rendered the 3D scene to a texture using double alternating texture buffers. While the 3D scene is rendering to the buffer, the HUD can render the previous frame with the video on top of it to the actual screen. When the next video frame is ready, if the next 3D scene hasn't finished rendering, just re-use the previous frame's texture. When the 3D scene does finish rendering, set a flag to indicate it should now be used by the HUD, and begin rendering the NEXT 3D frame. Share resources (a la wglShareLists) so that there isn't a copy overhead transferring the rendered texture around. I've never tried it, but I think it could work, as long as your hardware can deal with rendering a quick HUD in the midst of rendering the complex 3d background scene. -- Chris 'Xenon' Hanson, omo sanza lettere. xe...@alphapixel.com http://www.alphapixel.com/ Digital Imaging. OpenGL. Scene Graphs. GIS. GPS. Training. Consulting. Contracting. "There is no Truth. There is only Perception. To Perceive is to Exist." - Xen ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Supporting multiple frame rates
Ok Thanks. This gives an idea of how to pursue a couple of different options. I'll post back with any results. Thanks -Brad -Original Message- From: osg-users-boun...@lists.openscenegraph.org [mailto:osg-users-boun...@lists.openscenegraph.org] On Behalf Of Robert Osfield Sent: Thursday, April 28, 2011 12:27 PM To: OpenSceneGraph Users Subject: Re: [osg-users] Supporting multiple frame rates Hi Brad, I also thought about the two context route but didn't suggest it as I'm doubtful that the graphics driver/card will be able to interleave the rendering on the two threads in way that doesn't have one thread stalling the other as they contend for resources on the graphics card. Using a pbuffer to render the 3d scene and the main graphics window doing the compositing would be the easier option to implement though, so I'd suggest trying it out, you'd need to use two viewers to decouple the frames, at least then you'd know whether it's good enough a solution or not. Robert. On Thu, Apr 28, 2011 at 5:52 PM, Brad Huber wrote: > Robert, Farshid, > > Thanks for the ideas. > > Given what you've said, would it make more sense to pursue using two > contexts? Perhaps the main scene would render to an invisible "off screen" > context and then the visible context would copy the frame buffer from > the invisible context and render the video frames on top of that. I > imagine there would be some difficulty in having additional threading > complexity to deal with. > > -Brad > > -Original Message- > From: osg-users-boun...@lists.openscenegraph.org > [mailto:osg-users-boun...@lists.openscenegraph.org] On Behalf Of > Robert Osfield > Sent: Thursday, April 28, 2011 9:59 AM > To: OpenSceneGraph Users > Subject: Re: [osg-users] Supporting multiple frame rates > > Hi Brad, > > I'm afraid decoupling the rendering of two parts of graphics window so > that they can run at their own frame rate won't be a straight forward > task, and well off the standard viewer path used by the vast majority > of the community. You are infact the first I recall wanting to do > exactly this combination of video at 30fps and man scene at it's own frame rate. > > Thoughts off the top of my head: > > You only need to do something special when 3D scene takes longer > than 30ms and would cause > frame rate to drop below the min 30fps, so the follow suggestions > are for when frame rate would > drop below 30. > > Could you render the video to the front buffer and just overlay the > background 3D scene? This > would enable you to remove the need for a swap buffers call at the > end of the render of teh video texture. > > Another approach would be to render the 3D scene into a texture and > then compose each frame > from the 3D scene texture and the video texture and do the swap > buffers at video texture frame rate. > > In both cases you the 3D scene is taking too long to render, and > since you are using a single > graphics context there isn't a natural way to split up and decouple > the rendering completely, > so to be able to put out frames at 30fps you'll need to break the > render of the 3D scene into > segments each of which can fit into with the time available when > rendering at 30fps. > > Splitting the scene won't be trivial, the easist method would be > something like depth partitioning, > and render different partition in depth and balance the depth range > for each so that the load is > as well balanced as you can get it. > > That's my best suggestions so far... Far from trivial to set up, very > much on the bleeding edge! > > Robert. > > > > > > On Thu, Apr 28, 2011 at 4:39 PM, Brad Huber wrote: >> Hello, >> >> >> >> We currently have an OSG based application which renders at 30-60+ >> fps on our development hardware with discrete gpus. Unfortunately we >> have to be able to deploy on a wide range of hardware including >> stuff that is significantly less capable (Intel GPUs, etc). On some >> of these slower platforms we are seeing framerates that are 10-15 >> fps. This is actually ok for us with one exception, live video. >> >> >> >> We show live video in the scene graph (for now strictly in the HUD). >> We have a requirement to make the video render at native speed (~30 >> fps). I am looking for a way to make the HUD/video render at 30 fps >> and let the rest of the scene graph render at a floating rate >> (whatever > the machine can handle). >> >> >> >> How might this be accomplished? Since this is a HUD
Re: [osg-users] Supporting multiple frame rates
Overlay buffer comes to mind given the selection and clearing control. But, if I remember right, they are limited to color index mode. Newer hardware of the day may allow RGB mode. We did this type of multiplexing with Performer years ago, but with different windows. We rendered the scene at a full 60 Hz and alternated HUD insets at 30hz. Much easier than trying to do it on the same canvas. Bob Buckley Principal Software Engineer Raytheon -Original Message- From: osg-users-boun...@lists.openscenegraph.org [mailto:osg-users-boun...@lists.openscenegraph.org] On Behalf Of Brad Huber Sent: Thursday, April 28, 2011 9:40 AM To: osg-users@lists.openscenegraph.org Subject: [osg-users] Supporting multiple frame rates Hello, We currently have an OSG based application which renders at 30-60+ fps on our development hardware with discrete gpus. Unfortunately we have to be able to deploy on a wide range of hardware including stuff that is significantly less capable (Intel GPUs, etc). On some of these slower platforms we are seeing framerates that are 10-15 fps. This is actually ok for us with one exception, live video. We show live video in the scene graph (for now strictly in the HUD). We have a requirement to make the video render at native speed (~30 fps). I am looking for a way to make the HUD/video render at 30 fps and let the rest of the scene graph render at a floating rate (whatever the machine can handle). How might this be accomplished? Since this is a HUD, obviously it must ultimately render to the same graphics context, however it doesn't really need to respect the depth buffer, etc of the rest of the scene (it should always render on top). PS Should I be thinking about render to texture? Related threads I came across: http://forum.openscenegraph.org/viewtopic.php?t=7858 http://lists.openscenegraph.org/pipermail/osg-users-openscenegraph.org/2008-January/006000.html http://forum.openscenegraph.org/viewtopic.php?t=7117 Thanks -Brad ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Supporting multiple frame rates
Hi Brad, I also thought about the two context route but didn't suggest it as I'm doubtful that the graphics driver/card will be able to interleave the rendering on the two threads in way that doesn't have one thread stalling the other as they contend for resources on the graphics card. Using a pbuffer to render the 3d scene and the main graphics window doing the compositing would be the easier option to implement though, so I'd suggest trying it out, you'd need to use two viewers to decouple the frames, at least then you'd know whether it's good enough a solution or not. Robert. On Thu, Apr 28, 2011 at 5:52 PM, Brad Huber wrote: > Robert, Farshid, > > Thanks for the ideas. > > Given what you've said, would it make more sense to pursue using two > contexts? Perhaps the main scene would render to an invisible "off screen" > context and then the visible context would copy the frame buffer from the > invisible context and render the video frames on top of that. I imagine > there would be some difficulty in having additional threading complexity to > deal with. > > -Brad > > -Original Message- > From: osg-users-boun...@lists.openscenegraph.org > [mailto:osg-users-boun...@lists.openscenegraph.org] On Behalf Of Robert > Osfield > Sent: Thursday, April 28, 2011 9:59 AM > To: OpenSceneGraph Users > Subject: Re: [osg-users] Supporting multiple frame rates > > Hi Brad, > > I'm afraid decoupling the rendering of two parts of graphics window so that > they can run at their own frame rate won't be a straight forward task, and > well off the standard viewer path used by the vast majority of the > community. You are infact the first I recall wanting to do exactly this > combination of video at 30fps and man scene at it's own frame rate. > > Thoughts off the top of my head: > > You only need to do something special when 3D scene takes longer than > 30ms and would cause > frame rate to drop below the min 30fps, so the follow suggestions are > for when frame rate would > drop below 30. > > Could you render the video to the front buffer and just overlay the > background 3D scene? This > would enable you to remove the need for a swap buffers call at the end > of the render of teh video texture. > > Another approach would be to render the 3D scene into a texture and then > compose each frame > from the 3D scene texture and the video texture and do the swap buffers > at video texture frame rate. > > In both cases you the 3D scene is taking too long to render, and since > you are using a single > graphics context there isn't a natural way to split up and decouple the > rendering completely, > so to be able to put out frames at 30fps you'll need to break the render > of the 3D scene into > segments each of which can fit into with the time available when > rendering at 30fps. > > Splitting the scene won't be trivial, the easist method would be > something like depth partitioning, > and render different partition in depth and balance the depth range for > each so that the load is > as well balanced as you can get it. > > That's my best suggestions so far... Far from trivial to set up, very much > on the bleeding edge! > > Robert. > > > > > > On Thu, Apr 28, 2011 at 4:39 PM, Brad Huber wrote: >> Hello, >> >> >> >> We currently have an OSG based application which renders at 30-60+ fps >> on our development hardware with discrete gpus. Unfortunately we have >> to be able to deploy on a wide range of hardware including stuff that >> is significantly less capable (Intel GPUs, etc). On some of these >> slower platforms we are seeing framerates that are 10-15 fps. This is >> actually ok for us with one exception, live video. >> >> >> >> We show live video in the scene graph (for now strictly in the HUD). >> We have a requirement to make the video render at native speed (~30 >> fps). I am looking for a way to make the HUD/video render at 30 fps >> and let the rest of the scene graph render at a floating rate (whatever > the machine can handle). >> >> >> >> How might this be accomplished? Since this is a HUD, obviously it >> must ultimately render to the same graphics context, however it >> doesn’t really need to respect the depth buffer, etc of the rest of >> the scene (it should always render on top). >> >> >> >> PS Should I be thinking about render to texture? >> >> >> >> Related threads I came across: >> >> http://forum.openscenegraph.org/viewtopic.php?t=7858 >> &g
Re: [osg-users] Supporting multiple frame rates
Robert, Farshid, Thanks for the ideas. Given what you've said, would it make more sense to pursue using two contexts? Perhaps the main scene would render to an invisible "off screen" context and then the visible context would copy the frame buffer from the invisible context and render the video frames on top of that. I imagine there would be some difficulty in having additional threading complexity to deal with. -Brad -Original Message- From: osg-users-boun...@lists.openscenegraph.org [mailto:osg-users-boun...@lists.openscenegraph.org] On Behalf Of Robert Osfield Sent: Thursday, April 28, 2011 9:59 AM To: OpenSceneGraph Users Subject: Re: [osg-users] Supporting multiple frame rates Hi Brad, I'm afraid decoupling the rendering of two parts of graphics window so that they can run at their own frame rate won't be a straight forward task, and well off the standard viewer path used by the vast majority of the community. You are infact the first I recall wanting to do exactly this combination of video at 30fps and man scene at it's own frame rate. Thoughts off the top of my head: You only need to do something special when 3D scene takes longer than 30ms and would cause frame rate to drop below the min 30fps, so the follow suggestions are for when frame rate would drop below 30. Could you render the video to the front buffer and just overlay the background 3D scene? This would enable you to remove the need for a swap buffers call at the end of the render of teh video texture. Another approach would be to render the 3D scene into a texture and then compose each frame from the 3D scene texture and the video texture and do the swap buffers at video texture frame rate. In both cases you the 3D scene is taking too long to render, and since you are using a single graphics context there isn't a natural way to split up and decouple the rendering completely, so to be able to put out frames at 30fps you'll need to break the render of the 3D scene into segments each of which can fit into with the time available when rendering at 30fps. Splitting the scene won't be trivial, the easist method would be something like depth partitioning, and render different partition in depth and balance the depth range for each so that the load is as well balanced as you can get it. That's my best suggestions so far... Far from trivial to set up, very much on the bleeding edge! Robert. On Thu, Apr 28, 2011 at 4:39 PM, Brad Huber wrote: > Hello, > > > > We currently have an OSG based application which renders at 30-60+ fps > on our development hardware with discrete gpus. Unfortunately we have > to be able to deploy on a wide range of hardware including stuff that > is significantly less capable (Intel GPUs, etc). On some of these > slower platforms we are seeing framerates that are 10-15 fps. This is > actually ok for us with one exception, live video. > > > > We show live video in the scene graph (for now strictly in the HUD). > We have a requirement to make the video render at native speed (~30 > fps). I am looking for a way to make the HUD/video render at 30 fps > and let the rest of the scene graph render at a floating rate (whatever the machine can handle). > > > > How might this be accomplished? Since this is a HUD, obviously it > must ultimately render to the same graphics context, however it > doesnt really need to respect the depth buffer, etc of the rest of > the scene (it should always render on top). > > > > PS Should I be thinking about render to texture? > > > > Related threads I came across: > > http://forum.openscenegraph.org/viewtopic.php?t=7858 > > http://lists.openscenegraph.org/pipermail/osg-users-openscenegraph.org > /2008-January/006000.html > > http://forum.openscenegraph.org/viewtopic.php?t=7117 > > > > Thanks > > -Brad > > ___ > osg-users mailing list > osg-users@lists.openscenegraph.org > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph. > org > > ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Supporting multiple frame rates
Hi Brad, I haven't tried this myself, but it might work. Create a separate context/thread that renders the scene to a texture. Once it finishes rendering, use the NV_copy_image extension ( http://developer.download.nvidia.com/opengl/specs/GL_NV_copy_image.txt) to copy the image into the main context containing the HUD. I haven't used the NV_copy_image extension myself, but from my understanding it allows copying texture data between different contexts. I'm not sure if ATI has a similar extension. Cheers, Farshid On Thu, Apr 28, 2011 at 8:39 AM, Brad Huber wrote: > Hello, > > > > We currently have an OSG based application which renders at 30-60+ fps on > our development hardware with discrete gpus. Unfortunately we have to be > able to deploy on a wide range of hardware including stuff that is > significantly less capable (Intel GPUs, etc). On some of these slower > platforms we are seeing framerates that are 10-15 fps. This is actually ok > for us with one exception, live video. > > > > We show live video in the scene graph (for now strictly in the HUD). We > have a requirement to make the video render at native speed (~30 fps). I am > looking for a way to make the HUD/video render at 30 fps and let the rest of > the scene graph render at a floating rate (whatever the machine can handle). > > > > How might this be accomplished? Since this is a HUD, obviously it must > ultimately render to the same graphics context, however it doesn’t really > need to respect the depth buffer, etc of the rest of the scene (it should > always render on top). > > > > PS Should I be thinking about render to texture? > > > > Related threads I came across: > > http://forum.openscenegraph.org/viewtopic.php?t=7858 > > > http://lists.openscenegraph.org/pipermail/osg-users-openscenegraph.org/2008-January/006000.html > > http://forum.openscenegraph.org/viewtopic.php?t=7117 > > > > Thanks > > -Brad > > ___ > osg-users mailing list > osg-users@lists.openscenegraph.org > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org > > ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Supporting multiple frame rates
Hi Brad, I'm afraid decoupling the rendering of two parts of graphics window so that they can run at their own frame rate won't be a straight forward task, and well off the standard viewer path used by the vast majority of the community. You are infact the first I recall wanting to do exactly this combination of video at 30fps and man scene at it's own frame rate. Thoughts off the top of my head: You only need to do something special when 3D scene takes longer than 30ms and would cause frame rate to drop below the min 30fps, so the follow suggestions are for when frame rate would drop below 30. Could you render the video to the front buffer and just overlay the background 3D scene? This would enable you to remove the need for a swap buffers call at the end of the render of teh video texture. Another approach would be to render the 3D scene into a texture and then compose each frame from the 3D scene texture and the video texture and do the swap buffers at video texture frame rate. In both cases you the 3D scene is taking too long to render, and since you are using a single graphics context there isn't a natural way to split up and decouple the rendering completely, so to be able to put out frames at 30fps you'll need to break the render of the 3D scene into segments each of which can fit into with the time available when rendering at 30fps. Splitting the scene won't be trivial, the easist method would be something like depth partitioning, and render different partition in depth and balance the depth range for each so that the load is as well balanced as you can get it. That's my best suggestions so far... Far from trivial to set up, very much on the bleeding edge! Robert. On Thu, Apr 28, 2011 at 4:39 PM, Brad Huber wrote: > Hello, > > > > We currently have an OSG based application which renders at 30-60+ fps on > our development hardware with discrete gpus. Unfortunately we have to be > able to deploy on a wide range of hardware including stuff that is > significantly less capable (Intel GPUs, etc). On some of these slower > platforms we are seeing framerates that are 10-15 fps. This is actually ok > for us with one exception, live video. > > > > We show live video in the scene graph (for now strictly in the HUD). We > have a requirement to make the video render at native speed (~30 fps). I am > looking for a way to make the HUD/video render at 30 fps and let the rest of > the scene graph render at a floating rate (whatever the machine can handle). > > > > How might this be accomplished? Since this is a HUD, obviously it must > ultimately render to the same graphics context, however it doesn’t really > need to respect the depth buffer, etc of the rest of the scene (it should > always render on top). > > > > PS Should I be thinking about render to texture? > > > > Related threads I came across: > > http://forum.openscenegraph.org/viewtopic.php?t=7858 > > http://lists.openscenegraph.org/pipermail/osg-users-openscenegraph.org/2008-January/006000.html > > http://forum.openscenegraph.org/viewtopic.php?t=7117 > > > > Thanks > > -Brad > > ___ > osg-users mailing list > osg-users@lists.openscenegraph.org > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org > > ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
