RFC: OpenGL + XvMC
I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function to XvMC. I have implemented this in NVIDIA's binary drivers and am able to do full framerate HDTV video textures on the higher end GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer contents into a texture. Status XvMCCopySurfaceToGLXPbuffer ( Display *display, XvMCSurface *surface, XID pbuffer_id, short src_x, short src_y, unsigned short width, unsigned short height, short dst_x, short dst_y, int flags ); This function copies the rectangle specified by src_x, src_y, width, and height from the XvMCSurface denoted by "surface" to offset dst_x, dst_y within the pbuffer identified by its GLXPbuffer XID "pbuffer_id". Note that while the src_x, src_y are in XvMC's standard left-handed coordinate system and specify the upper left hand corner of the rectangle, dst_x and dst_y are in OpenGL's right-handed coordinate system and denote the lower left hand corner of the destination rectangle in the pbuffer. "Flags" may be XVMC_TOP_FIELD, XVMC_BOTTOM_FIELD or XVMC_FRAME_PICTURE. If flags is not XVMC_FRAME_PICTURE, the src_y and height are in field coordinates, not frame. That is, the total copyable height is half the height of the XvMCSurface. XvMCCopySurfaceToGLXPbuffer does not return until the copy to the pbuffer has completed. XvMCCopySurfaceToGLXPbuffer is pipelined with XvMCRenderSurface so no explicit synchronization between XvMCRenderSurface and XvMCCopySurfaceToGLXPbuffer is needed. The pbuffer must be of type GLX_RGBA, and the destination of the copy is the left front buffer of the pbuffer. Success is returned if no error occured, the error code is returned otherwise. Possible Errors: XvMCBadSurface - The surface is invalid. BadDrawable - The pbuffer_id is not a valid pbuffer. BadMatch - The pbuffer is not of type GLX_RGBA or the pbuffer does not have a front left buffer. XvMCCopySurfaceToGLXPbuffer is supported if the following flag: #define XVMC_COPY_TO_PBUFFER 0x0010 is set in the XvMCSurfaceInfo's flags field. I'd like to bump the API version up to 1.1 and add this. Comments? Mark. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
RE: OpenGL + XvMC
Let me preface my comment with "I don't know a lot about OGL" so some further clarification may be needed. I am assuming that pbuffers are basically buffers that can be used as textures by OGL. I would then assume that the OGL driver would have some mapping of pbuffer id to the texture memory it represents, maybe this memory is in video memory maybe it has been "swapped out" so-to-speak by some texture manager etc. So basically this copies data from an XvMC offscreen surface to an OGL offscreen surface to be used by OGL for normal rendering purposes. Seems easy enough... I expect anyone doing XvMC would use the drm for the direct access (or their own drm equivalent) which would also be the same drm used for OGL and therefore whatever texture management needs to be done should be possible without much of a problem. My main problem with the concept is that it seems that a copy is not always required, and is costly at 24fps. For YUV packed surfaces at least, an XvMC surface could be directly used as a texture. Some way to associate an XvMC surface with a pbuffer without a copy seems like something that would have a large performance gain. Also, what is the goal exactly? Are you trying to allow video to be used as textures within a 3d rendered scene, or are your trying to make it possible to do something like Xv, but using direct rendering and 3d hardware? If you are trying to do the latter, it seems far easier to just plug your XvMC extension into the 3d engine rather than into the overlay. I think you've done the equivalent with Xv already. -Matt -Original Message- From: Mark Vojkovich [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 4:30 PM To: [EMAIL PROTECTED] Subject: RFC: OpenGL + XvMC I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function to XvMC. I have implemented this in NVIDIA's binary drivers and am able to do full framerate HDTV video textures on the higher end GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer contents into a texture. Status XvMCCopySurfaceToGLXPbuffer ( Display *display, XvMCSurface *surface, XID pbuffer_id, short src_x, short src_y, unsigned short width, unsigned short height, short dst_x, short dst_y, int flags ); This function copies the rectangle specified by src_x, src_y, width, and height from the XvMCSurface denoted by "surface" to offset dst_x, dst_y within the pbuffer identified by its GLXPbuffer XID "pbuffer_id". Note that while the src_x, src_y are in XvMC's standard left-handed coordinate system and specify the upper left hand corner of the rectangle, dst_x and dst_y are in OpenGL's right-handed coordinate system and denote the lower left hand corner of the destination rectangle in the pbuffer. "Flags" may be XVMC_TOP_FIELD, XVMC_BOTTOM_FIELD or XVMC_FRAME_PICTURE. If flags is not XVMC_FRAME_PICTURE, the src_y and height are in field coordinates, not frame. That is, the total copyable height is half the height of the XvMCSurface. XvMCCopySurfaceToGLXPbuffer does not return until the copy to the pbuffer has completed. XvMCCopySurfaceToGLXPbuffer is pipelined with XvMCRenderSurface so no explicit synchronization between XvMCRenderSurface and XvMCCopySurfaceToGLXPbuffer is needed. The pbuffer must be of type GLX_RGBA, and the destination of the copy is the left front buffer of the pbuffer. Success is returned if no error occured, the error code is returned otherwise. Possible Errors: XvMCBadSurface - The surface is invalid. BadDrawable - The pbuffer_id is not a valid pbuffer. BadMatch - The pbuffer is not of type GLX_RGBA or the pbuffer does not have a front left buffer. XvMCCopySurfaceToGLXPbuffer is supported if the following flag: #define XVMC_COPY_TO_PBUFFER 0x0010 is set in the XvMCSurfaceInfo's flags field. I'd like to bump the API version up to 1.1 and add this. Comments? Mark. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: OpenGL + XvMC
Sottek, Matthew J wrote: Let me preface my comment with "I don't know a lot about OGL" so some further clarification may be needed. I am assuming that pbuffers are basically buffers that can be used as textures by OGL. I would then assume that the OGL driver would have some mapping of pbuffer id to the texture memory it represents, maybe this memory is in video memory maybe it has been "swapped out" so-to-speak by some texture manager etc. A pbuffer is (basically) just an off-screen window. You can do the same things to a pbuffer that you can do to a normal window. This includes copying its contents to a texture. There was a proposal to bring WGL_render_texture to GLX, but, in light of other developments, there wasn't much interest. It *may* be resurrected at some point for completeness sake, but I wouldn't hold my breath. So basically this copies data from an XvMC offscreen surface to an OGL offscreen surface to be used by OGL for normal rendering purposes. Seems easy enough... I expect anyone doing XvMC would use the drm for the direct access (or their own drm equivalent) which would also be the same drm used for OGL and therefore whatever texture management needs to be done should be possible without much of a problem. Well, except that, at least in the open-source DRI based drivers, the texture memory manager doesn't live in the DRM (anymore than malloc and free live in the kernel). My main problem with the concept is that it seems that a copy is not always required, and is costly at 24fps. For YUV packed surfaces at least, an XvMC surface could be directly used as a texture. Some way to associate an XvMC surface with a pbuffer without a copy seems like something that would have a large performance gain. It *may* not always be required. There have been GLX extensions in the past (see my first message in this thread) that worked that way. However, as we discussed earlier, this doesn't seem to work so well with MPEG video files. The main problem being that you don't get the frames exactly in order. You're stuck doing a copy either way. Also, what is the goal exactly? Are you trying to allow video to be used as textures within a 3d rendered scene, or are your trying to make it possible to do something like Xv, but using direct rendering and 3d hardware? If you are trying to do the latter, it seems far easier to just plug your XvMC extension into the 3d engine rather than into the overlay. I think you've done the equivalent with Xv already. I think the goal is to be able to do both. Although, the idea of using MPEG video files as animated textures in a game is pretty cool. :) ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
RE: OpenGL + XvMC
On Mon, 2 Jun 2003, Sottek, Matthew J wrote: > > Let me preface my comment with "I don't know a lot about OGL" so some > further clarification may be needed. > > I am assuming that pbuffers are basically buffers that can be used > as textures by OGL. I would then assume that the OGL driver would > have some mapping of pbuffer id to the texture memory it represents, > maybe this memory is in video memory maybe it has been "swapped out" > so-to-speak by some texture manager etc. A pbuffer is a GLXDrawable like a window is. You can make current on it and draw to it. The intention of pbuffers is that they're always in hardware. > > So basically this copies data from an XvMC offscreen surface to an > OGL offscreen surface to be used by OGL for normal rendering purposes. > Seems easy enough... I expect anyone doing XvMC would use the drm > for the direct access (or their own drm equivalent) which would also > be the same drm used for OGL and therefore whatever texture management > needs to be done should be possible without much of a problem. > > My main problem with the concept is that it seems that a copy is not > always required, and is costly at 24fps. For YUV packed surfaces at > least, an XvMC surface could be directly used as a texture. Some way > to associate an XvMC surface with a pbuffer without a copy seems > like something that would have a large performance gain. Can you use those non-power-of-two mpeg surfaces as normal textures without limitations? I don't think most hardware can do that, some possibly can't at all. I expect binding an XvMC surface as a texture isn't implementable without limitations on most hardware. It also requires that OpenGL do all the work for this since XvMC doesn't have access to OpenGL's texture namespace. It would also require cooperation between OpenGL and XvMC that makes me nervous. OpenGL would have to make sure that XvMC's Surface didn't go away while it was using it as a texture. An explict copy from the XvMCSurface to a pbuffer solves these problems with the cost of the extra copy, which at least for my hardware, isn't a big issue. > Also, what is the goal exactly? I am interested in getting mpeg into textures for the purpose of incorporating into 3D scenes and video editing/post production. Mark. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: OpenGL + XvMC
> It *may* not always be required. There have been GLX extensions in the > past (see my first message in this thread) that worked that way. > However, as we discussed earlier, this doesn't seem to work so well with > MPEG video files. The main problem being that you don't get the frames > exactly in order. You're stuck doing a copy either way. Why ? You have usually enough video/AGP/whatever texture memory to store multiple frames. I haven't looked at XvMc, but there is a difference between rendering frames and scheduling them for display, you render them to multiple buffers and schedule display when your next expected display frame is ready. I completely agree that it's a big waste to still have a copy in cases where the HW let you avoid it Ben. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
RE: OpenGL + XvMC
>Well, except that, at least in the open-source DRI based drivers, the >texture memory manager doesn't live in the DRM (anymore than malloc and >free live in the kernel). The texture manager itself may not but the drm is the arbitrator. As I understand it the app takes the lock and then manages the texture memory itself. The "good neighbor" approach, where the supervisor process just controls the locks and the apps are expected to play nice. An XvMC app can "play nice" by the same rules as the OGL app. The point was that there are no real technical roadblocks to making an direct rendered XvMC application use the same texture manager. >It *may* not always be required. There have been GLX extensions in the >past (see my first message in this thread) that worked that way. >However, as we discussed earlier, this doesn't seem to work so well >with MPEG video files. The main problem being that you don't get >the frames exactly in order. You're stuck doing a copy either way. That is only a problem if you want to connect a "stream" of data to a texture. I.e. just bind an XvMC stream to a texture and have it auto-update. That is nearly impossible... XvMC has individual surfaces for each frame. The surfaces "live" for the life of the decode, they are not freed when the frame is done being displayed. So you have maybe 6 XvMC surfaces that you use again and again. That surface may, on some hardware, be directly usable with the 3d engine without a needed copy. >I think the goal is to be able to do both. Although, the idea of using >MPEG video files as animated textures in a game is pretty cool. :) Mpeg in textures is a cool idea, and something of this nature would be needed for that. Using 3d hardware to display the video doesn't need this concept and due to the surface copy is probably drastically slower than using XvMC without OGL and just hooking it's final on-screen render up to the XvMCPutSurface. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
RE: OpenGL + XvMC
> Can you use those non-power-of-two mpeg surfaces as normal >textures without limitations? I don't think most hardware can >do that, some possibly can't at all. Well for one thing my XvMC surfaces ARE power of two, but that is not the point. The HWMC engine used the very same texture engine as is used for 3d. So while the planar surfaces are not _great_ for use as textures it can be done. It is probably just as much bandwidth due to an internal conversion but the YUV planar to YUV packed conversion could happen render time into a temporary buffer. In the end, I think this is more of a neat trick than anything else so I don't think it matters a whole lot if there is an extra copy. I keep thinking that some sort of Direct Rendered Video extension would be very useful for X. You could then alloc a direct video surface that was mappable. Populate it from the final write in the decode process (From ANY codec) then either do a Put() a Blend() or a CopytoPBuffer(). The CopytoPBuffer() may be unnecessary if you could do a CreatePBufferFromXvD() to share the surface. In such a scenario I think the ability to save the copy is quite important. > I am interested in getting mpeg into textures for the purpose >of incorporating into 3D scenes and video editing/post production. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
Here is the latest version of the proposed change to the XvMC API specification: Status XvMCCopySurfaceToGLXPbuffer ( Display *display, XvMCSurface *surface, XID pbuffer_id, short src_x, short src_y, unsigned short width, unsigned short height, short dst_x, short dst_y, unsigned int draw_buffer, int flags ); This function copies a rectangle of dimension "width" by "height" from the XvMCSurface indicated by "surface" to the GLXPbuffer identified by its XID "pbuffer_id". The locations of source and destination rectangles are specified differently for the XvMCSurface source and the GLXPbuffer destination as follows. On the XvMCSurface, the rectangle location is specified in the usual X-Window fashion with src_x and src_y indicating the location of the upper left hand corner of the rectangle relative to the XvMCSurface origin (the upper left hand corner of the XvMCSurface with positive Y axis going in the down direction). On the GLXPbuffer the rectangle location is specified in the usual OpenGL fashion with the dst_x and dst_y indicating the location of the lower left hand corner of the rectangle relative to the GLXPbuffer origin (the lower left hand corner of the GLXPbuffer with the positive Y axis going in the up direction). The "draw_buffer" takes the same OpenGL enums that glDrawBuffer() takes (ie. GL_FRONT_LEFT, GL_FRONT_RIGHT, GL_BACK_LEFT, GL_BACK_RIGHT, GL_FRONT, GL_BACK, GL_LEFT, GL_RIGHT or GL_FRONT_AND_BACK). This indicates which buffer of the GLXPbuffer is to be used for the destination of the copy. Buffers specified in the "draw_buffer" that do not exist in the GLXPbuffer are ignored. "Flags" may be XVMC_TOP_FIELD, XVMC_BOTTOM_FIELD or XVMC_FRAME_PICTURE. If flags is not XVMC_FRAME_PICTURE, the src_y and height are in field coordinates, not frame. That is, the total copyable height is half the height of the XvMCSurface. XvMCCopySurfaceToGLXPbuffer does not return until the copy to the pbuffer has completed. XvMCCopySurfaceToGLXPbuffer is pipelined with XvMCRenderSurface so no explicit synchronization between XvMCRenderSurface and XvMCCopySurfaceToGLXPbuffer is needed. The pbuffer must be of type GLX_RGBA. Success is returned if no error occured, the error code is returned otherwise. Possible Errors: XvMCBadSurface - The surface is invalid. BadDrawable - The pbuffer_id is not a valid pbuffer. BadMatch - The pbuffer is not of type GLX_RGBA. XvMCCopySurfaceToGLXPbuffer is supported if the following flag: #define XVMC_COPY_TO_PBUFFER 0x0010 is set in the XvMCSurfaceInfo's flags field. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
Mark Vojkovich wrote: I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function to XvMC. I have implemented this in NVIDIA's binary drivers and am able to do full framerate HDTV video textures on the higher end GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer contents into a texture. This shoulds like a good candidate for a GLX extension. I've been wondering when someone would suggest somthing like this. :) Although, I did expect it to come from someone doing video capture work first. Over the years there have been a couple extensions for doing things this, both from SGI. They both work by streaming video data into a new type of GLX drawable and use that to source pixel / texel data. http://oss.sgi.com/projects/ogl-sample/registry/SGIX/video_source.txt http://oss.sgi.com/projects/ogl-sample/registry/SGIX/dmbuffer.txt The function that you're suggesting here is a clear break from that. I don't think that's a bad thing. I suspect that you designed it this way so that the implementation would not have to live in the GLX subsystem or in the 3D driver, correct? With slightly closer integration between XvMC and the 3D driver, we ought to be able to do something along those lines. Basically, bind a XvMCSurface to a pbuffer. Then, each time a new frame of video is rendered the pbuffer would be automatically updated. Given the way the XvMC works, I'm not sure how well that would work, though. I'll have to think on it some more. Status XvMCCopySurfaceToGLXPbuffer ( Display *display, XvMCSurface *surface, XID pbuffer_id, short src_x, short src_y, unsigned short width, unsigned short height, short dst_x, short dst_y, int flags ); One quick comment. Don't use 'short', use 'int'. On every existing and future platform that we're likely to care about the shorts will take up as much space as an int on the stack anyway, and slower / larger / more instructions will need to be used to access them. This function copies the rectangle specified by src_x, src_y, width, and height from the XvMCSurface denoted by "surface" to offset dst_x, dst_y within the pbuffer identified by its GLXPbuffer XID "pbuffer_id". Note that while the src_x, src_y are in XvMC's standard left-handed coordinate system and specify the upper left hand corner of the rectangle, dst_x and dst_y are in OpenGL's right-handed coordinate system and denote the lower left hand corner of the destination rectangle in the pbuffer. This conceptually concerns me. Mixing coordinate systems is usually a bad call, and is likely to confuse developers. I assume this means that the image is implicitly inverted? Hmm... "Flags" may be XVMC_TOP_FIELD, XVMC_BOTTOM_FIELD or XVMC_FRAME_PICTURE. If flags is not XVMC_FRAME_PICTURE, the src_y and height are in field coordinates, not frame. That is, the total copyable height is half the height of the XvMCSurface. XvMCCopySurfaceToGLXPbuffer does not return until the copy to the pbuffer has completed. XvMCCopySurfaceToGLXPbuffer is pipelined with XvMCRenderSurface so no explicit synchronization between XvMCRenderSurface and XvMCCopySurfaceToGLXPbuffer is needed. The pbuffer must be of type GLX_RGBA, and the destination of the copy is the left front buffer of the pbuffer. Success is returned if no error occured, the error code is returned otherwise. This goes against common practice. The copy should obey the setting of glDrawBuffer. I assume you must have had some reason for doing this...I just can't imagine what it was. :) Possible Errors: XvMCBadSurface - The surface is invalid. BadDrawable - The pbuffer_id is not a valid pbuffer. BadMatch - The pbuffer is not of type GLX_RGBA or the pbuffer does not have a front left buffer. XvMCCopySurfaceToGLXPbuffer is supported if the following flag: #define XVMC_COPY_TO_PBUFFER 0x0010 is set in the XvMCSurfaceInfo's flags field. I'd like to bump the API version up to 1.1 and add this. Comments? ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
On Fri, 30 May 2003, Ian Romanick wrote: > Mark Vojkovich wrote: > >I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function > > to XvMC. I have implemented this in NVIDIA's binary drivers and > > am able to do full framerate HDTV video textures on the higher end > > GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer > > contents into a texture. > > This shoulds like a good candidate for a GLX extension. I've been > wondering when someone would suggest somthing like this. :) Although, I > did expect it to come from someone doing video capture work first. I wanted to avoid something from the GLX side. Introducing the concept of an XFree86 video extension buffer to GLX seemed like a hard sell. Introducting a well establish GLX drawable type to XvMC seemed more reasonable. > > Over the years there have been a couple extensions for doing things > this, both from SGI. They both work by streaming video data into a new > type of GLX drawable and use that to source pixel / texel data. > >http://oss.sgi.com/projects/ogl-sample/registry/SGIX/video_source.txt >http://oss.sgi.com/projects/ogl-sample/registry/SGIX/dmbuffer.txt > > The function that you're suggesting here is a clear break from that. I > don't think that's a bad thing. I suspect that you designed it this way > so that the implementation would not have to live in the GLX subsystem > or in the 3D driver, correct? That was one of the goals. I generally view trying to bind a video-specific buffer to an OpenGL buffer as a bad idea since they always end up as second class. While there have been implementations that could use video buffers as textures, etc... they've always had serious limitations like the inability to have mipmaps, or repeat, or limited filtering ability or other disapointing things that people are sad to learn about. I opted instead for an explicit copy from a video-specific surface to a first-class OpenGL drawable. Being able to do HDTV video textures on a P4 1.2 Gig PC with a $100 video card has show this to be a reasonable tradeoff. > > With slightly closer integration between XvMC and the 3D driver, we > ought to be able to do something along those lines. Basically, bind a > XvMCSurface to a pbuffer. Then, each time a new frame of video is > rendered the pbuffer would be automatically updated. Given the way the > XvMC works, I'm not sure how well that would work, though. I'll have to > think on it some more. Mpeg frames are displayed in a different order than they are rendered. It's best if the decoder has full control over what goes where and when. > > > Status > > XvMCCopySurfaceToGLXPbuffer ( > > Display *display, > > XvMCSurface *surface, > > XID pbuffer_id, > > short src_x, > > short src_y, > > unsigned short width, > > unsigned short height, > > short dst_x, > > short dst_y, > > int flags > > ); > > One quick comment. Don't use 'short', use 'int'. On every existing and > future platform that we're likely to care about the shorts will take up > as much space as an int on the stack anyway, and slower / larger / more > instructions will need to be used to access them. This is an X-window extension. It's limited to the signed 16 bit coordinate system like the X-window system itself, all of Xlib and the rest of XvMC. > > >This function copies the rectangle specified by src_x, src_y, width, > > and height from the XvMCSurface denoted by "surface" to offset dst_x, dst_y > > within the pbuffer identified by its GLXPbuffer XID "pbuffer_id". > > Note that while the src_x, src_y are in XvMC's standard left-handed > > coordinate system and specify the upper left hand corner of the > > rectangle, dst_x and dst_y are in OpenGL's right-handed coordinate > > system and denote the lower left hand corner of the destination > > rectangle in the pbuffer. > > This conceptually concerns me. Mixing coordinate systems is usually a > bad call, and is likely to confuse developers. I assume this means that > the image is implicitly inverted? Hmm... X is left handed. OpenGL is right handed. Addressing a pbuffer in anything other than a right handed coordinate system is perverse. Mixed coordinate systems seems a necessity here. There is no inversion, it's just a remap of the origins. > > > "Flags" may be XVMC_TOP_FIELD, XVMC_BOTTOM_FIELD or XVMC_FRAME_PICTURE. > > If flags is not XVMC_FRAME_PICTURE, the src_y and height are in field > > coordinates, not frame. That is, the total copyable height is half > > the height of the XvMCSurface. > > > > XvMCCopySurfaceToGLXPbuffer does not return until the copy to the > > pbuffer has completed. XvMCCopySurfaceToGLXPbuffer is pipelined > > with XvMCRenderSurface so no explicit synchronization between > > XvMCRenderSurface and XvMCCopySurfaceToGLXPbuffer is needed. > > > > The pbuffer must be of type GLX_RGBA, and the destinati
Re: RFC: OpenGL + XvMC
Mark Vojkovich wrote: On Fri, 30 May 2003, Ian Romanick wrote: Mark Vojkovich wrote: I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function to XvMC. I have implemented this in NVIDIA's binary drivers and am able to do full framerate HDTV video textures on the higher end GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer contents into a texture. This shoulds like a good candidate for a GLX extension. I've been wondering when someone would suggest somthing like this. :) Although, I did expect it to come from someone doing video capture work first. I wanted to avoid something from the GLX side. Introducing the concept of an XFree86 video extension buffer to GLX seemed like a hard sell. Introducting a well establish GLX drawable type to XvMC seemed more reasonable. Right. I thought about this a bit more last night. A better approach might be to expose this functionality as an XFree86 extension, then create a GLX extension on top of it. I was thinking of an extension where you would bind a "magic" buffer to a pbuffer, then take a snapshot from the input buffer to the pbuffer. Doing that we could create layered extensions for binding v4l streams to pbuffers. This would be like creating a subclass in C++ and just over-riding the virtual CaptureImage method. I think that would be much nicer for application code. At the same time, all of the real work would still be done in the X extension (or v4l). Only some light-weight bookkeeping would live in GLX. Over the years there have been a couple extensions for doing things this, both from SGI. They both work by streaming video data into a new type of GLX drawable and use that to source pixel / texel data. http://oss.sgi.com/projects/ogl-sample/registry/SGIX/video_source.txt http://oss.sgi.com/projects/ogl-sample/registry/SGIX/dmbuffer.txt The function that you're suggesting here is a clear break from that. I don't think that's a bad thing. I suspect that you designed it this way so that the implementation would not have to live in the GLX subsystem or in the 3D driver, correct? That was one of the goals. I generally view trying to bind a video-specific buffer to an OpenGL buffer as a bad idea since they always end up as second class. While there have been implementations that could use video buffers as textures, etc... they've always had serious limitations like the inability to have mipmaps, or repeat, or limited filtering ability or other disapointing things that people are sad to learn about. I opted instead for an explicit copy from a video-specific surface to a first-class OpenGL drawable. Being able to do HDTV video textures on a P4 1.2 Gig PC with a $100 video card has show this to be a reasonable tradeoff. The reason you would lose mipmaps and most of the texture wrap modes is because video streams rarely have power-of-two dimensions. In the past, hardware couldn't do mipmapping or GL_WRAP on non-power-of-two textures. For the most part, without NV_texture_rectangle, you can't even use npot textures. :( With slightly closer integration between XvMC and the 3D driver, we ought to be able to do something along those lines. Basically, bind a XvMCSurface to a pbuffer. Then, each time a new frame of video is rendered the pbuffer would be automatically updated. Given the way the XvMC works, I'm not sure how well that would work, though. I'll have to think on it some more. Mpeg frames are displayed in a different order than they are rendered. It's best if the decoder has full control over what goes where and when. Oh. That does change things a bit. Status XvMCCopySurfaceToGLXPbuffer ( Display *display, XvMCSurface *surface, XID pbuffer_id, short src_x, short src_y, unsigned short width, unsigned short height, short dst_x, short dst_y, int flags ); One quick comment. Don't use 'short', use 'int'. On every existing and future platform that we're likely to care about the shorts will take up as much space as an int on the stack anyway, and slower / larger / more instructions will need to be used to access them. This is an X-window extension. It's limited to the signed 16 bit coordinate system like the X-window system itself, all of Xlib and the rest of XvMC. So? Just because the values are limited to 16-bit doesn't necessitate that they be stored in a memory location that's only 16-bits. If X were being developed from scratch today, instead of calling everything short, it would likely be int_fast16_t. On IA-32, PowerPC, Alpha, SPARC, and x86-64, this is int. Maybe using the C99 types is the right answer anyway. This function copies the rectangle specified by src_x, src_y, width, and height from the XvMCSurface denoted by "surface" to offset dst_x, dst_y within the pbuffer identified by its GLXPbuffer XID "pbuffer_id". Note that while the src_x, src_y are in XvMC's standard left-handed coordinate system and specify the upper left hand
Re: RFC: OpenGL + XvMC
On Sat, 31 May 2003, Ian Romanick wrote: > Mark Vojkovich wrote: > > On Fri, 30 May 2003, Ian Romanick wrote: > > > >>Mark Vojkovich wrote: > >> > >>> I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function > >>>to XvMC. I have implemented this in NVIDIA's binary drivers and > >>>am able to do full framerate HDTV video textures on the higher end > >>>GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer > >>>contents into a texture. > >> > >>This shoulds like a good candidate for a GLX extension. I've been > >>wondering when someone would suggest somthing like this. :) Although, I > >>did expect it to come from someone doing video capture work first. > > > >I wanted to avoid something from the GLX side. Introducing the > > concept of an XFree86 video extension buffer to GLX seemed like a hard > > sell. Introducting a well establish GLX drawable type to XvMC > > seemed more reasonable. > > Right. I thought about this a bit more last night. A better approach > might be to expose this functionality as an XFree86 extension, then > create a GLX extension on top of it. I was thinking of an extension > where you would bind a "magic" buffer to a pbuffer, then take a snapshot > from the input buffer to the pbuffer. Doing that we could create > layered extensions for binding v4l streams to pbuffers. This would be > like creating a subclass in C++ and just over-riding the virtual > CaptureImage method. I think that would be much nicer for application code. This isn't capture. It's decode. XvMC is a video acceleration architecture not a capture architecture. Even with capture, HW capture buffer formats don't always line up nicely with pbuffer or texture formats. > > At the same time, all of the real work would still be done in the X > extension (or v4l). Only some light-weight bookkeeping would live in GLX. > > >>Over the years there have been a couple extensions for doing things > >>this, both from SGI. They both work by streaming video data into a new > >>type of GLX drawable and use that to source pixel / texel data. > >> > >> http://oss.sgi.com/projects/ogl-sample/registry/SGIX/video_source.txt > >> http://oss.sgi.com/projects/ogl-sample/registry/SGIX/dmbuffer.txt > >> > >>The function that you're suggesting here is a clear break from that. I > >>don't think that's a bad thing. I suspect that you designed it this way > >>so that the implementation would not have to live in the GLX subsystem > >>or in the 3D driver, correct? > > > >That was one of the goals. I generally view trying to bind > > a video-specific buffer to an OpenGL buffer as a bad idea since they > > always end up as second class. While there have been implementations > > that could use video buffers as textures, etc... they've always had > > serious limitations like the inability to have mipmaps, or repeat, or > > limited filtering ability or other disapointing things that people > > are sad to learn about. I opted instead for an explicit copy from > > a video-specific surface to a first-class OpenGL drawable. Being > > able to do HDTV video textures on a P4 1.2 Gig PC with a $100 video > > card has show this to be a reasonable tradeoff. > > The reason you would lose mipmaps and most of the texture wrap modes is > because video streams rarely have power-of-two dimensions. In the past, That's one of the reasons. > hardware couldn't do mipmapping or GL_WRAP on non-power-of-two textures. > For the most part, without NV_texture_rectangle, you can't even use > npot textures. :( And NV_texture_rectangle are still second class compared to normal textures. No video formats are powers of two, unfortunately. > > >>With slightly closer integration between XvMC and the 3D driver, we > >>ought to be able to do something along those lines. Basically, bind a > >>XvMCSurface to a pbuffer. Then, each time a new frame of video is > >>rendered the pbuffer would be automatically updated. Given the way the > >>XvMC works, I'm not sure how well that would work, though. I'll have to > >>think on it some more. > > > > > >Mpeg frames are displayed in a different order than they are > > rendered. It's best if the decoder has full control over what goes > > where and when. > > Oh. That does change things a bit. Yes. It's not capture. I'm decoding HDTV mpeg2 program streams from my harddisk. Normally XvMC would display these in the overlay. I've made a mechanism to copy to a pbuffer rather than (or in addition to) displaying in the video overlay. > > >>>Status > >>>XvMCCopySurfaceToGLXPbuffer ( > >>> Display *display, > >>> XvMCSurface *surface, > >>> XID pbuffer_id, > >>> short src_x, > >>> short src_y, > >>> unsigned short width, > >>> unsigned short height, > >>> short dst_x, > >>> short dst_y, > >>> int flags > >>>); > >> > >>One quick comment. Don't use 'short', use 'int'. On every existing and > >>future platform that we're
Re: RFC: OpenGL + XvMC
Just curious, as I'm not too familiar with XvMC yet, but would this provide another Xv adapter that used the 3D texture engine to display videos rather than othe video overlay? or something else. Sorry for my ignorance. Thanks, Alex --- Mark Vojkovich <[EMAIL PROTECTED]> wrote: >I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function > to XvMC. I have implemented this in NVIDIA's binary drivers and > am able to do full framerate HDTV video textures on the higher end > GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer > contents into a texture. > > > Status > XvMCCopySurfaceToGLXPbuffer ( > Display *display, > XvMCSurface *surface, > XID pbuffer_id, > short src_x, > short src_y, > unsigned short width, > unsigned short height, > short dst_x, > short dst_y, > int flags > ); > >This function copies the rectangle specified by src_x, src_y, > width, > and height from the XvMCSurface denoted by "surface" to offset > dst_x, dst_y > within the pbuffer identified by its GLXPbuffer XID "pbuffer_id". > Note that while the src_x, src_y are in XvMC's standard left-handed > coordinate system and specify the upper left hand corner of the > rectangle, dst_x and dst_y are in OpenGL's right-handed coordinate > > system and denote the lower left hand corner of the destination > rectangle in the pbuffer. > > "Flags" may be XVMC_TOP_FIELD, XVMC_BOTTOM_FIELD or > XVMC_FRAME_PICTURE. > If flags is not XVMC_FRAME_PICTURE, the src_y and height are in > field > coordinates, not frame. That is, the total copyable height is half > the height of the XvMCSurface. > > XvMCCopySurfaceToGLXPbuffer does not return until the copy to the > pbuffer has completed. XvMCCopySurfaceToGLXPbuffer is pipelined > with XvMCRenderSurface so no explicit synchronization between > XvMCRenderSurface and XvMCCopySurfaceToGLXPbuffer is needed. > > The pbuffer must be of type GLX_RGBA, and the destination of the > copy is the left front buffer of the pbuffer. Success is returned > if no error occured, the error code is returned otherwise. > > Possible Errors: > >XvMCBadSurface - The surface is invalid. > >BadDrawable - The pbuffer_id is not a valid pbuffer. > >BadMatch - The pbuffer is not of type GLX_RGBA or the > pbuffer does not have a front left buffer. > > XvMCCopySurfaceToGLXPbuffer is supported if the following flag: > > #define XVMC_COPY_TO_PBUFFER 0x0010 > > is set in the XvMCSurfaceInfo's flags field. > > I'd like to bump the API version up to 1.1 and add this. > Comments? > > > Mark. > __ Do you Yahoo!? Yahoo! Calendar - Free online calendar with sync to Outlook(TM). http://calendar.yahoo.com ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
On Sun, 1 Jun 2003, Alex Deucher wrote: > Just curious, as I'm not too familiar with XvMC yet, but would this > provide another Xv adapter that used the 3D texture engine to display > videos rather than othe video overlay? or something else. Sorry for > my ignorance. ... ... ... > --- Mark Vojkovich <[EMAIL PROTECTED]> wrote: > >I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function > > to XvMC. I have implemented this in NVIDIA's binary drivers and > > am able to do full framerate HDTV video textures on the higher end > > GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer > > contents into a texture. I don't really know any more than you but my guess is that while this may be presented as another Xv adapter, what it really does is make a video source available as a texture. This would allow you to, for instance, display a virtual reality living-room and whenever the camera falls on the TV it would be showing live pictures from a video feed. -- Andrew C. Aitchison Cambridge [EMAIL PROTECTED] ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
On Sun, 1 Jun 2003, Andrew C Aitchison wrote: >> Just curious, as I'm not too familiar with XvMC yet, but would this >> provide another Xv adapter that used the 3D texture engine to display >> videos rather than othe video overlay? or something else. Sorry for >> my ignorance. > ... ... ... >> --- Mark Vojkovich <[EMAIL PROTECTED]> wrote: >> >I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function >> > to XvMC. I have implemented this in NVIDIA's binary drivers and >> > am able to do full framerate HDTV video textures on the higher end >> > GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer >> > contents into a texture. > >I don't really know any more than you but my guess is that while >this may be presented as another Xv adapter, what it really does is >make a video source available as a texture. >This would allow you to, for instance, display a virtual reality >living-room and whenever the camera falls on the TV it would be >showing live pictures from a video feed. Another more useful use would be CNN like video effects where the fullscreen shrinks to an inset and then rotates on a 3D angle, etc. Various nifty 3D video effects for video editing/production could be done. -- Mike A. Harris ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
It's not another adaptor. I lets you copy video surfaces rendered with an existing XvMC adaptor into a GLXPbuffer. It's possible to both display in the overlay and save off to a Pbuffer. A while ago, I started getting inquiries from a couple film companies who store their rendered animated features as jpegs. They decompress them in software and copy them into textures so they can do post processing on them in OpenGL. I think mostly they're interested in doing things like adding film grain and doing color correction. The software decode is really a significant expense if you're trying to do 1920x1080 sized jpegs in real time. Unfortunately, I don't think the quality of 4:2:0 video is really is up their standards, but there are lower quality applications such as HDTV and DVD post processing that can benefit from accelerated decode of mpeg streams into an OpenGL drawable. There are probably some interesting multimedia projects as well. Mark. On Sun, 1 Jun 2003, Alex Deucher wrote: > Just curious, as I'm not too familiar with XvMC yet, but would this > provide another Xv adapter that used the 3D texture engine to display > videos rather than othe video overlay? or something else. Sorry for > my ignorance. > > Thanks, > > Alex > > --- Mark Vojkovich <[EMAIL PROTECTED]> wrote: > >I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function > > to XvMC. I have implemented this in NVIDIA's binary drivers and > > am able to do full framerate HDTV video textures on the higher end > > GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer > > contents into a texture. > > > > > > Status > > XvMCCopySurfaceToGLXPbuffer ( > > Display *display, > > XvMCSurface *surface, > > XID pbuffer_id, > > short src_x, > > short src_y, > > unsigned short width, > > unsigned short height, > > short dst_x, > > short dst_y, > > int flags > > ); > > > >This function copies the rectangle specified by src_x, src_y, > > width, > > and height from the XvMCSurface denoted by "surface" to offset > > dst_x, dst_y > > within the pbuffer identified by its GLXPbuffer XID "pbuffer_id". > > Note that while the src_x, src_y are in XvMC's standard left-handed > > coordinate system and specify the upper left hand corner of the > > rectangle, dst_x and dst_y are in OpenGL's right-handed coordinate > > > > system and denote the lower left hand corner of the destination > > rectangle in the pbuffer. > > > > "Flags" may be XVMC_TOP_FIELD, XVMC_BOTTOM_FIELD or > > XVMC_FRAME_PICTURE. > > If flags is not XVMC_FRAME_PICTURE, the src_y and height are in > > field > > coordinates, not frame. That is, the total copyable height is half > > the height of the XvMCSurface. > > > > XvMCCopySurfaceToGLXPbuffer does not return until the copy to the > > pbuffer has completed. XvMCCopySurfaceToGLXPbuffer is pipelined > > with XvMCRenderSurface so no explicit synchronization between > > XvMCRenderSurface and XvMCCopySurfaceToGLXPbuffer is needed. > > > > The pbuffer must be of type GLX_RGBA, and the destination of the > > copy is the left front buffer of the pbuffer. Success is returned > > if no error occured, the error code is returned otherwise. > > > > Possible Errors: > > > >XvMCBadSurface - The surface is invalid. > > > >BadDrawable - The pbuffer_id is not a valid pbuffer. > > > >BadMatch - The pbuffer is not of type GLX_RGBA or the > > pbuffer does not have a front left buffer. > > > > XvMCCopySurfaceToGLXPbuffer is supported if the following flag: > > > > #define XVMC_COPY_TO_PBUFFER 0x0010 > > > > is set in the XvMCSurfaceInfo's flags field. > > > > I'd like to bump the API version up to 1.1 and add this. > > Comments? > > > > > > Mark. > > > > > __ > Do you Yahoo!? > Yahoo! Calendar - Free online calendar with sync to Outlook(TM). > http://calendar.yahoo.com > ___ > Devel mailing list > [EMAIL PROTECTED] > http://XFree86.Org/mailman/listinfo/devel > ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
On Fri, May 30, 2003 at 05:42:20PM -0700, Ian Romanick wrote: > Mark Vojkovich wrote: > > I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function > >to XvMC. I have implemented this in NVIDIA's binary drivers and > >am able to do full framerate HDTV video textures on the higher end > >GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer > >contents into a texture. > > This shoulds like a good candidate for a GLX extension. I've been > wondering when someone would suggest somthing like this. :) Although, I > did expect it to come from someone doing video capture work first. I don't think it makes much difference whether it's considered as XvMC functionality or GLX functionality, but if you do it in XvMC I suggest paying as much attention to a detailed specification as OpenGL and GLX extensions do. In particular since the X and GL rendering streams are generally asynchronous, making sure apps can get repeatable behavior across drivers. You might want to think about how this could carry over to the upcoming super buffers extension, too, since that will probably replace pbuffers for most purposes within a few years. Since super buffers operate entirely within GL, and the GL namespace doesn't involve XIDs and is dependent on the current context, it might make more sense to pass the XvMC objects into GL than vice versa. Jon Leech SGI ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
On Sun, 1 Jun 2003, Jon Leech wrote: > On Fri, May 30, 2003 at 05:42:20PM -0700, Ian Romanick wrote: > > Mark Vojkovich wrote: > > > I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function > > >to XvMC. I have implemented this in NVIDIA's binary drivers and > > >am able to do full framerate HDTV video textures on the higher end > > >GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer > > >contents into a texture. > > > > This shoulds like a good candidate for a GLX extension. I've been > > wondering when someone would suggest somthing like this. :) Although, I > > did expect it to come from someone doing video capture work first. > > I don't think it makes much difference whether it's considered as > XvMC functionality or GLX functionality, but if you do it in XvMC I > suggest paying as much attention to a detailed specification as OpenGL > and GLX extensions do. In particular since the X and GL rendering > streams are generally asynchronous, making sure apps can get repeatable > behavior across drivers. > You might want to think about how this could carry over to the > upcoming super buffers extension, too, since that will probably replace > pbuffers for most purposes within a few years. Since super buffers There are alot of people who are just discovering pbuffers now. I would guess it would take years before superbuffers were widely used. > operate entirely within GL, and the GL namespace doesn't involve XIDs > and is dependent on the current context, it might make more sense to > pass the XvMC objects into GL than vice versa. > Jon Leech > SGI Extending GL to recognize a relatively unknown XFree86 format is a hard sell. I wouldn't even be able to convince my own company to dirty their code for it seeing as how relatively nobody is using XvMC. Of course you really want these things in objects that are entirely within the OpenGL namespace, such as textures. Speaking of slowly adopted extensions, if there's GLX_ARB_render_to_texture that sort of solves that problem. Haven't been paying much attention to super buffers (doesn't effect me yet). You can bind a pbuffer to a superbuffer in some way can't you? Mark. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
On Mon, Jun 02, 2003 at 01:09:59AM -0400, Mark Vojkovich wrote: >Extending GL to recognize a relatively unknown XFree86 format > is a hard sell. I wouldn't even be able to convince my own company > to dirty their code for it seeing as how relatively nobody is using > XvMC. Do you implement this without touching the GL driver code? Seems difficult to avoid touching the driver in the general case, when the format and location of pbuffer memory is intentionally opaque. >Of course you really want these things in objects that are > entirely within the OpenGL namespace, such as textures. Speaking > of slowly adopted extensions, if there's GLX_ARB_render_to_texture > that sort of solves that problem. Haven't been paying much attention > to super buffers (doesn't effect me yet). You can bind a pbuffer > to a superbuffer in some way can't you? No. There is a WGL_ARB_render_texture extension but we never finished the GLX version due to lack of interest + the expectation that super buffers will enable similar functionality. Jon ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
Mark Vojkovich wrote: On Sun, 1 Jun 2003, Jon Leech wrote: You might want to think about how this could carry over to the upcoming super buffers extension, too, since that will probably replace pbuffers for most purposes within a few years. Since super buffers There are alot of people who are just discovering pbuffers now. I would guess it would take years before superbuffers were widely used. I would re-think that assumption. :) A *lot* of people have known about pbuffers but have intentionally avoided them. When superbuffers are available, they are going to jump all over it! Not only that, on Linux on the Nvidia drivers and the ATI drivers for the FireGL 1/2/3 cards (not the Radeon based FireGL cards) support it at all currently. Since nobody supports superbuffers yet, I think we could probably re-visit this issue when it is available. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
Mark Vojkovich wrote: On Sat, 31 May 2003, Ian Romanick wrote: Mark Vojkovich wrote: On Fri, 30 May 2003, Ian Romanick wrote: Mark Vojkovich wrote: I'd like to propose adding a XvMCCopySurfaceToGLXPbuffer function to XvMC. I have implemented this in NVIDIA's binary drivers and am able to do full framerate HDTV video textures on the higher end GeForce4 MX cards by using glCopyTexSubImage2D to copy the Pbuffer contents into a texture. This shoulds like a good candidate for a GLX extension. I've been wondering when someone would suggest somthing like this. :) Although, I did expect it to come from someone doing video capture work first. I wanted to avoid something from the GLX side. Introducing the concept of an XFree86 video extension buffer to GLX seemed like a hard sell. Introducting a well establish GLX drawable type to XvMC seemed more reasonable. Right. I thought about this a bit more last night. A better approach might be to expose this functionality as an XFree86 extension, then create a GLX extension on top of it. I was thinking of an extension where you would bind a "magic" buffer to a pbuffer, then take a snapshot from the input buffer to the pbuffer. Doing that we could create layered extensions for binding v4l streams to pbuffers. This would be like creating a subclass in C++ and just over-riding the virtual CaptureImage method. I think that would be much nicer for application code. This isn't capture. It's decode. XvMC is a video acceleration architecture not a capture architecture. Even with capture, HW capture buffer formats don't always line up nicely with pbuffer or texture formats. I understand that it's not capture. However, it's conceptually similar. You have some opaque source that's generating frames of video. You take whatever is the current frame and slurp it over to your pbuffer. I doen't really matter if the video source is a file on your hard drive, a webcam on your PC, or a Dept. of Transportation traffic camera over the web. That's why I'm saying that this new XvMC functionality could be used at the basis of a more general GLX extension. I think a more abstract GLX extension is going to be far more useful to application developers. However, without something like XvMCCopySurfaceToGLXPbuffer in each thing that we want to use as a video source, the GLX extension can't be implemented. The fact that buffer formats don't match is why I propsed having a CaptureImage (or something) function in the GLX extension. That's the explicit copy of the next frame (either from the MPEG file or from the video camera) to the pbuffer. It's just a name that would redirect to XvMCCopySurfaceToGLXPbuffer in this case. The reason for the "bind" call is so that CaptureImage (which could probably use a better name, but it was late when I thought of it) knows what the video source is and what type of source (i.e., XvMC, v4l, etc.) it is. That way it knows how to do the copy. Then the application developer doesn't have to bother with that in their code. That way if they want to switch from a video file to live video, they just make a different bind call, their code path doesn't have to change. The really cool thing is that if all the real work (i.e., copying the data) is done in the XFree86 extensions, then all of the code for the GLX extension is *completely* device independent. That ends up being a *HUGE* win. hardware couldn't do mipmapping or GL_WRAP on non-power-of-two textures. For the most part, without NV_texture_rectangle, you can't even use npot textures. :( And NV_texture_rectangle are still second class compared to normal textures. No video formats are powers of two, unfortunately. Fair enough. But that should change soon. :) Status XvMCCopySurfaceToGLXPbuffer ( Display *display, XvMCSurface *surface, XID pbuffer_id, short src_x, short src_y, unsigned short width, unsigned short height, short dst_x, short dst_y, int flags ); One quick comment. Don't use 'short', use 'int'. On every existing and future platform that we're likely to care about the shorts will take up as much space as an int on the stack anyway, and slower / larger / more instructions will need to be used to access them. This is an X-window extension. It's limited to the signed 16 bit coordinate system like the X-window system itself, all of Xlib and the rest of XvMC. So? Just because the values are limited to 16-bit doesn't necessitate that they be stored in a memory location that's only 16-bits. If X were being developed from scratch today, instead of calling everything short, it would likely be int_fast16_t. On IA-32, PowerPC, Alpha, SPARC, and x86-64, this is int. Maybe using the C99 types is the right answer anyway. XvMC is already using shorts. No reason to be inconsistent now. It reflects the underlying protocol limitations anyhow. Something to keep in mind for future extensions though. True. It should be pretty easy t
Re: RFC: OpenGL + XvMC
On Sun, 1 Jun 2003, Jon Leech wrote: > On Mon, Jun 02, 2003 at 01:09:59AM -0400, Mark Vojkovich wrote: > >Extending GL to recognize a relatively unknown XFree86 format > > is a hard sell. I wouldn't even be able to convince my own company > > to dirty their code for it seeing as how relatively nobody is using > > XvMC. > > Do you implement this without touching the GL driver code? Seems > difficult to avoid touching the driver in the general case, when the > format and location of pbuffer memory is intentionally opaque. I haven't touched the GL driver at all. XvMC is direct rendered and the assumption is that it's using the same direct rendering architecture as OpenGL and should be able to get access to the pbuffer memory if it can name it, just like GL would be able to do. Mark. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
On Mon, 2 Jun 2003, Ian Romanick wrote: > Mark Vojkovich wrote: > > On Sun, 1 Jun 2003, Jon Leech wrote: > >>You might want to think about how this could carry over to the > >>upcoming super buffers extension, too, since that will probably replace > >>pbuffers for most purposes within a few years. Since super buffers > > > > There are alot of people who are just discovering pbuffers now. > > I would guess it would take years before superbuffers were widely used. > > I would re-think that assumption. :) A *lot* of people have known about > pbuffers but have intentionally avoided them. When superbuffers are > available, they are going to jump all over it! I'll believe it when I see it. Mark. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
Mark Vojkovich wrote: On Sun, 1 Jun 2003, Jon Leech wrote: On Mon, Jun 02, 2003 at 01:09:59AM -0400, Mark Vojkovich wrote: Extending GL to recognize a relatively unknown XFree86 format is a hard sell. I wouldn't even be able to convince my own company to dirty their code for it seeing as how relatively nobody is using XvMC. Do you implement this without touching the GL driver code? Seems difficult to avoid touching the driver in the general case, when the format and location of pbuffer memory is intentionally opaque. I haven't touched the GL driver at all. XvMC is direct rendered and the assumption is that it's using the same direct rendering architecture as OpenGL and should be able to get access to the pbuffer memory if it can name it, just like GL would be able to do. You may not have touched the GL driver at all, but you are using some sort of non-public interface to it to convert a pbuffer ID to an address. That was somewhat the point of Jon's comment. I certainly don't see anything in any pbuffer documentation that I've ever seen that describes how to get the address in video memory of a pbuffer. In fact, the documentation that I have seen goes to some length to explain that at certain points in time the pbuffer may not have an address in video memory. Instead of modifying your 3D driver, you've used an internal interface that, luckilly for you, just happened to already be there. The rest of us may not be so lucky. Given that, I have only three comments / requests for the function. 1. Please provide a way to specify the destination buffer (i.e., GL_FRONT, GL_BACK_RIGHT, etc.) of the copy. 2. Make explicit the coordinate conversion monkey business. 3. Is there a way for apps to determine if this function is available on their hardware? Later this year when pbuffers become available in the open-source drivers, we probably won't (initially) have support for this function. I fully expect that support will follow soon, but it won't be there initially. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
On Mon, 2 Jun 2003, Ian Romanick wrote: > >I haven't touched the GL driver at all. XvMC is direct rendered > > and the assumption is that it's using the same direct rendering > > architecture as OpenGL and should be able to get access to the > > pbuffer memory if it can name it, just like GL would be able to > > do. > > You may not have touched the GL driver at all, but you are using some > sort of non-public interface to it to convert a pbuffer ID to an > address. That was somewhat the point of Jon's comment. I certainly > don't see anything in any pbuffer documentation that I've ever seen that > describes how to get the address in video memory of a pbuffer. In fact, > the documentation that I have seen goes to some length to explain that > at certain points in time the pbuffer may not have an address in video > memory. > > Instead of modifying your 3D driver, you've used an internal interface > that, luckilly for you, just happened to already be there. The rest of > us may not be so lucky. I'd expect you to be able to do the same think with the DRI. Intel implements their XvMC driver using the DRI. > > Given that, I have only three comments / requests for the function. > > 1. Please provide a way to specify the destination buffer (i.e., > GL_FRONT, GL_BACK_RIGHT, etc.) of the copy. It's taking an argument like glDrawBuffer does. It uses the GL enums. > > 2. Make explicit the coordinate conversion monkey business. Will do. > > 3. Is there a way for apps to determine if this function is available on > their hardware? Later this year when pbuffers become available in the > open-source drivers, we probably won't (initially) have support for this > function. I fully expect that support will follow soon, but it won't be > there initially. There was a flag in the proposal that indicated it. That's the way other XvMC limitations/features are indicated, such as whether or not subpicture scaling is supported. What I'd really like to hear is if the DRI folks think they'd have implementation problems with this in the Intel driver or some hypothetical driver. Mark. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: RFC: OpenGL + XvMC
Mark Vojkovich wrote: On Mon, 2 Jun 2003, Ian Romanick wrote: Instead of modifying your 3D driver, you've used an internal interface that, luckilly for you, just happened to already be there. The rest of us may not be so lucky. I'd expect you to be able to do the same think with the DRI. Intel implements their XvMC driver using the DRI. Oh I know that we will be able to do it. We'll just have to make the necessary changes to the 3D driver... ;) Given that, I have only three comments / requests for the function. 1. Please provide a way to specify the destination buffer (i.e., GL_FRONT, GL_BACK_RIGHT, etc.) of the copy. It's taking an argument like glDrawBuffer does. It uses the GL enums. Perfect. 2. Make explicit the coordinate conversion monkey business. Will do. I think that will avoid having to answer a lot of questions later. :) 3. Is there a way for apps to determine if this function is available on their hardware? Later this year when pbuffers become available in the open-source drivers, we probably won't (initially) have support for this function. I fully expect that support will follow soon, but it won't be there initially. There was a flag in the proposal that indicated it. That's the way other XvMC limitations/features are indicated, such as whether or not subpicture scaling is supported. Okay, I missed that the first time. Sorry. What I'd really like to hear is if the DRI folks think they'd have implementation problems with this in the Intel driver or some hypothetical driver. Since I'll probably end up implementing it (at least in some drivers), I don't forsee any problems. Since both the 3D drivers and the XvMC drivers are client-side, it should be pretty easy. As soon as I get done writing all the parts so that we can have pbuffers (and ARB_vertex_buffer_objects, and superbuffers), I should be able to treat the XvMC copy call just like a glDrawPixels that sources from another pbuffer of a superbuffer. ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
