Re: [Xpert]Re: XVideo and refresh sync
On Sun, Oct 21, 2001 at 10:11:36AM -0700, Billy Biggs wrote: > How do the ATI cards currently export TV output capabilities when you're > writing interlaced frames? Hmm you can ask the card stuff like: - are you drawing or retracing? - are you drawing an odd or even frame? I am not sure if you can ask it whether it's NTSC or PAL. Anyway, currently when using TVOut on ATIs, the "Vertical Refresh" value xvidtune shows is wrong, it isn't taken from the tv encoder (I doubt my TV is 85Hz ). If you tell the yuv scaler to sync, it only syncs on one type of retrace (I don't know if it's even or odd frame, but it is definitely only one of them). Bye, Peter Surda (Shurdeek) <[EMAIL PROTECTED]>, ICQ 10236103, +436505122023 -- The best things in life are free, but the expensive ones are still worth a look. PGP signature
Re: [Xpert]Re: XVideo and refresh sync
On Sun, 21 Oct 2001 [EMAIL PROTECTED] wrote: > > I'm talking about planning how many refreshes to show each frame. I'd > > be really surprised if you could say to the ATI card 'show this 2 > > refreshes from now'. > > The card won't do that, but the driver can. In this case what you do is > submit a frame and a number - show this frame n-vsync interrupts from the > previous one - and the driver can do the rest. > If you had kernel support you could. You could easily add a port attribute that was a hint to tell the server to display the next frame/field N vsyncs from now. The data copy could happen immediately but the display would go into effect at the desired time. You'd probably need to offload some overlay programming into the kernel and have the kernel change the overlay offset at the right time for you, but the X-server could do the rest, like setup the destination and scaling registers. The kernel merely needs to update to the new source buffer at the correct time. I don't think that's too difficult of a problem to solve. Mark. ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
Vladimir Dergachev ([EMAIL PROTECTED]): > > I'm talking about planning how many refreshes to show each frame. > > I'd be really surprised if you could say to the ATI card 'show this > > 2 refreshes from now'. > > The card won't do that, but the driver can. In this case what you do > is submit a frame and a number - show this frame n-vsync interrupts > from the previous one - and the driver can do the rest. Yes, for all cards which generate an interrupt every vsync then we can make an awesome API for high quality video, so long as the driver code which copies the frame is also in the kernel, since we don't want the scheduling wait of having the X driver get loaded. Having a /dev/vsync device would give the smallest amount of code in the kernel as possible, but then you need SCHED_FIFO access to protect against scheduling delays. (read: hack) You still need calls to tell you when the timebase for the refresh is though to make sure we plan ourselves out corrrectly. Regardless that kind of information is needed for TV output if you want to get field-correctness. How do the ATI cards currently export TV output capabilities when you're writing interlaced frames? The TV output API of v4l2 looks like it might be a good start. Since the requirements for TV output sync and progressive video playback sync are so similar, I'm wondering if the frame queueing API shouldn't be the same. -- Billy Biggs [EMAIL PROTECTED] ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
On Sun, 21 Oct 2001, Billy Biggs wrote: > Vladimir Dergachev ([EMAIL PROTECTED]): > > > > and hope that someone in the kernel has an accurate clock. It would > > > also need to tell me the (exact) refresh rate + sync point. > > > > _You_ don't need to know that. All you need is to tell the card: > > display this frame after sync. It will do the rest ! > > I think that you think I'm trying to solve the tearing problem. No, > I'm not. I understand that you won't tear. > > I'm talking about planning how many refreshes to show each frame. I'd > be really surprised if you could say to the ATI card 'show this 2 > refreshes from now'. The card won't do that, but the driver can. In this case what you do is submit a frame and a number - show this frame n-vsync interrupts from the previous one - and the driver can do the rest. > > > > I'm not worried about tearing, I'm worried about amortizing frame > > > time over the refresh rate in a pleasing way (especially for special > > > refresh rates like 72hz: think projector). > > > > I am not sure I understand the problem here. What is it again ? > > I'm glad you asked. :) > > Say I'm playing 24fps content and I have a projector which I'm going > to run at exactly 72hz refresh. So, for every frame of video, I want to > show it for exactly 3 refreshes. > > To do this correctly, I want to make sure I do my blit safely inside > a refresh interval to account for some error (it takes time to have the > frame copied to video memory etc). Say I don't at all know when the > refresh is occuring (since I have no API for that), and I'm doing my > blits at _just_ before the refresh all the time. Sometimes I blit and > the card is in the middle of the refresh, sometimes I blit and it hasn't > happened yet. > > So, say I want to blit before the refresh. Here's a possible > scenario: > > Output Refreshes Blits > > 0 Blit 1 [ we make it in before the next refresh ] > 1 Refresh 1 > 1 Refresh 2 > 1 Refresh 3 > 1 Refresh 4 Blit 2 [ this time, we collide with the refresh, > 2 Refresh 5so the card waits for the next one to show > 2 Refresh 6our frame ] > Blit 3 [ we make it again, before the intended > 3 Refresh 7refresh ] > 3 Refresh 8 > 3 Refresh 9 > > So you can see in this case, frame 1 was shown for 4 refreshes, and > frame 2 for only 2, instead of the intended refresh-per-frame average we > wanted of 3. Over time we sometimes make it and sometimes don't, and > our video output quality suffers. > > Tonight I did up some discussion on the effects of 10ms scheduling > resolution on frame blits. However, the results do assume that we can > determine when the video refresh occurs so that we can avoid cases like > the above. > > http://www.dumbterm.net/graphics/refresh/ > > I'll try and do up a similar experiment to show the effects of not > being refresh-aware. Ok. Vladimir Dergachev > > -- > Billy Biggs > [EMAIL PROTECTED] > ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
Vladimir Dergachev ([EMAIL PROTECTED]): > > and hope that someone in the kernel has an accurate clock. It would > > also need to tell me the (exact) refresh rate + sync point. > > _You_ don't need to know that. All you need is to tell the card: > display this frame after sync. It will do the rest ! I think that you think I'm trying to solve the tearing problem. No, I'm not. I understand that you won't tear. I'm talking about planning how many refreshes to show each frame. I'd be really surprised if you could say to the ATI card 'show this 2 refreshes from now'. > > I'm not worried about tearing, I'm worried about amortizing frame > > time over the refresh rate in a pleasing way (especially for special > > refresh rates like 72hz: think projector). > > I am not sure I understand the problem here. What is it again ? I'm glad you asked. :) Say I'm playing 24fps content and I have a projector which I'm going to run at exactly 72hz refresh. So, for every frame of video, I want to show it for exactly 3 refreshes. To do this correctly, I want to make sure I do my blit safely inside a refresh interval to account for some error (it takes time to have the frame copied to video memory etc). Say I don't at all know when the refresh is occuring (since I have no API for that), and I'm doing my blits at _just_ before the refresh all the time. Sometimes I blit and the card is in the middle of the refresh, sometimes I blit and it hasn't happened yet. So, say I want to blit before the refresh. Here's a possible scenario: Output Refreshes Blits 0 Blit 1 [ we make it in before the next refresh ] 1 Refresh 1 1 Refresh 2 1 Refresh 3 1 Refresh 4 Blit 2 [ this time, we collide with the refresh, 2 Refresh 5so the card waits for the next one to show 2 Refresh 6our frame ] Blit 3 [ we make it again, before the intended 3 Refresh 7refresh ] 3 Refresh 8 3 Refresh 9 So you can see in this case, frame 1 was shown for 4 refreshes, and frame 2 for only 2, instead of the intended refresh-per-frame average we wanted of 3. Over time we sometimes make it and sometimes don't, and our video output quality suffers. Tonight I did up some discussion on the effects of 10ms scheduling resolution on frame blits. However, the results do assume that we can determine when the video refresh occurs so that we can avoid cases like the above. http://www.dumbterm.net/graphics/refresh/ I'll try and do up a similar experiment to show the effects of not being refresh-aware. -- Billy Biggs [EMAIL PROTECTED] ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
On Sun, 21 Oct 2001, Peter Surda wrote: > On Fri, Oct 19, 2001 at 06:45:07AM -0700, Billy Biggs wrote: > > > > The +/-5ms error here is visible, especially on big cinematic pans. > > > I REALLY doubt what you perceive as an error is a 5ms difference. > > No? I'll post up a good example later today. Consider a pan where > > the time between frames goes like this: 40ms, 45ms, 35ms, 45ms, 42ms... > > So, the second frame is shown for much longer than the first or third. > it's a 1frame difference if you have 100Hz refresh. I REALLY doubt a human eye > can see the difference. I bet it is something else causing you to see > disturbances. To see the 80ms disturbance I was talking about (missed frame on > TVout) you already have to watch out a little. 10ms (45 - 35) is 8 times as > difficult. > > Please don't understand me wrongly: I agree that the current situation is > insufficient. I simply think that the cause of your problems is not exactly > what you think :-). The human eye will see the difference in a specific case - stroboscope. Imagine that you have a film where every odd frame is green and every even frame is red. Than if it happened that the duration of odd frames is longer than duration of the even frames you'll notice. Now, we don't have movies like that, but you might see a fence, or bicycle wheel or something similar in the movie. Normally, a DVD will be designed for a specific refresh rate. I would think that the DVD player application should recommend a specific refresh rate for the screen as well - an integer multiple of whatever DVD was targeted for. Vladimir Dergachev > > > Speaking of TV output, if you're outputting interlaced material to a > > TV then you MUST have accurate vsync, field dominance, and some way of > > ensuring you never miss a field. Otherwise you get crazy effects of > > people jumping back in time when you miss a field blit. > My ATI does this on HW (AFAIK), i.e. it understands it can't blit both on even and >odd > frames. > > > Ideally for playing 24fps on a 25fps output, > You don't have 24fps source usually. DVDs, dvd rips and tv caps are either 25 > or 29.9 (PAL vs NTSC). But I agree that a 24fps on TV will look sucky unless > you modify playback speed. > > Bye, > > Peter Surda (Shurdeek) <[EMAIL PROTECTED]>, ICQ 10236103, +436505122023 > > -- >Intel: where Quality is job number 0.9998782345! > ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
On Fri, 19 Oct 2001, Billy Biggs wrote: > [EMAIL PROTECTED] ([EMAIL PROTECTED]): > > > > If we had the vsync as an interrupt (/dev/vsync or something), > > > then quality would increase with less system load! Very nice! > > > But SCHED_FIFO would still be needed to guarentee accurate blits. > > > > The other approach is to have a device which you mmap and write your > > frames into. Then you issue an ioctl and the driver DMAs the buffers > > at the next available opportunity. > > Yes, you could make a cool API to solve the issue. It would be really > neat. My only concern would be the difficulty of implementing it with > consumer video cards. Ideally I'd want 'here's a frame, blit at time X' What do you mean "consumer video cards" ? I was talking about ATI cards - l let's see: mach64, Rage Pro, Rage128 and Radeon. A good portion of notebooks (I would guess at least 50% - just by looking at which manufacturers use which chips) made uses these and a fair number of desktop systems. > and hope that someone in the kernel has an accurate clock. It would > also need to tell me the (exact) refresh rate + sync point. _You_ don't need to know that. All you need is to tell the card: display this frame after sync. It will do the rest ! > > > > Check out deinterlace.sf.net (aka dscaler.sf.net). [...] > > > > How big are the cpu requirements ? This is something that is quite > > useful, because, at the moment, v4l does not allow to convey such > > information as field parity.. > > Not sure yet. For v4l, I assume all frames are top-field-first. v4l2 > provides field parity information, and I'm getting flamed on the > video4linux list to switch to it. > > > One thing - did you try your dvd player with recent ATI drivers ? ATI > > can doublebuffer frames switching them automatically on vsync. If you > > still see the issue you are worried about something is wrong with our > > drivers - I'd appreciate a lot if you could write a test up to debug > > the issue. (or a test mpeg stream) > > I'm not worried about tearing, I'm worried about amortizing frame > time over the refresh rate in a pleasing way (especially for special > refresh rates like 72hz: think projector). I am not sure I understand the problem here. What is it again ? Vladimir Dergachev > > -- > Billy Biggs > [EMAIL PROTECTED] > ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
On Sun, Oct 21, 2001 at 05:19:12AM +0200, Peter Surda wrote: > > Ideally for playing 24fps on a 25fps output, > You don't have 24fps source usually. DVDs, dvd rips and tv caps are either 25 > or 29.9 (PAL vs NTSC). But I agree that a 24fps on TV will look sucky unless > you modify playback speed. Nitpick: 24 fps is _quite_ common in progressive DVDs (and in quality rips). Removing the artifical 3:2 pulldown greatly enhances quality. R C -- They said it was *daft* to build a space station in a swamp, but I showed them! It sank unto the swamp. So I built a second space station. That sank into the swamp too. My third space station sank into the swamp. So I built a fourth one. That fell into a time warp and _then_ sank into the swamp. But the fifth one... stayed up! --Monty Python/Babylon 5 ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
On Fri, Oct 19, 2001 at 06:45:07AM -0700, Billy Biggs wrote: > > > The +/-5ms error here is visible, especially on big cinematic pans. > > I REALLY doubt what you perceive as an error is a 5ms difference. > No? I'll post up a good example later today. Consider a pan where > the time between frames goes like this: 40ms, 45ms, 35ms, 45ms, 42ms... > So, the second frame is shown for much longer than the first or third. it's a 1frame difference if you have 100Hz refresh. I REALLY doubt a human eye can see the difference. I bet it is something else causing you to see disturbances. To see the 80ms disturbance I was talking about (missed frame on TVout) you already have to watch out a little. 10ms (45 - 35) is 8 times as difficult. Please don't understand me wrongly: I agree that the current situation is insufficient. I simply think that the cause of your problems is not exactly what you think :-). > Speaking of TV output, if you're outputting interlaced material to a > TV then you MUST have accurate vsync, field dominance, and some way of > ensuring you never miss a field. Otherwise you get crazy effects of > people jumping back in time when you miss a field blit. My ATI does this on HW (AFAIK), i.e. it understands it can't blit both on even and odd frames. > Ideally for playing 24fps on a 25fps output, You don't have 24fps source usually. DVDs, dvd rips and tv caps are either 25 or 29.9 (PAL vs NTSC). But I agree that a 24fps on TV will look sucky unless you modify playback speed. Bye, Peter Surda (Shurdeek) <[EMAIL PROTECTED]>, ICQ 10236103, +436505122023 -- Intel: where Quality is job number 0.9998782345! PGP signature
Re: [Xpert]Re: XVideo and refresh sync
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > > If we had the vsync as an interrupt (/dev/vsync or something), > > then quality would increase with less system load! Very nice! > > But SCHED_FIFO would still be needed to guarentee accurate blits. > > The other approach is to have a device which you mmap and write your > frames into. Then you issue an ioctl and the driver DMAs the buffers > at the next available opportunity. Yes, you could make a cool API to solve the issue. It would be really neat. My only concern would be the difficulty of implementing it with consumer video cards. Ideally I'd want 'here's a frame, blit at time X' and hope that someone in the kernel has an accurate clock. It would also need to tell me the (exact) refresh rate + sync point. > > Check out deinterlace.sf.net (aka dscaler.sf.net). [...] > > How big are the cpu requirements ? This is something that is quite > useful, because, at the moment, v4l does not allow to convey such > information as field parity.. Not sure yet. For v4l, I assume all frames are top-field-first. v4l2 provides field parity information, and I'm getting flamed on the video4linux list to switch to it. > One thing - did you try your dvd player with recent ATI drivers ? ATI > can doublebuffer frames switching them automatically on vsync. If you > still see the issue you are worried about something is wrong with our > drivers - I'd appreciate a lot if you could write a test up to debug > the issue. (or a test mpeg stream) I'm not worried about tearing, I'm worried about amortizing frame time over the refresh rate in a pleasing way (especially for special refresh rates like 72hz: think projector). -- Billy Biggs [EMAIL PROTECTED] ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
Peter Surda ([EMAIL PROTECTED]): > On Fri, Oct 19, 2001 at 12:21:12AM -0400, Billy Biggs wrote: > > The +/-5ms error here is visible, especially on big cinematic pans. > I REALLY doubt what you perceive as an error is a 5ms difference. No? I'll post up a good example later today. Consider a pan where the time between frames goes like this: 40ms, 45ms, 35ms, 45ms, 42ms... So, the second frame is shown for much longer than the first or third. On a smooth pan you'll see the background shudder as it goes by. Think of a skateboard video. It's really horrid. > Though I agree that current vsync adjusting in XF86drivers (those that > support it e.g. ATI) is insufficient. I see noticeable disturbaces on > TVout (thought not on monitor), I believe it is because from time to > time the thread that calls XvShmPutImage has "missed" one frame and > suddenly the difference between 2 frames becomes double. Speaking of TV output, if you're outputting interlaced material to a TV then you MUST have accurate vsync, field dominance, and some way of ensuring you never miss a field. Otherwise you get crazy effects of people jumping back in time when you miss a field blit. Currently it seems like all TV output APIs assume you're sending progressive video at some very low framerate. Argh. :( One reason you may be seeing disturbances is that your TV encoder is only blitting frames and will wait until the next frame refresh before showing the next. This gives your output display a refresh of 25hz, and if you're not synced to it, you'll see much worse effects. Consider trying to play a 24fps video: you'll cycle around the real output sync pretty badly. Ideally for playing 24fps on a 25fps output, apps should speed it up by 4% (same as film->PAL video). To do so though we need to know information about the output capabilities though. -- Billy Biggs [EMAIL PROTECTED] ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
On Fri, 19 Oct 2001, Billy Biggs wrote: > [EMAIL PROTECTED] ([EMAIL PROTECTED]): > > > What I don't understand is why there is such an issue about root > > permissions. v4l devices, for example, have been user-accessible for a > > while. [...] > > Please consider the root issue a separate matter: I don't want it to > hinder my suggestions for improving video output quality. > > That said, let me explain my reasoning: > > Consider a video player attempting to play a TV feed at 59.94fps. A > frame must be blit every 16ms. With current scheduling under linux, > nanosleep has an accuracy of 1/HZ, which for most systems is 10ms. > Film-source material at 24fps (41.6ms/blit) suffers similarily. The > +/-5ms error here is visible, especially on big cinematic pans. > Clearly, smooth video output is only possible with better accuracy. > > My current plan of attack is simple: > > * Use /dev/rtc to cause an interupt at 1024hz (need root for >64hz). > * Set your render thread SCHED_FIFO so that when the timer interrupt > occurs, you will preempt everything (requires root). > > I get +/-1ms accuracy for my frame blits. This brute force method > works fine, but I still suffer slightly from not being synced to the > refresh rate. If HZ was set to something reasonable (like, say, 1024), > then being SCHED_FIFO would only be required when you want to guarentee > that you're never very far off, which you probably only need when you're > going to sit down with your family for a sunday night movie. > > If we had the vsync as an interrupt (/dev/vsync or something), then > quality would increase with less system load! Very nice! But > SCHED_FIFO would still be needed to guarentee accurate blits. The other approach is to have a device which you mmap and write your frames into. Then you issue an ioctl and the driver DMAs the buffers at the next available opportunity. > > Not all video and multimedia applications require root access. > Single-frame video grabbers, teletext decoders, and some specific > hardware-assisted players are examples. But you do need something > special for realtime audio effects processors, MIDI synthesizers, and > high quality (even TV quality!) video players. Not necessarily, though you may want something like this for real-time effects,encoders and decoders. This still does not imply you can't provide a clean interface to userspace. > > Check out deinterlace.sf.net (aka dscaler.sf.net). It's an example of > a (Windows-based) video deinterlacer, something I've kinda got working > under linux. For realtime deinterlacing, you have about 16ms _total_ to > grab a field from the capture card, process it, and send it to the > display. This demands alot from the CPU, and will definitely require > root access to ensure accurate timing. We can't buffer more than maybe > 2 fields, since unlike using a software DVD player we have no control of > the audio (which goes direct from your DVD player into your AC3 decoder > etc). Ouch!! How big are the cpu requirements ? This is something that is quite useful, because, at the moment, v4l does not allow to convey such information as field parity.. > > > Branden Robinson mentioned the following: > > > FWIW, I've heard that the pre-emptible kernel patches to Linux improve > > latency a great deal. > > If you're talking about the low-latency kernel patches, these only > guarentee performance benefits to SCHED_FIFO processes running as root, > AFAIK. > > > What I'm telling you is that DVD playback software that has to be > > setuid root, or run only by the root user, is a very unpalatable > > prospect to most general-purpose distributions. I think that is the > > audience that most interests the current developers of XvMC, but I > > urge them to correct me if I'm wrong. > > Hey, I'm just saying that for 'high quality' output, that is, at > television framerates or theatre-smooth video, you need root. You can > play things just fine as a user but your quality will be nowhere near > what you can achieve. > > > Back to [EMAIL PROTECTED]: > > > Bill - if you are willing to settle for your application to > > work with one video card (i.e. ATI) you could get such functionality > > much sooner. Just e-mail me what you want (in terms of sync), and I'll > > try to include this in our "extended" v4l module (http://gatos.sf.net) > > Vladimir, the idea of a /dev/vsync device is looking very promising. > Maybe we should look at standardizing something in this line, but > ideally I want my software to work with all video cards eventually (so > long as they don't suck and can provide me with the vsync status or an > interrupt). > > Thoughts? Not /dev/vsync. just a device to write frame data into and issue ioctl for dma'ed transfer to video memory. One thing - did you try your dvd player with recent ATI drivers ? ATI can doublebuffer frames switching them automatically on vsync. If you still see the issue yo
Re: [Xpert]Re: XVideo and refresh sync
Mark Vojkovich writes: > >That's the accuracy of X's internal clock. I suspect we could > call get time of day and give it to you in microseconds since most > platforms support that. Those timestamps would have no relation > to X's normal times stamps like the ones in Events though. > I know form experience that gettiemofday() is not very accurate on certain systems. I found systems where it is not even monotonic. I had to patch GetTimeInMillis() to work around this. I frequently had problems with grabs not getting released any more because of bogus timestamps. Egbert. ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
On Fri, Oct 19, 2001 at 12:21:12AM -0400, Billy Biggs wrote: > The +/-5ms error here is visible, especially on big cinematic pans. I REALLY doubt what you perceive as an error is a 5ms difference. Though I agree that current vsync adjusting in XF86drivers (those that support it e.g. ATI) is insufficient. I see noticeable disturbaces on TVout (thought not on monitor), I believe it is because from time to time the thread that calls XvShmPutImage has "missed" one frame and suddenly the difference between 2 frames becomes double. I see that 80ms (2 frames on 25fps TV) difference is noticeable, though I doubt 5ms is. Bye, Peter Surda (Shurdeek) <[EMAIL PROTECTED]>, ICQ 10236103, +436505122023 -- 0 and 1. Now what could be so hard about that? PGP signature
Re: [Xpert]Re: XVideo and refresh sync
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > What I don't understand is why there is such an issue about root > permissions. v4l devices, for example, have been user-accessible for a > while. [...] Please consider the root issue a separate matter: I don't want it to hinder my suggestions for improving video output quality. That said, let me explain my reasoning: Consider a video player attempting to play a TV feed at 59.94fps. A frame must be blit every 16ms. With current scheduling under linux, nanosleep has an accuracy of 1/HZ, which for most systems is 10ms. Film-source material at 24fps (41.6ms/blit) suffers similarily. The +/-5ms error here is visible, especially on big cinematic pans. Clearly, smooth video output is only possible with better accuracy. My current plan of attack is simple: * Use /dev/rtc to cause an interupt at 1024hz (need root for >64hz). * Set your render thread SCHED_FIFO so that when the timer interrupt occurs, you will preempt everything (requires root). I get +/-1ms accuracy for my frame blits. This brute force method works fine, but I still suffer slightly from not being synced to the refresh rate. If HZ was set to something reasonable (like, say, 1024), then being SCHED_FIFO would only be required when you want to guarentee that you're never very far off, which you probably only need when you're going to sit down with your family for a sunday night movie. If we had the vsync as an interrupt (/dev/vsync or something), then quality would increase with less system load! Very nice! But SCHED_FIFO would still be needed to guarentee accurate blits. Not all video and multimedia applications require root access. Single-frame video grabbers, teletext decoders, and some specific hardware-assisted players are examples. But you do need something special for realtime audio effects processors, MIDI synthesizers, and high quality (even TV quality!) video players. Check out deinterlace.sf.net (aka dscaler.sf.net). It's an example of a (Windows-based) video deinterlacer, something I've kinda got working under linux. For realtime deinterlacing, you have about 16ms _total_ to grab a field from the capture card, process it, and send it to the display. This demands alot from the CPU, and will definitely require root access to ensure accurate timing. We can't buffer more than maybe 2 fields, since unlike using a software DVD player we have no control of the audio (which goes direct from your DVD player into your AC3 decoder etc). Ouch!! Branden Robinson mentioned the following: > FWIW, I've heard that the pre-emptible kernel patches to Linux improve > latency a great deal. If you're talking about the low-latency kernel patches, these only guarentee performance benefits to SCHED_FIFO processes running as root, AFAIK. > What I'm telling you is that DVD playback software that has to be > setuid root, or run only by the root user, is a very unpalatable > prospect to most general-purpose distributions. I think that is the > audience that most interests the current developers of XvMC, but I > urge them to correct me if I'm wrong. Hey, I'm just saying that for 'high quality' output, that is, at television framerates or theatre-smooth video, you need root. You can play things just fine as a user but your quality will be nowhere near what you can achieve. Back to [EMAIL PROTECTED]: > Bill - if you are willing to settle for your application to > work with one video card (i.e. ATI) you could get such functionality > much sooner. Just e-mail me what you want (in terms of sync), and I'll > try to include this in our "extended" v4l module (http://gatos.sf.net) Vladimir, the idea of a /dev/vsync device is looking very promising. Maybe we should look at standardizing something in this line, but ideally I want my software to work with all video cards eventually (so long as they don't suck and can provide me with the vsync status or an interrupt). Thoughts? -- Billy Biggs [EMAIL PROTECTED] http://www.billybiggs.com/ [EMAIL PROTECTED] ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
What I don't understand is why there is such an issue about root permissions. v4l devices, for example, have been user-accessible for a while. Bill - if you are willing to settle for your application to work with one video card (i.e. ATI) you could get such functionality much sooner. Just e-mail me what you want (in terms of sync), and I'll try to include this in our "extended" v4l module (http://gatos.sf.net ) Vladimir Dergachev On Thu, 18 Oct 2001, Billy Biggs wrote: > Branden Robinson ([EMAIL PROTECTED]): > > > > I disagree. If I'm showing a movie, have rented a large projector > > > and a hall, I'd really really like 24fps at exactly 72hz. Giving > > > high-priority multimedia applications root access is a no-brainer. > > > > That may be true for *your* preferred application of such an > > interface, but it's not acceptable for, e.g., most Linux distributors > > or groups like OpenBSD[1], who don't want to expose root security > > holes via multimedia software. > > There's a difference between have an application which must be run as > root, and one which gives a significant quality benefit when run as > root. My DVD player (movietime.sf.net) will work just fine if you > aren't root, but it will look just as bad as xine, mplayer, or any of > the others. For quality playback, root access is required. > > It's not like I'm demanding the app be installed setuid root. But for > it to work, you need guarentees of latency etc. I don't see how we can > give that to user applications without creating security holes. > > Still, I don't like the attitude that we should disallow users to use > the capabilities of their hardware completely because the security of a > standalone DVD playback machine might be compromised. > > > [1] I'm not qualified to speak on behalf of OpenBSD, but given their > > approach to security I'd be pretty surprised if I were wrong on this > > point. > > -- > Billy Biggs > [EMAIL PROTECTED] > ___ > Xpert mailing list > [EMAIL PROTECTED] > http://XFree86.Org/mailman/listinfo/xpert > ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
Please respect my mail headers: Mail-Followup-To: Xpert list <[EMAIL PROTECTED]> Mail-Copies-To: nobody X-No-CC: I subscribe to this list; do not CC me on replies. On Thu, Oct 18, 2001 at 03:14:20PM -0700, Billy Biggs wrote: > It's not like I'm demanding the app be installed setuid root. But for > it to work, you need guarentees of latency etc. I don't see how we can > give that to user applications without creating security holes. FWIW, I've heard that the pre-emptible kernel patches to Linux improve latency a great deal. > Still, I don't like the attitude that we should disallow users to use > the capabilities of their hardware completely because the security of a > standalone DVD playback machine might be compromised. No one's stopping you from coding an implementation that requires root access, and no one's telling you that can't or shouldn't do so particularly in an embedded environment, where the superuser/user distinction isn't important. What I'm telling you is that DVD playback software that has to be setuid root, or run only by the root user, is a very unpalatable prospect to most general-purpose distributions. I think that is the audience that most interests the current developers of XvMC, but I urge them to correct me if I'm wrong. -- G. Branden Robinson| "Why do we have to hide from the Debian GNU/Linux | police, Daddy?" [EMAIL PROTECTED] | "Because we use vi, son. They use http://people.debian.org/~branden/ | emacs." PGP signature
Re: [Xpert]Re: XVideo and refresh sync
On Thu, 18 Oct 2001, Billy Biggs wrote: > Mark Vojkovich ([EMAIL PROTECTED]): > > > > I'm sure most video cards can trigger an interrupt on the vertical > > > refresh, that would be pretty accurate. Even if I could just query > > > > Handling the interrupt requires kernel support. The X-server is a > > user-space app. > > Right, but aren't DRI modules kernel-space? Why not, as part of the > DRI effort, we do up an API for a /dev/vsync[0,1,...] device. Do you > think most video cards support sending interrupts on vsync? This is an OS issue then. If somebody wants to standardize having /dev/vsync[0,1,...] that's fine with me as long as it isn't tied to a particular implementation so we can provide that with NVIDIA's drivers. Mark. ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
Branden Robinson ([EMAIL PROTECTED]): > > I disagree. If I'm showing a movie, have rented a large projector > > and a hall, I'd really really like 24fps at exactly 72hz. Giving > > high-priority multimedia applications root access is a no-brainer. > > That may be true for *your* preferred application of such an > interface, but it's not acceptable for, e.g., most Linux distributors > or groups like OpenBSD[1], who don't want to expose root security > holes via multimedia software. There's a difference between have an application which must be run as root, and one which gives a significant quality benefit when run as root. My DVD player (movietime.sf.net) will work just fine if you aren't root, but it will look just as bad as xine, mplayer, or any of the others. For quality playback, root access is required. It's not like I'm demanding the app be installed setuid root. But for it to work, you need guarentees of latency etc. I don't see how we can give that to user applications without creating security holes. Still, I don't like the attitude that we should disallow users to use the capabilities of their hardware completely because the security of a standalone DVD playback machine might be compromised. > [1] I'm not qualified to speak on behalf of OpenBSD, but given their > approach to security I'd be pretty surprised if I were wrong on this > point. -- Billy Biggs [EMAIL PROTECTED] ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
On Thu, Oct 18, 2001 at 02:23:36PM -0700, Billy Biggs wrote: > Mark Vojkovich ([EMAIL PROTECTED]): > > We have tossed around an interface for this. But there needs to be > > kernel support to get around the permissions problems. If root > > permissions are required to be able to write to that XvImage, it's > > not much use. > > I disagree. If I'm showing a movie, have rented a large projector and > a hall, I'd really really like 24fps at exactly 72hz. Giving > high-priority multimedia applications root access is a no-brainer. That may be true for *your* preferred application of such an interface, but it's not acceptable for, e.g., most Linux distributors or groups like OpenBSD[1], who don't want to expose root security holes via multimedia software. [1] I'm not qualified to speak on behalf of OpenBSD, but given their approach to security I'd be pretty surprised if I were wrong on this point. -- G. Branden Robinson|America is at that awkward stage. Debian GNU/Linux |It's too late to work within the [EMAIL PROTECTED] |system, but too early to shoot the http://people.debian.org/~branden/ |bastards. -- Claire Wolfe PGP signature
Re: [Xpert]Re: XVideo and refresh sync
Mark Vojkovich ([EMAIL PROTECTED]): > > I'm sure most video cards can trigger an interrupt on the vertical > > refresh, that would be pretty accurate. Even if I could just query > > Handling the interrupt requires kernel support. The X-server is a > user-space app. Right, but aren't DRI modules kernel-space? Why not, as part of the DRI effort, we do up an API for a /dev/vsync[0,1,...] device. Do you think most video cards support sending interrupts on vsync? -- Billy Biggs [EMAIL PROTECTED] ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
Mark Vojkovich ([EMAIL PROTECTED]): > > > I can add a XvMC function to get you an XImage from the XvMC > > > surface so that you can use XvMC instead of Xv. Then you can get > > > rid of the delay between the XvShmPut() and the actual flip. > > > > But could this be a solution we could implement in general? > > We have tossed around an interface for this. But there needs to be > kernel support to get around the permissions problems. If root > permissions are required to be able to write to that XvImage, it's > not much use. I disagree. If I'm showing a movie, have rented a large projector and a hall, I'd really really like 24fps at exactly 72hz. Giving high-priority multimedia applications root access is a no-brainer. -- Billy Biggs [EMAIL PROTECTED] ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
On Wed, 17 Oct 2001, Billy Biggs wrote: > Sottek, Matthew J ([EMAIL PROTECTED]): > > > I can add a XvMC function to get you an XImage from the XvMC surface > > so that you can use XvMC instead of Xv. Then you can get rid of the > > delay between the XvShmPut() and the actual flip. > > But could this be a solution we could implement in general? We have tossed around an interface for this. But there needs to be kernel support to get around the permissions problems. If root permissions are required to be able to write to that XvImage, it's not much use. Mark. ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
On Wed, 17 Oct 2001, Billy Biggs wrote: > > To port this to the X world, maybe: > > o Can the driver tell me an exact value of the refresh rate? I bet > most cards can. Does xvidtune display that information? It must > get it from somewhere. You can already get the refresh rate via the VidMode extension but I'm not sure how accurate it is. The drivers rarely use the exact clock frequency specified in the modeline. Any value the driver would give you would only be as accurate as the clock on the card, which I suspect is not very accurate. > o Why would the timestamps from X be with only millisecond accuracy? That's the accuracy of X's internal clock. I suspect we could call get time of day and give it to you in microseconds since most platforms support that. Those timestamps would have no relation to X's normal times stamps like the ones in Events though. > I'm sure most video cards can trigger an interrupt on the vertical > refresh, that would be pretty accurate. Even if I could just query Handling the interrupt requires kernel support. The X-server is a user-space app. > the state: am I during a refresh or not, and have an accurate > timestamp for that, then I think I could probe this enough to get > some notion of a reference point. That won't work. You'll never be in the refresh. The X-server won't even be scheduled most of the time. The only thing that would be useful would be the scanline number and a timestamp if we can give you something other than X's normal millisecond time stamps, like the gettimeofday timeval. Mark. ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
Sottek, Matthew J ([EMAIL PROTECTED]): > Look at the ioctls for XvMC in i810's drm. There is one that > returns the overlay flip status. (fstatus). That returns a 32 Aha! Ok, see, I'm just not up to date with this DRI stuff. If we can do ioctl()s to /dev/dri and have this handled by the driver, the latency is now much more reasonable! Sounds like this is a good possibility for a standard way to query the refresh status. > I can add a XvMC function to get you an XImage from the XvMC surface > so that you can use XvMC instead of Xv. Then you can get rid of the > delay between the XvShmPut() and the actual flip. But could this be a solution we could implement in general? -- Billy Biggs [EMAIL PROTECTED] ___ Xpert mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/xpert
Re: [Xpert]Re: XVideo and refresh sync
Mark Vojkovich ([EMAIL PROTECTED]):
> > I would like to see:
> >
> > 1) A call returning the refresh rate and status. This way I can
> >calculate a playback strategy. It seems like it would be trivial
> >to implement for the i810 (good, cause I have one), and I'm sure
> >for many other drivers. Can we throw a func in somewhere, maybe
> >into Xv?
>
>The XF86VidMode extension is a more appropriate place, however, I'm
> not sure how you can make use of this. All the driver can basically
> tell you is a timestamp (millisecond accuracy is the best you'll get)
> and what line it was on at that time. And not all drivers will be
> able to tell you what line you're on.
I don't understand your pessimism. Using the VGA port and an nVidia
TNT2 card I was able to get sufficent accuracy. The problem, I guess,
is in the latency of an X call. How bad do you think this is? Is there
some way around it? Can the driver perform the hard work for me (it
probably can, but not as SCHED_FIFO).
Understandably, some drivers will suck. But I don't think this means
we should ever limit ourselves.
My strategy with the VGA port is as follows. Spawn a thread, set it
SCHED_FIFO, execute the following:
/* Spin until the refresh. */
while( !inb( 0x3da ) & 8 );
/* Spin until it ends. Now we're synced. */
while( inb( 0x3da ) & 8 );
/* Spin until the next refresh starts. */
while( !inb( 0x3da ) & 8 );
/* Get the time of day, this is our base time. */
gettimeofday( &basetime, 0 );
for( i = 0; i < bignumber; i++ ) {
/* Grab bignumber refreshes and mark their times using
* gettimeofday() or rdtsc. Use these to estimate the refresh
* rate.
*/
}
This gives us a time reference point of when a refresh occurred (+
some small error). We also have the refresh rate which is pretty
accurate. We can now use these to estimate when refreshes will occur in
the future, our accuracy hindered by our estimate of the refresh rate
and the assumption that it remains constant. However, during the
running of our program we can trivially spin and resync ourselves again
by finding a new basetime.
To port this to the X world, maybe:
o Can the driver tell me an exact value of the refresh rate? I bet
most cards can. Does xvidtune display that information? It must
get it from somewhere.
o Why would the timestamps from X be with only millisecond accuracy?
I'm sure most video cards can trigger an interrupt on the vertical
refresh, that would be pretty accurate. Even if I could just query
the state: am I during a refresh or not, and have an accurate
timestamp for that, then I think I could probe this enough to get
some notion of a reference point.
--
Billy Biggs
[EMAIL PROTECTED]
___
Xpert mailing list
[EMAIL PROTECTED]
http://XFree86.Org/mailman/listinfo/xpert
