Re: Neo1973 video (was Re: What would be a realistic but challenging level for Bryce announced trophy money for video playback on the Neo1973? Re: h.264 format is now open?)
Siarhei Siamashka wrote: > I'm just interested in improving video support for ARM based devices, > that's why I posted to this openmoko mailing list . I'm currently trying to > integrate a fast scaler for ARM into ffmpeg library (the engine used > by mplayer, vlc and the other video players for linux ): > http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-January/051209.html > > So now I'm trying to collect information about what scaling/color conversion > functionality would be useful for various devices. Looks like Neo 1973 > may need rgb565 => rgb565 scaler, image rotation support and probably > yv12 => rgb565 (grayscale). I don't have much free time, so support for > just Nokia internet tablets has the highest priority for me. But if anybody > else would like to work on the features needed for Neo 1973 video support, > I would be glad to help. This looks pretty interesting work. Do you have any interest in and would it be possible to do this as a gstreamer plugin? Thanks, Rob Taylor ___ OpenMoko community mailing list community@lists.openmoko.org https://lists.openmoko.org/mailman/listinfo/community
Re: Neo1973 video (was Re: What would be a realistic but challenging level for Bryce announced trophy money for video playback on the Neo1973? Re: h.264 format is now open?)
On Mon, Feb 05, 2007 at 02:36:04AM +0200, Siarhei Siamashka wrote: > On Saturday 03 February 2007 15:47, you wrote: > > > Also, the 770 has a landscape display. We have a portrait display. The > > S3C2410 cannot rotate the image, so you would have to rotate every frame > > in software, too! > > That's a good point. Anyway, rotation can be combined with scaling or color > format conversion and done in a single pass, so overhead should not be too > big. One more solution is rotation at video transcode stage as Mikko > suggested. > > > > Just for some experiment, I compiled mplayer for arm920t (not using > > > armv5te instructions), and benchmarked it with sdl video output (software > > > YUV->RGB conversion, generic nonoptimized scaling 320x240 => 640x480) and > > > libmad mp3 audio decoder. > > > > Please note that the LCM we use in the Neo1973 can do hardware scaling, > > e.g. theoretically you can software-reconfigure the LCM to behave as > > QVGA 240x320, and then change the s3c2410_fb kernel driver timings > > accordingly. > > If I understand that correctly, it is not arbitrary scaling but support for > 240x320 resolution? yes, you understand correctly. > > This has not been tested or implemented by us, since we're mainly > > interested in getting a high-res phone UI working right now :) > > I clearly understand that :) I guess it is one of the reasons, why you > announced early access to the device for open source developers. I hope > that some of them would try implementing some video support. exactly. I just wanted to make sure that this is not on our agenda, at least with any significant priority. > As it seems to be not quite trivial to do, this part of work can wait a bit > until some initial video benchmarks are available (centered nonscaled > 320x240 video playback vs. software scaled to fullscreen). Yes, obviously those benchmarks would help. Anyway, the "hardware scaling" is also interesting for non-video applicaitions such as many old video games (ScummVM and the like). As for the benchmarks: Obviously you consume only a quarter of memory bandwidth when you use 320x240 than 640x480. So native 320x240 playback should at least theoretically be faster than centered nonscaled 320x240 on VGA. > I'm just interested in improving video support for ARM based devices, > that's why I posted to this openmoko mailing list . I'm currently trying to > integrate a fast scaler for ARM into ffmpeg library (the engine used > by mplayer, vlc and the other video players for linux ): > http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-January/051209.html good luck! I've done some ffmpeg hacking before, and it was ... painful... ;) -- - Harald Welte <[EMAIL PROTECTED]> http://openmoko.org/ Software for the world's first truly open Free Software mobile phone ___ OpenMoko community mailing list community@lists.openmoko.org https://lists.openmoko.org/mailman/listinfo/community
Re: Neo1973 video (was Re: What would be a realistic but challenging level for Bryce announced trophy money for video playback on the Neo1973? Re: h.264 format is now open?)
On Saturday 03 February 2007 15:47, you wrote: > Also, the 770 has a landscape display. We have a portrait display. The > S3C2410 cannot rotate the image, so you would have to rotate every frame > in software, too! That's a good point. Anyway, rotation can be combined with scaling or color format conversion and done in a single pass, so overhead should not be too big. One more solution is rotation at video transcode stage as Mikko suggested. > > Just for some experiment, I compiled mplayer for arm920t (not using > > armv5te instructions), and benchmarked it with sdl video output (software > > YUV->RGB conversion, generic nonoptimized scaling 320x240 => 640x480) and > > libmad mp3 audio decoder. > > Please note that the LCM we use in the Neo1973 can do hardware scaling, > e.g. theoretically you can software-reconfigure the LCM to behave as > QVGA 240x320, and then change the s3c2410_fb kernel driver timings > accordingly. If I understand that correctly, it is not arbitrary scaling but support for 240x320 resolution? But as we should first aim at transcoded video support anyway, that would provide some performance improvement. > This has not been tested or implemented by us, since we're mainly > interested in getting a high-res phone UI working right now :) I clearly understand that :) I guess it is one of the reasons, why you announced early access to the device for open source developers. I hope that some of them would try implementing some video support. > However, if somebody skilled wants to write, integrate and test code for > this, I'm happy the help. We cannot disclose the LCM data sheet, but I > can add the software sequence for 240x320 / 480x640 switching to the > [GPL licensed] kernel LCM driver at least after phase-1 (March 11). > > The S3C2410 user manual is publicly available from Samsung > Semiconductors, so basically it's up to whoever is interested to > implement it. One interesting question is how to merge this into > X11/Kdrive... As far as I understand, only the very latest big > (non-kdrive) X11 has randr capabilities for dynamically adding/removing > viewports and changing resolutions. > > [But did I mention that Keith Packard gets a phase-0 phone? *g*] As it seems to be not quite trivial to do, this part of work can wait a bit until some initial video benchmarks are available (centered nonscaled 320x240 video playback vs. software scaled to fullscreen). About integration with x11/kdrive. Nokia 770 also uses kdrive, and implements some extensions to access extra features (pixel doubling). Unfortunately it does not provide access to hardware YUV colorspace (libxv exists, but it is just a stub and does nothing). So right not mplayer on Nokia 770 uses some kind of hack and accesses framebuffer concurrently with x11 server. Synchronization is not perfect, so it is possible to get garbage on the screen when you switch between different applications a lot while mplayer video playback is active. A more clean solution would be to patch xserver to interact with mplayer better to synchronize framebuffer access (kdrive on Nokia 770 already has some patches to synchronize with DSP video decoder used by built-in video player). Maybe I'll try this later. I'm just interested in improving video support for ARM based devices, that's why I posted to this openmoko mailing list . I'm currently trying to integrate a fast scaler for ARM into ffmpeg library (the engine used by mplayer, vlc and the other video players for linux ): http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-January/051209.html So now I'm trying to collect information about what scaling/color conversion functionality would be useful for various devices. Looks like Neo 1973 may need rgb565 => rgb565 scaler, image rotation support and probably yv12 => rgb565 (grayscale). I don't have much free time, so support for just Nokia internet tablets has the highest priority for me. But if anybody else would like to work on the features needed for Neo 1973 video support, I would be glad to help. ___ OpenMoko community mailing list community@lists.openmoko.org https://lists.openmoko.org/mailman/listinfo/community
Re: Neo1973 video (was Re: What would be a realistic but challenging level for Bryce announced trophy money for video playback on the Neo1973? Re: h.264 format is now open?)
su, 2007-02-04 kello 18:28 -0500, Bryce Leo kirjoitti: > I'm sorry guys... but instead of flipping the video or worrying about > re-coding... how bout a non-techical solution. Rotate the Device > You certainly wouldn't catch me wanting to watch a video in portrait > modethat would ruin the whole thing. Why would you want to watch a > video in portrait mode? We don't. The framebuffer, however, is in portrait orientation. Hence the need to rotate to landscape at some point, be it at play time in the player or the X server (probably rather the former, efficiency-wise), or at recode time to take some work off of the Neo CPU. -- Mikko Rauhala - [EMAIL PROTECTED] - http://www.iki.fi/mjr/> Transhumanist - WTA member - http://www.transhumanism.org/> Singularitarian - SIAI supporter - http://www.singinst.org/> ___ OpenMoko community mailing list community@lists.openmoko.org https://lists.openmoko.org/mailman/listinfo/community
Re: Neo1973 video (was Re: What would be a realistic but challenging level for Bryce announced trophy money for video playback on the Neo1973? Re: h.264 format is now open?)
I'm sorry guys... but instead of flipping the video or worrying about re-coding... how bout a non-techical solution. Rotate the Device You certainly wouldn't catch me wanting to watch a video in portrait modethat would ruin the whole thing. Why would you want to watch a video in portrait mode? Why don't we just relax for a bit. The bounty only applies if the solution is non-trivial. I'm certain that it's going to take just a compilation of vlc, or MPlayer to get the job done with very little work. ___ OpenMoko community mailing list community@lists.openmoko.org https://lists.openmoko.org/mailman/listinfo/community
Re: Neo1973 video (was Re: What would be a realistic but challenging level for Bryce announced trophy money for video playback on the Neo1973? Re: h.264 format is now open?)
Since you're going to have to reencode your videos for OpenMoko anyway, they could easily be prerotated in the process. That's what we were doing on the Tapwave Zodiac until the amazing people from CoreCodec worked their magic. On 2/4/07, Ian Stirling <[EMAIL PROTECTED]> wrote: Mikko Rauhala wrote: > la, 2007-02-03 kello 14:47 +0100, Harald Welte kirjoitti: >> Also, the 770 has a landscape display. We have a portrait display. The >> S3C2410 cannot rotate the image, so you would have to rotate every frame >> in software, too! > > Modifying a player to render the image to be rotated in the first place > shouldn't be hard, though perhaps any straightforward way to do that > would degrade cache behaviour? (A wild guess.) > > Also, as re-encoding files spesifically for the Neo would be quite ok > with me, rotating at that point would be quite a workable solution also. > (I'll add that I don't necessarily expect the Neo1973 to play back > decent video and won't blame FIC if it doesn't, but I'll sure _try_ > it ;) The comparison I'm using internally - Pentium 100, just about will manage that. I do expect to need to pick codecs very carefully - there won't be much spare. Rotating internally may be the straw that breaks the camels back. OTOH - going down to 12fps isn't _that_ visible on this size of screen, and that helps a fair bit - as does reducing the sound bandwidth, or going mono. _simple_ audio codecs - such as NICAM 728 equivalent - which is 10 bit, stereo companded from 14 bits, at 32Khz - may even be appropriate in some cases - to reduce the load on the CPU, and trade off for bandwidth. ___ OpenMoko community mailing list community@lists.openmoko.org https://lists.openmoko.org/mailman/listinfo/community ___ OpenMoko community mailing list community@lists.openmoko.org https://lists.openmoko.org/mailman/listinfo/community
Re: Neo1973 video (was Re: What would be a realistic but challenging level for Bryce announced trophy money for video playback on the Neo1973? Re: h.264 format is now open?)
Mikko Rauhala wrote: la, 2007-02-03 kello 14:47 +0100, Harald Welte kirjoitti: Also, the 770 has a landscape display. We have a portrait display. The S3C2410 cannot rotate the image, so you would have to rotate every frame in software, too! Modifying a player to render the image to be rotated in the first place shouldn't be hard, though perhaps any straightforward way to do that would degrade cache behaviour? (A wild guess.) Also, as re-encoding files spesifically for the Neo would be quite ok with me, rotating at that point would be quite a workable solution also. (I'll add that I don't necessarily expect the Neo1973 to play back decent video and won't blame FIC if it doesn't, but I'll sure _try_ it ;) The comparison I'm using internally - Pentium 100, just about will manage that. I do expect to need to pick codecs very carefully - there won't be much spare. Rotating internally may be the straw that breaks the camels back. OTOH - going down to 12fps isn't _that_ visible on this size of screen, and that helps a fair bit - as does reducing the sound bandwidth, or going mono. _simple_ audio codecs - such as NICAM 728 equivalent - which is 10 bit, stereo companded from 14 bits, at 32Khz - may even be appropriate in some cases - to reduce the load on the CPU, and trade off for bandwidth. ___ OpenMoko community mailing list community@lists.openmoko.org https://lists.openmoko.org/mailman/listinfo/community
Re: Neo1973 video (was Re: What would be a realistic but challenging level for Bryce announced trophy money for video playback on the Neo1973? Re: h.264 format is now open?)
la, 2007-02-03 kello 14:47 +0100, Harald Welte kirjoitti: > Also, the 770 has a landscape display. We have a portrait display. The > S3C2410 cannot rotate the image, so you would have to rotate every frame > in software, too! Modifying a player to render the image to be rotated in the first place shouldn't be hard, though perhaps any straightforward way to do that would degrade cache behaviour? (A wild guess.) Also, as re-encoding files spesifically for the Neo would be quite ok with me, rotating at that point would be quite a workable solution also. (I'll add that I don't necessarily expect the Neo1973 to play back decent video and won't blame FIC if it doesn't, but I'll sure _try_ it ;) > Please note that the LCM we use in the Neo1973 can do hardware scaling, > e.g. theoretically you can software-reconfigure the LCM to behave as > QVGA 240x320, and then change the s3c2410_fb kernel driver timings > accordingly. Now this is excellent news. I asked before if the XVideo extension present on the xdpyinfo output implied scaling support; apparently it didn't, but even having full screen QVGA capability should aid significantly. And yeah, do concentrate on the high-res UI for now, but great if you can at some point provide code/info for the QVGA mode. And Keith getting a p0 phone, that's a nice one too ;) -- Mikko Rauhala - [EMAIL PROTECTED] - http://www.iki.fi/mjr/> Transhumanist - WTA member - http://www.transhumanism.org/> Singularitarian - SIAI supporter - http://www.singinst.org/> ___ OpenMoko community mailing list community@lists.openmoko.org https://lists.openmoko.org/mailman/listinfo/community
Neo1973 video (was Re: What would be a realistic but challenging level for Bryce announced trophy money for video playback on the Neo1973? Re: h.264 format is now open?)
On Fri, Feb 02, 2007 at 11:06:43PM +0200, Siarhei Siamashka wrote: > On Friday 02 February 2007 12:31, Robert Michel wrote: > > > Some video information given by Sean in november: > > http://lists.openmoko.org/pipermail/community/2006-November/000340.html > > A few technical questions here. Is image data constantly transferred to LCD > from system memory (60 times per second)? yes, since the graphics controller is embedded in the S3C2410 SoC. The LCM signal lines are directly attached to the S3C2410. There is no other graphics memory than main memory. > If we compare it to Nokia 770, Nokia 770 has a similar design with framebuffer > stored in the system memory with data getting transferred to graphics chip > using DMA when needed (graphics chip has its own video memory). That's a significantly different design. Also, the 770 has a landscape display. We have a portrait display. The S3C2410 cannot rotate the image, so you would have to rotate every frame in software, too! > If video is really heavy on memory bus, is it possible to reduce refresh rate > somewhat (to 50Hz for example) to improve performance? There's not too much room since the minimum pixel clock frequency of the LCM is 20MHz. At a typical cycle of 520x648, you get 59Hz minimum refresh rate. You could probably go all the way to extend horzontal /vertical front/back porch, etc. But they can only be increased a tiny bit, the maximum values are determined by the S3C2410 register set. > Just for some experiment, I compiled mplayer for arm920t (not using armv5te > instructions), and benchmarked it with sdl video output (software YUV->RGB > conversion, generic nonoptimized scaling 320x240 => 640x480) and libmad > mp3 audio decoder. Please note that the LCM we use in the Neo1973 can do hardware scaling, e.g. theoretically you can software-reconfigure the LCM to behave as QVGA 240x320, and then change the s3c2410_fb kernel driver timings accordingly. This has not been tested or implemented by us, since we're mainly interested in getting a high-res phone UI working right now :) However, if somebody skilled wants to write, integrate and test code for this, I'm happy the help. We cannot disclose the LCM data sheet, but I can add the software sequence for 240x320 / 480x640 switching to the [GPL licensed] kernel LCM driver at least after phase-1 (March 11). The S3C2410 user manual is publicly available from Samsung Semiconductors, so basically it's up to whoever is interested to implement it. One interesting question is how to merge this into X11/Kdrive... As far as I understand, only the very latest big (non-kdrive) X11 has randr capabilities for dynamically adding/removing viewports and changing resolutions. [But did I mention that Keith Packard gets a phase-0 phone? *g*] Cheers, -- - Harald Welte <[EMAIL PROTECTED]> http://openmoko.org/ Software for the world's first truly open Free Software mobile phone ___ OpenMoko community mailing list community@lists.openmoko.org https://lists.openmoko.org/mailman/listinfo/community
