Re: GStreamer, mp3 playback and the DSP
On Sun, Apr 11, 2010 at 12:01 AM, Javier S. Pedro wrote: > Simon Pickering wrote: >> So yes, my understanding is that that is exactly what's involved, >> therefore it should be possible to write an OpenMAX compliant codec that >> uses the DSP bridge to transfer data back and forth to a decoder running >> on the DSP. > > There's a already a OpenMAX compliant MP3 decoder, the Palm Pre uses it. > (mp3dec_sn.dll64P dsp side, libOMX.TI.Mp3.decode.so arm side). > Unfortunately, I don't know the licensing of it, and Nokia for sure is not > including it. I don't know which specific version of TI's MP3 decoder is Palm using, but you can find public versions of it from TI (not sure if you can distribute them): http://www.omapedia.org/wiki/L23.i3.3_Release_Notes I haven't tried this, but my guess is that simply sending buffers back and forth to the DSP would take more CPU, specially taking into account the OpenMAX overhead. The most efficient way to do this would be to add support for MP3 decoding to gst-dsp (bypassing OpenMAX). Then profiling would be needed in order to tune gst-dsp for audio (audio buffers are much smaller). You can try dsp-dummy to simulate MP3 decoding and see if it's worth the trouble. An alternative would be to take FFmpeg's MP3 decoder and see if there's any NEON optimizations that can be done. > In fact, Nokia explicitly chose to remove the audio DSP decoders from the > firmwares (see gst-openmax Maemo patches), probably to save rootfs space > since the builtin media player would not be using them (for powersaving > reasons). > > It would be nice if we could get a word or two from "above" about shipping > those as an extras package, for the use cases like the one described by the > OP. gst-openmax is just a wrapper in order to use OpenMAX IL components; there's no point in including the omx_mp3dec element if it's going to fail due to missing omx component. First you would need the DSP socket-node, then the omx component (libOMX.TI.Mp3.decode.so), and then it would make sense to add the element to gst-openmax. This would require modifications to gst-openmax and could be done as a separate package. >> I imagine/hope the N900 + DSP bridge setup is better, but I really don't >> know. Does anyone know? Could a Nokian tell us about the possibility of >> direct audio output from the DSP and how efficient the data passing is >> in the absence of direct audio output? > > I am not an authority here either, but remember that audio to the speakers > should pass through PulseAudio's HP filter, or else they might be damaged. > (http://talk.maemo.org/showthread.php?t=36555). So I guess the question is > indeed the latter one :) Yeah, I don't know if it's possible to do that, but I wouldn't try. Cheers. -- Felipe Contreras ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: GStreamer, mp3 playback and the DSP
Hello, On Sat, Apr 10, 2010 at 3:29 PM, javicq wrote: > I'm currently using GStreamer to play mp3 files. I'm trying to move all mp3 > decoding to the DSP but I'm unable to find any useful info on the matter. > The only thing i got so far is this paragraph from the wiki: > > http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Architecture/Multimedia_Domain#Audio_Codecs Unfortunately that documentation is outdated. By the time N900 was released the audio codecs were not using OpenMAX IL, but that's irrelevant as the codecs were always meant to run on ARM side. -- Felipe Contreras ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
RE: GStreamer, mp3 playback and the DSP
Oops, managed to not reply to the list so forwarding now. Sorry to break the threading too. Simon -Original Message- From: Simon Pickering [mailto:s.g.picker...@bath.ac.uk] Sent: 11 April 2010 00:25 To: 'Javier S. Pedro' Subject: RE: GStreamer, mp3 playback and the DSP > > I imagine/hope the N900 + DSP bridge setup is better, but I really > > don't know. Does anyone know? Could a Nokian tell us about the > > possibility of direct audio output from the DSP and how > efficient the > > data passing is in the absence of direct audio output? > > I am not an authority here either, but remember that audio to the > speakers should pass through PulseAudio's HP filter, or else they > might be damaged. > (http://talk.maemo.org/showthread.php?t=36555). So I guess the > question is indeed the latter one :) Well an HP filter could also be implemented on the DSP (either currently or theoretically) so direct output is not necessarily out of the question, though I imagine my postulated test DSP audio codec would realistically be removed to save space before shipping as PulseAudio is used. But perhaps it might now be possible to add an audio codec driver without needing to recompile the entire DSP kernel (not having looked at how the linking is performed for "DSP tasks" nowadays)... Does anyone know off the top of their heads? Cheers, Simon ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: GStreamer, mp3 playback and the DSP
Simon Pickering wrote: > So yes, my understanding is that that is exactly what's involved, > therefore it should be possible to write an OpenMAX compliant codec that > uses the DSP bridge to transfer data back and forth to a decoder running > on the DSP. There's a already a OpenMAX compliant MP3 decoder, the Palm Pre uses it. (mp3dec_sn.dll64P dsp side, libOMX.TI.Mp3.decode.so arm side). Unfortunately, I don't know the licensing of it, and Nokia for sure is not including it. In fact, Nokia explicitly chose to remove the audio DSP decoders from the firmwares (see gst-openmax Maemo patches), probably to save rootfs space since the builtin media player would not be using them (for powersaving reasons). It would be nice if we could get a word or two from "above" about shipping those as an extras package, for the use cases like the one described by the OP. I'm pretty sure that at least the source for the AAC decoder's arm side is included with the tiopendsp tarball (because I saw it there), but I have no idea about the rest. > I imagine/hope the N900 + DSP bridge setup is better, but I really don't > know. Does anyone know? Could a Nokian tell us about the possibility of > direct audio output from the DSP and how efficient the data passing is > in the absence of direct audio output? I am not an authority here either, but remember that audio to the speakers should pass through PulseAudio's HP filter, or else they might be damaged. (http://talk.maemo.org/showthread.php?t=36555). So I guess the question is indeed the latter one :) -- Javier ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
RE: GStreamer, mp3 playback and the DSP
> >> I'm currently using GStreamer to play mp3 files. I'm > >> trying to move > >> all mp3 decoding to the DSP but I'm unable to find any > >> useful info on > >> the matter. > > > > May I ask why you're trying to move mp3 decoding to the > > DSP? Are the > > mp3s the sound track/effects for a game, etc., so you want to have > > more CPU cycles to do useful work? > > Yes, it's an emulator actually, and mp3 decoding is > noticeably stealing CPU cycles. Ok, I just wanted to check that your reason was something valid and indeed it is. > >> So, obviuosly a different solution is needed but I coudn't > >> find any > >> documentation whatsoever. :) Anyone can help here? > > > > Are there any existing OpenMAX mp3 decoders targetted at the > > TMS320c64x DSP? > > If not, it would probably not be too painful to port one > (where pain > > is all relative, but at least this would be less painful > than trying > > on the more constrained and 8bit-less c55 ;) > > > > Thats what the wiki article I quoted seems to imply, but it's > not very clear and I couldn't find any further info. There's > a multimedia framework diagram suggesting that there are > OpenMAX audio codecs that run in the DSP via DSP-bridge, but > not sure if this applies to maemo 5 and there's no info on > how to use them. I must admit that I've not done any DSP work on the c64, though I did pay attention to it when the N900 was originally announced (and I thought I'd get a Pandora to fill the gap until it was shipped, glad I didn't pin my hopes on that now ;). Anyway I imagine some of the Nokians (and others) will know from first hand experience so please someone jump in if I've got the wrong idea. So yes, my understanding is that that is exactly what's involved, therefore it should be possible to write an OpenMAX compliant codec that uses the DSP bridge to transfer data back and forth to a decoder running on the DSP. There were docs and examples on one of the websites (this one iirc: https://gforge.ti.com/gf/project/omapbridge/ and DSP coding docs on this site: http://www.ti.com/sc/docs/psheets/man_dsp.htm), not sure about the OpenMAX docs, but I guess there will be example code floating about somewhere (or you might just wrap the DSP bridge interface stuff directly in a Gstreamer plugin and skip the OpenMax stuff completely). I think that writing/porting this should be reasonably straight-forward, optimising it less so, but at least afaiu the code shouldn't need major changes to get rid of 8bit types. However on the N8x0 devices it was not possible to directly output audio from the DSP due to a lack of api docs (it would need a re-write of the DSP kernel or reverse engineering the DSP audio codec api, which are things that would now be good for Mer, but would take some time, but that's another story). I don't know if it is currently (or would be relatively easily) possible to output audio directly from the N900's DSP. This is something you should try to find out, as on the N8x0, needing to pass data both to and back from the DSP (in this case for SBC encoding) clobbered video decoding on the CPU (CPU load was lower with SBC encoding on the DSP, but something, presumably memory contention due to the way the data was passed to and fro for both the DSP and framebuffer, dropped the mplayer decoded framerate down from iirc ~25fps to ~1fps). I imagine/hope the N900 + DSP bridge setup is better, but I really don't know. Does anyone know? Could a Nokian tell us about the possibility of direct audio output from the DSP and how efficient the data passing is in the absence of direct audio output? Cheers, Simon ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
RE: GStreamer, mp3 playback and the DSP
Simon Pickering wrote: > > >> I'm currently using GStreamer to play mp3 files. I'm trying >> to move all mp3 decoding to the DSP but I'm unable to find >> any useful info on the matter. > > May I ask why you're trying to move mp3 decoding to the DSP? Are the mp3s > the sound track/effects for a game, etc., so you want to have more CPU > cycles to do useful work? > > Yes, it's an emulator actually, and mp3 decoding is noticeably stealing CPU cycles. Simon Pickering wrote: > > >> The only thing i got so far is this paragraph from the wiki: >> >> http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Ar > chitecture/Multimedia_Domain#Audio_Codecs >> >> Most of the audio encoders and decoders are wrapped under the >> OpenMAX IL interface [OMAX], which abstracts the codec >> specific implementation (which in general can run on ARM or >> DSP). Unless a different solution is needed (due e.g. to >> sourcing problems, performance requirements or to fulfill >> some specific use cases) all the audio codecs will be running >> on the ARM side to simplify the audio architecture and to >> avoid the additional latency and load over the data path due >> to the routing of the audio data first to DSP and then back to ARM. >> >> So, obviuosly a different solution is needed but I coudn't >> find any documentation whatsoever. :) Anyone can help here? > > Are there any existing OpenMAX mp3 decoders targetted at the TMS320c64x > DSP? > If not, it would probably not be too painful to port one (where pain is > all > relative, but at least this would be less painful than trying on the more > constrained and 8bit-less c55 ;) > > Thats what the wiki article I quoted seems to imply, but it's not very clear and I couldn't find any further info. There's a multimedia framework diagram suggesting that there are OpenMAX audio codecs that run in the DSP via DSP-bridge, but not sure if this applies to maemo 5 and there's no info on how to use them. -- View this message in context: http://n2.nabble.com/GStreamer-mp3-playback-and-the-DSP-tp4881751p4882126.html Sent from the maemo-developers mailing list archive at Nabble.com. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
RE: GStreamer, mp3 playback and the DSP
> I'm currently using GStreamer to play mp3 files. I'm trying > to move all mp3 decoding to the DSP but I'm unable to find > any useful info on the matter. May I ask why you're trying to move mp3 decoding to the DSP? Are the mp3s the sound track/effects for a game, etc., so you want to have more CPU cycles to do useful work? > The only thing i got so far is this paragraph from the wiki: > > http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Ar chitecture/Multimedia_Domain#Audio_Codecs > > Most of the audio encoders and decoders are wrapped under the > OpenMAX IL interface [OMAX], which abstracts the codec > specific implementation (which in general can run on ARM or > DSP). Unless a different solution is needed (due e.g. to > sourcing problems, performance requirements or to fulfill > some specific use cases) all the audio codecs will be running > on the ARM side to simplify the audio architecture and to > avoid the additional latency and load over the data path due > to the routing of the audio data first to DSP and then back to ARM. > > So, obviuosly a different solution is needed but I coudn't > find any documentation whatsoever. :) Anyone can help here? Are there any existing OpenMAX mp3 decoders targetted at the TMS320c64x DSP? If not, it would probably not be too painful to port one (where pain is all relative, but at least this would be less painful than trying on the more constrained and 8bit-less c55 ;) Cheers, Simon ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
