Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
On mer., 2010-06-16 at 13:15 +0300, Sivan Greenberg wrote: > > I think you are barking at the wrong tree - pulseaudio should get way > > better latency than 5000ms, otherwise it would be useless for pretty > > much everything. > > My thought exactly. If that really is the case, a proper phone > conversation would had never been possible on the device, but my > everyday shows it is :-) > Not wanting to put oil on the fire (or so we say), but he said that QAudioInput bug had 5000ms latency, not pulseaudio, and afaik Maemo 5 uses gstreamer, so I guess no QAudioInput there. -- Yves-Alexis signature.asc Description: This is a digitally signed message part ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
Benno Senoner wrote: Thanks to all for your responses, yes I agree with what you said. I will try to track the evolution of the audio subsystem on meego and provide my feedback and findings. I'll discuss the issue with the JACK authors too what they think about it, about adding power management to jack etc. (the main author of Jack2, Stephane Letz is one of the coauthors to LinuxSampler too). It's understandable that Nokia will probably not change the audio subsystem for the current Meego version, so at least they should try to provide a stable working system, giving developers a way to achieve <50msec mic to speaker latency. As I understand it, pulseaudio on the n900 achieves 5ms latency, and a few dozen microseconds of jitter. It's just not for normal users. I point you at: http://linuxplumbersconf.org/2009/slides/Jyri-Sarha-audio_miniconf_slides.pdf ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
Thanks to all for your responses, yes I agree with what you said. I will try to track the evolution of the audio subsystem on meego and provide my feedback and findings. I'll discuss the issue with the JACK authors too what they think about it, about adding power management to jack etc. (the main author of Jack2, Stephane Letz is one of the coauthors to LinuxSampler too). It's understandable that Nokia will probably not change the audio subsystem for the current Meego version, so at least they should try to provide a stable working system, giving developers a way to achieve <50msec mic to speaker latency. I'm not going to invest time to improve pulseaudio as I, like any audio developer I know, consider it a wastly inferiour audio server with regard to low latency and stability. But I will provide benchmarking apps and feedback in order to point out problems if they arise. And as said, if Nokia and Intel manages to provide stable low latency operation I'll happily use the API they provide and if pulse audio is capable to do it then QAudioInput/QAudioOutput can certainly be fixed to provide the same kind of performance since it's simply a wrapper. But if we discover that the performance of the current Meego for user space apps is not easily fixable then my advice is to swap out pulseaudio for JACK2, provide a compatiblity layer that exposes the pulse API (if needed) so that no app needs to be rewritten etc. It's still early to say how the whole thing turns out but I can certainly give my share of contribution to improve the audio subsystem. LinuxSampler in interesting for benchmarking an OS, because it is a very very demanding app, it really puts big stress on the OS since it streams large, multi GByte samples from disk, pre-caches the sample heads (first part) into memory and allows low latency playback of the samples (with single digit msec milatency). It runs one thread (or a callback) with real time priority that renders the audio, another that listens to MIDI data and a lower priority disk I/O thread which writes to lock-free zero-copy buffers. And if you load a lot of samples and play complex musical pieces with hundreds of voices you end up using 95% of your CPU, 95% of your RAM, 95% of your disk I/O bandwidth, while still providing only a few msec audio latency without dropout. Linux kernels with scheduling preemption enabled and JACK on ALSA have proved to be capable to satisfy all the above conditions. I'm not saying that Meego on a phone should be able to provide such excellent numbers as on a desktop, but Meego engineers should work hard to give it response times comparable to the leading mobile operating systems and sorry again for mentioning the Iphone :) I fully disagree with Apple's closed systems but apart of the brand and that's fashionable to buy their products, they get some stuff working right especially the audio/video parts. I don't know to which extent the impossibility/difficulty to achieve low audio latency "worsens" the appeal of a smartphone platform, but it is certainly depressing when you spend several hundred $/E on a phone with excellent specs and you cannot have apps like virtual musical instruments (eg piano,flute, etc), vocoders (voice changers), apps that performs by analyzing the audio in real time etc, while your Iphone colleagues have all sort of apps for that. Of course the usefulness of those apps is debatable but the fact is that the bigger the app market the more the platform becomes, and we all know that Ovi Store still pales compared to Apple's app store in terms of number of daily downloads and numbers of apps available, despite the fact that Nokia has a much bigger market share and has deals about sharing Ovi apps with big mobile operators like China Mobile ecc. Apple is yet another proof that getting the basics right your product sells without advertising. I know nowadays it's fashionable to bash Nokia at any occasion, especially in the US where journalists write as Nokia and Ovi Store did not exist and the whole company will fail soon, which is very unfair, since the US != World. So Nokia just needs to demonstrate that there is a reason why they are #1 and that they can beat the competitors with better hardware AND software. Nokia has been very open source friendly lately and developers highly esteem this, but as said they need to get the details right and working like a swiss clockwork, including audio :) Nokia needs to get their things right and release products and software much faster than in the past. Android is advancing very fast too (100k phones / day shipped ATM) and it's open source too so it's a big competitor for Symbian/Meego. OTOH it is good that Nokia has competitors so they are forced to evolve and raise the bar otherwise users will switch platforms and perhaps if Nokia was the only player on the market the world would probbly still use an old symbian version on keypad-style phones. Again sorry for the long digression, and as many here,
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
On Fri, Jun 18, 2010 at 2:19 PM, Benno Senoner wrote: > Sadly, this week Nokia stock again dropped like a rock, while Apple > gets preorders for 600k Iphone 4 in a day and have trouble > to process all the orders. Does that not ring alarm bell at Nokia ? Right, this is certainly not the appropriate medium for this kind of discussion, but still I had the give my argument to stand up for those who support and follow open source with concept operating systems and platforms before it reaches the polish of the competitors, in sack of open development process. What Nokia does and has done with Maemo and MeeGo is yet unprecedented from a commercial market point of view. Let me assure you that if you communicate your issues you will get a serious response and if you're right with your complaints action will be taken. Sivan ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
RE: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
Benno, Out of curiosity, why not just compile it yourself? According to the Jack2 and PulseAudio sites they can work together with PA automatically releasing the sound system for J2 when J2 needs it using DBUS. So rather than complain about it; just port it real quick and then you will have actual numbers and the component you need to continue your work on your main project. :-D I'm not saying you aren't right that J2 offers a lot; it just that I would venture a educated guess that Nokia will NOT waste resources to change the underlying sound system on a OS (Maemo) that is at a EOL status.As for Meego, you need to obviously present a case on the Meego mailing list -- but since they already shipped a version; gutting the sound system may not be something worth while that they would consider until a later major version.Although who knows, maybe they would -- and since it is Open Source, and you can submit patches; if you actually got J2 running great on Meego, that would make the process a whole lot easier for them to decide to switch. Nathan -Original Message- From: maemo-developers-boun...@maemo.org [mailto:maemo-developers-boun...@maemo.org] On Behalf Of Benno Senoner Sent: Friday, June 18, 2010 6:19 AM To: maemo-developers@maemo.org Subject: Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future I know that I'm a bit a repetitive person, but my intent is to raise an alarm, that nowadays companies cannot afford to ship, half-finished, half-baked, buggy systems, and moreover chose the vastly inferiour (pulseaudio) open source component over the better (JACK2 audio server). I reported my findings here, on the maemo-developers mailingslist, which is read by Nokia engineers too, and the only responses I get back is that I should not "whine". No technical responses at all, how the latency can be lowered using other APIs etc.. Sadly, this week Nokia stock again dropped like a rock, while Apple gets preorders for 600k Iphone 4 in a day and have trouble to process all the orders. Does that not ring alarm bell at Nokia ? As said nowadays either you deliver a smooth user experience (including dropout free, low latency audio for apps) and make it easy for developers to access that features, or your market share will continue to drop and drop. It seems like Maemo will be end of lined soon so I'm not expecting Maemo getting much improvements, but it's successor, Meego HAS to deliver a smooth user experience. I will continue those discussions on the Meego list and perform benchmarks and tests so that everyone can see what the real numbers are and if there are still problems regarding low latency audio. Unfortunately, given the weak implementation of pulseaudio I fear that the prerformance will not be stellar. I hope that I'm proved wrong Did Meego enginers publish such numbers, yet ? eg what latency can be achieved on certain hardware, audio dropout frequency etc ? Execpt for technical discussions about audio I will not continue my long diatribes on this list. But as said I will continue to point out possible flaws in Meego regarding audio backing my claims up with numbers and bencharking apps. If the hardware is capable to be responsive, there is no reason why Meego should not fully exploit it regards, Benno The LinuxSampler Project http://www.linuxsampler.org 2010/6/16 Ville M. Vainio : > On Wed, Jun 16, 2010 at 3:57 PM, Benno Senoner > wrote: > >> I just represent a multimedia developer wanting to use the platform >> and point out the flaws in order to contribute to improve it. > > ... and that is appreciated. OTOH it's waste of typing to engage in > long diatribes about business models and whatnot here (such discussion > is more appropriate for talk.maemo.org). > > -- > Ville M. Vainio > http://tinyurl.com/vainio > ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
On Fri, Jun 18, 2010 at 12:19 PM, Benno Senoner wrote: > and the only responses I get back is that I should not "whine". No > technical responses at all, how the latency can be lowered using other > APIs etc.. The actual responses from both myself and Ville were that you're really not in the right place for this kind of discussion, both technical and political (=> forum). Not to stop whining. Perhaps if you listened to that, you'd get feedback of the type you're after. > But as said I will continue to point out possible flaws in Meego > regarding audio backing my claims up with numbers and bencharking > apps. By all means. Just make sure you point them out in the right place. If you're running MeeGo (as you keep mentioning), point them out to the relevant channels inside MeeGo. (hint: probably not maemo-developers.) OTOH, if you've found problems in Qt, which is as I recall the only concrete issue you have mentioned so far, please report them, to Qt, so they can be addressed. Thanks. -- Robin Burchell http://rburchell.com ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
I know that I'm a bit a repetitive person, but my intent is to raise an alarm, that nowadays companies cannot afford to ship, half-finished, half-baked, buggy systems, and moreover chose the vastly inferiour (pulseaudio) open source component over the better (JACK2 audio server). I reported my findings here, on the maemo-developers mailingslist, which is read by Nokia engineers too, and the only responses I get back is that I should not "whine". No technical responses at all, how the latency can be lowered using other APIs etc.. Sadly, this week Nokia stock again dropped like a rock, while Apple gets preorders for 600k Iphone 4 in a day and have trouble to process all the orders. Does that not ring alarm bell at Nokia ? As said nowadays either you deliver a smooth user experience (including dropout free, low latency audio for apps) and make it easy for developers to access that features, or your market share will continue to drop and drop. It seems like Maemo will be end of lined soon so I'm not expecting Maemo getting much improvements, but it's successor, Meego HAS to deliver a smooth user experience. I will continue those discussions on the Meego list and perform benchmarks and tests so that everyone can see what the real numbers are and if there are still problems regarding low latency audio. Unfortunately, given the weak implementation of pulseaudio I fear that the prerformance will not be stellar. I hope that I'm proved wrong Did Meego enginers publish such numbers, yet ? eg what latency can be achieved on certain hardware, audio dropout frequency etc ? Execpt for technical discussions about audio I will not continue my long diatribes on this list. But as said I will continue to point out possible flaws in Meego regarding audio backing my claims up with numbers and bencharking apps. If the hardware is capable to be responsive, there is no reason why Meego should not fully exploit it regards, Benno The LinuxSampler Project http://www.linuxsampler.org 2010/6/16 Ville M. Vainio : > On Wed, Jun 16, 2010 at 3:57 PM, Benno Senoner > wrote: > >> I just represent a multimedia developer wanting to use the platform >> and point out the flaws in order to contribute to improve it. > > ... and that is appreciated. OTOH it's waste of typing to engage in > long diatribes about business models and whatnot here (such discussion > is more appropriate for talk.maemo.org). > > -- > Ville M. Vainio > http://tinyurl.com/vainio > ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
On Wed, Jun 16, 2010 at 3:57 PM, Benno Senoner wrote: > I just represent a multimedia developer wanting to use the platform > and point out the flaws in order to contribute to improve it. ... and that is appreciated. OTOH it's waste of typing to engage in long diatribes about business models and whatnot here (such discussion is more appropriate for talk.maemo.org). -- Ville M. Vainio http://tinyurl.com/vainio ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
regards, Benno The LinuxSampler project http://www.linuxsampler.org 2010/6/16 Ville M. Vainio : > > I think you are barking at the wrong tree - pulseaudio should get way > better latency than 5000ms, otherwise it would be useless for pretty > much everything. > > You should take this up at bugreports.qt.nokia.com => Qt => Multimedia. Yes, pulse audio should, regardless of it's bad implementation achieve better numbers than 5000msec latency but this is what I am seeing. from the user space in QAudioInput. I will fill a bug report but I want to write a testing app first to back up my claims, but it takes time do do that. I'm an engineer and I like numbers and diagrams. I already did such testing tools in the past and they helped to optimize the scheduler of the linux kernel by providing stable single digit millisecond latency for user space apps. So it seems like my "barking" helped to some extend. It's not that my main hobby is to shout and whining on developer mailing lists only because I want some feature implemented which only I need. I just represent a multimedia developer wanting to use the platform and point out the flaws in order to contribute to improve it. The fact is that the Iphone makes it transparent and easy for developers to achieve low latency audio while Nokia has still not achieved this goal, The day I can pipe audio in and out of QAudioInput/QAudioOutput with negligible delay and having it work well on both Symbian and Meego I will shut up and take all my complaints back! But at the moment the situation is quite embarassing. Since Nokia wants us to use only Qt, in order to make the code work on all their platforms, eliminating system-dependent code, they have to provide the developer with a working implementation. Otherwise as said Iphone and Android will continue to grow and grow, stealing precious market share and developer mindshare to Nokia, and how knows if those developers will come back one day. 2010/6/16 Xavier Bestel : > The typical use case is only one sound played at the same time. In that > case the sound server needs to: > - not do mixing > - use as few interrupts as possible, i.e. having the biggest acceptable > latency possible. Otherwise the sound hardware keeps waking up the > processor to fill a tiny buffer again and again. > > I don't know for the first, but I'm pretty sure JACK isn't designed to > do the last. A hardware audio driver usually exposes an buffer which is composed by fragments ( eg 2 x 1024 samples = 2048 samples buffer) and after each fragments got played an interrupt is generated and the user space app can read or write from/to that fragment while the other(s) are filled,read by the hardware. Of course the smaller the buffer size the higher the interrupt frequency and this can in turn cause higher latencies. So if power needs to be conserved then as I see it the only way is to increase the fragment size and this usually requires closing and reopening the audio device. AFAIK JACK does not do that, but it's much easier than making pulseaudio reealtime-safe, efficient and SMP capable as JACK is. download the JACK2 source code and compare it to pulseaudio, JACK is a a complex piece of software very well designed, it provides glitchfree graph changes etc, things that are far from trivial to implement. As said JACK2 in its current form, in order to be useful on mobile platforms perhaps requires some adaptions but I see no showstoppers hindering doing that. And the reward will be very big. We are very satisfied with JACK running on the Lionstracs MIDI keyboard, and the performance demands are very big in that cases. On a phone we don't need 1000 of audio voices and dozen of audio apps running. just a few apps and give the app developer a way to to achieve low audio latency hasslefree. I'm not expecting a phone doing the same as a quadcore desktop cpu etc. but given that the N900 runs at 600Mhz let's say 50-60 msec round trip audio latency in user space should be doable. Not only in the phone application but in user space apps too. If power consumption is a concern then give the app developer an API to tell it to the system. eg a mp3 player can tell the system , low latency not required, and the system returns an optimal buffersize (on the bigger side) to minimize power consumption. If an app needs to work with low audio latencies, for example a vocoder (voice changer), audio signal analyzer, virtual synth etc, then tell the system that low latency is required. At this point the system can change the audio driver's buffer size (eg by closing and opening it again) and the system continues to work as before, providing low latency but with increaased battery consumption due to higher IRQ rate etc. As soon as the app exits it increases the buffer size again. I've read a PDF about the audio on the N900 that something like this is done on Maemo but I think it is still not optimal. I have so far only used QAudioInput which uses ALSA on the N900 so I cannot
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
On Wed, Jun 16, 2010 at 1:03 PM, Ville M. Vainio wrote: > On Wed, Jun 16, 2010 at 12:56 PM, Benno Senoner > wrote: > >> If Nokia can show us that pulse audio is able to achieve >> stable,dropout-free 10-20 msec audio latencies in any audio app >> requiring it and working perfectly >> when using QAudioInput/QAudioOutput classes then I'll shut up and take >> my words back, but at the moment the situation does not seem so, and I >> fear that >> those "features" will be inherited by Meego. > > I think you are barking at the wrong tree - pulseaudio should get way > better latency than 5000ms, otherwise it would be useless for pretty > much everything. My thought exactly. If that really is the case, a proper phone conversation would had never been possible on the device, but my everyday shows it is :-) And to a more serious note, I am NO expert in sound but it feels to me that you might be wanting a bit more than what is currently expected from any other mobile handheld computing device - are there any examples of other mobile devices that allow you to achieve your goal? Research see what they use or how they tackle the problems you're facing. If you application is highly specialized you could always require things like being plugged to outlet or a car for specific power draining functionalities Why do you need multiple real time input source mixing for a voice analysis application ? Sivan ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
On Wed, Jun 16, 2010 at 12:56 PM, Benno Senoner wrote: > If Nokia can show us that pulse audio is able to achieve > stable,dropout-free 10-20 msec audio latencies in any audio app > requiring it and working perfectly > when using QAudioInput/QAudioOutput classes then I'll shut up and take > my words back, but at the moment the situation does not seem so, and I > fear that > those "features" will be inherited by Meego. I think you are barking at the wrong tree - pulseaudio should get way better latency than 5000ms, otherwise it would be useless for pretty much everything. You should take this up at bugreports.qt.nokia.com => Qt => Multimedia. -- Ville M. Vainio http://tinyurl.com/vainio ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
On Wed, 2010-06-16 at 11:56 +0200, Benno Senoner wrote: > Hi, > Can you back up your claims that pulseaudio uses less CPU than JACK ? > pulseaudio cannot do wonders, if mixing is required the code must sum > up the inputs, jack does this > already in the most efficient manner and one can easily add ARM vector > code to speed it up. The typical use case is only one sound played at the same time. In that case the sound server needs to: - not do mixing - use as few interrupts as possible, i.e. having the biggest acceptable latency possible. Otherwise the sound hardware keeps waking up the processor to fill a tiny buffer again and again. I don't know for the first, but I'm pretty sure JACK isn't designed to do the last. Xav ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
Hi, Can you back up your claims that pulseaudio uses less CPU than JACK ? pulseaudio cannot do wonders, if mixing is required the code must sum up the inputs, jack does this already in the most efficient manner and one can easily add ARM vector code to speed it up. I mean if battery is a concern then suspending the mixing of certain channels is not a problem and you save CPU. For example skype does not support jack but it supports ALSA. There is an alsa to jack module which provides a virtual ALSA device which is routed into jack. If I use skype with it, a jack input port (source) is created while the virtual ALSA device is open. Skype always closes the audio device so if you are not in a call and are only using the chat, then only while the chat beep sound is emitted the ALSA device (and thus the jack input port) is open. If you run qjackctl to visualize the connection graph, you see that the input port quickly appears and disappears after the sound has been played. And if the port is not present, JACK does not need to route and mix the audio to the physical outs, thus saving CPU. Since jack has not been deployed on battery powered devices yet, it's normal that it does not contain CPU saving power management functions. But adding those is much easier than adapting pulseaudio which was not designed with real time and multi processing in mind. Soon we will see multicore ARM CPUs on phones, pdas and tablets since it is cheaper and more power efficient than increasing the frequency of a single core. JACK can utilize all those cores giving providing smooth audio even under very high load and while many applications are started. If the manufacturers puts a fast CPU into a device, I guess the goal is to give the user a way to utilize it and exploit all its features. The big problem of digital audio is that it is a complex subject and getting things right is very hard and any initial design error will hurt you at a later stage. Finding a developer which fully understands all aspects of real time audio is hard, because you need knowledge in many fields, real time programming, priorities, multi threading, RT safe memory allocation, synchronization promitives, lock free FIFOs, DSP algorithms, etc etc. This is why I think companies don't try hard enough, they hire some random guy whose CV looks good on paper but then produces badly designed, inefficient stuff which later will plague the platform for the entire lifecycle and often it's to complex or expensive to fix the system later, so users do have to live with the flaws of the system. Millions of future users and developers don't want the same fate for Meego, so Nokia please listen and fix those issues. Of couse I'm not expecting that the N900 on Maemo should run JACK. Maemo will be supplanted with Meego and it makes no sense to push for changes on a platform which has reached it's end of line. I don't understand why big corporations like Nokia don't work in a more clean way, is benchmarking and running unit tests a foreign word ? Adopting pulseaudio which has no good track record of providing low latency as a key part of a mobile computing platform which is aimed to become the best that the market can offer is a quite irresponsible move. pulseaudio was just forced down to Linux desktop user's throats because it was adopted by Fedora and Ubuntu because it's packagers thought that it works well. But unfortunately only on paper. And Nokia blindly adopted it too (because it must be good since it was adopted by a few distros), causing lots of problems to users with it. I'm the first to admit errors or flaws or mistakes if someone points them out, but people must come up with numbers and proofs. JACK has already proven that it is the most efficient and versatile audio routing platform in existence and has been deployed in commercial products. If Nokia can show us that pulse audio is able to achieve stable,dropout-free 10-20 msec audio latencies in any audio app requiring it and working perfectly when using QAudioInput/QAudioOutput classes then I'll shut up and take my words back, but at the moment the situation does not seem so, and I fear that those "features" will be inherited by Meego. regards, Benno The LinuxSampler Project http://www.linuxsampler.org 2010/6/16 Xavier Bestel : > On Wed, 2010-06-16 at 01:27 +0200, Benno Senoner wrote: >> So my advice to Nokia, while still in time, cosider switching to JACK >> as an audio server for future versions of Meego, > > The N900 already sucks batteries like mad. JACK is a no-compromise > low-latency pro audio server, and does it at the expense of some > processor time. You don't want it on an N900, otherwise the poor battery > life would be even worse. > > Xav > > ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
On Wed, 2010-06-16 at 01:27 +0200, Benno Senoner wrote: > So my advice to Nokia, while still in time, cosider switching to JACK > as an audio server for future versions of Meego, The N900 already sucks batteries like mad. JACK is a no-compromise low-latency pro audio server, and does it at the expense of some processor time. You don't want it on an N900, otherwise the poor battery life would be even worse. Xav ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
On Wed, Jun 16, 2010 at 12:27 AM, Benno Senoner wrote: > Hi all, > I'm writing an interactive app on the N900 with PR 1.2. which captures > audio and analyzes it. Hi Benno, I think you would probably recieve better answers if you did two things: 1) split your points up into logical groupings 2) sent them to the appropriate place For instance, questions and suggestions for MeeGo architecture are probably better sent to the meego-dev mailing list (http://lists.meego.com/listinfo/meego-dev). I'm not sure who you could talk to about QAudioInput's performance, but I'd suggest either the qt-maemo-feedback mailing list (http://lists.trolltech.com/mailman/listinfo/qt-maemo-feedback) or alternatively filing a bug (http://bugreports.qt.nokia.com - they do have a Maemo5 category, and they do address issues) - though obviously, if you can do some investigation to help track down the issue, that would probably be appreciated. (It sounds like you know your way around an audio stack, so I'd presume you can help out a bit there :)) Also, please keep your audience in mind: developers - and developers talk tech best. So stick to your main points here.. e.g. when suggesting changes to architecture, I'd point out technical merits, and leave it at that. Hope this helps, > regards, > Benno > The LinuxSampler project > http://www.linuxsampler.org > ___ > maemo-developers mailing list > maemo-developers@maemo.org > https://lists.maemo.org/mailman/listinfo/maemo-developers -- Robin Burchell http://rburchell.com ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future
Hi all, I'm writing an interactive app on the N900 with PR 1.2. which captures audio and analyzes it. I used QAudioInput and while audio capture works, is has an EXTREMELY high latency, several seconds, (around 3-5secs). I tried several sample rates, 8000, 44100, 48000 Hz, tried with small buffer sizes eg 512 samples but QAudioInput always seems to set the buffer size to about 100msec. I did take a look at the source of QAudioInput and it confirmed my findings, QAudioInput::setBufferSize() is not honored and a fixed buffer size is used instead. QAudioInput is advertised to provide a low level API to talk directly to the audio device but in it's present form it's not working as it is supposed. And even if 100msec is used as the ALSA buffer size, something fishy is going on. AFAIK pulseaudio is providing a virtual ALSA device and it could cause this big delay. The same code works on Linux where the latency is as QAudioInput sets it to around 100msecs. So the problem lies in Maemo/N900. I'm one of the authors of LinuxSampler, the most widely used open source audio sampler, http://www.linuxsampler.org and we can target all common OSes and audio interfaces (we use an abstraction layer) and we consistently achieve very low latencies on any platform, including Windows, we are talking about <5-10msec latencies. So I see no reason why QAudio* cannot achieve those numbers too. we can help out with QAudio* by providing advices, bug reports and working code implementations. I think Nokia should take the audio latency issue very seriously since without low latency a range of apps cannot be implemented and it diminishes the value of Nokia's platforms and APIs as a valid multimedia system. Nokia's losing market share is caused too by things like this, lack of attention to the detail. On the Iphone, even the most stupid developer gets this audio application working with low latency, because the API does what it advertises and Apple has paid attention to these things. I'm an audio developer that programmed on many platforms and embedded systems but newer experienced something that bad like QAudioInput on the N900. So please can you point me to an API on the N900 how I could capture audio with lower latency ? since skype runs on the N900, I know it's possible, but the docs on maemo.org don't clearly say how to do it. some on maemo irc channel said I should use the gstreamer API. Any reference what the optimal parameters are ? Some reference code or pointers to an existing open source application would be welcome. Since Nokia's goal is to push Qt as the only API to use because of it's powerful features and crossplatform capabilities, they should pay attention to make it working correctly. As other professional audio developers said too (ask anyone on the linux-audio-dev list) the decision of using pulseaudio on a phone was very bad since it was not designed with real time in mind. The most advanced audio server on Linux is currently JACK, http://www.jackaudio.org/ it has been in use by professional linux audio developers for several years and IS DESIGNED with low latency in mind. there are hardware products based on jackd and linuxsampler like the Lionstracs Mediastation which achieve 5 msec latency running the software sampler, audio streaming, video players all in parallel. I contributed to that keyboard too so I know all the complexities behind the system. But the fact is that JACK's clean design made this possible. If we would have chosen to base it on pulseaudio, the product would have been unsellable. So it is puzzling why Nokia decides to continue with pulseaudio even in Meego. Meego is the successor of Maemo so it is natural to inherit features and code from Maemo, but nowhere is it written in stone that they need to continue with pulseaudio. Nokia take a look as JACK's clean design, it was refined over several iterations, jack2 was developed as a research project at a french university and is backed by research papers and many real world apps, and it's even multi processing capable by partitioning work loads while maintaining excellent low latency. The Lionstracs keyboard makes use of it and the quadcore CPU version can process an insane amount ov voices and audio data in real time. we are talking about around 1000 voices @ 24bit 44kHz So my advice to Nokia, while still in time, cosider switching to JACK as an audio server for future versions of Meego, I still not get why big corporations like Nokia don't do their research to look what's available on the market, why chose an inferiour product and customize it and put it in their devices (N900) when there is something better around that is completely free (LGPL) and is well tested and in use by professional audio users and hardware. Sorry for the rants, and sorry for sounding perhaps a bit arrogant but I just represent a developer which is frustrated that in 2010 modern hardware cannot provide the application developer with an easy way to achie
