Heya! At FOSDEM in Brussels this WE the GStreamer folks and I sat down to discuss how to improve interfacing between GStreamer and PulseAudio. Here's a short, rough summary of that meeting:
First topic: zero-copy/latency/... (This is a bit vague, I know) Currently writing sinks for GStreamer kind of forces you to make use of a GStRingBuffer, for communicating between the pipeline and the IO thread. For the PulseAudio sink we'd like to get rid of that, because that basically doubles latency, comes at the cost of one unnecessary context switch and some amount of needless memcpy()s. Given that Gstreamer elements can support allocation functions where elements connected to them can place data, we should implement this for PA to allow Gst to directly write into PA's shared memory segment. This needs some changes to the PA client librariers however since this kind of "zero-copy" writing is not exposed in the API. Hence: on the gst side i'd like to see a more "fine-grained" API that allows the pulse sink to bypass the GstRingBuffer entirely if the playback speed is 1, and only have it in line for the other cases. Also, the buffer should generally only filled on request, not preemptively. PA provides a function pa_stream_writable_size() that returns how much data is needed by PA and only that much data should be generated. To formalize this a bit: The PA API shall provide five functions (besides initialization/teardown): 1. a function that allows GstPulseSink to allocate arbitrary memory from the client's shared memory area. 2. a function to free/unref memory allocated like this 3. a write functin that takes memory like this plus a seek index where to write it to. 4. a function for querying how much data PA needs (pa_stream_writable_size() -- already available) 5. a way for notificating GstPulseSink when more data is needed, i.e. when pa_stream_writable_size() changes. (already available) Second topic: Volume interface We need a nice way to allow per-stream volume adjustement in GstPulseSink. Right now we have a 'volume' property for that which some folks implement. This should become part of an official interface that includes both a mute and a volume field plus some kind of notification mechanism for changes of these fields triggered from the server. Third Topic: Pause/Resume request mechanism PulseAudio would like to be able to ask applications to pause/resume their pipelines in certain situations. Background: audio players should pause when a voip call comes in. All that is missing is defining two new standard messages on GstBus. Fourth Topic: Tagging application streams To present streams nicely for the user in volume control UIs and for policy decisions PA relies on 'properties' set for client streams. Some of these properties can be deduced automatically, some cannot and need to be set explicitly by an application. The most important property here is the "role", i.e. a property that classifies a stream as "music", "movie", "phone call" and so on. We probably need an interface that allows setting this for a sink. How to do this in detail has not been discussed. The properties PA knows right now are listed here: http://0pointer.de/lennart/projects/pulseaudio/doxygen/proplist_8h.html The properties starting with "application." should probably not be configured explicitly for GStreamer but deduced from the process environment or additional GLib facilities. (The properties starting with "event." and "device." are irrelevant in this context). Leaves "media." and "window.". name/title/artist can be deduced automatically from the id3-style tag data. However as mentioned media.role needs to be explicitly set by the application. It would be very valuable to attach information about the window that a streams belongs to to the stream. Big question is how to do that cleanly. Given that Gtk doesn't want to depend on Gstreamer and Gstreamer not on Gtk this is not easy to solve... The window issue hasn't been discussed. Fifth Topic: Better format negotiation Right now GstPulseSink/GstPulseSrc have a static set of capabilities. This is suboptimal. We need a more elaborate way of doing negotiation since sometimes (i.e. AC3) the codec knows better how to upmix/downmix than PA. We discussed that PA should allow dynamic reconfiguration of the sample spec of a stream. However, I am not really convinced anymore that this is a good idea. Main reason is that the sample spec might be relevant for policy decisions (i.e. choose a different sound card when 5.1 is requested than when stereo is requested.). I need to think about this quite a bit more. Sixth Topic: GstPulseSrc Timing issues Apparently GstPulseSrc's timing is not that reliable. Needs to be debugged in more detail. So, thtat's mostly what we discussed. Now all that's missing is that someone actually starts coding on this ;-) Hope I didn't forget anything. Lennart -- Lennart Poettering Red Hat, Inc. lennart [at] poettering [dot] net ICQ# 11060553 http://0pointer.net/lennart/ GnuPG 0x1A015CC4 _______________________________________________ pulseaudio-discuss mailing list pulseaudio-discuss@mail.0pointer.de https://tango.0pointer.de/mailman/listinfo/pulseaudio-discuss
