[LAD] Re: Status of Pipewire
On Tue, 14 Feb 2023 at 16:32, Fons Adriaensen wrote: > > On Tue, Feb 14, 2023 at 03:57:05PM +0100, Wim Taymans wrote: > > > > The real difference between the two methods is 'sample count' > > > versus 'time' as the source of the event that starts a period. > > > > > > I always wondered why one would use a timer, it just amounts > > > to polling. Suppose you look every 1 ms to check if there > > > > You don't need to use polling with timerfd, just set the timeout > > according to some clock, > > add the timerfd to some poll loop and it wakes up on time. > > It's of course not 'active polling' (spending all CPU time on > testing a condition), but it is still polling in the sense > that it is NOT the event you wait for (having enough samples > to start a Jack cycle) that wakes you up. When using a timer > you just test for that condition periodically, which means > you can be up to that period late. The idea is to use a DLL to tweak the timeout and wake up exactly when you have the desired number of samples in the device. > > To avoid loss of period processing time, the timer period > must a very small fraction of the Jack period time. And > then I wonder what is the advantage. > If you want to implement dynamic latency changes with the IRQ based wakeup you need to do the opposite, use small ALSA periods and then accumulate a few until you have the desired period for the graph. It would be possible to implement something like this as an alternative for timers. > > Very much like how ALSA wakes you up when a period expires. > > AFAIK, ALSA doesn't use timers for that. > For a sound card on e.g. a PCI bus the start of cycle would > be the indirect result of an hardware interrupt. For USB > or firewire cards, it would be triggered by an event from > the lower (USB/firewire) layers. > > BTW, what about the 'signed differences' issue I pointed > out earlier ? Should be fixed with this: https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/274b63e9723ec00dd413bb64b6650d2004f7e4c2 Wim > > Ciao, > > -- > FA > ___ Linux-audio-dev mailing list -- linux-audio-dev@lists.linuxaudio.org To unsubscribe send an email to linux-audio-dev-le...@lists.linuxaudio.org
[LAD] Re: Status of Pipewire
On Fri, 10 Feb 2023 at 23:46, Fons Adriaensen wrote: > > On Thu, Feb 09, 2023 at 01:34:52PM +0100, Wim Taymans wrote: > > > real JACK is more mature and does things differently (mostly device > > wakeup with IRQ instead of timers) > > The real difference between the two methods is 'sample count' > versus 'time' as the source of the event that starts a period. > > I always wondered why one would use a timer, it just amounts > to polling. Suppose you look every 1 ms to check if there You don't need to use polling with timerfd, just set the timeout according to some clock, add the timerfd to some poll loop and it wakes up on time. Very much like how ALSA wakes you up when a period expires. Wim > are enough samples for a period. That means you can be up > to 1 ms late. Compare that to the period time of 1.33 ms > when using 64 samples / 48 kHz. Up to 3/4 of the available > time to compute a period could be lost... > > Or am I missing something ? > > Ciao, > > -- > FA > ___ Linux-audio-dev mailing list -- linux-audio-dev@lists.linuxaudio.org To unsubscribe send an email to linux-audio-dev-le...@lists.linuxaudio.org
[LAD] Re: Status of Pipewire
On Wed, 8 Feb 2023 at 12:09, Fons Adriaensen wrote: > > Hello all, > [snip] > So the ideal solution > for me would be the have Pipewire as a Jack client. > So first question: > > Q1. Is that still possible ? If not, why not ? It is possible although I have not tried this in a while. The way it works is a little bit like pulseaudio. There is a JACK device that you can activate and then it creates a jack client with N inputs and M outputs. > > If the answer is no, then all of the following become > relevant. > > Q2. Does Pipewire as a Jack replacement work, in a reliable > way [1], in real-life conditions, with tens of clients, > each maybe having up to a hundred ports ? I believe so. > > Q3. What overhead (memory, CPU) is incurred for such large > systems, compared to plain old Jack ? I have some performance measurements here: https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Performance TLDR; it's comparable, I would say maybe slightly more CPU usage than JACK when you do simple things. Probably more memory usage, didn't test that. > > A key feature of Jack is that all clients share a common idea > of what a 'period' is, including its timing. In particular > the information provided by jack_get_cycle_times(), which is > basically the state of the DLL and identical for all clients > in any particular period. Now if Pipewire allows (non-Jack) > clients with arbitrary periods (and even sample rates) PipeWire at the core uses the same period and samplerate for all clients attached to a driver, just like JACK. It can change dynamically but then it changes for everyone. When a JACK client is started, the automatic buffer size and samplerate switch is disabled but you can still force it (with settings metadata). You *can* have jack clients with different buffer size and samplerate when they are running on different drivers and when they are in no way related, it would be like starting 2 JACK servers on different cards. In the layer above (pipewire-pulse and ALSA plugin), each client gets a different ring buffer and resampler that is used to read and write to the clients with the latency and samplerate that they want. > > Q4. Where is the DLL and what does it lock to when Pipewire > is emulating Jack ? There is one DLL per driver in the graph, clients join a driver either explicitly or by linking to it directly or indirectly. If multiple drivers are joined, based on priority, one becomes master and the other uses an adaptive resampler to keep in sync (if not already sharing the same clock on the device or word clock). So normally you would have all clients look at the timings of one driver and so they share the same time. It's pretty similar to what JACK does. > > Q5. Do all Jack clients see the same (and correct) info > regarding the state of the DLL in all cases ? Yes, if they are using the same driver. > > The only way I can see this being OK would be that the Jack > emulation is not just a collection of Pipewire clients which > happen to have compatible parameters, but actually a dedicated > subsystem that operates almost independently of what the rest > of Pipewire is up to. Which in turn means that having Pipewire > as a Jack client would be the simpler (and hence preferred) > solution. PipeWire at the core is a JACK server with some more dynamic behaviour. You can in fact make a minimal server that just does what JACK does (without a session manager and pipewire-pulse server). The dynamic stuff and device detection and automatic routing (for pulse and ALSA apps) is implemented in the session manager. The adaption layer for pulseaudio clients is implemented in pipewire-pulse. The JACK emulation is a small wrapper to map the JACK methods to PipeWire IPC and methods. The only reason to run JACK clients on top of JACK instead of PipeWire is because real JACK is more mature and does things differently (mostly device wakeup with IRQ instead of timers) that makes it work better for some cases. Otherwise, there should be no difference. In any case, I should test the PipeWire as a JACK client code again and make it work well... Wim > > > [1] which means I won't fall flat on my face in front of > a customer or a concert audience because of some software > hickup. > > Ciao, > > -- > FA > > > > > ___ > Linux-audio-dev mailing list -- linux-audio-dev@lists.linuxaudio.org > To unsubscribe send an email to linux-audio-dev-le...@lists.linuxaudio.org ___ Linux-audio-dev mailing list -- linux-audio-dev@lists.linuxaudio.org To unsubscribe send an email to linux-audio-dev-le...@lists.linuxaudio.org
Re: [LAD] pipewire
On Thu, 20 Jan 2022 at 12:16, Fons Adriaensen wrote: > > Hello, Wim, > > > Sorry, git for now. I just started to implement the last bits to make a > > session manager optional. > > OK, I'll wait until this is available via Arch (don't want to mix up > two potential problems, build/install and configure...) > I'll probably make a release next week. > > > All alsa devices are wrapped in an adapter. This contains: > > > >-> channelmix -> resample -> convert -> alsa-device > > > > channelmix and resample are disabled when the graph sample rate > > and device channels all match the adapter ports, you would configure > > the same number of channels on the output as the alsa-device channels > > and set the graph rate to something the hw supports. > > In the example config, node.param.Portconfig.format.channels is > hardcoded. Is there a way to obtain the number of channels the > device supports (to make sure the values match) ? ALSA does > provide this info once the devide is opened... > I've added an option to disable the channelmixer and volume updates to it now. I've also added an option to automatically configure the device with the max amount of channels probed from the device. > What will happen if the configured number of channels does not > match ? What sort of channelmix will I get ? > It depends on the channel layout, first channels that match will be copied, then there are some simple heuristics to make channels out of the other ones (front center from stereo, copy front to rear channels) or mixing channels into other ones (rear channels into stereo). This is nothing fancy but good enough for default consumer use. > > The channelmix is mostly to support making a 5.1 sink that can downmix > > to dolby or some other tricks. > > This can be a very devious thing. I remember an occcasion some > years ago (when I was in Parma) when some students were doing > measurements in a rented anechoic room during an entire week. > Later, when the measurements were processed, they discovered > that all of them were useless because their (Windows) system > had been trying to be clever and had applied gain changes and > channel mixing without them being aware. Now work out the cost > of renting an anechoic room for 60 hours. Plus, if they hadn't > been students and 'free labour', the consultancy fees for the > same period. > > For any serious work, there are things that need to be > disabled without any chance of them ever be re-enabled by > accident. The required result when something does not match > is to fail and report, not to try and be clever and 'fix' > things behind the user's back. > > The idea of having the daemon do the 'plumbing' and the > session manager to define 'policies' is a very good one. > But to take that to its logical consequence, the defaults > for any optional processing (channel gains and mixing, > resampling,...) should be off and disabled. If any other > defaults make sense for the 'average user', they should > be defaults defined by the session manager, not by the > plumbing daemon. > I agree, I'm adding options to disable the extra automatic things by default. > > There is one feature that would be very desirable and > for which I would even be prepared to write an ad-hoc > session manager if that is the only place it can be > done: if a sound card becomes unavailable while in use, > substitute a dummy device with the same sample rate, > period, and number of channels, so the entire processing > graph remains intact and running. Then, when the device > becomes available again, allow the user to reconnect > to it (this must NOT be automatic). This is to minimise > 'down time' when someone accidentally pulls a cable > during a concert or recording (a classical orchestra > is orders of magnitude more expensive than an anechoic > room). Very doable with a little lua script in wireplumber.. > > Ciao, > > -- > FA > > > > > > > > > ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org https://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] pipewire
On Wed, 19 Jan 2022 at 14:04, Fons Adriaensen wrote: > > On Tue, Jan 18, 2022 at 07:16:39PM +0100, Wim Taymans wrote: > > > As a bare minimum you would need pipewire (the daemon) and > > pipewire-jack (the libjack.so client implementation). With a custom > > config file you can make this work exactly like jack (see below). > > Thanks, will try this, but mnay questions remain (see below). > > > All the system integration (dbus, systemd and the automatic stuff) > > happens in the session manager. You don't need to run this. > > Aha, that is good news. > > > You'll need the pipewire git version > > Will things work with the current Arch packages and do I > need git just becuase it has the minimal.conf file ? > If yes I'd prefer to use the Arch packages for now. Sorry, git for now. I just started to implement the last bits to make a session manager optional. > > Questions: > > * A lot of lines in the minimal.conf are commented. Can one > assume that these correspond to the defaults ? If not, what > are the defaults for all these parameters ? > They are defaults, yes. > What concerns me here is things like > > #channelmix.normalize = true All alsa devices are wrapped in an adapter. This contains: -> channelmix -> resample -> convert -> alsa-device channelmix and resample are disabled when the graph sample rate and device channels all match the adapter ports, you would configure the same number of channels on the output as the alsa-device channels and set the graph rate to something the hw supports. The channelmix is mostly to support making a 5.1 sink that can downmix to dolby or some other tricks. The resampler is used when slaving devices or when the graph rate doesn't match the device rate (some webcams only do 16KHz). The minimal config disables it as well. > > I certainly do not want any normalisation, so do I need > to set this to false explicitly ? > If you set the channels in node.param.PortConfig and audio.channels to the same value, there will be no channelmixing and so no normalization. > > * The sample rate (48000) is in many places, most of them > commented out. What is the relation between all of these ? > Why, for example, is 'default.clock.rate' commented ? default.clock.rate = 48000 is the default, If you change the graph clock rate, the devices will follow the rate so you can just leave them blank (the rate in node.param.PortConfig is ignored.. need to fix that..) > > * 'quantum', 'period-size' and 'period-num' are commented > out everywhere (except in 'vm.override'). So where is > the period size defined ? > The commented out values are the defaults > * If things like sample rate, period size, etc. are set > to some fixed values in the config, can they still be > modified by e.g. pw-metadata ? I hope not... > Yes, pw-metadata (but not accessible from jack apps) overrides the config settings and limits. I'll add a switch to disable this. Wim > > Ciao, > > -- > FA > ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org https://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] pipewire
On Tue, 18 Jan 2022 at 17:15, Will Godfrey wrote: > >and of course systemd. I do not think it will run without. It does run fine without systemd. > > If it *requires* systemd then that is a non-starter for me :( > It doesn't require systemd. You can compile with systemd support and then you have the capability of having systemd socket activate the server and all components. If you don't compile it in, you'll have to start things some other way. Wim ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org https://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] pipewire
On Mon, 17 Jan 2022 at 16:03, Lorenzo Sutton wrote: > > My problem with that set-up is that it seemed that something like Ardour > would need to be explicitly run via pw-jack so e.g. > > pw-jack ardour > You distro probably also has a package that puts the pipewire libjack.so in LD_LIBRARY path and then you don't have to type pw-jack anymore. > But then setting the samplerate (I have projects at different > samplerates), wasn't trivial. switch to fixed sample rate (on the fly): pw-metadata -n settings 0 clock.force-rate switch back to dynamic control pw-metadata -n settings 0 clock.force-rate 0 Same for buffersize (quantum) : pw-metadata -n settings 0 clock.force-quantum and back to dynamic: pw-metadata -n settings 0 clock.force-quantum 0 Wim ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org https://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] pipewire
Hi Fons, As a bare minimum you would need pipewire (the daemon) and pipewire-jack (the libjack.so client implementation). With a custom config file you can make this work exactly like jack (see below). The way PipeWire normally works is that when starting the daemon, nothing is in the graph. Devices are usually loaded into the graph (using udev for ALSA, bluez5 for bluetooth, avahi for network devices, ...) by a session manager. We have a basic pipewire-media-session and a more complete WirePlumber session manager. All the system integration (dbus, systemd and the automatic stuff) happens in the session manager. You don't need to run this. For pulseaudio compatibility, you can also install and run a separate server (pipewire-pulse). This basically just translates pulse protocol to pipewire protocol and makes pipewire clients for streams. You don't need to run this either. You'll need the pipewire git version to run a minimal setup with only pipewire using this config: https://gitlab.freedesktop.org/pipewire/pipewire/-/blob/master/src/daemon/minimal.conf.in You can change the source and sink ALSA devices, set up the channels and format and then use pipewire -c It should be enough to run all JACK apps to test things. pw-top is interesting, pw-profiler works like the jack2 profiler, pw-dump for a JSON dump of the object tree, pw-link (like jack_lsp, jack_connect), etc... Wim On Mon, 17 Jan 2022 at 14:56, Fons Adriaensen wrote: > > Hello all, > > I'd like to test pipewire as a replacement for Jack (on Arch), > and have been reading most (I think) of the available docs. > > What is clear is that I will need to install the pipewire > and pipewire-jack packages. > > And then ? > > How do I tell pipewire to use e.g. hw:3,0 and make all of > its 64 channels appear as capture/playback ports in qjackctl ? > > Note: I do not have anything PulseAudio (like pavucontrol) > installed and don't want to either. If that would be a > requirement then I'll just forget about using pipewire. > > TIA, > > -- > FA > > ___ > Linux-audio-dev mailing list > Linux-audio-dev@lists.linuxaudio.org > https://lists.linuxaudio.org/listinfo/linux-audio-dev ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org https://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] Simple Pipewire test request
You need to make sure the cron script can find the local socket to talk to pipewire. It searches for the socket named pipewire-0 in: $PIPEWIRE_RUNTIME_DIR $XDG_RUNTIME_DIR $HOME ~/ I'm guessing that the cron script is running with a different user and/or does not have the same environment variables as the main pipewire daemon and thus looks in a different place and can't find a socket. Normally the user running the daemon will set XDG_RUNTIME_DIR=/run/user/1000 and the socket will be named /run/user/1000/pipewire-0 Substitute 1000 for the user id. Try this in the cron script: PIPEWIRE_RUNTIME_DIR=/run/user/1000 pw-record An try again. I'm pretty sure it's a socket path thing. There is no dbus, systemd or device permissions involved. You need to check if the cronjob can find and has permissions to connect to the socket and that's it. Wim On Wed, 7 Jul 2021 at 17:47, John Murphy wrote: > > Could someone please do a simple test on any Pipewire installation > set up to use/replace Jack (or shed any light on this). > > If I run 'pw-record crontest.wav' it works fine, as expected. > If I run 'pw-cat 2> er.txt' ditto, of course. > > If I setup a user cronjob with 'crontab -e' to run the commands > (at the start of the next minute or so) the first does nothing, > while the second works. > > Cron writes to /var/log/syslog like: > > CRON[7694]: (john) CMD (pw-record crontest.wav) > rtkit-daemon[1277]: Supervising 7 threads of 4 processes of 1 users > > (second line repeated) > > -- > John (older than 95.6% of the population). > ___ > Linux-audio-dev mailing list > Linux-audio-dev@lists.linuxaudio.org > https://lists.linuxaudio.org/listinfo/linux-audio-dev ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org https://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] Is Piperware a successor to Jack/Pulseaudio?
On Wed, 7 Jul 2021, 22:55 Fons Adriaensen, wrote: > On Wed, Jul 07, 2021 at 10:44:23PM +0200, Fons Adriaensen wrote: > > > With jack2 this takes 0.5s to create the clients, and on average 0.1s to > > connect all of them in a chain (15 * 32 connect calls). > > Correction: after removing some print() statements the total connections > time was reduced to 50 ms on average (jack2 of course). > > My script spawns jack_connect 480 times, It's probably going to be faster with an application... I'll try to get some measurements of that later. Wim > Ciao, > > -- > FA > ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org https://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] Is Piperware a successor to Jack/Pulseaudio?
On Wed, 7 Jul 2021 at 14:05, Filipe Coelho wrote: > > On 07/07/21 12:37, Robin Gareus wrote: > > On 7/7/21 1:00 PM, Wim Taymans wrote: > >> This utterly fails with jackd on this system, it doesn't even want > >> to start all the clients, I'm sure it's something with the config > >> somewhere... > > jack has a port-limit (IIRC 256 by default). It is not dynamic and > > unbound for performance reasons. > I got the same script working on JACK2 (minus the youtube video), I needed to increase mlock limits. JackLinuxFutex uses MAP_LOCKED, which can fail when there is too much locked already. JACK2 takes about the same amount of time to create the clients and set up the links and uses about the same amount of DSP (-+25%) as PipeWire. Wim ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org https://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] Is Piperware a successor to Jack/Pulseaudio?
On Tue, 6 Jul 2021 at 21:41, Fons Adriaensen wrote: > > > I'll give PW its chance when the developers tell me it's ready for > real life. Which will mean a session with around 15 jack clients > with a total of 800 or so ports. Should run without hickups while > watching a youtube movie and compiling a kernel at the same time > (which I can do now without any problem). Challenge accepted!... I made a little jack client with 32 input and 32 output ports that memcpy the samples. Then I started 16 of those and linked them all in a long chain. Then I linked the input of the chain to a USB mic and the output to another USB card (it needs to do adaptive resampling to keep this going), That takes about 6 seconds to setup on my machine. I run this with a buffer size of 128 samples and 48KHz. Then I started firefox and loaded a video. Then I also started compiling all of GStreamer on all cores. Here is the screenshot: https://people.freedesktop.org/~wtay/pw-load.png Works okish, some xruns here and there and this is a stock fedora setup with extra rtprio for the user. No low latency kernel or any tuning. I had to increase the max fds to 8192. I'm sure you can eliminate more xruns with some tuning. This utterly fails with jackd on this system, it doesn't even want to start all the clients, I'm sure it's something with the config somewhere... This is probably not a representative setup but at 16+ clients and 1024+ ports we're ballpark.. It probably starts to fail more with some real processing. While testing I found a scalability bug in the feedback loop detection, which should be fixed now. It might explain startup delays with complex projects... Wim > > Ciao, > > -- > FA > > > ___ > Linux-audio-dev mailing list > Linux-audio-dev@lists.linuxaudio.org > https://lists.linuxaudio.org/listinfo/linux-audio-dev ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org https://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] PipeWire
On Mon, 20 Aug 2018 at 01:41, Robin Gareus wrote: > > On 02/19/2018 09:39 AM, Wim Taymans wrote: > [...] > > I would very much like to hear your ideas, comments, flames, thoughts on > > this > > idea. I think I'm at a stage where I can present this to a bigger audience > > and > > have enough experience with the matter to have meaningful discussions. > > Hi Wim, Hi Robin, Thanks for taking time to reply. > > I think the general lack of enthusiasm about pipewire here is because it > does not solve any issues for linux-audio and at best does not > introduces new ones. > > In the past years the most prominent question that I have received is > > * How can I use all of my USB Mics with Ardour on Linux? > * How do I uniquely identify my many MIDI devices? > * Why does my audio device not have proper port-names? > * Why can't I re-connect my device and resume work? > > These questions are mostly from Mac or Windows users moving to Linux ... > and many of them moving back to MacOS. > > If you try to come up with a new system (think pipewire), please copy as > many concepts as possible from Mac's CoreAudio. > I heard this before. Device management on linux is still pretty bad. I have not seriously looked at how to solve any of this yet.. > > While it is not impossible to combine multiple devices, it is not a > straightforward to set this up. Manging devices uniquely and handling > temporarily missing devices is not possible on GNU/Linux AFAIK. One of the ideas with PipeWire is to move much of the logic to set up devices and filters to another process. We would like to have the desktops come up with policies for what to connect when and where and implement those. This would also make it possible to do more configuration in the desktop control panels, like rank devices (to select a master device), setup filters for surround sound, bass boost, echo cancellation (the things you can configure in Windows and MacOs). I know there a problems with uniquely identifying devices in Linux that may not make this 100% perfect but we should be able to get to the same state as MacOs or Windows. The logic for combining devices exists and works well (zita-a2j/j2a) I would like to have built-in support for this in PipeWire as soon as 2 devices interact. MacOS has a panel for combining devices, we need something like that too. > > Both pulseaudio and jack had the correct idea to present audio as a > service to applications. The server is concerned with device(s) and > device settings. However, both fail to abstract multiple devices, map > their port uniquely and provide multiple apps to concurrently use those > devices for different purposes. Are you talking about JACK? PulseAudio pretty much has this right, no? > > The main issue with pulse is that it is a poll API. Also pulseaudio's > per device, per port-latency is incorrect (if set at all). What wrong with a poll API? To me PulseAudio has more of an event based API.. Not sure what you mean with the latency being reported incorrectly, latency is dynamic and you can query it, it pretty much gives you access to the read and write pointers of the device.. > JACK on the > other hand is too limited: single device, fixed buffersize. jackd also > periodically wakes ups the CPU and uses power (even if no client is > connected). These are the main points for objecting to JACK as a generic desktop replacement for audio and PulseAudio takes the complete opposite approach. To me, the ideal solution would be to keep the JACK design and remove the above mentioned limitations. > > Browsing around in the pipewire source I see several potential design > issues. > > In particular data format conversions: The nice part about JACK is that > uses float as only native format. Also port-memory is shared between > application with zero-copy. I 100% agree, arbitrary format conversions are not practical. In PipeWire, there are 2 scenarios: 1) exclusive access to a device. You can negotiate any format and buffer layout directly with the device. Very handy for compressed formats, or to get the maximum performance (games). Of course only one app can use the device but this can be allowed in certain cases. 2) non-exclusive access. A dsp module (exclusively) connects to the device that converts to and from the canonical format (float32 mono) and the device format. Clients then either connect with the canonical format (jack clients) or can use the stream API (like CoreAudio's AudioQueue) to play or record data with conversions being handled automatically. So only conversions at the entry and exit points of the graph, everything in between is float32. Port memory in PipeWire is also shared between applications and is pretty much a requirement to do anything related to video. Where JACK has 1 buffer per port allocate
[LAD] PipeWire
Hi everyone, I'm Wim Taymans and I'm working on a new project called PipeWire you might have heard about [1]. I have given some general presentations about it during its various stages of development, some of which are online [2]. PipeWire started as a way to share arbirary multimedia, wich requires vastly different requirements regarding format support, device and memory management than JACK. It wasn't until I started experimenting with audio processing that the design started to gravitate to JACK. And then some of JACKs features became a requirement for PipeWire. The end goal of PipeWire is to interconnect applications and devices through a shared graph in a secure and efficient way. Some of the first applications will be wayland screen sharing and camera sharing with access control for sandboxed applications. It would be great if we could also use this to connect audio apps and devices, possibly unifying the pulseaudio/JACK audio stack. Because the general design is, what I think, now very similar to JACK, many people have been asking me if I'm collaborating with the linux pro-audio community on this in any way at all. I have not but I really want to change that. In this mail I hope to start a conversation about what I'm doing and I hope to get some help and experience from the broader professional audio developers community on how we can make this into something useful for everybody. I've been looking hard at all the things that are out there, including Wayland, JACK, LV2, CRAS, GStreamer, MFT, OMX,.. and have been trying to combine the best ideas of these projects into PipeWire. A new plugin API was designed for hard realtime processing of any media type. PipeWire is LGPL licensed and depends only on a standard c library. It's currently targeting Linux. At the core of the PipeWire design is a graph of processing nodes with arbirary input/output ports. Before processing begins, ports need to be configured with a format and a set of buffers for the data. Buffer data and metadata generally lives in memfd shared memory but can also be dmabuf or anything that can be passed as an fd between processes. There is a lot of flexibility in doing this setup, reusing much of the GStreamer experience there is. This all happens on the main thread, infrequently, not very important for the actual execution of the graph. In the realtime thread (PipeWire currently has 1 main thread and 1 realtime data thread), events from various sources can start push/pull operations in the graph. For the purpose of this mail, the audio sink uses a timerfd to wake up when the alsa buffer fill level is below a threshold. This causes the sink to fetch a buffer from its input port queue and copy it to the alsa ringbuffer. It then issues a pull to fetch more data from all linked peer nodes for which there is nothing queued. These peers will then eventually push another buffer in the sink queue to be picked up in the next pull cycle of the sink. This is somewhat similar to the JACK async scheduling model. In the generic case, PipeWire has to walk upstream in the graph until it finds a node that can produce something (see below how this can be optimized). Scheduling of nodes is, contrary to JACKs (and LADSPA and LV2) single 'process' method, done with 2 methods: process_input and process_ouput. This is done to support more complex plugins that need to decouple input from output and to also support a pull model for plugins. For internal clients, we directly call the methods, for external clients we use an eventfd and a shared ringbuffer to send the right process command to the client. When the external client has finished processing or need to pull, it signals PipeWire, which then wakes up the next clients if needed. This is different from JACK, where a client directly wakes up the peers to avoid a server context switch. JACK can do this because the graph and all client semaphores are shared. PipeWire can't in general for a couple of reaons: 1) you need to bring mixing of arbitrary formats to the clients 2) sandboxed clients should not be trusted with this information and responsability. In some cases it would probably be possible to improve that in the future (see below). This kind of scheduling works well for generic desktop style audio and video. Apps can send buffers of the size of their liking. Bigger buffers means higher latency but less frequent wakeups. The sink wakeup frequency is determined by the smallest buffer size that needs to be mixed. There is an upper limit for the largest amount of data that is mixed in one go to avoid having to do rewinds in alsa and still have reasonable latency when doing volume changes or adding new streams etc. The idea is to make a separate part of the graph dedicated to pro-audio. This part of the graph runs with mono 32bit float sample buffers of a fixed size and samplerate. The nodes running in this part of the graph also need to have a fixed input-output pattern. In this part
