Re: Wine sound discussion summary
Maarten Lankhorst wrote: Hi Reif, 2009/12/8 Robert Reif r...@earthlink.net: Francois Gouget wrote: Also, as far as I know, DirectSound works on top of all our backend drivers. It works through the WAVE API on most drivers which requires software mixing and format conversions. Even the direct sound drivers only implement a single hardware buffer which means that even direct sound goes through the software mixer and format conversions. GASP, that's not such a problem that you make it out to be. Hell, winealsa even emulates a ring buffer with read calls, see f27d88e16fe.. Yes, a single ring buffer for all the software mixed direct sound buffers. A good driver implementation would have a ring buffer for every direct sound buffer. Therefore no requirement for any software mixing. That path is in the direct sound dll now but is not used because no driver supports it. In fact, when you rewrite the dll, you should be removing all existing code paths excepts this one. If any direct sound driver implemented multiple buffers of any format, there would be no software mixing done in the direct sound dll. Everything would just pass through the direct sound dll directly to the driver untouched. We would also get multiple open support since it wouldn't matter which application the buffers came from. The dsound timer would still tick and most of the time the app would still use the crappy remixer in dsound since games use DSBCAPS_LOCSOFTWARE these days.. Even more fundamentally, our winmm drivers are crap, full of literally copied versions of wineoss, just renamed and sedded, but never maintained. Spot the similar #if 0's... The wavein/out drivers will be thrown out after we can forward to WASAPI, since the midi code is still handled in winmm afaict, a few of them will continue to live for that purpose.. Can I please have some new discussion point instead of you trying to bring up the same over and over? I'm growing tired of having to repeat myself so much.. I know this discussion is academic at this point but sound in wine is poor not because of a bad model but because of real poor and incomplete driver implementations. With the right driver, the current model for pre vista applications would have had no issues. This is the same problem Microsoft had and is one of the reasons they changed their audio system model. With the introduction of vista, a new model is required but just like vista, audio capabilities will be significantly reduced and backwards compatibility will become a big issue requiring workarounds. Also putting in new incomplete drivers that don't support everything required will just be repeating the same mistakes we already made. I don't want to see a new implementation start out with the same problems that the old system had because people didn't learn any lessons. We are already talking about hacks to fix things before the new implementation is even fully conceptualized. Not a good start.
Re: Wine sound discussion summary
Am 08.12.2009 um 13:06 schrieb Robert Reif: Yes, a single ring buffer for all the software mixed direct sound buffers. A good driver implementation would have a ring buffer for every direct sound buffer. Therefore no requirement for any software mixing. That path is in the direct sound dll now but is not used because no driver supports it. In fact, when you rewrite the dll, you should be removing all existing code paths excepts this one. Maarten please correct me if I am wrong, but I think the reason why the Alsa driver does its own mixing is because Alsa does sample rate and similar conversions only when NOT using the mmap API. So you either get Alsa-side mixing or fast access using mmap. Back it the days it was considered that mixing ourselves and then using mmap was faster than not using mmap and then let Alsa mix things(which happens in Software as well in most cases)
Re: Wine sound discussion summary
Hi Stefan, Stefan Dösinger schreef: Am 08.12.2009 um 13:06 schrieb Robert Reif: Yes, a single ring buffer for all the software mixed direct sound buffers. A good driver implementation would have a ring buffer for every direct sound buffer. Therefore no requirement for any software mixing. That path is in the direct sound dll now but is not used because no driver supports it. In fact, when you rewrite the dll, you should be removing all existing code paths excepts this one. Maarten please correct me if I am wrong, but I think the reason why the Alsa driver does its own mixing is because Alsa does sample rate and similar conversions only when NOT using the mmap API. So you either get Alsa-side mixing or fast access using mmap. Back it the days it was considered that mixing ourselves and then using mmap was faster than not using mmap and then let Alsa mix things(which happens in Software as well in most cases) Wrong :) Alsa doesn't allow you to specify buffer sizes, or granularity. Furthermore rate resampling in alsa results in horrible things happening, you cannot set a alsa buffer in looping mode reliably, and if you could you cannot randomly access its memory, furthermore buffer notications don't work, and you cannot set per stream volume.. I probably missed a few other reasons but those are the biggest ones. Cheers, Maarten.
Re: Wine sound discussion summary
On Sun, 6 Dec 2009, Reece Dunn wrote: [...] If Wine is going to go down the Windows 7 audio route, why not redirect everything to pulseaudio? Is PulseAudio supported on Mac OS X, FreeBSD and Solaris? If not then we'd need an alternative for these platforms. That's also something that worries me a bit about OpenAL. It's meant to be portable but I wonder how well it's supported and integrated with each of these platforms. Note that this worry is in part caused by ignorance, that is I don't know that it's widely used and works well on all these platforms. -- Francois Gouget fgou...@free.fr http://fgouget.free.fr/ Stolen from an Internet user: f u cn rd ths, u cn gt a gd jb n cmptr prgrmmng !
Re: Wine sound discussion summary
On Mon, Dec 7, 2009 at 11:53 AM, Francois Gouget fgou...@free.fr wrote: On Sun, 6 Dec 2009, Reece Dunn wrote: [...] If Wine is going to go down the Windows 7 audio route, why not redirect everything to pulseaudio? Is PulseAudio supported on Mac OS X, FreeBSD and Solaris? If not then we'd need an alternative for these platforms. PulseAudio is around on FreeBSD and Solaris but not on OSX. That's also something that worries me a bit about OpenAL. It's meant to be portable but I wonder how well it's supported and integrated with each of these platforms. Note that this worry is in part caused by ignorance, that is I don't know that it's widely used and works well on all these platforms. OpenAL was designed by Loki and Creative as a cross-platform API. Since the fall of Loki Creative has continued it and also Apple has added it to OSX. The original opensource implementation which was used in Linux games didn't receive any (or else little) updates from Creative. In the end Chris has cleaned up the last Windows version of OpenAL and maintains it as 'OpenAL soft'. Officially Creative leads OpenAL but they don't seem to disclose specifications of improvements and extensions they have added. They don't seem to bother keeping all implementations in sync. Chris has added a lot of improvements to OpenAL soft and has defined new extensions. I have no idea to what degree other implemenations of OpenAL will pick up these changes and what the quality of the (closed) OSX branch is. It would be good if OpenAL became a real cross-platform standard again (for instance coordinated by Khronos who maintain OpenGL and other standards). Roderick
Re: Wine sound discussion summary
2009/12/7 Francois Gouget fgou...@free.fr On Sun, 6 Dec 2009, Reece Dunn wrote: [...] If Wine is going to go down the Windows 7 audio route, why not redirect everything to pulseaudio? Is PulseAudio supported on Mac OS X, FreeBSD and Solaris? If not then we'd need an alternative for these platforms. http://www.pulseaudio.org/wiki/AboutPulseAudio#SupportedOperatingSystems: * Linux (any modern distribution) * Solaris * FreeBSD * NetBSD * Native Win32 (no cygwin) They do have Mac support in progress (see the mailing list -- pa_fdsem racy?), but there are some issues at the moment. That's also something that worries me a bit about OpenAL. It's meant to be portable but I wonder how well it's supported and integrated with each of these platforms. Note that this worry is in part caused by ignorance, that is I don't know that it's widely used and works well on all these platforms. Stefan said that there were some issues with the OpenAL support on Mac, and that there are some bugs on that platform (as well as bugs in the Linux driver). However, there are bugs and issues whichever API is chosen (alsa, oss, pulseaudio, openal, ...). The way I see it, each choice has its own advantages: alsa/oss/coreaudio -- midi support; direct access to audio driver jack -- low-latency performance needed for wavert interfaces openal -- 3d sound effects (dsound 3d); flexible backend pulseaudio -- user-space audio mixing and application-based volume configuration; flexible backend - Reece
Re: Wine sound discussion summary
On Mon, 7 Dec 2009, Reece Dunn wrote: [...] Is PulseAudio supported on Mac OS X, FreeBSD and Solaris? If not then we'd need an alternative for these platforms. http://www.pulseaudio.org/wiki/AboutPulseAudio#SupportedOperatingSystems: * Linux (any modern distribution) * Solaris * FreeBSD * NetBSD * Native Win32 (no cygwin) Actually I did not find any libpulse or libopenal library on my Solaris 10u5 or Solaris 10 Companion DVDs. So they are both non-standard on this platform, meaning that unless Wine ships with them, then users of these platforms would have no sound. Concerning FreeBSD 7.0, neither are available as a package but both are available as a port. So FreeBSD users are unlikely to have them preinstalled but at least they have a reasonably easy and _standard_ way of installing them. -- Francois Gouget fgou...@free.fr http://fgouget.free.fr/ Linux: the choice of a GNU generation
Re: Wine sound discussion summary
On Sun, 6 Dec 2009, Stefan Dösinger wrote:
[...]
So you mean keeping the existing infrastructure around just for midi
and have something entirely different for the rest of the audio
features?
That might nt be too bad in that we could still drop all the current
current winmm audio backends that don't support Midi. At first glance:
wineaudioio, wineesd, winejack, and winenas. So we'd just keep the Midi
portion of winealsa, winecoreaudio and wineoss.
--
Francois Gouget fgou...@free.fr http://fgouget.free.fr/
$live{free} || die ;
Re: Wine sound discussion summary
Reece Dunn wrote: 2009/12/7 Francois Gouget fgou...@free.fr On Sun, 6 Dec 2009, Reece Dunn wrote: [...] If Wine is going to go down the Windows 7 audio route, why not redirect everything to pulseaudio? Is PulseAudio supported on Mac OS X, FreeBSD and Solaris? If not then we'd need an alternative for these platforms. http://www.pulseaudio.org/wiki/AboutPulseAudio#SupportedOperatingSystems: * Linux (any modern distribution) * Solaris * FreeBSD * NetBSD * Native Win32 (no cygwin) ROFL. A stone age version of PA was ported half-a^Hhearted to the other OSes just to have a tick in the portability checkbox. Doesn't mean current versions work there or are actively developed. bye michael
Re: Wine sound discussion summary
Stefan Dösinger wrote: A related topic: How do Joysticks connected to the gameport work(which is the same hardware connector as MIDI). I think that a joystick already takes an entirely different path in the Linux kernel and it doesn't get anywhere near a sound system, so we don't have to bother about it. Is that correct? Well, if you're interested... no, the gameport is *not* the same hardware connector as MIDI. A MIDI cable has a 5-pin circular DIN connector. A game port has a 15-pin D-sub connector. They're not the same thing. However, some of the pins of the original 15-pin game port were redundant. Early soundcard manufacturers decided that they could repurpose those pins, and let MIDI users use a breakout cable (from the soundcard's custom 15-pin connector to a standard 15-pin gameport connector and two 5-pin MIDI connectors). It was cheap, and a full-size MIDI connector probably wouldn't fit on a regular soundcard without using breakout cables anyway. And not many needed it. When gameports were provided by soundcards, then usually, the joystick interface was provided by the soundcard's drivers - although it could be in a different kernel module. After all, the joystick and the MIDI was driven by separate pieces of hardware, they just got multiplexed onto the same 15-pin connector through appropriate circuitry, with the assumption that the breakout cable would separate the signals again. By the time this stuff got integrated into motherboards, the multiplexed 15-pin gameport had become a standard. Yet still, two separate functional units drive the two separate sets of pins on the game port, so the kernel does see two separate hardware interfaces, controlled by separate pieces of software. So while I'm not sure you've gotten the facts completely straight, it's correct that you don't have to worry about it. And I suspect you will need ALSA to do real MIDI... I guess it's like in Windows, where wave APIs are redesigned, but if you want to do MIDI, you still need to use the exact same multimedia interface as in Windows 3.x. Just as in Linux, you can do wave with OpenAL and pulse, but for MIDI, you still need to use OSS/ALSA. I don't necessarily consider that a problem, actually. (If you play MIDI through ALSA and route it into a softsynth like timidity, then timidity could still be routed into pulseaudio, etc...)
Re: Wine sound discussion summary
On Dec 6, 2009, at 10:08 AM, Stefan Dösinger wrote: Am 06.12.2009 um 15:32 schrieb Roderick Colenbrander: This audio engine is quite similar to pulseaudio and it offers functionality like per stream volume which normal APIs like oss/alsa/openal don't offer, so these APIs would map better to a sound server than to a standard library. I am wondering: Does CoreAudio on OSX offer similar features? Or do we need a sound server there as well if we want to support them? I'm not particularly familiar with any audio API other than Core Audio, so I'm not sure how comparable the feature sets are, but per-stream volume and such are available. Core Audio is not particularly close to the hardware. I think it's roughly similar to a sound server type of architecture. There's no problem with multiple apps producing audio, or anything like that. It's pretty virtual/abstracted. On Dec 7, 2009, at 5:15 AM, Roderick Colenbrander wrote: I have no idea to what degree other implemenations of OpenAL will pick up these changes and what the quality of the (closed) OSX branch is. What makes you say the OS X branch is closed? svn://connect.creativelabs.com/OpenAL/trunk/OpenAL-MacOSX Apple has links at http://www.opensource.apple.com/ to the OpenAL that ships with each version of Mac OS X, although Creative seems to have reorganized their site and Apple's links are dead. Still, the above SVN URL should be current. -Ken
Re: Wine sound discussion summary
On Mon, Dec 7, 2009 at 11:00 PM, Ken Thomases k...@codeweavers.com wrote: On Dec 6, 2009, at 10:08 AM, Stefan Dösinger wrote: Am 06.12.2009 um 15:32 schrieb Roderick Colenbrander: This audio engine is quite similar to pulseaudio and it offers functionality like per stream volume which normal APIs like oss/alsa/openal don't offer, so these APIs would map better to a sound server than to a standard library. I am wondering: Does CoreAudio on OSX offer similar features? Or do we need a sound server there as well if we want to support them? I'm not particularly familiar with any audio API other than Core Audio, so I'm not sure how comparable the feature sets are, but per-stream volume and such are available. Core Audio is not particularly close to the hardware. I think it's roughly similar to a sound server type of architecture. There's no problem with multiple apps producing audio, or anything like that. It's pretty virtual/abstracted. On Dec 7, 2009, at 5:15 AM, Roderick Colenbrander wrote: I have no idea to what degree other implemenations of OpenAL will pick up these changes and what the quality of the (closed) OSX branch is. What makes you say the OS X branch is closed? svn://connect.creativelabs.com/OpenAL/trunk/OpenAL-MacOSX Apple has links at http://www.opensource.apple.com/ to the OpenAL that ships with each version of Mac OS X, although Creative seems to have reorganized their site and Apple's links are dead. Still, the above SVN URL should be current. -Ken The svn indeed contains their current code. Although it seems to be from 2006 and it looks like it had no recent activity. What worries me is that this is really a different OpenAL implementation (it is all c++ and shares no code with the other libs). I'm worried about the future of OpenAL on OSX though. Roderick
Re: Wine sound discussion summary
Francois Gouget wrote: On Sun, 6 Dec 2009, Stefan Dösinger wrote: [...] So you mean keeping the existing infrastructure around just for midi and have something entirely different for the rest of the audio features? That might nt be too bad in that we could still drop all the current current winmm audio backends that don't support Midi. At first glance: wineaudioio, wineesd, winejack, and winenas. So we'd just keep the Midi portion of winealsa, winecoreaudio and wineoss. Can we drop all current audio drivers now that don't have midi? Any driver that doesn't implement the complete winmm API and direct sound shouldn't be in the current tree anyway.
Re: Wine sound discussion summary
Hello, 2009/12/8 Robert Reif r...@earthlink.net: ... Can we drop all current audio drivers now that don't have midi? Any driver that doesn't implement the complete winmm API and direct sound shouldn't be in the current tree anyway. I wish, but aj wouldn't even let me delete winenas.drv that has been obviously broken for many years, even after I pointed out it even failed 'play test sound' in winecfg after you set it up. I guess it has to wait until after wine7audio hits. The lack of an accelerated dsound interface isn't that big, just means 1 more copy of the primary buffer kept around, wine dsound should handle it correctly, tested on windows some time back too. Our dsound won't run on windows though, mingw gets horribly confused by directsoundcrea...@24 and directsoundcre...@24, and no matter what I tried, either dynamic linking or static linking would fail (or both..) Cheers, Maaten.
Re: Wine sound discussion summary
Maarten Lankhorst wrote: The lack of an accelerated dsound interface isn't that big, just means I wish accelerated direct sound had been a big issue. Both OSS and alsa theoretically could support it under very limited circumstances but there was no guarantee that it could be supported under all circumstances. With full direct sound hardware acceleration in the low level driver, all software mixing would be bypassed in the current direct sound implementation. Direct sound would just forward calls from the application to the driver. That's where the direct in direct sound originally came from in the old days.
Re: Wine sound discussion summary
On Mon, 7 Dec 2009, Robert Reif wrote: [...] Can we drop all current audio drivers now that don't have midi? Any driver that doesn't implement the complete winmm API and direct sound shouldn't be in the current tree anyway. I don't see why drivers that don't support MIDI should be dropped. Lack of MIDI support does not hamper games, media players and email arrival notifiers. So a driver without MIDI support is still useful for a good 90% of the applications out there. As for not implementing the 'complete winmm API' I'm not sure what you mean. Also, as far as I know, DirectSound works on top of all our backend drivers. -- Francois Gouget fgou...@free.fr http://fgouget.free.fr/ War doesn't determine who's right. War determines who's left.
Re: Wine sound discussion summary
Francois Gouget wrote: Also, as far as I know, DirectSound works on top of all our backend drivers. It works through the WAVE API on most drivers which requires software mixing and format conversions. Even the direct sound drivers only implement a single hardware buffer which means that even direct sound goes through the software mixer and format conversions. If any direct sound driver implemented multiple buffers of any format, there would be no software mixing done in the direct sound dll. Everything would just pass through the direct sound dll directly to the driver untouched. We would also get multiple open support since it wouldn't matter which application the buffers came from. OpenAL and pulseaudio probably have the capabilities to implement a direct sound driver that supports multiple opens and mixing of multiple streams of different formats which would bypass the software mixing and format conversion path in the current direct sound dll implementation but I guess we will never know. Unfortunately OSS and alsa don't except under very limited conditions (if the drivers implemented it which they don't).
Re: Wine sound discussion summary
Hi Reif, 2009/12/8 Robert Reif r...@earthlink.net: Francois Gouget wrote: Also, as far as I know, DirectSound works on top of all our backend drivers. It works through the WAVE API on most drivers which requires software mixing and format conversions. Even the direct sound drivers only implement a single hardware buffer which means that even direct sound goes through the software mixer and format conversions. GASP, that's not such a problem that you make it out to be. Hell, winealsa even emulates a ring buffer with read calls, see f27d88e16fe.. If any direct sound driver implemented multiple buffers of any format, there would be no software mixing done in the direct sound dll. Everything would just pass through the direct sound dll directly to the driver untouched. We would also get multiple open support since it wouldn't matter which application the buffers came from. The dsound timer would still tick and most of the time the app would still use the crappy remixer in dsound since games use DSBCAPS_LOCSOFTWARE these days.. Even more fundamentally, our winmm drivers are crap, full of literally copied versions of wineoss, just renamed and sedded, but never maintained. Spot the similar #if 0's... The wavein/out drivers will be thrown out after we can forward to WASAPI, since the midi code is still handled in winmm afaict, a few of them will continue to live for that purpose.. Can I please have some new discussion point instead of you trying to bring up the same over and over? I'm growing tired of having to repeat myself so much..
Wine sound discussion summary
Hi, For those who didn't follow the big dsound argument thread, here is a quick summary of the main points. I originally inteded this as a summary mail, but while collecting the arguments I have instead taken the discussion off-list accidentally. But here's the current state: Pretty much everyone agrees that the current sound infrastructure is broken. In terms of how to fix it, there are 3 points to be worked out. 1) Use OpenAL or a more sophisticated driver infrastructure 2) If no OpenAL, what do we use 3) How do we get there. I'll mostly focus on (1) here, as the others depend on the decision of (1). On Wineconf we decided to give killing our entire sound infrastructure a try and replace it with a winmm and dsound based on OpenAL. That wasn't a promise by Alexandre to commit it, more an investigative study. The advantages of that would be: * A compact Wine sound system, we don't have many different sound drivers. OpenAL is available on all operating systems * We can let Linux(dmix, pulseaudio) do some or all of the mixing(with openal.so improvements) * Outsource workarounds for Alsa/PulseAudio/Coreaudio bugs to openal * Supporting DirectSound 3D is easy - openal just has it available. No need to reinvent the wheel. There are still a lot of dsound 3d games from WinXP and earlier. * Hardware accelerated 3D sound is possible. Currently there are no Linux drivers for that though. * Winmm, dsound and Windows openal.dll interaction still works fine as long as all libs interact with openal in the same way Disadvantages: * We have less control over our sound infrastructure and might get into troubles if some apps depend on the underlying driver system * We depend on the system openal. Linux distros have to keep their 32 bit openal updated. Getting bugfixes shipped to users might take years. MacOS OpenAL seems pretty broken. * Outsourcing bug workarounds is a slippery slope. It will be hard or impossible to work around problems without openal and distro help(and we need help again to get them fixed properly). * OpenAL is essentially just an abstraction layer not any system's native way to do sound. * Dsound 3D is dead in Win7, maybe we shouldn't focus our design decision on it 2) If no OpenAL, what do we use The consensus is mostly that if we don't use OpenAL for abstraction then we would aim for the Windows 7 sound driver API, or something very close to it. The goodies include a) Its the way Windows does it. b) If Win7 has user space audio driver support we might be able to use those drivers c) Technically native dsound and winmm might work, which can be helpful for debugging d) No bad surprises when apps start screwing around with the internals(e.g. load an DRM decryption APO into the sound processing chain). Problems of this sound infrastructure include: * It is very complex and maybe a lot more than we need. * The Windows 7 sound model mandates software mixing. We'd have a audiodg.exe process running that mixes all streams from all processes. Add in pulseaudio or dmix, and maybe dsound has to do its own mixing and you have 3 software mixers in a chain. But maybe that's a place where we can deviate from the MS design, or use mixing plugins provided by our drivers. * It rules out hardware accelerated dsound3d. Although a user might install Creative's openal wrapper or similar software to get back HW dsound 3D in the windows 7 way, with all its pros and cons. Most new 3D sound games use openal natively. * For full compatibility to drivers and native libs we'd need the DDK, and nobody checked the license terms so far. 3) How do we get there. This obviously depends on what we do, and we probably won't do much before the 1.2 release. For the OpenAL way, we could just throw in the code Maarten wrote so far, or Chris' Dsound-OpenAL wrapper. For Win7, one idea was to implement one more Win 3.1 sound driver in Wine that is based on the Win7 infrastructure, then implement that infrastructure and move winmm and dsound over. Once this is done, throw out the old Win 3.1 stuff. Or implement the Win7 infrastructure and maintain a separate dsound and winmm off-tree(developed on Windows), and put them in place once the Win7 driver infrastructure in Wine is solid enough.
Re: Wine sound discussion summary
2009/12/6 Stefan Dösinger stefandoesin...@gmx.at Hi, For those who didn't follow the big dsound argument thread, here is a quick summary of the main points. I originally inteded this as a summary mail, but while collecting the arguments I have instead taken the discussion off-list accidentally. But here's the current state: Nice summary. Pretty much everyone agrees that the current sound infrastructure is broken. In terms of how to fix it, there are 3 points to be worked out. 1) Use OpenAL or a more sophisticated driver infrastructure 2) If no OpenAL, what do we use 3) How do we get there. I'll mostly focus on (1) here, as the others depend on the decision of (1). On Wineconf we decided to give killing our entire sound infrastructure a try and replace it with a winmm and dsound based on OpenAL. That wasn't a promise by Alexandre to commit it, more an investigative study. The advantages of that would be: * A compact Wine sound system, we don't have many different sound drivers. OpenAL is available on all operating systems For any solution (OpenAL, Win7, other), I would say that having a unified audio codepath (like WineD3D does for ddraw/d3d) is a good thing. * Supporting DirectSound 3D is easy - openal just has it available. No need to reinvent the wheel. There are still a lot of dsound 3d games from WinXP and earlier. This is good, as those older games will then sound better on Linux+Wine than they do on Vista/Win7 with its default drivers. That is, the 3D effects in Diablo 2 and other games should just work. Disadvantages: * We have less control over our sound infrastructure and might get into troubles if some apps depend on the underlying driver system * We depend on the system openal. Linux distros have to keep their 32 bit openal updated. Getting bugfixes shipped to users might take years. MacOS OpenAL seems pretty broken. * Outsourcing bug workarounds is a slippery slope. It will be hard or impossible to work around problems without openal and distro help(and we need help again to get them fixed properly). There will be bugs in alsa, pulseaudio, oss4, openal, etc. that any adapter/abstraction layer will need to cope with. The key question is whether the benefits of creating and maintaining a wine-specific audio layer (like is done at present, with the duplication of bug-fixes for the different backends) outweigh the benefits of using an appropriate backend such as OpenAL. * OpenAL is essentially just an abstraction layer not any system's native way to do sound. This is true (in that it is like alsa, oss3, oss4 and the Windows/Mac audio driver layers), but is largely irrelevant. The key question here is how OpenAL interacts with the native system audio (i.e. how does it interact with alsa, oss and pulseaudio). * Dsound 3D is dead in Win7, maybe we shouldn't focus our design decision on it So. There are older games that are making use of dsound 3d, so it is not a deprecated API that no-one used. I thought Wine's policy was (in addition to getting Windows applications running on Linux/Mac/BSD/OpenSolaris/etc.) that it aims to support older applications, which means that if there is a way to properly support dsound 3d then Wine should support dsound 3d, irrespective of Microsoft's decisions to drop it. 2) If no OpenAL, what do we use The consensus is mostly that if we don't use OpenAL for abstraction then we would aim for the Windows 7 sound driver API, or something very close to it. The goodies include a) Its the way Windows does it. It is the way that Windows does it *now*. There is no guarantee that Windows 8 or later might do something different. b) If Win7 has user space audio driver support we might be able to use those drivers c) Technically native dsound and winmm might work, which can be helpful for debugging Wine isn't out to support the Windows driver model, it is aimed at supporting Windows applications and the Windows APIs that those applications use. d) No bad surprises when apps start screwing around with the internals(e.g. load an DRM decryption APO into the sound processing chain). What about applications that are dependent on the internals of pre-Vista Windows audio? Problems of this sound infrastructure include: * It is very complex and maybe a lot more than we need. * The Windows 7 sound model mandates software mixing. We'd have a audiodg.exe process running that mixes all streams from all processes. Add in pulseaudio or dmix, and maybe dsound has to do its own mixing and you have 3 software mixers in a chain. But maybe that's a place where we can deviate from the MS design, or use mixing plugins provided by our drivers. If Wine is going to go down the Windows 7 audio route, why not redirect everything to pulseaudio? Then you don't need yet another software mixer and you get the different applications showing up in the pulseaudio volume control mixer. * It rules out hardware
Re: Wine sound discussion summary
Am 06.12.2009 um 14:20 schrieb Reece Dunn: * Supporting DirectSound 3D is easy - openal just has it available. No need to reinvent the wheel. There are still a lot of dsound 3d games from WinXP and earlier. This is good, as those older games will then sound better on Linux+Wine than they do on Vista/Win7 with its default drivers. That is, the 3D effects in Diablo 2 and other games should just work. One thing to consider is that if old games break in Windows, people will write their own compatibility layers to fix that. Take for example the Glide-OpenGL wrapper, or the dsound-OpenAL wrappers that are out there and get these games working on Win7 with all the old features. We don't have a disadvantage compared to Windows if the same fixes work in the same way on Wine. We do however have a disadvantage if new apps don't work because we stick to old solutions for old apps. We killed a few Win16 and Win95 hacks in the past. * Dsound 3D is dead in Win7, maybe we shouldn't focus our design decision on it So. There are older games that are making use of dsound 3d, so it is not a deprecated API that no-one used. I thought Wine's policy was (in addition to getting Windows applications running on Linux/Mac/BSD/OpenSolaris/etc.) that it aims to support older applications, which means that if there is a way to properly support dsound 3d then Wine should support dsound 3d, irrespective of Microsoft's decisions to drop it. Actually, in many places we say that if an old app doesn't work on Wine and fails in the same way on some sort of reference windows version(currently probably XP, in a few years Windows 7), then letting the application fail is the correct thing to do. If its easy to get it running do it, but don't go to extreme lengths. For example, we don't support the Windows 95 memory management which breaks a lot of old apps. b) If Win7 has user space audio driver support we might be able to use those drivers c) Technically native dsound and winmm might work, which can be helpful for debugging Wine isn't out to support the Windows driver model, it is aimed at supporting Windows applications and the Windows APIs that those applications use. This is correct, and at times I have argued against trying to support other Windows drivers(e.g. 3D drivers). However, extra features that aren't the primary goal but still interesting aren't a drawback in my eyes. Being able to use Windows drivers definitely isn't our primary concern though. If Wine is going to go down the Windows 7 audio route, why not redirect everything to pulseaudio? Then you don't need yet another software mixer and you get the different applications showing up in the pulseaudio volume control mixer. This is something we've discussed on IRC a bit today. In fact, implementing the whole Win7 stuff with the audiodg.exe mixer process won't work - it will require lots of IPC, add latency and make games unplayable. See below for more: With the Vista/Win7 model, are there any new front-end audio APIs that applications can make use of, or is it still winmm+dsound? Yes, there's one new DLL for playback and one new DLL for sound card configuration - e.g. notification if a bluetooth sound card is added, PNP enumeration. We have to implement those new DLLs. However, do we reimplement the old DLLs on top of the new ones like Windows does(e.g. dsound.dll and winmm use the new Win7 sound DLLs) or just make every of our DLLs call OpenAL directly. (My personal opinion is that we have to keep the Windows way on the DLLs that can be considered the app frontend) Those new DLLs add a bunch of new features, like per-stream volume that more or less require a sound server like pulseaudio(Windows 7 has audiodg.exe, XP already had kmixer). That has some implications too.
Re: Wine sound discussion summary
I have made some drawings of the current Wine audio architecture, the WinXP architecture, the Vista architecture and the proposals we have seen. The drawings are based on information which can be found on MSDN. Regarding Vista/Win7 it would be useful to read: http://msdn.microsoft.com/en-us/library/dd316780%28VS.85%29.aspx which explains the architecture in detail. Basically all Microsoft APIs use WASAPI which in turn implements IAudioClient and other classes which talk to an 'audio engine' which is implemented in audiodg.dll/audiodg.exe. This audio engine is quite similar to pulseaudio and it offers functionality like per stream volume which normal APIs like oss/alsa/openal don't offer, so these APIs would map better to a sound server than to a standard library. Roderick sound.pdf Description: Adobe PDF document
Re: Wine sound discussion summary
Am 06.12.2009 um 15:32 schrieb Roderick Colenbrander: This audio engine is quite similar to pulseaudio and it offers functionality like per stream volume which normal APIs like oss/alsa/openal don't offer, so these APIs would map better to a sound server than to a standard library. I am wondering: Does CoreAudio on OSX offer similar features? Or do we need a sound server there as well if we want to support them? I CC'ed Ken. On the big picture I like the Wine WASAPI proposal. I do however think that we should retain the capability to have multiple backends. Maybe we should go all-in on PulseAudio with its API if that gives us all needed features, and use CoreAudio on OSX if it satisfies our needs without a separate sound server. Stefan
Re: Wine sound discussion summary
On Sun, Dec 6, 2009 at 5:08 PM, Stefan Dösinger stefandoesin...@gmx.at wrote: Am 06.12.2009 um 15:32 schrieb Roderick Colenbrander: This audio engine is quite similar to pulseaudio and it offers functionality like per stream volume which normal APIs like oss/alsa/openal don't offer, so these APIs would map better to a sound server than to a standard library. I am wondering: Does CoreAudio on OSX offer similar features? Or do we need a sound server there as well if we want to support them? I CC'ed Ken. On the big picture I like the Wine WASAPI proposal. I do however think that we should retain the capability to have multiple backends. Maybe we should go all-in on PulseAudio with its API if that gives us all needed features, and use CoreAudio on OSX if it satisfies our needs without a separate sound server. Stefan I just found this document. It is a good summary: http://support.creative.com/kb/ShowArticle.aspx?sid=25937 (expand all bullets) Roderick
Re: Wine sound discussion summary
Roderick Colenbrander skrev: This audio engine is quite similar to pulseaudio and it offers functionality like per stream volume which normal APIs like oss/alsa/openal don't offer, What? If configured appropriately, ALSA can do it, although with the current API it's not straightforward, as you need to map the stream to the corresponding control element, if there is one. The softvol ALSA plugin shows one implementation, but the method is also available in hardware on suitable sound cards (SB Live, I think). (Of course, you'd have to convince distros to use softvol by default, just like they now use dmix by default, so this may not be practical, but in principle, ALSA does offer the feature...) And obviously, OpenAL can do it, probably in several ways (though some might be more practical than others). Setting the corresponding buffer's AL_GAIN attribute appropriately seems like an obvious candidate. (Also wouldn't surprise me if the commercial OSS4 could do it...)
Re: Wine sound discussion summary
On Sun, Dec 6, 2009 at 5:10 PM, Roderick Colenbrander thunderbir...@gmail.com wrote: On Sun, Dec 6, 2009 at 5:08 PM, Stefan Dösinger stefandoesin...@gmx.at wrote: Am 06.12.2009 um 15:32 schrieb Roderick Colenbrander: This audio engine is quite similar to pulseaudio and it offers functionality like per stream volume which normal APIs like oss/alsa/openal don't offer, so these APIs would map better to a sound server than to a standard library. I am wondering: Does CoreAudio on OSX offer similar features? Or do we need a sound server there as well if we want to support them? I CC'ed Ken. On the big picture I like the Wine WASAPI proposal. I do however think that we should retain the capability to have multiple backends. Maybe we should go all-in on PulseAudio with its API if that gives us all needed features, and use CoreAudio on OSX if it satisfies our needs without a separate sound server. Stefan I just found this document. It is a good summary: http://support.creative.com/kb/ShowArticle.aspx?sid=25937 (expand all bullets) Roderick This is an update for the guys who aren't hanging out on IRC. We still aren't convinced whether dsound should be layered on top of OpenAL or WASAPI. An argument for going the OpenAL way would be that DSound3D is easier and that perhaps the latency is less (I will come back to latency later). The Vista way is to use MMDevAPI to enumerate 'endpoints' (speakers, headsets, microphones) and then to enable winmm/dsound/wasapi or whatever you want using MMDevAPI. You are able to use wasapi calls (IAudio*) with dsound. This allows you to access the sound buffer, adjust volume, get the position in the stream and so on. Some of those calls can be mapped to dsound itself (e.g. things to query the position or frequency) but the fear is that there are things which are not mappable. At this point the biggest part of the discussion focuses on WASAPI itself and how it should be implemented in Wine. We need to start taking this API into account because in Vista more and more apps rely on it. For instance World of Warcraft uses WASAPI in Vista and it doesn't have a fallback to DirectSound. Maarten verified this on Wine and indeed when you report Vista, you won't receive sound in WoW. Further XAudio2 and friends also use WASAPI on Vista but I don't know if those fall back (I guess not). Regarding on how to implement WASAPI. On Windows there is the 'Audio Engine' which is quite similar to PulseAudio. We want to avoid implementing this Audio Engine as it could mean layering our own sound server on top of another one (pulseaudio) and we are concerned about performance and overhead. We would like to let the 'audio library' do all the mixing and hard work. Maarten advocated to use OpenAL as the audio library. This would allow us to get rid of all the different audio backends which would make maintenance easier. The others thought that solely relying on OpenAL would not be wise. The WASAPI makes it easy to add other tiny backends since a lot of the code can be shared. The current idea would be to have wine 'audio drivers' implement the 'IAudioEngineEndpoint' interface (see the endpoint blocks in http://msdn.microsoft.com/en-us/library/dd316780(VS.85).aspx). The design would be made in such a way that multiple backends can be added with a minimum amount of code. Likely OpenAL would be a reference backend but we can easily add PulseAudio and CoreAudio. At the end of the discussion it felt like more and more people lean to a full WASAPI design. Some fear about latency and mention 'library X' adds a lot of latency and 'library Y adds even more'. When you implement your audio subsystem in software you will add more latency (on XP years ago microsoft accepted max 30ms, yes that's too much for an FPS). We need proof to justify claims about latency. Chris believes that pulseaudio contains decent mixing code these days and that the latency it adds might not be that bad. Personally I'm in favor of the full WASAPI backend and would have multiple backends including a Wine pulseaudio one. Roderick
Re: Wine sound discussion summary
Am 06.12.2009 um 20:20 schrieb Roderick Colenbrander: We would like to let the 'audio library' do all the mixing and hard work. Maarten advocated to use OpenAL as the audio library. This would allow us to get rid of all the different audio backends which would make maintenance easier. The others thought that solely relying on OpenAL would not be wise. I think have to define a interface for sound drivers in the way we currently have them. Whether or not we implement more than one is another topic. If OpenAL works I am fine with just a wineopenal driver *if the openal calls are properly separated from the rest of the code* . I don't like the idea of rewriting our wasapi implementation if we have to go away from openal for some reason.
Re: Wine sound discussion summary
Though, how would MIDI be handled. I think that OpenAL doesn't handle MIDI... Good question. As far as I can see PulseAudio doesn't support MIDI either, but it is certainly available in Alsa, CoreAudio and on Windows Vista, even though its more tricky to activate HW midi on Vista. Emulating MIDI through a software synthesizer won't fly, as this will make it impossible to use hardware instruments. A related topic: How do Joysticks connected to the gameport work(which is the same hardware connector as MIDI). I think that a joystick already takes an entirely different path in the Linux kernel and it doesn't get anywhere near a sound system, so we don't have to bother about it. Is that correct?
Re: Wine sound discussion summary
On Sun, Dec 6, 2009 at 10:42 PM, Stefan Dösinger stefandoesin...@gmx.at wrote: Though, how would MIDI be handled. I think that OpenAL doesn't handle MIDI... Good question. As far as I can see PulseAudio doesn't support MIDI either, but it is certainly available in Alsa, CoreAudio and on Windows Vista, even though its more tricky to activate HW midi on Vista. Emulating MIDI through a software synthesizer won't fly, as this will make it impossible to use hardware instruments. A related topic: How do Joysticks connected to the gameport work(which is the same hardware connector as MIDI). I think that a joystick already takes an entirely different path in the Linux kernel and it doesn't get anywhere near a sound system, so we don't have to bother about it. Is that correct? Joysticks work using /dev/js or evdev these days. It was hard to find some info on MIDI for Vista but I believe it still works using WinMM. You really need oss or alsa for midi I think. Roderick
Re: Wine sound discussion summary
Am 06.12.2009 um 22:57 schrieb Roderick Colenbrander: Joysticks work using /dev/js or evdev these days. It was hard to find some info on MIDI for Vista but I believe it still works using WinMM. You really need oss or alsa for midi I think. There's this: http://www.jcabs-rumblings.com/Programming/OpenALFeatures.html But it seems to be an early design prototype. All other google hits say that MIDI is not supported by openal. If you want to play MIDI files with openal you have to synthesize them first(if you want to have openal apply 3D effects on top of that) or use a different API.
Re: Wine sound discussion summary
Hello, Roderick Colenbrander schreef: On Sun, Dec 6, 2009 at 10:42 PM, Stefan Dösinger stefandoesin...@gmx.at wrote: Though, how would MIDI be handled. I think that OpenAL doesn't handle MIDI... Good question. As far as I can see PulseAudio doesn't support MIDI either, but it is certainly available in Alsa, CoreAudio and on Windows Vista, even though its more tricky to activate HW midi on Vista. Emulating MIDI through a software synthesizer won't fly, as this will make it impossible to use hardware instruments. A related topic: How do Joysticks connected to the gameport work(which is the same hardware connector as MIDI). I think that a joystick already takes an entirely different path in the Linux kernel and it doesn't get anywhere near a sound system, so we don't have to bother about it. Is that correct? Joysticks work using /dev/js or evdev these days. It was hard to find some info on MIDI for Vista but I believe it still works using WinMM. You really need oss or alsa for midi I think. I wouldn't be surprised if this still was the case, we could keep the midi interfaces around and just report 0 wave in/out devices for oss/coreaudio/alsaa once we complete wine7audio.drv. Cheers, Maarten.
Re: Wine sound discussion summary
On Sun, Dec 6, 2009 at 11:19 PM, Stefan Dösinger stefandoesin...@gmx.at wrote: Am 06.12.2009 um 22:57 schrieb Roderick Colenbrander: Joysticks work using /dev/js or evdev these days. It was hard to find some info on MIDI for Vista but I believe it still works using WinMM. You really need oss or alsa for midi I think. There's this: http://www.jcabs-rumblings.com/Programming/OpenALFeatures.html But it seems to be an early design prototype. All other google hits say that MIDI is not supported by openal. If you want to play MIDI files with openal you have to synthesize them first(if you want to have openal apply 3D effects on top of that) or use a different API. Midi playback isn't that important the main feature is Midi access to hardware like keyboards, drums, light equipment and so on. It is basically the 'RS232' of the audio/video world. Roderick
Re: Wine sound discussion summary
Am 06.12.2009 um 23:21 schrieb Maarten Lankhorst: I wouldn't be surprised if this still was the case, we could keep the midi interfaces around and just report 0 wave in/out devices for oss/coreaudio/alsaa once we complete wine7audio.drv. So you mean keeping the existing infrastructure around just for midi and have something entirely different for the rest of the audio features? Apparently winmm is still the only way to do MIDI on Windows Vista, and if winmm uses wasapi and friends there must be some way to tell the devices to talk to MIDI instruments.
Re: Wine sound discussion summary
On Sun, Dec 6, 2009 at 11:34 PM, Stefan Dösinger stefandoesin...@gmx.at wrote: Am 06.12.2009 um 23:21 schrieb Maarten Lankhorst: I wouldn't be surprised if this still was the case, we could keep the midi interfaces around and just report 0 wave in/out devices for oss/coreaudio/alsaa once we complete wine7audio.drv. So you mean keeping the existing infrastructure around just for midi and have something entirely different for the rest of the audio features? Apparently winmm is still the only way to do MIDI on Windows Vista, and if winmm uses wasapi and friends there must be some way to tell the devices to talk to MIDI instruments. I mainly wonder how Vista is able to sync MIDI with audio playback? I would say that's an important feature e.g. be able to enable light effects on a podium in sync with the music you play from your computer. Roderick
Re: Wine sound discussion summary
On Sun, Dec 6, 2009 at 4:37 PM, Roderick Colenbrander thunderbir...@gmail.com wrote: On Sun, Dec 6, 2009 at 11:34 PM, Stefan Dösinger stefandoesin...@gmx.at wrote: Am 06.12.2009 um 23:21 schrieb Maarten Lankhorst: I wouldn't be surprised if this still was the case, we could keep the midi interfaces around and just report 0 wave in/out devices for oss/coreaudio/alsaa once we complete wine7audio.drv. So you mean keeping the existing infrastructure around just for midi and have something entirely different for the rest of the audio features? Apparently winmm is still the only way to do MIDI on Windows Vista, and if winmm uses wasapi and friends there must be some way to tell the devices to talk to MIDI instruments. I mainly wonder how Vista is able to sync MIDI with audio playback? I would say that's an important feature e.g. be able to enable light effects on a podium in sync with the music you play from your computer. Roderick DirectMusic is the other way to work with MIDI I think.
