Re: [Sursound] about principled rendering of ambisonic to binaural

[Brendan Baker]

Hi Lorenzo,

I'm a longtime lurker on this discussion group and I think this is the
first time I've reached out to anyone directly via the listserv. But I
wanted to say I recently tried your 3DTI VSTs and was really impressed with
your binauralization algorithm. I'm not sure if it's just a happy
coincidence that your default settings happen to be a good fit for my HRTF
me or what (I haven't played with other HRTF settings yet), but I felt a
noticeable improvement in localization in comparison to many other
2ch-binaural or ambisonic-binaural decoders I've used. I was most impressed
when using it to make a virtual binaural "array" using different speaker
locations mixed together. I think it might be the closest-sounding to
native binaural that I've heard so far.

I did, however, run into a problem: I can't seem to save VST parameters.
I'm using the latest version of Reaper on an intel Mac. Whenever I reload a
project or copy/paste an instance of the VST, all the parameters including
azimuth/elevation get reset back to default. The only workaround I've found
so far is to write parameter automation points into Reaper itself. Is this
the intended behavior, or is there a way to save parameter settings without
having to write automation?

I'm also curious if you have plans to make a multichannel input version
that would allow for cube/7.1.4/5.1 etc inputs without having to build a
virtual array with many tracks/complex routing? Or an ambisonic decoder
using your same algorithm? I suspect it would sound amazing!

Thanks and much appreciated,
- Brendan

--
Brendan P. Baker
brendanpatrickbaker.com
Twitter: @brendanpbaker <https://twitter.com/BrendanPBaker>




On Tue, Dec 6, 2022 at 12:28 PM Picinali, Lorenzo 
wrote:

> Hi Bosse,
>
> so...for the reverberation, the 3DTI toolkit requires a set of Binaural
> Room Impulse Responses, specifically one measured on the front, one back,
> on left, one right, one above and one below. You can of course do it with
> just the ones on the horizontal plane, and also with just the left and
> right ones. In your case, you have a 1st Order IR for a few positions on
> the horizontal plane, so...you'll need to add the binaural "component.
>
> A "dirty" way of doing that could be to decode each of your 1st Order IRs
> to a virtual cube loudspeaker layout, and convert each of the loudspeaker
> channels to binaural by convolving them with the HRIR corresponding to
> their individual position (you can use your own HRTF or a KEMAR one, or
> else). The resulting binaural response (i.e. summing all eight
> "binauralised" loudspeaker channels) from the frontal channel source will
> be your front BRIR. You can then average the two from the back channels to
> obtain a rear BRIR. Then again average the front and back left ones to get
> your left BRIR, and front and back right ones to get your right BRIR.
> You'll then need to remove the direct path signal and substitute it with
> the same amount of zeroes (as the direct path rendering will be managed by
> the 3DTI Toolkit via direct convolution with the HRTF), save the 4 BRIRs in
> a sofa file, and import them in the VST. For these last steps (remove
> direct path and save in sofa file) I should have some ready Matlab scripts
> - please email me directly and I can send them over!
>
> I hope this helps!
>
> Best
> Lorenzo
>
>
> --
> Dr Lorenzo Picinali
> Reader in Audio Experience Design<https://www.axdesign.co.uk/>
> Dyson School of Design Engineering
> Imperial College London
> Dyson Building
> Imperial College Road
> South Kensington, SW7 2DB, London
> E: l.picin...@imperial.ac.uk
>
> http://www.imperial.ac.uk/people/l.picinali
> https://www.axdesign.co.uk/
> 
> From: Sursound  on behalf of Bo-Erik
> Sandholm 
> Sent: 06 December 2022 16:29
> To: Surround Sound discussion group 
> Subject: Re: [Sursound] about principled rendering of ambisonic to binaural
>
> Hi Lorenzo
> I have FOA recordings of sweeps from a very good listening room that i want
> to use as a virtual binaural listening room for commersial music.
>
> I have responses for stereo, virtual center and back channels for 5.1
>
> Is it possible to use your VSTs for such a setup?
> Any tips on how to do it if so.
>
> Bosse
>
> On Sun, 18 Sept 2022, 11:11 Picinali, Lorenzo, 
> wrote:
>
> > Hello,
> >
> > I can advertise our binaural 3D Tune-In Toolkit, which exists also in the
> > form of VST plugin (as well as standalone application, Unity and
> Javascript
> > wrappers). You can just create one instance of the plugin for every
> output
> > track, and spatialise that in the position where the loudspeaker shoul

Re: [Sursound] about principled rendering of ambisonic to binaural

[Picinali, Lorenzo]

Hi Bosse,

so...for the reverberation, the 3DTI toolkit requires a set of Binaural Room 
Impulse Responses, specifically one measured on the front, one back, on left, 
one right, one above and one below. You can of course do it with just the ones 
on the horizontal plane, and also with just the left and right ones. In your 
case, you have a 1st Order IR for a few positions on the horizontal plane, 
so...you'll need to add the binaural "component.

A "dirty" way of doing that could be to decode each of your 1st Order IRs to a 
virtual cube loudspeaker layout, and convert each of the loudspeaker channels 
to binaural by convolving them with the HRIR corresponding to their individual 
position (you can use your own HRTF or a KEMAR one, or else). The resulting 
binaural response (i.e. summing all eight "binauralised" loudspeaker channels) 
from the frontal channel source will be your front BRIR. You can then average 
the two from the back channels to obtain a rear BRIR. Then again average the 
front and back left ones to get your left BRIR, and front and back right ones 
to get your right BRIR. You'll then need to remove the direct path signal and 
substitute it with the same amount of zeroes (as the direct path rendering will 
be managed by the 3DTI Toolkit via direct convolution with the HRTF), save the 
4 BRIRs in a sofa file, and import them in the VST. For these last steps 
(remove direct path and save in sofa file) I should have some ready Matlab 
scripts - please email me directly and I can send them over!

I hope this helps!

Best
Lorenzo


--
Dr Lorenzo Picinali
Reader in Audio Experience Design<https://www.axdesign.co.uk/>
Dyson School of Design Engineering
Imperial College London
Dyson Building
Imperial College Road
South Kensington, SW7 2DB, London
E: l.picin...@imperial.ac.uk

http://www.imperial.ac.uk/people/l.picinali
https://www.axdesign.co.uk/

From: Sursound  on behalf of Bo-Erik Sandholm 

Sent: 06 December 2022 16:29
To: Surround Sound discussion group 
Subject: Re: [Sursound] about principled rendering of ambisonic to binaural

Hi Lorenzo
I have FOA recordings of sweeps from a very good listening room that i want
to use as a virtual binaural listening room for commersial music.

I have responses for stereo, virtual center and back channels for 5.1

Is it possible to use your VSTs for such a setup?
Any tips on how to do it if so.

Bosse

On Sun, 18 Sept 2022, 11:11 Picinali, Lorenzo, 
wrote:

> Hello,
>
> I can advertise our binaural 3D Tune-In Toolkit, which exists also in the
> form of VST plugin (as well as standalone application, Unity and Javascript
> wrappers). You can just create one instance of the plugin for every output
> track, and spatialise that in the position where the loudspeaker should be.
> We have a version of the plugin with reverb, which is easy to use but
> rather heavy from the computational point of view, or an anechoic version,
> with a bus reverb, which is definitely more efficient - in the downloads
> you will be able to find a template Reaper project which shows how to setup
> the anechoic and bus reverb plugins.
>
> An example of the functionalities of the Test Application can be found in
> this video (use headphones when listening) - the VST plugin creates of
> course the same spatialisation effect.
>
> https://www.youtube.com/watch?v=osJQ0Kxv1P0
>
> The Test Application (the one you see in the video above) is available for
> MacOS, Windows and Linux at the following link:
>
> https://github.com/3DTune-In/3dti_AudioToolkit/releases/latest
>
> At the link above, you can also download the VST plugin, both for MacOS
> and Windows, as well as the Unity wrapper.
>
> If interested, you can find some details about the 3DTI Toolkit spatial
> audio implementation in this paper:
>
> https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0211899
>
> And the open source code is available from the GitHub account:
>
> https://github.com/3DTune-In/3dti_AudioToolkit
>
> If you have a chance to come around in the London area, we can measure
> your HRTF so that you can use that for your mixes - just get in touch!
>
> best
> Lorenzo
>
>
>
>
>
>
>
> --
> Dr Lorenzo Picinali
> Reader in Audio Experience Design<https://www.axdesign.co.uk/>
> Dyson School of Design Engineering
> Imperial College London
> Dyson Building
> Imperial College Road
> South Kensington, SW7 2DB, London
> E: l.picin...@imperial.ac.uk
>
> http://www.imperial.ac.uk/people/l.picinali
> https://www.axdesign.co.uk/
> ____
> From: Sursound  on behalf of Ralph Jones <
> rjonesth...@comcast.net>
> Sent: 18 September 2022 01:51
> To: sursound@music.vt.edu 
> Subject: Re: [Sursound] about principled re

Re: [Sursound] about principled rendering of ambisonic to binaural

[Bo-Erik Sandholm]

Hi Lorenzo
I have FOA recordings of sweeps from a very good listening room that i want
to use as a virtual binaural listening room for commersial music.

I have responses for stereo, virtual center and back channels for 5.1

Is it possible to use your VSTs for such a setup?
Any tips on how to do it if so.

Bosse

On Sun, 18 Sept 2022, 11:11 Picinali, Lorenzo, 
wrote:

> Hello,
>
> I can advertise our binaural 3D Tune-In Toolkit, which exists also in the
> form of VST plugin (as well as standalone application, Unity and Javascript
> wrappers). You can just create one instance of the plugin for every output
> track, and spatialise that in the position where the loudspeaker should be.
> We have a version of the plugin with reverb, which is easy to use but
> rather heavy from the computational point of view, or an anechoic version,
> with a bus reverb, which is definitely more efficient - in the downloads
> you will be able to find a template Reaper project which shows how to setup
> the anechoic and bus reverb plugins.
>
> An example of the functionalities of the Test Application can be found in
> this video (use headphones when listening) - the VST plugin creates of
> course the same spatialisation effect.
>
> https://www.youtube.com/watch?v=osJQ0Kxv1P0
>
> The Test Application (the one you see in the video above) is available for
> MacOS, Windows and Linux at the following link:
>
> https://github.com/3DTune-In/3dti_AudioToolkit/releases/latest
>
> At the link above, you can also download the VST plugin, both for MacOS
> and Windows, as well as the Unity wrapper.
>
> If interested, you can find some details about the 3DTI Toolkit spatial
> audio implementation in this paper:
>
> https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0211899
>
> And the open source code is available from the GitHub account:
>
> https://github.com/3DTune-In/3dti_AudioToolkit
>
> If you have a chance to come around in the London area, we can measure
> your HRTF so that you can use that for your mixes - just get in touch!
>
> best
> Lorenzo
>
>
>
>
>
>
>
> --
> Dr Lorenzo Picinali
> Reader in Audio Experience Design<https://www.axdesign.co.uk/>
> Dyson School of Design Engineering
> Imperial College London
> Dyson Building
> Imperial College Road
> South Kensington, SW7 2DB, London
> E: l.picin...@imperial.ac.uk
>
> http://www.imperial.ac.uk/people/l.picinali
> https://www.axdesign.co.uk/
> ________
> From: Sursound  on behalf of Ralph Jones <
> rjonesth...@comcast.net>
> Sent: 18 September 2022 01:51
> To: sursound@music.vt.edu 
> Subject: Re: [Sursound] about principled rendering of ambisonic to binaural
>
>
> ***
> This email originates from outside Imperial. Do not click on links and
> attachments unless you recognise the sender.
> If you trust the sender, add them to your safe senders list
> https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping
> for this address.
> ***
> I’m a composer, not a mathematician, so while I try, I don’t get very far
> at understanding discussions like this. But the subject is of real concern
> for me, because I am currently working in 5.1.4 surround format
> (channel-based, not Atmos) and I would dearly love to find a mac-compatible
> VST plugin that would convincingly render my work in binaural. So, is there
> a plugin that does what Fons describes here? (i.e., given azimuth and
> elevation for each channel, render the signals to binaural convincingly,
> including an impression of elevation for height channels.)
>
> Ralph Jones
>
> > On Sep 13, 2022, at 9:00 AM,Fons Adriaensen wrote:
> >
> > Message: 1
> > Date: Tue, 13 Sep 2022 15:59:49 +0200
> > From: Fons Adriaensen 
> > To: sursound@music.vt.edu
> > Subject: Re: [Sursound] about principled rendering of ambisonic to
> >binaural
> > Message-ID:
> ><20220913135949.ugwflytibwa7p...@mail1.linuxaudio.cyso.net>
> > Content-Type: text/plain; charset=iso-8859-1
> >
> >
>
> [Snip]
>
> > Another question is if for high quality binaural rendering, starting from
> > Ambisonic content is a good idea at all.
> >
> > Simple fact is that if you want really good results you need very high
> > order, and
> >
> > 1. such content isn't available from direct recordings (we don't have
> even
> > 10th order microphpones), so it has to be synthetic,
> >
> > 2. rendering it from an Ambisonic format would be very inefficient. For
> > example for order 20 you'd need 441 convolutions if you assume L/R head
> > symmetry, twic

Re: [Sursound] about principled rendering of ambisonic to binaural

[Sampo Syreeni]

On 2022-09-17, Ralph Jones wrote:

I’m a composer, not a mathematician, so while I try, I don’t get very 
far at understanding discussions like this.


It's not such a long road from being a composer to being a math geek, 
you know. Especially to becoming a tehcno fiend, of the surround sound 
kind... ;)


My music, in my twenties or so, sounds like this: 
http://decoy.iki.fi/decoy/download/drone_1.mp3 . It's wholly by the 
numbers, so that quite literally it might have a DC offset. Certainly 
most of it's about high order nonlinearity, because I applied straight 
up aliasing on purpose.


Also, http://decoy.iki.fi/decoy/download/smokin.mp3 .
--
Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front
+358-40-3751464, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2
___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.

Re: [Sursound] about principled rendering of ambisonic to binaural

[Sampo Syreeni]

On 2022-09-17, Ralph Jones wrote:

But the subject is of real concern for me, because I am currently 
working in 5.1.4 surround format (channel-based, not Atmos) and I 
would dearly love to find a mac-compatible VST plugin that would 
convincingly render my work in binaural.


Hmm. Sursound has been a bit of a mathematicians' list for some time. 
Ambisonic and later on WFS. Adverse to the more usual 
x.y.z-sound-systems.


What if we now finally did some usable code or examples? Us fiends?

So, is there a plugin that does what Fons describes here? (i.e., given 
azimuth and elevation for each channel, render the signals to binaural 
convincingly, including an impression of elevation for height 
channels.)


You gave a channel arrangement. Or a speaker arrangement. That 5.1.4 
arrangement sort of tells us what you have, or where, but not 
*precisely*. It doesn't tell at which precise angles or at which 
distances.


As such, it doesn't tell us how those various channels *sound* to a 
listener. Not precisely. So it's impossible to even start to render them 
into binaural. Also, you'd need to specify a model of your ears, which 
you didn't give. (The KEMAR-model is a model of ears, so I'd probably 
start with those. But they are not *your* ears, but a make-do 
average-sounding one. Plus the set is symmetrized, unlike anybodys 
real.)

--
Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front
+358-40-3751464, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2
___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.

Re: [Sursound] about principled rendering of ambisonic to binaural

[Sampo Syreeni]

On 2022-09-13, Fons Adriaensen wrote:


Even if that case it isn't as simple as you seem to think.


Obviously I simplify. I know a thing or two already.

Any set of measured HRIR will need some non trivial preprocessing 
before it can be used. One reason is low-frequency errors. Accurate IR 
measurements below say 200 Hz are difficult (unless you have a very 
big and good anechoic room). OTOH we know that HRIR in that frequency 
range are very low order and can be synthesised quite easily.


As such we put in a priori knowledge into the model, and/or somehow 
repeat the measurement, coherently adding the resulting signal so as to 
bring down the noise floor.


Another reason is that you can't reduce a set of HRIR to low order 
(the order of the content you want to render) without introducing 
significant new errors.


I believe the systematic way to talk about this is to see how 
directional sources reduced to a central framework is about the 
Fourier-Bessel reduction, which doesn't reduce easily to the rectilinear 
Fourier decomposition. Even their reduced orders aren't comparible. In 
each frame, a low order, finite order field which looks perfectly 
even/in-quadrature, in the other has an infinite order decomposition.


But they work pretty well against each other for most outside-of-the-rig 
sources. The higher order crossterms in the transform fall off fast, so 
that you can approximate pretty well either way. That's where the 
Daniel, Nicol & Moreau NFC-HOA paper comes from. (They also did Hankel 
functions, in outwards going energy transfer. Their solution is exact, 
and they've talked about the connection to rectangular WFS, but even 
they didn't quantify this all fully.)


One way to reduce these is to reduce or even fully remove ITD at mid 
and high frequencies, again depending on the order the renderer is 
supposed to support.


This is all implicit in the order of reconstruction. ITD is just 
derivative of the soundfield over your ears. Of course Gerzon took 
Makita theory, but the latter is derivable from first the acoustic wave 
equation, and then its ecologically minded, reduced psychoacoustics. 
Once you go to third order ambisonic or beyond, no psychoacoustics are 
necessary.


Getting the magnitudes (and hence ILD) accurate requires much lower 
order than if you also want to keep the delays.


My point is that in binaural work, especially if head tracked, you can 
easily get to order twenty or so. No ITD/ILD-analysis needed, because 
it'll mimic physical reality.


If we can just do it right. How do we, from a sparse and anisotropic 
meaasurement set?


Compared to these and some other issues, not having a set on a regular 
grid (e.g. t-design or Lebedev) is the least of problems you will 
encounter.


Tell me more? I don't recognize these ones just yet.

There are other considerations. For best results you need head 
tracking and a plausible room sound (even if the content already 
includes its own).


In plausible room reverberation, I might have some ideas as well. :)

The practical solutions do not depend on such concepts and are much 
more ad-hoc. Some members of my team and myself worked on them for the 
last three years. Most of the results are confidential, although 
others (e.g. IEM) have arrived at some similar results and published 
them.


IEM is serious shit. If the results are confidential, so be it, but at 
the same time, if I'd have something to contribute, I'd happily sign an 
NDA. Just to be in the know, and contribute; I've done it before, and 
would do again.


It'd just be fun to actually solve or at least quantify the limits of 
this particular mathematical problem. How well can you actually go from 
say a KEMAR set to a binaural-to-ambisonic rendition? Isotropically? If 
you don't really have the bottomwards set? How do you interpolate, and 
where's your overall error metric? And so on?!? My problem.


Another question is if for high quality binaural rendering, starting 
from Ambisonic content is a good idea at all.


Obviously it isn't, in all cases. For example, if your content is such 
that the listener mostly looks forward, as say towards a movie screen, a 
fully isotropic ambisonic signal of any order wastes bandwidth/entropy. 
A lot of it. Even at first order POA, probably something like 2+epsilon 
channels worth of it. If they used just periphonic ambisonic, they'd get 
more typical theatrical sounds. In fact, if they really optimized the 
thing for frontal sound, and maybe a supplemental, theatrically minded 
surround track, such as in Dolby Surround, maybe it'd need even less.


But the thing is, ambisonic has always been an excercise in generality, 
regularity, and so virtual auditory reality, with mathematical 
certainty. Above instant efficiency or cheapness. It's never been about 
what is easy, but about being able to look allround, and to perceive the 
same isotropic soundfield, even if you look up or down. The auditory 
idea of what we now call "imm

Re: [Sursound] about principled rendering of ambisonic to binaural

[Sampo Syreeni]

On 2022-09-11, Picinali, Lorenzo wrote:


https://acta-acustica.edpsciences.org/articles/aacus/abs/2022/01/aacus210029/aacus210029.html


Thank you, will read.
--
Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front
+358-40-3751464, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2
___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.

Re: [Sursound] about principled rendering of ambisonic to binaural

[Marc Lavallée]

Le 2022-09-23 à 17 h 26, Ralph Jones a écrit :

Any further suggestions would be gratefully accepted.


Explore the SPARTA suite, specifically the AmbiENC plugin to encode 
virtual sources to ambisonics (up to 7th order), then render the 
resulting ambisonics stream with the AmbiBIN plugin. AmbiENC already 
have a 5.0 preset and you can add your own. AmbiBIN have a few options 
to enhance the spatial perception, which is useful when using a generic 
HRIR set. It won't be perfect (of the best possible option), but it 
could be good enough.


Marc
-- next part --
An HTML attachment was scrubbed...
URL: 

___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.

Re: [Sursound] about principled rendering of ambisonic to binaural

[Ralph Jones]

Thank you all who responded to my inquiry regarding a method to encode a 5.1.4 
channel stream to binaural. Some responses made clear that I didn’t provide 
enough information in my initial post. Specifically:

1) I am using Reaper on a Mac Mini. I own Logic, but while I tried at first to 
use it for 2D surround work, I rapidly became frustrated by its limitations 
when venturing beyond stereo. And I’m interested primarily in 3D surround 
(including height channels), which means I really need the much greater 
flexibility and channel counts that you get with Reaper. 

2) My interest in encoding my channels to binaural is not for personal 
monitoring. My studio is based around a physical 5.1.4 loudspeaker system, 
which I’ll probably expand to 7.1.4 relatively soon. And it’s all Meyer, so I 
pretty much have monitoring nailed down.

3) The reason I asked about encoding to binaural is that it’s the only way I 
can see to make my work available for more people. AFAIK, pretty much everyone 
now experiences music through earbuds. And basically no one can be expected to 
have a full-blown surround speaker system like mine, anyway, so I think it’s to 
my advantage that folks are using earbuds: at least there’s a possibility to 
give them an approximation of 3D surround using binaural. I hope.

So, that said, I really appreciate all the great information and I’m eagerly 
looking forward to experimenting with 3D Tune-In Toolkit, Anaglyph and the IEM 
plugin suite. Any further suggestions would be gratefully accepted.

Ralph Jones
___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.

Re: [Sursound] about principled rendering of ambisonic to binaural

[brian.katz]

Ralph, 

If you are looking for direct/object binaural rendering in a VST, I would 
recommend checking out Anaglyph (http://anaglyph.dalembert.upmc.fr/), a free 
VST (OSX and Win compatible) from our academic research over the years, 
highlighting improved externalisation features. Simply provide coordinate 
(azimuth/elevation/distance) for each track and away you go. 

-Brian
--
Brian FG Katz, Research Director, CNRS
Groupe Lutheries - Acoustique – Musique
Sorbonne Université, CNRS, UMR 7190, Institut Jean Le Rond ∂'Alembert 
http://www.dalembert.upmc.fr/home/katz


___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.

Re: [Sursound] about principled rendering of ambisonic to binaural

[Picinali, Lorenzo]

Hello,

I can advertise our binaural 3D Tune-In Toolkit, which exists also in the form 
of VST plugin (as well as standalone application, Unity and Javascript 
wrappers). You can just create one instance of the plugin for every output 
track, and spatialise that in the position where the loudspeaker should be. We 
have a version of the plugin with reverb, which is easy to use but rather heavy 
from the computational point of view, or an anechoic version, with a bus 
reverb, which is definitely more efficient - in the downloads you will be able 
to find a template Reaper project which shows how to setup the anechoic and bus 
reverb plugins.

An example of the functionalities of the Test Application can be found in this 
video (use headphones when listening) - the VST plugin creates of course the 
same spatialisation effect.

https://www.youtube.com/watch?v=osJQ0Kxv1P0

The Test Application (the one you see in the video above) is available for 
MacOS, Windows and Linux at the following link:

https://github.com/3DTune-In/3dti_AudioToolkit/releases/latest

At the link above, you can also download the VST plugin, both for MacOS and 
Windows, as well as the Unity wrapper.

If interested, you can find some details about the 3DTI Toolkit spatial audio 
implementation in this paper:

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0211899

And the open source code is available from the GitHub account:

https://github.com/3DTune-In/3dti_AudioToolkit

If you have a chance to come around in the London area, we can measure your 
HRTF so that you can use that for your mixes - just get in touch!

best
Lorenzo







--
Dr Lorenzo Picinali
Reader in Audio Experience Design<https://www.axdesign.co.uk/>
Dyson School of Design Engineering
Imperial College London
Dyson Building
Imperial College Road
South Kensington, SW7 2DB, London
E: l.picin...@imperial.ac.uk

http://www.imperial.ac.uk/people/l.picinali
https://www.axdesign.co.uk/

From: Sursound  on behalf of Ralph Jones 

Sent: 18 September 2022 01:51
To: sursound@music.vt.edu 
Subject: Re: [Sursound] about principled rendering of ambisonic to binaural


***
This email originates from outside Imperial. Do not click on links and 
attachments unless you recognise the sender.
If you trust the sender, add them to your safe senders list 
https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for 
this address.
***
I’m a composer, not a mathematician, so while I try, I don’t get very far at 
understanding discussions like this. But the subject is of real concern for me, 
because I am currently working in 5.1.4 surround format (channel-based, not 
Atmos) and I would dearly love to find a mac-compatible VST plugin that would 
convincingly render my work in binaural. So, is there a plugin that does what 
Fons describes here? (i.e., given azimuth and elevation for each channel, 
render the signals to binaural convincingly, including an impression of 
elevation for height channels.)

Ralph Jones

> On Sep 13, 2022, at 9:00 AM,Fons Adriaensen wrote:
>
> Message: 1
> Date: Tue, 13 Sep 2022 15:59:49 +0200
> From: Fons Adriaensen 
> To: sursound@music.vt.edu
> Subject: Re: [Sursound] about principled rendering of ambisonic to
>binaural
> Message-ID:
><20220913135949.ugwflytibwa7p...@mail1.linuxaudio.cyso.net>
> Content-Type: text/plain; charset=iso-8859-1
>
>

[Snip]

> Another question is if for high quality binaural rendering, starting from
> Ambisonic content is a good idea at all.
>
> Simple fact is that if you want really good results you need very high
> order, and
>
> 1. such content isn't available from direct recordings (we don't have even
> 10th order microphpones), so it has to be synthetic,
>
> 2. rendering it from an Ambisonic format would be very inefficient. For
> example for order 20 you'd need 441 convolutions if you assume L/R head
> symmetry, twice that number if you don't.
>
> Compare this to rendering from object encoded content (i.e. mono signals
> plus directional metadata). You need only two convolutions per object.
> Starting from a sufficiently dense HRIR set, you can easily generate a
> new set on a regular grid with a few thousand points, and interpolate
> them (VBAP style) in real time. This can give you the same resolution
> as e.g. order 40 Ambisonics at fraction of the complexity.
>
>
> Ciao,
>
> --
> FA

___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.
-- next part --
An HTML attachment was scrubbed...
URL: 
<https://mail.music.vt.edu/mailman/private/sursound/attachments/20220918/54475ea0/attachment

Re: [Sursound] about principled rendering of ambisonic to binaural

[Søren Bendixen]

I´m also "only" a musician/composer, trying to get wiser..:-)
But I do know that VST is not working in Logic - this is AU format only.

I would try stuff, and listen.
Maybe you could use Reasurroundpan the new surroundplugin in reaper. if U don´t 
have reaper U can get a 60 days free evaluation (or more)
Another plug in I use for listening back is DearVR MONITOR.  Penteo can do the 
same and go directly to Binaural but goes from atmos 5.1.4.
Or you can try Flux spat revolution (expensive)
There is trial versions of all this plug ins (Spat R is Not a plug in though)

Med venlig hilsen/Best regards

Søren Bendixen
Composer/Sound Designer &  stringed instruments player

Company: Audiotect
Web:audiotect.dk <http://audiotect.dk/>

Latest Music and Sounddesign

14 september - ?
Sound and Music for New Exhibition at Danish Jewish Museum, Copenhagen.

August 2022 - ?
Music and sounddesign for the Exhibition "Auf den Spuren einer Königin", at 
Großraden, Germany

29 april 2022 - february 23
Sounddesign for Neanderthal exhibition at the National History Museum - 
Copenhagen

22 january - 11 september 2022
Music and sounddesign for "RUS Viking" at Moesgaard Museum




In progress
Sound and Music for the Exhibition KAOS. Moesgaard Museum. Opens 11 nov. 2022

Recordings for new album :"Musik" by Søren Bendixen.Release fall 2022

> Den 18. sep. 2022 kl. 04.57 skrev David McKevy :
> 
> Ralph you mentioned wanting a Mac based VST so I assume you have access to
> Logic and can try their binaural decoder?
> 
> It's been a while since I worked with channel based surround but I assume
> that by now they may have varying HRTF's...
> 
> Just a thought...
> 
> On Sat, Sep 17, 2022, 8:52 PM Ralph Jones  wrote:
> 
>> I’m a composer, not a mathematician, so while I try, I don’t get very far
>> at understanding discussions like this. But the subject is of real concern
>> for me, because I am currently working in 5.1.4 surround format
>> (channel-based, not Atmos) and I would dearly love to find a mac-compatible
>> VST plugin that would convincingly render my work in binaural. So, is there
>> a plugin that does what Fons describes here? (i.e., given azimuth and
>> elevation for each channel, render the signals to binaural convincingly,
>> including an impression of elevation for height channels.)
>> 
>> Ralph Jones
>> 
>>> On Sep 13, 2022, at 9:00 AM,Fons Adriaensen wrote:
>>> 
>>> Message: 1
>>> Date: Tue, 13 Sep 2022 15:59:49 +0200
>>> From: Fons Adriaensen 
>>> To: sursound@music.vt.edu
>>> Subject: Re: [Sursound] about principled rendering of ambisonic to
>>>  binaural
>>> Message-ID:
>>>  <20220913135949.ugwflytibwa7p...@mail1.linuxaudio.cyso.net>
>>> Content-Type: text/plain; charset=iso-8859-1
>>> 
>>> 
>> 
>> [Snip]
>> 
>>> Another question is if for high quality binaural rendering, starting from
>>> Ambisonic content is a good idea at all.
>>> 
>>> Simple fact is that if you want really good results you need very high
>>> order, and
>>> 
>>> 1. such content isn't available from direct recordings (we don't have
>> even
>>> 10th order microphpones), so it has to be synthetic,
>>> 
>>> 2. rendering it from an Ambisonic format would be very inefficient. For
>>> example for order 20 you'd need 441 convolutions if you assume L/R head
>>> symmetry, twice that number if you don't.
>>> 
>>> Compare this to rendering from object encoded content (i.e. mono signals
>>> plus directional metadata). You need only two convolutions per object.
>>> Starting from a sufficiently dense HRIR set, you can easily generate a
>>> new set on a regular grid with a few thousand points, and interpolate
>>> them (VBAP style) in real time. This can give you the same resolution
>>> as e.g. order 40 Ambisonics at fraction of the complexity.
>>> 
>>> 
>>> Ciao,
>>> 
>>> --
>>> FA
>> 
>> ___
>> Sursound mailing list
>> Sursound@music.vt.edu
>> https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here,
>> edit account or options, view archives and so on.
>> 
> -- next part --
> An HTML attachment was scrubbed...
> URL: 
> <https://mail.music.vt.edu/mailman/private/sursound/attachments/20220917/acff5f4e/attachment.htm>
> ___
> Sursound mailing list
> Sursound@music.vt.edu
> https://mail.music.vt.edu/mailman/listi

Re: [Sursound] about principled rendering of ambisonic to binaural

[David McKevy]

Ralph you mentioned wanting a Mac based VST so I assume you have access to
Logic and can try their binaural decoder?

It's been a while since I worked with channel based surround but I assume
that by now they may have varying HRTF's...

Just a thought...

On Sat, Sep 17, 2022, 8:52 PM Ralph Jones  wrote:

> I’m a composer, not a mathematician, so while I try, I don’t get very far
> at understanding discussions like this. But the subject is of real concern
> for me, because I am currently working in 5.1.4 surround format
> (channel-based, not Atmos) and I would dearly love to find a mac-compatible
> VST plugin that would convincingly render my work in binaural. So, is there
> a plugin that does what Fons describes here? (i.e., given azimuth and
> elevation for each channel, render the signals to binaural convincingly,
> including an impression of elevation for height channels.)
>
> Ralph Jones
>
> > On Sep 13, 2022, at 9:00 AM,Fons Adriaensen wrote:
> >
> > Message: 1
> > Date: Tue, 13 Sep 2022 15:59:49 +0200
> > From: Fons Adriaensen 
> > To: sursound@music.vt.edu
> > Subject: Re: [Sursound] about principled rendering of ambisonic to
> >   binaural
> > Message-ID:
> >   <20220913135949.ugwflytibwa7p...@mail1.linuxaudio.cyso.net>
> > Content-Type: text/plain; charset=iso-8859-1
> >
> >
>
> [Snip]
>
> > Another question is if for high quality binaural rendering, starting from
> > Ambisonic content is a good idea at all.
> >
> > Simple fact is that if you want really good results you need very high
> > order, and
> >
> > 1. such content isn't available from direct recordings (we don't have
> even
> > 10th order microphpones), so it has to be synthetic,
> >
> > 2. rendering it from an Ambisonic format would be very inefficient. For
> > example for order 20 you'd need 441 convolutions if you assume L/R head
> > symmetry, twice that number if you don't.
> >
> > Compare this to rendering from object encoded content (i.e. mono signals
> > plus directional metadata). You need only two convolutions per object.
> > Starting from a sufficiently dense HRIR set, you can easily generate a
> > new set on a regular grid with a few thousand points, and interpolate
> > them (VBAP style) in real time. This can give you the same resolution
> > as e.g. order 40 Ambisonics at fraction of the complexity.
> >
> >
> > Ciao,
> >
> > --
> > FA
>
> ___
> Sursound mailing list
> Sursound@music.vt.edu
> https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here,
> edit account or options, view archives and so on.
>
-- next part --
An HTML attachment was scrubbed...
URL: 
<https://mail.music.vt.edu/mailman/private/sursound/attachments/20220917/acff5f4e/attachment.htm>
___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.

Re: [Sursound] about principled rendering of ambisonic to binaural

[Ralph Jones]

I’m a composer, not a mathematician, so while I try, I don’t get very far at 
understanding discussions like this. But the subject is of real concern for me, 
because I am currently working in 5.1.4 surround format (channel-based, not 
Atmos) and I would dearly love to find a mac-compatible VST plugin that would 
convincingly render my work in binaural. So, is there a plugin that does what 
Fons describes here? (i.e., given azimuth and elevation for each channel, 
render the signals to binaural convincingly, including an impression of 
elevation for height channels.)

Ralph Jones

> On Sep 13, 2022, at 9:00 AM,Fons Adriaensen wrote:
> 
> Message: 1
> Date: Tue, 13 Sep 2022 15:59:49 +0200
> From: Fons Adriaensen 
> To: sursound@music.vt.edu
> Subject: Re: [Sursound] about principled rendering of ambisonic to
>   binaural
> Message-ID:
>   <20220913135949.ugwflytibwa7p...@mail1.linuxaudio.cyso.net>
> Content-Type: text/plain; charset=iso-8859-1
> 
> 

[Snip]

> Another question is if for high quality binaural rendering, starting from
> Ambisonic content is a good idea at all. 
> 
> Simple fact is that if you want really good results you need very high
> order, and
> 
> 1. such content isn't available from direct recordings (we don't have even
> 10th order microphpones), so it has to be synthetic,
> 
> 2. rendering it from an Ambisonic format would be very inefficient. For
> example for order 20 you'd need 441 convolutions if you assume L/R head
> symmetry, twice that number if you don't.
> 
> Compare this to rendering from object encoded content (i.e. mono signals 
> plus directional metadata). You need only two convolutions per object.
> Starting from a sufficiently dense HRIR set, you can easily generate a
> new set on a regular grid with a few thousand points, and interpolate 
> them (VBAP style) in real time. This can give you the same resolution
> as e.g. order 40 Ambisonics at fraction of the complexity.
> 
> 
> Ciao,
> 
> -- 
> FA

___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.

Re: [Sursound] about principled rendering of ambisonic to binaural

[Fons Adriaensen]

On Sun, Sep 11, 2022 at 07:21:50PM +0300, Sampo Syreeni wrote:

> If the directional
> sampling was statistically uniform over the whole sphere of directions, and
> in addition the sample of directions probed was to be in quadrature, it
> would be an easy excercise in discrete summation to gain the transform
> matrix we need. 

Even if that case it isn't as simple as you seem to think.

Any set of measured HRIR will need some non trivial preprocessing before
it can be used. One reason is low-frequency errors. Accurate IR measurements
below say 200 Hz are difficult (unless you have a very big and good anechoic
room). OTOH we know that HRIR in that frequency range are very low order and
can be synthesised quite easily.

Another reason is that you can't reduce a set of HRIR to low order (the order
of the content you want to render) without introducing significant new errors.
One way to reduce these is to reduce or even fully remove ITD at mid and high
frequencies, again depending on the order the renderer is supposed to support.
Getting the magnitudes (and hence ILD) accurate requires much lower order than
if you also want to keep the delays.

Compared to these and some other issues, not having a set on a regular grid
(e.g. t-design or Lebedev) is the least of problems you will encounter.

There are other considerations. For best results you need head tracking
and a plausible room sound (even if the content already includes its own).

> So the best framework I could think of, years past, was to try and
> interpolate the incoming directional point cloud from the KEMAR and other
> sets, to the whole sphere, and then integrate. Using a priori knowledge for
> the edge, singular cases, where a number of the empirical observations prove
> to be co-planar, and as such singular in inversion. I tried stuff such as
> information theoretical Kullback-Leibner divergence, and Vapnik-Cervonenkis
> dimension, in order to pare down the stuff. The thing I settled on was a
> kind of mutual recursion between the directional mutual information between
> empirical point gained/removed and Mahalanobis distance to each spherical
> harmonic added/removed. It ought to have worked.

The practical solutions do not depend on such concepts and are much more
ad-hoc. Some members of my team and myself worked on them for the last
three years. Most of the results are confidential, although others (e.g.
IEM) have arrived at some similar results and published them.  

Another question is if for high quality binaural rendering, starting from
Ambisonic content is a good idea at all. 

Simple fact is that if you want really good results you need very high
order, and

1. such content isn't available from direct recordings (we don't have even
10th order microphpones), so it has to be synthetic,

2. rendering it from an Ambisonic format would be very inefficient. For
example for order 20 you'd need 441 convolutions if you assume L/R head
symmetry, twice that number if you don't.

Compare this to rendering from object encoded content (i.e. mono signals 
plus directional metadata). You need only two convolutions per object.
Starting from a sufficiently dense HRIR set, you can easily generate a
new set on a regular grid with a few thousand points, and interpolate 
them (VBAP style) in real time. This can give you the same resolution
as e.g. order 40 Ambisonics at fraction of the complexity.

 
Ciao,

-- 
FA

___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.

Re: [Sursound] about principled rendering of ambisonic to binaural

[Picinali, Lorenzo]

Hello,

I don't know if this is of help, as it doesn't deal directly with the matter 
you raised about the uniformity of the HRTF measurements, but...I'll anyway 
mention it:

https://acta-acustica.edpsciences.org/articles/aacus/abs/2022/01/aacus210029/aacus210029.html

Best
Lorenzo




--
Dr Lorenzo Picinali
Reader in Audio Experience Design<https://www.axdesign.co.uk/>
Dyson School of Design Engineering
Imperial College London
Dyson Building
Imperial College Road
South Kensington, SW7 2DB, London
E: l.picin...@imperial.ac.uk

http://www.imperial.ac.uk/people/l.picinali
https://www.axdesign.co.uk/

From: Sursound  on behalf of Sampo Syreeni 

Sent: 11 September 2022 17:21
To: sursound-list 
Cc: music-dsp list 
Subject: [Sursound] about principled rendering of ambisonic to binaural


***
This email originates from outside Imperial. Do not click on links and 
attachments unless you recognise the sender.
If you trust the sender, add them to your safe senders list 
https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for 
this address.
***
Hi y'all, for the longest time now... There's been a lot of discussion
about rendering ambisonic soundfields down to binaural of late, and in
the past couple of years. I don't really think this is a problem that
has been solved in any principled fashion as of yet, so I'd like to
invite some discussion. Especially since I once was about to try my hand
at the problem, but found my skills woefully lacking.

AFAICS, the thing here is to have a set of HRTF measurements -- the
well-known and open KEMAR set, but also any other -- and then to derive
from it an LTI coupling from a representation of the soundfield to two
ears perceiving the field. The representation ought to be isotropic, as
per basic ambisonic principles, and it ought to be matched to the order
of the ambisonic field. If you had a neat set of measurements, over the
whole sphere of directions, which was designed to be in perfect
quadrature, this would be easy as cheese.

The trouble is that no set of measurements really behaves this way.
They're not in quadrature at all, and almost *always* you'll have a
sparsity, or even a full gap, towards the direction straight down. If
the directional sampling was statistically uniform over the whole
sphere of directions, and in addition the sample of directions probed
was to be in quadrature, it would be an easy excercise in discrete
summation to gain the transform matrix we need. But now it very much
isn't.

It truly isn't so when you have those gaps of coverage in the HRTF data
to the above, and especially below. It leads to divergent, numerically
touchy problems in *very* high dimension: if even one of your points
in the KEMAR set happens to be out of perfect quadrature, you're led
to an infinite order contribution from that one data point.

It also doesn't help that, directionally speaking, our known HRTF/HRIF's
don't really come in quadrature, so that they actually contribute to
directional aliasing, *statistically*. To negate their individual
error contributions out, to a degree. But then, again, I know of *no*
global, stochastic error metric out there, nor any optimization
strategy, proven to be optimal for this sort of optimization task.

So the best framework I could think of, years past, was to try and
interpolate the incoming directional point cloud from the KEMAR and
other sets, to the whole sphere, and then integrate. Using a priori
knowledge for the edge, singular cases, where a number of the empirical
observations prove to be co-planar, and as such singular in inversion. I
tried stuff such as information theoretical Kullback-Leibner divergence,
and Vapnik-Cervonenkis dimension, in order to pare down the stuff. The
thing I settled on was a kind of mutual recursion between the
directional mutual information between empirical point gained/removed
and Mahalanobis distance to each spherical harmonic added/removed. It
ought to have worked.

But it didn't. My heuristic, even utilizing exhaustive search at points,
didn't cut it even close. It didn't even approach what Gerzon did
analytically in 4.0 or 5.1.

So, any better ideas on how to interpolate and integrate, using ex ante
knowledge? In order to go from arbitrary point clouds to regularized,
isotropic, optimized, ambisonic -> binaural mappings?
--
Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front
+358-40-3751464, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2
___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.
-- next part --
An HTML attachment was scrubbed...
URL: 
<https://mail.music.vt.edu/mailman/private/sursound/attachments/2

[Sursound] about principled rendering of ambisonic to binaural

[Sampo Syreeni]

Hi y'all, for the longest time now... There's been a lot of discussion 
about rendering ambisonic soundfields down to binaural of late, and in 
the past couple of years. I don't really think this is a problem that 
has been solved in any principled fashion as of yet, so I'd like to 
invite some discussion. Especially since I once was about to try my hand 
at the problem, but found my skills woefully lacking.


AFAICS, the thing here is to have a set of HRTF measurements -- the 
well-known and open KEMAR set, but also any other -- and then to derive 
from it an LTI coupling from a representation of the soundfield to two 
ears perceiving the field. The representation ought to be isotropic, as 
per basic ambisonic principles, and it ought to be matched to the order 
of the ambisonic field. If you had a neat set of measurements, over the 
whole sphere of directions, which was designed to be in perfect 
quadrature, this would be easy as cheese.


The trouble is that no set of measurements really behaves this way. 
They're not in quadrature at all, and almost *always* you'll have a 
sparsity, or even a full gap, towards the direction straight down. If 
the directional sampling was statistically uniform over the whole 
sphere of directions, and in addition the sample of directions probed 
was to be in quadrature, it would be an easy excercise in discrete 
summation to gain the transform matrix we need. But now it very much 
isn't.


It truly isn't so when you have those gaps of coverage in the HRTF data 
to the above, and especially below. It leads to divergent, numerically 
touchy problems in *very* high dimension: if even one of your points 
in the KEMAR set happens to be out of perfect quadrature, you're led 
to an infinite order contribution from that one data point.


It also doesn't help that, directionally speaking, our known HRTF/HRIF's 
don't really come in quadrature, so that they actually contribute to 
directional aliasing, *statistically*. To negate their individual 
error contributions out, to a degree. But then, again, I know of *no* 
global, stochastic error metric out there, nor any optimization 
strategy, proven to be optimal for this sort of optimization task.


So the best framework I could think of, years past, was to try and 
interpolate the incoming directional point cloud from the KEMAR and 
other sets, to the whole sphere, and then integrate. Using a priori 
knowledge for the edge, singular cases, where a number of the empirical 
observations prove to be co-planar, and as such singular in inversion. I 
tried stuff such as information theoretical Kullback-Leibner divergence, 
and Vapnik-Cervonenkis dimension, in order to pare down the stuff. The 
thing I settled on was a kind of mutual recursion between the 
directional mutual information between empirical point gained/removed 
and Mahalanobis distance to each spherical harmonic added/removed. It 
ought to have worked.


But it didn't. My heuristic, even utilizing exhaustive search at points, 
didn't cut it even close. It didn't even approach what Gerzon did 
analytically in 4.0 or 5.1.


So, any better ideas on how to interpolate and integrate, using ex ante 
knowledge? In order to go from arbitrary point clouds to regularized, 
isotropic, optimized, ambisonic -> binaural mappings?

--
Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front
+358-40-3751464, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2
___
Sursound mailing list
Sursound@music.vt.edu
https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit 
account or options, view archives and so on.