The sync issue to video raises a number of questions, for which the question of a tracker and reaper HOA decoder are secondary to me.
First, is the video individual or projected? Is this HMD/GoogleCardboard type situation? If the first, what are the audio/video sync latency tolerances for the piece? Second, is the sound scene a "shared" experience, in that do the actions of one person affect the sound heard by another? Is the scene dynamically generated/rendered in real-time or is this basic off-line rendered audio HOA playback? Single point of view in the sound scene, or does each listener have their own view, and it is fixed or moving with the user's position? If the audio is pre-rendered, without data upload from the user to the server, then a local HOA player started by a sync signal should suffice. This is far easier then hosting the audio on a server (REAPER) for which the tracker info needs to get from the user to the server, decoded, then sent back. Having more than just a few wireless streams can be difficult, and broadcast streaming of HOA has issues, as true "broadcast" doesn't exist in WiFi the same as it does for Wired LAN (i.e. don't bet on it being able to send audio with low latency). We had some similar questions on a project almost a decade ago now (see references below) where a shared sound-scene was to be shared to multiple users in a common (gallery) setting. The solution was local mini-PCs decoding the Ambisonic stream with the local head-tracker. If there is no upstream data, and only "play" commands, this would be a better solution in my mind. - N. Mariette, B. Katz, K. Boussetta, and O. Guillerminet, “SoundDelta : a study of audio augmented reality using WiFi-distributed Ambisonic cell rendering,” in Audio Eng Soc Conv 128, (London), pp. 1–15, 2010. - N. Mariette and B. Katz, “SoundDelta - Largescale, multi-user audio augmented reality,” in EAA Symp. on Auralization, (Espoo), pp. 1–6, 2009. - http://remu.fr/remu/realisations/sound-delta/ For the plug-in, if the tracking is coming in via OSC, or midi, or most other means, I would think that you could redirect this control signal to different HOA-binaural converters (if you use a client-server architecture and send the binaural audio from the server). Of course, if you want to go beyond HOA and do dynamic binaural rendering, I offer a small bit of promo for Anaglyph, our research developed binaural VST plug-in: http://anaglyph.dalembert.upmc.fr/ Best of luck, -- Brian FG Katz, Ph.D, HDR Research Director, CNRS Groupe Lutheries - Acoustique - Musique Sorbonne Université, CNRS, UMR 7190, Institut Jean Le Rond ∂'Alembert bureau 510, 5ème, allé des tours 55-65 4, Place Jussieu 75005 Paris Tel: (+33) 01 44 27 80 34 web_perso: http://www.dalembert.upmc.fr/home/katz web_lab: http://www.dalembert.upmc.fr web_group: http://www.dalembert.upmc.fr/lam -----Original Message----- From: Simon Connor <sico...@gmail.com> To: sursound@music.vt.edu Subject: Re: [Sursound] Multi-user head tracked binaural ambisonics query Message-ID: <cacmg1nmqdv0cjdzrzho0t6ragqt5fovpok4i37et_pffjf9...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" Thanks for the swift reply Lorenzo! This is a great start. I shall check these out, one key thing I should have mentioned is that the audio feed needs to be synced to video (hence the Reaper functionality - as this will simultaneously run the audio and video). So the audio will need to be streamed to all users synchronously. I'm not sure if this has an implication on the use of mobile phones or not? Cheers Simon _______________________________________________ Sursound mailing list Sursound@music.vt.edu https://mail.music.vt.edu/mailman/listinfo/sursound - unsubscribe here, edit account or options, view archives and so on.
