I was able to match a single "a" - although even with a straight "a" there can be some subtle variation. So I mapped variations that come close and I don't need to match every value in the complete waveform over time... every couple together or even the first value with buffer comes pretty close. this is with a known, unchanging vocal waveform. So I doubt this would be very useful outside of this current system, which is a bummer.
I think it's time for me to retire this code and move on. Oh well... Eric On Fri, Jun 4, 2010 at 9:28 AM, Eric E. Dolecki <edole...@gmail.com> wrote: > I can get waveforms... but say "a" takes 1 second to speak. I get different > waveforms over that 1 second... so I'm not matching against a single > waveform, but many waveforms in succession. This seems like a tricky thing > to match against. > > What might be a good approach to matching values over a certain amount of > time? Is AS3 fast enough to sync quick enough? I imagine it would need to > check for all vowels every frame matching values in waveforms over a certain > amount of time. > > Eric > > > On Fri, Jun 4, 2010 at 8:56 AM, Eric E. Dolecki <edole...@gmail.com>wrote: > >> I've started implementing some code this morning in the hopes to match the >> vowel "a" this morning. Of course there are several intonations for this >> depending on the word it's located in, but if I can get a match on a naked >> "a" I may be on to something. Like you said, I have a higher chance of >> success since the voice is software generated and not from random people's >> speech patterns. >> >> If I don't get something today I'm going to bail on the engine in the >> hopes of finding something useful some other time. This isn't a critical >> feature for me as I have the jaw moving with precision and the effect comes >> across. Mouth shapes would be the icing on the cake. >> >> Eric >> >> >> On Fri, Jun 4, 2010 at 8:34 AM, Karim Beyrouti <ka...@kurst.co.uk> wrote: >> >>> Yeh - not sure this will help >>> >>> however - a (very talented) colleague of mine worked on a simple speech >>> recognition software for mobile - it was built to recognise about 20 >>> commands with 90% success rate. >>> >>> His approach (in my simplistic terms) was: >>> >>> 1) get recordings / audio samples of the commands (in your case vowels - >>> it should be easier as it's generated so you wont have to compare against >>> too many/different intonations ) - >>> 2) create / store a graph of the audio commands ( this used FFT (s) - to >>> abstract and simplify, the pattern of the commands - the result was a square >>> voice print graph ) >>> 3) The stored patterns/voiceprints were then compared against the users >>> voice recording. >>> >>> The trickiest part of this whole business were the Fast Fourier >>> Transforms - these things get very complicated, and confuse the life out of >>> me. Anyway, hopefully this >>> will help you - seems like it might be the best approach. if you do crack >>> it - you will end up with a simple voice recognition system. Which would be >>> a brilliant and useful thing bit of code to >>> have... >>> >>> hope this was of any use.. >>> >>> - karim >>> >>> On 4 Jun 2010, at 01:23, Karl DeSaulniers wrote: >>> >>> > I would try using that to figure out a way of maping the sounds and >>> then translate that to your project. You are able to see the wave forms in >>> soundbooth? Haven't used it. If so, can you run your cursor over it at any >>> point to get the readings? Might be a little trivial, but may yeild a >>> pattern that you can utilize. >>> > >>> > JAT >>> > >>> > Karl >>> > >>> > Sent from losPhone >>> > >>> > On Jun 3, 2010, at 6:18 PM, "Eric E. Dolecki" <edole...@gmail.com> >>> wrote: >>> > >>> >> SoundBooth >>> >> >>> >> On Thu, Jun 3, 2010 at 6:39 PM, Karl DeSaulniers < >>> k...@designdrumm.com>wrote: >>> >> >>> >>> Do you have SoundEdit? Or the like? >>> >>> >>> >>> >>> >>> Karl >>> >>> >>> >>> >>> >>> >>> >>> On Jun 3, 2010, at 5:09 PM, Eric E. Dolecki wrote: >>> >>> >>> >>> I think I might make waveform bitmaps and then try and compare >>> against the >>> >>>> current waveform (block EQ) - and if it's a close match, then fire >>> off >>> >>>> specific vowel events. If that works, I could do consonants too. If >>> this >>> >>>> works, I'll do jumping jacks and shots of Jack. >>> >>>> >>> >>>> So how would I compare two bitmaps to see if a waveform ( >>> >>>> On Thu, Jun 3, 2010 at 5:18 PM, Karl DeSaulniers < >>> k...@designdrumm.com >>> >>>>> wrote: >>> >>>> >>> >>>> If you need any of these files or can't find them, lmk and I can >>> send off >>> >>>>> list. >>> >>>>> >>> >>>>> Best, >>> >>>>> >>> >>>>> Karl >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> On Jun 3, 2010, at 3:37 PM, Karl DeSaulniers wrote: >>> >>>>> >>> >>>>> Don't know if this will help, but have you looked into >>> WaveAnalyzer.as >>> >>>>> or >>> >>>>> >>> >>>>>> Flash MX - Audio: Sound completion event (The source files for >>> this can >>> >>>>>> be >>> >>>>>> found in the Flash MX/Samples folder.) >>> >>>>>> They both let you control the sound. I am thinking this will point >>> you >>> >>>>>> in >>> >>>>>> a good direction. Its AS2 though. >>> >>>>>> >>> >>>>>> HTH, >>> >>>>>> >>> >>>>>> Karl >>> >>>>>> >>> >>>>>> >>> >>>>>> On Jun 3, 2010, at 2:42 PM, Eric E. Dolecki wrote: >>> >>>>>> >>> >>>>>> Ya - I have the data for both things, but they extend over time >>> and are >>> >>>>>> >>> >>>>>>> difficult to compare. It's the boiling down the signatures into >>> >>>>>>> something >>> >>>>>>> simple and being able to read the playing audio looking for the >>> match >>> >>>>>>> (or >>> >>>>>>> near match). I thought about using bitmap data and trying to >>> match up >>> >>>>>>> waveforms, etc. but I don't know enough about it to pull that >>> off. It >>> >>>>>>> seems >>> >>>>>>> like a hack in a way, but if it worked, who cares I suppose. >>> >>>>>>> >>> >>>>>>> On Thu, Jun 3, 2010 at 3:31 PM, Juan Pablo Califano < >>> >>>>>>> califa010.flashcod...@gmail.com> wrote: >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> >>>>>>>>>>> I'm not Henrik, but I've done some lip-synch stuff for >>> Disney. We >>> >>>>>>>> did >>> >>>>>>>> it pretty much the way Eric described--we just used amplitude. >>> It's >>> >>>>>>>> not as accurate as Disney would demand on a film, but it's ok in >>> the >>> >>>>>>>> kids' game market. >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>>>>> I see, amplitudes could be just good enough for some stuff. >>> >>>>>>>> >>> >>>>>>>> Although the "speed" and the intensitiy of the speech could give >>> >>>>>>>> misleading >>> >>>>>>>> results, I think. I'm under the impression that you should >>> somehow try >>> >>>>>>>> to >>> >>>>>>>> compare the shape of the waves (somehow simplifiy your input to >>> some >>> >>>>>>>> value >>> >>>>>>>> of sets of values that are easier to compare, possibly in a >>> "time >>> >>>>>>>> window") >>> >>>>>>>> and compare it in some meaningful way to precalculated samples >>> to find >>> >>>>>>>> a >>> >>>>>>>> matching pattern. That's the part I have no clue about! >>> >>>>>>>> >>> >>>>>>>> Cheers >>> >>>>>>>> Juan Pablo Califano >>> >>>>>>>> >>> >>>>>>>> 2010/6/3 Kerry Thompson <al...@cyberiantiger.biz> >>> >>>>>>>> >>> >>>>>>>> Juan Pablo Califano wrote: >>> >>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> Wow. That was really uncalled for. >>> >>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>> That was my reaction, too. I didn't see Eric as >>> complaining--just >>> >>>>>>>>> asking. Maybe Henrik was just having a bad day. >>> >>>>>>>>> >>> >>>>>>>>> For me, the hard part, which you seem to imply is rather simple >>> >>>>>>>>> here, >>> >>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> is >>> >>>>>>>>> >>> >>>>>>>> >>> >>>>>>>> *matching+ the input audio against said profiles. Admitedly, I >>> don't >>> >>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> know >>> >>>>>>>>> >>> >>>>>>>> >>> >>>>>>>> anything about digital signal processing and audio programming >>> in >>> >>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> general, >>> >>>>>>>>> >>> >>>>>>>>> but "matching" sounds a bit vague. Perhaps you could enlighten >>> us, I >>> >>>>>>>>>> >>> >>>>>>>>>> you >>> >>>>>>>>> >>> >>>>>>>> >>> >>>>>>>> feel like. >>> >>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>>> >>> >>>>>>>>> I'm not Henrik, but I've done some lip-synch stuff for Disney. >>> We did >>> >>>>>>>>> it pretty much the way Eric described--we just used amplitude. >>> It's >>> >>>>>>>>> not as accurate as Disney would demand on a film, but it's ok >>> in the >>> >>>>>>>>> kids' game market. >>> >>>>>>>>> >>> >>>>>>>>> Doing something more accurate would probably involve at least 6 >>> mouth >>> >>>>>>>>> positions, and if you're doing it in real time, you'd have to >>> do a >>> >>>>>>>>> reverse FFT. It can be done--there was a really good commercial >>> >>>>>>>>> lip-synch program that generated Action Script to control mouth >>> >>>>>>>>> positions. I don't know if it's still around--that was 5 years >>> ago, >>> >>>>>>>>> and it was pretty expensive (about $2,500 for one seat, I >>> think). It >>> >>>>>>>>> may even have been a Director Xtra that worked with a Flash >>> Sprite, >>> >>>>>>>>> but let's not talk about Director :-P >>> >>>>>>>>> >>> >>>>>>>>> Cordially, >>> >>>>>>>>> >>> >>>>>>>>> Kerry Thompson >>> >>>>>>>>> _______________________________________________ >>> >>>>>>>>> Flashcoders mailing list >>> >>>>>>>>> Flashcoders@chattyfig.figleaf.com >>> >>>>>>>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> >>>>>>>>> >>> >>>>>>>>> _______________________________________________ >>> >>>>>>>>> >>> >>>>>>>> Flashcoders mailing list >>> >>>>>>>> Flashcoders@chattyfig.figleaf.com >>> >>>>>>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>> >>> >>>>>>> -- >>> >>>>>>> http://ericd.net >>> >>>>>>> Interactive design and development >>> >>>>>>> _______________________________________________ >>> >>>>>>> Flashcoders mailing list >>> >>>>>>> Flashcoders@chattyfig.figleaf.com >>> >>>>>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> >>>>>>> >>> >>>>>>> >>> >>>>>> Karl DeSaulniers >>> >>>>>> Design Drumm >>> >>>>>> http://designdrumm.com >>> >>>>>> >>> >>>>>> _______________________________________________ >>> >>>>>> Flashcoders mailing list >>> >>>>>> Flashcoders@chattyfig.figleaf.com >>> >>>>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> >>>>>> >>> >>>>>> >>> >>>>> Karl DeSaulniers >>> >>>>> Design Drumm >>> >>>>> http://designdrumm.com >>> >>>>> >>> >>>>> _______________________________________________ >>> >>>>> Flashcoders mailing list >>> >>>>> Flashcoders@chattyfig.figleaf.com >>> >>>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> >>>>> >>> >>>>> >>> >>>> >>> >>>> >>> >>>> -- >>> >>>> http://ericd.net >>> >>>> Interactive design and development >>> >>>> _______________________________________________ >>> >>>> Flashcoders mailing list >>> >>>> Flashcoders@chattyfig.figleaf.com >>> >>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> >>>> >>> >>> >>> >>> Karl DeSaulniers >>> >>> Design Drumm >>> >>> http://designdrumm.com >>> >>> >>> >>> _______________________________________________ >>> >>> Flashcoders mailing list >>> >>> Flashcoders@chattyfig.figleaf.com >>> >>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> >>> >>> >> >>> >> >>> >> >>> >> -- >>> >> http://ericd.net >>> >> Interactive design and development >>> >> _______________________________________________ >>> >> Flashcoders mailing list >>> >> Flashcoders@chattyfig.figleaf.com >>> >> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> > _______________________________________________ >>> > Flashcoders mailing list >>> > Flashcoders@chattyfig.figleaf.com >>> > http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> >>> >>> _______________________________________________ >>> Flashcoders mailing list >>> Flashcoders@chattyfig.figleaf.com >>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> >> >> >> >> -- >> http://ericd.net >> Interactive design and development >> > > > > -- > http://ericd.net > Interactive design and development > -- http://ericd.net Interactive design and development _______________________________________________ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
