On 10/31/2014 02:21 AM, smaug wrote:
Intent to ship is too strong for this.
We need to first have implementation landed and tested ;)
I wouldn't ship the implementation in desktop FF without plenty of more testing.
But I guess the question is what people think about shipping the pocketspinx +
API, even if disabled by default.
Andre, we need some numbers here. How much does Pocketsphinx increase binary
size? or download size?
When the pref is enabled, how much does it use memory on desktop, what about on
b2g?
-Olli
On 10/31/2014 01:18 AM, Andre Natal wrote:
I've been researching speech recognition in Firefox for two years. First
SpeechRTC, then emscripten, and now Web Speech API with CMU pocketsphinx
[1] embedded in Gecko C++ layer, project that I had the luck to develop for
Google Summer of Code with the mentoring of Olli Pettay, Guilherme
Gonçalves, Steven Lee, Randell Jesup plus others and with the management of
Sandip Kamat.
The implementation already works in B2G, Fennec and all FF desktop
versions, and the first language supported will be english. The API and
implementation are in conformity with W3C standard [2]. The preference to
enable it is: media.webspeech.service.default = pocketsphinx
The required patches for achieve this are:
- Import pocketsphinx sources in Gecko. Bug 1051146 [3]
- Embed english models. Bug 1065911 [4]
- Change SpeechGrammarList to store grammars inside SpeechGrammar objects.
Bug 1088336 [5]
- Creation of a SpeechRecognitionService for Pocketsphinx. Bug 1051148 [6]
Also, other important features that we don't have patches yet:
- Relax VAD strategy to be les strict and avoid stop in the middle of
speech when speaking low volume phonemes [7]
- Integrate or develop a grapheme to phoneme algorithm to realtime
generator when compiling grammars [8]
- Inlcude and build models for other languages [9]
- Continuous and wordspotting recognition [10]
The wip repo is here [11] and this Air Mozilla video [12] plus this wiki
has more detailed info [13].
At this comment you can see a cpu usage on flame while recognition is
happening [14]
I wish to hear your comments.
Thanks,
Andre Natal
[1] http://cmusphinx.sourceforge.net/
[2] https://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=1051146
[4] https://bugzilla.mozilla.org/show_bug.cgi?id=1065911
[5] https://bugzilla.mozilla.org/show_bug.cgi?id=1088336
[6] https://bugzilla.mozilla.org/show_bug.cgi?id=1051148
[7] https://bugzilla.mozilla.org/show_bug.cgi?id=1051604
[8] https://bugzilla.mozilla.org/show_bug.cgi?id=1051554
[9] https://bugzilla.mozilla.org/show_bug.cgi?id=1065904 and
https://bugzilla.mozilla.org/show_bug.cgi?id=1051607
[10] https://bugzilla.mozilla.org/show_bug.cgi?id=967896
[11] https://github.com/andrenatal/gecko-dev
[12] https://air.mozilla.org/mozilla-weekly-project-meeting-20141027/ (Jump
to 12:00)
[13] https://wiki.mozilla.org/SpeechRTC_-_Speech_enabling_the_open_web
[14] https://bugzilla.mozilla.org/show_bug.cgi?id=1051148#c14
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
