I am working on an Activity for the One Laptop Per Child project. As
you may know, OLPC activities often use pygtk. My activity is for
reading Project Gutenberg Etexts, and I have been asked to add text to
speech to that Activity using speech-dispatcher. The idea is to have
karaoke-style highlighting, so that as each word is spoken it is also
highlighted in the text. This should help children learn to read. To
do this I have to synchronize highlighting and speaking.
Speech-dispatcher can do callbacks when it is finished speaking, so my
plan was to use gobject.timeout_add() to call a function at regular
intervals. This function would check a flag that would be set to True
if speech-dispatcher was finished speaking. If True I would highlight
and speak the next word, if false I would not. I would use
speech-dispatcher callbacks to set the flag to True every time it
finished speaking.
What I found out was that speech-dispatcher would not consistently
execute the callbacks. When I pointed this out to the speech-dispatcher
mailing list one of the developers recreated my problem, then found that
it he just executed my method in a loop rather than using
gobject.timeout_add() that the code in speech-dispatcher worked
perfectly and all callbacks got executed. He is convinced that using
gobject.timeout_add() interferes with socket activities in other
threads. I have forwarded his email below.
Unfortunately, running that code in a loop is less than optimum. I need
to be able to turn speech on and off, which I can't do in a simple
loop. I need gobject.timeout_add() to work, or something similar, to
get text to speech with highlighting.
I would appreciate any suggestions you have to offer.
James Simmons
--- Begin Message ---
The code is attached. This is not an OLPC Activity but a standalone
program I use to try out new features
Hello James,
I have tried to run the code you've sent and I can reliably reproduce
the issue. I believe it has something to do with the way how you call
speechd.client.speak() from inside the gtk loop, because outside of it,
everything works normally (even in your application).
I'm not familiar with PyGTK however, so I don't see what exactly is
going wrong. I'll try to describe my findings in hope you or someone
else will have someidea.
As you describe, in your program, the SSIP Python interface often fails
to deliver a callback to the client application (in fact, it freezes and
no other communication with Speech Dispatcher is possible). The exact
reason is that in self._socket.recv(1024) (socket.socket.recv()) fails
to return although the logs of Speech Dispatcher prove some new data
have been written to the socket.
I found that this happens when cb_next_word is
called as a callback from keypress_cb() via
timeout_id = gobject.timeout_add(100, self.next_word_cb)
but it doesn't happen, if I call it directly (around line 75):
self.karaoke = True
self.finished_flag = True
#timeout_id = gobject.timeout_add(100,
self.next_word_cb)
while True:
self.next_word_cb()
time.sleep(0.1);
return True
This variant will reliably speak the whole document word-by-word with
all the expected callbacks received.
Another interesting observation is that the mentioned _socket.recv(1024)
returns and correctly receives all the pending callbacks just after I
hit CTRL-C on the python interpreter running ReadETextsActivity.py and
thus kill the main program thread.
Now, the _socket.recv() function in the python speechd library runs in a
separate thread that is being launched from the Client object
constructor. So apparently gobject.timeout_add() somehow disturbs socket
opperations in other threads.
It seems really strange to me and quite unlikely that the python
bindings themselves would be responsible.
With regards,
Hynek Hanke
--- End Message ---
_______________________________________________
pygtk mailing list pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://www.async.com.br/faq/pygtk/
