Re: GTK and threaded applications

[Michael Torrie]

On Fri, 2006-02-03 at 23:14 +0100, kornelix wrote:
> If I have understood the implications correctly, then writing 
> multi-threaded applications with GTK may be possible, but I am almost 
> ready to give it up because of the complexity and limitations. I will 
> check how hard it will be to convert my application to QT or X11 Motif.

Writing multi-threaded GTK apps is done all the time and it works well
if you follow the GTK docs guidelines on threading and follow the
instructions in the post by Brian Tarricone, especially the part about
the enter/leave stuff around the gtk_main call and the thread creation
calls.

The lock up problems you've experiences really indicate that there's an
architecture problem in there somewhere.  I have a feeling until your
work out the architecture (worker threads, synchronization queues, IPC),
you'll have the same problems with QT and X11 Motif (especially Motif
since it is much older and I believe has even less threading support).
My first multi-threaded Qt program failed miserably because of my
architecture design.  So in short I think that writing multi-threaded
GUI apps just as difficult on Qt or even Windows.  But once you conquer
these initial issues I think you'll be fine on any toolkit.

If you want to submit a short example of your code to the list (that
illustrates the problems you are encountering) then I'm sure that things
can be worked out in very short order.

Michael


___
gtk-app-devel-list mailing list
gtk-app-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-app-devel-list

Re: GTK and threaded applications

[Tor Lillqvist]

 > > FYI, here is something I pulled from the Microsoft web site on Win32
 > > programming with threads.

 > That's a bit irrelevant: the Win32 API doesn't need to run on different
 > architectures, different OSes, and with different graphics-drawing backends.

Indeed. And actually, I was a bit surprised that Win32 was brought up
as a role model, as it isn't like the Win32 windowing system was
completely without thread-related gotchas. (I am talking about the
core API, now, not any MFC, Window.Forms, or whatever layers on top of
that.) A Win32 window is tightly coupled to the thread that created
it. Many of the Win32 API calls that refer to a window handle must be
done in the same thread as created the window. Window messages are
delivered to the thread that created the window. Etc, this is in no
way straightforward. If you read the window management and messaging
MSDN documentation carefully, you will notice that it mentions
thread-related restrictions quite a lot.

If fact, compared to this, the X11 protocol is much simpler. It is,
after all, just a protocol on top of any bidirectional byte
stream. There is no mention of threads (or even processes, for that
matter) in the X11 requests, replies or events. The X server has no
knowledge of things like that.

--tml

___
gtk-app-devel-list mailing list
gtk-app-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-app-devel-list

Re: GTK and threaded applications

[Brian J. Tarricone]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 2/3/2006 2:14 PM, kornelix wrote:
>  I last reported that GTK was making my multi-threaded application run
> as multiple single threads, one after the other. It was suggested to
> lock only the GTK calls instead of the whole thread. I tried this.
> 
> I wrapped the individual GTK calls as follows:
>   gdk_threads_enter();
>   do GTK calls to update the GUI or get user input
>   gdk_flush();
>   gdk_threads_leave();

That looks reasonable, but remember, you don't wrap *all* gtk calls in
the application.  See below.

> I managed to lock up my system so badly that I had to power off and
> reboot.

It's highly doubtful that GTK caused this.

> I think the core problem is that some of my functions can be
> called from within main() dialogs and also from within thread dialogs.
> Hence calling "gtk_threads_enter()" was likely invalid when being called
> from a dialog that was initiated from a menu created in main(). 
> Correct?  Then I need to write different versions of my functions, with
> and without "gtk_threads_enter()" etc., and make sure the right version
> is being called. Correct? Or perhaps keep track of whether I am in
> main() or a thread, and conditionally execute the above function calls
> (there must be a library function that can tell me if I am in a thread
> or not).

I rarely encounter this problem, but when I do, usually I use a stub
function, like:

void do_important_gui_stuff {
   /* ... */
}

void do_important_gui_stuff_locked() {
gdk_threads_enter();
do_important_gui_stuff();
gdk_threads_leave();
}

It's not beautiful, but it works.  More likely, you can avoid the issue
entirely by rethinking your program flow a bit.

> If I have understood the implications correctly, then writing
> multi-threaded applications with GTK may be possible, but I am almost
> ready to give it up because of the complexity and limitations. I will
> check how hard it will be to convert my application to QT or X11 Motif.

You're looking at this in an overly-complicated way.  Here are a few
guidelines, all of which probably have appropriate documentation somewhere:

1.  Start off your app with this:

g_thread_init(NULL);
gdk_threads_init();
gtk_init(&argc, &argv);

2.  Only call gtk_main() *once*, usually from your main() function in
the main thread.  Surround this with
gdk_threads_enter()/gdk_threads_leave().  If you start up your other
threads in main(), then surround the whole bit.  So you might have
something like:

gdk_threads_enter();
start_other_threads();
win = gtk_window_new(...);
// ...
gtk_widget_show_all(win);
gtk_main();
gdk_threads_leave();

3.  GTK signal handlers are *always* run in the main thread, and are
already inside the GDK lock, so do *not* call _enter()/_leave().

4.  Glib "source" fuctions (e.g., functions run via g_idle_add() or
g_timeout_add(), etc.), are *always* run in the main thread, but are
*not* inside the GDK lock, so gtk/gdk calls in source functions *must*
be surrounded by _enter()/_leave() pairs.

5.  And of course, gtk/gdk calls that you explicitly make in other
threads need to be surrounded by _enter()/_leave() pairs as well.

> FYI, here is something I pulled from the Microsoft web site on Win32
> programming with threads.
[snip]

That's a bit irrelevant: the Win32 API doesn't need to run on different
architectures, different OSes, and with different graphics-drawing backends.

> GTK gurus, please put this on your roadmap. Threads need to work without
> the user having to manage this with extra complexity and the risk of
> subtle bugs.

They do.  You just need to work within some constraints.  Building
multithreaded applications (especially in a language like C or C++) will
always have gotchas and pitfalls.

> GTK functions need to assume they are in a threaded
> environment and do their own locking.

This would increase complexity quite a bit.  How is a GTK function to
know if it's being called inside the lock or not?  Recursive mutexes
would help, but with a performance penalty.  My bet is it would hurt
maintainability of GTK itself quite a bit.

> Making the user worry about this is wrong.

The user doesn't have to worry about it.  The application developer
does.  If you're not familiar with how to the toolkit in a multithreaded
environment, sure, there's a learning curve, but I don't believe the 5
points I mentioned above are an unreasonable burden.

> Since obtaining a lock requires nanoseconds, this should not
> be a performance concern.

Do you have benchmarks to back this up?

Note that the method I outlined above isn't the One True Way.  If you
want to be totally safe, don't let your worker threads touch the GUI at
all.  Communicate with the main (GUI) thread via GAsyncQueues and Glib
idle/timeout functions.  With this method, you don't need to call
gdk_threads_init() or use gdk_threads_enter/leave() at all.

-brian

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (MingW32)

iD8DBQFD490W6XyW6V

Re: GTK and threaded applications

[Daniel Atallah]

On 2/3/06, kornelix <[EMAIL PROTECTED]> wrote:
>   I last reported that GTK was making my multi-threaded application run
> as multiple single threads, one after the other. It was suggested to
> lock only the GTK calls instead of the whole thread. I tried this.
>
> I wrapped the individual GTK calls as follows:
>gdk_threads_enter();
>do GTK calls to update the GUI or get user input
>gdk_flush();
>gdk_threads_leave();
>
> I managed to lock up my system so badly that I had to power off and
> reboot. I think the core problem is that some of my functions can be
> called from within main() dialogs and also from within thread dialogs.
> Hence calling "gtk_threads_enter()" was likely invalid when being called
> from a dialog that was initiated from a menu created in main().
> Correct?  Then I need to write different versions of my functions, with
> and without "gtk_threads_enter()" etc., and make sure the right version
> is being called. Correct? Or perhaps keep track of whether I am in
> main() or a thread, and conditionally execute the above function calls
> (there must be a library function that can tell me if I am in a thread
> or not).
>
> If I have understood the implications correctly, then writing
> multi-threaded applications with GTK may be possible, but I am almost
> ready to give it up because of the complexity and limitations. I will
> check how hard it will be to convert my application to QT or X11 Motif.
>
> FYI, here is something I pulled from the Microsoft web site on Win32
> programming with threads.
>
> << begin Microsoft
>
> MSDN Library
>  > 
> Development
> Tools and Languages
>  > 
> Visual
> Studio
>  > 
> Visual
> C++
>  > 
> Programming
> Guide
>  > 
> General
> Concepts
>  > 
> Multithreading
>  > 
> Multithreading
> with C and Win32
> Visual C++
> Multithreading with C and Win32
>
> Microsoft Visual C++ provides support for creating multithread
> applications with 32-bit versions of Microsoft Windows: Windows XP,
> Windows 2000, Windows NT, Windows Me, and Windows 98. You should
> consider using more than one thread if your application needs to manage
> multiple activities, such as simultaneous keyboard and mouse input. One
> thread can process keyboard input while a second thread filters mouse
> activities. A third thread can update the display screen based on data
> from the mouse and keyboard threads. At the same time, other threads can
> access disk files or get data from a communications port.
>
> With Visual C++, there are two ways to program with multiple threads:
> use the Microsoft Foundation Class (MFC) library or the C run-time
> library and the Win32 API. For information about creating multithread
> applications with MFC, see Multithreading with C++ and MFC
>  after reading
> the following topics about multithreading in C.
>
>  >> end Microsoft
>
> This was my experience writing a multi-threaded application for Win32. I
> did not have to single-thread the Win32 calls themselves, I only had to
> worry about the obvious things like conflicting access to data or two
> threads writing to the same window at the same time.
>
> GTK gurus, please put this on your roadmap. Threads need to work without
> the user having to manage this with extra complexity and the risk of
> subtle bugs. GTK functions need to assume they are in a threaded
> environment and do their own locking. Making the user worry about this
> is wrong. Since obtaining a lock requires nanoseconds, this should not
> be a performance concern.
>
> regards,
> Mike

It seems to me that you're going about this in the wrong way.

When I read the win32 example of three threads above, that is exactly
what you want to be doing.  You should have your worker threads doing
whatever it is that they need to do - read from the network, process
data, whatever, and when you actually want to update the UI based on
these changes, you need to make sure that you do that in the main
thread.

You can easily get your worker threads to trigger an UI update by
using g_idle_add() to schedule a function to be executed in the main
thread at the next main loop iteration.

-D
___
gtk-app-devel-list mailing list
gtk-app-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-app-devel-list

Re: GTK and threaded applications

[kornelix]

 I last reported that GTK was making my multi-threaded application run 
as multiple single threads, one after the other. It was suggested to 
lock only the GTK calls instead of the whole thread. I tried this.


I wrapped the individual GTK calls as follows:
  gdk_threads_enter();
  do GTK calls to update the GUI or get user input
  gdk_flush();
  gdk_threads_leave();

I managed to lock up my system so badly that I had to power off and 
reboot. I think the core problem is that some of my functions can be 
called from within main() dialogs and also from within thread dialogs. 
Hence calling "gtk_threads_enter()" was likely invalid when being called 
from a dialog that was initiated from a menu created in main().  
Correct?  Then I need to write different versions of my functions, with 
and without "gtk_threads_enter()" etc., and make sure the right version 
is being called. Correct? Or perhaps keep track of whether I am in 
main() or a thread, and conditionally execute the above function calls 
(there must be a library function that can tell me if I am in a thread 
or not).


If I have understood the implications correctly, then writing 
multi-threaded applications with GTK may be possible, but I am almost 
ready to give it up because of the complexity and limitations. I will 
check how hard it will be to convert my application to QT or X11 Motif.


FYI, here is something I pulled from the Microsoft web site on Win32 
programming with threads.


<< begin Microsoft

MSDN Library 
 > Development 
Tools and Languages 
 > Visual 
Studio 
 > Visual 
C++ 
 > Programming 
Guide 
 > General 
Concepts 
 > Multithreading 
 > Multithreading 
with C and Win32
Visual C++  
Multithreading with C and Win32  

Microsoft Visual C++ provides support for creating multithread 
applications with 32-bit versions of Microsoft Windows: Windows XP, 
Windows 2000, Windows NT, Windows Me, and Windows 98. You should 
consider using more than one thread if your application needs to manage 
multiple activities, such as simultaneous keyboard and mouse input. One 
thread can process keyboard input while a second thread filters mouse 
activities. A third thread can update the display screen based on data 
from the mouse and keyboard threads. At the same time, other threads can 
access disk files or get data from a communications port.


With Visual C++, there are two ways to program with multiple threads: 
use the Microsoft Foundation Class (MFC) library or the C run-time 
library and the Win32 API. For information about creating multithread 
applications with MFC, see Multithreading with C++ and MFC 
 after reading 
the following topics about multithreading in C.


>> end Microsoft

This was my experience writing a multi-threaded application for Win32. I 
did not have to single-thread the Win32 calls themselves, I only had to 
worry about the obvious things like conflicting access to data or two 
threads writing to the same window at the same time.


GTK gurus, please put this on your roadmap. Threads need to work without 
the user having to manage this with extra complexity and the risk of 
subtle bugs. GTK functions need to assume they are in a threaded 
environment and do their own locking. Making the user worry about this 
is wrong. Since obtaining a lock requires nanoseconds, this should not 
be a performance concern.


regards,
Mike

Michael L Torrie wrote:


On Wed, 2006-02-01 at 10:44 +0100, kornelix wrote:
 

Thanks for your generous help. I will try Michael's suggestion and see 
how it works.


I would like to implement all GTK calls in the main program as Tristan 
suggested, but this seems to be very complex (must implement asynch. 
queues of data going back and forth between threads and main(), where 
screen updates and user inputs take place.


Question: it seems to me that GTK puts the burden of locking (and the 
responsibility to understand GTK internals) in the wrong place: the user 
of GTK. Would it not be better if GTK took care of its own locking and 
blocking in those places where it is necessary? Is this in the roadmap 
for GTK, or should it be?
   



I believe if such locking were made automatic in the GTK  libraries
themselves, we'd be open to possible deadlock situations.  Better to
leave the locking in control of the programmer who can then resolve any
deadlocks in his code.

 

I have written multi-threaded applications in Win32, and I never worried

Re: GTK and threaded applications

[Michael L Torrie]

On Wed, 2006-02-01 at 10:44 +0100, kornelix wrote:
> Thanks for your generous help. I will try Michael's suggestion and see 
> how it works.
> 
> I would like to implement all GTK calls in the main program as Tristan 
> suggested, but this seems to be very complex (must implement asynch. 
> queues of data going back and forth between threads and main(), where 
> screen updates and user inputs take place.
> 
> Question: it seems to me that GTK puts the burden of locking (and the 
> responsibility to understand GTK internals) in the wrong place: the user 
> of GTK. Would it not be better if GTK took care of its own locking and 
> blocking in those places where it is necessary? Is this in the roadmap 
> for GTK, or should it be?

I believe if such locking were made automatic in the GTK  libraries
themselves, we'd be open to possible deadlock situations.  Better to
leave the locking in control of the programmer who can then resolve any
deadlocks in his code.

> 
> I have written multi-threaded applications in Win32, and I never worried 
> about locks and thread blocking (except for my own application's business).

It has been nearly 8 years since I did any Win32 GUI programming, but at
the time I recall that because the Win32 GUI calls had to be
synchronized too, I ended up have a thread just for the gui and had a
message queue that I used to send it messages to change things in the
GUI.  So unless you were using a toolkit that hid the synchronization
details from you (not MFC!) then perhaps you were lucky you didn't have
any problems.

> 
> I have not looked at KDE. Any better?

Qt is now supposed to be thread-safe as of somewhere in version 2 (we're
now up to 3.3 and 4.1).  I don't know how they implement the
synchronization stuff.

Michael



> 
> thanks again,
> Mike


___
gtk-app-devel-list mailing list
gtk-app-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-app-devel-list

Re: GTK and threaded applications

[kornelix]

Thanks for your generous help. I will try Michael's suggestion and see 
how it works.


I would like to implement all GTK calls in the main program as Tristan 
suggested, but this seems to be very complex (must implement asynch. 
queues of data going back and forth between threads and main(), where 
screen updates and user inputs take place.


Question: it seems to me that GTK puts the burden of locking (and the 
responsibility to understand GTK internals) in the wrong place: the user 
of GTK. Would it not be better if GTK took care of its own locking and 
blocking in those places where it is necessary? Is this in the roadmap 
for GTK, or should it be?


I have written multi-threaded applications in Win32, and I never worried 
about locks and thread blocking (except for my own application's business).


I have not looked at KDE. Any better?

thanks again,
Mike


Just put the gdk_* threads and flush calls around the actual GTK calls.
that way only gui updates themselves are put in mutexes (locked).  The
way you are currently doing it, your actual business logic ends up in
the mutex, and so if it is lengthy, it will block all the other threads.

Michael

 




IMO,
 overall you'll have a cleaner program design if you put
all your GUI interaction in the main thread.

Now if I'm reading you right; what your experiencing is normal,
calling gdk_threads_enter/leave is locking a global mutex for
GTK/GDK interaction.

The rule is, you cant have two threads access the GTK+ api
at once, so:
 o You can have a multithreaded application using GTK+
 o You can even go to the extreme of calling GTK+ functions
   from seperate threads, but you must take care to lock
   GTK+ down with gdk_threads_enter/leave()
 o You *cant* call GTK+ functions from more than one thread
   and expect GTK+ to execute simultainiously.

Cheers,
 -Tristan



___
gtk-app-devel-list mailing list
gtk-app-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-app-devel-list

Re: GTK and threaded applications

[Michael L Torrie]

On Tue, 2006-01-31 at 18:28 +0100, kornelix wrote:
> Following the guidelines in the FAQ, I constructed my application 
> threads as follows:
> 
>gdk_threads_enter();//  enter thread
> (do some work, including GTK calls)
>gdk_flush();//  exit thread
>gdk_threads_leave(); 
>return 0;
> 
> Now my multi-threaded application executes its threads one after the 
> other, instead of in parallel. Apparently the above method introduces a 
> lock or mutex which allows only one thread at a time to execute.
> 
> Is this a hopeless situation, or is there some other way?
> (other than putting all GTK calls in the main program)

Just put the gdk_* threads and flush calls around the actual GTK calls.
that way only gui updates themselves are put in mutexes (locked).  The
way you are currently doing it, your actual business logic ends up in
the mutex, and so if it is lengthy, it will block all the other threads.

Michael


> 
> thanks,
> Mike
> 
> ___
> gtk-app-devel-list mailing list
> gtk-app-devel-list@gnome.org
> http://mail.gnome.org/mailman/listinfo/gtk-app-devel-list
> 

___
gtk-app-devel-list mailing list
gtk-app-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-app-devel-list

Re: GTK and threaded applications

[Tristan Van Berkom]

kornelix wrote:
Following the guidelines in the FAQ, I constructed my application 
threads as follows:


  gdk_threads_enter();//  enter thread
   (do some work, including GTK calls)
  gdk_flush();//  exit thread
  gdk_threads_leave();   return 0;

Now my multi-threaded application executes its threads one after the 
other, instead of in parallel. Apparently the above method introduces a 
lock or mutex which allows only one thread at a time to execute.


Is this a hopeless situation, or is there some other way?
(other than putting all GTK calls in the main program)


IMO,
 overall you'll have a cleaner program design if you put
all your GUI interaction in the main thread.

Now if I'm reading you right; what your experiencing is normal,
calling gdk_threads_enter/leave is locking a global mutex for
GTK/GDK interaction.

The rule is, you cant have two threads access the GTK+ api
at once, so:
 o You can have a multithreaded application using GTK+
 o You can even go to the extreme of calling GTK+ functions
   from seperate threads, but you must take care to lock
   GTK+ down with gdk_threads_enter/leave()
 o You *cant* call GTK+ functions from more than one thread
   and expect GTK+ to execute simultainiously.

Cheers,
 -Tristan

___
gtk-app-devel-list mailing list
gtk-app-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-app-devel-list

GTK and threaded applications

[kornelix]

Following the guidelines in the FAQ, I constructed my application 
threads as follows:


  gdk_threads_enter();//  enter thread
   (do some work, including GTK calls)
  gdk_flush();//  exit thread
  gdk_threads_leave(); 
  return 0;


Now my multi-threaded application executes its threads one after the 
other, instead of in parallel. Apparently the above method introduces a 
lock or mutex which allows only one thread at a time to execute.


Is this a hopeless situation, or is there some other way?
(other than putting all GTK calls in the main program)

thanks,
Mike

___
gtk-app-devel-list mailing list
gtk-app-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-app-devel-list