This is a proposal to introduce a way to post and run events on the main UI thread.
Currently (as far as I know), IUP is hands-off on threads. The problem I need to solve is that there are many libraries or situations where you are on another thread, but then need to change something in the GUI which is on the main thread. This can lead to disaster depending on the platform and implementation since most are not thread-safe. Examples of libraries that I often encounter that like to callback on background threads are networking and audio. (In fact, the native speech synthesizer I presented in my IUP Next talk at the Lua Workshop struggled with these issues since many of the speech synth implementation utilized background threads.) Since threads are a fact of life (even the web is getting web workers now), and since platform thread-safety GUI issues involve native implementation details that leak out everywhere, we need a cross-platform solution in IUP to tame these issues. So my proposal is very small and simple, “post message” mechanism that I believe can be implemented for all the major platforms moving forward. To be clear, this is NOT a proposal to introduce threads to IUP, but merely be able to message/dispatch something on the main thread for those who find themselves on a background thread because of any variety of reasons (including sometimes out of their control). To set up some context about the backends: Mac, iOS, and Android *must* do UI stuff on the main UI thread. Attempts to call GUI APIs on other threads will lead to crashes, halting, etc. But because not-blocking the UI thread is so important for responsiveness, all these platforms are aggressive about finding non-UI things to compute on background threads. Thus all of these platforms provide ways to dispatch code back on the UI thread from the other threads. And because these mechanisms are so easy to invoke, many libraries that are provided callback on background threads and assume the user will take responsibility for dispatching back to the main UI thread if they need to. But since IUP abstracts away the native GUI parts, but doesn’t currently abstract the main UI dispatch mechanisms, we are left with a roadblock. Additionally, Windows and GTK are not guaranteed to be thread safe. While these platforms aren’t as explicit about advertising the threading rules as Apple and Android, the restrictions are effectively the same. Hence why Windows has a PostMessage() API and GTK offers g_idle_add, both of which are advertised as techniques for communicating back with the main UI thread. And it appears the web worker design also provides a postMessage() mechanism in this same spirt. So now. let me give a concrete example of the threading problem which I propose to solve. I have written a normal IUP program that works across all the different platforms I mentioned. But my program also uses an audio library, ALmixer, to play back sounds. ALmixer however has an audio-callback API for certain types events, which calls back on a background thread. For simplicity, let’s just focus on ALmixer’s “sound finished playing” callback notification. In my IUP program, - I have a button you press to start playing a sound. - When you press the button, I make the button inactive (ACTIVE=0) and play the sound. - When the audio finishes playing, I want to make the button active, so the user can play the sound again by pressing the button again. The problem is that the callback is on a background thread, so it is unsafe to call any IUP GUI APIs. So I need a mechanism where I can send a message to the main UI thread and start running a callback there, so I can make the button active again. My proposal is to introduce a new callback, in spirit of the Windows PostMessage(), maybe also web worker’s postMessage(). On GTK we can implement this with g_idle_add. On Apple, we can implement this easily with Apple’s libdispatch: dispatch_async/dispatch_get_main_queue. Android can be done with Handler.post() using Looper.getMainLooper() (assuming you designed the backend correctly, which I’ve done). The overall proposal idea for IUP is to introduce a new callback event, e.g. IupSetCallback(ih, “UITHREAD”, uicallback); and a API to trigger it from a background thread, e.g. IupPostMessage(Ihandle* ih, void* message_data); (I’ve suggest more proposal variations of this further below.) I’ve actually already implemented a working proof-of-concept for all the platforms I mentioned, except Windows. (I’ll explain why not-Windows in a moment.) My proof of concept example using ALmixer with IUP can be found here. https://bitbucket.org/ewing/iupalmixerthread/src (The bulk of the code is in the source directory, but the Android .java file resides in the CMakeModules subdirectory.) The proof-of-concept I wrote actually was implemented directly in my app, and outside (not-modifying) of IUP (mostly). For Mac, iOS, and GTK, this is pretty trivial to do. Android on the other-hand was a nightmare (as usual with Android), but because I’ve already been keenly aware of this problem, I’ve already taken great care in designing IUPAndroid’s backend so this problem could be solved, so all the internal mechanisms I need are already inside my implementation. That leaves Windows. The reason I couldn’t implement this is because IUP abstracts the event loop for all the platforms in the API. For the Windows implementation, I will need to make a few additions to IUP’s Windows implementation in how it deals with the loop. But I believe I found a working solution. The solution actually from Raymond Chen’s blog: "Rescuing thread messages from modal loops via message filters” https://blogs.msdn.microsoft.com/oldnewthing/20050428-00/?p=35753 The solution uses PostThreadMessage. Generally, this is not advised because it can lose messages during modal loops. PostMessage is usually the preferred solution. However, the problem with PostMessage is you must post a message to a explicit window handle. It is not always the case you know which window you want to deal with, if any. For example, in my audio example, maybe instead of updating the GUI, I just want to unload the audio file now that it is done playing. Additionally, none of the other platforms tie their thread messaging with specific GUI widgets, so this would create an impedance mismatch. But fortunately, Raymond Chen’s article demonstrates how to not lose messages when using PostThreadMessage. So even though it is not the usual recommended solution, in our case, I think it is the correct design. He warns that the downside to this implementation is that it requires “all modal loops need to remember to call CallMsgFilter as part of their dispatch loop”. But since IUP wraps/abstracts all the event loop stuff, this should be quite simple to do for us. Since this was an “end-run” prototype that didn’t modify IUP (except for Android), this is not using the API I am proposing for IUP. But it similar and should capture the heart of what needs to be done and how it works. So I’d like to work with you to bring this into IUP. But I wanted to ask before doing, and maybe find out if you have better ideas on both the API and the Windows implementation approach. I think needs to be in IUP because: - IUP abstracts/wrap the event loop, so IUP is the appropriate place to solve it. - (And as I mentioned, the Windows implementation seems to require that it be in IUP.) - Additionally, IUP also somewhat hides the backend implementation (e.g GTK2, GTK3, Motif, etc.) So trying to do this outside IUP (aka “end-run”) is fragile because it depends on me implementing and using the exact same backend. So in my linked ALmixer example, I wrote a GTK backend and forced me to write GTK code and link to it, on top of also using IUP. And if IUP used a different backend (e.g. Motif) my end-run would break. As an IUP user, I shouldn’t have to write native code for this type of thing. And of course, my example cross-platform code is now littered with Mac, iOS, and Android implementations on top of all this. - The thread safety rules and issues are real in all the backends, and each native platform offers APIs to deal with thread messaging. IUP wrapping native GUIs should wrap this small piece too. - New libraries continue to push hard into callbacks on background threads. For example, but Apple and Android’s in-app purchase (which talk over the network) modules may callback on background threads. So below are 3 variations of my API proposal for consideration: ///////////////////// Idea 1: Global callback // Should the term be UITHREAD or THREAD_DISPATCH or MAIN_DISPATCH? IupSetGlobal(“UITHREAD”, uicallback); void uicallback(int type, void* message_data) { } // Should the term be Message or Thread or Dispatch? IupPostMessage(int user_type, void* message_data); ///////////////////// Example: void Init() { IupSetGlobal(“UITHREAD”, main_thread_callback); AudioSetFinishedCallback(audio_background_thread_finished_playing); struct AudioData* audio_data = LoadAudioData(); void* audio_user_data = NULL; AudioPlaySound(audio_data, audio_user_data); } // This happens on a background thread void audio_background_thread_finished_playing(struct Audio* audio_data, void* audio_user_data) { // I give audio types an arbitrary type of 42 IupPostMessage(42, audio_data); // does not block } // calls back on the main thread void main_thread_callback(int user_type, void* audio_data) { switch(user_type) { case 42: // got audio type UpdateUI(); break; default: // unhandled } } ///////////////////// Notes: Since there is only one global callback, there may be multiple systems that want to use it. So to allow this, the int user_type can be used to distinguish different use cases. Requires Icallback with signature: viv Downsides: - int user_types will need to be coordinated by end users so they don’t trample over each other. - There is only one global callback, so library developers who need this but want to hide implementation details about their threads, cannot do so because the user may steal the global callback out from under them. Upsides: - Does not require any explicit widget (Ihandle*) to work. - Reuses existing IupSetGlobal() API. Additional thoughts: - Maybe the user should be able to set one more void* user_data parameter they can get back in the callback? (That way, an Ihandle* can be passed through which can be used to UpdateUI() on the other side. IupSetAttribute(NULL, “UITHREAD”, some_user_data_pointer); // Then get it back when inside the uicallback? void* some_user_data_pointer = IupGetAttrbitue(NULL, “UITHREAD”); ///////////////////// ///////////////////// Idea 2: Per Ihandle* IupSetCallback(ih, “UITHREAD”, uicallback); void uicallback(Ihandle* ih, int type, void* message_data) { } // Should the term be Message or Thread or Dispatch? IupPostMessage(Ihandle* ih, int user_type, void* message_data); ///////////////////// Example: void Init() { Ihandle* ih = IupButton(NULL, NULL); // (omitted) set up button stuff IupSetCallback(ih, “UITHREAD”, main_thread_callback); AudioSetFinishedCallback(audio_background_thread_finished_playing); struct AudioData* audio_data = LoadAudioData(); void* audio_user_data = ih; AudioPlaySound(audio_data, audio_user_data); } // This happens on a background thread void audio_background_thread_finished_playing(struct Audio* audio_data, void* audio_user_data) { Ihandle* ih = (Ihandle*) audio_user_data; // I give audio types an arbitrary type of 42 IupPostMessage(ih, 42, audio_data); // does not block } // calls back on the main thread void main_thread_callback(Ihandle* ih, int user_type, void* audio_data) { switch(user_type) { case 42: // got audio type UpdateUI(ih); break; default: // unhandled } } ///////////////////// Notes: Since we do this on a per-Ihandle basis, we might be able to do-away with the int user_type. I still think it could be handy though. Requires Icallback with signature: viv Downsides: - Now requires an explicit, designated Ihandle* to work. This could create impedance for GUIs that are very fluid and tear down widgets a lot. In those, dialogs and other widgets go away a lot, so if you expect a callback to fire in some distant future, you have to keep those widgets alive. This may lead to the creation of dummy widgets (and may tempt people to create unmapped widgets…I’m not sure if this is a bad thing or not.) - Also notice that the widget itself is not necessarily that important to the thread dispatch stuff. It is just a proxy object. Instead of a button, I could have used a label or dialog. And the widget itself does not need to be involved in anything else. Perhaps I just wanted to delete the AudioData in the callback instead of doing GUI stuff, yet I still brought along the Ihandle*. Hence, the association between Ihandle* and main thread dispatch is kind of weak and mostly just serves bookkeeping. Upsides: - We now get one callback per Ihandle*, so library writers can use this to hide implementation details without being trampled by end users. - Reuses existing IupSetCallback() API. Additional thoughts: - Maybe the user should be able to set one more void* user_data parameter they can get back in the callback? IupSetAttribute(NULL, “UITHREAD”, some_user_data_pointer); // Then get it back when inside the uicallback? void* some_user_data_pointer = IupGetAttrbitue(NULL, “UITHREAD”); ///////////////////// ///////////////////// Idea 3: New API to split the difference between Idea 1 and Idea 2? Allow separate callbacks for each different type of user type. IupSetUiThreadCallback(const char* user_type, “UITHREAD”, uicallback); void uicallback(const char* user_type, void* message_data) { } // Message or Thread or Dispatch? IupPostMessage(const char* user_type, void* message_data); ///////////////////// Example: void Init() { Ihandle* ih = IupButton(NULL, NULL); // (omitted) set up button stuff IupSetUiThreadCallback(“audio”, “UITHREAD”, main_thread_audio_callback); AudioSetFinishedCallback(audio_background_thread_finished_playing); struct AudioData* audio_data = LoadAudioData(); void* audio_user_data = ih; AudioPlaySound(audio_data, audio_user_data); void* network_user_data = ih; NetworkSetFinishedCallback(network_thread_finished_download); NetworkFetchData(ih); IupSetUiThreadCallback(“network”, “UITHREAD”, main_thread_network_callback); } // This happens on a background thread void audio_background_thread_finished_playing(struct Audio* audio_data, void* audio_user_data) { Ihandle* ih = (Ihandle*) audio_user_data; IupPostMessage(“audio”, audio_data, audio_user_data); // does not block } // This happens on a background thread void network_thread_finished_download(struct NetworkData* download_data, void* network_user_data) { Ihandle* ih = (Ihandle*) network_user_data; IupPostMessage(“network”, download_data, network_user_data); // does not block } // calls back on the main thread for just “audio” events void main_thread_callback(const char* user_type, void* audio_data, void* user_data) { // assert: user_type == “audio” Ihandle* ih = (Ihandle*)user_data; UpdateUI(ih); } // calls back on the main thread for just “network” events void main_thread_callback(const char* user_type, void* network_data, void* user_data) { // assert: user_type == “network” Ihandle* ih = (Ihandle*)user_data; UpdateUI(ih); } ///////////////////// Notes: This idea decouples the thread dispatch callbacks from actual Ihandle* widgets (like Idea 1), but we still can get multiple callbacks so people don’t trample each other fighting over a single callback (like Idea 2). Additionally, this idea filters callbacks so that each user_type can potentially get a separate function pointer, if the user desires. The idea is somewhat similar to IupButton callbacks and how you can either have separate callbacks for each Ihandle*, or share the same one and distinguish on const char* action). My thinking is the const char* user_type can be very similar to the IupSetAttribute and IupSetStrAttribute in that maybe the user can use it as either a string or arbitrary pointer. Downsides: - One more new API function - Probably requires more implementation code for us to save and invoke the different callbacks. Upsides: - Thread dispatch and Ihandle* widgets are not artificially coupled. - Callbacks can be separated based on type. Anyway, since my proof-of-concept works (sans Windows), I expect this to be a relatively easy thing to implement. (Fingers crossed that Chen’s solution just works.) The main thing I believe is to decide on the API design so we can implement it. Thanks, Eric ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Iup-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/iup-users
