Suppose we replace "draw X, Y,Z" with "draw button". I think this then
spins out in either of two possible ways:
(1) the "button" widget on the GUI side now takes care of mouse-hit
detection and making itself flash (and eventually widgets with text
can grab focus, handle copy/paste, etc). In this case the GUI widget
has to maintain a connection with its corresponding object in Pd.
This gets very complicated when, for instance, changing text causes
inlets or outlets to appear, which perhaps on the Pd side the text is
getting hammered by messages from elsewhere at the same time.
Distributed database management, anyone?
(2) it's just a drawing with a tag, not an active widget; perhaps
mouse hit detection is done in the GUI but everything else in Pd. In
this case it's not a big enough change to make much difference, except
for this noisome one: "GUI externals" such as the knob now have to
load dynamic libraries into two programs, instead of just one.
Depending on which scenario we want to consider (or perhaps this is
somehow a false choice) we can go into more detail about this...
These are important points! I think there are two separate concerns:
1. who is doing the mouse-hit-detection?
2. who initiates the state changes (e.g. flashing, selection)
For example, it would be possible to do the hit detection on the GUI,
but still do the state changes on the core. For example, in IOhannes
draft PR, the "bang" object has an "activate" message that is sent
whenever the flash state changes:
https://github.com/pure-data/pure-data/pull/1765/files#diff-08294559d6a971f74472cc2fca7cce724cdd9e055ac46f6e38c4633b27a97bcbR46-R47
We could move such internal state changes to the GUI, but we don't have
to if it only complicates things.
However, I would love to move the hit detection to the GUI! Note that
some GUI frameworks already do hit detection for canvas items. (In Qt,
graphic items in a QGraphicsScene can receive mouse events!) In this
case, doing the hit detection on the core is just duplicate work. It is
also rather expensive since the core only stores the objects in a linked
list, without spatial indexing, so every mouse movement has to
potentially iterate over hundreds or even thousands gobjects and do
rectangle intersection tests until it finds a hit. This is not something
that you would want to do on the audio thread :)
The same thing applies to selections. The GUI typically already knows
which objects are selected, so we only need to send this information to
the core.
If we do the hit detection on the GUI, you are right that we need to
keep an association between GUI widget and gobject. I think we just need
to have a hashtable of tag -> gobject on the core; when the core
receives a mouse event, it looks up the tag in the hashtable and then
dispatches the event to the corresponding object. If the tag is not
found, it means that the object has been destroyed in flight and we can
just ignore the event.
There's one slight problem: some objects need to know *where* the user
clicked. E.g. a slider with steady-on-click needs to know the position
to set the new value. This means that the core side has to maintain some
knowledge about the graphical representation. This does not seem to much
of a problem to me, but it's something to be aware of.
---
Now here's the big catch: what should we do with GUI externals?
Ad 1: externals could define a region that acts as the bounding
rectangle and which can be filled with drawing commands. This way they
can partipicate in hit detection on the GUI.
Ad 2: I think state changes for externals have to be handled on the core
because the external has no access to the GUI features. It does not even
know which GUI it runs in.
One idea I had was to provide the GUI features as API functions. E.g.
pdgui_drawrect() to draw a rectangle or pdgui_delay() to start a timer,
etc. These API functions have to be implemented by the GUI, the core
would just delegate the calls to the GUI. The external would register a
callback function that is called whenever the object needs to be
redrawn. Also, the GUI external could register callbacks for mouse
events and paintings
One problem I see is that not every GUI can export C functions. For
example, this probably wouldn't work with a web browser GUI...
The easiest solution is to just keep state changes for external GUI
objects on the core, i.e. the external needs to send raw drawing
commands. As an upside, this would make it easier to update existing
externals as they would just have to update the drawing commands. Of
course, as I said, sending all these individual commands not only
increases traffic, it also means that the GUI has to maintain individual
graphical primitives instead of just a single widget.
Either way, we should provide a stable GUI/drawing API for externals.
Currently, it's all unofficial.
Thanks to Antoine for mentioning pdlua! I will need to check it out. I
have already seen people doing some very cool stuff with it. Maybe they
already have the perfect solution :)
---
I think all of this needs quite a bit of planning and design. We already
had an online meeting two or three years ago before we started the GUI
refactoring process. I would suggest to have another meeting where we
can talk through all the potential issues and get a feel for the overall
design.
Anyway, I'm very excited that the GUI refactoring is moving forward!
Christof
cheers
M
---
[email protected] - the Pd developers' mailinglist
https://lists.iem.at/hyperkitty/list/[email protected]/message/6WFFUCOQCFNEKCVI2QTY6XYZN6QAPFZY/
---
[email protected] - the Pd developers' mailinglist
https://lists.iem.at/hyperkitty/list/[email protected]/message/U4ED4AGPILGDGQYNHNVZ7KPEHUICTUNG/
