oops, re-sending....
On 30.09.2024 19:19, Miller Puckette wrote:
On 9/30/24 6:23 PM, Christof Ressi wrote:
Suppose we replace "draw X, Y,Z" with "draw button". I think this
then spins out in either of two possible ways:
(1) the "button" widget on the GUI side now takes care of mouse-hit
detection and making itself flash (and eventually widgets with text
can grab focus, handle copy/paste, etc). In this case the GUI widget
has to maintain a connection with its corresponding object in Pd.
This gets very complicated when, for instance, changing text causes
inlets or outlets to appear, which perhaps on the Pd side the text
is getting hammered by messages from elsewhere at the same time.
Distributed database management, anyone?
(2) it's just a drawing with a tag, not an active widget; perhaps
mouse hit detection is done in the GUI but everything else in Pd. In
this case it's not a big enough change to make much difference,
except for this noisome one: "GUI externals" such as the knob now
have to load dynamic libraries into two programs, instead of just one.
Depending on which scenario we want to consider (or perhaps this is
somehow a false choice) we can go into more detail about this...
These are important points! I think there are two separate concerns:
1. who is doing the mouse-hit-detection?
2. who initiates the state changes (e.g. flashing, selection)
For example, it would be possible to do the hit detection on the GUI,
but still do the state changes on the core. For example, in IOhannes
draft PR, the "bang" object has an "activate" message that is sent
whenever the flash state changes:
https://urldefense.com/v3/__https://github.com/pure-data/pure-data/pull/1765/files*diff-08294559d6a971f74472cc2fca7cce724cdd9e055ac46f6e38c4633b27a97bcbR46-R47__;Iw!!Mih3wA!DMJmlfmkmALCSAhXhDnbQnMhQSg8DlxeR4TAY7O3YsaFJP1hRcbCXmzibkOfxx9v_QN4aCGo1NJQcA$
We could move such internal state changes to the GUI, but we don't
have to if it only complicates things.
However, I would love to move the hit detection to the GUI! Note that
some GUI frameworks already do hit detection for canvas items. (In
Qt, graphic items in a QGraphicsScene can receive mouse events!) In
this case, doing the hit detection on the core is just duplicate
work. It is also rather expensive since the core only stores the
objects in a linked list, without spatial indexing, so every mouse
movement has to potentially iterate over hundreds or even thousands
gobjects and do rectangle intersection tests until it finds a hit.
This is not something that you would want to do on the audio thread :)
The same thing applies to selections. The GUI typically already knows
which objects are selected, so we only need to send this information
to the core.
If we do the hit detection on the GUI, you are right that we need to
keep an association between GUI widget and gobject. I think we just
need to have a hashtable of tag -> gobject on the core; when the core
receives a mouse event, it looks up the tag in the hashtable and then
dispatches the event to the corresponding object. If the tag is not
found, it means that the object has been destroyed in flight and we
can just ignore the event.
There's one slight problem: some objects need to know *where* the
user clicked. E.g. a slider with steady-on-click needs to know the
position to set the new value. This means that the core side has to
maintain some knowledge about the graphical representation. This does
not seem to much of a problem to me, but it's something to be aware of.
One idea would be to do the hit detection in the GUI and simply pass
Pd the (x,y) coords along with a list of all the GUI objects that got
hit (each with a "fuzziness" quotient so that we can be smart about
disambiguating multiple hits). The only reason I didn't do that
already is because I didn't want to get too deeply involved with
TclTk's canvases. And yes, anything we kick up to the GUI has to be
done both in TCL/TK and in whatever 5 new GUIs are tried out -- that's
one reason I've kept as much of the intelligence as possible down in
Pd land.
I also hate the way every canvas I've seen does rectangle-selection.
Maybe you know of one I won't hate :)
And... every widget set I've seen thinks widgets live in rectangles.
The second most important graphical item in Pd is the patch line.
Giving it a widget-style selection method? Disaster. I don't think
patch lines can be TK or GTK r QT or anyone's widgets - imagine how
those bounding rectangles would fill th screen :)
---
Now here's the big catch: what should we do with GUI externals?
Ad 1: externals could define a region that acts as the bounding
rectangle and which can be filled with drawing commands. This way
they can partipicate in hit detection on the GUI.
Ad 2: I think state changes for externals have to be handled on the
core because the external has no access to the GUI features. It does
not even know which GUI it runs in.
One idea I had was to provide the GUI features as API functions. E.g.
pdgui_drawrect() to draw a rectangle or pdgui_delay() to start a
timer, etc. These API functions have to be implemented by the GUI,
the core would just delegate the calls to the GUI. The external would
register a callback function that is called whenever the object needs
to be redrawn. Also, the GUI external could register callbacks for
mouse events and paintings
One problem I see is that not every GUI can export C functions. For
example, this probably wouldn't work with a web browser GUI...
The easiest solution is to just keep state changes for external GUI
objects on the core, i.e. the external needs to send raw drawing
commands. As an upside, this would make it easier to update existing
externals as they would just have to update the drawing commands. Of
course, as I said, sending all these individual commands not only
increases traffic, it also means that the GUI has to maintain
individual graphical primitives instead of just a single widget.
Either way, we should provide a stable GUI/drawing API for externals.
Currently, it's all unofficial.
Thanks to Antoine for mentioning pdlua! I will need to check it out.
I have already seen people doing some very cool stuff with it. Maybe
they already have the perfect solution :)
I'm also looking at pdlua, which might want to become the official way
to make graphical externs AND to add scripting in a fundamental way to
Pd :)
---
I think all of this needs quite a bit of planning and design. We
already had an online meeting two or three years ago before we
started the GUI refactoring process. I would suggest to have another
meeting where we can talk through all the potential issues and get a
feel for the overall design.
Anyway, I'm very excited that the GUI refactoring is moving forward!
Christof
cheers
M
---
[email protected] - the Pd developers' mailinglist
https://urldefense.com/v3/__https://lists.iem.at/hyperkitty/list/[email protected]/message/6WFFUCOQCFNEKCVI2QTY6XYZN6QAPFZY/__;!!Mih3wA!DMJmlfmkmALCSAhXhDnbQnMhQSg8DlxeR4TAY7O3YsaFJP1hRcbCXmzibkOfxx9v_QN4aCFvp3A6bw$
---
[email protected] - the Pd developers' mailinglist
https://urldefense.com/v3/__https://lists.iem.at/hyperkitty/list/[email protected]/message/U4ED4AGPILGDGQYNHNVZ7KPEHUICTUNG/__;!!Mih3wA!DMJmlfmkmALCSAhXhDnbQnMhQSg8DlxeR4TAY7O3YsaFJP1hRcbCXmzibkOfxx9v_QN4aCHXM2jDtA$
---
[email protected] - the Pd developers' mailinglist
https://lists.iem.at/hyperkitty/list/[email protected]/message/STWRP25DA7TATO4A3HSHY3ULVH3VERDY/
