I have tried using "weston-debug scene-graph" and I am coming to the
conclusion that qtwayland 6.5.0 is not really using native Wayland
surfaces when Qt::WA_NativeWindow is used. From what I can see (and I
could easily be wrong) the Wayland protocol shows wl_surfaces being
created and two QWidget's QPlatformNativeInterface
nativeResourceForWindow("surface", windowHandle()) function does return
different wl_surface pointers but even at the QWidget level (ignoring
gstreamer), a QPainter paint into each of these QWidgets actually uses
Wayland to draw into just the one top level surface and "weston-debug
scene-graph" shows only one application xdg_toplevel surface and no
subsurfaces. I don't know how to determine the Wayland surface ID from a
wl_surface pointer unfortunately to really check this.
If my Video QWidget(0) is a top level QWidget, then video is shown and
"weston-debug scene-graph" shows the application xdg_toplevel and two
wl_subsurfaces as children.
Unfortunately I think "weston-debug scene-graph" only shows surfaces
that are actually "active" so I can't see all of the surfaces that
Weston actually knows about (is there a method of doing this ?).
My feeling is that although Qtwayland is creating native surfaces, it
actually only uses the one top level one and presumably doesn't
"activate" (set a role, do something ?) with the other surfaces.
Does anyone know a good list/place where I can ask such detailed
qtwayland questions ?
I guess I can work around this by manually creating a Wayland subsurface
from the Qt top level surface and handing that to waylandsink and then
manage this subsurface, like hiding, showing and resizing, when the
QWidget is hidden/shown/resized.
Or could there be a way of "activating" the child QWidget's Wayland
surface ?
On 22/02/2024 18:44, Terry Barnaby wrote:
Hi Marius,
Many thanks for the info.
Some notes/questions below:
Terry
On 22/02/2024 17:49, Marius Vlad wrote:
Hi,
On Thu, Feb 22, 2024 at 03:21:01PM +0000, Terry Barnaby wrote:
Hi,
We are developing a video processing system that runs on an NXP imx8
processor using a Yocto embedded Linux system that has Qt6, GStreamer,
Wayland and Weston.
We are having a problem displaying the video stream from GStreamer on a
QWidget. In the past we had this working with Qt5 and older GStreamer,
Wayland and Weston.
A simple test program also shows the issue on Fedora37 with QT6 and
KDE/Plasma/Wayland.
I'm tempted to say if this happens on a desktop with the same Qt
version and
other compositors to be an issue with Qt rather than waylandsink or
the compositor. Note that on NXP they have their own modified Weston
version.
That is my current feeling and is one reason why I tried it on Fedora
with whatever Wayland compositor KDE/Plasma is using.
The technique we are using is to get the Wayland surface from the
QWidget is
using (It has been configured to use a Qt::WA_NativeWindow) and pass
this to
the GStreamer's waylandsink which should then update this surface
with video
frames (via hardware). This works when the QWidget is a top level
Window
widget (QWidget(0)), but if this QWidget is below others in the
hierarchy no
video is seen and the gstreamer pipeline line is stalled.
So the assumption is that aren't there other widgets which obscures this
one, when you move it below others?
My simple test example has two QWidgets with the one for video being
created as a child of the first so it should be above all others. I
have even tried drawing in it to make sure and it displays its Qt
drawn contents fine, just not the video stream.
It appears that waylandsink does:
Creates a surface callback:
callback = wl_surface_frame (surface);
wl_callback_add_listener (callback, &frame_callback_listener, self);
Then adds a buffer to a surface:
gst_wl_buffer_attach (buffer, priv->video_surface_wrapper);
wl_surface_set_buffer_scale (priv->video_surface_wrapper,
priv->scale);
wl_surface_damage_buffer (priv->video_surface_wrapper, 0, 0,
G_MAXINT32,
G_MAXINT32);
wl_surface_commit (priv->video_surface_wrapper);
But never gets a callback and just sits in a loop awaiting that
callback.
I assume that the surface waylandsink is using, which is created
using the
original QWidget surface (sub-surface ? with window ?) is not
"active" for
some reason.
Possibly when QWidget is below in hierarcy to be a child of of a parent,
as described in
https://wayland.app/protocols/xdg-shell#xdg_toplevel:request:set_parent,
so I assume to have a different surface than the parent one. This would
be easy to determine with WAYLAND_DEBUG. Seems unlikely to a itself a
sub-surface of a surface.
I haven't really got the gist of whats going on, but waylandsink
certainly creates a subsurface from the QWidget surface, in fact it
seems to create a few things.
I assume a subsurface is used so the video can be displayed in that
subsurface separately from the parent (de synced from it).
I am trying to debug this, but this graphics stack is quite
complicated with
waylandsink, qtwayland, wayland-lib and Weston not to mention the NXP
hardware levels. My thoughts are that it is something qtwayland is
doing
with the surface stack or thread locking issues (gstreamer uses
separate
threads). I also don't understand Wayland or Weston in detail. So some
questions:
1. Anyone seen something like this ?
Someone else reported something similar but that by causing damage,
or moving pointer to make the video sub-surface to show up:
https://gitlab.freedesktop.org/wayland/weston/-/issues/843.
Thanks, I will have a look. Moving the mouse cursor in my case (at
least with Weston) does not affect things.
2. Anyone any idea one where to look ?
3. Given the wl_surface in the Qt app or in waylandsink is there a
way I can
print out its state and the surface hierarchy easily ?
In Weston there's something called scene-graph. You can grab it by
starting Weston with with the --debug argument, then you can print
with `weston-debug scene-graph` command. A more recent Weston version
would indent sub-surfaces by their (main) surface parent.
Thanks, that could be useful.
4. Any idea on any debug methods to use ?
WAYLAND_DEBUG=1 as env variable.
Any idea on how to get a surfaces ID from a C pointer so I can match
up the QtWidget/waylandsink surface with the Wayland debug output ?
Cheers
Terry
