RE: Full-motion zero-copy screen capture in Weston
>> >> >> -Original Message- >> From: Hoosier, Matt >> Sent: Monday, June 17, 2024 8:28 AM >> To: Pekka Paalanen >> Cc: 'Marius Vlad' ; >> 'wayland-devel@lists.freedesktop.org' ; >> 'Daniel Stone' >> Subject: RE: Full-motion zero-copy screen capture in Weston >> >> > -Original Message- >> > From: Pekka Paalanen >> > Sent: Monday, June 17, 2024 3:28 AM >> > To: Hoosier, Matt >> > Cc: 's...@cmpwn.com' ; 'cont...@emersion.fr' >> > ; 'Marius Vlad' ; >> > 'wayland-devel@lists.freedesktop.org' > > de...@lists.freedesktop.org>; 'Daniel Stone' >> > Subject: Re: Full-motion zero-copy screen capture in Weston >> > >> > On Fri, 14 Jun 2024 18:36:57 + >> > "Hoosier, Matt" wrote: >> > >> > > >> > > Hmm. As I read through the history of the original support for >> > > writeback screenshots >> > > (https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/458) >> > > and get some initial results just trying to drive a steady stream of >> > > writeback composition, I'm not sure this can work effectively. >> > > >> > > MR 458 has a fairly long discussion about the restriction that no >> > > further commits must be made to the source CRTC or any of the other >> > > KMS objects reachable on its graph, while a writeback composition >> > > referencing that CRTC is still in flight. >> > > >> > > Daniel Vetter confirmed that interpretation. >> > > >> > > I think this means that unless you are extremely lucky that: >> > > >> > > (a) your hardware's writeback mechanism is synchronous with scanout, >> > > and (b) the fence fd manages to fire and get dispatched immediately >> > > upon scanout/writeback passing through the final scanline, and (c) >> > > there's a really large vertical back porch >> > > >> > > , there will be no time left to enqueue a page flip for the >> > > immediately succeeding vblank. >> > > >> > > Seems like this will almost always cut the frame achievable update >> > > rate on the main connector in half. >> > > >> > > Did I misinterpret something in there? Early results look like the >> > > kernel state machine gets confused (with VKMS anyway) if I queue up >> > > two writeback operations in flight at the same time. >> > >> > Hi Matt, >> > >> > I really don't know. If hardware and drivers require that, and cannot >> > stream at full refresh rate, then there is not much we can do. >> > >> > Not much, because I got one idea: Weston could repaint pre-emptively >> > according to its own schedule, but KMS-flip only when the writeback >> > situation allows. That should reduce the time needed between the >> > writeback completion fence firing and the KMS deadline for the flip. >> > >> > VKMS might rely on the above rules, or maybe it's not fully developed, >> > I'm not sure. You'd have to ask its developers. OTOH, if all hardware >> > drivers need the above rules, then VKMS should too. Maybe it just >> > needs to check and fail better. >> > >> > >> > Thanks, >> > pq >> >> Okay. I think maybe it makes sense to figure out who originally added this >> kernel documentation and see whether the situation is really so grim. >> >> I'll see about writing something to whoever that was, perhaps with dri-devel >> CC'ed. >> It turns out that the official interpretation of the documentation on the KMS writeback connector's properties is that the framerate of the CRTC driving the writeback operations will almost always be cut in half: https://lists.freedesktop.org/archives/dri-devel/2024-June/458351.html This isn't a hit that the use-cases I have in mind can afford, so I probably won't continue trying to make an implementation of this Pipewire stream driven by the writeback connector. Some of the responders to that thread over on dri-devel are beginning to speculate about updated DRM uAPI properties that would better support streaming of those writeback connectors whose hardware can do it. Maybe I'll revisit if that discussion goes far and something new lands in the framework. Thanks to everybody for the advice along the way. -Matt
RE: Full-motion zero-copy screen capture in Weston
> -Original Message- > From: Pekka Paalanen > Sent: Monday, June 17, 2024 3:28 AM > To: Hoosier, Matt > Cc: 's...@cmpwn.com' ; 'cont...@emersion.fr' > ; 'Marius Vlad' ; > 'wayland-devel@lists.freedesktop.org' de...@lists.freedesktop.org>; 'Daniel Stone' > Subject: Re: Full-motion zero-copy screen capture in Weston > > On Fri, 14 Jun 2024 18:36:57 + > "Hoosier, Matt" wrote: > > > > > Hmm. As I read through the history of the original support for > > writeback screenshots > > (https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/458) > > and get some initial results just trying to drive a steady stream of > > writeback composition, I'm not sure this can work effectively. > > > > MR 458 has a fairly long discussion about the restriction that no > > further commits must be made to the source CRTC or any of the other > > KMS objects reachable on its graph, while a writeback composition > > referencing that CRTC is still in flight. > > > > Daniel Vetter confirmed that interpretation. > > > > I think this means that unless you are extremely lucky that: > > > > (a) your hardware's writeback mechanism is synchronous with scanout, > > and (b) the fence fd manages to fire and get dispatched immediately > > upon scanout/writeback passing through the final scanline, and (c) > > there's a really large vertical back porch > > > > , there will be no time left to enqueue a page flip for the > > immediately succeeding vblank. > > > > Seems like this will almost always cut the frame achievable update > > rate on the main connector in half. > > > > Did I misinterpret something in there? Early results look like the > > kernel state machine gets confused (with VKMS anyway) if I queue up > > two writeback operations in flight at the same time. > > Hi Matt, > > I really don't know. If hardware and drivers require that, and cannot stream > at > full refresh rate, then there is not much we can do. > > Not much, because I got one idea: Weston could repaint pre-emptively > according to its own schedule, but KMS-flip only when the writeback situation > allows. That should reduce the time needed between the writeback > completion fence firing and the KMS deadline for the flip. > > VKMS might rely on the above rules, or maybe it's not fully developed, I'm not > sure. You'd have to ask its developers. OTOH, if all hardware drivers need the > above rules, then VKMS should too. Maybe it just needs to check and fail > better. > > > Thanks, > pq Okay. I think maybe it makes sense to figure out who originally added this kernel documentation and see whether the situation is really so grim. I'll see about writing something to whoever that was, perhaps with dri-devel CC'ed.
Re: Full-motion zero-copy screen capture in Weston
On Fri, 14 Jun 2024 18:36:57 + "Hoosier, Matt" wrote: > > Hmm. As I read through the history of the original support for > writeback screenshots > (https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/458) > and get some initial results just trying to drive a steady stream of > writeback composition, I'm not sure this can work effectively. > > MR 458 has a fairly long discussion about the restriction that no > further commits must be made to the source CRTC or any of the other > KMS objects reachable on its graph, while a writeback composition > referencing that CRTC is still in flight. > > Daniel Vetter confirmed that interpretation. > > I think this means that unless you are extremely lucky that: > > (a) your hardware's writeback mechanism is synchronous with scanout, > and (b) the fence fd manages to fire and get dispatched immediately > upon scanout/writeback passing through the final scanline, and (c) > there's a really large vertical back porch > > , there will be no time left to enqueue a page flip for the > immediately succeeding vblank. > > Seems like this will almost always cut the frame achievable update > rate on the main connector in half. > > Did I misinterpret something in there? Early results look like the > kernel state machine gets confused (with VKMS anyway) if I queue up > two writeback operations in flight at the same time. Hi Matt, I really don't know. If hardware and drivers require that, and cannot stream at full refresh rate, then there is not much we can do. Not much, because I got one idea: Weston could repaint pre-emptively according to its own schedule, but KMS-flip only when the writeback situation allows. That should reduce the time needed between the writeback completion fence firing and the KMS deadline for the flip. VKMS might rely on the above rules, or maybe it's not fully developed, I'm not sure. You'd have to ask its developers. OTOH, if all hardware drivers need the above rules, then VKMS should too. Maybe it just needs to check and fail better. Thanks, pq pgpoe7g0nY5K_.pgp Description: OpenPGP digital signature
RE: Full-motion zero-copy screen capture in Weston
> -Original Message- > From: Pekka Paalanen > Sent: Thursday, June 13, 2024 3:32 AM > To: Hoosier, Matt > Cc: 's...@cmpwn.com' ; 'cont...@emersion.fr' > ; 'Marius Vlad' ; > 'wayland-devel@lists.freedesktop.org' de...@lists.freedesktop.org>; 'Daniel Stone' > Subject: Re: Full-motion zero-copy screen capture in Weston > > On Wed, 12 Jun 2024 21:35:48 + > "Hoosier, Matt" wrote: > > > > -Original Message- > > > From: Hoosier, Matt > > > Sent: Wednesday, June 12, 2024 8:37 AM > > > To: Pekka Paalanen > > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > > ; wayland-devel@lists.freedesktop.org; > > > Daniel Stone > > > Subject: RE: Full-motion zero-copy screen capture in Weston > > > > > > > -Original Message- > > > > From: Pekka Paalanen > > > > Sent: Wednesday, June 12, 2024 4:03 AM > > > > To: Hoosier, Matt > > > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > > > ; wayland-devel@lists.freedesktop.org; > > > > Daniel Stone > > > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > > > > > On Mon, 10 Jun 2024 14:18:39 + "Hoosier, Matt" > > > > wrote: > > > > > > > > > Yes, the native Linux driver for the hardware still does end up > > > > > being responsible for one or more planes. > > > > > > > > > > Other than trying to arrange the pieces so that there's some API > > > > > that offers the client an option to guarantee that the source of > > > > > the screen capture involves the writeback connector (similar to > > > > > what you've done with weston_output_capture), I don't really > > > > > think it would be a good idea to make Weston explicitly aware of > > > > > any of this funny hypervisor business going on. > > > > > > > > > > > > > I very much agree. > > > > > > Ack > > > > > > > > > > > > > > > The title on this email conversation was probably poorly chosen, > > > > > in retrospect. I'm not so much concerned with keeping absolutely > > > > > to a zero-copy mechanism as I am to using hardware mechanisms > > > > > (GL rendering, DRI writeback, hardware-accelerated blits as > > > > > necessary) all the way through. > > > > > > > > > > After seeing the way that the Pipewire backend handles streaming > > > > > (especially with > > > > > https://gitlab.freedesktop.org/wayland/weston/- > > > /merge_requests/1366), > > > > > I'm almost thinking that it would be preferable to maybe > > > > > structure the design of this feature something like this: > > > > > > > > > > * Add some sort of API on weston_output by which it can > > > > > advertise the ability to lay hands on the explicit renderbuffer (e.g. > > > > > gbm_surface_get_front_buffer()). This roughly equates to whether > > > > > it's a primary, non-nested backend. That is, DRM. > > > > > > > > > > * When processing the weston.ini mirror-of relationships at > > > > > startup, check whether the source output of the mirror-of > > > > > declaration supports that ability above. If so: > > > > > - Wire up a signal so that the source output announces each > > > > > newly rendered frame to any/all mirror-of outputs. > > > > > - The virtual output's backend allocates its own buffer pool > > > > > in exactly the same way that MR 1366 already does. > > > > > - Upon receipt of each signal announcing a new frame's > > > > > renderbuffer from the source output, the virtual output does a > > > > > very cheap > > > > > glBlitFramebuffer() to get the contents into its own buffer pool. > > > > > This avoids the possibility of an unresponsive client tying up > > > > > the source output's buffer. > > > > > > > > > > * If the source output isn't one that supports exposing the > > > > > underlying renderbuffer, then the mirror-of relationship > > > > > continues as with MR > > > > > 1476 just to invoke an explicit weston_renderer pass to draw the > > >
Re: Full-motion zero-copy screen capture in Weston
On Wed, Jun 12, 2024 at 09:35:48PM +, Hoosier, Matt wrote: > > -Original Message- > > From: Hoosier, Matt > > Sent: Wednesday, June 12, 2024 8:37 AM > > To: Pekka Paalanen > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > ; wayland-devel@lists.freedesktop.org; Daniel > > Stone > > Subject: RE: Full-motion zero-copy screen capture in Weston > > > > > -Original Message- > > > From: Pekka Paalanen > > > Sent: Wednesday, June 12, 2024 4:03 AM > > > To: Hoosier, Matt > > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > > ; wayland-devel@lists.freedesktop.org; Daniel > > > Stone > > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > > > On Mon, 10 Jun 2024 14:18:39 + > > > "Hoosier, Matt" wrote: > > > > > > > Yes, the native Linux driver for the hardware still does end up being > > > > responsible for one or more planes. > > > > > > > > Other than trying to arrange the pieces so that there's some API that > > > > offers the client an option to guarantee that the source of the screen > > > > capture involves the writeback connector (similar to what you've done > > > > with weston_output_capture), I don't really think it would be a good > > > > idea to make Weston explicitly aware of any of this funny hypervisor > > > > business going on. > > > > > > > > > > I very much agree. > > > > Ack > > > > > > > > > > > The title on this email conversation was probably poorly chosen, in > > > > retrospect. I'm not so much concerned with keeping absolutely to a > > > > zero-copy mechanism as I am to using hardware mechanisms (GL > > > > rendering, DRI writeback, hardware-accelerated blits as necessary) all > > > > the way through. > > > > > > > > After seeing the way that the Pipewire backend handles streaming > > > > (especially with > > > > https://gitlab.freedesktop.org/wayland/weston/- > > /merge_requests/1366), > > > > I'm almost thinking that it would be preferable to maybe structure the > > > > design of this feature something like this: > > > > > > > > * Add some sort of API on weston_output by which it can advertise the > > > > ability to lay hands on the explicit renderbuffer (e.g. > > > > gbm_surface_get_front_buffer()). This roughly equates to whether it's > > > > a primary, non-nested backend. That is, DRM. > > > > > > > > * When processing the weston.ini mirror-of relationships at startup, > > > > check whether the source output of the mirror-of declaration supports > > > > that ability above. If so: > > > > - Wire up a signal so that the source output announces each newly > > > > rendered frame to any/all mirror-of outputs. > > > > - The virtual output's backend allocates its own buffer pool in > > > > exactly the same way that MR 1366 already does. > > > > - Upon receipt of each signal announcing a new frame's renderbuffer > > > > from the source output, the virtual output does a very cheap > > > > glBlitFramebuffer() to get the contents into its own buffer pool. > > > > This avoids the possibility of an unresponsive client tying up the > > > > source output's buffer. > > > > > > > > * If the source output isn't one that supports exposing the underlying > > > > renderbuffer, then the mirror-of relationship continues as with MR > > > > 1476 just to invoke an explicit weston_renderer pass to draw the > > > > correct logical contents. > > > > > > > > How does this strike you? > > > > > > Sorry, I don't understand how that idea is relevant to the KMS writeback > > case. > > > Did you imply that DRM-backend could deliver a KMS-writeback FB instead > > of > > > the rendered FB? > > > > Just for the same of argument, yes. But I take your point below that this > > entire > > idea to drive the mirroring output directly from the source output's > > rendering > > doesn't fit the plan for the mirror-of semantics. > > > > > > > > This is not the current plan for mirror-of. It does not allow the mirror > > > output > > to > > > run on its own pace independently of the source out
Re: Full-motion zero-copy screen capture in Weston
On Wed, 12 Jun 2024 21:35:48 + "Hoosier, Matt" wrote: > > -Original Message- > > From: Hoosier, Matt > > Sent: Wednesday, June 12, 2024 8:37 AM > > To: Pekka Paalanen > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > ; wayland-devel@lists.freedesktop.org; Daniel > > Stone > > Subject: RE: Full-motion zero-copy screen capture in Weston > > > > > -Original Message- > > > From: Pekka Paalanen > > > Sent: Wednesday, June 12, 2024 4:03 AM > > > To: Hoosier, Matt > > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > > ; wayland-devel@lists.freedesktop.org; Daniel > > > Stone > > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > > > On Mon, 10 Jun 2024 14:18:39 + > > > "Hoosier, Matt" wrote: > > > > > > > Yes, the native Linux driver for the hardware still does end up being > > > > responsible for one or more planes. > > > > > > > > Other than trying to arrange the pieces so that there's some API that > > > > offers the client an option to guarantee that the source of the screen > > > > capture involves the writeback connector (similar to what you've done > > > > with weston_output_capture), I don't really think it would be a good > > > > idea to make Weston explicitly aware of any of this funny hypervisor > > > > business going on. > > > > > > > > > > I very much agree. > > > > Ack > > > > > > > > > > > The title on this email conversation was probably poorly chosen, in > > > > retrospect. I'm not so much concerned with keeping absolutely to a > > > > zero-copy mechanism as I am to using hardware mechanisms (GL > > > > rendering, DRI writeback, hardware-accelerated blits as necessary) all > > > > the way through. > > > > > > > > After seeing the way that the Pipewire backend handles streaming > > > > (especially with > > > > https://gitlab.freedesktop.org/wayland/weston/- > > /merge_requests/1366), > > > > I'm almost thinking that it would be preferable to maybe structure the > > > > design of this feature something like this: > > > > > > > > * Add some sort of API on weston_output by which it can advertise the > > > > ability to lay hands on the explicit renderbuffer (e.g. > > > > gbm_surface_get_front_buffer()). This roughly equates to whether it's > > > > a primary, non-nested backend. That is, DRM. > > > > > > > > * When processing the weston.ini mirror-of relationships at startup, > > > > check whether the source output of the mirror-of declaration supports > > > > that ability above. If so: > > > > - Wire up a signal so that the source output announces each newly > > > > rendered frame to any/all mirror-of outputs. > > > > - The virtual output's backend allocates its own buffer pool in > > > > exactly the same way that MR 1366 already does. > > > > - Upon receipt of each signal announcing a new frame's renderbuffer > > > > from the source output, the virtual output does a very cheap > > > > glBlitFramebuffer() to get the contents into its own buffer pool. > > > > This avoids the possibility of an unresponsive client tying up the > > > > source output's buffer. > > > > > > > > * If the source output isn't one that supports exposing the underlying > > > > renderbuffer, then the mirror-of relationship continues as with MR > > > > 1476 just to invoke an explicit weston_renderer pass to draw the > > > > correct logical contents. > > > > > > > > How does this strike you? > > > > > > Sorry, I don't understand how that idea is relevant to the KMS writeback > > case. > > > Did you imply that DRM-backend could deliver a KMS-writeback FB instead > > of > > > the rendered FB? > > > > Just for the same of argument, yes. But I take your point below that this > > entire > > idea to drive the mirroring output directly from the source output's > > rendering > > doesn't fit the plan for the mirror-of semantics. > > > > > > > > This is not the current plan for mirror-of. It does not allow the mirror > > > output > > to > > > run on i
RE: Full-motion zero-copy screen capture in Weston
> -Original Message- > From: Hoosier, Matt > Sent: Wednesday, June 12, 2024 8:37 AM > To: Pekka Paalanen > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > ; wayland-devel@lists.freedesktop.org; Daniel > Stone > Subject: RE: Full-motion zero-copy screen capture in Weston > > > -Original Message- > > From: Pekka Paalanen > > Sent: Wednesday, June 12, 2024 4:03 AM > > To: Hoosier, Matt > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > ; wayland-devel@lists.freedesktop.org; Daniel > > Stone > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > On Mon, 10 Jun 2024 14:18:39 + > > "Hoosier, Matt" wrote: > > > > > Yes, the native Linux driver for the hardware still does end up being > > > responsible for one or more planes. > > > > > > Other than trying to arrange the pieces so that there's some API that > > > offers the client an option to guarantee that the source of the screen > > > capture involves the writeback connector (similar to what you've done > > > with weston_output_capture), I don't really think it would be a good > > > idea to make Weston explicitly aware of any of this funny hypervisor > > > business going on. > > > > > > > I very much agree. > > Ack > > > > > > > The title on this email conversation was probably poorly chosen, in > > > retrospect. I'm not so much concerned with keeping absolutely to a > > > zero-copy mechanism as I am to using hardware mechanisms (GL > > > rendering, DRI writeback, hardware-accelerated blits as necessary) all > > > the way through. > > > > > > After seeing the way that the Pipewire backend handles streaming > > > (especially with > > > https://gitlab.freedesktop.org/wayland/weston/- > /merge_requests/1366), > > > I'm almost thinking that it would be preferable to maybe structure the > > > design of this feature something like this: > > > > > > * Add some sort of API on weston_output by which it can advertise the > > > ability to lay hands on the explicit renderbuffer (e.g. > > > gbm_surface_get_front_buffer()). This roughly equates to whether it's > > > a primary, non-nested backend. That is, DRM. > > > > > > * When processing the weston.ini mirror-of relationships at startup, > > > check whether the source output of the mirror-of declaration supports > > > that ability above. If so: > > > - Wire up a signal so that the source output announces each newly > > > rendered frame to any/all mirror-of outputs. > > > - The virtual output's backend allocates its own buffer pool in > > > exactly the same way that MR 1366 already does. > > > - Upon receipt of each signal announcing a new frame's renderbuffer > > > from the source output, the virtual output does a very cheap > > > glBlitFramebuffer() to get the contents into its own buffer pool. > > > This avoids the possibility of an unresponsive client tying up the > > > source output's buffer. > > > > > > * If the source output isn't one that supports exposing the underlying > > > renderbuffer, then the mirror-of relationship continues as with MR > > > 1476 just to invoke an explicit weston_renderer pass to draw the > > > correct logical contents. > > > > > > How does this strike you? > > > > Sorry, I don't understand how that idea is relevant to the KMS writeback > case. > > Did you imply that DRM-backend could deliver a KMS-writeback FB instead > of > > the rendered FB? > > Just for the same of argument, yes. But I take your point below that this > entire > idea to drive the mirroring output directly from the source output's rendering > doesn't fit the plan for the mirror-of semantics. > > > > > This is not the current plan for mirror-of. It does not allow the mirror > > output > to > > run on its own pace independently of the source output. Re-using the source > > output's rendered FB would also be a problem for color management. The FB > > is rendered for that output, and the color properties of the mirror may be > > different, needing an independent rendering anyway. > > > > The fundamental difference is who defines the properties of the stream. > > KMS writeback steam properties are necessarily defined by the KMS output. > > Mirror-of is for the opposite, for full flexibility. E.g. a remote mirror > > may
RE: Full-motion zero-copy screen capture in Weston
> -Original Message- > From: Pekka Paalanen > Sent: Wednesday, June 12, 2024 4:03 AM > To: Hoosier, Matt > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > ; wayland-devel@lists.freedesktop.org; Daniel > Stone > Subject: Re: Full-motion zero-copy screen capture in Weston > > On Mon, 10 Jun 2024 14:18:39 + > "Hoosier, Matt" wrote: > > > Yes, the native Linux driver for the hardware still does end up being > > responsible for one or more planes. > > > > Other than trying to arrange the pieces so that there's some API that > > offers the client an option to guarantee that the source of the screen > > capture involves the writeback connector (similar to what you've done > > with weston_output_capture), I don't really think it would be a good > > idea to make Weston explicitly aware of any of this funny hypervisor > > business going on. > > > > I very much agree. Ack > > > The title on this email conversation was probably poorly chosen, in > > retrospect. I'm not so much concerned with keeping absolutely to a > > zero-copy mechanism as I am to using hardware mechanisms (GL > > rendering, DRI writeback, hardware-accelerated blits as necessary) all > > the way through. > > > > After seeing the way that the Pipewire backend handles streaming > > (especially with > > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366), > > I'm almost thinking that it would be preferable to maybe structure the > > design of this feature something like this: > > > > * Add some sort of API on weston_output by which it can advertise the > > ability to lay hands on the explicit renderbuffer (e.g. > > gbm_surface_get_front_buffer()). This roughly equates to whether it's > > a primary, non-nested backend. That is, DRM. > > > > * When processing the weston.ini mirror-of relationships at startup, > > check whether the source output of the mirror-of declaration supports > > that ability above. If so: > > - Wire up a signal so that the source output announces each newly > > rendered frame to any/all mirror-of outputs. > > - The virtual output's backend allocates its own buffer pool in > > exactly the same way that MR 1366 already does. > > - Upon receipt of each signal announcing a new frame's renderbuffer > > from the source output, the virtual output does a very cheap > > glBlitFramebuffer() to get the contents into its own buffer pool. > > This avoids the possibility of an unresponsive client tying up the > > source output's buffer. > > > > * If the source output isn't one that supports exposing the underlying > > renderbuffer, then the mirror-of relationship continues as with MR > > 1476 just to invoke an explicit weston_renderer pass to draw the > > correct logical contents. > > > > How does this strike you? > > Sorry, I don't understand how that idea is relevant to the KMS writeback case. > Did you imply that DRM-backend could deliver a KMS-writeback FB instead of > the rendered FB? Just for the same of argument, yes. But I take your point below that this entire idea to drive the mirroring output directly from the source output's rendering doesn't fit the plan for the mirror-of semantics. > > This is not the current plan for mirror-of. It does not allow the mirror > output to > run on its own pace independently of the source output. Re-using the source > output's rendered FB would also be a problem for color management. The FB > is rendered for that output, and the color properties of the mirror may be > different, needing an independent rendering anyway. > > The fundamental difference is who defines the properties of the stream. > KMS writeback steam properties are necessarily defined by the KMS output. > Mirror-of is for the opposite, for full flexibility. E.g. a remote mirror may > not > want to run at the full framerate, the physical monitor resolution, or with > the > color capabilities of the source output in order to save bandwidth and improve > latency. Okay, understood. Although I'm a bit curious how you can say that one output "mirrors" another whose resolution doesn't even match. Maybe you have scaling in mind? > > It seems to me that we will need two different mechanisms for the two cases. I > believe KMS writeback streaming is worthwhile to support, but not via mirror- > of key. Maybe the writeback streaming should be built into the DRM-backend > as a special pipewire source? Then it would also be guaranteed to be KMS > writeback. It is some amount of code and testi
Re: Full-motion zero-copy screen capture in Weston
On Mon, 10 Jun 2024 14:18:39 + "Hoosier, Matt" wrote: > Yes, the native Linux driver for the hardware still does end up being > responsible for one or more planes. > > Other than trying to arrange the pieces so that there's some API that > offers the client an option to guarantee that the source of the > screen capture involves the writeback connector (similar to what > you've done with weston_output_capture), I don't really think it > would be a good idea to make Weston explicitly aware of any of this > funny hypervisor business going on. > I very much agree. > The title on this email conversation was probably poorly chosen, in > retrospect. I'm not so much concerned with keeping absolutely to a > zero-copy mechanism as I am to using hardware mechanisms (GL > rendering, DRI writeback, hardware-accelerated blits as necessary) > all the way through. > > After seeing the way that the Pipewire backend handles streaming > (especially with > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366), > I'm almost thinking that it would be preferable to maybe structure > the design of this feature something like this: > > * Add some sort of API on weston_output by which it can advertise the > ability to lay hands on the explicit renderbuffer (e.g. > gbm_surface_get_front_buffer()). This roughly equates to whether it's > a primary, non-nested backend. That is, DRM. > > * When processing the weston.ini mirror-of relationships at startup, > check whether the source output of the mirror-of declaration supports > that ability above. If so: > - Wire up a signal so that the source output announces each newly > rendered frame to any/all mirror-of outputs. > - The virtual output's backend allocates its own buffer pool in > exactly the same way that MR 1366 already does. > - Upon receipt of each signal announcing a new frame's renderbuffer > from the source output, the virtual output does a very cheap > glBlitFramebuffer() to get the contents into its own buffer pool. > This avoids the possibility of an unresponsive client tying up the > source output's buffer. > > * If the source output isn't one that supports exposing the > underlying renderbuffer, then the mirror-of relationship continues as > with MR 1476 just to invoke an explicit weston_renderer pass to draw > the correct logical contents. > > How does this strike you? Sorry, I don't understand how that idea is relevant to the KMS writeback case. Did you imply that DRM-backend could deliver a KMS-writeback FB instead of the rendered FB? This is not the current plan for mirror-of. It does not allow the mirror output to run on its own pace independently of the source output. Re-using the source output's rendered FB would also be a problem for color management. The FB is rendered for that output, and the color properties of the mirror may be different, needing an independent rendering anyway. The fundamental difference is who defines the properties of the stream. KMS writeback steam properties are necessarily defined by the KMS output. Mirror-of is for the opposite, for full flexibility. E.g. a remote mirror may not want to run at the full framerate, the physical monitor resolution, or with the color capabilities of the source output in order to save bandwidth and improve latency. It seems to me that we will need two different mechanisms for the two cases. I believe KMS writeback streaming is worthwhile to support, but not via mirror-of key. Maybe the writeback streaming should be built into the DRM-backend as a special pipewire source? Then it would also be guaranteed to be KMS writeback. It is some amount of code and testing duplication, but I think it would be the simplest. I don't see KMS writeback streaming as specific to your curious virtualization case. I can imagine it being useful in general, to ensure that display controller output is correct for example, or when overall rendering efficiency is more important than optimal stream format. Thanks, pq pgpvurOu3w1aS.pgp Description: OpenPGP digital signature
Re: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston)
Hi Matt, On Fri, 7 Jun 2024 at 16:30, Hoosier, Matt wrote: > Okay, makes sense that you don’t want to have to repeat the dependencies’ > builds for every CI test. I’m not arguing that you should – it was just more > a thought experiment to see whether riding Meson subprojects is a reasonable > idea for establishing a development environment. > > I get your point that that can become a deep rabbit hole. But it seems that > you didn’t have any need to build LLVM and similar just to support the > hand-built copy of Mesa that’s in the CI. Is there some reason why a deeper > set of transitive dependencies would be needed using Meson subprojects than > when building by hand? Seems like I could probably just mimic what you’ve > done. Maybe your point is that the CI is a very constrained environment > that’s known not to need ATI or llvmpipe, but a general developer situation > with physical machines would? Oh no, the CI environment absolutely needs llvmpipe! We install quite a few development packages (cf .gitlab-ci/debian-install.sh) into the CI environment though, so although we don't build LLVM, we do absolutely depend on distro LLVM development packages which aren't present in a clean distro install. You're completely right though that it makes no difference to the dependency chain whether the dependencies come from Meson subprojects or previous installs though. Cheers, Daniel
RE: Full-motion zero-copy screen capture in Weston
Yes, the native Linux driver for the hardware still does end up being responsible for one or more planes. Other than trying to arrange the pieces so that there's some API that offers the client an option to guarantee that the source of the screen capture involves the writeback connector (similar to what you've done with weston_output_capture), I don't really think it would be a good idea to make Weston explicitly aware of any of this funny hypervisor business going on. The title on this email conversation was probably poorly chosen, in retrospect. I'm not so much concerned with keeping absolutely to a zero-copy mechanism as I am to using hardware mechanisms (GL rendering, DRI writeback, hardware-accelerated blits as necessary) all the way through. After seeing the way that the Pipewire backend handles streaming (especially with https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366), I'm almost thinking that it would be preferable to maybe structure the design of this feature something like this: * Add some sort of API on weston_output by which it can advertise the ability to lay hands on the explicit renderbuffer (e.g. gbm_surface_get_front_buffer()). This roughly equates to whether it's a primary, non-nested backend. That is, DRM. * When processing the weston.ini mirror-of relationships at startup, check whether the source output of the mirror-of declaration supports that ability above. If so: - Wire up a signal so that the source output announces each newly rendered frame to any/all mirror-of outputs. - The virtual output's backend allocates its own buffer pool in exactly the same way that MR 1366 already does. - Upon receipt of each signal announcing a new frame's renderbuffer from the source output, the virtual output does a very cheap glBlitFramebuffer() to get the contents into its own buffer pool. This avoids the possibility of an unresponsive client tying up the source output's buffer. * If the source output isn't one that supports exposing the underlying renderbuffer, then the mirror-of relationship continues as with MR 1476 just to invoke an explicit weston_renderer pass to draw the correct logical contents. How does this strike you? -Matt > -Original Message- > From: Pekka Paalanen > Sent: Monday, June 10, 2024 3:03 AM > To: Hoosier, Matt > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > ; wayland-devel@lists.freedesktop.org > Subject: Re: Full-motion zero-copy screen capture in Weston > > On Fri, 7 Jun 2024 14:16:32 + > "Hoosier, Matt" wrote: > > > > What do you mean you can capture all virtual machines with KMS > > > writeback? > > > > > > What's the architecture there? How do virtual machines access KMS > > > hardware? Why would Weston be able to capture their outputs at all? > > > > The world of virtualization on commercially supported embedded SOCs > > for big-scale production use is wild. Every vendor typically has only > > a narrow range of supported hypervisors. Usually, one -- and an > > in-house model at that. There will typically be at least one bizarre > > twist on the virtualization method for display controllers. > > > > One fad is for one of the virtual machines -- typically, the one > > running a normal-style GNU/Linux Yocto system -- to own privileged I/O > > maps of most of the hardware, and it more or less runs the same > > drivers inside this VM that the SOC maker has already written for > > their bare-metal Linux deployment. Most hardware peripherals are just > > exposed to the other guest VMs by having the privileged Linux VM host > > export some sort of hypervisor-integrated VirtIO-style backend for the > > hardware. The guests see VirtIO devices. This approach goes by the > > name of "paravirtualization". > > > > But for graphics and display, there is almost always some additional > > mechanism to sidestep this pure paravirtualization because it's > > perceived as too expensive. So the vendor may do something like > > designate some subset of planes on each connector to be "directly" > > manipulated by the non-GNU/Linux guest VMs. The hypervisor executive > > runs a tiny little server that receives the stream of plane updates, > > and during the vsync it programs the appropriate display controller > > hardware registers to refer to the new frame's contents. It's very > > limited -- the guests VMs whose scene updates are couched using this > > mechanism are not able to do modesets or reconfigure the overall DRI > > scene topology. > > > > The key point here is that because Linux is running a full physical > > driver, the writebac
Re: Full-motion zero-copy screen capture in Weston
On Fri, 7 Jun 2024 14:16:32 + "Hoosier, Matt" wrote: > > What do you mean you can capture all virtual machines with KMS > > writeback? > > > > What's the architecture there? How do virtual machines access KMS > > hardware? Why would Weston be able to capture their outputs at all? > > The world of virtualization on commercially supported embedded SOCs > for big-scale production use is wild. Every vendor typically has only > a narrow range of supported hypervisors. Usually, one -- and an > in-house model at that. There will typically be at least one bizarre > twist on the virtualization method for display controllers. > > One fad is for one of the virtual machines -- typically, the one > running a normal-style GNU/Linux Yocto system -- to own privileged > I/O maps of most of the hardware, and it more or less runs the same > drivers inside this VM that the SOC maker has already written for > their bare-metal Linux deployment. Most hardware peripherals are just > exposed to the other guest VMs by having the privileged Linux VM host > export some sort of hypervisor-integrated VirtIO-style backend for > the hardware. The guests see VirtIO devices. This approach goes by > the name of "paravirtualization". > > But for graphics and display, there is almost always some additional > mechanism to sidestep this pure paravirtualization because it's > perceived as too expensive. So the vendor may do something like > designate some subset of planes on each connector to be "directly" > manipulated by the non-GNU/Linux guest VMs. The hypervisor executive > runs a tiny little server that receives the stream of plane updates, > and during the vsync it programs the appropriate display controller > hardware registers to refer to the new frame's contents. It's very > limited -- the guests VMs whose scene updates are couched using this > mechanism are not able to do modesets or reconfigure the overall DRI > scene topology. > > The key point here is that because Linux is running a full physical > driver, the writeback connector gets the results of blending all the > layers -- even those whose contents are programmed using the awkward > side channel. > > I'm not a big fan on this approach. But it is there, and I'd like to > cope with it. I have a use-case that requires Linux to get a complete > picture of the physical contents getting scanned out by the connector. > I see. On such connectors, does Weston still paint at least one KMS plane? If we can pretend that the connector is still a normal output for Weston, just with less planes, I feel that this should be implementable in upstream Weston. Thanks, pq pgpqzxgORl36l.pgp Description: OpenPGP digital signature
RE: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston)
Hi Daniel, Okay, makes sense that you don’t want to have to repeat the dependencies’ builds for every CI test. I’m not arguing that you should – it was just more a thought experiment to see whether riding Meson subprojects is a reasonable idea for establishing a development environment. I get your point that that can become a deep rabbit hole. But it seems that you didn’t have any need to build LLVM and similar just to support the hand-built copy of Mesa that’s in the CI. Is there some reason why a deeper set of transitive dependencies would be needed using Meson subprojects than when building by hand? Seems like I could probably just mimic what you’ve done. Maybe your point is that the CI is a very constrained environment that’s known not to need ATI or llvmpipe, but a general developer situation with physical machines would? -Matt From: Daniel Stone Sent: Friday, June 7, 2024 10:17 AM To: Hoosier, Matt Cc: Pekka Paalanen ; Marius Vlad ; wayland-devel@lists.freedesktop.org Subject: Re: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston) Hi Matt, On Fri, 7 Jun 2024 at 15: 30, Hoosier, Matt wrote: > Would Meson’s dependency wrapping capabilities be a viable solution here? I think that most of Weston’s dependencies that have aggressive version jQcmQRYFpfptBannerStart Hi Matt, On Fri, 7 Jun 2024 at 15:30, Hoosier, Matt mailto:matt.hoos...@garmin.com>> wrote: > Would Meson’s dependency wrapping capabilities be a viable solution here? I > think that most of Weston’s dependencies that have aggressive version > requirements are themselves also Meson projects. > > The Weston CI configuration builds a bunch of its dependencies (Mesa, libdrm, > libwayland …) manually. I wonder why Meson wrapping was not used for this? We don't want to rebuild Mesa every time. We could've built it as a subproject and cached it, but it didn't seem to offer much any advantage over just installing it into the system. We could probably add some subprojects, but you'd probably end up pulling in more components as well - e.g. if you want to run Mesa with its software renderer or the AMD drivers, you'll also need to use LLVM - and at what point does your easy subproject build turn into, well, a full distribution? I guess one thing we could do is to jazz the CI build up a little so it's easier to pull the OCI and run it inside a toolbox, as well as reuse those scripts locally. Cheers, Daniel
Re: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston)
Hi Matt, On Fri, 7 Jun 2024 at 15:30, Hoosier, Matt wrote: > Would Meson’s dependency wrapping capabilities be a viable solution here? I > think that most of Weston’s dependencies that have aggressive version > requirements are themselves also Meson projects. > > The Weston CI configuration builds a bunch of its dependencies (Mesa, libdrm, > libwayland …) manually. I wonder why Meson wrapping was not used for this? We don't want to rebuild Mesa every time. We could've built it as a subproject and cached it, but it didn't seem to offer much any advantage over just installing it into the system. We could probably add some subprojects, but you'd probably end up pulling in more components as well - e.g. if you want to run Mesa with its software renderer or the AMD drivers, you'll also need to use LLVM - and at what point does your easy subproject build turn into, well, a full distribution? I guess one thing we could do is to jazz the CI build up a little so it's easier to pull the OCI and run it inside a toolbox, as well as reuse those scripts locally. Cheers, Daniel
RE: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston)
Would Meson’s dependency wrapping capabilities be a viable solution here? I think that most of Weston’s dependencies that have aggressive version requirements are themselves also Meson projects. The Weston CI configuration builds a bunch of its dependencies (Mesa, libdrm, libwayland …) manually. I wonder why Meson wrapping was not used for this? -Matt From: Hoosier, Matt Sent: Wednesday, June 5, 2024 7:44 AM To: Daniel Stone ; Pekka Paalanen Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad ; wayland-devel@lists.freedesktop.org Subject: Re: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston) Thanks, everybody. After some trial and error, I find that if I install seatd in the host and the seatd dev package in the Toolbox container and I then symlink the host seatd socket into /tmp on the container, Weston seems to start up okay on my physical KMS connectors: user@host:~$ toolbox enter … user@toolbox:~$ sudo ln -s /run/host/run/seatd.socket /run/ user@toolbox:~$ weston --backend=drm -Matt From: Daniel Stone mailto:dan...@fooishbar.org>> Date: Wednesday, June 5, 2024 at 5:07 AM To: Pekka Paalanen mailto:pekka.paala...@collabora.com>> Cc: "Hoosier, Matt" mailto:matt.hoos...@garmin.com>>, "s...@cmpwn.com<mailto:s...@cmpwn.com>" mailto:s...@cmpwn.com>>, "cont...@emersion.fr<mailto:cont...@emersion.fr>" mailto:cont...@emersion.fr>>, Marius Vlad mailto:marius.v...@collabora.com>>, "wayland-devel@lists.freedesktop.org<mailto:wayland-devel@lists.freedesktop.org>" mailto:wayland-devel@lists.freedesktop.org>> Subject: Re: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston) Hi, On Wed, 5 Jun 2024 at 09: 09, Pekka Paalanen wrote: > On Tue, 4 Jun 2024 20: 33: 48 + > "Hoosier, Matt" wrote: > > Tactical question: I somehow missed until Hi, On Wed, 5 Jun 2024 at 09:09, Pekka Paalanen mailto:pekka.paala...@collabora.com>> wrote: > On Tue, 4 Jun 2024 20:33:48 + > "Hoosier, Matt" mailto:matt.hoos...@garmin.com>> > wrote: > > Tactical question: I somehow missed until this point that the remote > > and pipewire plugins will only run if the DRM backend is being used. > > > > But the DRM backend *really* doesn't want to start nowadays unless > > you're running on a system with seatd and/or logind available. > > Toolbox [1] is the de facto way to develop on bleeding edge copies of > > components these days. But it logind and seatd aren't exposed into it. > > > > How do Weston people interactively develop on the Weston DRM backend > > nowadays? > > > > [1] > > https://urldefense.com/v3/__https://docs.fedoraproject.org/en-US/fedora-silverblue/toolbox/__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK2SJoQzfs$<https://urldefense.com/v3/__https:/docs.fedoraproject.org/en-US/fedora-silverblue/toolbox/__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK2SJoQzfs$> > > I'm doing it old-school on my workstation, without any containers. What > dependencies my distribution does not provide, I build and install > manually into a prefix under $HOME: > > https://urldefense.com/v3/__https://www.collabora.com/news-and-blog/blog/2020/04/10/clean-reliable-setup-for-dependency-installation/__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK2Y5E0gB4$<https://urldefense.com/v3/__https:/www.collabora.com/news-and-blog/blog/2020/04/10/clean-reliable-setup-for-dependency-installation/__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK2Y5E0gB4$> > > The "clean and reliable" is probably outdated in this era of > containers... Yes, doing it in containers is a little bit tricky since it's not exactly the design case. Honestly, on my Silverblue systems, I just install a bunch of relevant dependencies into the system image with rpm-ostree, and have a pile of self-built dependencies in a local prefix. This might give you some insight however: https://urldefense.com/v3/__https://github.com/containers/toolbox/issues/992__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK21Tr-34M$<https://urldefense.com/v3/__https:/github.com/containers/toolbox/issues/992__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK21Tr-34M$> It probably needs some minor changes in Weston but does at least seem doable ... Cheers, Daniel
Re: Full-motion zero-copy screen capture in Weston
> What do you mean you can capture all virtual machines with KMS writeback? > > What's the architecture there? How do virtual machines access KMS hardware? Why would Weston be able to capture their outputs at all? The world of virtualization on commercially supported embedded SOCs for big-scale production use is wild. Every vendor typically has only a narrow range of supported hypervisors. Usually, one -- and an in-house model at that. There will typically be at least one bizarre twist on the virtualization method for display controllers. One fad is for one of the virtual machines -- typically, the one running a normal-style GNU/Linux Yocto system -- to own privileged I/O maps of most of the hardware, and it more or less runs the same drivers inside this VM that the SOC maker has already written for their bare-metal Linux deployment. Most hardware peripherals are just exposed to the other guest VMs by having the privileged Linux VM host export some sort of hypervisor-integrated VirtIO-style backend for the hardware. The guests see VirtIO devices. This approach goes by the name of "paravirtualization". But for graphics and display, there is almost always some additional mechanism to sidestep this pure paravirtualization because it's perceived as too expensive. So the vendor may do something like designate some subset of planes on each connector to be "directly" manipulated by the non-GNU/Linux guest VMs. The hypervisor executive runs a tiny little server that receives the stream of plane updates, and during the vsync it programs the appropriate display controller hardware registers to refer to the new frame's contents. It's very limited -- the guests VMs whose scene updates are couched using this mechanism are not able to do modesets or reconfigure the overall DRI scene topology. The key point here is that because Linux is running a full physical driver, the writeback connector gets the results of blending all the layers -- even those whose contents are programmed using the awkward side channel. I'm not a big fan on this approach. But it is there, and I'd like to cope with it. I have a use-case that requires Linux to get a complete picture of the physical contents getting scanned out by the connector. -Matt On 6/3/24, 3:54 AM, "Pekka Paalanen" mailto:pekka.paala...@collabora.com>> wrote: On Fri, 31 May 2024 13:26:12 + "Hoosier, Matt" mailto:matt.hoos...@garmin.com>> wrote: > > My goal is to implement this screen capture with a guarantee that the > copy comes from a KMS writeback connector. I know this sounds like an > odd thing to insist on. Let's say that in my industry, the system is > rarely only running Linux directly on the bare metal. Using the > writeback hardware on the display controller allows to get a copy of > content from all the virtual machines in the system. > What do you mean you can capture all virtual machines with KMS writeback? What's the architecture there? How do virtual machines access KMS hardware? Why would Weston be able to capture their outputs at all? > Frankly, weston_output_capture_v1's model that clients allocate the > buffers would make it very difficult to support efficient screen > capture for more than one simultaneous client. You can only target > one writeback framebuffer per page flip. Having the compositor manage > the buffers' lifetimes and just publishing out handles (in the style > of those two wlr extensions) to those probably fits better. That's certainly true. The disadvantage is that buffer allocations are accounted to the compositor (which can manage its own memory consumption, sure), and either the compositor allocates a new buffer for every frame (possibly serious overhead) or it needs to wait for all consumers to tell that they are done with the buffer before it can be used again. Clients might hold on to the buffer indefinitely or just a little too long, which is the risk. Thanks, pq
Re: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston)
Thanks, everybody. After some trial and error, I find that if I install seatd in the host and the seatd dev package in the Toolbox container and I then symlink the host seatd socket into /tmp on the container, Weston seems to start up okay on my physical KMS connectors: user@host:~$ toolbox enter … user@toolbox:~$ sudo ln -s /run/host/run/seatd.socket /run/ user@toolbox:~$ weston --backend=drm -Matt From: Daniel Stone Date: Wednesday, June 5, 2024 at 5:07 AM To: Pekka Paalanen Cc: "Hoosier, Matt" , "s...@cmpwn.com" , "cont...@emersion.fr" , Marius Vlad , "wayland-devel@lists.freedesktop.org" Subject: Re: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston) Hi, On Wed, 5 Jun 2024 at 09: 09, Pekka Paalanen wrote: > On Tue, 4 Jun 2024 20: 33: 48 + > "Hoosier, Matt" wrote: > > Tactical question: I somehow missed until Hi, On Wed, 5 Jun 2024 at 09:09, Pekka Paalanen wrote: > On Tue, 4 Jun 2024 20:33:48 + > "Hoosier, Matt" wrote: > > Tactical question: I somehow missed until this point that the remote > > and pipewire plugins will only run if the DRM backend is being used. > > > > But the DRM backend *really* doesn't want to start nowadays unless > > you're running on a system with seatd and/or logind available. > > Toolbox [1] is the de facto way to develop on bleeding edge copies of > > components these days. But it logind and seatd aren't exposed into it. > > > > How do Weston people interactively develop on the Weston DRM backend > > nowadays? > > > > [1] > > https://urldefense.com/v3/__https://docs.fedoraproject.org/en-US/fedora-silverblue/toolbox/__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK2SJoQzfs$<https://urldefense.com/v3/__https:/docs.fedoraproject.org/en-US/fedora-silverblue/toolbox/__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK2SJoQzfs$> > > I'm doing it old-school on my workstation, without any containers. What > dependencies my distribution does not provide, I build and install > manually into a prefix under $HOME: > > https://urldefense.com/v3/__https://www.collabora.com/news-and-blog/blog/2020/04/10/clean-reliable-setup-for-dependency-installation/__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK2Y5E0gB4$<https://urldefense.com/v3/__https:/www.collabora.com/news-and-blog/blog/2020/04/10/clean-reliable-setup-for-dependency-installation/__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK2Y5E0gB4$> > > The "clean and reliable" is probably outdated in this era of > containers... Yes, doing it in containers is a little bit tricky since it's not exactly the design case. Honestly, on my Silverblue systems, I just install a bunch of relevant dependencies into the system image with rpm-ostree, and have a pile of self-built dependencies in a local prefix. This might give you some insight however: https://urldefense.com/v3/__https://github.com/containers/toolbox/issues/992__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK21Tr-34M$<https://urldefense.com/v3/__https:/github.com/containers/toolbox/issues/992__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK21Tr-34M$> It probably needs some minor changes in Weston but does at least seem doable ... Cheers, Daniel
Re: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston)
Hi, On Wed, 5 Jun 2024 at 09:09, Pekka Paalanen wrote: > On Tue, 4 Jun 2024 20:33:48 + > "Hoosier, Matt" wrote: > > Tactical question: I somehow missed until this point that the remote > > and pipewire plugins will only run if the DRM backend is being used. > > > > But the DRM backend *really* doesn't want to start nowadays unless > > you're running on a system with seatd and/or logind available. > > Toolbox [1] is the de facto way to develop on bleeding edge copies of > > components these days. But it logind and seatd aren't exposed into it. > > > > How do Weston people interactively develop on the Weston DRM backend > > nowadays? > > > > [1] https://docs.fedoraproject.org/en-US/fedora-silverblue/toolbox/ > > I'm doing it old-school on my workstation, without any containers. What > dependencies my distribution does not provide, I build and install > manually into a prefix under $HOME: > > https://www.collabora.com/news-and-blog/blog/2020/04/10/clean-reliable-setup-for-dependency-installation/ > > The "clean and reliable" is probably outdated in this era of > containers... Yes, doing it in containers is a little bit tricky since it's not exactly the design case. Honestly, on my Silverblue systems, I just install a bunch of relevant dependencies into the system image with rpm-ostree, and have a pile of self-built dependencies in a local prefix. This might give you some insight however: https://github.com/containers/toolbox/issues/992 It probably needs some minor changes in Weston but does at least seem doable ... Cheers, Daniel
Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston)
On Tue, 4 Jun 2024 20:33:48 + "Hoosier, Matt" wrote: > Tactical question: I somehow missed until this point that the remote > and pipewire plugins will only run if the DRM backend is being used. > > But the DRM backend *really* doesn't want to start nowadays unless > you're running on a system with seatd and/or logind available. > Toolbox [1] is the de facto way to develop on bleeding edge copies of > components these days. But it logind and seatd aren't exposed into it. > > How do Weston people interactively develop on the Weston DRM backend > nowadays? > > [1] https://docs.fedoraproject.org/en-US/fedora-silverblue/toolbox/ Hi Matt, I'm doing it old-school on my workstation, without any containers. What dependencies my distribution does not provide, I build and install manually into a prefix under $HOME: https://www.collabora.com/news-and-blog/blog/2020/04/10/clean-reliable-setup-for-dependency-installation/ The "clean and reliable" is probably outdated in this era of containers... There is also 'meson devenv', but since I need multiple manually built projects in the same session, I haven't really looked into how to make that work for me. https://mesonbuild.com/Commands.html#devenv When launching Weston, I have two different cases. Both need my prefix installed environment as in the blog post. - When I want to run on the same gfx card as my desktop, I VT-switch to a text mode login, log in, enter the environment, and simply run 'weston'. This gets input and output devices from logind. - When I want to run on my dedicated testing gfx card (no desktop there), I do it from a terminal window from my normal desktop: enter the environment, and run 'SEATD_VTBOUND=0 weston -Bdrm-backend.so --seat=seat-insecure -i0'. This requires prior setup: https://wayland.pages.freedesktop.org/weston/toc/running-weston.html#running-weston-on-a-different-seat-on-a-stand-alone-back-end It uses libseat's built-in backend, so no seatd and no logind. The prior setup gives my user direct access to the necessary device nodes. I have a separate monitor, keyboard and mouse reserved for this. Thanks, pq pgpi5sye1gU6b.pgp Description: OpenPGP digital signature
Re: Full-motion zero-copy screen capture in Weston
Hi, Note that systemd-udev will not work in a container. That usually has the side effect of libinput not working either. On Tue, Jun 04, 2024 at 08:33:48PM +, Hoosier, Matt wrote: > Tactical question: I somehow missed until this point that the remote and > pipewire plugins will only run if the DRM backend is being used. > > But the DRM backend *really* doesn't want to start nowadays unless you're > running on a system with seatd and/or logind available. Toolbox [1] is the de > facto way to develop on bleeding edge copies of components these days. But it > logind and seatd aren't exposed into it. > > How do Weston people interactively develop on the Weston DRM backend nowadays? With either seatd or logind. I suspect people use containers to build but run it on the device. > > [1] https://docs.fedoraproject.org/en-US/fedora-silverblue/toolbox/ > > > -Original Message- > > From: Pekka Paalanen > > Sent: Tuesday, June 4, 2024 3:20 AM > > To: Hoosier, Matt > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > ; wayland-devel@lists.freedesktop.org > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > On Mon, 3 Jun 2024 17:57:18 + > > "Hoosier, Matt" wrote: > > > > > > What do you mean you can capture all virtual machines with KMS > > > > writeback? > > > > > > > > What's the architecture there? How do virtual machines access KMS > > > > hardware? Why would Weston be able to capture their outputs at all? > > > > > > > > > > I hope to have more to say about this shortly. > > > > I'm interested because I worry whether it will work. I don't see how it > > could > > work with the traditional display controllers and DRM drivers. > > > > Maybe some kind of hardware assisted plane "leasing" that works around the > > DRM KMS software model limitations? > > > > > > The disadvantage is that buffer allocations are accounted to the > > > > compositor (which can manage its own memory consumption, sure), and > > > > either the compositor allocates a new buffer for every frame > > > > (possibly serious overhead) or it needs to wait for all consumers to > > > > tell that they are done with the buffer before it can be used again. > > > > Clients might hold on to the buffer indefinitely or just a little > > > > too long, which is the risk. > > > > > > It seems to me this would be a risk of the existing Pipewire backend, > > > no? At least, if I understood the buffering model of the dmabuf MR > > > that Marius linked earlier. > > > > > > > I haven't looked at any of that, so I can't say. I don't know if pipewire > > is only > > allocate-send-forget, or does it include consumers signalling they are done > > to > > allow re-use as well. Or maybe the sources trust that rotating N buffers is > > always enough, and if a sink needs the data for longer, it will make a copy > > in > > time. > > > > It's a trade-off. Sometimes the overhead of allocate-send-forget with the > > risk > > of OOM (KMS writeback might require a specific type of memory that could be > > scarce), maybe even with an added copy to avoid exhausting writeback- > > capable memory, may be justified. > > > > > > Thanks, > > pq signature.asc Description: PGP signature
RE: Full-motion zero-copy screen capture in Weston
Tactical question: I somehow missed until this point that the remote and pipewire plugins will only run if the DRM backend is being used. But the DRM backend *really* doesn't want to start nowadays unless you're running on a system with seatd and/or logind available. Toolbox [1] is the de facto way to develop on bleeding edge copies of components these days. But it logind and seatd aren't exposed into it. How do Weston people interactively develop on the Weston DRM backend nowadays? [1] https://docs.fedoraproject.org/en-US/fedora-silverblue/toolbox/ > -Original Message- > From: Pekka Paalanen > Sent: Tuesday, June 4, 2024 3:20 AM > To: Hoosier, Matt > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > ; wayland-devel@lists.freedesktop.org > Subject: Re: Full-motion zero-copy screen capture in Weston > > On Mon, 3 Jun 2024 17:57:18 + > "Hoosier, Matt" wrote: > > > > What do you mean you can capture all virtual machines with KMS > > > writeback? > > > > > > What's the architecture there? How do virtual machines access KMS > > > hardware? Why would Weston be able to capture their outputs at all? > > > > > > > I hope to have more to say about this shortly. > > I'm interested because I worry whether it will work. I don't see how it could > work with the traditional display controllers and DRM drivers. > > Maybe some kind of hardware assisted plane "leasing" that works around the > DRM KMS software model limitations? > > > > The disadvantage is that buffer allocations are accounted to the > > > compositor (which can manage its own memory consumption, sure), and > > > either the compositor allocates a new buffer for every frame > > > (possibly serious overhead) or it needs to wait for all consumers to > > > tell that they are done with the buffer before it can be used again. > > > Clients might hold on to the buffer indefinitely or just a little > > > too long, which is the risk. > > > > It seems to me this would be a risk of the existing Pipewire backend, > > no? At least, if I understood the buffering model of the dmabuf MR > > that Marius linked earlier. > > > > I haven't looked at any of that, so I can't say. I don't know if pipewire is > only > allocate-send-forget, or does it include consumers signalling they are done to > allow re-use as well. Or maybe the sources trust that rotating N buffers is > always enough, and if a sink needs the data for longer, it will make a copy in > time. > > It's a trade-off. Sometimes the overhead of allocate-send-forget with the risk > of OOM (KMS writeback might require a specific type of memory that could be > scarce), maybe even with an added copy to avoid exhausting writeback- > capable memory, may be justified. > > > Thanks, > pq
Re: Full-motion zero-copy screen capture in Weston
On Mon, 3 Jun 2024 17:57:18 + "Hoosier, Matt" wrote: > > What do you mean you can capture all virtual machines with KMS > > writeback? > > > > What's the architecture there? How do virtual machines access KMS > > hardware? Why would Weston be able to capture their outputs at all? > > > > I hope to have more to say about this shortly. I'm interested because I worry whether it will work. I don't see how it could work with the traditional display controllers and DRM drivers. Maybe some kind of hardware assisted plane "leasing" that works around the DRM KMS software model limitations? > > The disadvantage is that buffer allocations are accounted to the > > compositor (which can manage its own memory consumption, sure), and > > either the compositor allocates a new buffer for every frame > > (possibly serious overhead) or it needs to wait for all consumers > > to tell that they are done with the buffer before it can be used > > again. Clients might hold on to the buffer indefinitely or just a > > little too long, which is the risk. > > It seems to me this would be a risk of the existing Pipewire backend, > no? At least, if I understood the buffering model of the dmabuf MR > that Marius linked earlier. > I haven't looked at any of that, so I can't say. I don't know if pipewire is only allocate-send-forget, or does it include consumers signalling they are done to allow re-use as well. Or maybe the sources trust that rotating N buffers is always enough, and if a sink needs the data for longer, it will make a copy in time. It's a trade-off. Sometimes the overhead of allocate-send-forget with the risk of OOM (KMS writeback might require a specific type of memory that could be scarce), maybe even with an added copy to avoid exhausting writeback-capable memory, may be justified. Thanks, pq pgpUuGG4KCRnG.pgp Description: OpenPGP digital signature
Re: Full-motion zero-copy screen capture in Weston
> What do you mean you can capture all virtual machines with KMS > writeback? > > What's the architecture there? How do virtual machines access KMS > hardware? Why would Weston be able to capture their outputs at all? I hope to have more to say about this shortly. > The disadvantage is that buffer allocations are accounted to the > compositor (which can manage its own memory consumption, sure), and > either the compositor allocates a new buffer for every frame (possibly > serious overhead) or it needs to wait for all consumers to tell that > they are done with the buffer before it can be used again. Clients > might hold on to the buffer indefinitely or just a little too long, > which is the risk. It seems to me this would be a risk of the existing Pipewire backend, no? At least, if I understood the buffering model of the dmabuf MR that Marius linked earlier. On 6/3/24, 3:54 AM, "Pekka Paalanen" mailto:pekka.paala...@collabora.com>> wrote: On Fri, 31 May 2024 13:26:12 + "Hoosier, Matt" mailto:matt.hoos...@garmin.com>> wrote: > > My goal is to implement this screen capture with a guarantee that the > copy comes from a KMS writeback connector. I know this sounds like an > odd thing to insist on. Let's say that in my industry, the system is > rarely only running Linux directly on the bare metal. Using the > writeback hardware on the display controller allows to get a copy of > content from all the virtual machines in the system. > What do you mean you can capture all virtual machines with KMS writeback? What's the architecture there? How do virtual machines access KMS hardware? Why would Weston be able to capture their outputs at all? > Frankly, weston_output_capture_v1's model that clients allocate the > buffers would make it very difficult to support efficient screen > capture for more than one simultaneous client. You can only target > one writeback framebuffer per page flip. Having the compositor manage > the buffers' lifetimes and just publishing out handles (in the style > of those two wlr extensions) to those probably fits better. That's certainly true. The disadvantage is that buffer allocations are accounted to the compositor (which can manage its own memory consumption, sure), and either the compositor allocates a new buffer for every frame (possibly serious overhead) or it needs to wait for all consumers to tell that they are done with the buffer before it can be used again. Clients might hold on to the buffer indefinitely or just a little too long, which is the risk. Thanks, pq
Re: Full-motion zero-copy screen capture in Weston
On Fri, 31 May 2024 13:26:12 + "Hoosier, Matt" wrote: > > My goal is to implement this screen capture with a guarantee that the > copy comes from a KMS writeback connector. I know this sounds like an > odd thing to insist on. Let's say that in my industry, the system is > rarely only running Linux directly on the bare metal. Using the > writeback hardware on the display controller allows to get a copy of > content from all the virtual machines in the system. > What do you mean you can capture all virtual machines with KMS writeback? What's the architecture there? How do virtual machines access KMS hardware? Why would Weston be able to capture their outputs at all? > Frankly, weston_output_capture_v1's model that clients allocate the > buffers would make it very difficult to support efficient screen > capture for more than one simultaneous client. You can only target > one writeback framebuffer per page flip. Having the compositor manage > the buffers' lifetimes and just publishing out handles (in the style > of those two wlr extensions) to those probably fits better. That's certainly true. The disadvantage is that buffer allocations are accounted to the compositor (which can manage its own memory consumption, sure), and either the compositor allocates a new buffer for every frame (possibly serious overhead) or it needs to wait for all consumers to tell that they are done with the buffer before it can be used again. Clients might hold on to the buffer indefinitely or just a little too long, which is the risk. Thanks, pq pgpKFYV6jmA3W.pgp Description: OpenPGP digital signature
Re: Full-motion zero-copy screen capture in Weston
Hi Matt, fös., 31. maí 2024 kl. 15:34 skrifaði Hoosier, Matt : > > Thanks, understood. I would expect that I need to take some care in > allocating a buffer that my DRM driver accepts for writeback, in this usage. > > > > Do I interpret right that the wlroots screencopy extension wants the client > to create a fresh zwlr_screencopy_frame_v1 for each successive frame? I > wonder then: > > > > * Whether this might lead to the possibility of dropped frames, if the client > doesn’t manage to send back the requests to start capture for frame N+1 soon > enough after the delivery of the copied buffer for frame N; and If you send a frame request as soon as you receive an update, frames will not be dropped. > * Will the explicit request for each frame via zwlr_screencopy_frame_v1:: > copy(), result in artificial redraws to satisfy the request? Ideally I would > just like to receive a passive copy of each frame that naturally got drawn by > the compositor. Perhaps with one artificial redraw at the very beginning of > the screencopy stream just to have a defined starting point. For wlr-screencopy-v1, you can use the the "copy_with_damage" request to avoid "artificial redraws". See: https://wayland.app/protocols/wlr-screencopy-unstable-v1#zwlr_screencopy_frame_v1:request:copy_with_damage For ext-screencopy-v1, when you create a session, a redraw may be required for the first frame. After that, only damaged frames are copied for that session. See: https://wayland.app/protocols/wayland-protocols/124 Regards, Andri
Re: Full-motion zero-copy screen capture in Weston
Thanks, understood. I would expect that I need to take some care in allocating a buffer that my DRM driver accepts for writeback, in this usage. Do I interpret right that the wlroots screencopy extension wants the client to create a fresh zwlr_screencopy_frame_v1 for each successive frame? I wonder then: * Whether this might lead to the possibility of dropped frames, if the client doesn’t manage to send back the requests to start capture for frame N+1 soon enough after the delivery of the copied buffer for frame N; and * Will the explicit request for each frame via zwlr_screencopy_frame_v1:: copy(), result in artificial redraws to satisfy the request? Ideally I would just like to receive a passive copy of each frame that naturally got drawn by the compositor. Perhaps with one artificial redraw at the very beginning of the screencopy stream just to have a defined starting point. -Matt From: Simon Ser Date: Friday, May 31, 2024 at 10:16 AM To: Andri Yngvason Cc: "Hoosier, Matt" , Pekka Paalanen , "s...@cmpwn.com" , Marius Vlad , "wayland-devel@lists.freedesktop.org" Subject: Re: Full-motion zero-copy screen capture in Weston On Friday, May 31st, 2024 at 16: 45, Simon Ser wrote: > > See: https: //urldefense. com/v3/__https: //gitlab. freedesktop. org/wayland/wayland-protocols/-/merge_requests/124__;!!EJc4YC3iFmQ!WaOP607PgstX4CpWrn4DV6X4TIDRrqQW5XEZfczyVDFZdkpmCfNDh0lUQetO5TuCK5ct4NVlCqcAz4-8ehlw$ On Friday, May 31st, 2024 at 16:45, Simon Ser wrote: > > See: > > https://urldefense.com/v3/__https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/124__;!!EJc4YC3iFmQ!WaOP607PgstX4CpWrn4DV6X4TIDRrqQW5XEZfczyVDFZdkpmCfNDh0lUQetO5TuCK5ct4NVlCqcAz4-8ehlw$<https://urldefense.com/v3/__https:/gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/124__;!!EJc4YC3iFmQ!WaOP607PgstX4CpWrn4DV6X4TIDRrqQW5XEZfczyVDFZdkpmCfNDh0lUQetO5TuCK5ct4NVlCqcAz4-8ehlw$> > > > > > My goal is to implement this screen capture with a guarantee that the > > > copy comes from a KMS writeback connector. I know this sounds like an odd > > > thing to insist on. Let's say that in my industry, the system is rarely > > > only running Linux directly on the bare metal. Using the writeback > > > hardware on the display controller allows to get a copy of content from > > > all the virtual machines in the system. > > > > Although the protocol is called "screencopy", it does not need to be > > implemented via copying. I don't think anything precludes drm > > write-back or rendering directly to the client-submitted buffers. > > Not sure that's true… With screencopy the client necessarily provides > buffers that the compositor copies into. Correction: Andri said that it should be possible to pass client-allocated buffers to the writeback connector. There might be allocation considerations to take care of, but it might work fine indeed.
Re: Full-motion zero-copy screen capture in Weston
On Friday, May 31st, 2024 at 16:45, Simon Ser wrote: > > See: > > https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/124 > > > > > My goal is to implement this screen capture with a guarantee that the > > > copy comes from a KMS writeback connector. I know this sounds like an odd > > > thing to insist on. Let's say that in my industry, the system is rarely > > > only running Linux directly on the bare metal. Using the writeback > > > hardware on the display controller allows to get a copy of content from > > > all the virtual machines in the system. > > > > Although the protocol is called "screencopy", it does not need to be > > implemented via copying. I don't think anything precludes drm > > write-back or rendering directly to the client-submitted buffers. > > Not sure that's true… With screencopy the client necessarily provides > buffers that the compositor copies into. Correction: Andri said that it should be possible to pass client-allocated buffers to the writeback connector. There might be allocation considerations to take care of, but it might work fine indeed.
Re: Full-motion zero-copy screen capture in Weston
On Friday, May 31st, 2024 at 16:39, Andri Yngvason wrote: > Hi Matt, > > fös., 31. maí 2024 kl. 13:26 skrifaði Hoosier, Matt matt.hoos...@garmin.com: > > > Hi, > > > > Yeah. I agree that although I can prototype something quick and dirty here > > as a change to weston_output_capture_v1, it's probably not a good fit there. > > > > Drew or Simon, does either of > > https://github.com/swaywm/wlroots/blob/master/protocol/wlr-screencopy-unstable-v1.xml > > or > > https://github.com/swaywm/wlroots/blob/master/protocol/wlr-export-dmabuf-unstable-v1.xml > > strike you as a good fit for the use-case of getting a streamed copy of an > > output? They seem very similar – not sure what differentiates them from > > each other. > > > I'd say ext-screencopy-v1 is pretty close to being complete. It is > designed for both single shot capture and screen casting. It has 2 > acks and it has been implemented in wlroots, cosmic and wayvnc. > > I would not recommend wlr-export-dmabuf for anything as it suffers > from synchronisation issues. There are no guarantees for life-times of > the dmabufs. wlr-screencopy works well enough, but ext-screencopy-v1 > is better. For fixing the sync issue, there is: https://gitlab.freedesktop.org/wlroots/wlr-protocols/-/merge_requests/121 But not enough interest it seems. > See: > https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/124 > > > My goal is to implement this screen capture with a guarantee that the copy > > comes from a KMS writeback connector. I know this sounds like an odd thing > > to insist on. Let's say that in my industry, the system is rarely only > > running Linux directly on the bare metal. Using the writeback hardware on > > the display controller allows to get a copy of content from all the virtual > > machines in the system. > > > Although the protocol is called "screencopy", it does not need to be > implemented via copying. I don't think anything precludes drm > write-back or rendering directly to the client-submitted buffers. Not sure that's true… With screencopy the client necessarily provides buffers that the compositor copies into.
Re: Full-motion zero-copy screen capture in Weston
Hi Matt, fös., 31. maí 2024 kl. 13:26 skrifaði Hoosier, Matt : > > Hi, > > Yeah. I agree that although I can prototype something quick and dirty here as > a change to weston_output_capture_v1, it's probably not a good fit there. > > Drew or Simon, does either of > https://github.com/swaywm/wlroots/blob/master/protocol/wlr-screencopy-unstable-v1.xml > or > https://github.com/swaywm/wlroots/blob/master/protocol/wlr-export-dmabuf-unstable-v1.xml > strike you as a good fit for the use-case of getting a streamed copy of an > output? They seem very similar – not sure what differentiates them from each > other. > I'd say ext-screencopy-v1 is pretty close to being complete. It is designed for both single shot capture and screen casting. It has 2 acks and it has been implemented in wlroots, cosmic and wayvnc. I would not recommend wlr-export-dmabuf for anything as it suffers from synchronisation issues. There are no guarantees for life-times of the dmabufs. wlr-screencopy works well enough, but ext-screencopy-v1 is better. See: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/124 > My goal is to implement this screen capture with a guarantee that the copy > comes from a KMS writeback connector. I know this sounds like an odd thing to > insist on. Let's say that in my industry, the system is rarely only running > Linux directly on the bare metal. Using the writeback hardware on the display > controller allows to get a copy of content from all the virtual machines in > the system. Although the protocol is called "screencopy", it does not need to be implemented via copying. I don't think anything precludes drm write-back or rendering directly to the client-submitted buffers. Regards, Andri
Re: Full-motion zero-copy screen capture in Weston
On Friday, May 31st, 2024 at 15:26, Hoosier, Matt wrote: > Drew or Simon, does either of > https://github.com/swaywm/wlroots/blob/master/protocol/wlr-screencopy-unstable-v1.xml > or > https://github.com/swaywm/wlroots/blob/master/protocol/wlr-export-dmabuf-unstable-v1.xml > strike you as a good fit for the use-case of getting a streamed copy of an > output? They seem very similar – not sure what differentiates them from each > other. screencopy is designed around client-allocated buffers. For compositor-allocated buffers, export-dmabuf is what's used in the wlroots ecosystem. > My goal is to implement this screen capture with a guarantee that the copy > comes from a KMS writeback connector. I know this sounds like an odd thing to > insist on. Let's say that in my industry, the system is rarely only running > Linux directly on the bare metal. Using the writeback hardware on the display > controller allows to get a copy of content from all the virtual machines in > the system. > Frankly, weston_output_capture_v1's model that clients allocate the buffers > would make it very difficult to support efficient screen capture for more > than one simultaneous client. You can only target one writeback framebuffer > per page flip. Having the compositor manage the buffers' lifetimes and just > publishing out handles (in the style of those two wlr extensions) to those > probably fits better. IIRC Weston is not interested in supporting the wlroots protocol, they'd prefer to use e.g. PipeWire. Should be possible to pass the result of the writeback into a PipeWire stream. > From: Pekka Paalanen > Sent: Friday, May 31, 2024 3:02 AM > To: Hoosier, Matt > Cc: Marius Vlad; wayland-devel@lists.freedesktop.org > Subject: Re: Full-motion zero-copy screen capture in Weston > On Thu, 30 May 2024 13:40:15 + > "Hoosier, Matt" wrote: > > > Okay, interesting thoughts on all of that. > > > > I'm not sure how far I'm going to get toward a complete overhaul of > > the capture mechanism. Maybe it would be useful to know a couple > > things about the current one (weston_output_capture_v1), so that I > > don't commit mistakes that somebody already considered and guarded > > against: > > > > * Why was it explicitly picked only for still shots? I can image that > > it wouldn't have been a whole lot more work to design this API with a > > bounce-buffering scheme rather than a setup/do-it-once/destroy model. > > > > There was no need for streaming, as it was purely for the test suite. > > The test suite is very particular about when and how the shot is taken: > it must be produced by the defined source, and it must be produced on > the very next repaint of the output, exactly once. > > Did you mean a buffer pool (queueing/dequeueing) rather than > bounce-buffering? Sure. > > > * Why was wl_shm explicitly picked for the buffer type? I was > > thinking here that it would have worked equally well to specify that > > the client must supply a linear-layout buffer manufactured through > > zwp_linux_dmabuf. This would be eligible for direct targeting by the > > GL renderer/KMS writeback, but also can be mapped with gbm_bo_map() > > in the compositor's Pixman backend and/or a client who wants to map > > it to the CPU upon delivery. > > Because the test suite specifically needs CPU access to the screenshot. > There was no use yet for dmabuf, and GL-renderer was already doing > glReadPixels() for screenshots. > > IOW, all the limitations are just "was not needed yet". > > Note, that re-using or extending the protocol extension > weston_capture_v1 for streaming outputs for non-test-suite use cases > may not be the best idea. If the interface needs to be a Wayland > extension, then maybe something from the wlr extensions would suit > better. OTOH, for e.g. Pipewire there would not be a Wayland extension > used at all. > > The internal API (weston_output_capture) is very much geared for > weston_capture_v1 only. The internal API might take improving rather > than weston_capture_v1. > > > Thanks, > pq
Re: Full-motion zero-copy screen capture in Weston
Hi, Yeah. I agree that although I can prototype something quick and dirty here as a change to weston_output_capture_v1, it's probably not a good fit there. Drew or Simon, does either of https://github.com/swaywm/wlroots/blob/master/protocol/wlr-screencopy-unstable-v1.xml or https://github.com/swaywm/wlroots/blob/master/protocol/wlr-export-dmabuf-unstable-v1.xml strike you as a good fit for the use-case of getting a streamed copy of an output? They seem very similar – not sure what differentiates them from each other. My goal is to implement this screen capture with a guarantee that the copy comes from a KMS writeback connector. I know this sounds like an odd thing to insist on. Let's say that in my industry, the system is rarely only running Linux directly on the bare metal. Using the writeback hardware on the display controller allows to get a copy of content from all the virtual machines in the system. Frankly, weston_output_capture_v1's model that clients allocate the buffers would make it very difficult to support efficient screen capture for more than one simultaneous client. You can only target one writeback framebuffer per page flip. Having the compositor manage the buffers' lifetimes and just publishing out handles (in the style of those two wlr extensions) to those probably fits better. -Matt From: Pekka Paalanen Sent: Friday, May 31, 2024 3:02 AM To: Hoosier, Matt Cc: Marius Vlad; wayland-devel@lists.freedesktop.org Subject: Re: Full-motion zero-copy screen capture in Weston On Thu, 30 May 2024 13:40:15 + "Hoosier, Matt" wrote: > Okay, interesting thoughts on all of that. > > I'm not sure how far I'm going to get toward a complete overhaul of > the capture mechanism. Maybe it would be useful to know a couple > things about the current one (weston_output_capture_v1), so that I > don't commit mistakes that somebody already considered and guarded > against: > > * Why was it explicitly picked only for still shots? I can image that > it wouldn't have been a whole lot more work to design this API with a > bounce-buffering scheme rather than a setup/do-it-once/destroy model. > There was no need for streaming, as it was purely for the test suite. The test suite is very particular about when and how the shot is taken: it must be produced by the defined source, and it must be produced on the very next repaint of the output, exactly once. Did you mean a buffer pool (queueing/dequeueing) rather than bounce-buffering? Sure. > * Why was wl_shm explicitly picked for the buffer type? I was > thinking here that it would have worked equally well to specify that > the client must supply a linear-layout buffer manufactured through > zwp_linux_dmabuf. This would be eligible for direct targeting by the > GL renderer/KMS writeback, but also can be mapped with gbm_bo_map() > in the compositor's Pixman backend and/or a client who wants to map > it to the CPU upon delivery. Because the test suite specifically needs CPU access to the screenshot. There was no use yet for dmabuf, and GL-renderer was already doing glReadPixels() for screenshots. IOW, all the limitations are just "was not needed yet". Note, that re-using or extending the protocol extension weston_capture_v1 for streaming outputs for non-test-suite use cases may not be the best idea. If the interface needs to be a Wayland extension, then maybe something from the wlr extensions would suit better. OTOH, for e.g. Pipewire there would not be a Wayland extension used at all. The internal API (weston_output_capture) is very much geared for weston_capture_v1 only. The internal API might take improving rather than weston_capture_v1. Thanks, pq
Re: Full-motion zero-copy screen capture in Weston
On Thu, 30 May 2024 13:40:15 + "Hoosier, Matt" wrote: > Okay, interesting thoughts on all of that. > > I'm not sure how far I'm going to get toward a complete overhaul of > the capture mechanism. Maybe it would be useful to know a couple > things about the current one (weston_output_capture_v1), so that I > don't commit mistakes that somebody already considered and guarded > against: > > * Why was it explicitly picked only for still shots? I can image that > it wouldn't have been a whole lot more work to design this API with a > bounce-buffering scheme rather than a setup/do-it-once/destroy model. > There was no need for streaming, as it was purely for the test suite. The test suite is very particular about when and how the shot is taken: it must be produced by the defined source, and it must be produced on the very next repaint of the output, exactly once. Did you mean a buffer pool (queueing/dequeueing) rather than bounce-buffering? Sure. > * Why was wl_shm explicitly picked for the buffer type? I was > thinking here that it would have worked equally well to specify that > the client must supply a linear-layout buffer manufactured through > zwp_linux_dmabuf. This would be eligible for direct targeting by the > GL renderer/KMS writeback, but also can be mapped with gbm_bo_map() > in the compositor's Pixman backend and/or a client who wants to map > it to the CPU upon delivery. Because the test suite specifically needs CPU access to the screenshot. There was no use yet for dmabuf, and GL-renderer was already doing glReadPixels() for screenshots. IOW, all the limitations are just "was not needed yet". Note, that re-using or extending the protocol extension weston_capture_v1 for streaming outputs for non-test-suite use cases may not be the best idea. If the interface needs to be a Wayland extension, then maybe something from the wlr extensions would suit better. OTOH, for e.g. Pipewire there would not be a Wayland extension used at all. The internal API (weston_output_capture) is very much geared for weston_capture_v1 only. The internal API might take improving rather than weston_capture_v1. Thanks, pq pgpGdEI0WVF9N.pgp Description: OpenPGP digital signature
Re: Full-motion zero-copy screen capture in Weston
Okay, interesting thoughts on all of that. I'm not sure how far I'm going to get toward a complete overhaul of the capture mechanism. Maybe it would be useful to know a couple things about the current one (weston_output_capture_v1), so that I don't commit mistakes that somebody already considered and guarded against: * Why was it explicitly picked only for still shots? I can image that it wouldn't have been a whole lot more work to design this API with a bounce-buffering scheme rather than a setup/do-it-once/destroy model. * Why was wl_shm explicitly picked for the buffer type? I was thinking here that it would have worked equally well to specify that the client must supply a linear-layout buffer manufactured through zwp_linux_dmabuf. This would be eligible for direct targeting by the GL renderer/KMS writeback, but also can be mapped with gbm_bo_map() in the compositor's Pixman backend and/or a client who wants to map it to the CPU upon delivery. -Matt On 5/30/24, 3:22 AM, "Pekka Paalanen" mailto:pekka.paala...@collabora.com>> wrote: On Wed, 29 May 2024 15:18:16 + "Hoosier, Matt" mailto:matt.hoos...@garmin.com>> wrote: > Thanks, Pekka. > > So what would be an upstream-friendly way to put hooks deep down into > the DRM backend to exploit the KMS writeback connectors, but still > allow an XDG Desktop portal implementation to be usable on all the > backends? Hi Matt, I don't know how that would look like. "Hooks" feels like a side-channel to me, while I'd prefer a first class API. This probably requires adjusting some of the existing internal interfaces so that they can handle both one-shot and streaming captures (timestamps!), and even multiple capture consumers simultaneously. While the private screenshooting protocol currently explicitly defines how the screenshot must be taken, this improved API should be able to fall back as necessary and if desired: use KMS writeback when it works, fall back to renderer otherwise (which disables KMS planes composition). There is also the dmabuf vs. wl_shm buffer issue. All the external APIs should be implemented on top of such improved internal API: screenshots, screen streaming, anything that wants to get the contents of another backend's output and not create an independent output of its own with an independent update/rendering cadence. > Or maybe that's an unnecessary goal to begin with; a desktop session > somewhat implies the compositor is driving the physical hardware? I think the improved API needs to be usable on all backends, yes. Only KMS writeback needs backend support, the rest should be doable in the renderers which are shared by all backends. Non-DRM backends just don't have KMS writeback. Something like this would be ideal, I believe. Unfortunately I don't think I can personally spare time to help with this. This is more of a wish than a requirement. Thanks, pq > > > -Original Message- > > From: Pekka Paalanen > <mailto:pekka.paala...@collabora.com>> > > Sent: Wednesday, May 29, 2024 2:43 AM > > To: Marius Vlad > <mailto:marius.v...@collabora.com>>; Hoosier, Matt > > mailto:matt.hoos...@garmin.com>> > > Cc: wayland-devel@lists.freedesktop.org > > <mailto:wayland-devel@lists.freedesktop.org> > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > On Tue, 28 May 2024 20:04:45 +0300 > > Marius Vlad mailto:marius.v...@collabora.com>> > > wrote: > > > > > On Tue, May 28, 2024 at 03:53:23PM +, Hoosier, Matt wrote: > > > > Hi Marius, > > > Hi, > > > > > > > > Okay, I guess that answers the bit about needing to screen-scrape to > > > > get > > content into Pipewire now. > > > > > > > > But I'm still a little unclear about a couple things, if I were to try > > > > to build on > > this PW backend as a starting point: > > > > > > > > First, it looks to me like when you use the PW backend to Weston, > > > > that becomes your display. That is, rendering directly targets it. I > > > > was hoping for a way to get it to broadcast the very same > > > > framebuffer(s) that are getting scanned out for the current frame by > > > > the DRM backend. > > > With > > > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341 > > > <https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341> > > > the PipeWire output can mirror out the native DRM one. It aims at > > > having a way to configure VNC, RDP and PipeWire to screen-share the > > output. > > > > > > > &
Re: Full-motion zero-copy screen capture in Weston
On Wed, 29 May 2024 15:18:16 + "Hoosier, Matt" wrote: > Thanks, Pekka. > > So what would be an upstream-friendly way to put hooks deep down into > the DRM backend to exploit the KMS writeback connectors, but still > allow an XDG Desktop portal implementation to be usable on all the > backends? Hi Matt, I don't know how that would look like. "Hooks" feels like a side-channel to me, while I'd prefer a first class API. This probably requires adjusting some of the existing internal interfaces so that they can handle both one-shot and streaming captures (timestamps!), and even multiple capture consumers simultaneously. While the private screenshooting protocol currently explicitly defines how the screenshot must be taken, this improved API should be able to fall back as necessary and if desired: use KMS writeback when it works, fall back to renderer otherwise (which disables KMS planes composition). There is also the dmabuf vs. wl_shm buffer issue. All the external APIs should be implemented on top of such improved internal API: screenshots, screen streaming, anything that wants to get the contents of another backend's output and not create an independent output of its own with an independent update/rendering cadence. > Or maybe that's an unnecessary goal to begin with; a desktop session > somewhat implies the compositor is driving the physical hardware? I think the improved API needs to be usable on all backends, yes. Only KMS writeback needs backend support, the rest should be doable in the renderers which are shared by all backends. Non-DRM backends just don't have KMS writeback. Something like this would be ideal, I believe. Unfortunately I don't think I can personally spare time to help with this. This is more of a wish than a requirement. Thanks, pq > > > -Original Message- > > From: Pekka Paalanen > > Sent: Wednesday, May 29, 2024 2:43 AM > > To: Marius Vlad ; Hoosier, Matt > > > > Cc: wayland-devel@lists.freedesktop.org > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > On Tue, 28 May 2024 20:04:45 +0300 > > Marius Vlad wrote: > > > > > On Tue, May 28, 2024 at 03:53:23PM +, Hoosier, Matt wrote: > > > > Hi Marius, > > > Hi, > > > > > > > > Okay, I guess that answers the bit about needing to screen-scrape to > > > > get > > content into Pipewire now. > > > > > > > > But I'm still a little unclear about a couple things, if I were to try > > > > to build on > > this PW backend as a starting point: > > > > > > > > First, it looks to me like when you use the PW backend to Weston, > > > > that becomes your display. That is, rendering directly targets it. I > > > > was hoping for a way to get it to broadcast the very same > > > > framebuffer(s) that are getting scanned out for the current frame by > > > > the DRM backend. > > > With > > > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341 > > > the PipeWire output can mirror out the native DRM one. It aims at > > > having a way to configure VNC, RDP and PipeWire to screen-share the > > output. > > > > > > > > Second, I'm don't see the path to getting this to leverage the > > > > DRM_MODE_CONNECTOR_WRITEBACK hardware (like the > > > > weston_screen_recorder does). I think that any layering would be > > > > forced to be offloaded to the GPU ahead of time. Maybe I missed some > > > > implication of what you were pointing out here? > > > No sorry, I haven't really implied that, just pointed that out there's > > > some work for PipeWire gaining dmabuf. > > > > > > Screen-sharing an output is done > > > with multiple backends, and configuring Weston front-end is it with > > > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341. > > > > Hi Matt, Marius, > > > > multiple backends necessarily means that each backend renders its own > > weston_output independently. While it's fine for most screen-sharing use > > cases, I don't think it can ever be as efficient as KMS writeback could be. > > > > Getting a dmabuf-based video stream from DRM-backend using KMS > > writeback needs integration in the DRM-backend. I don't know how that > > would look like, but I suspect these ideas would be good: > > > > - builds on and extends the existing writeback support instead of > > something new on the side > > > > - does not use the DRM-backend "virtual output" API; this was a > > workaround for the lack of multi-backend support in the past and has > > the same disadvantages as multi-backend. (I don't think the DRM > > virtual output API has any reason to exist anymore, multi-backend > > should supersede it after converting the users.) > > > > I also think that implementing the usual desktop portals in Weston would be > > a > > good thing, and not something to avoid, if those can provide a nice external > > interface for the use case. > > > > > > Thanks, > > pq pgpE2PXYICwdX.pgp Description: OpenPGP digital signature
RE: Full-motion zero-copy screen capture in Weston
Thanks, Pekka. So what would be an upstream-friendly way to put hooks deep down into the DRM backend to exploit the KMS writeback connectors, but still allow an XDG Desktop portal implementation to be usable on all the backends? Or maybe that's an unnecessary goal to begin with; a desktop session somewhat implies the compositor is driving the physical hardware? > -Original Message- > From: Pekka Paalanen > Sent: Wednesday, May 29, 2024 2:43 AM > To: Marius Vlad ; Hoosier, Matt > > Cc: wayland-devel@lists.freedesktop.org > Subject: Re: Full-motion zero-copy screen capture in Weston > > On Tue, 28 May 2024 20:04:45 +0300 > Marius Vlad wrote: > > > On Tue, May 28, 2024 at 03:53:23PM +, Hoosier, Matt wrote: > > > Hi Marius, > > Hi, > > > > > > Okay, I guess that answers the bit about needing to screen-scrape to get > content into Pipewire now. > > > > > > But I'm still a little unclear about a couple things, if I were to try to > > > build on > this PW backend as a starting point: > > > > > > First, it looks to me like when you use the PW backend to Weston, > > > that becomes your display. That is, rendering directly targets it. I > > > was hoping for a way to get it to broadcast the very same > > > framebuffer(s) that are getting scanned out for the current frame by > > > the DRM backend. > > With > > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341 > > the PipeWire output can mirror out the native DRM one. It aims at > > having a way to configure VNC, RDP and PipeWire to screen-share the > output. > > > > > > Second, I'm don't see the path to getting this to leverage the > > > DRM_MODE_CONNECTOR_WRITEBACK hardware (like the > > > weston_screen_recorder does). I think that any layering would be > > > forced to be offloaded to the GPU ahead of time. Maybe I missed some > > > implication of what you were pointing out here? > > No sorry, I haven't really implied that, just pointed that out there's > > some work for PipeWire gaining dmabuf. > > > > Screen-sharing an output is done > > with multiple backends, and configuring Weston front-end is it with > > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341. > > Hi Matt, Marius, > > multiple backends necessarily means that each backend renders its own > weston_output independently. While it's fine for most screen-sharing use > cases, I don't think it can ever be as efficient as KMS writeback could be. > > Getting a dmabuf-based video stream from DRM-backend using KMS > writeback needs integration in the DRM-backend. I don't know how that > would look like, but I suspect these ideas would be good: > > - builds on and extends the existing writeback support instead of > something new on the side > > - does not use the DRM-backend "virtual output" API; this was a > workaround for the lack of multi-backend support in the past and has > the same disadvantages as multi-backend. (I don't think the DRM > virtual output API has any reason to exist anymore, multi-backend > should supersede it after converting the users.) > > I also think that implementing the usual desktop portals in Weston would be a > good thing, and not something to avoid, if those can provide a nice external > interface for the use case. > > > Thanks, > pq
Re: Full-motion zero-copy screen capture in Weston
On Tue, 28 May 2024 20:04:45 +0300 Marius Vlad wrote: > On Tue, May 28, 2024 at 03:53:23PM +, Hoosier, Matt wrote: > > Hi Marius, > Hi, > > > > Okay, I guess that answers the bit about needing to screen-scrape to get > > content into Pipewire now. > > > > But I'm still a little unclear about a couple things, if I were to try to > > build on this PW backend as a starting point: > > > > First, it looks to me like when you use the PW backend to Weston, that > > becomes your display. That is, rendering directly targets it. I was > > hoping for a way to get it to broadcast the very same framebuffer(s) > > that are getting scanned out for the current frame by the DRM > > backend. > With https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341 > the PipeWire output can mirror out the native DRM one. It aims at having > a way to configure VNC, RDP and PipeWire to screen-share the output. > > > > Second, I'm don't see the path to getting this to leverage the > > DRM_MODE_CONNECTOR_WRITEBACK hardware (like the weston_screen_recorder > > does). I think that any layering would be forced to > > be offloaded to the GPU ahead of time. Maybe I missed some implication > > of what you were pointing out here? > No sorry, I haven't really implied that, just pointed that out there's > some work for PipeWire gaining dmabuf. > > Screen-sharing an output is done > with multiple backends, and configuring Weston front-end is it with > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341. Hi Matt, Marius, multiple backends necessarily means that each backend renders its own weston_output independently. While it's fine for most screen-sharing use cases, I don't think it can ever be as efficient as KMS writeback could be. Getting a dmabuf-based video stream from DRM-backend using KMS writeback needs integration in the DRM-backend. I don't know how that would look like, but I suspect these ideas would be good: - builds on and extends the existing writeback support instead of something new on the side - does not use the DRM-backend "virtual output" API; this was a workaround for the lack of multi-backend support in the past and has the same disadvantages as multi-backend. (I don't think the DRM virtual output API has any reason to exist anymore, multi-backend should supersede it after converting the users.) I also think that implementing the usual desktop portals in Weston would be a good thing, and not something to avoid, if those can provide a nice external interface for the use case. Thanks, pq pgp2frA_CAvcb.pgp Description: OpenPGP digital signature
Re: Full-motion zero-copy screen capture in Weston
Hi again, The MR I'd like to link is actually at https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1476 On Tue, May 28, 2024 at 08:04:52PM +0300, Marius Vlad wrote: > On Tue, May 28, 2024 at 03:53:23PM +, Hoosier, Matt wrote: > > Hi Marius, > Hi, > > > > Okay, I guess that answers the bit about needing to screen-scrape to get > > content into Pipewire now. > > > > But I'm still a little unclear about a couple things, if I were to try to > > build on this PW backend as a starting point: > > > > First, it looks to me like when you use the PW backend to Weston, that > > becomes your display. That is, rendering directly targets it. I was > > hoping for a way to get it to broadcast the very same framebuffer(s) > > that are getting scanned out for the current frame by the DRM > > backend. > With https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341 > the PipeWire output can mirror out the native DRM one. It aims at having > a way to configure VNC, RDP and PipeWire to screen-share the output. > > > > Second, I'm don't see the path to getting this to leverage the > > DRM_MODE_CONNECTOR_WRITEBACK hardware (like the weston_screen_recorder > > does). I think that any layering would be forced to > > be offloaded to the GPU ahead of time. Maybe I missed some implication > > of what you were pointing out here? > No sorry, I haven't really implied that, just pointed that out there's > some work for PipeWire gaining dmabuf. > > Screen-sharing an output is done > with multiple backends, and configuring Weston front-end is it with > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341. > > > > > > -----Original Message- > > > From: Marius Vlad > > > Sent: Saturday, May 25, 2024 6:12 AM > > > To: Hoosier, Matt > > > Cc: wayland-devel@lists.freedesktop.org > > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > > > On Fri, May 24, 2024 at 09:43:58PM +, Hoosier, Matt wrote: > > > > Hi, > > > > > > > > I'm interested in finding or contributing some mechanism to get sort of > > > > the > > > same effect as a cross between: > > > > > > > > > > > > * > > > > weston_output_capture's support for using the > > > > DRM_MODE_CONNECTOR_WRITEBACK connectors, and > > > > > > > > * > > > > the streamed orientation of weston_screen_recorder, and > > > > > > > > * > > > > no forced reliance on the GPU to pre-blend the 2D scene – whatever > > > > plane blending would otherwise have occurred must still occur when the > > > > screen recording mechanism is active > > > > > > > > The desktop environments' compositors implement the XDG screencast > > > portal. If I read things correctly, that one deposits the stream of > > > dmabuf frame > > > fds into the Pipewire stream indicated by the user invoking the D-Bus > > > Screencast API. > > > > > > > > That doesn't really seem like a starter for doing this in Weston. > > > > There was conversation back in 2019 about trying to add zero-copy > > > > dmabuf support in Weston's own Pipewire integration, but I think that > > > > didn't happen? > > > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366 > > > > > > > > Alternately, I see that the remoting plugin on today's main branch > > > > supports > > > GStreamer dmabuf allocators. Does this mean that I could build something > > > using a virtual weston_output in the drm-backend? > > > > > > > > Cheers, > > > > Matt signature.asc Description: PGP signature
Re: Full-motion zero-copy screen capture in Weston
On Tue, May 28, 2024 at 03:53:23PM +, Hoosier, Matt wrote: > Hi Marius, Hi, > > Okay, I guess that answers the bit about needing to screen-scrape to get > content into Pipewire now. > > But I'm still a little unclear about a couple things, if I were to try to > build on this PW backend as a starting point: > > First, it looks to me like when you use the PW backend to Weston, that > becomes your display. That is, rendering directly targets it. I was > hoping for a way to get it to broadcast the very same framebuffer(s) > that are getting scanned out for the current frame by the DRM > backend. With https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341 the PipeWire output can mirror out the native DRM one. It aims at having a way to configure VNC, RDP and PipeWire to screen-share the output. > > Second, I'm don't see the path to getting this to leverage the > DRM_MODE_CONNECTOR_WRITEBACK hardware (like the weston_screen_recorder > does). I think that any layering would be forced to > be offloaded to the GPU ahead of time. Maybe I missed some implication > of what you were pointing out here? No sorry, I haven't really implied that, just pointed that out there's some work for PipeWire gaining dmabuf. Screen-sharing an output is done with multiple backends, and configuring Weston front-end is it with https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341. > > > -Original Message- > > From: Marius Vlad > > Sent: Saturday, May 25, 2024 6:12 AM > > To: Hoosier, Matt > > Cc: wayland-devel@lists.freedesktop.org > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > On Fri, May 24, 2024 at 09:43:58PM +, Hoosier, Matt wrote: > > > Hi, > > > > > > I'm interested in finding or contributing some mechanism to get sort of > > > the > > same effect as a cross between: > > > > > > > > > * > > > weston_output_capture's support for using the > > > DRM_MODE_CONNECTOR_WRITEBACK connectors, and > > > > > > * > > > the streamed orientation of weston_screen_recorder, and > > > > > > * > > > no forced reliance on the GPU to pre-blend the 2D scene – whatever > > > plane blending would otherwise have occurred must still occur when the > > > screen recording mechanism is active > > > > > > The desktop environments' compositors implement the XDG screencast > > portal. If I read things correctly, that one deposits the stream of dmabuf > > frame > > fds into the Pipewire stream indicated by the user invoking the D-Bus > > Screencast API. > > > > > > That doesn't really seem like a starter for doing this in Weston. > > > There was conversation back in 2019 about trying to add zero-copy > > > dmabuf support in Weston's own Pipewire integration, but I think that > > > didn't happen? > > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366 > > > > > > Alternately, I see that the remoting plugin on today's main branch > > > supports > > GStreamer dmabuf allocators. Does this mean that I could build something > > using a virtual weston_output in the drm-backend? > > > > > > Cheers, > > > Matt signature.asc Description: PGP signature
RE: Full-motion zero-copy screen capture in Weston
Hi Marius, Okay, I guess that answers the bit about needing to screen-scrape to get content into Pipewire now. But I'm still a little unclear about a couple things, if I were to try to build on this PW backend as a starting point: First, it looks to me like when you use the PW backend to Weston, that becomes your display. That is, rendering directly targets it. I was hoping for a way to get it to broadcast the very same framebuffer(s) that are getting scanned out for the current frame by the DRM backend. Second, I'm don't see the path to getting this to leverage the DRM_MODE_CONNECTOR_WRITEBACK hardware (like the weston_screen_recorder does). I think that any layering would be forced to be offloaded to the GPU ahead of time. Maybe I missed some implication of what you were pointing out here? > -Original Message- > From: Marius Vlad > Sent: Saturday, May 25, 2024 6:12 AM > To: Hoosier, Matt > Cc: wayland-devel@lists.freedesktop.org > Subject: Re: Full-motion zero-copy screen capture in Weston > > On Fri, May 24, 2024 at 09:43:58PM +, Hoosier, Matt wrote: > > Hi, > > > > I'm interested in finding or contributing some mechanism to get sort of the > same effect as a cross between: > > > > > > * > > weston_output_capture's support for using the > > DRM_MODE_CONNECTOR_WRITEBACK connectors, and > > > > * > > the streamed orientation of weston_screen_recorder, and > > > > * > > no forced reliance on the GPU to pre-blend the 2D scene – whatever > > plane blending would otherwise have occurred must still occur when the > > screen recording mechanism is active > > > > The desktop environments' compositors implement the XDG screencast > portal. If I read things correctly, that one deposits the stream of dmabuf > frame > fds into the Pipewire stream indicated by the user invoking the D-Bus > Screencast API. > > > > That doesn't really seem like a starter for doing this in Weston. > > There was conversation back in 2019 about trying to add zero-copy > > dmabuf support in Weston's own Pipewire integration, but I think that > > didn't happen? > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366 > > > > Alternately, I see that the remoting plugin on today's main branch supports > GStreamer dmabuf allocators. Does this mean that I could build something > using a virtual weston_output in the drm-backend? > > > > Cheers, > > Matt
Re: Full-motion zero-copy screen capture in Weston
On Fri, May 24, 2024 at 09:43:58PM +, Hoosier, Matt wrote: > Hi, > > I'm interested in finding or contributing some mechanism to get sort of the > same effect as a cross between: > > > * > weston_output_capture's support for using the DRM_MODE_CONNECTOR_WRITEBACK > connectors, and > > * > the streamed orientation of weston_screen_recorder, and > > * > no forced reliance on the GPU to pre-blend the 2D scene – whatever plane > blending would otherwise have occurred must still occur when the screen > recording mechanism is active > > The desktop environments' compositors implement the XDG screencast portal. If > I read things correctly, that one deposits the stream of dmabuf frame fds > into the Pipewire stream indicated by the user invoking the D-Bus Screencast > API. > > That doesn't really seem like a starter for doing this in Weston. > There was conversation back in 2019 about trying to add zero-copy > dmabuf support in Weston's own Pipewire integration, but I think that > didn't happen? https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366 > > Alternately, I see that the remoting plugin on today's main branch supports > GStreamer dmabuf allocators. Does this mean that I could build something > using a virtual weston_output in the drm-backend? > > Cheers, > Matt signature.asc Description: PGP signature
