RE: Full-motion zero-copy screen capture in Weston
> -Original Message- > From: Hoosier, Matt > Sent: Wednesday, June 12, 2024 8:37 AM > To: Pekka Paalanen > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > ; wayland-devel@lists.freedesktop.org; Daniel > Stone > Subject: RE: Full-motion zero-copy screen capture in Weston > > > -Original Message- > > From: Pekka Paalanen > > Sent: Wednesday, June 12, 2024 4:03 AM > > To: Hoosier, Matt > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > ; wayland-devel@lists.freedesktop.org; Daniel > > Stone > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > On Mon, 10 Jun 2024 14:18:39 + > > "Hoosier, Matt" wrote: > > > > > Yes, the native Linux driver for the hardware still does end up being > > > responsible for one or more planes. > > > > > > Other than trying to arrange the pieces so that there's some API that > > > offers the client an option to guarantee that the source of the screen > > > capture involves the writeback connector (similar to what you've done > > > with weston_output_capture), I don't really think it would be a good > > > idea to make Weston explicitly aware of any of this funny hypervisor > > > business going on. > > > > > > > I very much agree. > > Ack > > > > > > > The title on this email conversation was probably poorly chosen, in > > > retrospect. I'm not so much concerned with keeping absolutely to a > > > zero-copy mechanism as I am to using hardware mechanisms (GL > > > rendering, DRI writeback, hardware-accelerated blits as necessary) all > > > the way through. > > > > > > After seeing the way that the Pipewire backend handles streaming > > > (especially with > > > https://gitlab.freedesktop.org/wayland/weston/- > /merge_requests/1366), > > > I'm almost thinking that it would be preferable to maybe structure the > > > design of this feature something like this: > > > > > > * Add some sort of API on weston_output by which it can advertise the > > > ability to lay hands on the explicit renderbuffer (e.g. > > > gbm_surface_get_front_buffer()). This roughly equates to whether it's > > > a primary, non-nested backend. That is, DRM. > > > > > > * When processing the weston.ini mirror-of relationships at startup, > > > check whether the source output of the mirror-of declaration supports > > > that ability above. If so: > > > - Wire up a signal so that the source output announces each newly > > > rendered frame to any/all mirror-of outputs. > > > - The virtual output's backend allocates its own buffer pool in > > > exactly the same way that MR 1366 already does. > > > - Upon receipt of each signal announcing a new frame's renderbuffer > > > from the source output, the virtual output does a very cheap > > > glBlitFramebuffer() to get the contents into its own buffer pool. > > > This avoids the possibility of an unresponsive client tying up the > > > source output's buffer. > > > > > > * If the source output isn't one that supports exposing the underlying > > > renderbuffer, then the mirror-of relationship continues as with MR > > > 1476 just to invoke an explicit weston_renderer pass to draw the > > > correct logical contents. > > > > > > How does this strike you? > > > > Sorry, I don't understand how that idea is relevant to the KMS writeback > case. > > Did you imply that DRM-backend could deliver a KMS-writeback FB instead > of > > the rendered FB? > > Just for the same of argument, yes. But I take your point below that this > entire > idea to drive the mirroring output directly from the source output's rendering > doesn't fit the plan for the mirror-of semantics. > > > > > This is not the current plan for mirror-of. It does not allow the mirror > > output > to > > run on its own pace independently of the source output. Re-using the source > > output's rendered FB would also be a problem for color management. The FB > > is rendered for that output, and the color properties of the mirror may be > > different, needing an independent rendering anyway. > > > > The fundamental difference is who defines the properties of the stream. > > KMS writeback steam properties are necessarily defined by the KMS output. > > Mirror-of is for the opposite, for full flexibility. E.g. a remote mirror > > may not > > want to run at the full framerate, the physical monitor resolution, or with > the > > color capabilities of the source output in order to save bandwidth and > improve > > latency. > > Okay, understood. Although I'm a bit curious how you can say that one > output "mirrors" another whose resolution doesn't even match. Maybe you > have scaling in mind? > > > > > It seems to me that we will need two different mechanisms for the two > cases. I > > believe KMS writeback streaming is worthwhile to support, but not via > mirror- > > of key. Maybe the writeback streaming should be built into the DRM- > backend > > as a special pipewire source? Then it would also be guaranteed to be KMS > > writeback. It is some amount of code and testing
RE: Full-motion zero-copy screen capture in Weston
> -Original Message- > From: Pekka Paalanen > Sent: Wednesday, June 12, 2024 4:03 AM > To: Hoosier, Matt > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > ; wayland-devel@lists.freedesktop.org; Daniel > Stone > Subject: Re: Full-motion zero-copy screen capture in Weston > > On Mon, 10 Jun 2024 14:18:39 + > "Hoosier, Matt" wrote: > > > Yes, the native Linux driver for the hardware still does end up being > > responsible for one or more planes. > > > > Other than trying to arrange the pieces so that there's some API that > > offers the client an option to guarantee that the source of the screen > > capture involves the writeback connector (similar to what you've done > > with weston_output_capture), I don't really think it would be a good > > idea to make Weston explicitly aware of any of this funny hypervisor > > business going on. > > > > I very much agree. Ack > > > The title on this email conversation was probably poorly chosen, in > > retrospect. I'm not so much concerned with keeping absolutely to a > > zero-copy mechanism as I am to using hardware mechanisms (GL > > rendering, DRI writeback, hardware-accelerated blits as necessary) all > > the way through. > > > > After seeing the way that the Pipewire backend handles streaming > > (especially with > > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366), > > I'm almost thinking that it would be preferable to maybe structure the > > design of this feature something like this: > > > > * Add some sort of API on weston_output by which it can advertise the > > ability to lay hands on the explicit renderbuffer (e.g. > > gbm_surface_get_front_buffer()). This roughly equates to whether it's > > a primary, non-nested backend. That is, DRM. > > > > * When processing the weston.ini mirror-of relationships at startup, > > check whether the source output of the mirror-of declaration supports > > that ability above. If so: > > - Wire up a signal so that the source output announces each newly > > rendered frame to any/all mirror-of outputs. > > - The virtual output's backend allocates its own buffer pool in > > exactly the same way that MR 1366 already does. > > - Upon receipt of each signal announcing a new frame's renderbuffer > > from the source output, the virtual output does a very cheap > > glBlitFramebuffer() to get the contents into its own buffer pool. > > This avoids the possibility of an unresponsive client tying up the > > source output's buffer. > > > > * If the source output isn't one that supports exposing the underlying > > renderbuffer, then the mirror-of relationship continues as with MR > > 1476 just to invoke an explicit weston_renderer pass to draw the > > correct logical contents. > > > > How does this strike you? > > Sorry, I don't understand how that idea is relevant to the KMS writeback case. > Did you imply that DRM-backend could deliver a KMS-writeback FB instead of > the rendered FB? Just for the same of argument, yes. But I take your point below that this entire idea to drive the mirroring output directly from the source output's rendering doesn't fit the plan for the mirror-of semantics. > > This is not the current plan for mirror-of. It does not allow the mirror > output to > run on its own pace independently of the source output. Re-using the source > output's rendered FB would also be a problem for color management. The FB > is rendered for that output, and the color properties of the mirror may be > different, needing an independent rendering anyway. > > The fundamental difference is who defines the properties of the stream. > KMS writeback steam properties are necessarily defined by the KMS output. > Mirror-of is for the opposite, for full flexibility. E.g. a remote mirror may > not > want to run at the full framerate, the physical monitor resolution, or with > the > color capabilities of the source output in order to save bandwidth and improve > latency. Okay, understood. Although I'm a bit curious how you can say that one output "mirrors" another whose resolution doesn't even match. Maybe you have scaling in mind? > > It seems to me that we will need two different mechanisms for the two cases. I > believe KMS writeback streaming is worthwhile to support, but not via mirror- > of key. Maybe the writeback streaming should be built into the DRM-backend > as a special pipewire source? Then it would also be guaranteed to be KMS > writeback. It is some amount of code and testing duplication, but I think it > would be the simplest. Fair enough. I had the beginning of a complicated scheme of signals fallback rendering in mind. If you think it's justified to just add a special case directly for this, that's better to me. > > I don't see KMS writeback streaming as specific to your curious virtualization > case. I can imagine it being useful in general, to ensure that display > controller > output is correct for example, or when overall rendering efficiency is more >
Re: Full-motion zero-copy screen capture in Weston
On Mon, 10 Jun 2024 14:18:39 + "Hoosier, Matt" wrote: > Yes, the native Linux driver for the hardware still does end up being > responsible for one or more planes. > > Other than trying to arrange the pieces so that there's some API that > offers the client an option to guarantee that the source of the > screen capture involves the writeback connector (similar to what > you've done with weston_output_capture), I don't really think it > would be a good idea to make Weston explicitly aware of any of this > funny hypervisor business going on. > I very much agree. > The title on this email conversation was probably poorly chosen, in > retrospect. I'm not so much concerned with keeping absolutely to a > zero-copy mechanism as I am to using hardware mechanisms (GL > rendering, DRI writeback, hardware-accelerated blits as necessary) > all the way through. > > After seeing the way that the Pipewire backend handles streaming > (especially with > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366), > I'm almost thinking that it would be preferable to maybe structure > the design of this feature something like this: > > * Add some sort of API on weston_output by which it can advertise the > ability to lay hands on the explicit renderbuffer (e.g. > gbm_surface_get_front_buffer()). This roughly equates to whether it's > a primary, non-nested backend. That is, DRM. > > * When processing the weston.ini mirror-of relationships at startup, > check whether the source output of the mirror-of declaration supports > that ability above. If so: > - Wire up a signal so that the source output announces each newly > rendered frame to any/all mirror-of outputs. > - The virtual output's backend allocates its own buffer pool in > exactly the same way that MR 1366 already does. > - Upon receipt of each signal announcing a new frame's renderbuffer > from the source output, the virtual output does a very cheap > glBlitFramebuffer() to get the contents into its own buffer pool. > This avoids the possibility of an unresponsive client tying up the > source output's buffer. > > * If the source output isn't one that supports exposing the > underlying renderbuffer, then the mirror-of relationship continues as > with MR 1476 just to invoke an explicit weston_renderer pass to draw > the correct logical contents. > > How does this strike you? Sorry, I don't understand how that idea is relevant to the KMS writeback case. Did you imply that DRM-backend could deliver a KMS-writeback FB instead of the rendered FB? This is not the current plan for mirror-of. It does not allow the mirror output to run on its own pace independently of the source output. Re-using the source output's rendered FB would also be a problem for color management. The FB is rendered for that output, and the color properties of the mirror may be different, needing an independent rendering anyway. The fundamental difference is who defines the properties of the stream. KMS writeback steam properties are necessarily defined by the KMS output. Mirror-of is for the opposite, for full flexibility. E.g. a remote mirror may not want to run at the full framerate, the physical monitor resolution, or with the color capabilities of the source output in order to save bandwidth and improve latency. It seems to me that we will need two different mechanisms for the two cases. I believe KMS writeback streaming is worthwhile to support, but not via mirror-of key. Maybe the writeback streaming should be built into the DRM-backend as a special pipewire source? Then it would also be guaranteed to be KMS writeback. It is some amount of code and testing duplication, but I think it would be the simplest. I don't see KMS writeback streaming as specific to your curious virtualization case. I can imagine it being useful in general, to ensure that display controller output is correct for example, or when overall rendering efficiency is more important than optimal stream format. Thanks, pq pgpvurOu3w1aS.pgp Description: OpenPGP digital signature
