Re: [dev-servo] How to efficiently react to an element changing visibility status from DOM code?
Note that WR is now keeping the DOM up to date with the current scroll offset (or at least as close to "up to date" as you can be with APZ) so this could reasonably be done DOM-side, I imagine. Patrick On Jun 17, 2016 4:02 AM, "Josh Matthews" wrote: > While listening to the Gecko platform talk about shutting down video > decoders for offscreen elements, I wondered about how we could do the same > in Servo. We obviously don't support decoding video yet, but what would be > an efficient way of doing that in the future? > > Cheers, > Josh > ___ > dev-servo mailing list > dev-servo@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-servo > ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] How to efficiently react to an element changing visibility status from DOM code?
On Tue, Jun 21, 2016 at 9:16 AM, Jack Moffitt wrote: > > The problem is that you want notifications ahead of time that an element > is > > predicted to become visible, so you can decode images/video etc hopefully > > in time to render the element the moment it becomes visible. > > Does this mean you need scroll velocity and other such data to make > such a prediction? > Yes. > > It's implemented as a seek operation, so you decode a keyframe and then > as > > many frames as necessary to reach the nominal current position. > > Does this mean it doesn't work for live video sources or MSE? > I don't know. Dan Glastonbury or Anthony Jones are on the video side, Timothy Nikkel and Seth Fowler are on the layout/image rendering side. Rob -- lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf toD selthor stor edna siewaoeodm or v sstvr esBa kbvted,t rdsme,aoreseoouoto o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea lurpr .a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr esn ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] How to efficiently react to an element changing visibility status from DOM code?
> The problem is that you want notifications ahead of time that an element is > predicted to become visible, so you can decode images/video etc hopefully > in time to render the element the moment it becomes visible. Does this mean you need scroll velocity and other such data to make such a prediction? > It's implemented as a seek operation, so you decode a keyframe and then as > many frames as necessary to reach the nominal current position. Does this mean it doesn't work for live video sources or MSE? jack. ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] How to efficiently react to an element changing visibility status from DOM code?
On Tue, Jun 21, 2016 at 4:41 AM, Jack Moffitt wrote: > How is this done in Gecko? > Display list analysis in the main thread. We learn something is not visible during one of the various cullings > of WebRender. So I guess the idea would be to have some notification > you send when something goes from visible previously to invisible. I > suppose there is an analog when doing flow construction that you might > find a flow that was previously visible is now display: none. > The problem is that you want notifications ahead of time that an element is predicted to become visible, so you can decode images/video etc hopefully in time to render the element the moment it becomes visible. One issue that wasn't covered in that lightning talk is that what > happens when the video comes back into view? Decoding can only be > started at a keyframe, so how do they resume playback? > It's implemented as a seek operation, so you decode a keyframe and then as many frames as necessary to reach the nominal current position. Rob -- lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf toD selthor stor edna siewaoeodm or v sstvr esBa kbvted,t rdsme,aoreseoouoto o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea lurpr .a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr esn ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] How to efficiently react to an element changing visibility status from DOM code?
On 17/06/16 13:01, Josh Matthews wrote: While listening to the Gecko platform talk about shutting down video decoders for offscreen elements, I wondered about how we could do the same in Servo. We obviously don't support decoding video yet, but what would be an efficient way of doing that in the future? I think there are two parts to this: * Whether a Servo tab is visible at all (the window could be showing another tab, or be minimized, in another workspace, behind another opaque window, ...) * Whether a given element intersects the visual viewport. I believe WebRender tracks the latter to skip painting such elements. Maybe it could add a way to query that information? Or maybe display lists already carry that information, without going to webrender? -- Simon Sapin ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] How to efficiently react to an element changing visibility status from DOM code?
How is this done in Gecko? We learn something is not visible during one of the various cullings of WebRender. So I guess the idea would be to have some notification you send when something goes from visible previously to invisible. I suppose there is an analog when doing flow construction that you might find a flow that was previously visible is now display: none. So it almost seems like we want something in the DOM elements which allow them to be notified of visibility changes. For videos, this could stop video decode, and for images, this may let their cache memory be reclaimed, etc. One issue that wasn't covered in that lightning talk is that what happens when the video comes back into view? Decoding can only be started at a keyframe, so how do they resume playback? Does the demuxer tell them which frames are prediction sources so they can save those and then resume decoding by decoding those plus the current frame? VLC for example does not have a check for this which results in very odd results where only the part of the image that is moving is visible until a new keyframe appears and the whole scene pops into view. jack. On Fri, Jun 17, 2016 at 5:01 AM, Josh Matthews wrote: > While listening to the Gecko platform talk about shutting down video > decoders for offscreen elements, I wondered about how we could do the same > in Servo. We obviously don't support decoding video yet, but what would be > an efficient way of doing that in the future? > > Cheers, > Josh > ___ > dev-servo mailing list > dev-servo@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-servo ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
