Re: [whatwg] Media elements statistics
On Fri, May 6, 2011 at 3:04 AM, Steve Lacey s...@chromium.org wrote: On Thu, May 5, 2011 at 2:54 AM, Jeroen Wijering jer...@longtailvideo.com wrote: Hey Steve, This looks great; would be a really useful set of data for video players / publishers. Since none of the metrics have a time component, developers can sample the data over the window / at the frequency they prefer. Would jitter be calculated over decodedFrames or decodedFrames+droppedFrames? In the first case, jitter is a more granular metric for measuring exact presentation performance. In the latter case, jitter can serve as a single metric for tracking processing power (simple!). In either case, it's fairly straightforward to calculate towards the other metric. Actually, I would expect presentedFrames + droppedFrames would be used. This implies all decoded frames that were supposed to be presented. Do we also expose to the JS developer how many frames we actually requested, so we can calculate rates? Silvia.
Re: [whatwg] Media elements statistics
On 3/05/2011 10:15 a.m., Steve Lacey wrote: I've updated the wiki with a proposal... http://wiki.whatwg.org/wiki/Video_Metrics#Proposal This proposal looks good to us at Mozilla. Thanks for putting this together! Chris Pearce.
Re: [whatwg] Media elements statistics
Hey Steve, This looks great; would be a really useful set of data for video players / publishers. Since none of the metrics have a time component, developers can sample the data over the window / at the frequency they prefer. Would jitter be calculated over decodedFrames or decodedFrames+droppedFrames? In the first case, jitter is a more granular metric for measuring exact presentation performance. In the latter case, jitter can serve as a single metric for tracking processing power (simple!). In either case, it's fairly straightforward to calculate towards the other metric. Resetting the values when @src changes is a good idea. Changing @src is much used for advertising and playlist support (it works around iOS not supporting the play() call). Kind regards, Jeroen Wijering On May 3, 2011, at 12:15 AM, Steve Lacey wrote: All, I've updated the wiki with a proposal... http://wiki.whatwg.org/wiki/Video_Metrics#Proposal Cheers! Steve On Sat, Apr 9, 2011 at 7:08 AM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: Ah, thanks for the link. I've included Silverlight stats, too, for completeness. If somebody knows about QuickTime stats, that would be another good one to add, I guess. Cheers, Silvia. On Fri, Apr 8, 2011 at 5:21 PM, Jeroen Wijering jer...@longtailvideo.com wrote: On Apr 7, 2011, at 8:11 AM, Silvia Pfeiffer wrote: I've also just added a section with the stats that the Adobe Flash player exposes. Great. Perhaps Silverlight stats might be of use too - though they're fairly similar: http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ Apart from the statistics that are not currently available from the HTML5 player, there are stats that are already available, such as currentSrc, currentTime, and all the events which can be turned into hooks for measurement. Yes, the network and ready states are very useful to determine if clients are stalling for buffering etc. I think the page now has a lot of analysis of currently used stats - probably a sufficient amount. All the video publishing sites likely just use a subpart of the ones that Adobe Flash exposes in their analytics. Especially all the separate A/V bytecounts are overkill IMO. One useful metric I didn't list for JW Player but is very nice is Flash's isLive property. Kind regards, Jeroen On Thu, Apr 7, 2011 at 4:52 AM, Mark Watson wats...@netflix.com wrote: All, I added some material to the wiki page based on our experience here at Netflix and based on the metrics defined in MPEG DASH for adaptive streaming. I'd love to here what people think. Statistics about presentation/rendering seem to be covered, but what should also be considered are network performance statistics, which become increasingly difficult to collect from the server when sessions are making use of multiple servers, possibly across multiple CDNs. Another aspect important for performance management is error reporting. Some thoughts on that on the page. ...Mark On Mar 31, 2011, at 7:07 PM, Robert O'Callahan wrote: On Fri, Apr 1, 2011 at 1:33 PM, Chris Pearce ch...@pearce.org.nz wrote: On 1/04/2011 12:22 p.m., Steve Lacey wrote: Chris - in the mozilla stats, I agree on the need for a frame count of frames that actually make it the the screen, but am interested in why we need both presented and painted? Wouldn't just a simple 'presented' (i.e. presented to the user) suffice? We distinguish between painted and presented so we have a measure of the latency in our rendering pipeline. It's more for our benefit as browser developers than for web developers. Yeah, just to be clear, we don't necessarily think that everything in our stats API should be standardized. We should wait and see what authors actually use. Rob -- Now the Bereans were of more noble character than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if what Paul said was true. [Acts 17:11]
Re: [whatwg] Media elements statistics
On Thu, May 5, 2011 at 2:54 AM, Jeroen Wijering jer...@longtailvideo.comwrote: Hey Steve, This looks great; would be a really useful set of data for video players / publishers. Since none of the metrics have a time component, developers can sample the data over the window / at the frequency they prefer. Would jitter be calculated over decodedFrames or decodedFrames+droppedFrames? In the first case, jitter is a more granular metric for measuring exact presentation performance. In the latter case, jitter can serve as a single metric for tracking processing power (simple!). In either case, it's fairly straightforward to calculate towards the other metric. Actually, I would expect presentedFrames + droppedFrames would be used. This implies all decoded frames that were supposed to be presented. Resetting the values when @src changes is a good idea. Changing @src is much used for advertising and playlist support (it works around iOS not supporting the play() call). Kind regards, Jeroen Wijering On May 3, 2011, at 12:15 AM, Steve Lacey wrote: All, I've updated the wiki with a proposal... http://wiki.whatwg.org/wiki/Video_Metrics#Proposal Cheers! Steve On Sat, Apr 9, 2011 at 7:08 AM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: Ah, thanks for the link. I've included Silverlight stats, too, for completeness. If somebody knows about QuickTime stats, that would be another good one to add, I guess. Cheers, Silvia. On Fri, Apr 8, 2011 at 5:21 PM, Jeroen Wijering jer...@longtailvideo.com wrote: On Apr 7, 2011, at 8:11 AM, Silvia Pfeiffer wrote: I've also just added a section with the stats that the Adobe Flash player exposes. Great. Perhaps Silverlight stats might be of use too - though they're fairly similar: http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ Apart from the statistics that are not currently available from the HTML5 player, there are stats that are already available, such as currentSrc, currentTime, and all the events which can be turned into hooks for measurement. Yes, the network and ready states are very useful to determine if clients are stalling for buffering etc. I think the page now has a lot of analysis of currently used stats - probably a sufficient amount. All the video publishing sites likely just use a subpart of the ones that Adobe Flash exposes in their analytics. Especially all the separate A/V bytecounts are overkill IMO. One useful metric I didn't list for JW Player but is very nice is Flash's isLive property. Kind regards, Jeroen On Thu, Apr 7, 2011 at 4:52 AM, Mark Watson wats...@netflix.com wrote: All, I added some material to the wiki page based on our experience here at Netflix and based on the metrics defined in MPEG DASH for adaptive streaming. I'd love to here what people think. Statistics about presentation/rendering seem to be covered, but what should also be considered are network performance statistics, which become increasingly difficult to collect from the server when sessions are making use of multiple servers, possibly across multiple CDNs. Another aspect important for performance management is error reporting. Some thoughts on that on the page. ...Mark On Mar 31, 2011, at 7:07 PM, Robert O'Callahan wrote: On Fri, Apr 1, 2011 at 1:33 PM, Chris Pearce ch...@pearce.org.nz wrote: On 1/04/2011 12:22 p.m., Steve Lacey wrote: Chris - in the mozilla stats, I agree on the need for a frame count of frames that actually make it the the screen, but am interested in why we need both presented and painted? Wouldn't just a simple 'presented' (i.e. presented to the user) suffice? We distinguish between painted and presented so we have a measure of the latency in our rendering pipeline. It's more for our benefit as browser developers than for web developers. Yeah, just to be clear, we don't necessarily think that everything in our stats API should be standardized. We should wait and see what authors actually use. Rob -- Now the Bereans were of more noble character than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if what Paul said was true. [Acts 17:11]
Re: [whatwg] Media elements statistics
On May 5, 2011, at 7:04 PM, Steve Lacey wrote: Would jitter be calculated over decodedFrames or decodedFrames+droppedFrames? In the first case, jitter is a more granular metric for measuring exact presentation performance. In the latter case, jitter can serve as a single metric for tracking processing power (simple!). In either case, it's fairly straightforward to calculate towards the other metric. Actually, I would expect presentedFrames + droppedFrames would be used. This implies all decoded frames that were supposed to be presented. Ah yes, sorry. That makes sense. presentedFrames of course need to be included. - Jeroen
Re: [whatwg] Media elements statistics
All, I've updated the wiki with a proposal... http://wiki.whatwg.org/wiki/Video_Metrics#Proposal Cheers! Steve On Sat, Apr 9, 2011 at 7:08 AM, Silvia Pfeiffer silviapfeiff...@gmail.comwrote: Ah, thanks for the link. I've included Silverlight stats, too, for completeness. If somebody knows about QuickTime stats, that would be another good one to add, I guess. Cheers, Silvia. On Fri, Apr 8, 2011 at 5:21 PM, Jeroen Wijering jer...@longtailvideo.com wrote: On Apr 7, 2011, at 8:11 AM, Silvia Pfeiffer wrote: I've also just added a section with the stats that the Adobe Flash player exposes. Great. Perhaps Silverlight stats might be of use too - though they're fairly similar: http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ Apart from the statistics that are not currently available from the HTML5 player, there are stats that are already available, such as currentSrc, currentTime, and all the events which can be turned into hooks for measurement. Yes, the network and ready states are very useful to determine if clients are stalling for buffering etc. I think the page now has a lot of analysis of currently used stats - probably a sufficient amount. All the video publishing sites likely just use a subpart of the ones that Adobe Flash exposes in their analytics. Especially all the separate A/V bytecounts are overkill IMO. One useful metric I didn't list for JW Player but is very nice is Flash's isLive property. Kind regards, Jeroen On Thu, Apr 7, 2011 at 4:52 AM, Mark Watson wats...@netflix.com wrote: All, I added some material to the wiki page based on our experience here at Netflix and based on the metrics defined in MPEG DASH for adaptive streaming. I'd love to here what people think. Statistics about presentation/rendering seem to be covered, but what should also be considered are network performance statistics, which become increasingly difficult to collect from the server when sessions are making use of multiple servers, possibly across multiple CDNs. Another aspect important for performance management is error reporting. Some thoughts on that on the page. ...Mark On Mar 31, 2011, at 7:07 PM, Robert O'Callahan wrote: On Fri, Apr 1, 2011 at 1:33 PM, Chris Pearce ch...@pearce.org.nz wrote: On 1/04/2011 12:22 p.m., Steve Lacey wrote: Chris - in the mozilla stats, I agree on the need for a frame count of frames that actually make it the the screen, but am interested in why we need both presented and painted? Wouldn't just a simple 'presented' (i.e. presented to the user) suffice? We distinguish between painted and presented so we have a measure of the latency in our rendering pipeline. It's more for our benefit as browser developers than for web developers. Yeah, just to be clear, we don't necessarily think that everything in our stats API should be standardized. We should wait and see what authors actually use. Rob -- Now the Bereans were of more noble character than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if what Paul said was true. [Acts 17:11]
Re: [whatwg] Media elements statistics
Ah, thanks for the link. I've included Silverlight stats, too, for completeness. If somebody knows about QuickTime stats, that would be another good one to add, I guess. Cheers, Silvia. On Fri, Apr 8, 2011 at 5:21 PM, Jeroen Wijering jer...@longtailvideo.com wrote: On Apr 7, 2011, at 8:11 AM, Silvia Pfeiffer wrote: I've also just added a section with the stats that the Adobe Flash player exposes. Great. Perhaps Silverlight stats might be of use too - though they're fairly similar: http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ Apart from the statistics that are not currently available from the HTML5 player, there are stats that are already available, such as currentSrc, currentTime, and all the events which can be turned into hooks for measurement. Yes, the network and ready states are very useful to determine if clients are stalling for buffering etc. I think the page now has a lot of analysis of currently used stats - probably a sufficient amount. All the video publishing sites likely just use a subpart of the ones that Adobe Flash exposes in their analytics. Especially all the separate A/V bytecounts are overkill IMO. One useful metric I didn't list for JW Player but is very nice is Flash's isLive property. Kind regards, Jeroen On Thu, Apr 7, 2011 at 4:52 AM, Mark Watson wats...@netflix.com wrote: All, I added some material to the wiki page based on our experience here at Netflix and based on the metrics defined in MPEG DASH for adaptive streaming. I'd love to here what people think. Statistics about presentation/rendering seem to be covered, but what should also be considered are network performance statistics, which become increasingly difficult to collect from the server when sessions are making use of multiple servers, possibly across multiple CDNs. Another aspect important for performance management is error reporting. Some thoughts on that on the page. ...Mark On Mar 31, 2011, at 7:07 PM, Robert O'Callahan wrote: On Fri, Apr 1, 2011 at 1:33 PM, Chris Pearce ch...@pearce.org.nz wrote: On 1/04/2011 12:22 p.m., Steve Lacey wrote: Chris - in the mozilla stats, I agree on the need for a frame count of frames that actually make it the the screen, but am interested in why we need both presented and painted? Wouldn't just a simple 'presented' (i.e. presented to the user) suffice? We distinguish between painted and presented so we have a measure of the latency in our rendering pipeline. It's more for our benefit as browser developers than for web developers. Yeah, just to be clear, we don't necessarily think that everything in our stats API should be standardized. We should wait and see what authors actually use. Rob -- Now the Bereans were of more noble character than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if what Paul said was true. [Acts 17:11]
Re: [whatwg] Media elements statistics
On Apr 7, 2011, at 8:11 AM, Silvia Pfeiffer wrote: I've also just added a section with the stats that the Adobe Flash player exposes. Great. Perhaps Silverlight stats might be of use too - though they're fairly similar: http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ Apart from the statistics that are not currently available from the HTML5 player, there are stats that are already available, such as currentSrc, currentTime, and all the events which can be turned into hooks for measurement. Yes, the network and ready states are very useful to determine if clients are stalling for buffering etc. I think the page now has a lot of analysis of currently used stats - probably a sufficient amount. All the video publishing sites likely just use a subpart of the ones that Adobe Flash exposes in their analytics. Especially all the separate A/V bytecounts are overkill IMO. One useful metric I didn't list for JW Player but is very nice is Flash's isLive property. Kind regards, Jeroen On Thu, Apr 7, 2011 at 4:52 AM, Mark Watson wats...@netflix.com wrote: All, I added some material to the wiki page based on our experience here at Netflix and based on the metrics defined in MPEG DASH for adaptive streaming. I'd love to here what people think. Statistics about presentation/rendering seem to be covered, but what should also be considered are network performance statistics, which become increasingly difficult to collect from the server when sessions are making use of multiple servers, possibly across multiple CDNs. Another aspect important for performance management is error reporting. Some thoughts on that on the page. ...Mark On Mar 31, 2011, at 7:07 PM, Robert O'Callahan wrote: On Fri, Apr 1, 2011 at 1:33 PM, Chris Pearce ch...@pearce.org.nz wrote: On 1/04/2011 12:22 p.m., Steve Lacey wrote: Chris - in the mozilla stats, I agree on the need for a frame count of frames that actually make it the the screen, but am interested in why we need both presented and painted? Wouldn't just a simple 'presented' (i.e. presented to the user) suffice? We distinguish between painted and presented so we have a measure of the latency in our rendering pipeline. It's more for our benefit as browser developers than for web developers. Yeah, just to be clear, we don't necessarily think that everything in our stats API should be standardized. We should wait and see what authors actually use. Rob -- Now the Bereans were of more noble character than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if what Paul said was true. [Acts 17:11]
Re: [whatwg] Media elements statistics
I've also just added a section with the stats that the Adobe Flash player exposes. Apart from the statistics that are not currently available from the HTML5 player, there are stats that are already available, such as currentSrc, currentTime, and all the events which can be turned into hooks for measurement. I think the page now has a lot of analysis of currently used stats - probably a sufficient amount. All the video publishing sites likely just use a subpart of the ones that Adobe Flash exposes in their analytics. Cheers, Silvia. On Thu, Apr 7, 2011 at 4:52 AM, Mark Watson wats...@netflix.com wrote: All, I added some material to the wiki page based on our experience here at Netflix and based on the metrics defined in MPEG DASH for adaptive streaming. I'd love to here what people think. Statistics about presentation/rendering seem to be covered, but what should also be considered are network performance statistics, which become increasingly difficult to collect from the server when sessions are making use of multiple servers, possibly across multiple CDNs. Another aspect important for performance management is error reporting. Some thoughts on that on the page. ...Mark On Mar 31, 2011, at 7:07 PM, Robert O'Callahan wrote: On Fri, Apr 1, 2011 at 1:33 PM, Chris Pearce ch...@pearce.org.nz wrote: On 1/04/2011 12:22 p.m., Steve Lacey wrote: Chris - in the mozilla stats, I agree on the need for a frame count of frames that actually make it the the screen, but am interested in why we need both presented and painted? Wouldn't just a simple 'presented' (i.e. presented to the user) suffice? We distinguish between painted and presented so we have a measure of the latency in our rendering pipeline. It's more for our benefit as browser developers than for web developers. Yeah, just to be clear, we don't necessarily think that everything in our stats API should be standardized. We should wait and see what authors actually use. Rob -- Now the Bereans were of more noble character than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if what Paul said was true. [Acts 17:11]
Re: [whatwg] Media elements statistics
All, I added some material to the wiki page based on our experience here at Netflix and based on the metrics defined in MPEG DASH for adaptive streaming. I'd love to here what people think. Statistics about presentation/rendering seem to be covered, but what should also be considered are network performance statistics, which become increasingly difficult to collect from the server when sessions are making use of multiple servers, possibly across multiple CDNs. Another aspect important for performance management is error reporting. Some thoughts on that on the page. ...Mark On Mar 31, 2011, at 7:07 PM, Robert O'Callahan wrote: On Fri, Apr 1, 2011 at 1:33 PM, Chris Pearce ch...@pearce.org.nz wrote: On 1/04/2011 12:22 p.m., Steve Lacey wrote: Chris - in the mozilla stats, I agree on the need for a frame count of frames that actually make it the the screen, but am interested in why we need both presented and painted? Wouldn't just a simple 'presented' (i.e. presented to the user) suffice? We distinguish between painted and presented so we have a measure of the latency in our rendering pipeline. It's more for our benefit as browser developers than for web developers. Yeah, just to be clear, we don't necessarily think that everything in our stats API should be standardized. We should wait and see what authors actually use. Rob -- Now the Bereans were of more noble character than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if what Paul said was true. [Acts 17:11]
Re: [whatwg] Media elements statistics
On Wed, Mar 30, 2011 at 9:24 PM, Silvia Pfeiffer silviapfeiff...@gmail.comwrote: Please note that I've started a wiki page at http://wiki.whatwg.org/wiki/Video_Metrics to try and collect all ideas around media element statistics. Please add freely! Thanks! That was on my todo list for this week. Note that the webkit proposal has landed and is available in Chrome 11. Silvia. On Thu, Mar 31, 2011 at 2:16 PM, Chris Pearce ch...@pearce.org.nz wrote: Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Reviving thread... I have an initial patch in webkit (http://trac.webkit.org/changeset/77394) and the chromium work is underway - I wonder what might be a good approach to drive the apis closer together towards a real spec that everyone is happy with? There seems to be a lot of general agreement here (at least in principal :-) that this is needed. We'll be doing a bunch of experimentation once this has landed in chromium. I've landed support for some Mozilla-specific video decoding/painting statistics in Firefox trunk (tracking in Mozilla bug 580531 mentioned above), it should ship in Firefox 5 in a few months. For a few details, see: http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html Regards, Chris Pearce.
Re: [whatwg] Media elements statistics
Ha - I should have waited! ;-) But seriously: if you have any other sources of information for what stats may be useful to people, please add them. And: congrats on the stats released in Chromium Mozilla! Cheers, Silvia. On Fri, Apr 1, 2011 at 5:11 AM, Steve Lacey s...@chromium.org wrote: On Wed, Mar 30, 2011 at 9:24 PM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: Please note that I've started a wiki page at http://wiki.whatwg.org/wiki/Video_Metrics to try and collect all ideas around media element statistics. Please add freely! Thanks! That was on my todo list for this week. Note that the webkit proposal has landed and is available in Chrome 11. Silvia. On Thu, Mar 31, 2011 at 2:16 PM, Chris Pearce ch...@pearce.org.nz wrote: Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Reviving thread... I have an initial patch in webkit (http://trac.webkit.org/changeset/77394) and the chromium work is underway - I wonder what might be a good approach to drive the apis closer together towards a real spec that everyone is happy with? There seems to be a lot of general agreement here (at least in principal :-) that this is needed. We'll be doing a bunch of experimentation once this has landed in chromium. I've landed support for some Mozilla-specific video decoding/painting statistics in Firefox trunk (tracking in Mozilla bug 580531 mentioned above), it should ship in Firefox 5 in a few months. For a few details, see: http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html Regards, Chris Pearce.
Re: [whatwg] Media elements statistics
On Thu, Mar 31, 2011 at 3:36 PM, Silvia Pfeiffer silviapfeiff...@gmail.comwrote: Ha - I should have waited! ;-) But seriously: if you have any other sources of information for what stats may be useful to people, please add them. I'm pretty sure that a combination of the mozilla/webkit stats cover all bases. Chris - in the mozilla stats, I agree on the need for a frame count of frames that actually make it the the screen, but am interested in why we need both presented and painted? Wouldn't just a simple 'presented' (i.e. presented to the user) suffice? Cheers! Steve And: congrats on the stats released in Chromium Mozilla! Cheers, Silvia. On Fri, Apr 1, 2011 at 5:11 AM, Steve Lacey s...@chromium.org wrote: On Wed, Mar 30, 2011 at 9:24 PM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: Please note that I've started a wiki page at http://wiki.whatwg.org/wiki/Video_Metrics to try and collect all ideas around media element statistics. Please add freely! Thanks! That was on my todo list for this week. Note that the webkit proposal has landed and is available in Chrome 11. Silvia. On Thu, Mar 31, 2011 at 2:16 PM, Chris Pearce ch...@pearce.org.nz wrote: Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Reviving thread... I have an initial patch in webkit (http://trac.webkit.org/changeset/77394) and the chromium work is underway - I wonder what might be a good approach to drive the apis closer together towards a real spec that everyone is happy with? There seems to be a lot of general agreement here (at least in principal :-) that this is needed. We'll be doing a bunch of experimentation once this has landed in chromium. I've landed support for some Mozilla-specific video decoding/painting statistics in Firefox trunk (tracking in Mozilla bug 580531 mentioned above), it should ship in Firefox 5 in a few months. For a few details, see: http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html Regards, Chris Pearce.
Re: [whatwg] Media elements statistics
On 1/04/2011 12:22 p.m., Steve Lacey wrote: Chris - in the mozilla stats, I agree on the need for a frame count of frames that actually make it the the screen, but am interested in why we need both presented and painted? Wouldn't just a simple 'presented' (i.e. presented to the user) suffice? We distinguish between painted and presented so we have a measure of the latency in our rendering pipeline. It's more for our benefit as browser developers than for web developers. Chris
Re: [whatwg] Media elements statistics
On Fri, Apr 1, 2011 at 1:33 PM, Chris Pearce ch...@pearce.org.nz wrote: On 1/04/2011 12:22 p.m., Steve Lacey wrote: Chris - in the mozilla stats, I agree on the need for a frame count of frames that actually make it the the screen, but am interested in why we need both presented and painted? Wouldn't just a simple 'presented' (i.e. presented to the user) suffice? We distinguish between painted and presented so we have a measure of the latency in our rendering pipeline. It's more for our benefit as browser developers than for web developers. Yeah, just to be clear, we don't necessarily think that everything in our stats API should be standardized. We should wait and see what authors actually use. Rob -- Now the Bereans were of more noble character than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if what Paul said was true. [Acts 17:11]
Re: [whatwg] Media elements statistics
Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Reviving thread... I have an initial patch in webkit (http://trac.webkit.org/changeset/77394) and the chromium work is underway - I wonder what might be a good approach to drive the apis closer together towards a real spec that everyone is happy with? There seems to be a lot of general agreement here (at least in principal :-) that this is needed. We'll be doing a bunch of experimentation once this has landed in chromium. I've landed support for some Mozilla-specific video decoding/painting statistics in Firefox trunk (tracking in Mozilla bug 580531 mentioned above), it should ship in Firefox 5 in a few months. For a few details, see: http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html Regards, Chris Pearce.
Re: [whatwg] Media elements statistics
Please note that I've started a wiki page at http://wiki.whatwg.org/wiki/Video_Metrics to try and collect all ideas around media element statistics. Please add freely! Silvia. On Thu, Mar 31, 2011 at 2:16 PM, Chris Pearce ch...@pearce.org.nz wrote: Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Reviving thread... I have an initial patch in webkit (http://trac.webkit.org/changeset/77394) and the chromium work is underway - I wonder what might be a good approach to drive the apis closer together towards a real spec that everyone is happy with? There seems to be a lot of general agreement here (at least in principal :-) that this is needed. We'll be doing a bunch of experimentation once this has landed in chromium. I've landed support for some Mozilla-specific video decoding/painting statistics in Firefox trunk (tracking in Mozilla bug 580531 mentioned above), it should ship in Firefox 5 in a few months. For a few details, see: http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html Regards, Chris Pearce.
Re: [whatwg] Media elements statistics
You can always chuck it in the whatwg wiki http://wiki.whatwg.org/wiki/Main_Page. :-) Silvia. On Sat, Feb 12, 2011 at 7:02 PM, Steve Lacey s...@chromium.org wrote: crickets... I'll work on a proposal spec for this ;-) On Mon, Feb 7, 2011 at 1:55 PM, Steve Lacey s...@chromium.org wrote: On Thu, Jan 27, 2011 at 4:18 PM, Steve Lacey s...@chromium.org wrote: On Thu, Jan 27, 2011 at 3:38 PM, Chris Double chris.dou...@double.co.nz wrote: On Fri, Jan 28, 2011 at 12:22 PM, Steve Lacey s...@chromium.org wrote: But for the media element I'd like to propose raw bytes instead of a rate as this allows the developer to construct their own rates (if needed) based on whatever window they want. It would also be useful to separate audio from video. A suggestion might be: Raw bytes sounds good. unsigned long audioBytesDecoded; unsigned long videoBytesDecoded; Though this seems a little strange to have these specifically on the media element as they reference particular media types. Another idea would be to move these to the video element and also add audioBytesDecoded to the audio element. Moving them to the video and audio element would mean you can't get the audioBytesDecoded on a video element which is what I'm assuming you want by having the two values. Yeah - I'd also add audioBytesDecoded to the audio element, which is an unpleasant dupe. Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Reviving thread... I have an initial patch in webkit (http://trac.webkit.org/changeset/77394) and the chromium work is underway - I wonder what might be a good approach to drive the apis closer together towards a real spec that everyone is happy with? There seems to be a lot of general agreement here (at least in principal :-) that this is needed. We'll be doing a bunch of experimentation once this has landed in chromium. Cheers! Steve
Re: [whatwg] Media elements statistics
crickets... I'll work on a proposal spec for this ;-) On Mon, Feb 7, 2011 at 1:55 PM, Steve Lacey s...@chromium.org wrote: On Thu, Jan 27, 2011 at 4:18 PM, Steve Lacey s...@chromium.org wrote: On Thu, Jan 27, 2011 at 3:38 PM, Chris Double chris.dou...@double.co.nz wrote: On Fri, Jan 28, 2011 at 12:22 PM, Steve Lacey s...@chromium.org wrote: But for the media element I'd like to propose raw bytes instead of a rate as this allows the developer to construct their own rates (if needed) based on whatever window they want. It would also be useful to separate audio from video. A suggestion might be: Raw bytes sounds good. unsigned long audioBytesDecoded; unsigned long videoBytesDecoded; Though this seems a little strange to have these specifically on the media element as they reference particular media types. Another idea would be to move these to the video element and also add audioBytesDecoded to the audio element. Moving them to the video and audio element would mean you can't get the audioBytesDecoded on a video element which is what I'm assuming you want by having the two values. Yeah - I'd also add audioBytesDecoded to the audio element, which is an unpleasant dupe. Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Reviving thread... I have an initial patch in webkit (http://trac.webkit.org/changeset/77394) and the chromium work is underway - I wonder what might be a good approach to drive the apis closer together towards a real spec that everyone is happy with? There seems to be a lot of general agreement here (at least in principal :-) that this is needed. We'll be doing a bunch of experimentation once this has landed in chromium. Cheers! Steve
Re: [whatwg] Media elements statistics
On Thu, Jan 27, 2011 at 4:18 PM, Steve Lacey s...@chromium.org wrote: On Thu, Jan 27, 2011 at 3:38 PM, Chris Double chris.dou...@double.co.nz wrote: On Fri, Jan 28, 2011 at 12:22 PM, Steve Lacey s...@chromium.org wrote: But for the media element I'd like to propose raw bytes instead of a rate as this allows the developer to construct their own rates (if needed) based on whatever window they want. It would also be useful to separate audio from video. A suggestion might be: Raw bytes sounds good. unsigned long audioBytesDecoded; unsigned long videoBytesDecoded; Though this seems a little strange to have these specifically on the media element as they reference particular media types. Another idea would be to move these to the video element and also add audioBytesDecoded to the audio element. Moving them to the video and audio element would mean you can't get the audioBytesDecoded on a video element which is what I'm assuming you want by having the two values. Yeah - I'd also add audioBytesDecoded to the audio element, which is an unpleasant dupe. Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Reviving thread... I have an initial patch in webkit (http://trac.webkit.org/changeset/77394) and the chromium work is underway - I wonder what might be a good approach to drive the apis closer together towards a real spec that everyone is happy with? There seems to be a lot of general agreement here (at least in principal :-) that this is needed. We'll be doing a bunch of experimentation once this has landed in chromium. Cheers! Steve
Re: [whatwg] Media elements statistics
Steve Lacey s...@chromium.org wrote: [Media elements] Another open question: what are sensible values if the information is not available. Zero seems wrong. This is a question that I have considered for some time for all the properties in HTMLMediaElement interface, not especially for your new proposed statistics. I have not come to any particular conclusions as yet. One option is to try to invent plausible values for the property values. e.g. you can seek anywhere. For us, with a third-party black box media player sitting at several levels of abstraction away (and sometimes on a separate processor), we do not have access to most of the information that is necessary to drive the algorithms. For example, network usage (if indeed there is any at all), which frames you have, seekable ranges, buffering statistics - all of these are unavailable. -- Stewart Brodie Team Leader - ANT Galio Browser ANT Software Limited
Re: [whatwg] Media elements statistics
On Thu, Jan 27, 2011 at 6:22 PM, Steve Lacey s...@chromium.org wrote: Another open question: what are sensible values if the information is not available. Zero seems wrong. Make them unsigned long? instead of unsigned long, and return null?
[whatwg] Media elements statistics
Hi, I'd like the raise this thread again: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-August/027929.html (I wasn't on the list at that point, so starting a new thread here and cc'ing a couple of folks from it...) I work on the media stack in Chromium and we'd like to implement something pretty similar. So I'm looking for comments... The original suggestion for the video element looks good: [Video Element] // Frames decoded and available for playback. unsigned long decodedFrames; // Frames dropped during playback for performance reasons. unsigned long droppedFrames; But for the media element I'd like to propose raw bytes instead of a rate as this allows the developer to construct their own rates (if needed) based on whatever window they want. It would also be useful to separate audio from video. A suggestion might be: [Media Element] unsigned long audioBytesDecoded; unsigned long videoBytesDecoded; Though this seems a little strange to have these specifically on the media element as they reference particular media types. Another idea would be to move these to the video element and also add audioBytesDecoded to the audio element. Another open question: what are sensible values if the information is not available. Zero seems wrong. Thoughts? Cheers! Steve
Re: [whatwg] Media elements statistics
Hi Steve et al, I'm working on a similar feature for Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Though we're implementing this more as a way of measuring the performance of our decoding and rendering pipeline, rather than providing playback/decode-rate stats. unsigned long audioBytesDecoded; unsigned long videoBytesDecoded; Out of curiosity, why do you want this feature? What does it give you that @buffered and @currentTime does not? Raw bytes reasonable to me, the feedback on the FOMS list regarding playback statistics showed webdevs liked that idea. How would you handle frames dropped by the decoder in order to prevent it falling behind? Would you count their bytes as decoded? Another open question: what are sensible values if the information is not available. Zero seems wrong. Return Number.NaN? Or provide some kind of ability to query whether there is audio and video? Regards, Chris P. On 28/01/2011 12:22 p.m., Steve Lacey wrote: Hi, I'd like the raise this thread again: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-August/027929.html (I wasn't on the list at that point, so starting a new thread here and cc'ing a couple of folks from it...) I work on the media stack in Chromium and we'd like to implement something pretty similar. So I'm looking for comments... The original suggestion for the video element looks good: [Video Element] // Frames decoded and available for playback. unsigned long decodedFrames; // Frames dropped during playback for performance reasons. unsigned long droppedFrames; But for the media element I'd like to propose raw bytes instead of a rate as this allows the developer to construct their own rates (if needed) based on whatever window they want. It would also be useful to separate audio from video. A suggestion might be: [Media Element] unsigned long audioBytesDecoded; unsigned long videoBytesDecoded; Though this seems a little strange to have these specifically on the media element as they reference particular media types. Another idea would be to move these to the video element and also add audioBytesDecoded to the audio element. Another open question: what are sensible values if the information is not available. Zero seems wrong. Thoughts? Cheers! Steve
Re: [whatwg] Media elements statistics
On Thu, Jan 27, 2011 at 3:53 PM, Chris Pearce ch...@pearce.org.nz wrote: Hi Steve et al, I'm working on a similar feature for Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Though we're implementing this more as a way of measuring the performance of our decoding and rendering pipeline, rather than providing playback/decode-rate stats. unsigned long audioBytesDecoded; unsigned long videoBytesDecoded; Out of curiosity, why do you want this feature? What does it give you that @buffered and @currentTime does not? Being able to determine the bitrate that's currently being decoded has been a request from devs (for similar reasons that devs on the FOMS list have I expect). Raw data seems generally useful. Raw bytes reasonable to me, the feedback on the FOMS list regarding playback statistics showed webdevs liked that idea. How would you handle frames dropped by the decoder in order to prevent it falling behind? Would you count their bytes as decoded? Right now I include the frames dropped in the decoded count. It's kinda orthogonal to frames decoded/dropped as one (bytesdecoded) gives the performance for decoding and the frame counts give performance of presentation. Another open question: what are sensible values if the information is not available. Zero seems wrong. Return Number.NaN? Or provide some kind of ability to query whether there is audio and video? Thanks! Regards, Chris P. On 28/01/2011 12:22 p.m., Steve Lacey wrote: Hi, I'd like the raise this thread again: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-August/027929.html (I wasn't on the list at that point, so starting a new thread here and cc'ing a couple of folks from it...) I work on the media stack in Chromium and we'd like to implement something pretty similar. So I'm looking for comments... The original suggestion for the video element looks good: [Video Element] // Frames decoded and available for playback. unsigned long decodedFrames; // Frames dropped during playback for performance reasons. unsigned long droppedFrames; But for the media element I'd like to propose raw bytes instead of a rate as this allows the developer to construct their own rates (if needed) based on whatever window they want. It would also be useful to separate audio from video. A suggestion might be: [Media Element] unsigned long audioBytesDecoded; unsigned long videoBytesDecoded; Though this seems a little strange to have these specifically on the media element as they reference particular media types. Another idea would be to move these to the video element and also add audioBytesDecoded to the audio element. Another open question: what are sensible values if the information is not available. Zero seems wrong. Thoughts? Cheers! Steve
Re: [whatwg] Media elements statistics
On Thu, Jan 27, 2011 at 3:38 PM, Chris Double chris.dou...@double.co.nz wrote: On Fri, Jan 28, 2011 at 12:22 PM, Steve Lacey s...@chromium.org wrote: But for the media element I'd like to propose raw bytes instead of a rate as this allows the developer to construct their own rates (if needed) based on whatever window they want. It would also be useful to separate audio from video. A suggestion might be: Raw bytes sounds good. unsigned long audioBytesDecoded; unsigned long videoBytesDecoded; Though this seems a little strange to have these specifically on the media element as they reference particular media types. Another idea would be to move these to the video element and also add audioBytesDecoded to the audio element. Moving them to the video and audio element would mean you can't get the audioBytesDecoded on a video element which is what I'm assuming you want by having the two values. Yeah - I'd also add audioBytesDecoded to the audio element, which is an unpleasant dupe. Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Chris. -- http://www.bluishcoder.co.nz
