Re: We should write memory reporters for new features as they're being developed
On Wed, Dec 18, 2013 at 6:17 PM, bent wrote: > > Didn't we have a plan for asynchronous report collection to make this kind of > thing easier? "Plan" is a strong word :) It was suggested as a way of dealing with multiple processes, but it turned out to be more complicated than the more direct approach I ended up taking. And until now web workers have been the only case where synchronousness was a problem. Let's see how the web audio stuff pans out. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: We should write memory reporters for new features as they're being developed
On Tuesday, December 17, 2013 11:13:50 PM UTC-5, Nicholas Nethercote wrote: > Web workers use a hack: code running on the main thread completely > pauses (via locks) each worker's actions and then the main thread code > measures the worker's data structures. It should be noted that it took a considerable amount of time to get this code bug-free, and I would hate to tell people that they should emulate its approach. Didn't we have a plan for asynchronous report collection to make this kind of thing easier? ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: We should write memory reporters for new features as they're being developed
On Tue, Dec 17, 2013 at 6:54 PM, Ehsan Akhgari wrote: > > How does one write a memory reporter that captures data stored on more than > just the main thread? In the case of Web Audio for example, some of our > buffers live in another thread, and as far as I can see the memory reporter > API is mostly synchronous. Is locking the right way to do this? Good question. It depends on the details. Each memory reporter's CollectReports() method is called only from the main thread, and the CollectReports() call is synchronous. There isn't a good way to do asynchronous reporting. Web workers use a hack: code running on the main thread completely pauses (via locks) each worker's actions and then the main thread code measures the worker's data structures. So you may need to do something similar. One thing that's possibly in your favour is that memory reporting isn't performance-critical, since it only happens in response to user requests. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: We should write memory reporters for new features as they're being developed
First of all, sorry for neglecting that bug! And +1 on this suggestion. How does one write a memory reporter that captures data stored on more than just the main thread? In the case of Web Audio for example, some of our buffers live in another thread, and as far as I can see the memory reporter API is mostly synchronous. Is locking the right way to do this? Thanks! Ehsan On 12/16/2013, 10:57 PM, Nicholas Nethercote wrote: Hi, For over a month I've been working with a user to identify the cause of a bad memory leak (https://bugzilla.mozilla.org/show_bug.cgi?id=936784). Just today we got DMD working sufficiently well on Windows that the user was able to run it, and it pointed the finger at webaudio. Which is great progress! But if we had memory reporters for web audio, all this would have been so much easier. And (queue sad-face) we actually have a six-month old bug open for that: https://bugzilla.mozilla.org/show_bug.cgi?id=884368 So I want to propose something: if you're working on a change that will introduce significant new causes of memory consumption, you should write a memory reporter for it at the same time, rather than (maybe) doing it later, or letting someone else do it. And in this context, "significant" may be smaller than you expect. For example, we have numerous reporters for things that are typically only 100s of KBs. On B2G, 100KB per process is significant. Understanding the data structures is usually the hard part of writing a memory reporter. The actual reporter registration side isn't hard, and there are plenty of examples to refer to. So the author of the new code is typically the best person to write a reporter for it. And I'm happy to help (and review). Furthermore, memory reporters are best verified by doing a DMD run, and DMD now runs on all tier-1 platforms and is well documented (https://wiki.mozilla.org/Performance/MemShrink/DMD). So that shouldn't be an obstacle. This couldn't be a hard-and-fast rule, but I would like for it to be something that developers and reviewers keep in mind -- Does this code need a memory reporter? And have you verified it with DMD? Thoughts? Nick p.s.: The web audio bug prompted me to make this suggestion, and is a good example of the potential benefits. But I don't mean to criticize those who implemented web audio; apologies if it comes across that way. In that spirit, let's keep discussion on the general proposal as much as possible, rather than web audio. Thanks! ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: We should write memory reporters for new features as they're being developed
On Tue, Dec 17, 2013 at 2:55 PM, David Rajchenbach-Teller wrote: > > Is it possible to write memory reporters for JS-implemented code? It's possible. Some points about this... - Memory used by the JS engine already has good coverage in the "explicit" tree. - It's hard to measure the sizes of things in JS code. And there are lots of JS things (e.g. shapes) that aren't even visible from JS code. It might be interesting to measure counts of some things, though. - https://bugzilla.mozilla.org/show_bug.cgi?id=932156 is an idea to mark particular JS objects as special, and thus get identified specially in about:memory. IMO this is the most promising direction for improving reporting of JS code. - The DownloadThemAll! add-on implements some JS-side reporters. I'm not sure exactly what is measured, though. But in general, memory reporters are more relevant for C++ code. > Also, is it possible to write memory reporters for Chrome Worker code? That's also JS code, right? JS runtimes for workers get measured the same way that the main JS runtime does. So I don't think Chrome worker code needs any treatment different to other kinds of JS code. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: We should write memory reporters for new features as they're being developed
Generally, I like the idea. Is it possible to write memory reporters for JS-implemented code? Also, is it possible to write memory reporters for Chrome Worker code? Cheers, David On 12/17/13 4:57 AM, Nicholas Nethercote wrote: > So I want to propose something: if you're working on a change that > will introduce significant new causes of memory consumption, you > should write a memory reporter for it at the same time, rather than > (maybe) doing it later, or letting someone else do it. And in this > context, "significant" may be smaller than you expect. For example, > we have numerous reporters for things that are typically only 100s of > KBs. On B2G, 100KB per process is significant. -- David Rajchenbach-Teller, PhD Performance Team, Mozilla ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
We should write memory reporters for new features as they're being developed
Hi, For over a month I've been working with a user to identify the cause of a bad memory leak (https://bugzilla.mozilla.org/show_bug.cgi?id=936784). Just today we got DMD working sufficiently well on Windows that the user was able to run it, and it pointed the finger at webaudio. Which is great progress! But if we had memory reporters for web audio, all this would have been so much easier. And (queue sad-face) we actually have a six-month old bug open for that: https://bugzilla.mozilla.org/show_bug.cgi?id=884368 So I want to propose something: if you're working on a change that will introduce significant new causes of memory consumption, you should write a memory reporter for it at the same time, rather than (maybe) doing it later, or letting someone else do it. And in this context, "significant" may be smaller than you expect. For example, we have numerous reporters for things that are typically only 100s of KBs. On B2G, 100KB per process is significant. Understanding the data structures is usually the hard part of writing a memory reporter. The actual reporter registration side isn't hard, and there are plenty of examples to refer to. So the author of the new code is typically the best person to write a reporter for it. And I'm happy to help (and review). Furthermore, memory reporters are best verified by doing a DMD run, and DMD now runs on all tier-1 platforms and is well documented (https://wiki.mozilla.org/Performance/MemShrink/DMD). So that shouldn't be an obstacle. This couldn't be a hard-and-fast rule, but I would like for it to be something that developers and reviewers keep in mind -- Does this code need a memory reporter? And have you verified it with DMD? Thoughts? Nick p.s.: The web audio bug prompted me to make this suggestion, and is a good example of the potential benefits. But I don't mean to criticize those who implemented web audio; apologies if it comes across that way. In that spirit, let's keep discussion on the general proposal as much as possible, rather than web audio. Thanks! ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
