[chromium-dev] Re: Memory usage in chrome

[Ben Laurie]

On Wed, Jun 24, 2009 at 8:19 AM, Mike Belshe wrote:
> Anecdotally, a couple of people have said "it works" and a couple of people
> have said "it makes no difference".  I do believe that people doing compiles
> could see a difference.
> To determine if it was real, we did an experiment with memory model high and
> memory model medium and measured the whiteout duration.  We did not see any
> statistical difference between the two.  It could be that that was not a
> good test, but we don't know of a better test.

FWIW, it makes a huge difference on my machine, too. The main
competition for RAM is Thunderbird (VM 1.3GB right now) and Firefox
(480 MB). The biggest Chrome process is 400 MB.

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Beltzner]

On 29-Jun-09, at 5:37 PM, Marc-Antoine Ruel wrote:

> The measurement timing is important since the OS will trigger page  
> flush after X number of seconds. If you are not in a hurry, usually  
> taking measurements 60 seconds after your test is idle will be quite  
> stable.

For what it's worth, our test waited about 3-5 minutes once done to  
allow the browsers to flush memory.

cheers,
mike

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Marc-Antoine Ruel]

Actually, I had done testing like that in a VM to get a sense of raw
process/threads limits on a low memory system. It's relatively easy to
achieve a "fair" level of confidence with a live VM snapshot and making sure
to reuse the same timings for measurement.
The measurement timing is important since the OS will trigger page flush
after X number of seconds. If you are not in a hurry, usually taking
measurements 60 seconds after your test is idle will be quite stable.

M-A

On Sat, Jun 27, 2009 at 7:39 PM, Mike Belshe  wrote:

> This one is the hardest to test, you need to run a pristinely clean system
> to execute.
> Also - don't forget to make the browser window sizes the same (and with the
> same amount of visible window) for all browsers under test, because if the
> kernel can't offload to the graphics card, the display memory will be
> counted here.
>
> But yeah, if you can make all that work, then it is a good test!
> mike
>
>
> On Sat, Jun 27, 2009 at 2:50 PM, Linus Upson  wrote:
>
>> If I recall correctly, the best way we found to measure the total memory
>> usage of a multi-process system like chrome was to measure the total commit
>> charge of windows as you run the test. This will correctly account for
>> shared memory, mapped pages that have been touched, kernel memory, etc. I
>> don't recall if it includes virtual alloced paged that haven't been made
>> real. The big limitation is that your test needs to be the only thing
>> running on the machine.
>> Linus
>>
>>
>> On Thu, Jun 25, 2009 at 4:11 PM, Mike Beltzner wrote:
>>
>>>
>>> On 25-Jun-09, at 7:02 PM, Mike Belshe wrote:
>>>
>>> > This screen actually confuses me a little, as the Summary statistics
>>> > don't match the summation of the process based statistics. Do you
>>> > mean to say your summary statistics take into account the memory
>>> > that's being shared across the various processes?
>>> >
>>> > Correct.
>>> >
>>> > The "shared" across all processes is a bit of a hack, because you
>>> > can't know exactly which pages are shared across every single
>>> > process.  We do a heuristic.
>>>
>>> Cool! Good to know. I'll take a peek into that code you mentioned to
>>> see what the heuristic is that you're using.
>>>
>>> > Interestingly, as I watched this value change while webpages were
>>> > loading, it tracked the same pattern of growth/decline as "Memory
>>> > (Private Working Set)" in the Task Manager, though the values were
>>> > usually about 2x or so more. I suppose this is due to the heap
>>> > sharing you were speaking of earlier?
>>> >
>>> > I'm not quite sure what you mean.
>>>
>>> I'm basically being lazy. I'd like to not have to make my own counter
>>> for Private Working Set, so I watched the values of "Memory (Private
>>> Working Set)" and "Commit Size" in the Task Manager as the test ran,
>>> and noticed that they increased/decreased at the same time, and the
>>> delta between them was a near constant 2x. Since my interest here is
>>> developing a metric that can help us understand when we're regressing/
>>> improving memory usage, the exact value isn't as important to me as
>>> the delta. If the deltas are simply off by a constant factor, I could
>>> live with that.
>>>
>>> As I said: lazy!
>>>
>>> >
>>> > The "Working Set - Private" counter doesn't seem to have a structure
>>> > according to the MSDN document; that's what maps to the "Memory
>>> > (Private Working Set)" column in the TaskManager.
>>> >
>>> > Right, I think you have to use QueryWorkingSet, walk the pages and
>>> > categorize them yourself.
>>> >
>>> > OK, I can look into trying that. Though I'm wondering if it's worth
>>> > the bother, as the meta-pattern, to me, is more interesting than the
>>> > precise megabyte count.
>>> >
>>> > For a single process browser, it's not worth the effort; I think
>>> > it's the only way to know how to account for shared memory.
>>>
>>>
>>> > The closest thing I can find is the "Working Set" counter, which
>>> > uses the PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure and
>>> > shows up in the Vista Task Manager as "Working Set (Memory)"
>>> >
>>> > For multi-proc browsers like chrome, this will way overstate RAM;
>>> > there is a good 5-6MB of shared working set in each process.  So for
>>> > 10 tabs, you'd could an extra 50MB for Chrome if you do it this way.
>>> >
>>> > Looking both in Task Manager and about:memory, when I have 30 tabs
>>> > open I'm not seeing 30 processes. Are you sure you're right about
>>> > this point?
>>> >
>>> > You don't always get a new process for every tab.  If two tabs are
>>> > connected via javascript, then they'll be in the same process (the
>>> > about:memory shows which tabs are in the same process).  So,
>>> > clicking a link, for example, will open in the same tab, but typing
>>> > the URL in the omnibox will create a new process.  Others could tell
>>> > you more about the exact policy for when you get a new process and
>>> > when you don't.
>>>
>>> Someone just did in IRC, actua

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

This one is the hardest to test, you need to run a pristinely clean system
to execute.
Also - don't forget to make the browser window sizes the same (and with the
same amount of visible window) for all browsers under test, because if the
kernel can't offload to the graphics card, the display memory will be
counted here.

But yeah, if you can make all that work, then it is a good test!
mike


On Sat, Jun 27, 2009 at 2:50 PM, Linus Upson  wrote:

> If I recall correctly, the best way we found to measure the total memory
> usage of a multi-process system like chrome was to measure the total commit
> charge of windows as you run the test. This will correctly account for
> shared memory, mapped pages that have been touched, kernel memory, etc. I
> don't recall if it includes virtual alloced paged that haven't been made
> real. The big limitation is that your test needs to be the only thing
> running on the machine.
> Linus
>
>
> On Thu, Jun 25, 2009 at 4:11 PM, Mike Beltzner wrote:
>
>>
>> On 25-Jun-09, at 7:02 PM, Mike Belshe wrote:
>>
>> > This screen actually confuses me a little, as the Summary statistics
>> > don't match the summation of the process based statistics. Do you
>> > mean to say your summary statistics take into account the memory
>> > that's being shared across the various processes?
>> >
>> > Correct.
>> >
>> > The "shared" across all processes is a bit of a hack, because you
>> > can't know exactly which pages are shared across every single
>> > process.  We do a heuristic.
>>
>> Cool! Good to know. I'll take a peek into that code you mentioned to
>> see what the heuristic is that you're using.
>>
>> > Interestingly, as I watched this value change while webpages were
>> > loading, it tracked the same pattern of growth/decline as "Memory
>> > (Private Working Set)" in the Task Manager, though the values were
>> > usually about 2x or so more. I suppose this is due to the heap
>> > sharing you were speaking of earlier?
>> >
>> > I'm not quite sure what you mean.
>>
>> I'm basically being lazy. I'd like to not have to make my own counter
>> for Private Working Set, so I watched the values of "Memory (Private
>> Working Set)" and "Commit Size" in the Task Manager as the test ran,
>> and noticed that they increased/decreased at the same time, and the
>> delta between them was a near constant 2x. Since my interest here is
>> developing a metric that can help us understand when we're regressing/
>> improving memory usage, the exact value isn't as important to me as
>> the delta. If the deltas are simply off by a constant factor, I could
>> live with that.
>>
>> As I said: lazy!
>>
>> >
>> > The "Working Set - Private" counter doesn't seem to have a structure
>> > according to the MSDN document; that's what maps to the "Memory
>> > (Private Working Set)" column in the TaskManager.
>> >
>> > Right, I think you have to use QueryWorkingSet, walk the pages and
>> > categorize them yourself.
>> >
>> > OK, I can look into trying that. Though I'm wondering if it's worth
>> > the bother, as the meta-pattern, to me, is more interesting than the
>> > precise megabyte count.
>> >
>> > For a single process browser, it's not worth the effort; I think
>> > it's the only way to know how to account for shared memory.
>>
>>
>> > The closest thing I can find is the "Working Set" counter, which
>> > uses the PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure and
>> > shows up in the Vista Task Manager as "Working Set (Memory)"
>> >
>> > For multi-proc browsers like chrome, this will way overstate RAM;
>> > there is a good 5-6MB of shared working set in each process.  So for
>> > 10 tabs, you'd could an extra 50MB for Chrome if you do it this way.
>> >
>> > Looking both in Task Manager and about:memory, when I have 30 tabs
>> > open I'm not seeing 30 processes. Are you sure you're right about
>> > this point?
>> >
>> > You don't always get a new process for every tab.  If two tabs are
>> > connected via javascript, then they'll be in the same process (the
>> > about:memory shows which tabs are in the same process).  So,
>> > clicking a link, for example, will open in the same tab, but typing
>> > the URL in the omnibox will create a new process.  Others could tell
>> > you more about the exact policy for when you get a new process and
>> > when you don't.
>>
>> Someone just did in IRC, actually. Apparently in addition to what you
>> said, as soon as a page is in cache, processes get pooled. I clear
>> caches between test runs, but it sounds like since we're calling these
>> with window.open() in our test, they all get placed in the same process.
>>
>> Overall, though, that should mean that we're *not* double counting
>> memory. In fact, when I observed as the test ran, there were only
>> three processes: one for the browser, one for the single content
>> process from which all tabs were spawned, and one for Shockwave/Flash.
>> Good news, I guess, in terms of reporting accurately!
>>
>> > OK - I think this might basical

[chromium-dev] Re: Memory usage in chrome

[Dan Kegel]

On Sat, Jun 27, 2009 at 2:50 PM, Linus Upson wrote:
> If I recall correctly, the best way we found to measure the total memory
> usage of a multi-process system like chrome was to measure the total commit
> charge of windows as you run the test.

My favorite test is to plot the performance of the app on some
benchmark as a function of the amount of memory the computer
is booted with.
Comparing that plot for two different apps gives you
a pretty clear picture of relative RAM requirements.

Chrome kicked the bejeesus out of Firefox last time I tried
that, but I think that was before Firefox 3.5.
- Dan

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Linus Upson]

If I recall correctly, the best way we found to measure the total memory
usage of a multi-process system like chrome was to measure the total commit
charge of windows as you run the test. This will correctly account for
shared memory, mapped pages that have been touched, kernel memory, etc. I
don't recall if it includes virtual alloced paged that haven't been made
real. The big limitation is that your test needs to be the only thing
running on the machine.
Linus


On Thu, Jun 25, 2009 at 4:11 PM, Mike Beltzner  wrote:

>
> On 25-Jun-09, at 7:02 PM, Mike Belshe wrote:
>
> > This screen actually confuses me a little, as the Summary statistics
> > don't match the summation of the process based statistics. Do you
> > mean to say your summary statistics take into account the memory
> > that's being shared across the various processes?
> >
> > Correct.
> >
> > The "shared" across all processes is a bit of a hack, because you
> > can't know exactly which pages are shared across every single
> > process.  We do a heuristic.
>
> Cool! Good to know. I'll take a peek into that code you mentioned to
> see what the heuristic is that you're using.
>
> > Interestingly, as I watched this value change while webpages were
> > loading, it tracked the same pattern of growth/decline as "Memory
> > (Private Working Set)" in the Task Manager, though the values were
> > usually about 2x or so more. I suppose this is due to the heap
> > sharing you were speaking of earlier?
> >
> > I'm not quite sure what you mean.
>
> I'm basically being lazy. I'd like to not have to make my own counter
> for Private Working Set, so I watched the values of "Memory (Private
> Working Set)" and "Commit Size" in the Task Manager as the test ran,
> and noticed that they increased/decreased at the same time, and the
> delta between them was a near constant 2x. Since my interest here is
> developing a metric that can help us understand when we're regressing/
> improving memory usage, the exact value isn't as important to me as
> the delta. If the deltas are simply off by a constant factor, I could
> live with that.
>
> As I said: lazy!
>
> >
> > The "Working Set - Private" counter doesn't seem to have a structure
> > according to the MSDN document; that's what maps to the "Memory
> > (Private Working Set)" column in the TaskManager.
> >
> > Right, I think you have to use QueryWorkingSet, walk the pages and
> > categorize them yourself.
> >
> > OK, I can look into trying that. Though I'm wondering if it's worth
> > the bother, as the meta-pattern, to me, is more interesting than the
> > precise megabyte count.
> >
> > For a single process browser, it's not worth the effort; I think
> > it's the only way to know how to account for shared memory.
>
>
> > The closest thing I can find is the "Working Set" counter, which
> > uses the PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure and
> > shows up in the Vista Task Manager as "Working Set (Memory)"
> >
> > For multi-proc browsers like chrome, this will way overstate RAM;
> > there is a good 5-6MB of shared working set in each process.  So for
> > 10 tabs, you'd could an extra 50MB for Chrome if you do it this way.
> >
> > Looking both in Task Manager and about:memory, when I have 30 tabs
> > open I'm not seeing 30 processes. Are you sure you're right about
> > this point?
> >
> > You don't always get a new process for every tab.  If two tabs are
> > connected via javascript, then they'll be in the same process (the
> > about:memory shows which tabs are in the same process).  So,
> > clicking a link, for example, will open in the same tab, but typing
> > the URL in the omnibox will create a new process.  Others could tell
> > you more about the exact policy for when you get a new process and
> > when you don't.
>
> Someone just did in IRC, actually. Apparently in addition to what you
> said, as soon as a page is in cache, processes get pooled. I clear
> caches between test runs, but it sounds like since we're calling these
> with window.open() in our test, they all get placed in the same process.
>
> Overall, though, that should mean that we're *not* double counting
> memory. In fact, when I observed as the test ran, there were only
> three processes: one for the browser, one for the single content
> process from which all tabs were spawned, and one for Shockwave/Flash.
> Good news, I guess, in terms of reporting accurately!
>
> > OK - I think this might basically use one renderer process in
> > chrome?  Because of the new-process creation policy, it may not be
> > representative of real world usage.  Darin?
>
> Right, but AIUI, it's an erring on the side of reporting less, not
> more. If there's a better way to automate pageloads that represents
> real world usage, please let me know.
>
> > The whole while, we measure the amount of memory taken using the
> > PROCESS_MEMORY_COUNTERS structure, summating over processes when
> > multiple exist (as they do in the case of Internet Explorer 8 and
> > Chrome 2)

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

On Fri, Jun 26, 2009 at 10:11 AM, Mike Beltzner wrote:

> On 26-Jun-09, at 12:59 PM, Mike Belshe wrote:
>
>  Overall, though, that should mean that we're *not* double counting memory.
>> In fact, when I observed as the test ran, there were only three processes:
>> one for the browser, one for the single content process from which all tabs
>> were spawned, and one for Shockwave/Flash. Good news, I guess, in terms of
>> reporting accurately!
>>
>> Good news and bad news :-)  If you publish results saying how Chrome did,
>> Chrome doesn't get to cleanup as cleanly in this case.  It still *should*,
>> but it's not what users do :-)
>>
>
> Hm, so the thinking is that since users will close tabs and open new ones
> and navigate from the Omnibox that will close and open processes, thus
> freeing memory?
>
> I don't think I'm going to publish results, per se, but if I do, I'll point
> this part out for sure.
>
>  Right, but AIUI, it's an erring on the side of reporting less, not more.
>> If there's a better way to automate pageloads that represents real world
>> usage, please let me know.
>>
>> I don't know of cross-browser code that will accomplish what you want.
>>  Maybe we should add that.
>>
>
> What about command line invocation? Does that spawn new processes?


Oh - good call - yes - that should work.


>
>
> cheers,
> mike
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Beltzner]

On 26-Jun-09, at 12:59 PM, Mike Belshe wrote:

> Overall, though, that should mean that we're *not* double counting  
> memory. In fact, when I observed as the test ran, there were only  
> three processes: one for the browser, one for the single content  
> process from which all tabs were spawned, and one for Shockwave/ 
> Flash. Good news, I guess, in terms of reporting accurately!
>
> Good news and bad news :-)  If you publish results saying how Chrome  
> did, Chrome doesn't get to cleanup as cleanly in this case.  It  
> still *should*, but it's not what users do :-)

Hm, so the thinking is that since users will close tabs and open new  
ones and navigate from the Omnibox that will close and open processes,  
thus freeing memory?

I don't think I'm going to publish results, per se, but if I do, I'll  
point this part out for sure.

> Right, but AIUI, it's an erring on the side of reporting less, not  
> more. If there's a better way to automate pageloads that represents  
> real world usage, please let me know.
>
> I don't know of cross-browser code that will accomplish what you  
> want.  Maybe we should add that.

What about command line invocation? Does that spawn new processes?

cheers,
mike

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

On Thu, Jun 25, 2009 at 4:11 PM, Mike Beltzner  wrote:

> On 25-Jun-09, at 7:02 PM, Mike Belshe wrote:
>
>  This screen actually confuses me a little, as the Summary statistics don't
>> match the summation of the process based statistics. Do you mean to say your
>> summary statistics take into account the memory that's being shared across
>> the various processes?
>>
>> Correct.
>>
>> The "shared" across all processes is a bit of a hack, because you can't
>> know exactly which pages are shared across every single process.  We do a
>> heuristic.
>>
>
> Cool! Good to know. I'll take a peek into that code you mentioned to see
> what the heuristic is that you're using.
>
>  Interestingly, as I watched this value change while webpages were loading,
>> it tracked the same pattern of growth/decline as "Memory (Private Working
>> Set)" in the Task Manager, though the values were usually about 2x or so
>> more. I suppose this is due to the heap sharing you were speaking of
>> earlier?
>>
>> I'm not quite sure what you mean.
>>
>
> I'm basically being lazy. I'd like to not have to make my own counter for
> Private Working Set, so I watched the values of "Memory (Private Working
> Set)" and "Commit Size" in the Task Manager as the test ran, and noticed
> that they increased/decreased at the same time, and the delta between them
> was a near constant 2x. Since my interest here is developing a metric that
> can help us understand when we're regressing/improving memory usage, the
> exact value isn't as important to me as the delta. If the deltas are simply
> off by a constant factor, I could live with that.
>
> As I said: lazy!
>

understood.

Just in case, here is the code for you to take:
http://src.chromium.org/viewvc/chrome/trunk/src/base/process_util_win.cc?view=markup&pathrev=14705





>
>
>
>> The "Working Set - Private" counter doesn't seem to have a structure
>> according to the MSDN document; that's what maps to the "Memory (Private
>> Working Set)" column in the TaskManager.
>>
>> Right, I think you have to use QueryWorkingSet, walk the pages and
>> categorize them yourself.
>>
>> OK, I can look into trying that. Though I'm wondering if it's worth the
>> bother, as the meta-pattern, to me, is more interesting than the precise
>> megabyte count.
>>
>> For a single process browser, it's not worth the effort; I think it's the
>> only way to know how to account for shared memory.
>>
>
>
>  The closest thing I can find is the "Working Set" counter, which uses the
>> PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure and shows up in the
>> Vista Task Manager as "Working Set (Memory)"
>>
>> For multi-proc browsers like chrome, this will way overstate RAM; there is
>> a good 5-6MB of shared working set in each process.  So for 10 tabs, you'd
>> could an extra 50MB for Chrome if you do it this way.
>>
>> Looking both in Task Manager and about:memory, when I have 30 tabs open
>> I'm not seeing 30 processes. Are you sure you're right about this point?
>>
>> You don't always get a new process for every tab.  If two tabs are
>> connected via javascript, then they'll be in the same process (the
>> about:memory shows which tabs are in the same process).  So, clicking a
>> link, for example, will open in the same tab, but typing the URL in the
>> omnibox will create a new process.  Others could tell you more about the
>> exact policy for when you get a new process and when you don't.
>>
>
> Someone just did in IRC, actually. Apparently in addition to what you said,
> as soon as a page is in cache, processes get pooled. I clear caches between
> test runs, but it sounds like since we're calling these with window.open()
> in our test, they all get placed in the same process.
>
> Overall, though, that should mean that we're *not* double counting memory.
> In fact, when I observed as the test ran, there were only three processes:
> one for the browser, one for the single content process from which all tabs
> were spawned, and one for Shockwave/Flash. Good news, I guess, in terms of
> reporting accurately!


Good news and bad news :-)  If you publish results saying how Chrome did,
Chrome doesn't get to cleanup as cleanly in this case.  It still *should*,
but it's not what users do :-)



>
>
>  OK - I think this might basically use one renderer process in chrome?
>>  Because of the new-process creation policy, it may not be representative of
>> real world usage.  Darin?
>>
>
> Right, but AIUI, it's an erring on the side of reporting less, not more. If
> there's a better way to automate pageloads that represents real world usage,
> please let me know.


I don't know of cross-browser code that will accomplish what you want.
 Maybe we should add that.



>
>
>  The whole while, we measure the amount of memory taken using the
>> PROCESS_MEMORY_COUNTERS structure, summating over processes when multiple
>> exist (as they do in the case of Internet Explorer 8 and Chrome 2)
>>
>> Ok - that will double count shared memory.  I'd estimate 

[chromium-dev] Re: Memory usage in chrome

[Mike Beltzner]

On 25-Jun-09, at 7:02 PM, Mike Belshe wrote:

> This screen actually confuses me a little, as the Summary statistics  
> don't match the summation of the process based statistics. Do you  
> mean to say your summary statistics take into account the memory  
> that's being shared across the various processes?
>
> Correct.
>
> The "shared" across all processes is a bit of a hack, because you  
> can't know exactly which pages are shared across every single  
> process.  We do a heuristic.

Cool! Good to know. I'll take a peek into that code you mentioned to  
see what the heuristic is that you're using.

> Interestingly, as I watched this value change while webpages were  
> loading, it tracked the same pattern of growth/decline as "Memory  
> (Private Working Set)" in the Task Manager, though the values were  
> usually about 2x or so more. I suppose this is due to the heap  
> sharing you were speaking of earlier?
>
> I'm not quite sure what you mean.

I'm basically being lazy. I'd like to not have to make my own counter  
for Private Working Set, so I watched the values of "Memory (Private  
Working Set)" and "Commit Size" in the Task Manager as the test ran,  
and noticed that they increased/decreased at the same time, and the  
delta between them was a near constant 2x. Since my interest here is  
developing a metric that can help us understand when we're regressing/ 
improving memory usage, the exact value isn't as important to me as  
the delta. If the deltas are simply off by a constant factor, I could  
live with that.

As I said: lazy!

>
> The "Working Set - Private" counter doesn't seem to have a structure  
> according to the MSDN document; that's what maps to the "Memory  
> (Private Working Set)" column in the TaskManager.
>
> Right, I think you have to use QueryWorkingSet, walk the pages and  
> categorize them yourself.
>
> OK, I can look into trying that. Though I'm wondering if it's worth  
> the bother, as the meta-pattern, to me, is more interesting than the  
> precise megabyte count.
>
> For a single process browser, it's not worth the effort; I think  
> it's the only way to know how to account for shared memory.


> The closest thing I can find is the "Working Set" counter, which  
> uses the PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure and  
> shows up in the Vista Task Manager as "Working Set (Memory)"
>
> For multi-proc browsers like chrome, this will way overstate RAM;  
> there is a good 5-6MB of shared working set in each process.  So for  
> 10 tabs, you'd could an extra 50MB for Chrome if you do it this way.
>
> Looking both in Task Manager and about:memory, when I have 30 tabs  
> open I'm not seeing 30 processes. Are you sure you're right about  
> this point?
>
> You don't always get a new process for every tab.  If two tabs are  
> connected via javascript, then they'll be in the same process (the  
> about:memory shows which tabs are in the same process).  So,  
> clicking a link, for example, will open in the same tab, but typing  
> the URL in the omnibox will create a new process.  Others could tell  
> you more about the exact policy for when you get a new process and  
> when you don't.

Someone just did in IRC, actually. Apparently in addition to what you  
said, as soon as a page is in cache, processes get pooled. I clear  
caches between test runs, but it sounds like since we're calling these  
with window.open() in our test, they all get placed in the same process.

Overall, though, that should mean that we're *not* double counting  
memory. In fact, when I observed as the test ran, there were only  
three processes: one for the browser, one for the single content  
process from which all tabs were spawned, and one for Shockwave/Flash.  
Good news, I guess, in terms of reporting accurately!

> OK - I think this might basically use one renderer process in  
> chrome?  Because of the new-process creation policy, it may not be  
> representative of real world usage.  Darin?

Right, but AIUI, it's an erring on the side of reporting less, not  
more. If there's a better way to automate pageloads that represents  
real world usage, please let me know.

> The whole while, we measure the amount of memory taken using the  
> PROCESS_MEMORY_COUNTERS structure, summating over processes when  
> multiple exist (as they do in the case of Internet Explorer 8 and  
> Chrome 2)
>
> Ok - that will double count shared memory.  I'd estimate 3-5MB per  
> process.

So we're talking about over-reporting by 9-15MB across the test.  
Again, good to know.

> I'll try to take a closer look at your test, but I'm not sure when  
> I'll have time :-(

No rush here, and I appreciate your time and candor to date!

cheers,
mike

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

On Thu, Jun 25, 2009 at 3:26 PM, Mike Beltzner  wrote:

> On 25-Jun-09, at 12:52 PM, Mike Belshe wrote:
>
>  Yeah, the APIs all have constraints.  We end up walking the pages and
>> adding them up.  See process_util_win.cc in the chromium tree.  Be sure to
>> check about:memory and hover over the little "?" icons to see what we
>> measure.
>>
>
> This screen actually confuses me a little, as the Summary statistics don't
> match the summation of the process based statistics. Do you mean to say your
> summary statistics take into account the memory that's being shared across
> the various processes?


Correct.

The "shared" across all processes is a bit of a hack, because you can't know
exactly which pages are shared across every single process.  We do a
heuristic.


>
>
> If so, is there any command-line switch that will output those summary
> stats to console on a regular basis?


You can use --enable-logging --log-level=0  - each time you refresh the
about:memory page, it will drop a line to the log file.  This can be useful
for plots over time.


>
>
>
>> If someone measures the "Private Bytes" counter, which uses the
>> PROCESS_MEMORY_COUNTERS_EX.PrivateUsage structure, that seems to map to the
>> Task Manager "Commit Size" which isn't the thing I believe we want to
>> measure.
>>
>> Right.
>>
>
> Interestingly, as I watched this value change while webpages were loading,
> it tracked the same pattern of growth/decline as "Memory (Private Working
> Set)" in the Task Manager, though the values were usually about 2x or so
> more. I suppose this is due to the heap sharing you were speaking of
> earlier?


I'm not quite sure what you mean.

But of course, as you load more tabs, you'll see the memory use go up.


>
>
>  The "Working Set - Private" counter doesn't seem to have a structure
>> according to the MSDN document; that's what maps to the "Memory (Private
>> Working Set)" column in the TaskManager.
>>
>> Right, I think you have to use QueryWorkingSet, walk the pages and
>> categorize them yourself.
>>
>
> OK, I can look into trying that. Though I'm wondering if it's worth the
> bother, as the meta-pattern, to me, is more interesting than the precise
> megabyte count.


For a single process browser, it's not worth the effort; I think it's the
only way to know how to account for shared memory.


>
>
>
>> The closest thing I can find is the "Working Set" counter, which uses the
>> PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure and shows up in the
>> Vista Task Manager as "Working Set (Memory)"
>>
>> For multi-proc browsers like chrome, this will way overstate RAM; there is
>> a good 5-6MB of shared working set in each process.  So for 10 tabs, you'd
>> could an extra 50MB for Chrome if you do it this way.
>>
>
> Looking both in Task Manager and about:memory, when I have 30 tabs open I'm
> not seeing 30 processes. Are you sure you're right about this point?
>

You don't always get a new process for every tab.  If two tabs are connected
via javascript, then they'll be in the same process (the about:memory shows
which tabs are in the same process).  So, clicking a link, for example, will
open in the same tab, but typing the URL in the omnibox will create a new
process.  Others could tell you more about the exact policy for when you get
a new process and when you don't.



>
>
>  If you come up with a better way, please let me know!
>>
>
> Well, I can tell you what we've done and what we're doing, just trying to
> get confidence on various metrics. We've replicated the "membuster" test we
> used around the release of Firefox 3 (see
> http://blog.pavlov.net/2008/03/11/firefox-3-memory-usage/ ) as a way of
> measuring Firefox's ability to hold a "steady" state of memory across a
> browsing session (ie: not leak) and then release that memory when the
> session ends (ie: not bogart system resources). The test:
>
>  - runs on Windows 7, using Python and our Standalone Talos code to make
> measurements
>  - uses a local web proxy to ensure we're always viewing the same web
> content
>  - uses the JS at view-source:
> http://random.pavlov.net/membuster/index.html to:
>  --- open 30 pages using window.open() calls
>  --- load ~300 pages, opening and closing windows with each load (always 30
> open)
>  --- close all 30 windows when done cycling


OK - I think this might basically use one renderer process in chrome?
 Because of the new-process creation policy, it may not be representative of
real world usage.  Darin?


> The whole while, we measure the amount of memory taken using the
> PROCESS_MEMORY_COUNTERS structure, summating over processes when multiple
> exist (as they do in the case of Internet Explorer 8 and Chrome 2)


Ok - that will double count shared memory.  I'd estimate 3-5MB per process.


> The results can be seen here, using both the PrivateUsage (shows as "Commit
> Size" in Windows 7 Task Manager) and WorkingSet (shows as "Working Set
> (Memory)" in Windows 7 Task Manager) counters:
>
> http://p

[chromium-dev] Re: Memory usage in chrome

[Mike Beltzner]

On 25-Jun-09, at 6:26 PM, Mike Beltzner wrote:

> here's a ZIP with the
> required code (needs Python 2.5 or later to be installed on your
> system):

Oops, forgot the link!

http://people.mozilla.org/~beltzner/membuster-talos.rar

cheers,
mike

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Beltzner]

On 25-Jun-09, at 12:52 PM, Mike Belshe wrote:

> Yeah, the APIs all have constraints.  We end up walking the pages  
> and adding them up.  See process_util_win.cc in the chromium tree.   
> Be sure to check about:memory and hover over the little "?" icons to  
> see what we measure.

This screen actually confuses me a little, as the Summary statistics  
don't match the summation of the process based statistics. Do you mean  
to say your summary statistics take into account the memory that's  
being shared across the various processes?

If so, is there any command-line switch that will output those summary  
stats to console on a regular basis?

>
> If someone measures the "Private Bytes" counter, which uses the  
> PROCESS_MEMORY_COUNTERS_EX.PrivateUsage structure, that seems to map  
> to the Task Manager "Commit Size" which isn't the thing I believe we  
> want to measure.
>
> Right.

Interestingly, as I watched this value change while webpages were  
loading, it tracked the same pattern of growth/decline as "Memory  
(Private Working Set)" in the Task Manager, though the values were  
usually about 2x or so more. I suppose this is due to the heap sharing  
you were speaking of earlier?

> The "Working Set - Private" counter doesn't seem to have a structure  
> according to the MSDN document; that's what maps to the "Memory  
> (Private Working Set)" column in the TaskManager.
>
> Right, I think you have to use QueryWorkingSet, walk the pages and  
> categorize them yourself.

OK, I can look into trying that. Though I'm wondering if it's worth  
the bother, as the meta-pattern, to me, is more interesting than the  
precise megabyte count.

>
> The closest thing I can find is the "Working Set" counter, which  
> uses the PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure and  
> shows up in the Vista Task Manager as "Working Set (Memory)"
>
> For multi-proc browsers like chrome, this will way overstate RAM;  
> there is a good 5-6MB of shared working set in each process.  So for  
> 10 tabs, you'd could an extra 50MB for Chrome if you do it this way.

Looking both in Task Manager and about:memory, when I have 30 tabs  
open I'm not seeing 30 processes. Are you sure you're right about this  
point?

> If you come up with a better way, please let me know!

Well, I can tell you what we've done and what we're doing, just trying  
to get confidence on various metrics. We've replicated the "membuster"  
test we used around the release of Firefox 3 (see 
http://blog.pavlov.net/2008/03/11/firefox-3-memory-usage/ 
  ) as a way of measuring Firefox's ability to hold a "steady" state  
of memory across a browsing session (ie: not leak) and then release  
that memory when the session ends (ie: not bogart system resources).  
The test:

  - runs on Windows 7, using Python and our Standalone Talos code to  
make measurements
  - uses a local web proxy to ensure we're always viewing the same web  
content
  - uses the JS at view-source:http://random.pavlov.net/membuster/index.html 
  to:
  --- open 30 pages using window.open() calls
  --- load ~300 pages, opening and closing windows with each load  
(always 30 open)
  --- close all 30 windows when done cycling

The whole while, we measure the amount of memory taken using the  
PROCESS_MEMORY_COUNTERS structure, summating over processes when  
multiple exist (as they do in the case of Internet Explorer 8 and  
Chrome 2)

The results can be seen here, using both the PrivateUsage (shows as  
"Commit Size" in Windows 7 Task Manager) and WorkingSet (shows as  
"Working Set (Memory)" in Windows 7 Task Manager) counters:

http://people.mozilla.com/~beltzner/images/private-usage.png
http://people.mozilla.com/~beltzner/images/working-set.png

The datapoint we wanted to get out of this was primarily about Firefox  
3.5 vs Firefox 3.0.11, and as you can see we actually take slightly  
(7-10MB) more memory during the page cycling, but manage to release  
more when done. Safari had previously been unable to complete this  
test, but now does.

I'd love to get your feedback on how we can improve our recording  
here. I think we're going to try and bundle together the test and  
files into an easier-to-use-tool than our delicately cobbled together  
solution, but if you wanted to try it yourself, here's a ZIP with the  
required code (needs Python 2.5 or later to be installed on your  
system):

Once unzipped, enter the directory and run the following commands:

\path\to\python\python.exe proxyserver.py -v -l -u proxy-cache.db
\path\to\python\python.exe measure_any_windows_app.py > \path\to 
\resultsfile\resultsfile.txt

You'll want to edit measure_any_windows_app.py to have the correct  
paths to the various applications, and you'll also want to make sure  
your browsers are set to use 127.0.0.1 port 8000 as a proxy (under  
Advanced > Network > Settings in Firefox, Internet Properties >  
Connections > LAN Settings for Chrome/IE/Safari)

(Thanks to David Dahl for modifying the re

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

On Thu, Jun 25, 2009 at 9:40 AM, Mike Beltzner  wrote:

> On 22-Jun-09, at 12:57 PM, Mike Belshe wrote:
>
>  Yes, that accurately represents the private memory for a process, but it
>> doesn't reflect the user's experience.  Windows generally tracks working
>> set.  Why?  Because the working set is the amount of memory *not available
>> to other apps*.  If other apps can have the memory, then using the bytes is
>> inconsequential.
>>
>
> Figuring out how to track this is a bit of a terminology mess ;) Referring
> to the following document:
>
> http://msdn.microsoft.com/en-us/library/aa965225(VS.85).aspx


Yeah, the APIs all have constraints.  We end up walking the pages and adding
them up.  See process_util_win.cc in the chromium tree.  Be sure to check
about:memory and hover over the little "?" icons to see what we measure.



>
> If someone measures the "Private Bytes" counter, which uses the
> PROCESS_MEMORY_COUNTERS_EX.PrivateUsage structure, that seems to map to the
> Task Manager "Commit Size" which isn't the thing I believe we want to
> measure.


Right.


>
>
> The "Working Set - Private" counter doesn't seem to have a structure
> according to the MSDN document; that's what maps to the "Memory (Private
> Working Set)" column in the TaskManager.


Right, I think you have to use QueryWorkingSet, walk the pages and
categorize them yourself.


>
>
> The closest thing I can find is the "Working Set" counter, which uses the
> PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure and shows up in the
> Vista Task Manager as "Working Set (Memory)"


For multi-proc browsers like chrome, this will way overstate RAM; there is a
good 5-6MB of shared working set in each process.  So for 10 tabs, you'd
could an extra 50MB for Chrome if you do it this way.

I wish it were easier too.  In Vista's task manager, the primary metric is
"Memory (Private Working Set)", and I believe the only way to get this
number is to call QueryWorkingSet, walk the pages, and add up the ones which
are marked as private.

If you come up with a better way, please let me know!

Mike



>
>
> Would you agree that summating the data from the
> PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure would give a fair
> indication of memory usage?
>
> cheers,
> mike
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Beltzner]

On 22-Jun-09, at 12:57 PM, Mike Belshe wrote:

> Yes, that accurately represents the private memory for a process,  
> but it doesn't reflect the user's experience.  Windows generally  
> tracks working set.  Why?  Because the working set is the amount of  
> memory *not available to other apps*.  If other apps can have the  
> memory, then using the bytes is inconsequential.

Figuring out how to track this is a bit of a terminology mess ;)  
Referring to the following document:

http://msdn.microsoft.com/en-us/library/aa965225(VS.85).aspx

If someone measures the "Private Bytes" counter, which uses the  
PROCESS_MEMORY_COUNTERS_EX.PrivateUsage structure, that seems to map  
to the Task Manager "Commit Size" which isn't the thing I believe we  
want to measure.

The "Working Set - Private" counter doesn't seem to have a structure  
according to the MSDN document; that's what maps to the "Memory  
(Private Working Set)" column in the TaskManager.

The closest thing I can find is the "Working Set" counter, which uses  
the PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure and shows up  
in the Vista Task Manager as "Working Set (Memory)"

Would you agree that summating the data from the  
PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure would give a fair  
indication of memory usage?

cheers,
mike

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[PhistucK]

I am willing to volunteer to be a test case if you need one, tell me what to
do so you will be able to monitor.Before I added that switch, it was pretty
horrible.

Though I am building stuff with Java, XSLT, JavaScript that process a lot of
files (HTMLs) and viewing and refreshing a lot of pages (in Chrome, mostly,
but sometimes in other browsers for cross browser support), plus Eclipse and
Outlook, if it matters.

☆PhistucK


On Wed, Jun 24, 2009 at 19:46, Mike Belshe  wrote:

>
>
> On Wed, Jun 24, 2009 at 9:39 AM, Ian Fette  wrote:
>
>> +1. Most people are not doing compiles, we're trying to say that people
>> live in the web and in their browser, and that their browser is the primary
>> application. For me at least, that is true. The browser is the app I use the
>> most -- the only other app I use regularly is an ssh client, which can
>> usually fit in memory or get swapped back in much quicker than Chrome.
>
>
>
> This doesn't make sense as a use case, because if there is nothing else
> eating your memory, then the memory doesn't have to "swap in".  It's already
> there! :-)  For users with plenty of RAM like you, memory-model=medium
> should work just fine.
>
> The only people that might be effected by this is people that do have
> something else competing for the memory (like a compile, or they are on a
> low-memory box and outlook is eating it).
>
>  I do worry that virus scanners could churn through memory causing similar
> effects, but again, we measured for this and so far have been unable to
> detect any difference.
>
> At this point we could change to memory-model=high, despite having no
> real-world data that this is a problem.  Chrome would be perceived to use
> 25% more memory, and the folks on this thread that have compiles going might
> feel a better experience.  If we don't care about a 25% jump in memory use
> then we should switch the default even though data suggests it won't help.
>
> One more possibility:  maybe there is a bug we haven't yet identified.
>
> Mike
>
>
>
>>
>>
>> 2009/6/23 Peter Kasting 
>>
>>>  FWIW, I strongly believe we should move the default to
>>> --memory-model=high.  This is what pretty much every other app in the world
>>> does, and we mostly penalize ourselves when the OS aggressively swaps us out
>>> for a dumb reason (which yes, Windows does do).
>>> We have a lot of complaints of "I came back the next hour/day/whatever
>>> and everything was unresponsive".  I don't think our current tradeoff is the
>>> right one.
>>>
>>> I know Mike wants to be a good citizen and feels like if the OS swapped
>>> you out it really needed that RAM, but in my own observations of my machine
>>> the OS swaps for retarded reasons and I gain nothing but headaches.
>>>
>>> PK
>>>
>>> >>>
>>>
>>
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

On Wed, Jun 24, 2009 at 9:39 AM, Ian Fette  wrote:

> +1. Most people are not doing compiles, we're trying to say that people
> live in the web and in their browser, and that their browser is the primary
> application. For me at least, that is true. The browser is the app I use the
> most -- the only other app I use regularly is an ssh client, which can
> usually fit in memory or get swapped back in much quicker than Chrome.



This doesn't make sense as a use case, because if there is nothing else
eating your memory, then the memory doesn't have to "swap in".  It's already
there! :-)  For users with plenty of RAM like you, memory-model=medium
should work just fine.

The only people that might be effected by this is people that do have
something else competing for the memory (like a compile, or they are on a
low-memory box and outlook is eating it).

I do worry that virus scanners could churn through memory causing similar
effects, but again, we measured for this and so far have been unable to
detect any difference.

At this point we could change to memory-model=high, despite having no
real-world data that this is a problem.  Chrome would be perceived to use
25% more memory, and the folks on this thread that have compiles going might
feel a better experience.  If we don't care about a 25% jump in memory use
then we should switch the default even though data suggests it won't help.

One more possibility:  maybe there is a bug we haven't yet identified.

Mike



>
>
> 2009/6/23 Peter Kasting 
>
>> FWIW, I strongly believe we should move the default to
>> --memory-model=high.  This is what pretty much every other app in the world
>> does, and we mostly penalize ourselves when the OS aggressively swaps us out
>> for a dumb reason (which yes, Windows does do).
>> We have a lot of complaints of "I came back the next hour/day/whatever and
>> everything was unresponsive".  I don't think our current tradeoff is the
>> right one.
>>
>> I know Mike wants to be a good citizen and feels like if the OS swapped
>> you out it really needed that RAM, but in my own observations of my machine
>> the OS swaps for retarded reasons and I gain nothing but headaches.
>>
>> PK
>>
>> >>
>>
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Elliot Glaysher (Chromium)]

Could we stat at memory-model=hight and then change our memory model
mid-flight if there are any large, non-chrome, memory hungry
processes?

-- Elliot

On Wed, Jun 24, 2009 at 9:39 AM, Ian Fette wrote:
> +1. Most people are not doing compiles, we're trying to say that people live
> in the web and in their browser, and that their browser is the primary
> application. For me at least, that is true. The browser is the app I use the
> most -- the only other app I use regularly is an ssh client, which can
> usually fit in memory or get swapped back in much quicker than Chrome.
>
> 2009/6/23 Peter Kasting 
>>
>> FWIW, I strongly believe we should move the default to
>> --memory-model=high.  This is what pretty much every other app in the world
>> does, and we mostly penalize ourselves when the OS aggressively swaps us out
>> for a dumb reason (which yes, Windows does do).
>> We have a lot of complaints of "I came back the next hour/day/whatever and
>> everything was unresponsive".  I don't think our current tradeoff is the
>> right one.
>> I know Mike wants to be a good citizen and feels like if the OS swapped
>> you out it really needed that RAM, but in my own observations of my machine
>> the OS swaps for retarded reasons and I gain nothing but headaches.
>> PK
>>
>
>
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Ian Fette]

+1. Most people are not doing compiles, we're trying to say that people live
in the web and in their browser, and that their browser is the primary
application. For me at least, that is true. The browser is the app I use the
most -- the only other app I use regularly is an ssh client, which can
usually fit in memory or get swapped back in much quicker than Chrome.

2009/6/23 Peter Kasting 

> FWIW, I strongly believe we should move the default to --memory-model=high.
>  This is what pretty much every other app in the world does, and we mostly
> penalize ourselves when the OS aggressively swaps us out for a dumb reason
> (which yes, Windows does do).
> We have a lot of complaints of "I came back the next hour/day/whatever and
> everything was unresponsive".  I don't think our current tradeoff is the
> right one.
>
> I know Mike wants to be a good citizen and feels like if the OS swapped you
> out it really needed that RAM, but in my own observations of my machine the
> OS swaps for retarded reasons and I gain nothing but headaches.
>
> PK
>
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

Anecdotally, a couple of people have said "it works" and a couple of people
have said "it makes no difference".  I do believe that people doing compiles
could see a difference.

To determine if it was real, we did an experiment with memory model high and
memory model medium and measured the whiteout duration.  We did not see any
statistical difference between the two.  It could be that that was not a
good test, but we don't know of a better test.

If we have a better test, let's do an A/B test and find out the answer to
this puzzling question.  We also know that memory model high on real world
users uses about 25% more RAM.

Another option is to be less aggressive on the memory model medium.

Mike



On Tue, Jun 23, 2009 at 11:34 PM, Peter Kasting wrote:

> FWIW, I strongly believe we should move the default to --memory-model=high.
>  This is what pretty much every other app in the world does, and we mostly
> penalize ourselves when the OS aggressively swaps us out for a dumb reason
> (which yes, Windows does do).
> We have a lot of complaints of "I came back the next hour/day/whatever and
> everything was unresponsive".  I don't think our current tradeoff is the
> right one.
>
> I know Mike wants to be a good citizen and feels like if the OS swapped you
> out it really needed that RAM, but in my own observations of my machine the
> OS swaps for retarded reasons and I gain nothing but headaches.
>
> PK
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Peter Kasting]

FWIW, I strongly believe we should move the default to --memory-model=high.
 This is what pretty much every other app in the world does, and we mostly
penalize ourselves when the OS aggressively swaps us out for a dumb reason
(which yes, Windows does do).
We have a lot of complaints of "I came back the next hour/day/whatever and
everything was unresponsive".  I don't think our current tradeoff is the
right one.

I know Mike wants to be a good citizen and feels like if the OS swapped you
out it really needed that RAM, but in my own observations of my machine the
OS swaps for retarded reasons and I gain nothing but headaches.

PK

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[PhistucK]

Wow, that changed my whole browsing experience.I got back to the computer
after it has been on all of the night, with Chrome (and others) running and
when I got back to Chrome, I did not even have to wait for a bit!!!
Thank you very much for pointing it out. It was quite frustrating before
(but I liked everything else about Chrome, so I kept on using it).

What are the implications, though?
Not giving up the space to other applications and that is it?

☆PhistucK


2009/6/23 Dean McNamee 

> Have you tried running with --memory-model=high ?
>
> 2009/6/23 PhistucK :
> > This explanation actually shows me the source of this serious jank (I
> hope I
> > am using the term in the right context) I am having all of the time.
> > I am getting back to Chrome after a few minutes of dealing with some
> other
> > application and I have to wait, sometimes even for twenty seconds or
> more,
> > until I get the control back on the tab contents.
> > Sometimes I am not using a tab for a few minutes (or more) and when I
> switch
> > back to it, it takes it twenty seconds or more until I get the control
> back
> > of the tab contents.
> > :(
> >
> > ☆PhistucK
> >
> >
> > On Mon, Jun 22, 2009 at 19:57, Mike Belshe  wrote:
> >>
> >>
> >> On Mon, Jun 22, 2009 at 9:29 AM, Mike Beltzner 
> >> wrote:
> >>>
> >>> On 21-Jun-09, at 10:22 AM, Mike Belshe wrote:
> >>>
>  Second, the author is basically right.  Since he's running on Vista,
> its
>  a bit hard to tell whether his stats included shared memory or not;
> using
>  the default memory statistic ("Memory (Private Working Set)") is
> actually a
>  pretty good measure to just sum.  But he doesn't say which measurement
> he
>  used.
> >>>
> >>> Doesn't Private Bytes accurately represent the private memory for a
> given
> >>> process? I thought the whole point of that was that it didn't measure
> any
> >>> shared memory pools.
> >>
> >> Yes, that accurately represents the private memory for a process, but it
> >> doesn't reflect the user's experience.  Windows generally tracks working
> >> set.  Why?  Because the working set is the amount of memory *not
> available
> >> to other apps*.  If other apps can have the memory, then using the bytes
> is
> >> inconsequential.
> >> For most applications, there isn't much difference between private bytes
> >> and working set private bytes.  However, because of Chrome's multi-proc
> >> architecture, there is a big difference.  The reason is because
> >> Chrome intentionally gives memory back to the OS.  For instance, on my
> >> current instance of Chrome, I'm using 16 tabs.  The sum of the private
> bytes
> >> is 514408.  The sum of the private working set bytes is 275040, nearly
> half
> >> of the private bytes number.  This is on a machine with 8GB of RAM, so
> there
> >> is plenty of memory to go around.  But if some other app wants the
> memory,
> >> Chrome gave it back to the OS and will suffer the page faults to get it
> >> back.  Since Chrome has given it back to the OS (and has volunteered to
> take
> >> the performance consequences of getting it back), I don't think it
> should be
> >> counted as Chrome usage.  Others may disagree. But Windows uses working
> set
> >> as the primary metric for all applications the OS folks agree that
> working
> >> set is the right way to account for memory usage.
> >> Single process browsers have a hard time giving memory back, because
> they
> >> can't differentiate which pages are accounted to unused, background tabs
> and
> >> which pages are accounted to the active, in-use tabs.
> >> Finally, the common metric which definitely doesn't work well is Windows
> >> XP's default metric:  working set (private + shared).  Because of shared
> >> memory between processes, simply summing the working set will far
> overstate
> >> the actual RAM used.  Part of the motivation with Vista changing the
> default
> >> metric from working set to private working set was precisely to deal
> with
> >> the issue of better accounting of shared memory.
> >> Mike
> >>
> >>
> >>
> >
> >
> > > >
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[cpu]

Mike, yes we (I) increased the number of renderer processes for
machines with lots of ram. I think it tops now to 40 processes.

Our previous limit was not based on calculation but because we had
WaitForMultipleObjects(..) which has a 64 objects maximum and we had 2
objects per process so our limit was 31 so to give us some slack we
settled on 20

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[PhistucK]

No. I added it now and I will post the results later.
Thank you!
I really hope it will help...

☆PhistucK


2009/6/23 Dean McNamee 

> Have you tried running with --memory-model=high ?
>
> 2009/6/23 PhistucK :
> > This explanation actually shows me the source of this serious jank (I
> hope I
> > am using the term in the right context) I am having all of the time.
> > I am getting back to Chrome after a few minutes of dealing with some
> other
> > application and I have to wait, sometimes even for twenty seconds or
> more,
> > until I get the control back on the tab contents.
> > Sometimes I am not using a tab for a few minutes (or more) and when I
> switch
> > back to it, it takes it twenty seconds or more until I get the control
> back
> > of the tab contents.
> > :(
> >
> > ☆PhistucK
> >
> >
> > On Mon, Jun 22, 2009 at 19:57, Mike Belshe  wrote:
> >>
> >>
> >> On Mon, Jun 22, 2009 at 9:29 AM, Mike Beltzner 
> >> wrote:
> >>>
> >>> On 21-Jun-09, at 10:22 AM, Mike Belshe wrote:
> >>>
>  Second, the author is basically right.  Since he's running on Vista,
> its
>  a bit hard to tell whether his stats included shared memory or not;
> using
>  the default memory statistic ("Memory (Private Working Set)") is
> actually a
>  pretty good measure to just sum.  But he doesn't say which measurement
> he
>  used.
> >>>
> >>> Doesn't Private Bytes accurately represent the private memory for a
> given
> >>> process? I thought the whole point of that was that it didn't measure
> any
> >>> shared memory pools.
> >>
> >> Yes, that accurately represents the private memory for a process, but it
> >> doesn't reflect the user's experience.  Windows generally tracks working
> >> set.  Why?  Because the working set is the amount of memory *not
> available
> >> to other apps*.  If other apps can have the memory, then using the bytes
> is
> >> inconsequential.
> >> For most applications, there isn't much difference between private bytes
> >> and working set private bytes.  However, because of Chrome's multi-proc
> >> architecture, there is a big difference.  The reason is because
> >> Chrome intentionally gives memory back to the OS.  For instance, on my
> >> current instance of Chrome, I'm using 16 tabs.  The sum of the private
> bytes
> >> is 514408.  The sum of the private working set bytes is 275040, nearly
> half
> >> of the private bytes number.  This is on a machine with 8GB of RAM, so
> there
> >> is plenty of memory to go around.  But if some other app wants the
> memory,
> >> Chrome gave it back to the OS and will suffer the page faults to get it
> >> back.  Since Chrome has given it back to the OS (and has volunteered to
> take
> >> the performance consequences of getting it back), I don't think it
> should be
> >> counted as Chrome usage.  Others may disagree. But Windows uses working
> set
> >> as the primary metric for all applications the OS folks agree that
> working
> >> set is the right way to account for memory usage.
> >> Single process browsers have a hard time giving memory back, because
> they
> >> can't differentiate which pages are accounted to unused, background tabs
> and
> >> which pages are accounted to the active, in-use tabs.
> >> Finally, the common metric which definitely doesn't work well is Windows
> >> XP's default metric:  working set (private + shared).  Because of shared
> >> memory between processes, simply summing the working set will far
> overstate
> >> the actual RAM used.  Part of the motivation with Vista changing the
> default
> >> metric from working set to private working set was precisely to deal
> with
> >> the issue of better accounting of shared memory.
> >> Mike
> >>
> >>
> >>
> >
> >
> > > >
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Dean McNamee]

Have you tried running with --memory-model=high ?

2009/6/23 PhistucK :
> This explanation actually shows me the source of this serious jank (I hope I
> am using the term in the right context) I am having all of the time.
> I am getting back to Chrome after a few minutes of dealing with some other
> application and I have to wait, sometimes even for twenty seconds or more,
> until I get the control back on the tab contents.
> Sometimes I am not using a tab for a few minutes (or more) and when I switch
> back to it, it takes it twenty seconds or more until I get the control back
> of the tab contents.
> :(
>
> ☆PhistucK
>
>
> On Mon, Jun 22, 2009 at 19:57, Mike Belshe  wrote:
>>
>>
>> On Mon, Jun 22, 2009 at 9:29 AM, Mike Beltzner 
>> wrote:
>>>
>>> On 21-Jun-09, at 10:22 AM, Mike Belshe wrote:
>>>
 Second, the author is basically right.  Since he's running on Vista, its
 a bit hard to tell whether his stats included shared memory or not; using
 the default memory statistic ("Memory (Private Working Set)") is actually a
 pretty good measure to just sum.  But he doesn't say which measurement he
 used.
>>>
>>> Doesn't Private Bytes accurately represent the private memory for a given
>>> process? I thought the whole point of that was that it didn't measure any
>>> shared memory pools.
>>
>> Yes, that accurately represents the private memory for a process, but it
>> doesn't reflect the user's experience.  Windows generally tracks working
>> set.  Why?  Because the working set is the amount of memory *not available
>> to other apps*.  If other apps can have the memory, then using the bytes is
>> inconsequential.
>> For most applications, there isn't much difference between private bytes
>> and working set private bytes.  However, because of Chrome's multi-proc
>> architecture, there is a big difference.  The reason is because
>> Chrome intentionally gives memory back to the OS.  For instance, on my
>> current instance of Chrome, I'm using 16 tabs.  The sum of the private bytes
>> is 514408.  The sum of the private working set bytes is 275040, nearly half
>> of the private bytes number.  This is on a machine with 8GB of RAM, so there
>> is plenty of memory to go around.  But if some other app wants the memory,
>> Chrome gave it back to the OS and will suffer the page faults to get it
>> back.  Since Chrome has given it back to the OS (and has volunteered to take
>> the performance consequences of getting it back), I don't think it should be
>> counted as Chrome usage.  Others may disagree. But Windows uses working set
>> as the primary metric for all applications the OS folks agree that working
>> set is the right way to account for memory usage.
>> Single process browsers have a hard time giving memory back, because they
>> can't differentiate which pages are accounted to unused, background tabs and
>> which pages are accounted to the active, in-use tabs.
>> Finally, the common metric which definitely doesn't work well is Windows
>> XP's default metric:  working set (private + shared).  Because of shared
>> memory between processes, simply summing the working set will far overstate
>> the actual RAM used.  Part of the motivation with Vista changing the default
>> metric from working set to private working set was precisely to deal with
>> the issue of better accounting of shared memory.
>> Mike
>>
>>
>>
>
>
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[PhistucK]

This explanation actually shows me the source of this serious jank (I hope I
am using the term in the right context) I am having all of the time.I am
getting back to Chrome after a few minutes of dealing with some other
application and I have to wait, sometimes even for twenty seconds or more,
until I get the control back on the tab contents.
Sometimes I am not using a tab for a few minutes (or more) and when I switch
back to it, it takes it twenty seconds or more until I get the control back
of the tab contents.
:(


☆PhistucK


On Mon, Jun 22, 2009 at 19:57, Mike Belshe  wrote:

>
>
> On Mon, Jun 22, 2009 at 9:29 AM, Mike Beltzner wrote:
>
>> On 21-Jun-09, at 10:22 AM, Mike Belshe wrote:
>>
>>  Second, the author is basically right.  Since he's running on Vista, its
>>> a bit hard to tell whether his stats included shared memory or not; using
>>> the default memory statistic ("Memory (Private Working Set)") is actually a
>>> pretty good measure to just sum.  But he doesn't say which measurement he
>>> used.
>>>
>>
>> Doesn't Private Bytes accurately represent the private memory for a given
>> process? I thought the whole point of that was that it didn't measure any
>> shared memory pools.
>
>
> Yes, that accurately represents the private memory for a process, but it
> doesn't reflect the user's experience.  Windows generally tracks working
> set.  Why?  Because the working set is the amount of memory *not available
> to other apps*.  If other apps can have the memory, then using the bytes is
> inconsequential.
>
> For most applications, there isn't much difference between private bytes
> and working set private bytes.  However, because of Chrome's multi-proc
> architecture, there is a big difference.  The reason is because
> Chrome intentionally gives memory back to the OS.  For instance, on my
> current instance of Chrome, I'm using 16 tabs.  The sum of the private bytes
> is 514408.  The sum of the private working set bytes is 275040, nearly half
> of the private bytes number.  This is on a machine with 8GB of RAM, so there
> is plenty of memory to go around.  But if some other app wants the memory,
> Chrome gave it back to the OS and will suffer the page faults to get it
> back.  Since Chrome has given it back to the OS (and has volunteered to take
> the performance consequences of getting it back), I don't think it should be
> counted as Chrome usage.  Others may disagree. But Windows uses working set
> as the primary metric for all applications the OS folks agree that working
> set is the right way to account for memory usage.
>
> Single process browsers have a hard time giving memory back, because they
> can't differentiate which pages are accounted to unused, background tabs and
> which pages are accounted to the active, in-use tabs.
>
> Finally, the common metric which definitely doesn't work well is Windows
> XP's default metric:  working set (private + shared).  Because of shared
> memory between processes, simply summing the working set will far overstate
> the actual RAM used.  Part of the motivation with Vista changing the default
> metric from working set to private working set was precisely to deal with
> the issue of better accounting of shared memory.
>
> Mike
>
>
>
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

On Mon, Jun 22, 2009 at 9:29 AM, Mike Beltzner  wrote:

> On 21-Jun-09, at 10:22 AM, Mike Belshe wrote:
>
>  Second, the author is basically right.  Since he's running on Vista, its a
>> bit hard to tell whether his stats included shared memory or not; using the
>> default memory statistic ("Memory (Private Working Set)") is actually a
>> pretty good measure to just sum.  But he doesn't say which measurement he
>> used.
>>
>
> Doesn't Private Bytes accurately represent the private memory for a given
> process? I thought the whole point of that was that it didn't measure any
> shared memory pools.


Yes, that accurately represents the private memory for a process, but it
doesn't reflect the user's experience.  Windows generally tracks working
set.  Why?  Because the working set is the amount of memory *not available
to other apps*.  If other apps can have the memory, then using the bytes is
inconsequential.

For most applications, there isn't much difference between private bytes and
working set private bytes.  However, because of Chrome's multi-proc
architecture, there is a big difference.  The reason is because
Chrome intentionally gives memory back to the OS.  For instance, on my
current instance of Chrome, I'm using 16 tabs.  The sum of the private bytes
is 514408.  The sum of the private working set bytes is 275040, nearly half
of the private bytes number.  This is on a machine with 8GB of RAM, so there
is plenty of memory to go around.  But if some other app wants the memory,
Chrome gave it back to the OS and will suffer the page faults to get it
back.  Since Chrome has given it back to the OS (and has volunteered to take
the performance consequences of getting it back), I don't think it should be
counted as Chrome usage.  Others may disagree. But Windows uses working set
as the primary metric for all applications the OS folks agree that working
set is the right way to account for memory usage.

Single process browsers have a hard time giving memory back, because they
can't differentiate which pages are accounted to unused, background tabs and
which pages are accounted to the active, in-use tabs.

Finally, the common metric which definitely doesn't work well is Windows
XP's default metric:  working set (private + shared).  Because of shared
memory between processes, simply summing the working set will far overstate
the actual RAM used.  Part of the motivation with Vista changing the default
metric from working set to private working set was precisely to deal with
the issue of better accounting of shared memory.

Mike

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Beltzner]

On 21-Jun-09, at 10:22 AM, Mike Belshe wrote:

> Second, the author is basically right.  Since he's running on Vista,  
> its a bit hard to tell whether his stats included shared memory or  
> not; using the default memory statistic ("Memory (Private Working  
> Set)") is actually a pretty good measure to just sum.  But he  
> doesn't say which measurement he used.

Doesn't Private Bytes accurately represent the private memory for a  
given process? I thought the whole point of that was that it didn't  
measure any shared memory pools.

cheers,
mike

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[bush]

I built a web based application in php to upload dvd vob files to a
server via the browser. The only browser that makes this project a non
failure is chrome. Forget trying to upload 4+ gigs with Internet
Explorer. It can not handle the memory addressing. Firefox fails 90%
of the time. However chrome always gets the 4 gigs over the web
interface.

Chrome, an actual browser that doesnt need compatibility buttons, and
also can handle moving 4+gigs with the browser. Chrome is kick ass.

On Jun 22, 12:21 am, PhistucK  wrote:
> Really? the statistics show that many people are using the app mode?Or did
> you mean web apps, as in web application websites?
>
> ☆PhistucK
>
> On Sun, Jun 21, 2009 at 21:03, Mike Belshe  wrote:
> > I  assume he's not a benchmark pro, but he did a decent job already.  We
> > can nitpick his sampling methodology - but it won't change the result.  He
> > is correct that many procs is far more memory consuming than single proc,
> > and we already knew this.
> > This is a tradeoff we made consciously and deliberately.  When firefox
> > crashes, all tabs go down.  When firefox memory is compromised (security),
> > all tabs are compromised.  In chrome, we don't have those problems, but
> > instead use more RAM.  Further, Chrome is also able to implement per-tab
> > prioritization, so that background tabs don't make foreground tabs go slow.
> >  Firefox can't do that.
> > Lastly, lets bring the test back to reality.  People don't visit 150 random
> > home pages.  They may have 20-30 tabs open, but many are applications, with
> > cookies, javascript state and much more than just the "home page".  When
> > apps are in use, the memory gap between chrome and FF shrinks a lot.
>
> > Mike
>
> > On Sun, Jun 21, 2009 at 10:46 AM, Dan Kegel  wrote:
>
> >> On Sun, Jun 21, 2009 at 10:22 AM, Mike Belshe wrote:
> >> > First off - kudos to the author for posting the source and steps to
> >> > reproduce!  Most don't do that!
>
> >> > Second, the author is basically right.  Since he's running on Vista, its
> >> a
> >> > bit hard to tell whether his stats included shared memory or not; using
> >> the
> >> > default memory statistic ("Memory (Private Working Set)") is actually a
> >> > pretty good measure to just sum.  But he doesn't say which measurement
> >> he
> >> > used.
>
> >> Wait, why doesn't his program itself do the summing?
> >> (I don't see it in there.)
> >> Wouldn't that get rid of the ambiguity?
> >> How hard would it be to add that and repost?
> >> - Dan

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Toby DiPasquale]

On Jun 21, 3:37 am, n179911  wrote:
> Hi,
>
> There is a test which compares memory usage among rendering 
> engineshttp://dotnetperls.com/chrome-memory
>
> From the site, it shows the maximum memory usage of Chrome is more
> than Safari is > 2 times.
> Since both of them are Webkit base, does that mean the V8 engine uses
> twice as much memory as squirrel fish?
>
> --- Maximum memory used ---
>     Peak memory usage measured during experiment.
>
> Chrome:  1216.16 MB      [Largest]
> Firefox:  327.65 MB      [Smallest]
> Opera:    554.11 MB
> Safari:   517.00 MB

I find these results somewhat disingenuous. I left both Firefox and
Chrome open over the weekend (both on Linux, with the Chromium daily
build), Firefox with 1 tab, Chrome with 4 (one of them being Gmail).
Chrome's memory usage didn't move while I had to kill Firefox for
eating up 1.5GB of RAM and being generally unresponsive.

Maybe Chrome uses more memory to do its job initially, but it
certainly doesn't have the leaks that Firefox does which cause me to
have to restart it several times a day. Keep up the awesome work,
Chrome peeps. I can't wait until I'm using Chrome as my default
browser on all my platforms.

--
Toby DiPasquale

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

2009/6/21 PhistucK 

> Really? the statistics show that many people are using the app mode?Or did
> you mean web apps, as in web application websites?
>

I mean web applications like gmail, hotmail, zoho, shopping carts, etc etc.
 But it's an unsubstantiated claim.

Mike





>
> ☆PhistucK
>
>
>
> On Sun, Jun 21, 2009 at 21:03, Mike Belshe  wrote:
>
>> I  assume he's not a benchmark pro, but he did a decent job already.  We
>> can nitpick his sampling methodology - but it won't change the result.  He
>> is correct that many procs is far more memory consuming than single proc,
>> and we already knew this.
>> This is a tradeoff we made consciously and deliberately.  When firefox
>> crashes, all tabs go down.  When firefox memory is compromised (security),
>> all tabs are compromised.  In chrome, we don't have those problems, but
>> instead use more RAM.  Further, Chrome is also able to implement per-tab
>> prioritization, so that background tabs don't make foreground tabs go slow.
>>  Firefox can't do that.
>> Lastly, lets bring the test back to reality.  People don't visit 150
>> random home pages.  They may have 20-30 tabs open, but many are
>> applications, with cookies, javascript state and much more than just the
>> "home page".  When apps are in use, the memory gap between chrome and FF
>> shrinks a lot.
>>
>> Mike
>>
>>
>> On Sun, Jun 21, 2009 at 10:46 AM, Dan Kegel  wrote:
>>
>>> On Sun, Jun 21, 2009 at 10:22 AM, Mike Belshe wrote:
>>> > First off - kudos to the author for posting the source and steps to
>>> > reproduce!  Most don't do that!
>>> >
>>> > Second, the author is basically right.  Since he's running on Vista,
>>> its a
>>> > bit hard to tell whether his stats included shared memory or not; using
>>> the
>>> > default memory statistic ("Memory (Private Working Set)") is actually a
>>> > pretty good measure to just sum.  But he doesn't say which measurement
>>> he
>>> > used.
>>>
>>> Wait, why doesn't his program itself do the summing?
>>> (I don't see it in there.)
>>> Wouldn't that get rid of the ambiguity?
>>> How hard would it be to add that and repost?
>>> - Dan
>>>
>>
>>
>>
>>
>
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[PhistucK]

Really? the statistics show that many people are using the app mode?Or did
you mean web apps, as in web application websites?

☆PhistucK


On Sun, Jun 21, 2009 at 21:03, Mike Belshe  wrote:

> I  assume he's not a benchmark pro, but he did a decent job already.  We
> can nitpick his sampling methodology - but it won't change the result.  He
> is correct that many procs is far more memory consuming than single proc,
> and we already knew this.
> This is a tradeoff we made consciously and deliberately.  When firefox
> crashes, all tabs go down.  When firefox memory is compromised (security),
> all tabs are compromised.  In chrome, we don't have those problems, but
> instead use more RAM.  Further, Chrome is also able to implement per-tab
> prioritization, so that background tabs don't make foreground tabs go slow.
>  Firefox can't do that.
> Lastly, lets bring the test back to reality.  People don't visit 150 random
> home pages.  They may have 20-30 tabs open, but many are applications, with
> cookies, javascript state and much more than just the "home page".  When
> apps are in use, the memory gap between chrome and FF shrinks a lot.
>
> Mike
>
>
> On Sun, Jun 21, 2009 at 10:46 AM, Dan Kegel  wrote:
>
>> On Sun, Jun 21, 2009 at 10:22 AM, Mike Belshe wrote:
>> > First off - kudos to the author for posting the source and steps to
>> > reproduce!  Most don't do that!
>> >
>> > Second, the author is basically right.  Since he's running on Vista, its
>> a
>> > bit hard to tell whether his stats included shared memory or not; using
>> the
>> > default memory statistic ("Memory (Private Working Set)") is actually a
>> > pretty good measure to just sum.  But he doesn't say which measurement
>> he
>> > used.
>>
>> Wait, why doesn't his program itself do the summing?
>> (I don't see it in there.)
>> Wouldn't that get rid of the ambiguity?
>> How hard would it be to add that and repost?
>> - Dan
>>
>
>
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Lei Zhang]

FYI, --single-process only works in Chromium. It's disabled in Google Chrome.

Search for switches::kSingleProcess in
http://src.chromium.org/viewvc/chrome/trunk/src/chrome/app/chrome_dll_main.cc?revision=18801&view=markup

On Sun, Jun 21, 2009 at 4:22 PM, est wrote:
>
> use --single-process
>
> On Jun 21, 3:37 pm, n179911  wrote:
>> Hi,
>>
>> There is a test which compares memory usage among rendering 
>> engineshttp://dotnetperls.com/chrome-memory
>>
>> From the site, it shows the maximum memory usage of Chrome is more
>> than Safari is > 2 times.
>> Since both of them are Webkit base, does that mean the V8 engine uses
>> twice as much memory as squirrel fish?
>>
>> --- Maximum memory used ---
>>     Peak memory usage measured during experiment.
>>
>> Chrome:  1216.16 MB      [Largest]
>> Firefox:  327.65 MB      [Smallest]
>> Opera:    554.11 MB
>> Safari:   517.00 MB
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[est]

use --single-process

On Jun 21, 3:37 pm, n179911  wrote:
> Hi,
>
> There is a test which compares memory usage among rendering 
> engineshttp://dotnetperls.com/chrome-memory
>
> From the site, it shows the maximum memory usage of Chrome is more
> than Safari is > 2 times.
> Since both of them are Webkit base, does that mean the V8 engine uses
> twice as much memory as squirrel fish?
>
> --- Maximum memory used ---
>     Peak memory usage measured during experiment.
>
> Chrome:  1216.16 MB      [Largest]
> Firefox:  327.65 MB      [Smallest]
> Opera:    554.11 MB
> Safari:   517.00 MB
--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

I  assume he's not a benchmark pro, but he did a decent job already.  We can
nitpick his sampling methodology - but it won't change the result.  He is
correct that many procs is far more memory consuming than single proc, and
we already knew this.
This is a tradeoff we made consciously and deliberately.  When firefox
crashes, all tabs go down.  When firefox memory is compromised (security),
all tabs are compromised.  In chrome, we don't have those problems, but
instead use more RAM.  Further, Chrome is also able to implement per-tab
prioritization, so that background tabs don't make foreground tabs go slow.
 Firefox can't do that.
Lastly, lets bring the test back to reality.  People don't visit 150 random
home pages.  They may have 20-30 tabs open, but many are applications, with
cookies, javascript state and much more than just the "home page".  When
apps are in use, the memory gap between chrome and FF shrinks a lot.

Mike


On Sun, Jun 21, 2009 at 10:46 AM, Dan Kegel  wrote:

> On Sun, Jun 21, 2009 at 10:22 AM, Mike Belshe wrote:
> > First off - kudos to the author for posting the source and steps to
> > reproduce!  Most don't do that!
> >
> > Second, the author is basically right.  Since he's running on Vista, its
> a
> > bit hard to tell whether his stats included shared memory or not; using
> the
> > default memory statistic ("Memory (Private Working Set)") is actually a
> > pretty good measure to just sum.  But he doesn't say which measurement he
> > used.
>
> Wait, why doesn't his program itself do the summing?
> (I don't see it in there.)
> Wouldn't that get rid of the ambiguity?
> How hard would it be to add that and repost?
> - Dan
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Mike Belshe]

On Sun, Jun 21, 2009 at 12:37 AM, n179911  wrote:

>
> Hi,
>
> There is a test which compares memory usage among rendering engines
> http://dotnetperls.com/chrome-memory
>
> From the site, it shows the maximum memory usage of Chrome is more
> than Safari is > 2 times.
> Since both of them are Webkit base, does that mean the V8 engine uses
> twice as much memory as squirrel fish?


No.

Chrome's multi-process architecture means that each renderer runs in its own
Javascript heap.  When you start up any browser, the JS engine will allocate
some amount of memory to its heap.  It will garbage collect under some
policy (such as when it reaches certain size limits).

In this case, chrome has 30 processes, and thus 30 javascript heaps.  Since
the test just hits home pages, each heap is probably fairly small.  If you
wanted to compare Squirrelfish to this test, you'd have to create 30
instances of it :-)

Anyway, if you want to compare V8 to SquirrelFish, there are better ways to
do it (you can take the browser out completely and run tests using each JS
engine's command line shell).  You'll find that V8's heap is very compact.

The problem demonstrated in this site's test has more to do with how Chrome
behaves with multiple processes than it does with Javascript.  I also think
that the overall performance and reliability benefits of the multi-process
architecture far outweigh the memory impacts.  And this test, IMHO, is not
very close to a realistic test of what users do.

Mike




>
>
>
>
> --- Maximum memory used ---
>Peak memory usage measured during experiment.
>
> Chrome:  1216.16 MB  [Largest]
> Firefox:  327.65 MB  [Smallest]
> Opera:554.11 MB
> Safari:   517.00 MB
>
> >
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Peter Kasting]

On Sun, Jun 21, 2009 at 1:25 AM, Elliot Glaysher (Chromium) <
e...@chromium.org> wrote:

> "Google Chrome posted the highest maximum memory usage ***when all
> chrome.exe processes were summed***, reaching 1.18 gigabytes, while
> Firefox posted the lowest maximum memory levels of 327.65 megabytes."
> [Emphasis mine]
>
> Doing any memory test on a multi-process program where naive summation
> is used will yield inaccurate results.


I have just mailed the article's author pointing out this flaw and
mentioning both the blog post Elliot refers to and our about:memory page.

PK

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Memory usage in chrome

[Elliot Glaysher (Chromium)]

On Sun, Jun 21, 2009 at 12:37 AM, n179911 wrote:
> There is a test which compares memory usage among rendering engines
> http://dotnetperls.com/chrome-memory
>
> From the site, it shows the maximum memory usage of Chrome is more
> than Safari is > 2 times.

>From TFA:

"Google Chrome posted the highest maximum memory usage ***when all
chrome.exe processes were summed***, reaching 1.18 gigabytes, while
Firefox posted the lowest maximum memory levels of 327.65 megabytes."
[Emphasis mine]

Doing any memory test on a multi-process program where naive summation
is used will yield inaccurate results. We wrote a blog post about this
problem back when we launched in September:

http://blog.chromium.org/2008/09/google-chrome-memory-usage-good-and-bad.html

(I'm not claiming that Google Chrome uses the least memory; I'm just
pointing out that the above experiment's methodology is invalid.
Calculating actual memory consumption is really hard.)

-- Elliot

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---