Re: [chromium-dev] [MEMORY] CreateDIBSection: memory usage for bitmaps in Windows
On Thu, Nov 26, 2009 at 12:32 PM, Jim Roskind wrote: > SUMMARY: > Our handling of bitmap memory may be a fertile ground for saving both > physical and virtual memory. I'm still investigating this, but I wanted to > share my thoughts and very preliminary findings so that other's might chime > in and correct/extend what I'm seeing. > > PRELIMINARY DETAILS and thoughts. > AntonM identified that at least part of the under-counting in the memory > blame utility appeared because of allocations that took place in > CreateDIBSection. That routine creates bitmaps that provide backing store > for the display. The allocations appear to be made in kernel space, even > though the memory is made accessible to users (Chrome) in data returned by > this call.. Disclaimer: My Windows display driver knowledge is aged. I think the DIB storage is from user address space, not from kernel address space. There is a process affinity here, right? Does not make much sense to waste kernel-mode address space for it. Note - I have heard that in Vista the display driver model has been significantly changed. So in theory it is possible that you receive a direct pointer to device driver managed bitmap if the hardware directly supports DIBs. I doubt it, but I really don't know. :) > a) CreateDIBSection is called in both renderer and in the browser. > Allocation sizes often reflect the height and width of nearly the entire > display device, times 4 (32bits per pixel results in 4 bytes per pixel). On > my machine, this regularly produced allocations in the range of 7+megs. If, > as I was told by Will Chan, we write to one store and then "swap it into > place," then each user (tab? embedded video?) may actually need double this > amount to get their work done. > b) This function is called a LOT. It would appear that we might almost be > thrashing allocation and deallocations at times... but I'm still > investigating. Maybe the debugger was impacting the display... and causing > an evil feedback loop. Remote debugging with dbgsrv.exe on virtual of another physical machine should significantly reduce the impact. If you feel brave (something I would recommend), you may use kenel-mode debugging to see the actual Eng* and Drv* calls. > c) I'm not completely clear on the lifetime of these large allocations. > They are to some extent reportedly (re: msdn) released when we call > DeleteObject, but it is not immediately clear if they are simply put on a > free list, or decommitted, or fully returned to the OS. I'm also not clear > on the intended lifetime in the context of Chrome/Webkit, renderers, > browsers, and plugins. Can we get away with a more minimal set of such > bitmaps? > d) We also call CreateDIBSection with really small areas. Specifically, we > call it with 1 x 1 pixel areas (there is a comment in the code that we'd > like to ask for a zero size region... but CreateDIBSection reportedly does > not deal with zero very well). I noticed that the tiny bitmap that is > returned is always 64K aligned, and I'm *suspicious* that we've actually > allocated more memory, and I'm sure we've at least fragmented our VM space > with such tiny allocations. > e) There is a possibility that we could more directly manage the large > bitmap storage by calling into CreateDIBSection with pre-allocated areas > (re: mapped objects). Perhaps I'm confused, but I think we're using these > bitmaps to allow renderers to write directly to the graphical context (or to > a secondary backing store that is swapped into place). Perchance such more > direct memory handling could assure we fragment less, and decommit as much > as possible. > In chatting with Will Chan last night, I also heard that JamesR is looking > at the impact on virtual address space of having a multitude of backing > store bitmaps active. Will indicated that when a page has a LOT of videos, > that he was shown that each might require its own bitmap, He said that > JamesR thought this may be playing a role in our out-of-memory problems by > exhausting/fragmenting virtual space. > > Jim > > -- > Chromium Developers mailing list: chromium-dev@googlegroups.com > View archives, change email options, or unsubscribe: > http://groups.google.com/group/chromium-dev -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
Re: [chromium-dev] [MEMORY] CreateDIBSection: memory usage for bitmaps in Windows
This time from my chromium account: On Fri, Nov 27, 2009 at 5:43 PM, Kenneth Russell wrote: > On Thu, Nov 26, 2009 at 12:32 PM, Jim Roskind wrote: >> SUMMARY: >> Our handling of bitmap memory may be a fertile ground for saving both >> physical and virtual memory. I'm still investigating this, but I wanted to >> share my thoughts and very preliminary findings so that other's might chime >> in and correct/extend what I'm seeing. >> >> PRELIMINARY DETAILS and thoughts. >> AntonM identified that at least part of the under-counting in the memory >> blame utility appeared because of allocations that took place in >> CreateDIBSection. That routine creates bitmaps that provide backing store >> for the display. The allocations appear to be made in kernel space, even >> though the memory is made accessible to users (Chrome) in data returned by >> this call.. >> a) CreateDIBSection is called in both renderer and in the browser. >> Allocation sizes often reflect the height and width of nearly the entire >> display device, times 4 (32bits per pixel results in 4 bytes per pixel). On >> my machine, this regularly produced allocations in the range of 7+megs. If, >> as I was told by Will Chan, we write to one store and then "swap it into >> place," then each user (tab? embedded video?) may actually need double this >> amount to get their work done. >> b) This function is called a LOT. It would appear that we might almost be >> thrashing allocation and deallocations at times... but I'm still >> investigating. Maybe the debugger was impacting the display... and causing >> an evil feedback loop. > > While investigating > http://code.google.com/p/chromium/issues/detail?id=21921 I observed > that on Windows that when the Chrome window is resized, a Skia canvas > the size of the entire window is allocated and discarded in order to > paint the window background. I don't think this happens during normal > repainting operations but is probably one cause of allocation > thrashing of these objects. If it would be helpful I can try to > reconstruct where this happens but others on this list probably know > where it is off the top of their heads. That's probably in the browser process in response to a WM_PAINT. The allocation is done in canvas_paint_win.h. -Scott >> c) I'm not completely clear on the lifetime of these large allocations. >> They are to some extent reportedly (re: msdn) released when we call >> DeleteObject, but it is not immediately clear if they are simply put on a >> free list, or decommitted, or fully returned to the OS. I'm also not clear >> on the intended lifetime in the context of Chrome/Webkit, renderers, >> browsers, and plugins. Can we get away with a more minimal set of such >> bitmaps? >> d) We also call CreateDIBSection with really small areas. Specifically, we >> call it with 1 x 1 pixel areas (there is a comment in the code that we'd >> like to ask for a zero size region... but CreateDIBSection reportedly does >> not deal with zero very well). I noticed that the tiny bitmap that is >> returned is always 64K aligned, and I'm *suspicious* that we've actually >> allocated more memory, and I'm sure we've at least fragmented our VM space >> with such tiny allocations. >> e) There is a possibility that we could more directly manage the large >> bitmap storage by calling into CreateDIBSection with pre-allocated areas >> (re: mapped objects). Perhaps I'm confused, but I think we're using these >> bitmaps to allow renderers to write directly to the graphical context (or to >> a secondary backing store that is swapped into place). Perchance such more >> direct memory handling could assure we fragment less, and decommit as much >> as possible. >> In chatting with Will Chan last night, I also heard that JamesR is looking >> at the impact on virtual address space of having a multitude of backing >> store bitmaps active. Will indicated that when a page has a LOT of videos, >> that he was shown that each might require its own bitmap, He said that >> JamesR thought this may be playing a role in our out-of-memory problems by >> exhausting/fragmenting virtual space. >> >> Jim >> >> -- >> Chromium Developers mailing list: chromium-dev@googlegroups.com >> View archives, change email options, or unsubscribe: >> http://groups.google.com/group/chromium-dev > > -- > Chromium Developers mailing list: chromium-dev@googlegroups.com > View archives, change email options, or unsubscribe: > http://groups.google.com/group/chromium-dev > -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
Re: [chromium-dev] [MEMORY] CreateDIBSection: memory usage for bitmaps in Windows
On Fri, Nov 27, 2009 at 5:43 PM, Kenneth Russell wrote: > http://code.google.com/p/chromium/issues/detail?id=21921 I observed > that on Windows that when the Chrome window is resized, a Skia canvas > the size of the entire window is allocated and discarded in order to > paint the window background. I don't think this happens during normal > repainting operations but is probably one cause of allocation > thrashing of these objects. If it would be helpful I can try to > reconstruct where this happens but others on this list probably know > where it is off the top of their heads. When resizing, we need to repaint everything so we'll allocate a TransportDIB that is large enough to paint the whole window. TransportDIBs are cached at both ends it'll persist for a few seconds. If the window keeps getting large, then the old TransportDIB will be too small now and we'll allocate a new one. I think there's a little headroom added when we allocate, so hopefully we don't reallocate on every single frame, but if repaint takes a while then it's possible. AGL -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
Re: [chromium-dev] [MEMORY] CreateDIBSection: memory usage for bitmaps in Windows
On Fri, Nov 27, 2009 at 5:43 PM, Kenneth Russell wrote: > While investigating > http://code.google.com/p/chromium/issues/detail?id=21921 I observed > that on Windows that when the Chrome window is resized, a Skia canvas > the size of the entire window is allocated and discarded in order to > paint the window background. I don't think this happens during normal > repainting operations but is probably one cause of allocation > thrashing of these objects. If it would be helpful I can try to > reconstruct where this happens but others on this list probably know > where it is off the top of their heads. > +CC brettw, darin PK -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
Re: [chromium-dev] [MEMORY] CreateDIBSection: memory usage for bitmaps in Windows
On Thu, Nov 26, 2009 at 12:32 PM, Jim Roskind wrote: > SUMMARY: > Our handling of bitmap memory may be a fertile ground for saving both > physical and virtual memory. I'm still investigating this, but I wanted to > share my thoughts and very preliminary findings so that other's might chime > in and correct/extend what I'm seeing. > > PRELIMINARY DETAILS and thoughts. > AntonM identified that at least part of the under-counting in the memory > blame utility appeared because of allocations that took place in > CreateDIBSection. That routine creates bitmaps that provide backing store > for the display. The allocations appear to be made in kernel space, even > though the memory is made accessible to users (Chrome) in data returned by > this call.. > a) CreateDIBSection is called in both renderer and in the browser. > Allocation sizes often reflect the height and width of nearly the entire > display device, times 4 (32bits per pixel results in 4 bytes per pixel). On > my machine, this regularly produced allocations in the range of 7+megs. If, > as I was told by Will Chan, we write to one store and then "swap it into > place," then each user (tab? embedded video?) may actually need double this > amount to get their work done. > b) This function is called a LOT. It would appear that we might almost be > thrashing allocation and deallocations at times... but I'm still > investigating. Maybe the debugger was impacting the display... and causing > an evil feedback loop. While investigating http://code.google.com/p/chromium/issues/detail?id=21921 I observed that on Windows that when the Chrome window is resized, a Skia canvas the size of the entire window is allocated and discarded in order to paint the window background. I don't think this happens during normal repainting operations but is probably one cause of allocation thrashing of these objects. If it would be helpful I can try to reconstruct where this happens but others on this list probably know where it is off the top of their heads. -Ken > c) I'm not completely clear on the lifetime of these large allocations. > They are to some extent reportedly (re: msdn) released when we call > DeleteObject, but it is not immediately clear if they are simply put on a > free list, or decommitted, or fully returned to the OS. I'm also not clear > on the intended lifetime in the context of Chrome/Webkit, renderers, > browsers, and plugins. Can we get away with a more minimal set of such > bitmaps? > d) We also call CreateDIBSection with really small areas. Specifically, we > call it with 1 x 1 pixel areas (there is a comment in the code that we'd > like to ask for a zero size region... but CreateDIBSection reportedly does > not deal with zero very well). I noticed that the tiny bitmap that is > returned is always 64K aligned, and I'm *suspicious* that we've actually > allocated more memory, and I'm sure we've at least fragmented our VM space > with such tiny allocations. > e) There is a possibility that we could more directly manage the large > bitmap storage by calling into CreateDIBSection with pre-allocated areas > (re: mapped objects). Perhaps I'm confused, but I think we're using these > bitmaps to allow renderers to write directly to the graphical context (or to > a secondary backing store that is swapped into place). Perchance such more > direct memory handling could assure we fragment less, and decommit as much > as possible. > In chatting with Will Chan last night, I also heard that JamesR is looking > at the impact on virtual address space of having a multitude of backing > store bitmaps active. Will indicated that when a page has a LOT of videos, > that he was shown that each might require its own bitmap, He said that > JamesR thought this may be playing a role in our out-of-memory problems by > exhausting/fragmenting virtual space. > > Jim > > -- > Chromium Developers mailing list: chromium-dev@googlegroups.com > View archives, change email options, or unsubscribe: > http://groups.google.com/group/chromium-dev -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
