Re: Virtual Memory fragmentation issues

2013-04-09 Thread Jet Villegas
ot; , dev-platform@lists.mozilla.org, "Nicholas Nethercote" , "Brian R. Bondy" , "Benjamin Smedberg" , "Bas Schouten" Sent: Tuesday, April 9, 2013 12:45:51 PM Subject: Re: Virtual Memory fragmentation issues Just allocating a bunch of Texture memory and

Re: Virtual Memory fragmentation issues

2013-04-09 Thread Jeff Muizelaar
n Lebar" , "Aaron Klotz" , > dev-platform@lists.mozilla.org, "Nicholas Nethercote" > , "Brian R. Bondy" > Sent: Tuesday, April 9, 2013 12:24:32 PM > Subject: Re: Virtual Memory fragmentation issues > > > On 2013-04-08, at 7:46 PM, Benjamin Smed

Re: Virtual Memory fragmentation issues

2013-04-09 Thread Jet Villegas
Klotz" , dev-platform@lists.mozilla.org, "Nicholas Nethercote" , "Brian R. Bondy" Sent: Tuesday, April 9, 2013 12:24:32 PM Subject: Re: Virtual Memory fragmentation issues On 2013-04-08, at 7:46 PM, Benjamin Smedberg wrote: > In stability-land we're starting

Re: Virtual Memory fragmentation issues

2013-04-09 Thread Jeff Muizelaar
On 2013-04-08, at 7:46 PM, Benjamin Smedberg wrote: > In stability-land we're starting to see some "interesting" problems relating > to virtual memory usage in Firefox on Windows. > Either our code or the ATI driver is leaking mapped memory references in a > way that chews up VM space without

Re: Virtual Memory fragmentation issues

2013-04-09 Thread Benjamin Smedberg
On 4/8/2013 7:46 PM, Benjamin Smedberg wrote: In stability-land we're starting to see some "interesting" problems relating to virtual memory usage in Firefox on Windows. See bug 875030 This should have been bug 857030. 2) later on, there appear to be a large number of 8.7MB allocated blocks

Re: Virtual Memory fragmentation issues

2013-04-09 Thread Justin Lebar
> I see, so the hypothesis is that 100% of the fragmentation is coming from > VirtualAlloc/MapViewOfFile, not from allocations in general? jemalloc does not make 4kb allocations, I think ever. So yes. On Tue, Apr 9, 2013 at 9:23 AM, Kevin Gadd wrote: > I see, so the hypothesis is that 100% of t

Re: Virtual Memory fragmentation issues

2013-04-09 Thread Kevin Gadd
I see, so the hypothesis is that 100% of the fragmentation is coming from VirtualAlloc/MapViewOfFile, not from allocations in general? I believe the kernel instrumentation tools could give you data on this but it would probably require modifications to the profiler tool, so maybe it can't be used t

Re: Virtual Memory fragmentation issues

2013-04-09 Thread Aaron Klotz
On 4/08/13 5:57 PM, Nicholas Nethercote wrote: AIUI, on Windows the smallest block you can ask for with VirtualAlloc is 4 KiB. However, no more than one VirtualAlloc block can exist per 64 KiB chunk. So if you ask for 4 KiB you'll end up wasting the remaining 60 KiB of address space in the 64 K

Re: Virtual Memory fragmentation issues

2013-04-09 Thread Benjamin Smedberg
On 4/9/13 7:12 AM, Kevin Gadd wrote: This is one of the problems Heap Profiler was written to solve: https://github.com/kevingadd/HeapProfiler It enables stack trace captures at allocation time, so at any given point in the capture you can open the heap and see who allocated a given block of me

Re: Virtual Memory fragmentation issues

2013-04-09 Thread Kevin Gadd
This is one of the problems Heap Profiler was written to solve: https://github.com/kevingadd/HeapProfiler It enables stack trace captures at allocation time, so at any given point in the capture you can open the heap and see who allocated a given block of memory. It can also display a fragmentati

Re: Virtual Memory fragmentation issues

2013-04-08 Thread Mike Hommey
On Mon, Apr 08, 2013 at 08:09:08PM -0400, Justin Lebar wrote: > > AIUI, on Windows the smallest block you can ask for with VirtualAlloc > > is 4 KiB. However, no more than one VirtualAlloc block can exist per > > 64 KiB chunk. So if you ask for 4 KiB you'll end up wasting the > > remaining 60 KiB

Re: Virtual Memory fragmentation issues

2013-04-08 Thread Wayne
On 4/8/2013 7:46 PM, Benjamin Smedberg wrote: See bug 875030 for information that led me along this path. In this case, the user has plenty of memory (according to Windows) and is still failing to allocate moderate-size graphics buffers (1-2MB). This has very similar symptoms to a bunch of other

Re: Virtual Memory fragmentation issues

2013-04-08 Thread Justin Lebar
> AIUI, on Windows the smallest block you can ask for with VirtualAlloc > is 4 KiB. However, no more than one VirtualAlloc block can exist per > 64 KiB chunk. So if you ask for 4 KiB you'll end up wasting the > remaining 60 KiB of address space in the 64 KiB chunk. Awesome memory, Nick. MSDN se

Re: Virtual Memory fragmentation issues

2013-04-08 Thread Nicholas Nethercote
On Mon, Apr 8, 2013 at 4:46 PM, Benjamin Smedberg wrote: > > 1) early in the memory info, there appears to be a common pattern of a > committed block and then 15 free blocks. AIUI, on Windows the smallest block you can ask for with VirtualAlloc is 4 KiB. However, no more than one VirtualAlloc bl

Re: Virtual Memory fragmentation issues

2013-04-08 Thread Benjamin Smedberg
On 4/8/2013 7:46 PM, Benjamin Smedberg wrote: In this case, the crash report shows "Available Virtual Memory" as 303558656, but in fact the largest contiguous block of available VM is 64k. Correct, I had wrong math. The largest block available is about 3MB, and the allocation being requested

Virtual Memory fragmentation issues

2013-04-08 Thread Benjamin Smedberg
In stability-land we're starting to see some "interesting" problems relating to virtual memory usage in Firefox on Windows. See bug 875030 for information that led me along this path. In this case, the user has plenty of memory (according to Windows) and is still failing to allocate moderate-s