mmap of large amount returns invalid pointer
I am experience a problem where mmap does not return -1, but the pointer returned is not to a valid chunk of memory. I have included a sample program, the output that I see, and the output of cygcheck below. I have tried this program both with and withouth the registry setting HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\heap_chunk_in_mb which, when set, I have set to 512Mb. BTW, I notice that this registry value isn't checked for by cygcheck. That would seem useful. Even if I got the registry setting wrong, I would expect mmap to return -1 if it is unable to get the space. Thanks for your help. #include #include #include #include void try (uint length) { void *p; fprintf(stderr, "attempting to mmap %u bytes ...", length); p = mmap (NULL, length, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0); if ((void*)-1 == p) { fprintf(stderr, " out of memory\n"); exit(0); } fprintf(stderr, "zeroing ... "); memset(p, 0, length); fprintf(stderr, "unmapping ... "); munmap(p, length); fprintf(stderr, "succeeded\n"); length *= 2; } int main (int argc, char **argv) { MEMORYSTATUS ms; ms.dwLength = sizeof(MEMORYSTATUS); GlobalMemoryStatus(&ms); fprintf(stderr, "TotalPhys: %ld\nAvailPhys: %ld\nTotalPageFile: %ld\nAvailPageFile: %ld\nTotalVirtual: %ld\nAvailVirtual: %ld\n", ms.dwTotalPhys, ms.dwAvailPhys, ms.dwTotalPageFile, ms.dwAvailPageFile, ms.dwTotalVirtual, ms.dwAvailVirtual); try(1); try(13000); exit(0); } TotalPhys: 209113088 AvailPhys: 166580224 TotalPageFile: 309587968 AvailPageFile: 275365888 TotalVirtual: 2147352576 AvailVirtual: 1589841920 attempting to mmap 1 bytes ...zeroing ... unmapping ... succeeded attempting to mmap 13000 bytes ...zeroing ... Segmentation fault (core dumped) Cygwin Win95/NT Configuration Diagnostics Current System Time: Mon Mar 11 19:42:08 2002 Windows NT Ver 4.0 Build 1381 Service Pack 6 Path: z:\cygwin\home\sweeks\bin z:\cygwin\sbin z:\cygwin\usr\sbin z:\cygwin\usr\local\bin z:\cygwin\bin z:\cygwin\bin c:\WINNT\system32 c:\WINNT SysDir: C:\WINNT\System32 WinDir: C:\WINNT CYGWIN = `binmode ntsec tty ' HOME = `z:\cygwin\home\sweeks' MAKE_MODE = `unix' PWD = `/cygdrive/c' USER = `sweeks' COMPUTERNAME = `WINDOZE' COMSPEC = `C:\WINNT\system32\cmd.exe' HOMEDRIVE = `C:' HOMEPATH = `\' LOGONSERVER = `\\WINDOZE' MANPATH = `:/usr/ssl/man' NUMBER_OF_PROCESSORS = `1' OLDPWD = `/home/sweeks' OS2LIBPATH = `C:\WINNT\system32\os2\dll;' OS = `Windows_NT' PATHEXT = `.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH' PROCESSOR_ARCHITECTURE = `x86' PROCESSOR_IDENTIFIER = `x86 Family 15 Model 1 Stepping 2, GenuineIntel' PROCESSOR_LEVEL = `15' PROCESSOR_REVISION = `0102' PROMPT = `$P$G' PS1 = `\W% ' SHLVL = `1' SYSTEMDRIVE = `C:' SYSTEMROOT = `C:\WINNT' TEMP = `c:\TEMP' TERM = `cygwin' TMP = `c:\TEMP' USERDOMAIN = `WINDOZE' USERNAME = `sweeks' USERPROFILE = `C:\WINNT\Profiles\sweeks' WINDIR = `C:\WINNT' _ = `/usr/bin/cygcheck' HKEY_CURRENT_USER\Software\Cygnus Solutions HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin (default) = 0x0200 HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\mounts v2 (default) = `/cygdrive' cygdrive flags = 0x0022 HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\Program Options HKEY_CURRENT_USER\Software\Cygnus Solutions\CYGWIN.DLL setup HKEY_CURRENT_USER\Software\Cygnus Solutions\CYGWIN.DLL setup\b15.0 HKEY_CURRENT_USER\Software\Cygnus Solutions\CYGWIN.DLL setup\b15.0\mounts HKEY_CURRENT_USER\Software\Cygnus Solutions\CYGWIN.DLL setup\b15.0\mounts\00 (default) = `C:' unix = `/' fbinary = 0x fsilent = 0x HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2 HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/ (default) = `z:\cygwin' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/bin (default) = `z:/cygwin/bin' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/lib (default) = `z:/cygwin/lib' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\Program Options HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin B20 HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin B20\B20.1 HKE
Re: mmap of large amount returns invalid pointer
On Thu, Mar 21, 2002 at 11:15:52AM -0800, Stephen Weeks wrote: > > > Could you please describe what happens in the error case > > in plain English? I see that there could be a timing problem in > > fork() but I'd like to read how the error looks like. > > The process mmaps some memory, writes to the memory, and then forks. > After the fork, the parent unmaps the memory and exits, while the > child reads from the memory. From looking at the strace, I see that > the Cygwin fork implementation allows the parent process to continue > before ensuring that the child has completed the memory copy. In this > case, the parent unmaps the memory before the child can copy it. > Hence, we see the following line in the strace, which indicates the > failure in the child process (138). > > 1813 914381 [main] fork 138 fixup_mmaps_after_fork: ReadProcessMemory failed for >MAP_PRIVATE address 0x2A23, Win32 error 998 > c:\sweeks\tmp\fork.exe: *** recreate_mmaps_after_fork_failed I've checked in a fix. Please try the next devlopers snapshot or current from CVS. Thanks for the report and the testcase, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Developermailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: mmap of large amount returns invalid pointer
On Mon, Mar 11, 2002 at 08:38:34PM -0800, Stephen Weeks wrote: > > I am experience a problem where mmap does not return -1, but the > pointer returned is not to a valid chunk of memory. I have included a > sample program, the output that I see, and the output of cygcheck > below. > > I have tried this program both with and withouth the registry setting > HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\heap_chunk_in_mb > which, when set, I have set to 512Mb. BTW, I notice that this > registry value isn't checked for by cygcheck. That would seem > useful. > > Even if I got the registry setting wrong, I would expect mmap to > return -1 if it is unable to get the space. This registry setting has absolutely no effect on mmap(). It affects the application heap but the mmap memory is taken from the shared memory area of the system which isn't controlled by Cygwin but only by Win32 itself. I ran your below test application with slightly different values in the `try' calls to reflect the size of the page file in my box. However, before talking further about this I'd like you to run your below test application again but this time under strace. Could you please send the strace output to this list (it's not that long)? Thanks in advance, Corinna > > > #include > #include > #include > #include > > void try (uint length) { > void *p; > > fprintf(stderr, "attempting to mmap %u bytes ...", length); > p = mmap (NULL, length, > PROT_READ | PROT_WRITE, > MAP_PRIVATE | MAP_ANON, -1, 0); > if ((void*)-1 == p) { > fprintf(stderr, " out of memory\n"); > exit(0); > } > fprintf(stderr, "zeroing ... "); > memset(p, 0, length); > fprintf(stderr, "unmapping ... "); > munmap(p, length); > fprintf(stderr, "succeeded\n"); > length *= 2; > } > > int main (int argc, char **argv) { > MEMORYSTATUS ms; > > ms.dwLength = sizeof(MEMORYSTATUS); > GlobalMemoryStatus(&ms); > fprintf(stderr, "TotalPhys: %ld\nAvailPhys: %ld\nTotalPageFile: >%ld\nAvailPageFile: %ld\nTotalVirtual: %ld\nAvailVirtual: %ld\n", >ms.dwTotalPhys, >ms.dwAvailPhys, >ms.dwTotalPageFile, >ms.dwAvailPageFile, >ms.dwTotalVirtual, >ms.dwAvailVirtual); > try(1); > try(13000); > exit(0); > } > > > > TotalPhys: 209113088 > AvailPhys: 166580224 > TotalPageFile: 309587968 > AvailPageFile: 275365888 > TotalVirtual: 2147352576 > AvailVirtual: 1589841920 > attempting to mmap 1 bytes ...zeroing ... unmapping ... succeeded > attempting to mmap 13000 bytes ...zeroing ... Segmentation fault (core dumped) > > > > Cygwin Win95/NT Configuration Diagnostics > Current System Time: Mon Mar 11 19:42:08 2002 > > Windows NT Ver 4.0 Build 1381 Service Pack 6 > > Path: z:\cygwin\home\sweeks\bin > z:\cygwin\sbin > z:\cygwin\usr\sbin > z:\cygwin\usr\local\bin > z:\cygwin\bin > z:\cygwin\bin > c:\WINNT\system32 > c:\WINNT > > SysDir: C:\WINNT\System32 > WinDir: C:\WINNT > > CYGWIN = `binmode ntsec tty ' > HOME = `z:\cygwin\home\sweeks' > MAKE_MODE = `unix' > PWD = `/cygdrive/c' > USER = `sweeks' > > COMPUTERNAME = `WINDOZE' > COMSPEC = `C:\WINNT\system32\cmd.exe' > HOMEDRIVE = `C:' > HOMEPATH = `\' > LOGONSERVER = `\\WINDOZE' > MANPATH = `:/usr/ssl/man' > NUMBER_OF_PROCESSORS = `1' > OLDPWD = `/home/sweeks' > OS2LIBPATH = `C:\WINNT\system32\os2\dll;' > OS = `Windows_NT' > PATHEXT = `.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH' > PROCESSOR_ARCHITECTURE = `x86' > PROCESSOR_IDENTIFIER = `x86 Family 15 Model 1 Stepping 2, GenuineIntel' > PROCESSOR_LEVEL = `15' > PROCESSOR_REVISION = `0102' > PROMPT = `$P$G' > PS1 = `\W% ' > SHLVL = `1' > SYSTEMDRIVE = `C:' > SYSTEMROOT = `C:\WINNT' > TEMP = `c:\TEMP' > TERM = `cygwin' > TMP = `c:\TEMP' > USERDOMAIN = `WINDOZE' > USERNAME = `sweeks' > USERPROFILE = `C:\WINNT\Profiles\sweeks' > WINDIR = `C:\WINNT' > _ = `/usr/bin/cygcheck' > > HKEY_CURRENT_USER\Software\Cygnus Solutions > HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin > (default) = 0x0200 > HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\mounts v2 > (default) = `/cygdrive' > cygdrive flags = 0x0022 > HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\Program Options > HKEY_CURRENT_USER\Software\Cygnus Solutions\CYGWIN.DLL setup > HKEY_CURRENT_USER\Software\Cygnus Solutions\CYGWIN.DLL setup\b15.0 > HKEY_CURRENT_USER\Software\Cygnus Solutions\CYGWIN.DLL setup\b15.0\mounts > HKEY_CURRENT_USER\Software\Cygnus Soluti
Re: mmap of large amount returns invalid pointer
> However, before talking further about this I'd like you to run your > below test application again but this time under strace. Could > you please send the strace output to this list (it's not that long)? Here is the strace output. ** Program name: c:\mmap.exe (159) App version: 1003.10, api: 0.51 DLL version: 1003.10, api: 0.51 DLL build:2002-02-25 11:14 OS version: Windows NT-4.0 Date/Time:2002-03-12 10:02:03 ** 7637 19265 [main] mmap 159 environ_init: 0xA010420: !C:=C:\WINNT\Profiles\sweeks\Desktop 2224 21489 [main] mmap 159 environ_init: 0xA010450: !Z:=Z:\cygwin\bin 2542 24031 [main] mmap 159 environ_init: 0xA010468: COMPUTERNAME=WINDOZE 2498 26529 [main] mmap 159 environ_init: 0xA010488: COMSPEC=C:\WINNT\system32\cmd.exe 2338 28867 [main] mmap 159 parse_options: binmode 65536 2466 31333 [main] mmap 159 parse_options: ntsec 1 2297 33630 [main] mmap 159 parse_options: tty 1001 2517 36147 [main] mmap 159 parse_options: returning 1180 37327 [main] mmap 159 environ_init: 0xA0104B0: CYGWIN=binmode ntsec tty 2166 39493 [main] mmap 159 getwinenv: can't set native for HOME= since no environ yet 2358 41851 [main] mmap 159 mount_info::conv_to_posix_path: conv_to_posix_path (z:\cygwin\home\sweeks, no-keep-rel, no-add-slash) 1392 43243 [main] mmap 159 normalize_win32_path: z:\cygwin\home\sweeks = normalize_win32_path (z:\cygwin\home\sweeks) 2272 45515 [main] mmap 159 mount_info::conv_to_posix_path: /home/sweeks = conv_to_posix_path (z:\cygwin\home\sweeks) 3429 48944 [main] mmap 159 win_env::add_cache: posix /home/sweeks 1132 50076 [main] mmap 159 win_env::add_cache: native HOME=z:\cygwin\home\sweeks 1137 51213 [main] mmap 159 posify: env var converted to HOME=/home/sweeks 2183 53396 [main] mmap 159 environ_init: 0xA010520: HOME=/home/sweeks 2708 56104 [main] mmap 159 environ_init: 0xA010500: HOMEDRIVE=C: 2249 58353 [main] mmap 159 environ_init: 0xA010668: HOMEPATH=\ 2184 60537 [main] mmap 159 environ_init: 0xA010678: LOGONSERVER=\\WINDOZE 2133 62670 [main] mmap 159 environ_init: 0xA010698: MAKE_MODE=unix 2164 64834 [main] mmap 159 environ_init: 0xA0106B0: MANPATH=:/usr/ssl/man 2669 67503 [main] mmap 159 environ_init: 0xA0106D0: NUMBER_OF_PROCESSORS=1 2156 69659 [main] mmap 159 environ_init: 0xA0106F0: OLDPWD=/home/sweeks 2169 71828 [main] mmap 159 environ_init: 0xA010708: OS2LIBPATH=C:\WINNT\system32\os2\dll; 2164 73992 [main] mmap 159 environ_init: 0xA010738: OS=Windows_NT 3044 77036 [main] mmap 159 getwinenv: can't set native for PATH= since no environ yet 2261 79297 [main] mmap 159 mount_info::conv_to_posix_path: conv_to_posix_path (z:\cygwin\home\sweeks\bin, keep-rel, no-add-slash) 1177 80474 [main] mmap 159 normalize_win32_path: z:\cygwin\home\sweeks\bin = normalize_win32_path (z:\cygwin\home\sweeks\bin) 1156 81630 [main] mmap 159 mount_info::conv_to_posix_path: /home/sweeks/bin = conv_to_posix_path (z:\cygwin\home\sweeks\bin) 1115 82745 [main] mmap 159 mount_info::conv_to_posix_path: conv_to_posix_path (z:\cygwin\sbin, keep-rel, no-add-slash) 1134 83879 [main] mmap 159 normalize_win32_path: z:\cygwin\sbin = normalize_win32_path (z:\cygwin\sbin) 1131 85010 [main] mmap 159 mount_info::conv_to_posix_path: /sbin = conv_to_posix_path (z:\cygwin\sbin) 1606 86616 [main] mmap 159 mount_info::conv_to_posix_path: conv_to_posix_path (z:\cygwin\usr\sbin, keep-rel, no-add-slash) 1192 87808 [main] mmap 159 normalize_win32_path: z:\cygwin\usr\sbin = normalize_win32_path (z:\cygwin\usr\sbin) 1142 88950 [main] mmap 159 mount_info::conv_to_posix_path: /usr/sbin = conv_to_posix_path (z:\cygwin\usr\sbin) 1104 90054 [main] mmap 159 mount_info::conv_to_posix_path: conv_to_posix_path (z:\cygwin\usr\local\bin, keep-rel, no-add-slash) 1134 91188 [main] mmap 159 normalize_win32_path: z:\cygwin\usr\local\bin = normalize_win32_path (z:\cygwin\usr\local\bin) 1130 92318 [main] mmap 159 mount_info::conv_to_posix_path: /usr/local/bin = conv_to_posix_path (z:\cygwin\usr\local\bin) 1134 93452 [main] mmap 159 mount_info::conv_to_posix_path: conv_to_posix_path (z:\cygwin\bin, keep-rel, no-add-slash) 1396 94848 [main] mmap 159 normalize_win32_path: z:\cygwin\bin = normalize_win32_path (z:\cygwin\bin) 1618 96466 [main] mmap 159 mount_info::conv_to_posix_path: /usr/bin = conv_to_posix_path (z:\cygwin\bin) 1193 97659 [main] mmap 159 mount_info::conv_to_posix_path: conv_to_posix_path (z:\cygwin\bin, keep-rel, no-add-slash) 1145 98804 [main] mmap 159 normalize_win32_path: z:\cygwin\bin = normalize_win32_path (z:\cygwin\bin) 1139 99943 [main] mmap 159 mount_info::conv_to_posix_path: /usr/bin = conv_to_posix_path (z:\cygwin\bin) 1132 101075 [main] mmap 159 mount_info::conv_to_posix_path: conv_to_posix_path (c:\WINNT\system32, keep-rel, no-add-slash) 1133 102208 [main] mmap 159 normal
Re: mmap of large amount returns invalid pointer
On Tue, Mar 12, 2002 at 10:08:05AM -0800, Stephen Weeks wrote: > > > However, before talking further about this I'd like you to run your > > below test application again but this time under strace. Could > > you please send the strace output to this list (it's not that long)? > > Here is the strace output. Thanks for the strace. Boy, that's a problem... the mmap() succeeds, the following printf SEGV's as a follow up error. I asked you for the strace to be really sure. The same happens on my box. It's a surprising error raised by a Win32 call. Currently I don't know how to easily avoid that. Hmm. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Developermailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: mmap of large amount returns invalid pointer
Hi Stephen and Corinna, looks like you have overlooked that mmap doesn't return NULL despite win32 error in strace (see below): c:\>net helpmsg 1455 The paging file is too small for this operation to complete. Second, (may mean nothing) why is h:188 first and h:190 second time? Bye, Heribert ([EMAIL PROTECTED]) > -Original Message- > From: Stephen Weeks [SMTP:[EMAIL PROTECTED]] > Sent: Tuesday, March 12, 2002 19:08 > To: [EMAIL PROTECTED] > Subject: Re: mmap of large amount returns invalid pointer > [Heribert] [snip] > 1668 719099 [main] mmap 159 mmap: addr 0, len 1, prot 3, flags > 22, fd -1, off 0 > 5487 724586 [main] mmap 159 fhandler_disk_file::mmap: 2A23 = > MapViewOfFileEx (h:188, access:1, 0, off:0, len:17936, addr:0) > 26042 750628 [main] mmap 159 mmap: 2A23 = mmap() succeeded > [Heribert] [snip] > 1302 2076991 [main] mmap 159 mmap: addr 0, len 13000, prot 3, flags > 22, fd -1, off 0 > 2949 2079940 [main] mmap 159 fhandler_disk_file::mmap: 2A23 = > MapViewOfFileEx (h:190, access:1, 0, off:0, len:130023424, addr:0) > 7425 2087365 [main] mmap 159 mmap_record::map_map: -1 = map_map (): Win32 > error 1455 > 1830 2089195 [main] mmap 159 mmap: 2A23 = mmap() succeeded > [Heribert] [snip] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: mmap of large amount returns invalid pointer
> looks like you have overlooked that mmap doesn't return NULL > despite win32 error in strace (see below): > > c:\>net helpmsg 1455 > > The paging file is too small for this operation to complete. Thanks for pointing that out. I agree that it looks like the Cygwin dll is failing to check for the Win32 error, which should have caused mmap to return -1. But even if that is fixed, I don't understand why Cygwin/mmap is unable to obtain the memory. My earlier message showed that at the start of the program the information returned by GlobalMemoryStatus shows that there is easily >300Mb available. TotalPhys: 209113088 AvailPhys: 166580224 TotalPageFile: 309587968 AvailPageFile: 275365888 TotalVirtual: 2147352576 AvailVirtual: 1589841920 I tried a modified version of the program that uses VirtualAlloc/Free instead of mmap/munmap and it is able to allocate, zero, and free 300Mb. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: mmap of large amount returns invalid pointer
On Tue, Mar 12, 2002 at 06:21:15PM -0800, Stephen Weeks wrote: > >> looks like you have overlooked that mmap doesn't return NULL >> despite win32 error in strace (see below): >> >> c:\>net helpmsg 1455 >> >> The paging file is too small for this operation to complete. > >Thanks for pointing that out. I agree that it looks like the Cygwin >dll is failing to check for the Win32 error, which should have caused >mmap to return -1. > >But even if that is fixed, I don't understand why Cygwin/mmap is >unable to obtain the memory. My earlier message showed that at the >start of the program the information returned by GlobalMemoryStatus >shows that there is easily >300Mb available. > >TotalPhys: 209113088 >AvailPhys: 166580224 >TotalPageFile: 309587968 >AvailPageFile: 275365888 >TotalVirtual: 2147352576 >AvailVirtual: 1589841920 > >I tried a modified version of the program that uses VirtualAlloc/Free >instead of mmap/munmap and it is able to allocate, zero, and free >300Mb. Sounds like you should be looking at the cygwin source code. That's the best way to achieve clarity in cases like this. cgf -- Please do not send me personal email with cygwin questions. Use the resources at http://cygwin.com/ . -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: mmap of large amount returns invalid pointer
On Tue, Mar 12, 2002 at 11:44:36PM +0100, Heribert Dahms wrote: > Hi Stephen and Corinna, > > looks like you have overlooked that mmap doesn't return NULL > despite win32 error in strace (see below): > > c:\>net helpmsg 1455 > > The paging file is too small for this operation to complete. No, I didn't overlook it, it's exactly what I've been seeing on my machine as well as I mentioned yesterday. If you look into the strace you'll see that MapViewOfFileEx() returns a valid memory area. And no, it does *not* return an error code. MapViewOfFileEx() is reliable enough to not return a memory area and an error code. The failing function is a following VirtualProtect() which job is to set the memory protection on the allocated memory area correctly. And that's actually surprising. VirtualProtect() shouldn't have problems with memory since it doesn't allocate any. At least it shouldn't. But the real problem is that the page file is, well, full after the mmap() call. A following printf is trying to allocate a small amount of memory but allocation fails apparently and at one point a check for a failed malloc() is missing. So the SEGV is actually a followup of the fact that there isn't any memory left to allocate. > Second, (may mean nothing) why is h:188 first and h:190 second time? Means nothing. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Developermailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: mmap of large amount returns invalid pointer
> If you look into the strace you'll see that MapViewOfFileEx() > returns a valid memory area. And no, it does *not* return an > error code. MapViewOfFileEx() is reliable enough to not return > a memory area and an error code. > > The failing function is a following VirtualProtect() which job > is to set the memory protection on the allocated memory area > correctly. And that's actually surprising. VirtualProtect() > shouldn't have problems with memory since it doesn't allocate > any. At least it shouldn't. Having now read mmap.cc, this is how I interpret the strace as well. I'm afraid I know almost nothing about Windows and have no explanation as to why the VirtualProtect fails. > But the real problem is that the page file is, well, full after > the mmap() call. A following printf is trying to allocate a > small amount of memory but allocation fails apparently and at > one point a check for a failed malloc() is missing. > So the SEGV is actually a followup of the fact that there isn't > any memory left to allocate. I don't think this explanation is correct. I still think that mmap is returning a pointer to an invalid chunk of memory. To demonstrate this, here is a program that does an mmap, fprintf, and then attempts to write to the first byte of the mmap'ed memory. #include #include #include #include #include void die(char *fmt, ...) { va_list args; fflush(stdout); va_start(args, fmt); vfprintf(stderr, fmt, args); va_end(args); fprintf(stderr, "\n"); exit(1); } int main (int argc, char **argv) { uint length; void *p; length = 13000; p = mmap (NULL, length, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0); if ((void*)-1 == p) die("mmap failed\n"); fprintf(stderr, "mmap succeeded\n"); ((char*)p)[0] = 'a'; if (-1 == munmap(p, length)) die("munmap failed\n"); return 0; } When I run this program, the mmap and fprintf succeed, and then there is a segfault, presumably in the store to p[0]. Here is a snippet of the strace to corroborate. 1546 597904 [main] bug 161 mmap: addr 0, len 13000, prot 3, flags 22, fd -1, off 0 5208 603112 [main] bug 161 fhandler_disk_file::mmap: 2A23 = MapViewOfFileEx (h:144, access:1, 0, off:0, len:130023424, addr:0) 7918 611030 [main] bug 161 mmap_record::map_map: -1 = map_map (): Win32 error 1455 1380 612410 [main] bug 161 mmap: 2A23 = mmap() succeeded 1076 613486 [main] bug 161 _write: write (2, 0x22F758, 15) 1050 614536 [main] bug 161 fhandler_base::write: binary write mmap succeeded 1060 615596 [main] bug 161 fhandler_base::write: 15 = write (0x22F758, 15) 1005 616601 [main] bug 161 _write: 15 = write (2, 0x22F758, 15) 2225 618826 [main] bug 161 handle_exceptions: In cygwin_except_handler exc 0xC005 at 0x401140 sp 0x22FE80 1043 619869 [main] bug 161 handle_exceptions: In cygwin_except_handler sig = 11 at 0x401140 1236 621105 [main] bug 161 handle_exceptions: In cygwin_except_handler calling 0x0 622178 [main] bug 161 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION 1073 622178 [main] bug 161 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION On the other hand, if I decrease the length from 130,000,000 to 13,000,000, then the VirtualProtect does not complain, and the store to p[0] and munmap both succeed. Here is the strace snippet. 1140 594523 [main] bug 68 mmap: addr 0, len 1300, prot 3, flags 22, fd -1, off 0 7013 601536 [main] bug 68 fhandler_disk_file::mmap: 2A23 = MapViewOfFileEx (h:144, access:1, 0, off:0, len:13041664, addr:0) 4912 606448 [main] bug 68 mmap: 2A23 = mmap() succeeded 1185 607633 [main] bug 68 _write: write (2, 0x22F758, 15) 1614 609247 [main] bug 68 fhandler_base::write: binary write mmap succeeded 1150 610397 [main] bug 68 fhandler_base::write: 15 = write (0x22F758, 15) 1024 611421 [main] bug 68 _write: 15 = write (2, 0x22F758, 15) 1141 612562 [main] bug 68 munmap: munmap (addr 2A23, len 1300) 3703 616265 [main] bug 68 munmap: 0 = munmap(): 2A23 My conclusion remains that the mmap(130,000,000) is returning a pointer to invalid memory. I don't know if the problem is connected to the failure of VirtualProtect or not. I will continue to investigate, but would still appreciate any assistance. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: mmap of large amount returns invalid pointer
On Wed, Mar 13, 2002 at 01:57:33PM -0800, Stephen Weeks wrote: > I don't think this explanation is correct. I still think that mmap is > returning a pointer to an invalid chunk of memory. To demonstrate > this, here is a program that does an mmap, fprintf, and then attempts > to write to the first byte of the mmap'ed memory. The explanation wasn't quite correct, you're right, but the memory is not invalid. It's correctly returned by MapViewOfFileEx and the same value is returned by mmap(). However, it's *non-accessible* since all pages are protected using PAGE_NOACCESS after the call to MapViewOfFileEx. The next call to VirtualProtect which is responsible for setting the correct protection mode on the used memory (used mem != allocated mem) unfortunately fails. The result is that mmap() returns a correct memory pointer which is nevertheless inaccessible. :-( Anyway, I've checked in a patch. It now checks if VirtualProtect failed and then mmap() also fails. Try the next developers snapshot, please. Thanks for the testcases, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Developermailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: mmap of large amount returns invalid pointer
Corinna: > Anyway, I've checked in a patch. It now checks if VirtualProtect > failed and then mmap() also fails. Try the next developers snapshot, > please. Thanks for the fix. I tried the 2002-Mar-16 snapshot and the test program works correctly. However, a slightly modified program that munmaps the memory in the parent process and then exits does not work reliably. I have included the test case and an strace of a failed run below. Because of timing issues, you may have to run the program several times before you see the failure. As far as I can tell, the problem is that the parent unmaps the memory before the child has a chance to do "fixup_mmaps_after_fork". #include #include #include #include #include void die (char *s) { fprintf(stderr, "%d", s); exit(1); } int main (int argc, char **argv) { pid_t pid; size_t length; char *buf; fprintf(stderr, "starting\n"); length = getpagesize (); buf = (char*) mmap (NULL, length, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0); if (buf == (void*)-1) die ("mmap failed"); strcpy (buf, "hello"); pid = fork (); if (0 == pid) { fprintf(stderr, "%s\n", buf); munmap (buf, length); } else { fprintf(stderr, "%s\n", buf); munmap (buf, length); exit(0); } } ** Program name: c:\sweeks\tmp\fork.exe (63) App version: 1003.10, api: 0.51 DLL version: 1003.11, api: 0.51 DLL build:20020316 02:33:42SNP OS version: Windows NT-4.0 Date/Time:2002-03-17 10:41:52 ** 6712 17708 [main] fork 63 environ_init: 0xA010420: !C:=C:\WINNT\Profiles\sweeks\Desktop 3119 20827 [main] fork 63 environ_init: 0xA010450: !Z:=Z:\cygwin\bin 2436 23263 [main] fork 63 environ_init: 0xA010468: COMPUTERNAME=WINDOZE 2324 25587 [main] fork 63 environ_init: 0xA010488: COMSPEC=C:\WINNT\system32\cmd.exe 2327 27914 [main] fork 63 environ_init: 0xA0104B0: CVSROOT=:pserver:[EMAIL PROTECTED]:/cvs/src 2813 30727 [main] fork 63 parse_options: binmode 65536 2395 33122 [main] fork 63 parse_options: ntsec 1 2318 35440 [main] fork 63 parse_options: tty 1001 2314 37754 [main] fork 63 parse_options: returning 1375 39129 [main] fork 63 environ_init: 0xA0104F0: CYGWIN=binmode ntsec tty 2943 42072 [main] fork 63 getwinenv: can't set native for HOME= since no environ yet 2774 44846 [main] fork 63 mount_info::conv_to_posix_path: conv_to_posix_path (z:\cygwin\home\sweeks, no-keep-rel, no-add-slash) 1531 46377 [main] fork 63 normalize_win32_path: z:\cygwin\home\sweeks = normalize_win32_path (z:\cygwin\home\sweeks) 1923 48300 [main] fork 63 mount_info::conv_to_posix_path: /home/sweeks = conv_to_posix_path (z:\cygwin\home\sweeks) 4062 52362 [main] fork 63 win_env::add_cache: posix /home/sweeks 1223 53585 [main] fork 63 win_env::add_cache: native HOME=z:\cygwin\home\sweeks 1217 54802 [main] fork 63 posify: env var converted to HOME=/home/sweeks 2330 57132 [main] fork 63 environ_init: 0xA010560: HOME=/home/sweeks 3177 60309 [main] fork 63 environ_init: 0xA010540: HOMEDRIVE=C: 2429 62738 [main] fork 63 environ_init: 0xA0106A8: HOMEPATH=\ 2314 65052 [main] fork 63 environ_init: 0xA0106B8: LOGNAME=sweeks 2314 67366 [main] fork 63 environ_init: 0xA0106D0: LOGONSERVER=\\WINDOZE 2851 70217 [main] fork 63 environ_init: 0xA0106F0: MAIL=/var/spool/mail/sweeks 2407 72624 [main] fork 63 environ_init: 0xA010710: MAKE_MODE=unix 2323 74947 [main] fork 63 environ_init: 0xA010728: MANPATH=:/usr/ssl/man:/usr/ssl/man 2318 77265 [main] fork 63 environ_init: 0xA010750: NUMBER_OF_PROCESSORS=1 4838 82103 [main] fork 63 environ_init: 0xA010770: OLDPWD=/cygdrive/c/sweeks 3683 85786 [main] fork 63 environ_init: 0xA010790: OS2LIBPATH=C:\WINNT\system32\os2\dll; 2469 88255 [main] fork 63 environ_init: 0xA0107C0: OS=Windows_NT 2803 91058 [main] fork 63 getwinenv: can't set native for PATH= since no environ yet 2392 93450 [main] fork 63 mount_info::conv_to_posix_path: conv_to_posix_path (z:\cygwin\home\sweeks\bin, keep-rel, no-add-slash) 1247 94697 [main] fork 63 normalize_win32_path: z:\cygwin\home\sweeks\bin = normalize_win32_path (z:\cygwin\home\sweeks\bin) 1231 95928 [main] fork 63 mount_info::conv_to_posix_path: /home/sweeks/bin = conv_to_posix_path (z:\cygwin\home\sweeks\bin) 1210 97138 [main] fork 63 mount_info::conv_to_posix_path: conv_to_posix_path (z:\cygwin\sbin, keep-rel, no-add-slash) 1355 98493 [main] fork 63 normalize_win32_path: z:\cygwin\sbin = normalize_win32_path (z:\cygwin\sbin) 1718 100211 [main] fork 63 moun
Re: mmap of large amount returns invalid pointer
On Sun, Mar 17, 2002 at 10:51:29AM -0800, Stephen Weeks wrote: > > Corinna: > > Anyway, I've checked in a patch. It now checks if VirtualProtect > > failed and then mmap() also fails. Try the next developers snapshot, > > please. > > Thanks for the fix. I tried the 2002-Mar-16 snapshot and the test > program works correctly. However, a slightly modified program that > munmaps the memory in the parent process and then exits does not work > reliably. I have included the test case and an strace of a failed run I'm on vacation currently so I'm not really inclined to test that by myself. Could you please describe what happens in the error case in plain English? I see that there could be a timing problem in fork() but I'd like to read how the error looks like. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Developermailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: mmap of large amount returns invalid pointer
> Could you please describe what happens in the error case > in plain English? I see that there could be a timing problem in > fork() but I'd like to read how the error looks like. The process mmaps some memory, writes to the memory, and then forks. After the fork, the parent unmaps the memory and exits, while the child reads from the memory. From looking at the strace, I see that the Cygwin fork implementation allows the parent process to continue before ensuring that the child has completed the memory copy. In this case, the parent unmaps the memory before the child can copy it. Hence, we see the following line in the strace, which indicates the failure in the child process (138). 1813 914381 [main] fork 138 fixup_mmaps_after_fork: ReadProcessMemory failed for MAP_PRIVATE address 0x2A23, Win32 error 998 c:\sweeks\tmp\fork.exe: *** recreate_mmaps_after_fork_failed -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/