Re: [osg-users] [Not OSG related question] Virtual memory management on Windows
Thanks a lot for your explanation Gordon, I ended to the same conclusion, I'll need to go to a 64bits platform. I'll install an XP 64, and not a Linux David... :) Thanks again ! On Thu, Jun 26, 2008 at 12:30 AM, Gordon Tomlinson [EMAIL PROTECTED] wrote: Hi There's many issues why you will struggle with this and no it's not just a windows issues it effects other OS's some do a better job off moving the issues forward but they will still crop up Simplest solution is to go to a 64bit OS with a good 8gb or more. There is another limitation you will hit on 32bit windows is you can only have an address space per process of 1.8gb , other OS's such as Unix's and Linux's do a much better job and get you near the true 32bit limit Another problem is that you need a contiguous memory area for malloc/new on windows this is a big problem , Some of the reasons why this is an issue is that Windows has already eaten up a chunk of the available memory, not only with programs , services , dll's being loaded they sadly simply don't get then next serial memory address, they may be load smack bang in the middle of the address space, so straight away that can l half the size of the max malloc/new you can do. As you load more programs more dll's the longer windows is running the more fragmented the memory will get and the smaller the max malloc/new can create will get lower, the MAC's OS's are the best at handling this sort of thing and Linux is typically better than window's What you can try is all the normal traditional tips, only run [processes, services that absolutely need to etc see http://www.vis-sim.com/vega/vegafaq1.htm#f39 ( needs modernizing but the gist is valid) This use be a big problem back in the heyday of IRIX, it would load is system SO's(dll's) smack bang in the middle of memory the same for programs. What had to be do there was to force the system to load its libs either high or low and you has to rebase the loading address of all the SO's your program used. You can do a similar thing in Windows and for all your dll's to re-base and control were they load. If you do that the final trick is that as some as your application starts you need to create the large memory stuff straight away, otherwise your address space will get fragmented and your back to square one At my company we have to handle multi-terra byte imagery and have to use processes like I have described, so it can be done. you just need an engineer that knows this hard stuff, thankfully we have an engineer that does ;) and no you cannot have him ;) __ *Gordon Tomlinson * Email : [EMAIL PROTECTED] YIM/AIM : *gordon3dBrit* MSN IM : [EMAIL PROTECTED] Website : *www.vis-sim.com www.gordontomlinson.com* __ *From:* [EMAIL PROTECTED] [mailto: [EMAIL PROTECTED] *On Behalf Of *David Callu *Sent:* Wednesday, June 25, 2008 3:05 PM *To:* OpenSceneGraph Users *Subject:* Re: [osg-users] [Not OSG related question] Virtual memory management on Windows power linux Serge ;-). Regards David Callu 2008/6/25 Serge Lages [EMAIL PROTECTED]: Hi all, I have a question not related to OSG but I can't find any answer, and this is something that some of you probably knows. That's why I try here to find some help. Here is my problem : I have a big image database with some images larger than 1.5Go uncompressed, and I fail to load them (Win XP SP2 32bits with Visual Studio 8). My computer has 3Go of virtual memory and the option /3GB is activated on the system. In this document (page 13) : http://actes.sstic.org/SSTIC05/Vulnerabilites_et_gestion_des_limites_memoire/SSTIC05-article-Delalleau-Vulnerabilites_et_gestion_des_limites_memoire.pdf It says it's not possible to allocate more than 1.3Go in one call, and it's actually the limit where it crashs. If I do 2 allocations of 1Go each, it works, but 1 allocation of 1.4Go crashs... Has someone any idea if it's possible to change this limit ? My only hope will be to make smaller images, or even to develop under Linux ? :) Thanks in advance ! -- Serge Lages http://www.tharsis-software.com ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org -- Serge Lages http://www.tharsis-software.com ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] [Not OSG related question] Virtual memory management on Windows
power linux Serge ;-). Regards David Callu 2008/6/25 Serge Lages [EMAIL PROTECTED]: Hi all, I have a question not related to OSG but I can't find any answer, and this is something that some of you probably knows. That's why I try here to find some help. Here is my problem : I have a big image database with some images larger than 1.5Go uncompressed, and I fail to load them (Win XP SP2 32bits with Visual Studio 8). My computer has 3Go of virtual memory and the option /3GB is activated on the system. In this document (page 13) : http://actes.sstic.org/SSTIC05/Vulnerabilites_et_gestion_des_limites_memoire/SSTIC05-article-Delalleau-Vulnerabilites_et_gestion_des_limites_memoire.pdf It says it's not possible to allocate more than 1.3Go in one call, and it's actually the limit where it crashs. If I do 2 allocations of 1Go each, it works, but 1 allocation of 1.4Go crashs... Has someone any idea if it's possible to change this limit ? My only hope will be to make smaller images, or even to develop under Linux ? :) Thanks in advance ! -- Serge Lages http://www.tharsis-software.com ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] [Not OSG related question] Virtual memory management on Windows
Hi There's many issues why you will struggle with this and no it's not just a windows issues it effects other OS's some do a better job off moving the issues forward but they will still crop up Simplest solution is to go to a 64bit OS with a good 8gb or more. There is another limitation you will hit on 32bit windows is you can only have an address space per process of 1.8gb , other OS's such as Unix's and Linux's do a much better job and get you near the true 32bit limit Another problem is that you need a contiguous memory area for malloc/new on windows this is a big problem , Some of the reasons why this is an issue is that Windows has already eaten up a chunk of the available memory, not only with programs , services , dll's being loaded they sadly simply don't get then next serial memory address, they may be load smack bang in the middle of the address space, so straight away that can l half the size of the max malloc/new you can do. As you load more programs more dll's the longer windows is running the more fragmented the memory will get and the smaller the max malloc/new can create will get lower, the MAC's OS's are the best at handling this sort of thing and Linux is typically better than window's What you can try is all the normal traditional tips, only run [processes, services that absolutely need to etc see http://www.vis-sim.com/vega/vegafaq1.htm#f39 ( needs modernizing but the gist is valid) This use be a big problem back in the heyday of IRIX, it would load is system SO's(dll's) smack bang in the middle of memory the same for programs. What had to be do there was to force the system to load its libs either high or low and you has to rebase the loading address of all the SO's your program used. You can do a similar thing in Windows and for all your dll's to re-base and control were they load. If you do that the final trick is that as some as your application starts you need to create the large memory stuff straight away, otherwise your address space will get fragmented and your back to square one At my company we have to handle multi-terra byte imagery and have to use processes like I have described, so it can be done. you just need an engineer that knows this hard stuff, thankfully we have an engineer that does ;) and no you cannot have him ;) __ Gordon Tomlinson Email : mailto:[EMAIL PROTECTED] [EMAIL PROTECTED] YIM/AIM : gordon3dBrit MSN IM : mailto:[EMAIL PROTECTED] [EMAIL PROTECTED] Website : http://www.vis-sim.com www.vis-sim.com http://www.gordontomlinson.com www.gordontomlinson.com __ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Callu Sent: Wednesday, June 25, 2008 3:05 PM To: OpenSceneGraph Users Subject: Re: [osg-users] [Not OSG related question] Virtual memory management on Windows power linux Serge ;-). Regards David Callu 2008/6/25 Serge Lages [EMAIL PROTECTED]: Hi all, I have a question not related to OSG but I can't find any answer, and this is something that some of you probably knows. That's why I try here to find some help. Here is my problem : I have a big image database with some images larger than 1.5Go uncompressed, and I fail to load them (Win XP SP2 32bits with Visual Studio 8). My computer has 3Go of virtual memory and the option /3GB is activated on the system. In this document (page 13) : http://actes.sstic.org/SSTIC05/Vulnerabilites_et_gestion_des_limites_memoire /SSTIC05-article-Delalleau-Vulnerabilites_et_gestion_des_limites_memoire.pdf It says it's not possible to allocate more than 1.3Go in one call, and it's actually the limit where it crashs. If I do 2 allocations of 1Go each, it works, but 1 allocation of 1.4Go crashs... Has someone any idea if it's possible to change this limit ? My only hope will be to make smaller images, or even to develop under Linux ? :) Thanks in advance ! -- Serge Lages http://www.tharsis-software.com ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
