Re: [CentOS] High memory needs

2012-10-01 Thread Jérémie Dubois-Lacoste
> Anyone using VIRT to make decisions about resource utilization is > completely ignorant of its function. I agree. I don't understand why Sun Grid Engine does exactly that. I posted a full explanation of this problem and the solution we used here: https://www.centos.org/modules/newbb/viewtopic.ph

Re: [CentOS] High memory needs

2012-09-29 Thread Gordon Messmer
On 09/27/2012 09:34 AM, Les Mikesell wrote: > That's an interesting difference on its own, since the underlying > files are about 95M and 54M respectively. Does the 32 bit kernel use > some tricks to sparsely map files where the 64 bit one does it > directly with page tables? No, it's because gli

Re: [CentOS] High memory needs [SOLVED]

2012-09-28 Thread Jérémie Dubois-Lacoste
> did you test something like this? > > ls -al /usr/lib/locale/locale-archive > lrwxrwxrwx 1 root root 9 Sep 3 12:35 /usr/lib/locale/locale-archive -> > /dev/null Didn't test it. I thought that at least the languages set to be used must be accessible. Jeremie > > -- > LF > > >

Re: [CentOS] High memory needs [SOLVED]

2012-09-28 Thread Leon Fauster
Am 28.09.2012 um 16:10 schrieb Jérémie Dubois-Lacoste: > The solution that worked for us is a post-install fix that consists of > removing all languages that we don't use. We checked with "locale" > which languages are used on the system. Then we remove all the others: > $ localedef --list-archive

Re: [CentOS] High memory needs [SOLVED]

2012-09-28 Thread Jérémie Dubois-Lacoste
I finally found a solution to our problem. I think some people running like us a combination CentOS 64 bits\Sun Grid Engine, could encounter the same situation. Here is a detailed explanation, hope it can be useful to someone! The file /usr/lib/locale/locale-archive is a memory-mapped file used by

Re: [CentOS] High memory needs

2012-09-27 Thread Les Mikesell
On Thu, Sep 27, 2012 at 10:46 AM, Gordon Messmer wrote: > >> I understand it may not be very precise, however I still don't >> understant the difference compared to other x64 ditributions, >> under CentOS the value is 7 times higher! This might explain it: https://bugzilla.redhat.com/show_bug.cgi

Re: [CentOS] High memory needs

2012-09-27 Thread Gordon Messmer
On 09/27/2012 01:57 AM, Jérémie Dubois-Lacoste wrote: > We have a computing cluster running Sun Grid Engine, which > considers this value to check if a process exceeds the memory > limit or not. So somehow I'm bound to consider it. > > I installed a machine from scratch with CentOS 6.2 x64, nothing

Re: [CentOS] High memory needs

2012-09-27 Thread Jérémie Dubois-Lacoste
We have a computing cluster running Sun Grid Engine, which considers this value to check if a process exceeds the memory limit or not. So somehow I'm bound to consider it. I installed a machine from scratch with CentOS 6.2 x64, nothing else, I open a terminal, I run this simple bash script and VIR

Re: [CentOS] High memory needs

2012-09-26 Thread Gordon Messmer
On 09/26/2012 09:14 AM, Jérémie Dubois-Lacoste wrote: > 1. Run a python script and check the memory that > it requires (field "VIRT" of the "top" command). Don't use VIRT as a reference for memory used. RES is a better indication, but even that won't tell you anything useful about shared memory

Re: [CentOS] High memory needs

2012-09-26 Thread Jérémie Dubois-Lacoste
You may have misunderstood. The detailed number I gave are obtained on distributions that are not CentOS, and ok, it can makes sense between 32 and 64 bits. But on CentOS 6.2, 64bits, I obtain: SH: 103MB PYTHON: 114MB R:200MB This is from a freshly installed CentOS 6.2 machine

Re: [CentOS] High memory needs

2012-09-26 Thread Michael Hennebry
On Wed, 26 Sep 2012, m.r...@5-cent.us wrote: Jérémie Dubois-Lacoste wrote: Python script: Avg Min Max 32 bits8500 5004 11132 64 bits32800 3 36336 8500 * 2 = 17000 5004 * 2 = 10007 11132 * 2 = 22264 So that ranges from 2-2.5 larger. Hu

Re: [CentOS] High memory needs

2012-09-26 Thread m . roth
Jérémie Dubois-Lacoste wrote: > Dear All, > > We recently reinstalled our computing cluster. We were using CentOS > 5.3 (32 bits). It is now CentOS 6.3 (64 bits), installed from the > CentOS 6.2 x64 CD, then upgraded to 6.3. > > We have some issues with the memory needs of our running jobs. They

Re: [CentOS] High memory needs

2012-09-26 Thread Jérémie Dubois-Lacoste
Hm, interesting suggestion. But didn't change anything. :( Thanks anyway, Jérémie 2012/9/26 Adrian Sevcenco : > On 09/26/12 19:14, Jérémie Dubois-Lacoste wrote: >> >> Dear All, > > Hi! > > >> We recently reinstalled our computing cluster. We were using CentOS >> 5.3 (32 bits). It is now CentO

Re: [CentOS] High memory needs

2012-09-26 Thread Adrian Sevcenco
On 09/26/12 19:14, Jérémie Dubois-Lacoste wrote: Dear All, Hi! We recently reinstalled our computing cluster. We were using CentOS 5.3 (32 bits). It is now CentOS 6.3 (64 bits), installed from the CentOS 6.2 x64 CD, then upgraded to 6.3. We have some issues with the memory needs of our runn

[CentOS] High memory needs

2012-09-26 Thread Jérémie Dubois-Lacoste
Dear All, We recently reinstalled our computing cluster. We were using CentOS 5.3 (32 bits). It is now CentOS 6.3 (64 bits), installed from the CentOS 6.2 x64 CD, then upgraded to 6.3. We have some issues with the memory needs of our running jobs. They require much more than before, it may be d