Re: [OMPI users] Tmpdir work for first process only

2007-11-15 Thread Aurelien Bouteiller
Hi Clement, First, if you run 400 jobs on 16 nodes you will end up with around 32 processes on each nodes. Depending on the memory footprint of the application it will fail because of memory exhaustion. Usually I am able to oversubscribe up to 64 NAS class B processes on 2GB, and less tha

Re: [OMPI users] Tmpdir work for first process only

2007-11-15 Thread Clement Kam Man Chu
Jeff Squyres wrote: Thanks for your reply. I am using pbs job scheduler and I reqested 16 cpus to run 400 processes, but I don't how many processes are allocated on each cpus. Do you think it is a problem? Clement Are you running all of these processes on the same machine, or multiple dif

Re: [OMPI users] Tmpdir work for first process only

2007-11-15 Thread Jeff Squyres
Are you running all of these processes on the same machine, or multiple different machines? If you're running 400 processes on the same machine, it may well be that you are simply running out of memory or other OS resources. In particular, I've never seem iof fail that way before (iof is o

Re: [OMPI users] Tmpdir work for first process only

2007-11-15 Thread Clement Kam Man Chu
Hi, I have configured out why the tmpdir parameter works for the first process. I got another problem if I tried to run 400 processes (no problem if under 400 processes). I got an error "ORTE_ERROR_LOG: Out of resource in file base/iof_base_setup.c at line 106". I attached the message as belo

[OMPI users] Tmpdir work for first process only

2007-11-14 Thread Clement Kam Man Chu
Hi, I am using openmpi 1.2.3 under ia64 machine. I typed "mpirun -d --tmpdir /home/565/kxc565/tmpdir -mca btl sm -np 400 ./testprogram". I found only the first process can use my parameter setting to store tmp file, but the second process used its default setting to store tmp file in /tmp dir