Thanks for the ideas. I did finally end up getting this working by sending back to the master process. It's quite ugly, and added a good bit of MPI to the code, but it works for now, and I will revisit this later. I am not sure what the file system is, I think it is XFS, but I don't know much about why this has an effect on the output - just the way files can be opened at once or something?
I did have to end up using an MPI Data type, because this 3D domain was strided nicely in X, but not the other dimensions. The domain is larger in Z, so I wanted to order my loops such that Z is the innermost. This helped cut down some of the MPI overhead. It would have been nice to avoid this, but I could not think of the way to do it, and still have all of the computes working on the largest section of data possible. Derek -----Original Message----- From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Ricardo Reis Sent: Monday, April 05, 2010 3:20 PM To: Open MPI Users Subject: Re: [OMPI users] Best way to reduce 3D array On Mon, 5 Apr 2010, Rob Latham wrote: > On Tue, Mar 30, 2010 at 11:51:39PM +0100, Ricardo Reis wrote: >> >> If using the master/slace IO model, would it be better to cicle >> through all the process and each one would write it's part of the >> array into the file. This file would be open in "stream" mode... >> >> like >> >> do p=0,nprocs-1 >> >> if(my_rank.eq.i)then >> >> openfile (append mode) >> write_to_file >> closefile >> >> endif >> >> call MPI_Barrier(world,ierr) >> >> enddo > > Note that there's no guarantee of the order here, though. Nothing > prevents rank 30 from hitting that loop before rank 2 does. To ensure don't they all have to hit the same Barrier? I think that will ensure order in this business... or am I being blind to something? I will agree, though, this is not the best solution to do it. I use this kind of arrangment when I'm desperate to do some prinf kind of debugging and want it ordered by process. Never had a problem with it. I mean, I assume there is some sort of sync before the do cycle starts. cheers! Ricardo Reis 'Non Serviam' PhD candidate @ Lasef Computational Fluid Dynamics, High Performance Computing, Turbulence http://www.lasef.ist.utl.pt Cultural Instigator @ Rádio Zero http://www.radiozero.pt Keep them Flying! Ajude a/help Aero Fénix! http://www.aeronauta.com/aero.fenix http://www.flickr.com/photos/rreis/ < sent with alpine 2.00 >
