Dear Aron, Actually the file is an output of another application, and is stored in a simple text file. I have read about MPIIO, however couldn't figure out if it is a parallel file reader or not.
Have you ever tried or worked on a parallel file reader to a common global matrix in PETSc, where each processor has part of the file? Thank you Regards, Zuhair Khayyat On Sat, Jan 8, 2011 at 8:42 PM, Aron Ahmadia <aron.ahmadia at kaust.edu.sa>wrote: > if the file is stored in the PETSc format, you can use PETSc to pull the > file in using MPIIO, which should be (hopefully) faster. > > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Viewer/PetscViewerBinarySetMPIIO.html > > Feel free to pop these questions to petsc-users (cc'd), there are more of > them and they usually know more than me :) > > -Aron > > > On Sat, Jan 8, 2011 at 2:21 PM, Zuhair Khayyat < > zuhair.khayyat at kaust.edu.sa> wrote: > >> Dear Aron, >> >> Currently I am using PETSc for my research to implement a graph mining >> algorithm on a large cluster, and I would like ask you some questions due to >> your experience with this tool. >> >> Have you ever tried to optimize allocating a very large matrix from input >> file in parallel? I have a very large file (around 10 GB) and it takes too >> long to allocate the matrix through the main node. I am planning to >> implement a distributed parallel file reader that split the original file >> and make each node reads it separately into a common matrix. >> >> Is there and other tools that are comparable to PETSc in which they have >> parallel file reader? >> >> Thank you for your help >> >> Regards, >> Zuhair Khayyat >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110109/84955856/attachment.htm>
