Hi Carlos, See my answers below.
I also CC'ed my answer to the mailing list. On 14/09/12 02:01 PM, Carlos P. Sosa wrote: > Hi Sebastien, > > > > Quick question. Do you make use of parallel file systems, e.g., lustre with > RAY? > Ray uses <stdlib.h> from the standard C library and <fstream> from C++ 1998. These function calls will perform VFS (virtual file system) system calls in Linux. If the file is in a lustre mount point in the VFS, then the VFS system calls will be deleguated to the lustre linux kernel module. If the file is in a IBM gpfs mount point, it will be deleguated to the gpfs linux kernel module, and so on. For Lustre: When each MPI rank writes a file, it obviously uses the parallel system because file blocks are distributed across OST (object storage targets). One case that would be useful is the use of MPI I/O. In this setting, many processes can write to the same file, at different offsets, simultaneously. At the moment, parallel file input/output operations are not really useful for Ray, except in the initial step of loading files because a file can be loaded partially by many processes. Ray does not use intermediate files -- it is all in-memory and message passing. Linux cache blocks using memory pages anyway on both the OSS (object storage server) and on the Lustre LOV (Logical Object Volume manager) client, so I am not sure that reading files in parallel would be that of an improvement for Ray. > > > Thanks > > > > *Carlos P Sosa*** ** > > 380 Jackson Street, Suite 210 | Saint Paul, MN 55101 | [email protected] > <mailto:[email protected]> |www.cray.com <http://www.cray.com/> > > > > *Description: Description: cid:[email protected]* > <http://www.cray.com/> Description: Description: facebook > <http://www.facebook.com/crayinc> Description: Description: twitter > <http://www.twitter.com/cray_inc> Description: Description: youtube > <http://www.youtube.com/user/crayvideo> > > > ------------------------------------------------------------------------------ Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html _______________________________________________ Denovoassembler-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/denovoassembler-users
