Hi Todd, Based on the vmstat output I would guess that mpiblast is running normally on your system. To be absolutely certain, you could compare the user/system time split of mpiblast to that of ncbi's blastall when searching the same query against one of the database fragments. Just use -d /full/path/to/db_frag.### where ### is the fragment number. No .nal alias file necessary. By the way, are you using mpiblast 1.4 or pio or something else? As I recall, blastn can be very I/O intensive, and if the database is already in the filesystem buffer-cache then disk I/O becomes memory I/O. memory I/O is much faster than disk I/O, but still takes time, and that may be what's happening during all the system time. i think the -s option to vmstat can give paging statistics, at least on newer linux kernels.
It's hard to say what the expected runtime for your particular query set would be since blast runtimes are very dependent on the level of sequence identity shared by the query and database. -Aaron Heywood, Todd wrote: > Aaron, > > Thanks for the reply. I am referring to the workers, which all seem to > be chugging along. They all seem to have plenty of memory, and I'm not > seeing local I/O or network traffic (iostat and netstat). The mpiPBLAST > search is blastn on the nt database, with 100 fragments, and I asked for > 102 MPI tasks. The query file is 92MB and I'm running on dual-CPU > dual-core Opterons (anyone have any prediction of how long this should > take with everything running optimally?). > > So you can see what I'm talking about, here's the vmstat output for 3 > nodes: (1) 2GB node with 1 worker, (2) 4GB node with 3 workers, and (3) > 8GB node with 4 workers. > > I'm using a beta-version of Open-MPI 1.2, and I had the thought that > maybe Open-MPI might be causing mis-representation of user v.s. system > time. > > Thanks, > > Todd > > > > > procs -----------memory---------- ---swap-- -----io---- --system-- > ----cpu---- > r b swpd free buff cache si so bi bo in cs us sy > id wa > 1 0 0 606540 88732 977100 0 0 0 1 2 2 1 3 > 96 0 > 1 0 0 606540 88732 977100 0 0 0 0 1004 20 2 23 > 75 0 > 1 0 0 606396 88732 977100 0 0 0 0 1005 24 3 22 > 75 0 > 1 0 0 606396 88732 977100 0 0 0 14 1006 21 3 23 > 75 0 > > > procs -----------memory---------- ---swap-- -----io---- --system-- > ----cpu---- > r b swpd free buff cache si so bi bo in cs us sy > id wa > 4 1 4 2130840 85972 988492 0 0 0 1 2 2 1 > 9 90 0 > 3 0 4 2130840 85972 988492 0 0 0 88 1008 27 7 > 68 24 1 > 3 0 4 2130840 85972 988492 0 0 0 0 1004 19 7 > 68 25 0 > 3 0 4 2130848 85972 988492 0 0 0 0 1003 18 8 > 67 25 0 > > > > procs -----------memory---------- ---swap-- -----io---- --system-- > ----cpu---- > r b swpd free buff cache si so bi bo in cs us sy > id wa > 4 1 0 4794384 96300 1384740 0 0 0 1 2 0 2 > 12 86 0 > 4 0 0 4794384 96300 1384740 0 0 0 90 1039 340 8 > 92 0 0 > 4 0 0 4794344 96300 1384740 0 0 0 0 1064 424 10 > 90 0 0 > 4 0 0 4794352 96300 1384740 0 0 0 0 1056 360 9 > 91 0 0 > > > > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Aaron > Darling > Sent: Friday, March 02, 2007 1:31 AM > To: [email protected] > Subject: Re: [Mpiblast-users] cpu time partitioning > > Hi Todd... > > Heywood, Todd wrote: > >> I'm just starting to run mpiBLAST jobs, and I'm noticing, using >> > vmstat, that the vast majority of CPU time is system time, not user > time. Is this normal? > >> Thanks. >> >> Todd Heywood >> >> > > Whether that's normal depends on which mpiblast process you were > monitoring and a few other details. mpiblast processes have three > roles: writer, scheduler, and worker. For a perfectly load-balanced > search, the workers should probably be more user-time than system time. > > The writer can have substantial system time because it does lots of disk > > I/O and communication, and what little CPU time the scheduler uses will > probably be system time for communication overhead. Depending on the > search type and database, i.e. blastn, blastp, tblastx etc, the relative > > amounts of system time to user time for workers can change > substantially. blastn can be extremely I/O intensive, while tblastx > tends to be compute intensive. > > -Aaron > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Mpiblast-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/mpiblast-users
