One important correction: none of what you described uses openmp (OMP), it is all mpi. In the most recent Wien2k openmp is controlled by omp_X commands in the .machines file.
_____ Professor Laurence Marks "Research is to see what everybody else has seen, and to think what nobody else has thought", Albert Szent-Gyorgi www.numis.northwestern.edu On Thu, Oct 15, 2020, 13:45 Christian Søndergaard Pedersen <chr...@dtu.dk> wrote: > Deat Professors Blaha and Marks > > > Thank you kindly for your explanations, the code is running smoothly - and > efficiently! - now. For the sake of future WIEN2k newbies, I will summarize > the mistakes I made; hopefully this can save someone else's time: > > > 1: calling 'mpirun run_lapw -p' > > As Professor Marks explained, calling mpirun explicitly led to overload of > the compute node I was using. mpirun spawned one process per CPU on the > node, each of which spawned additional processes as specified in the > .machines file. The correct way is calling 'run_lapw -p'. > > > 2: Issuing only one job in the .machines file, regardless of how many > nodes/cores the job was using. For instance, for a Xeon16 node, I would > write: > > > 1:node1:4 > > > which uses 4 cores for lapw1/lapw2 while leaving the remaining 12 cores > idle. I corrected this to: > > > 1:node1:4 > > 1:node1:4 > > 1:node1:4 > > 1:node1:4 > > > ... which works, and which is explained in the example for OMP on page 86 > of the manual. > > > Best regards > > Christian > ------------------------------ > *Fra:* Wien <wien-boun...@zeus.theochem.tuwien.ac.at> på vegne af > Laurence Marks <laurence.ma...@gmail.com> > *Sendt:* 15. oktober 2020 17:15:30 > *Til:* A Mailing list for WIEN2k users > *Emne:* Re: [Wien] .machines for several nodes > > Let me expand why not to use mpirun yourself, unless you are doing > something "special" > > Wien2k uses the .machines file to setup how to use mpi and (in the most > recent versions) omp. As discussed by Peter, in most cases mpi is best with > close to square matrices and often powers of 2. OMP is good for having 2-4 > cores collaborate, not more. Depending upon your architecture OMP may be > better than mpi or worse. (On my nodes mpi is always best; I know on some > of Peter's that OMP is better.) > > The code internally sets the number of threads to use (for omp), and will > call mpirun or its equivalent depending upon what you have in > parallel_options. While many codes/programs are structured so they operate > mpi via "mpirun MyCode", Wien2k does not. The danger is that you will end > up with multiple copies of run_lapw running which is not what you want. > > There might be special cases where you would want to use "mpirun run_lapw" > to remotely start a single version, but until you know how to use Wien2k do > not go this complicated, it is likely to create problems. > > > On Thu, Oct 15, 2020 at 5:01 AM Laurence Marks <laurence.ma...@gmail.com> > wrote: > >> As an addendum to what Peter said, "mpirun run_lapw" is totally wrong. >> Remove the mpirun. >> >> _____ >> Professor Laurence Marks >> "Research is to see what everybody else has seen, and to think what >> nobody else has thought", Albert Szent-Gyorgi >> www.numis.northwestern.edu >> >> On Thu, Oct 15, 2020, 03:35 Peter Blaha <pbl...@theochem.tuwien.ac.at> >> wrote: >> >>> Well, 99% cpu efficiency does not mean that you run efficiently, but my >>> estimat is that you run at least 2 times slower than what is possible. >>> >>> Anyway, please save the dayfile and compare the wall time of the >>> different parts with a different setup. >>> >>> At least now we know that you have 24 cores/node. So the lapw0/dstart >>> lines are perfectly ok. >>> >>> However, lapw1 you run on 3 mpi cores. This is "maximally inefficient". >>> This gives a division of your matrix into 3x1, but it should be as close >>> as possible to an even decomposition. So 4x4=16 or 8x8=64 cores is >>> optimal. With your 24 cores and 96 atom/cell I'd probably go for 12 >>> cores in mpi and 2-kparallel jobs per node: >>> >>> 1:x073:12 >>> 1:x082:12 >>> 1:x073:12 >>> 1:x082:12 >>> >>> Maybe one can even overload the nodes a bit using 16 instead of 12 >>> cores, but this could be dangerous on some machines because of your >>> admins might have forced cpu-binding, .... (You can even change the >>> .machines file (12-->16) "by hand" while your job is running (and maybe >>> change it back once you have seen whether timing is better or worse). >>> >>> In any case, compare the timeings in the dayfile in order to find the >>> optimal setup. >>> _______________________________________________ >>> Wien mailing list >>> Wien@zeus.theochem.tuwien.ac.at >>> >>> https://urldefense.com/v3/__http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien__;!!Dq0X2DkFhyF93HkjWTBQKhk!A8sB9-qFfbOGiCLPnA6iSE84ZZQy6mW4l0zuzz3NpWm1Wmn2GKqNPUMWg1UBjmQOGPID6g$ >>> SEARCH the MAILING-LIST at: >>> https://urldefense.com/v3/__http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html__;!!Dq0X2DkFhyF93HkjWTBQKhk!A8sB9-qFfbOGiCLPnA6iSE84ZZQy6mW4l0zuzz3NpWm1Wmn2GKqNPUMWg1UBjmSyxhK3Ng$ >>> >> > > -- > Professor Laurence Marks > Department of Materials Science and Engineering > Northwestern University > www.numis.northwestern.edu > Corrosion in 4D: www.numis.northwestern.edu/MURI > Co-Editor, Acta Cryst A > "Research is to see what everybody else has seen, and to think what nobody > else has thought" > Albert Szent-Gyorgi > _______________________________________________ > Wien mailing list > Wien@zeus.theochem.tuwien.ac.at > > https://urldefense.com/v3/__http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien__;!!Dq0X2DkFhyF93HkjWTBQKhk!E486wu9mjEojVuBx9BmE4_SFsnfFc15_HVRm2cgBlrXVqtPX4yfLBohlMh65-_6IQmAb0g$ > SEARCH the MAILING-LIST at: > https://urldefense.com/v3/__http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html__;!!Dq0X2DkFhyF93HkjWTBQKhk!E486wu9mjEojVuBx9BmE4_SFsnfFc15_HVRm2cgBlrXVqtPX4yfLBohlMh65-_6nK9GYkg$ >
_______________________________________________ Wien mailing list Wien@zeus.theochem.tuwien.ac.at http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien SEARCH the MAILING-LIST at: http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html
