Deat Professors Blaha and Marks
Thank you kindly for your explanations, the code is running smoothly - and efficiently! - now. For the sake of future WIEN2k newbies, I will summarize the mistakes I made; hopefully this can save someone else's time: 1: calling 'mpirun run_lapw -p' As Professor Marks explained, calling mpirun explicitly led to overload of the compute node I was using. mpirun spawned one process per CPU on the node, each of which spawned additional processes as specified in the .machines file. The correct way is calling 'run_lapw -p'. 2: Issuing only one job in the .machines file, regardless of how many nodes/cores the job was using. For instance, for a Xeon16 node, I would write: 1:node1:4 which uses 4 cores for lapw1/lapw2 while leaving the remaining 12 cores idle. I corrected this to: 1:node1:4 1:node1:4 1:node1:4 1:node1:4 ... which works, and which is explained in the example for OMP on page 86 of the manual. Best regards Christian ________________________________ Fra: Wien <wien-boun...@zeus.theochem.tuwien.ac.at> på vegne af Laurence Marks <laurence.ma...@gmail.com> Sendt: 15. oktober 2020 17:15:30 Til: A Mailing list for WIEN2k users Emne: Re: [Wien] .machines for several nodes Let me expand why not to use mpirun yourself, unless you are doing something "special" Wien2k uses the .machines file to setup how to use mpi and (in the most recent versions) omp. As discussed by Peter, in most cases mpi is best with close to square matrices and often powers of 2. OMP is good for having 2-4 cores collaborate, not more. Depending upon your architecture OMP may be better than mpi or worse. (On my nodes mpi is always best; I know on some of Peter's that OMP is better.) The code internally sets the number of threads to use (for omp), and will call mpirun or its equivalent depending upon what you have in parallel_options. While many codes/programs are structured so they operate mpi via "mpirun MyCode", Wien2k does not. The danger is that you will end up with multiple copies of run_lapw running which is not what you want. There might be special cases where you would want to use "mpirun run_lapw" to remotely start a single version, but until you know how to use Wien2k do not go this complicated, it is likely to create problems. On Thu, Oct 15, 2020 at 5:01 AM Laurence Marks <laurence.ma...@gmail.com<mailto:laurence.ma...@gmail.com>> wrote: As an addendum to what Peter said, "mpirun run_lapw" is totally wrong. Remove the mpirun. _____ Professor Laurence Marks "Research is to see what everybody else has seen, and to think what nobody else has thought", Albert Szent-Gyorgi www.numis.northwestern.edu<http://www.numis.northwestern.edu> On Thu, Oct 15, 2020, 03:35 Peter Blaha <pbl...@theochem.tuwien.ac.at<mailto:pbl...@theochem.tuwien.ac.at>> wrote: Well, 99% cpu efficiency does not mean that you run efficiently, but my estimat is that you run at least 2 times slower than what is possible. Anyway, please save the dayfile and compare the wall time of the different parts with a different setup. At least now we know that you have 24 cores/node. So the lapw0/dstart lines are perfectly ok. However, lapw1 you run on 3 mpi cores. This is "maximally inefficient". This gives a division of your matrix into 3x1, but it should be as close as possible to an even decomposition. So 4x4=16 or 8x8=64 cores is optimal. With your 24 cores and 96 atom/cell I'd probably go for 12 cores in mpi and 2-kparallel jobs per node: 1:x073:12 1:x082:12 1:x073:12 1:x082:12 Maybe one can even overload the nodes a bit using 16 instead of 12 cores, but this could be dangerous on some machines because of your admins might have forced cpu-binding, .... (You can even change the .machines file (12-->16) "by hand" while your job is running (and maybe change it back once you have seen whether timing is better or worse). In any case, compare the timeings in the dayfile in order to find the optimal setup. _______________________________________________ Wien mailing list Wien@zeus.theochem.tuwien.ac.at<mailto:Wien@zeus.theochem.tuwien.ac.at> https://urldefense.com/v3/__http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien__;!!Dq0X2DkFhyF93HkjWTBQKhk!A8sB9-qFfbOGiCLPnA6iSE84ZZQy6mW4l0zuzz3NpWm1Wmn2GKqNPUMWg1UBjmQOGPID6g$ SEARCH the MAILING-LIST at: https://urldefense.com/v3/__http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html__;!!Dq0X2DkFhyF93HkjWTBQKhk!A8sB9-qFfbOGiCLPnA6iSE84ZZQy6mW4l0zuzz3NpWm1Wmn2GKqNPUMWg1UBjmSyxhK3Ng$ -- Professor Laurence Marks Department of Materials Science and Engineering Northwestern University www.numis.northwestern.edu<http://www.numis.northwestern.edu/> Corrosion in 4D: www.numis.northwestern.edu/MURI<http://www.numis.northwestern.edu/MURI> Co-Editor, Acta Cryst A "Research is to see what everybody else has seen, and to think what nobody else has thought" Albert Szent-Gyorgi
_______________________________________________ Wien mailing list Wien@zeus.theochem.tuwien.ac.at http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien SEARCH the MAILING-LIST at: http://www.mail-archive.com/wien@zeus.theochem.tuwien.ac.at/index.html