I did some tests. For 1000 Si atoms, I use 2010 bands because I need to
get the band gap value; moreover, being a cluster, the surface states of
the truncated bonds might close the gap, especially at the first steps
of the geometry optimization, so it's better I use few empty bands. I
managed to run the calculation by using 10 nodes and a max of 40 cores
per node. My question now is: can you suggest me optimal command line
options and/or input settings to speed up the calculation? And, if
possible, also to reduce the number of nodes? The relevant parameters in
the input file are the following:
input_dft= 'pz'
ecutwfc= 25
occupations= 'smearing'
smearing= 'cold'
degauss= 0.05 ! I know it's quite large, but necessary to stabilize
the SCF at this preliminary stage (no geometry step done yet)
nbnd= 2010
diagonalization= 'ppcg'
mixing_mode= 'plain'
mixing_beta= 0.4
The actual time spent per scf cycle is about 33 minutes. I use QE v. 7.3
compiled with openmpi and scalapack. I have access to the intel
compilers too but I did some tests and the difference is just tens of
seconds. I have only the Gamma point; please, here you have some info
about the grid and the estimated RAM usage:
Dense grid: 24616397 G-vectors FFT dimensions: ( 375, 375, 375)
Dynamical RAM for wfc: 235.91 MB
Dynamical RAM for wfc (w. buffer): 235.91 MB
Dynamical RAM for str. fact: 0.94 MB
Dynamical RAM for local pot: 0.00 MB
Dynamical RAM for nlocal pot: 2112.67 MB
Dynamical RAM for qrad: 0.80 MB
Dynamical RAM for rho,v,vnew: 6.04 MB
Dynamical RAM for rhoin: 2.01 MB
Dynamical RAM for rho*nmix: 15.03 MB
Dynamical RAM for G-vectors: 3.99 MB
Dynamical RAM for h,s,v(r/c): 0.46 MB
Dynamical RAM for <psi|beta>: 552.06 MB
Dynamical RAM for wfcinit/wfcrot: 1305.21 MB
Estimated static dynamical RAM per process > 2.31 GB
Estimated max dynamical RAM per process > 3.60 GB
Estimated total dynamical RAM > 1441.34 GB
Thanks a lot in advance for your kind help.
All the best
Antonio
On 10. 05. 24 12:01, Paolo Giannozzi wrote:
On 5/10/24 08:58, Antonio Cammarata via users wrote:
pw.x -nk 1 -nt 1 -nb 1 -nd 768 -inp qe.in > qe.out
too many processors for linear-algebra parallelization. 1000 Si atoms
= 2000 bands (assuming an insulator with no spin polarization). Use a
few tens of processors at most
"some processors have no G-vectors for symmetrization".
which sounds strange to me: with the Gamma point symmetrization is not
even needed
Dense grid: 30754065 G-vectors FFT dimensions: ( 400, 400, 400)
This is what a 256-atom Si supercell with 30 Ry cutoff yields:
Dense grid: 825897 G-vectors FFT dimensions: ( 162, 162, 162)
I guess you may reduce the size of your supercell
Paolo
Dynamical RAM for wfc: 153.50 MB
Dynamical RAM for wfc (w. buffer): 153.50 MB
Dynamical RAM for str. fact: 0.61 MB
Dynamical RAM for local pot: 0.00 MB
Dynamical RAM for nlocal pot: 1374.66 MB
Dynamical RAM for qrad: 0.87 MB
Dynamical RAM for rho,v,vnew: 5.50 MB
Dynamical RAM for rhoin: 1.83 MB
Dynamical RAM for rho*nmix: 9.78 MB
Dynamical RAM for G-vectors: 2.60 MB
Dynamical RAM for h,s,v(r/c): 0.25 MB
Dynamical RAM for <psi|beta>: 552.06 MB
Dynamical RAM for wfcinit/wfcrot: 977.20 MB
Estimated static dynamical RAM per process > 1.51 GB
Estimated max dynamical RAM per process > 2.47 GB
Estimated total dynamical RAM > 1900.41 GB
I managed to run the simulation with 512 atoms, cg diagonalization
and 3 nodes on the same machine with command line
pw.x -nk 1 -nt 1 -nd 484 -inp qe.in > qe.out
Please, do you have any suggestion on how to set optimal
parallelization parameters to avoid the memory issue and run the
calculation? I am also planning to run simulations on nanoclusters
with more than 1000 atoms.
Thanks a lot in advance for your kind help.
Antonio
--
_______________________________________________
Antonio Cammarata, PhD in Physics
Associate Professor in Applied Physics
Advanced Materials Group
Department of Control Engineering - KN:G-204
Faculty of Electrical Engineering
Czech Technical University in Prague
Karlovo Náměstí, 13
121 35, Prague 2, Czech Republic
Phone: +420 224 35 5711
Fax: +420 224 91 8646
ORCID: orcid.org/0000-0002-5691-0682
WoS ResearcherID: A-4883-2014
_______________________________________________
The Quantum ESPRESSO community stands by the Ukrainian
people and expresses its concerns about the devastating
effects that the Russian military offensive has on their
country and on the free and peaceful scientific, cultural,
and economic cooperation amongst peoples
_______________________________________________
Quantum ESPRESSO is supported by MaX (www.max-centre.eu)
users mailing list users@lists.quantum-espresso.org
https://lists.quantum-espresso.org/mailman/listinfo/users