[QE-users] [QE-GPU] Running GPU calculations optimally
Dear all, I am trying to use Quantum ESPRESSO v 7.1 with GPU acceleration. However, since I am new to this field, I need to learn about the flags and variables that could be exported to achieve optimum performance. Here is a bunch of variables that were suggested to me: export GPU_FORCE_64BIT_PTR=0 export GPU_MAX_HEAP_SIZE=100 export GPU_USE_SYNC_OBJECTS=1 export GPU_MAX_ALLOC_PERCENT=100 export GPU_SINGLE_ALLOC_PERCENT=100 export OMP_NUM_THREADS=1 export MPI_PER_GPU=20 export ENABLE_MPS=false export LS_HYPERTHREAD=true I could not find the documentation about these variables, nor do I understand how these work. The GPUs are not being utilized fully. I say this since I tried to run some basic profiling, all of which indicate the same: [image: image.png] I am using a SLURM script to run my jobs. Every GPU node on the cluster has 2 Tesla V100-SXM2-16GB and 40CPU cores. The job is launched using "mpirun -np $SLURM_NTASKS pw.x -ndiag 1 -in input.in > output.out" Could anyone shed some light on the following: 1. What flags and variables could be set, and how do I decide which are relevant for my calculations? 2. Where can I read more about these variables and flags to test which ones are relevant for the calculations? 3. Can the mpirun command be modified for better performance? Any insights would be very helpful. Sincerely, *Anson Thomas* Research Scholar, Department of Chemistry, Indian Institute of Technology, Roorkee ___ The Quantum ESPRESSO community stands by the Ukrainian people and expresses its concerns about the devastating effects that the Russian military offensive has on their country and on the free and peaceful scientific, cultural, and economic cooperation amongst peoples ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
[QE-users] What is the correct method to calculate ELF?
Dear all, I am trying to calculate the electron localization function (ELF) using Quantum ESPRESSO. However, I realize that it is not possible to correctly calculate it with either the Ultrasoft or the PAW pseudopotentials. With ultrasoft pseudos, the method is *not implemented fully*, while with the PAW pseudopotentials, as per an old answer here <https://narkive.com/10wFnvfE:2.689.38>, seems to indicate that ELF with PAW is also not implemented. As detailed in the answer here <https://narkive.com/F2GAQGzz:2.375.462>: "The correct calculation of the ELF with ultrasoft pseudopotentials or with PAW requires to compute the contribution from augmentation charges." Can someone provide some insights on how this contribution may be computed? Alternatively, is there another software that can calculate the ELF correctly for a periodic system? Any insights provided would be much helpful. Anson Thomas, Indian Institute of Technology, Roorkee ___ The Quantum ESPRESSO community stands by the Ukrainian people and expresses its concerns about the devastating effects that the Russian military offensive has on their country and on the free and peaceful scientific, cultural, and economic cooperation amongst peoples ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
[QE-users] What is the artifact of the Pt valence density referred to in example04 of the Environ module?
Dear QE Users, I am trying to model a metal oxide system with an adsorbate in Quantum Espresso with the Environ module. For this, I am following an example from the module itself, example04 which "shows how to use pw.x to model 2D periodic systems in contact with a continuum solvent." (from README file) The file further goes on to state that: "Note that the keyword solvent_mode='full' is mandatory in this calculation, *in order to remove the artifact of the Pt valence density, which presents a hole close to the position of ion, due to the missing core electrons*. In order to avoid the continuum solvent to enter such a hole, a quickly vanishing gaussian density is added at the position of the nuclei when defining the dielectric. The same problem is likely to occur in halogens and other transition metals." Can anyone give some idea as to what this artifact is in the Pt valence density, the hole close to the ion, and missing core electrons are? (The pseudopotentials used in the example are: O.pbe-rrkjus.UPF C.pbe-rrkjus.UPF Pt.pbe-nd-rrkjus.UPF) Any insights on this would be greatly appreciated. Sincerely, Anson *Anson Thomas* Research Scholar, IIT Roorkee ___ The Quantum ESPRESSO community stands by the Ukrainian people and expresses its concerns about the devastating effects that the Russian military offensive has on their country and on the free and peaceful scientific, cultural, and economic cooperation amongst peoples ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
Re: [QE-users] [EXT] Error in installation of Environ module
Dear Oliviero, Thank you for the response. Although I do desire to use GPUs for my calculations, as of now, I am only using GPUs. I started off using the gfortran compiler already present on my system (GNU Fortran (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)). However, it was giving many fatal errors and the setup finally came to an abrupt end. Part of the output has been shown below: Pre-compiling QE... mpif-sizeof.h:66.41: Included at mpif.h:63: Included at xclib_utils_and_para.f90:21: CHARACTER, DIMENSION(1,1,1,1,1,1,1,*)::x 1 Error: Array specification at (1) has more than 7 dimensions mpif-sizeof.h:72.41: Included at mpif.h:63: Included at xclib_utils_and_para.f90:21: CHARACTER, DIMENSION(1,1,1,1,1,1,1,1,*)::x 1 Error: Array specification at (1) has more than 7 dimensions mpif-sizeof.h:78.41: Included at mpif.h:63: Included at xclib_utils_and_para.f90:21: CHARACTER, DIMENSION(1,1,1,1,1,1,1,1,1,*)::x 1 Error: Array specification at (1) has more than 7 dimensions mpif-sizeof.h:84.41: Included at mpif.h:63: Included at xclib_utils_and_para.f90:21: CHARACTER, DIMENSION(1,1,1,1,1,1,1,1,1,1,*)::x 1 Using the Nvidia compilers resolved these problems but said that it was "Unable to open MODULE" in various files, as I mentioned in my question originally Do you have any suggestions? I would be happy to provide any extra information to help troubleshoot this issue. Sincerely, Anson IIT Roorkee -- Sent by *ANSON THOMAS* ___ The Quantum ESPRESSO community stands by the Ukrainian people and expresses its concerns about the devastating effects that the Russian military offensive has on their country and on the free and peaceful scientific, cultural, and economic cooperation amongst peoples ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
[QE-users] Error in installation of Environ module
Dear users,
I am trying to install Environ 2.0.3 with QE 6.8 on a Centos 7 machine. The
compilers being used are nvfortran, nvcc etc. provided as part of the
latest NVIDIA HPC SDK package.
I am, following the instruction given here (
https://environ.readthedocs.io/en/latest/install/configuring_install.html),
and accordingly, I ran ./configure from the QE root directory, then from
Environ directory. On running make install, I encounter the following error:
NVFORTRAN-F-0004-Unable to open MODULE file class_io.mod
(../../Environ/src/calculator.f90: 36)
NVFORTRAN/x86-64 Linux 22.5-0: compilation aborted
make[4]: *** [../../Environ/src/calculator.o] Error 2
make[4]: *** Waiting for unfinished jobs
NVFORTRAN-F-0004-Unable to open MODULE file class_setup.mod
(../../Environ/src/global_objects.f90: 32)
NVFORTRAN/x86-64 Linux 22.5-0: compilation aborted
make[4]: *** [../../Environ/src/global_objects.o] Error 2
NVFORTRAN-F-0004-Unable to open MODULE file env_char_ops.mod
(../../Environ/utils/io.f90: 32)
NVFORTRAN/x86-64 Linux 22.5-0: compilation aborted
make[4]: *** [../../Environ/utils/io.o] Error 2
NVFORTRAN-F-0004-Unable to open MODULE file class_io.mod
(../../Environ/src/destructor.f90: 33)
NVFORTRAN/x86-64 Linux 22.5-0: compilation aborted
make[4]: *** [../../Environ/src/destructor.o] Error 2
NVFORTRAN-F-0004-Unable to open MODULE file class_io.mod
(../../Environ/src/input.f90: 32)
NVFORTRAN/x86-64 Linux 22.5-0: compilation aborted
I also tried running configure with --enable-environment option, yet it is
giving the same errors. The {QE_ROOT}/Environ/install/QE_comp.log file is
attached here:
https://drive.google.com/file/d/1agxCVeh1pnuvH0DSsXLi67nxFy4QsBOJ/view?usp=sharing
Kindly help me with this installation.
Sincerely,
Anson
--
Sent by *ANSON THOMAS*
___
The Quantum ESPRESSO community stands by the Ukrainian
people and expresses its concerns about the devastating
effects that the Russian military offensive has on their
country and on the free and peaceful scientific, cultural,
and economic cooperation amongst peoples
___
Quantum ESPRESSO is supported by MaX (www.max-centre.eu)
users mailing list users@lists.quantum-espresso.org
https://lists.quantum-espresso.org/mailman/listinfo/users
[QE-users] [QE-Users]Bader charge analysis using USPP
Dear QE-Users, I wish to do a Bader charge analysis on my system. I have carried out the calculations using USPP I'm aware that Bader code requires the all-electron density which is generated using PAW. Is there any way to do it using USPP-generated data?? Any help would be appreciated. Thanks in advance -- *Anson Thomas* M.Sc. Chemistry, IIT Roorkee ___ The Quantum ESPRESSO community stands by the Ukrainian people and expresses its concerns about the devastating effects that the Russian military offensive has on their country and on the free and peaceful scientific, cultural, and economic cooperation amongst peoples ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
[QE-users] Choice of awin for average.x calculations
Dear QE Users, I have a general (and maybe naive) question to ask. What value should we choose for awin for average.x calculations? Any insights on this would be appreciated. Thank you -- *Anson Thomas* M.Sc. Chemistry, IIT Roorkee ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
[QE-users] [QE-GPU] Disable GPU acceleration for some calculations
Dear QE experts, I have installed QE 6.8 with GPU acceleration (Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-135-generic x86_64, Processor: Intel Xeon Gold 5120 CPU 2.20 GHz (2 Processor) RAM: 96 GB Graphics Card: NVIDIA Quadro P5000 (16 GB)). For some pw.x calculations, however, I desire to not use the GPU acceleration. Is there a way (some command-line option, or adding lines to bashrc or some other way) to not use GPU acceleration for a particular calculation without recompiling QE? -- *Anson Thomas* M.Sc. Chemistry, IIT Roorkee ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
Re: [QE-users] [QE-GPU] How to "Fill the CPU with OpenMP threads" to run QE-GPU
Yes, this works. Thank you so much. On Fri, Oct 8, 2021 at 9:41 PM Pietro Bonfa' wrote: > Does this work? > > export OMP_NUM_THREADS=1 > mpirun -np 1 pw.x -inp input.in > > > Best, > Pietro > > On 10/8/21 18:08, Anson Thomas wrote: > > Thanks for the suggestion. > > I exported OMP_NUM_THREADS= 1 to my bashrc (I am not using a script to > > run QE; doing it directly using mpirun pw.x -inp input.in > > <http://input.in/ > > > output). However, it is still giving the same error > > > > from addusdens_gpu : error # 1 > >cannot allocate aux2_d > > > > I even tried experimenting with OMP_NUM_THREADS= 2,3 .. but all in vain. > > Also, I have noted that always MPI processes is distributed on 1 node > > only regardless of the number of threads or processors (using mpirun -np > > ) provided. > > > > Do you have any other suggestions? > > > > > > Regards, > > Anson > > > > > > > > > > > > > > -- > > Sent by *ANSON THOMAS* > > *M.Sc. Chemistry, IIT Roorkee, India* > > > Firma il tuo 5x1000 all’Università di Parma, aiutaci a essere sempre più > accoglienti e inclusivi verso le nostre studentesse e i nostri studenti - > Indica 00308780345 nella tua denuncia dei redditi. > -- Sent by *ANSON THOMAS* *M.Sc. Chemistry, IIT Roorkee, India* ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
Re: [QE-users] [QE-GPU] How to "Fill the CPU with OpenMP threads" to run QE-GPU
Thanks for the suggestion. I exported OMP_NUM_THREADS= 1 to my bashrc (I am not using a script to run QE; doing it directly using mpirun pw.x -inp input.in > output). However, it is still giving the same error from addusdens_gpu : error # 1 cannot allocate aux2_d I even tried experimenting with OMP_NUM_THREADS= 2,3 .. but all in vain. Also, I have noted that always MPI processes is distributed on 1 node only regardless of the number of threads or processors (using mpirun -np ) provided. Do you have any other suggestions? Regards, Anson -- Sent by *ANSON THOMAS* *M.Sc. Chemistry, IIT Roorkee, India* ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
[QE-users] [QE-GPU] How to "Fill the CPU with OpenMP threads" to run QE-GPU
ss returned a non-zero exit code. Per user-direction, the job has been aborted. -- -- mpirun detected that one or more processes exited with non-zero status, thus causing the job to be terminated. The first process to do so was: Process name: [[58344,1],12] Exit code:127 -- I believe I am not "filling the CPUs with OpenMP threads", or running 1 MPI on 1 GPU, as suggested in this document. Can someone please give some suggestions? Sorry for the long post. I am totally new to this field. Any help would be appreciated. Thanks in advance -- Sent by *ANSON THOMAS* *M.Sc. Chemistry, IIT Roorkee, India* ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
[QE-users] [QE-GPU] - Error 'making' pw
I am trying to install the latest version of quantum espresso (6.8) with GPU- support on an Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-135-generic x86_64) System Configuration: Processor: Intel Xeon Gold 5120 CPU ],[ 2.20 GHz (2 Processor) RAM: 96 GB HDD: 6 TB Graphics Card: NVIDIA Quadro P5000 (16 GB) Following the steps given at https://gitlab.com/QEF/q-e-gpu/-/wikis/home, I installed all required packages (CUDA Toolkit v8+, PGI Compilers v17.10+, OpenMP package v3+), and tried configuring using ./configure --with-cuda="/opt/nvidia/hpc_sdk/Linux_x86_64/21.7/cuda/11.4/" --with-cuda-runtime=11.4 --with-cuda-cc=6.1 --enable-openmp --with- scalapack=no It successfully configures, but on doing 'make all', It works fine until it starts compiling pw.x. This is the error that I am getting make[1]: Entering directory '/home/anson/qe/qe-6.8/PW' ( cd src ; make all || exit 1 ) make[2]: Entering directory '/home/anson/qe/qe-6.8/PW/src' if test -n "" ; then \ ( cd ../.. ; make || exit 1 ) ; fi mpif90 -mp -cuda -gpu=cc6.1,cuda11.4 -o pw.x \ pwscf.o libpw.a ../../Modules/libqemod.a ../../KS_Solvers/libks_solvers.a ../../upflib/libupf.a ../../XClib/xc_lib.a ../../FFTXlib/libqefft.a ../../LAXlib/libqela.a ../../UtilXlib/libutil.a ../../dft-d3/libdftd3qe.a /home/anson/qe/qe-6.8//clib/clib.a /home/anson/qe/qe-6.8//MBD/libmbd.a -cudalib=cufft,cublas,cusolver /home/anson/qe/qe-6.8//external/devxlib/src/libdevXlib.a /home/anson/qe/qe-6.8//EIGENSOLVER_GPU/lib_eigsolve/lib_eigsolve.a -L/home/anson/qe/qe-6.8//external/devxlib/src -ldevXlib -L/usr/local/lib -llapack -lblas -L/home/anson/qe/qe-6.8//FoX/lib -lFoX_dom -lFoX_sax -lFoX_wxml -lFoX_common -lFoX_utils -lFoX_fsys -lblas ../../Modules/libqemod.a(random_numbers_gpu.o): In function `random_numbers_gpum_randy_vect_gpu_': /home/anson/qe/qe-6.8/Modules/random_numbers_gpu.f90:67: undefined reference to `curandDestroyGenerator' /home/anson/qe/qe-6.8/Modules/random_numbers_gpu.f90:68: undefined reference to `curandCreateGenerator' /home/anson/qe/qe-6.8/Modules/random_numbers_gpu.f90:69: undefined reference to `curandSetPseudoRandomGeneratorSeed' /home/anson/qe/qe-6.8/Modules/random_numbers_gpu.f90:73: undefined reference to `curandGenerateUniformDouble' pgacclnk: child process exit status 1: /usr/bin/ld Makefile:315: recipe for target 'pw.x' failed make[2]: *** [pw.x] Error 2 make[2]: Leaving directory '/home/anson/qe/qe-6.8/PW/src' Makefile:9: recipe for target 'pw' failed make[1]: *** [pw] Error 1 make[1]: Leaving directory '/home/anson/qe/qe-6.8/PW' Makefile:70: recipe for target 'pw' failed make: *** [pw] Error 1 Any suggestions to solve this would be greatly appreciated. Thank you. -- Sent by *ANSON THOMAS* *IIT Roorkee, India* ___ Quantum ESPRESSO is supported by MaX (www.max-centre.eu) users mailing list users@lists.quantum-espresso.org https://lists.quantum-espresso.org/mailman/listinfo/users
