[Pw_forum] qe-6.0 consistent errors when testing

[Fabricio Cannini]

Hello there


I'm running tests with QE version 6.0 and i'm having repeatedly the same 
errors, no matter what compiler version (gfortran 4.8.2 and 5.3.0) , 
library (openblas, atlas, netlib) openmpi version (1.8.8 , 1.10.4) or 
flags used. The results differ from the expected always in the exact 
same amounts. The errors only happen when running the parallel tests 
(make run-tests-pw-parallel).


Compilation was configured with the following options:
'''
--enable-openmp
--enable-xml
--enable-parallel
--without-scalapack
--without-elpa
'''

Here's the link to the errors.
http://pastebin.com/z5B3GgJu


In the words of Barry Burton, WHAT IS THIS?



TIA,
Fabricio
___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

Re: [Pw_forum] errors testing espresso 5.4.0

[Fabricio Cannini]

Em 21-09-2016 02:39, Paolo Giannozzi escreveu:
>
> On Wed, Sep 21, 2016 at 7:20 AM, Fabricio Cannini <fcann...@gmail.com
> <mailto:fcann...@gmail.com>> wrote:
>
>
> However, I'm no longer able to compile Phonon with the version you
> sent me, which is a dependency for EPW.
>
>
> which version are you referring to? I do not remember sending anything

Sorry, it was Filippo who sent the link.
http://qe-forge.org/snapshots/espresso-r12924-2016-09-07.tar.gz

> So, is it possible to have a patch with only the fixes, or even
> better, a 5.4.1 release?
>
>
> short answer: no, it is too much work and the new version of everything
> will be out in a few days

Ok, I'll wait for the 6.0 announcement.


[ ]'s
___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

Re: [Pw_forum] errors testing espresso 5.4.0

[Fabricio Cannini]

Em 21-09-2016 02:20, Fabricio Cannini escreveu:
> Em 10-09-2016 04:17, Paolo Giannozzi escreveu:
>> Tests are typically small. When run on many processors, they may give
>> some funny errors. It was nothing to worry about, but I fixed it anyway
>> in the development version. Thank you for reporting these errors
>>
>> Paolo

> If I understand it right, there have been changes in the trunk wvfct
> module API that makes it incompatible with the 5.4.0 wvfct module API.

Correction:

Changes to the mentioned module API expected by Phonon and EPW.
___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

Re: [Pw_forum] errors testing espresso 5.4.0

[Fabricio Cannini]

Em 10-09-2016 04:17, Paolo Giannozzi escreveu:
> Tests are typically small. When run on many processors, they may give
> some funny errors. It was nothing to worry about, but I fixed it anyway
> in the development version. Thank you for reporting these errors
>
> Paolo

Hi Paolo, Filippo

The tests went fine, thank you very much!


However, I'm no longer able to compile Phonon with the version you sent 
me, which is a dependency for EPW.


add_dkmds.f90(24): error #6580: Name in only-list does not exist.   [IGK]
   USE wvfct, ONLY : npwx, npw, nbnd, igk
-^
add_dkmds.f90(106): error #6404: This name does not have a type, and 
must have an explicit type.   [IGK]
 ( xk(ipol,kpoint) + g(ipol,igk(ig)) )
---^
compilation aborted for add_dkmds.f90 (code 1)


If I understand it right, there have been changes in the trunk wvfct 
module API that makes it incompatible with the 5.4.0 wvfct module API.

So, is it possible to have a patch with only the fixes, or even better, 
a 5.4.1 release?


TIA,
Fabricio
___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

Re: [Pw_forum] errors testing espresso 5.4.0

[Fabricio Cannini]

Em 09-09-2016 03:15, Paolo Giannozzi escreveu:
> It does not look like a true bug, because the out-of-bound array element
> is not used when the array is zero-sized. How many processors did you
> use to make your test, by the way? at least one processor has no G-vectors

I can reliably trigger this error using 16 cores, less than that it 
passes the test.

Other tests like thos in the 'pw_metal' directory begin to fail when 
using more than 16 cores.

[ ]'s

___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

Re: [Pw_forum] errors testing espresso 5.4.0

[Fabricio Cannini]

Em 06-09-2016 19:26, Filippo SPIGA escreveu:
> On Sep 5, 2016, at 6:19 PM, Fabricio Cannini <fcann...@gmail.com> wrote:
>> I can make some more tests if it helps, just tell me what to do.
>
> Grab this latest full snapshot of QE SVN repo I just generated 
> http://qe-forge.org/snapshots/espresso-r12924-2016-09-07.tar.gz
>
> It contains Paolo's fixes. Feel free to try run it and give us feedback if 
> the issue you saw still persists on the machine you run the initial tests.

Hi Paolo and Filippo


The errors that I first reported are not happening anymore, thanks a lot.

However, a new error appeared when running 
'test-suite/pw_uspp/uspp-singlegrid.in' :
--
forrtl: severe (408): fort: (2): Subscript #1 of the array RHOG_ has 
value 1 which is greater than the upper bound of 0

Image  PCRoutineLine 
Source
pw.x   013D3560  Unknown   Unknown  Unknown
pw.x   0063ADE1  symme_mp_sym_rho_ 701 
symme.f90
pw.x   0060D72E  sum_band_ 184 
sum_band.f90
pw.x   0040CDA0  electrons_scf_509 
electrons.f90
pw.x   0040A661  electrons_138 
electrons.f90
pw.x   005E19E5  run_pwscf_103 
run_pwscf.f90
pw.x   004083D8  MAIN__ 36 
pwscf.f90
pw.x   004080FE  Unknown   Unknown  Unknown
libc.so.6  0034F221ED1D  Unknown   Unknown  Unknown
pw.x   00408009  Unknown   Unknown  Unknown
-


[ ]'s
___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

Re: [Pw_forum] errors testing espresso 5.4.0

[Fabricio Cannini]

Em 05-09-2016 14:11, Paolo Giannozzi escreveu:
> ... and, I think, only the case in which the physical dimensions of FFT
> arrays are larger than the FFT dimensions

Hi Filippo and Paolo,

I can make some more tests if it helps, just tell me what to do.


[ ]'d
___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

Re: [Pw_forum] errors testing espresso 5.4.0

[Fabricio Cannini]

Em 04-09-2016 10:08, Filippo SPIGA escreveu:
> Dear Fabricio,

Hello Filippo,

> are you running these tests manually or via test-suite?

I'm running the tests via test-suite.

> Well, the error messages are quite clear so it is a matter of investigate if 
> these are repsoducible. We are tracking bugs in a very "amateurial way" at 
> the moment but we will look into it.
>
> Thanks for reporting

You're welcome.
___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

[Pw_forum] errors testing espresso 5.4.0

[Fabricio Cannini]

Hello there

I'm facing errors in a few tests of espresso 5.4.0.
I'm compiling it a centos 6.x machine in the following manner:
=
FC = intel 15.0
MPI = impi 5.0
BLAS/LAPACK = mkl 11.2
FFT = fftw 3.3.5

BLAS_LIBS="-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core"
LAPACK_LIBS="-lmkl_core"
FFT_LIBS="-lfftw3"
FFLAGS="-O2 -assume byterecl -g -traceback -fpe0 -CB -openmp"
MPIF90=mpiifort

./configure --enable-openmp --enable-parallel --without-scalapack

make pw cp ph neb epw
=


When running the pw tests, some of those fail no matter how many mpi 
processes I use.

'pw_b3lyp/b3lyp-h2o.in' and 'pw_b3lyp/b3lyp-O.in' fail with the error 
message:
---
forrtl: severe (408): fort: (2): Subscript #1 of the array CORR has 
value 12 which is greater than the upper bound of 10

Image  PCRoutineLine 
Source
pw.x   016F0EF0  Unknown   Unknown  Unknown
pw.x   00D7B085  funct_mp_set_dft_ 597 
funct.f90
pw.x   00D79837  funct_mp_enforce_ 723 
funct.f90
pw.x   00E2E054  read_pseudo_mod_m 101 
read_pseudo.f90
pw.x   006EA301  iosys_   1444 
input.f90
pw.x   004080B9  run_pwscf_ 63 
run_pwscf.f90
pw.x   00407FBD  MAIN__ 30 
pwscf.f90
pw.x   00407F1E  Unknown   Unknown  Unknown
libc.so.6  0034F221ED1D  Unknown   Unknown  Unknown
pw.x   00407E29  Unknown   Unknown  Unknown
---


'pw_uspp/uspp-hyb-g.in' fails with the error message:
---
forrtl: severe (408): fort: (2): Subscript #1 of the array DSPHER has 
value 1 which is greater than the upper bound of 0

Image  PCRoutineLine 
Source
pw.x   016F0EF0  Unknown   Unknown  Unknown
pw.x   00517B8C  realus_mp_real_sp 602 
realus.f90
pw.x   0050D056  realus_mp_addusfo1284 
realus.f90
pw.x   00AFB1F6  force_us_.L   113 
force_us.f90
pw.x   006A3415  forces_90 
forces.f90
pw.x   004081B8  run_pwscf_129 
run_pwscf.f90
pw.x   00407FBD  MAIN__ 30 
pwscf.f90
pw.x   00407F1E  Unknown   Unknown  Unknown
libc.so.6  0034F221ED1D  Unknown   Unknown  Unknown
pw.x   00407E29  Unknown   Unknown  Unknown
---


'pw_vdw/vdw-ts.in' fails with the error message:
---
forrtl: severe (408): fort: (2): Subscript #1 of the array UTSVDW has 
value 5201 which is greater than the upper bound of 5200

Image  PCRoutineLine 
Source
pw.x   016F0EF0  Unknown   Unknown  Unknown
pw.x   00470436  v_of_rho_  92 
v_of_rho.f90
pw.x   0080AE0B  potinit_  227 
potinit.f90
pw.x   006D98F8  init_run_  99 
init_run.f90
pw.x   00408111  run_pwscf_ 78 
run_pwscf.f90
pw.x   00407FBD  MAIN__ 30 
pwscf.f90
pw.x   00407F1E  Unknown   Unknown  Unknown
libc.so.6  0034F221ED1D  Unknown   Unknown  Unknown
pw.x   00407E29  Unknown   Unknown  Unknown
---



All messages are similar so they may have a common cause, but I'm unable 
to tell why exactly. Any ideas?


TIA,
Fabricio
___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

Re: [Pw_forum] Version 5.4.0 of Quantum ESPRESSO is available for download

[Fabricio Cannini]

On 28-04-2016 10:00, Filippo SPIGA wrote:
> Hello Fabricio,
>
> On Apr 25, 2016, at 10:46 PM, Fabricio Cannini <fcann...@gmail.com> wrote:
>> I'm not sseing 'QE-GPU-5.4.0.tar.gz' on the qe-forge link, and no
>> mention of it in 'Doc/release-notes'. Am I missing something ?
>
> I will upload it tomorrow afternoon.

Thanks a lot!


[ ]'s
___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

Re: [Pw_forum] Version 5.4.0 of Quantum ESPRESSO is available for download

[Fabricio Cannini]

On 25-04-2016 17:09, Filippo SPIGA wrote:
> Dear everybody,
>
> I am pleased to announce you that version 5.4.0 of Quantum ESPRESSO (SVN 
> revision r12350) is now available for download.
>
> You can find all related packages published on the QE-FORGE website at this 
> link:
> http://qe-forge.org/gf/project/q-e/frs/?action=FrsReleaseView_id=211
>
> Or download directly espresso-5.4.0.tar.gz here:
> http://qe-forge.org/gf/download/frsrelease/211/968/espresso-5.4.0.tar.gz
>
> Please refer to the file "Doc/release-notes" for additional details about the 
> release (new features, fixes, incompatibilities, known bugs). For any new 
> bug, suspicious misbehavior or any proven reproducible wrong result please 
> get in contact with the developers by writing directly to 
> q-e-develop...@qe-forge.org .
>
> Happy Computing!

Hi Filippo

I'm not sseing 'QE-GPU-5.4.0.tar.gz' on the qe-forge link, and no 
mention of it in 'Doc/release-notes'. Am I missing something ?

[ ]'s
___
Pw_forum mailing list
Pw_forum@pwscf.org
http://pwscf.org/mailman/listinfo/pw_forum

[Pw_forum] SATA vs SAS

[Fabricio Cannini]

On 07-08-2014 09:48, Carlo Nervi wrote:
> I completely agree with Axel, expecially speaking about a middle-range
> machine (as mine, for example, a linux home-assembled server with a
> careful look at the costs).

I was under the same impression that SSDs have a shorter life span than 
HDDs, but after i saw this test I don't think this is so much of an 
issue as it was when SSDs appeared.
Sure there is still plenty of room for improvement, but the SSDs of 
today are IMO doing pretty well in terms of durability.

http://techreport.com/review/26523/the-ssd-endurance-experiment-casualties-on-the-way-to-a-petabyte

[Pw_forum] Problem with espresso and cuda 6.0

[Fabricio Cannini]

2014-07-12 10:36 GMT-03:00 Filippo Spiga :

> Dear Fabricio,
>

Hi Filippo


> On Jul 11, 2014, at 2:55 AM, Fabricio Cannini  wrote:
> > ./configure BLAS_LIBS=-lopenblas
> > MANUAL_DFLAGS="-D__DISABLE_CUDA_ADDUSDENS -D__DISABLE_CUDA_VLOCPSI"
> > --enable-cuda --with-cuda-dir=/usr/local/cuda-6.0/ --with-gpu-arch=sm_35
>
> Is there any reason why you select "-D__DISABLE_CUDA_ADDUSDENS
> -D__DISABLE_CUDA_VLOCPSI"? Do you have an input case that generate
> weird/wrong results if these two routines are enabled?
>

Like I said in a previous email, I cannot compile cp-gpu and neb-gpu
without using these two flags.


> > Any one of thess fails with the following messages:
> >
> > "espresso-5.1/PW/src/newd.f90:33: undefined reference to
> newq_compute_gpu_"
> > "espresso-5.1/Modules/mp.f90:184: undefined reference to initcudaenv_"
> > "espresso-5.1/Modules/mp.f90:213: undefined reference to closecudaenv_"
>
> I got the problem and I will fix it but It will not appear in the package
> until next release.
>

5.1.1 ?

Ok then, I'll hack something for the time being.
-- next part --
An HTML attachment was scrubbed...
URL: 
http://pwscf.org/pipermail/pw_forum/attachments/20140712/e861969c/attachment.html
 

[Pw_forum] Problem with espresso and cuda 6.0

[Fabricio Cannini]

On 08-07-2014 06:42, Filippo Spiga wrote:
> Dear Fabricio,
>
> On Jul 8, 2014, at 12:04 AM, Fabricio Cannini 
> wrote:
>> I'm using 14.06 version of qe-gpu from here [
>> https://github.com/fspiga/QE-GPU/ ] . Strangely, it has no
>> 'espresso-5.1' directory nor 'espresso-5.0_GPU-14.06.patch' . Is it
>> ok ?
>
> About espresso 5.0.2 I am afraid it requires a dedicated patch not
> available yet. If you want it I can generate it and make it
> available. However if you can move to espresso-5.1 then please do
> so.
>
>
>> I've tried with espressp 5.1 too, but all patches fail at
>> 'Modules/mp.f90' , 'PW/src/pwscf.f90' and 'PW/src/vloc_psi.f90' .
>
> 14.06 does not require patch for espresso-5.1. Have you tried without
> any patch? It should work.

Yes, now it worked!

>> I've followed the instructions from the link above, but
>> compilation fails. Here's the configure options:
>>
>> ===
>> ./configure --enable-openmp --with-cuda-dir=/usr/local/cuda-6.0
>> --without-magma --disable-parallel
>> 
>
> Is nvcc in PATH? It seems it is not able to locate nvcc compiler.
>
> I also suggest to add "--enable-cuda" and "--with-gpu-arch=<...>"

Oh, sorry for this brain fart, I copied the wrong configure string.
The current configure string is:


./configure BLAS_LIBS=-lopenblas 
MANUAL_DFLAGS="-D__DISABLE_CUDA_ADDUSDENS -D__DISABLE_CUDA_VLOCPSI" 
--enable-cuda --with-cuda-dir=/usr/local/cuda-6.0/ --with-gpu-arch=sm_35


So, now i'm able to do the following:
- compile pw-gpu
- compile cp-gpu and neb-gpu using '-D__DISABLE_CUDA_ADDUSDENS 
-D__DISABLE_CUDA_VLOCPSI'

Not yet able:
- compile ph-gpu
- compile pwall-gpu
- compile all-gpu


Any one of thess fails with the following messages:


"espresso-5.1/PW/src/newd.f90:33: undefined reference to newq_compute_gpu_"
"espresso-5.1/Modules/mp.f90:184: undefined reference to initcudaenv_"
"espresso-5.1/Modules/mp.f90:213: undefined reference to closecudaenv_"


This error is because several makefiles lack the following string in the 
"LIBOBJS" variable:


../../GPU/Modules/libqemodgpu.a ../../GPU/PW/libpwgpu.a

in:
PW/src/
PW/tools/
PHonon/PH/
PHonon/Gamma/
PHonon/D3/
PHonon/FD/


What should I do to make sure that this string is inserted in the 
LIBOBJS variable of all the above makefiles ?


TIA,
Fabricio

[Pw_forum] Problem with espresso and cuda 6.0

[Fabricio Cannini]

Hi there

I'm having trouble compiling espresso 5.0.2 with cuda 6.0 on a Centos 
6.5 x86_64 machine. ( gcc 4.4.x )

I'm using 14.06 version of qe-gpu from here [ 
https://github.com/fspiga/QE-GPU/ ] . Strangely, it has no 
'espresso-5.1' directory nor 'espresso-5.0_GPU-14.06.patch' . Is it ok ?

I've tried with espressp 5.1 too, but all patches fail at 
'Modules/mp.f90' , 'PW/src/pwscf.f90' and 'PW/src/vloc_psi.f90' .


I've followed the instructions from the link above, but compilation 
fails. Here's the configure options:

===
./configure --enable-openmp --with-cuda-dir=/usr/local/cuda-6.0 
--without-magma --disable-parallel



Compilation fails with this error:
http://pastebin.com/J9mDpRxV

Notice the lines with "I../../include" but no dash in the "I"

Is it a bug in the './configure' script ?


TIA,
Fabricio

[Pw_forum] Using Intel Fortran, C++ complier- parallel, serial versions - reg.,

[Fabricio Cannini]

Em 04-06-2014 13:42, Muthu V escreveu:
> Dear users
> usually i install QE by installing FFTW lib and Fortran complier.
> everything goes fine with serial and parallel installation of QE.
>
> but i heard that  using intel C++, fortran (ifort, icc ) compliers for
> intel core(i am using intel i5 core)  improves performance so i decided
> to install above two for this i used *Intel? Parallel Studio XE 2013 for
> Linux*
> 
> intel non-commerial package which includes C++,  fortran compliers and
> mkl lib.
>
> Here i faced problem in configuring QE. i did this without installing
> *mpich2. *all process goes fine no problem i got serial version QE. but
> if i try with*mpich2, *pre-installed before QE configuration, i got the
> following error message and configuration stops.
>
> slave2 at slave2:~/espresso-5.0.1$ ./configure

Try this:

CC=icc CXX=icpc F90=ifort ./configure

If it doesn't work, google is your friend. ;)

[Pw_forum] espresso-5.0.2 error when testing modules

[Fabricio Cannini]

Em 14-04-2014 13:51, ??? escreveu:

Hi there.

> 2.For PP/examples, still the same error: gamma_only case not implemented

IIRC, this was fixed in 5.0.3 .

I'm afraid i can't answer your other questions.

> I don?t know its specific reasons.
>
> 3.For testfiles under PWCOND, there is one common error:
>
> task # 3
>
> from gep_x : error #82
>
> error on zggev
>
> really need your help
>
> thanks
>
> below are my files.

[Pw_forum] Compile QE v-4.3.2

[Fabricio Cannini]

Em 05-02-2014 21:16, Vo, Trinh (398C) escreveu:
> Dear QE users,
>
> In compiling QE in order to know whether the  external fftw  library
> included in Intel MKL (version 13)  is used,  instead of internal fftw
> is used, I changed the name of the file "fftw.c" in the directory "clib"
> into different name.  The compilation failed with following message:
>
> make[1]: *** No rule to make target `fftw.c', needed by `fft_stick.o'.
>   Stop.
> make[1]: Leaving directory `/home6/ttvo/espresso-4.3.2_test/clib'
> make: *** [libs] Error 2
>
> This may mean that the the code failed to find the external library and
> continued to use the internal one.  I have one question to confirm what
> I understand, and  I  appreciate if  you could help. if the external
> library is used, does the internal "fftw.c" is still called by some
> subroutine in the code?
>
> Thank you,
>
> Trinh Vo


You may want to try mkl's fftw interface instead :

http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-90E46168-7AD3-4EF3-958B-8224AD2E249D.htm


Good luck .

[Pw_forum] Undefined reference to gfortran

[Fabricio Cannini]

Em 24-01-2014 15:02, Axel Kohlmeyer escreveu:
> On Fri, Jan 24, 2014 at 11:57 AM, BENYAHIA NEZHA
>  wrote:
>> Hello,
>>   I'am using quantum espresso in my searches, but i'am still a beginner  in
>> this domain. I have faced a problem in which it prevent me to compile my
>> program, i have looked all around the internet for the hope of finding a
>> solution to this issue but unfortunately nothing was found.
>
> that is probably because the solution is *too* obvious. you need to
> use a Fortran compiler to compile fortran code.

Axel, what is obvious to me and you may not be to others. Please calm down.

> where did you learn programming???

Perhaps he/she ( i'm sorry, i cannot tell from your name ) have not 
learned programming. I would expect this kind of mistake in such case.


[ ]'s

[Pw_forum] Problem with QE 4.2.1 and AMD Opteron 6200 / 6300

[Fabricio Cannini]

Em 12-12-2013 08:20, Ivan Girotto escreveu:

Hi there

I've finally figured out the problem.

First, i changed the benchmark to DEISA Medium so that i could use a 
wider range of machines, as it ( predictably ) used less than half of 
the memory than the PRACE medium benchmark demanded. Then i started to 
have results.

http://qe-forge.org/gf/project/q-e/frs/?action=FrsReleaseView_id=45


Second, i noticed that no matter what, the machines running ubuntu 12.04 
were faster than the machines running debian 6.0 , and that it was 
causing by a series of factors. In order :

- OS runtime :
http://www.eglibc.org/cgi-bin/viewvc.cgi/branches/eglibc-2_15/libc/NEWS?view=markup
( "Lots of generic, 64-bit, and x86-64-specific performance 
optimizations to math functions." )


- Math Library :
Old MKL ( 10.0 ) and ACML ( 5.1.0 ) versions . Openblas gave much better 
results in both debian and ubuntu, even better in ubuntu.


- Compiler :
And this is why openblas ran even better in ubuntu.
http://gcc.gnu.org/gcc-4.6/changes.html
( "Support for AMD Bulldozer (family 15) processors is now available 
through the -march=bdver1 and -mtune=bdver1 options" )

The machines were running old versions of intel ifort ( 11.1 ), which 
could not possibly have good performance in a much newer machine.


Once again, Thanks for your time.
Fabricio

[Pw_forum] Strange benchmark results

[Fabricio Cannini]

Hi there


I'm running some tests on the following machine :
- 2x AMD Opteron 6380 ( 32 cores )
- 64GB RAM
- Ubuntu 12.04 ( gfortran 4.6, fftw 3.3.1, openblas 0.2.8, openmpi 1.4.3 )


Using these benchmarks :
- Medium size DEISA benchmark :
http://qe-forge.org/gf/project/q-e/frs/?action=FrsReleaseView_id=45

- Medium-size :
http://qe-forge.org/gf/project/q-e/frs/?action=FrsReleaseView_id=44


And the machines behavior was exactly the same. An example:

DEISA Benchmark:

-8 cores : 46m57.62s
16 cores : 38m26.81s
24 cores : 39m48.91s
32 cores : 46m2.46s




After more research on it, i've found the following graphic from other, 
older tests of quantum espresso, which shows the same behavior ( Page 8 ) :

http://hpcadvisorycouncil.com/pdf/ESPRESSO_Analysis_Intel_x5670.pdf


Is this the expected behavior of QE for these benchmarks ? If yes, why ?

[Pw_forum] Problem with QE 4.2.1 and AMD Opteron 6200 / 6300

[Fabricio Cannini]

Em 12-12-2013 08:20, Ivan Girotto escreveu:
> Dear Fabricio,
>
> I reckon there is some inconsistency in the results you are obtaining.
> The AMD 6380 is a 16-core model. I'm wondering how do you map the
> process affinity while running on 8 cores.
> Without controlling such mapping you can obtain substantial performance
> variation at each execution.
> Indeed, cache memory and FPU are shared among a given set of cores and
> the increasing concurrency on shared resources goes along with a
> degradation of the performances.

I tried using 'taskset' and 'hwloc-bind' , and the results were *worse* 
than without any of them.

binary 1 :
intel 13.2 + mkl 11.0 + openmpi 1.6.5 / 8 cores

binary 2:
gfortran 4.6 + openblas 0.2.8 + openmpi 1.6.5 / 8 cores


binary 1 time = 1h8m
binary 2 time = 46m57.62s

binary 1 with hwloc = 1h22m
binary 2 with hwloc = 56m40.50s

binary 1 with taskset = 2h27m
binary 2 with taskset = 1h48m



As I understand it, they're creating an overhead to the execution, which 
I'm sure is not the intent of them both.

> The AMD 6380 also supports the AVX instruction set extension for vector
> operations at 256bit. Does your O.S. support that too?
> Compile a simple source with -mavx and see whether you can run it. Or
> check if the "avx" flag is present in your /proc/cpuinfo.

I tried it too, but still wasn't enough to make binary 1 faster.


I'm not sure what to make of these results. Any clues ?



TIA,
Fabricio

[Pw_forum] Problem with QE 4.2.1 and AMD Opteron 6200 / 6300

[Fabricio Cannini]

Em 12-12-2013 08:20, Ivan Girotto escreveu:
> Dear Fabricio,
>
> I reckon there is some inconsistency in the results you are obtaining.
> The AMD 6380 is a 16-core model. I'm wondering how do you map the
> process affinity while running on 8 cores.
> Without controlling such mapping you can obtain substantial performance
> variation at each execution.
> Indeed, cache memory and FPU are shared among a given set of cores and
> the increasing concurrency on shared resources goes along with a
> degradation of the performances.

I tried using 'taskset' and 'hwloc-bind' , and the results were *worse* 
than without any of them.

binary 1 :
intel 13.2 + mkl 11.0 + openmpi 1.6.5 / 8 cores

binary 2:
gfortran 4.6 + openblas 0.2.8 + openmpi 1.6.5 / 8 cores


binary 1 time = 1h8m
binary 2 time = 46m57.62s

binary 1 with hwloc = 1h22m
binary 2 with hwloc = 56m40.50s

binary 1 with taskset = 2h27m
binary 2 with taskset = 1h48m



As I understand it, they're creating an overhead to the execution, which 
I'm sure is not the intent of them both.

> The AMD 6380 also supports the AVX instruction set extension for vector
> operations at 256bit. Does your O.S. support that too?
> Compile a simple source with -mavx and see whether you can run it. Or
> check if the "avx" flag is present in your /proc/cpuinfo.

I tried it too, but still wasn't enough to make binary 1 faster.


I'm not sure what to make of these results. Any clues ?



TIA,
Fabricio

[Pw_forum] Problem with QE 4.2.1 and AMD Opteron 6200 / 6300

[Fabricio Cannini]

Em 11-12-2013 15:51, Paolo Giannozzi escreveu:
> On Tue, 2013-12-10 at 19:49 -0200, Fabricio Cannini wrote:
>> Em 10-12-2013 18:34, Paolo Giannozzi escreveu:
>>> First of all you should verify if multi-threading libraries
>>> are conflicting with MPI parallelization.
>>
>> Yes, i did look into it already.


Hi there


So, what else can I look into ?
I did more tests, on the same Opteron 6380 machine, using the same 
binaries, but now using the "DEISA medium benchmark" and the results 
were interesting.

http://qe-forge.org/gf/project/q-e/frs/?action=FrsReleaseView_id=45


ifort 13.2 + mkl 11.0 / 8 cores = 1h8m
gfortran 4.6 + openblas 0.2.8 / 8 cores = 46m57.62s



This is making me even more suspicious of intel compiler being the problem.

TIA,
Fabricio

[Pw_forum] Problem with QE 4.2.1 and AMD Opteron 6200 / 6300

[Fabricio Cannini]

Em 10-12-2013 18:34, Paolo Giannozzi escreveu:
> First of all you should verify if multi-threading libraries
> are conflicting with MPI parallelization.

Yes, i did look into it already.
I can send you the 'make.sys' files of all compilations if it helps .

TIA,
Fabricio

[Pw_forum] Problem with QE 4.2.1 and AMD Opteron 6200 / 6300

[Fabricio Cannini]

Em 10-12-2013 06:01, Paolo Giannozzi escreveu:
> First of all: why do you want to use an old QE version? P.

Because the problem is of a client of mine, and he is using this very 
version , so I'm trying to reproduce the problem as exactly as possible.
Also, I'm a sysadmin, not a scientist . ;)

Should I re-run the tests with the newest QE ?


TIA,
Fabricio

[Pw_forum] Problem with QE 4.2.1 and AMD Opteron 6200 / 6300

[Fabricio Cannini]

Hi All


I'm facing a very strange situation with the above version of QE, 
compiled with :

- intel 12.1.3
- mkl 10.0,
- fftw 3.2.2 ( OS package )
- openmpi 1.4.5

Running 'prace-medium' benchmark as a test .
http://qe-forge.org/gf/project/q-e/frs/?action=FrsReleaseView_id=47

The OS is centos 5.6 x86-64 .

And the results were :

- Intel Xeon E5430 2.66GHz / 8 cores / 16 GB RAM = 00h:38m:21s

- Intel Core i7-2600 @ 3.40GHz / 8 cores / 16 GB = 00h:19m:40s

- AMD Opteron Processor 6276 / 8 cores / 256 GB = more than 8h ( process 
killed )



Then i tried another compilation :

- pgi 12.5
- acml 5.1.0 64
- fftw 3.2.2 ( OS package )
- openmpi 1.4.5

And the results were even worse . None of the machines above were able 
to finish the test in *24h* .




My third attempt was the following :

- intel 13.2
- mkl 11.0
- fftw 3.3.3 ( OS package )
- openmpi 1.6.5

- OS = Ubuntu 12.04 LTS

- AMD Opteron 6380 / 8 cores / 64 GB RAM

- Same "prace medium benchmark" test input.


Result : Also didn't finish in *24h* .




I was suspicious of the intel compiler , so I setup a 4th test :

- gfortran 4.6 ( OS package )
- openblas 0.2.8 ( compiled with gcc 4.6 )
- fftw 3.3.3 ( OS package )
- openmpi 1.4.3 ( OS package )


Same machine as the third test, and the result was the same too, with a 
difference that the binary compiled with gfortran used *much more* 
memory , running into as much as 15GB of swap memory , before i kill the 
process ( it took some 30 min to reach this point )

It should be noted that when running the 'small size' benchmark on the 
Opteron 6380 machine, the gfortran/openblas binary is faster than the 
intel 13.2/mkl binary ( up to a minute on the 3rd and 4th test ) .


Do you have a clue of what could be happening ?

Should i attach the 'make.sys' files to another message or paste it 
somewhere ?

TIA
Fabricio