Hi Barry,
Thanks for adding this functionality. A couple of questions:
1) Do we need to keep the --with-magma-device flag after the package checks for
clAmdBlas or MIC have been added? Will PETSc determine automatically which
device to use?
2) I am getting the following error while configuring:
Duplicate configure option -with-magma-device=<HAVE_CUBLAS or
HAVE_clAmdBlas or HAVE_MIC> in section MAGMA
I'm running configure with the command
./configure --download-magma --with-magma-device="HAVE_CUBLAS"
--with-cuda-dir=/home/harshad/cuda-5.5 --with-mpi=0 -CFLAGS="-malign-double"
configure.log is attached.
Harshad
----- Original Message -----
From: "Barry Smith" <[email protected]>
To: "Harshad Sahasrabudhe" <[email protected]>
Cc: "Matthew Knepley" <[email protected]>, "For users of the development
version of PETSc" mcs.anl.gov>
Sent: Friday, August 16, 2013 12:44:33 AM
Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
Harshad,
Here is the new version of magma.py
In your source code that includes magma.h you will need to #define
HAVE_CUBLAS just before that include.
Eventually you will want to be able to build PETSc with any of the three
devices: CUDA, MIC, or the AMDCL. I have put some code into magma.py and some
notes on how to handle that. Basically you need to list different libraries etc
depending on which one the user requests. It default to CUDA.
Barry
On Aug 15, 2013, at 6:26 PM, Harshad Sahasrabudhe <[email protected]> wrote:
> Hi Barry,
>
> I've attached magma.py.
>
> Thanks,
> Harshad
>
> ----- Original Message -----
> From: "Barry Smith" <[email protected]>
> To: "Harshad Sahasrabudhe" <[email protected]>
> Cc: "Matthew Knepley" <[email protected]>, "For users of the development
> version of PETSc" mcs.anl.gov>
> Sent: Thursday, August 15, 2013 6:05:47 PM
> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>
>
> If you send me your magma.py file I'll try self, self.framework and whatever
> else it takes to add the compiler flag and then send you back the working
> version.
>
> Barry
>
>
> On Aug 15, 2013, at 4:53 PM, Harshad Sahasrabudhe <[email protected]> wrote:
>
>> I am able to make python write this file and include it. But I'm all ears
>> for any other way in which we don't have to change the MAGMA source.
>>
>> Harshad
>>
>> ----- Original Message -----
>> From: "Matthew Knepley" <[email protected]>
>> To: "Barry Smith" <[email protected]>
>> Cc: "Harshad Sahasrabudhe" <[email protected]>, "For users of the
>> development version of PETSc" mcs.anl.gov>
>> Sent: Thursday, August 15, 2013 5:50:50 PM
>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>
>>
>> On Thu, Aug 15, 2013 at 4:33 PM, Barry Smith < [email protected] > wrote:
>>
>>
>>
>>
>>
>>
>> On Aug 15, 2013, at 4:30 PM, Harshad Sahasrabudhe < [email protected] >
>> wrote:
>>
>>> Yes, I made a stupid mistake. I put the include after some other includes.
>>>
>>> It's working now. Thanks!
>>
>> Yes but you needed to change magma source code AND make the changed code
>> only work for CUBLAS not for the other possibilities. So you change is not
>> scalable for others using magma
>>
>>
>> This is the right way to do it, and we do it for every other package. You
>> just have Python write this file, which is one line long.
>>
>>
>> Matt
>>
>>
>>
>>
>>>
>>> ----- Original Message -----
>>> From: "Matthew Knepley" < [email protected] >
>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>> Cc: "Karl Rupp" < [email protected] >, "For users of the development version
>>> of PETSc" < [email protected] >
>>> Sent: Thursday, August 15, 2013 5:20:44 PM
>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>
>>>
>>> On Thu, Aug 15, 2013 at 4:08 PM, Harshad Sahasrabudhe < [email protected]
>>> > wrote:
>>>
>>>
>>>
>>>
>>> Hi,
>>>
>>> Just for testing, I manually created magmaconfig.h and included it in
>>> magma.h, then ran configure. I also directly tried putting #define
>>> HAVE_CUBLAS 1 in magma.h in the extracted package.
>>>
>>> Both these methods result in the same error that I got before.
>>>
>>>
>>>
>>> That is impossible. You made a mistake somewhere.
>>>
>>>
>>> Matt
>>>
>>>
>>> I'm guessing you were suggesting creation og magmaconfig.h in magma.py?
>>>
>>> Harshad
>>>
>>> ----- Original Message -----
>>> From: "Matthew Knepley" < [email protected] >
>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>> Cc: "Karl Rupp" < [email protected] >, "For users of the development version
>>> of PETSc" < [email protected] >
>>> Sent: Thursday, August 15, 2013 4:38:57 PM
>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>
>>>
>>> On Thu, Aug 15, 2013 at 3:35 PM, Harshad Sahasrabudhe < [email protected]
>>> > wrote:
>>>
>>>
>>>
>>>
>>> I don't know how to write a config header. Can you give me an example
>>> package which I can look at?
>>>
>>>
>>>
>>> ---magmaconfig.h
>>>
>>>
>>> #define HAVE_CUBLAS 1
>>>
>>>
>>>
>>>
>>> Matt
>>>
>>>
>>> ----- Original Message -----
>>> From: "Matthew Knepley" < [email protected] >
>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>> Cc: "Karl Rupp" < [email protected] >, "For users of the development version
>>> of PETSc" < [email protected] >
>>> Sent: Thursday, August 15, 2013 4:29:12 PM
>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>
>>>
>>> On Thu, Aug 15, 2013 at 3:24 PM, Harshad Sahasrabudhe < [email protected]
>>> > wrote:
>>>
>>>
>>>
>>>
>>> It is getting passed... Here is one line from configure.log:
>>>
>>> nvcc -g -DADD_ -Xcompiler -fno-strict-aliasing -DGPUSHMEM=130 -DHAVE_CUBLAS
>>> -gencode arch=compute_13,code=sm_13 -gencode
>>> arch=compute_13,code=compute_13 -gencode arch=compute_10,code=compute_10
>>> -I/home/harshad/cuda-5.5/include -I../include -I../control -c
>>> zgeadd_batched.cu -o zgeadd_batched.cu_o
>>>
>>>
>>> During the Magma build. HOWEVER, when you compile with this header, you
>>> need to have this defined as well.
>>> I suggest writing a config header for that package since this "define on
>>> the command
>>> line stuff" is horrible. Then include that header in the regular Magma
>>> header that you are checking for.
>>>
>>>
>>> Matt
>>>
>>>
>>>
>>> ----- Original Message -----
>>> From: "Karl Rupp" < [email protected] >
>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>> Cc: "Barry Smith" < [email protected] >, "For users of the development
>>> version of PETSc" < [email protected] >
>>>
>>>
>>> Sent: Thursday, August 15, 2013 4:23:37 PM
>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>
>>> Hi again,
>>>
>>>> Yes, -DHAVE_CUBLAS is defined while compilation, but somehow I'm
>>> getting this error.
>>>
>>> Make sure this also gets passed to the CUDA compiler (nvcc).
>>>
>>> Best regards,
>>> Karli
>>>
>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Karl Rupp" < [email protected] >
>>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>>> Cc: "Barry Smith" < [email protected] >, "For users of the development
>>>> version of PETSc" < [email protected] >
>>>> Sent: Thursday, August 15, 2013 4:18:57 PM
>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>
>>>> Hey,
>>>>
>>>> apparently there must be one out of {CUBLAS, clAmdBlas, ICC for MIC}
>>>> installed. CUBLAS is already made available via --with-cuda, so you
>>>> should be fine pointing Magma to the CUDA folder. We don't have package
>>>> checks for clAmdBlas or MIC.
>>>>
>>>> Best regards,
>>>> Karli
>>>>
>>>>
>>>> On 08/15/2013 03:13 PM, Harshad Sahasrabudhe wrote:
>>>>> How does the testing work? What headers and functions do I need to
>>>>> include in magma.py?
>>>>>
>>>>> ----- Original Message -----
>>>>> From: "Barry Smith" < [email protected] >
>>>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>>>> Cc: "Jed Brown" < [email protected] >, "For users of the development
>>>>> version of PETSc" < [email protected] >
>>>>> Sent: Thursday, August 15, 2013 4:01:29 PM
>>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>>
>>>>>
>>>>> Possible ERROR while running preprocessor: In file included from
>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magmablas_z.h:14:0,
>>>>>
>>>>> from
>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magmablas.h:12,
>>>>>
>>>>> from
>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magma.h:15,
>>>>> from /tmp/petsc-Z_wD6e/config.headers/conftest.c:3:
>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magma_types.h:146:6:
>>>>> error: #error "One of HAVE_CUBLAS, HAVE_clAmdBlas, or HAVE_MIC must be
>>>>> defined. This typically happens in Makefile.internal."
>>>>> ret = 256
>>>>> error message = {In file included from
>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magmablas_z.h:14:0,
>>>>>
>>>>> from
>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magmablas.h:12,
>>>>>
>>>>> from
>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magma.h:15,
>>>>> from /tmp/petsc-Z_wD6e/config.headers/conftest.c:3:
>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magma_types.h:146:6:
>>>>> error: #error "One of HAVE_CUBLAS, HAVE_clAmdBlas, or HAVE_MIC must be
>>>>> defined. This typically happens in Makefile.internal."
>>>>>
>>>>>
>>>>> On Aug 15, 2013, at 2:43 PM, Harshad Sahasrabudhe < [email protected] >
>>>>> wrote:
>>>>>
>>>>>> Attached is the diff and configure.log
>>>>>>
>>>>>> ----- Original Message -----
>>>>>> From: "Jed Brown" < [email protected] >
>>>>>> To: "Matthew Knepley" < [email protected] >
>>>>>> Cc: "Harshad Sahasrabudhe" < [email protected] >, "For users of the
>>>>>> development version of PETSc" < [email protected] >
>>>>>> Sent: Thursday, August 15, 2013 3:33:33 PM
>>>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>>>
>>>>>>
>>>>>>
>>>>>> ... and send the changes you have made. One way to do that is to 'git
>>>>>> add config/PETSc/packages/magma.py' and send the output if 'git diff
>>>>>> master'.
>>>>>> On Aug 15, 2013 2:28 PM, "Matthew Knepley" < [email protected] > wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 15, 2013 at 1:56 PM, Harshad Sahasrabudhe <
>>>>>> [email protected] > wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Now MAGMA compiles correctly and libmagma.a is generated, however PETSc
>>>>>> says
>>>>>>
>>>>>> Downloaded magma could not be used. Please check install in
>>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug
>>>>>>
>>>>>> I am copying the generated library and include files to the install
>>>>>> directory using
>>>>>>
>>>>>> includeDir = os.path.join(self.installDir, self.includedir)
>>>>>> output,err,ret = PETSc.package.NewPackage.executeShellCommand('cd
>>>>>> '+self.packageDir+' && mv -f lib/*.* '+libDir+'/. && cp -f include/*.*
>>>>>> '+includeDir+'/.', timeout=2500, log = self.framework.log)
>>>>>>
>>>>>> I need help in resolving this error.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Always send configure.log.
>>>>>>
>>>>>>
>>>>>> Matt
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Harshad
>>>>>>
>>>>>> ----- Original Message -----
>>>>>> From: "Matthew Knepley" < [email protected] >
>>>>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>>>>> Cc: "For users of the development version of PETSc" <
>>>>>> [email protected] >
>>>>>> Sent: Wednesday, August 14, 2013 6:27:04 PM
>>>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>>>
>>>>>>
>>>>>> On Wed, Aug 14, 2013 at 5:23 PM, Harshad Sahasrabudhe <
>>>>>> [email protected] > wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm trying the write the package compiler for MAGMA using PETSc build
>>>>>> system. I'm getting the following error while building MAGMA. With the
>>>>>> error, I'm also getting a couple hundred warnings:
>>>>>>
>>>>>> sormqr_gpu_2stage.cpp: In function ‘magma_int_t
>>>>>> magma_sormqr_gpu_2stages(char, char, magma_int_t, magma_int_t,
>>>>>> magma_int_t, float*, magma_int_t, float*, magma_int_t, float*,
>>>>>> magma_int_t, magma_int_t*)’:
>>>>>> sormqr_gpu_2stage.cpp:98: warning: unused variable ‘c_one’
>>>>>> sormqr_gpu_2stage.cpp:107: warning: unused variable ‘lwkopt’
>>>>>> strsm_m.cpp: In function ‘magma_int_t magma_strsm_m(magma_int_t, char,
>>>>>> char, char, char, magma_int_t, magma_int_t, float, float*, magma_int_t,
>>>>>> float*, magma_int_t)’:
>>>>>> strsm_m.cpp:181: warning: unused variable ‘jj’
>>>>>> strsm_m.cpp:181: warning: unused variable ‘jjb’
>>>>>> sgeev.cpp: In function ‘magma_int_t magma_sgeev(char, char, magma_int_t,
>>>>>> float*, magma_int_t, float*, float*, float*, magma_int_t, float*,
>>>>>> magma_int_t, float*, magma_int_t, magma_int_t*)’:
>>>>>> sgeev.cpp:126: warning: unused variable ‘c_n1’
>>>>>> nvcc fatal : redefinition of argument 'gpu-architecture'
>>>>>>
>>>>>>
>>>>>> What is the easiest way to debug this kind of an error?
>>>>>>
>>>>>>
>>>>>>
>>>>>> 1) Make the MAGMA build system print out the full compile line
>>>>>>
>>>>>>
>>>>>> 2) Make the MAGMA people clean up their sloppy code, which would
>>>>>> eliminate the warnings
>>>>>>
>>>>>>
>>>>>> Matt
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Harshad
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> What most experimenters take for granted before they begin their
>>>>>> experiments is infinitely more interesting than any results to which
>>>>>> their experiments lead.
>>>>>> -- Norbert Wiener
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> What most experimenters take for granted before they begin their
>>>>>> experiments is infinitely more interesting than any results to which
>>>>>> their experiments lead.
>>>>>> -- Norbert Wiener
>>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>>
>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>>
>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>
>>
>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their experiments
>> is infinitely more interesting than any results to which their experiments
>> lead.
>> -- Norbert Wiener
>
> py>
Pushing language C
Popping language C
Pushing language CUDA
Popping language CUDA
Pushing language Cxx
Popping language Cxx
Pushing language FC
Popping language FC
**** Configure header conftest.h ****
#if !defined(INCLUDED_UNKNOWN)
#define INCLUDED_UNKNOWN
#endif
**** C specific Configure header conffix.h ****
#if !defined(INCLUDED_UNKNOWN)
#define INCLUDED_UNKNOWN
#if defined(__cplusplus)
extern "C" {
}
#else
#endif
#endif
*******************************************************************************
UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):
-------------------------------------------------------------------------------
Duplicate configure option -with-magma-device=<HAVE_CUBLAS or HAVE_clAmdBlas or HAVE_MIC> in section MAGMA
*******************************************************************************
File "./config/configure.py", line 291, in petsc_configure
framework.setup()
File "/home/harshad/workspace/petsc-hg/config/BuildSystem/script.py", line 82, in setup
logger.Logger.setup(self)
File "/home/harshad/workspace/petsc-hg/config/BuildSystem/logger.py", line 83, in setup
args.ArgumentProcessor.setup(self)
File "/home/harshad/workspace/petsc-hg/config/BuildSystem/args.py", line 75, in setup
self.setupArguments(self.argDB)
File "/home/harshad/workspace/petsc-hg/config/BuildSystem/config/framework.py", line 221, in setupArguments
if hasattr(child, 'setupHelp'): child.setupHelp(self.help)
File "/home/harshad/workspace/petsc-hg/config/PETSc/packages/magma_Barry.py", line 29, in setupHelp
help.addArgument('MAGMA', '-with-magma-device=<HAVE_CUBLAS or HAVE_clAmdBlas or HAVE_MIC>', nargs.ArgString(None, "HAVE_CUBLAS", 'Type of device to use'))
File "/home/harshad/workspace/petsc-hg/config/BuildSystem/help.py", line 98, in addArgument
raise RuntimeError('Duplicate configure option '+name+' in section '+section)
