On Aug 16, 2013, at 9:07 AM, Harshad Sahasrabudhe <[email protected]> wrote:
> Hi Barry,
>
> Thanks for adding this functionality. A couple of questions:
>
> 1) Do we need to keep the --with-magma-device flag after the package checks
> for clAmdBlas or MIC have been added? Will PETSc determine automatically
> which device to use?
I don't know. For now I assumed that the user would know what device they
wanted to use and set it. If you added logic to check if a gnu was attached or
clamdblas was available or MIC was available then I guess you could remove this
option.
>
> 2) I am getting the following error while configuring:
>
> Duplicate configure option -with-magma-device=<HAVE_CUBLAS or
> HAVE_clAmdBlas or HAVE_MIC> in section MAGMA
>
> I'm running configure with the command
>
> ./configure --download-magma --with-magma-device="HAVE_CUBLAS"
> --with-cuda-dir=/home/harshad/cuda-5.5 --with-mpi=0 -CFLAGS="-malign-double"
>
> configure.log is attached.
I the only way I guess this is happening is that the line
help.addArgument('MAGMA', '-with-magma-device=<HAVE_CUBLAS or
HAVE_clAmdBlas or HAVE_MIC>', nargs.ArgString(None, "HAVE_CUBLAS", 'Type of
device to use'))
is being seen twice which means that it is maybe in magma.py and magma_Barry.py
Unfortunately BuildSystem doesn't say where it was seen before.
Barry
>
> Harshad
>
> ----- Original Message -----
> From: "Barry Smith" <[email protected]>
> To: "Harshad Sahasrabudhe" <[email protected]>
> Cc: "Matthew Knepley" <[email protected]>, "For users of the development
> version of PETSc" mcs.anl.gov>
> Sent: Friday, August 16, 2013 12:44:33 AM
> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>
>
> Harshad,
>
> Here is the new version of magma.py
>
> In your source code that includes magma.h you will need to #define
> HAVE_CUBLAS just before that include.
>
> Eventually you will want to be able to build PETSc with any of the three
> devices: CUDA, MIC, or the AMDCL. I have put some code into magma.py and some
> notes on how to handle that. Basically you need to list different libraries
> etc depending on which one the user requests. It default to CUDA.
>
>
> Barry
>
>
>
>
>
> On Aug 15, 2013, at 6:26 PM, Harshad Sahasrabudhe <[email protected]> wrote:
>
>> Hi Barry,
>>
>> I've attached magma.py.
>>
>> Thanks,
>> Harshad
>>
>> ----- Original Message -----
>> From: "Barry Smith" <[email protected]>
>> To: "Harshad Sahasrabudhe" <[email protected]>
>> Cc: "Matthew Knepley" <[email protected]>, "For users of the development
>> version of PETSc" mcs.anl.gov>
>> Sent: Thursday, August 15, 2013 6:05:47 PM
>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>
>>
>> If you send me your magma.py file I'll try self, self.framework and whatever
>> else it takes to add the compiler flag and then send you back the working
>> version.
>>
>> Barry
>>
>>
>> On Aug 15, 2013, at 4:53 PM, Harshad Sahasrabudhe <[email protected]>
>> wrote:
>>
>>> I am able to make python write this file and include it. But I'm all ears
>>> for any other way in which we don't have to change the MAGMA source.
>>>
>>> Harshad
>>>
>>> ----- Original Message -----
>>> From: "Matthew Knepley" <[email protected]>
>>> To: "Barry Smith" <[email protected]>
>>> Cc: "Harshad Sahasrabudhe" <[email protected]>, "For users of the
>>> development version of PETSc" mcs.anl.gov>
>>> Sent: Thursday, August 15, 2013 5:50:50 PM
>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>
>>>
>>> On Thu, Aug 15, 2013 at 4:33 PM, Barry Smith < [email protected] > wrote:
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Aug 15, 2013, at 4:30 PM, Harshad Sahasrabudhe < [email protected] >
>>> wrote:
>>>
>>>> Yes, I made a stupid mistake. I put the include after some other includes.
>>>>
>>>> It's working now. Thanks!
>>>
>>> Yes but you needed to change magma source code AND make the changed code
>>> only work for CUBLAS not for the other possibilities. So you change is not
>>> scalable for others using magma
>>>
>>>
>>> This is the right way to do it, and we do it for every other package. You
>>> just have Python write this file, which is one line long.
>>>
>>>
>>> Matt
>>>
>>>
>>>
>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Matthew Knepley" < [email protected] >
>>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>>> Cc: "Karl Rupp" < [email protected] >, "For users of the development
>>>> version of PETSc" < [email protected] >
>>>> Sent: Thursday, August 15, 2013 5:20:44 PM
>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>
>>>>
>>>> On Thu, Aug 15, 2013 at 4:08 PM, Harshad Sahasrabudhe <
>>>> [email protected] > wrote:
>>>>
>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> Just for testing, I manually created magmaconfig.h and included it in
>>>> magma.h, then ran configure. I also directly tried putting #define
>>>> HAVE_CUBLAS 1 in magma.h in the extracted package.
>>>>
>>>> Both these methods result in the same error that I got before.
>>>>
>>>>
>>>>
>>>> That is impossible. You made a mistake somewhere.
>>>>
>>>>
>>>> Matt
>>>>
>>>>
>>>> I'm guessing you were suggesting creation og magmaconfig.h in magma.py?
>>>>
>>>> Harshad
>>>>
>>>> ----- Original Message -----
>>>> From: "Matthew Knepley" < [email protected] >
>>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>>> Cc: "Karl Rupp" < [email protected] >, "For users of the development
>>>> version of PETSc" < [email protected] >
>>>> Sent: Thursday, August 15, 2013 4:38:57 PM
>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>
>>>>
>>>> On Thu, Aug 15, 2013 at 3:35 PM, Harshad Sahasrabudhe <
>>>> [email protected] > wrote:
>>>>
>>>>
>>>>
>>>>
>>>> I don't know how to write a config header. Can you give me an example
>>>> package which I can look at?
>>>>
>>>>
>>>>
>>>> ---magmaconfig.h
>>>>
>>>>
>>>> #define HAVE_CUBLAS 1
>>>>
>>>>
>>>>
>>>>
>>>> Matt
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Matthew Knepley" < [email protected] >
>>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>>> Cc: "Karl Rupp" < [email protected] >, "For users of the development
>>>> version of PETSc" < [email protected] >
>>>> Sent: Thursday, August 15, 2013 4:29:12 PM
>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>
>>>>
>>>> On Thu, Aug 15, 2013 at 3:24 PM, Harshad Sahasrabudhe <
>>>> [email protected] > wrote:
>>>>
>>>>
>>>>
>>>>
>>>> It is getting passed... Here is one line from configure.log:
>>>>
>>>> nvcc -g -DADD_ -Xcompiler -fno-strict-aliasing -DGPUSHMEM=130
>>>> -DHAVE_CUBLAS -gencode arch=compute_13,code=sm_13 -gencode
>>>> arch=compute_13,code=compute_13 -gencode arch=compute_10,code=compute_10
>>>> -I/home/harshad/cuda-5.5/include -I../include -I../control -c
>>>> zgeadd_batched.cu -o zgeadd_batched.cu_o
>>>>
>>>>
>>>> During the Magma build. HOWEVER, when you compile with this header, you
>>>> need to have this defined as well.
>>>> I suggest writing a config header for that package since this "define on
>>>> the command
>>>> line stuff" is horrible. Then include that header in the regular Magma
>>>> header that you are checking for.
>>>>
>>>>
>>>> Matt
>>>>
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Karl Rupp" < [email protected] >
>>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>>> Cc: "Barry Smith" < [email protected] >, "For users of the development
>>>> version of PETSc" < [email protected] >
>>>>
>>>>
>>>> Sent: Thursday, August 15, 2013 4:23:37 PM
>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>
>>>> Hi again,
>>>>
>>>>> Yes, -DHAVE_CUBLAS is defined while compilation, but somehow I'm
>>>> getting this error.
>>>>
>>>> Make sure this also gets passed to the CUDA compiler (nvcc).
>>>>
>>>> Best regards,
>>>> Karli
>>>>
>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>> From: "Karl Rupp" < [email protected] >
>>>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>>>> Cc: "Barry Smith" < [email protected] >, "For users of the development
>>>>> version of PETSc" < [email protected] >
>>>>> Sent: Thursday, August 15, 2013 4:18:57 PM
>>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>>
>>>>> Hey,
>>>>>
>>>>> apparently there must be one out of {CUBLAS, clAmdBlas, ICC for MIC}
>>>>> installed. CUBLAS is already made available via --with-cuda, so you
>>>>> should be fine pointing Magma to the CUDA folder. We don't have package
>>>>> checks for clAmdBlas or MIC.
>>>>>
>>>>> Best regards,
>>>>> Karli
>>>>>
>>>>>
>>>>> On 08/15/2013 03:13 PM, Harshad Sahasrabudhe wrote:
>>>>>> How does the testing work? What headers and functions do I need to
>>>>>> include in magma.py?
>>>>>>
>>>>>> ----- Original Message -----
>>>>>> From: "Barry Smith" < [email protected] >
>>>>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>>>>> Cc: "Jed Brown" < [email protected] >, "For users of the development
>>>>>> version of PETSc" < [email protected] >
>>>>>> Sent: Thursday, August 15, 2013 4:01:29 PM
>>>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>>>
>>>>>>
>>>>>> Possible ERROR while running preprocessor: In file included from
>>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magmablas_z.h:14:0,
>>>>>>
>>>>>> from
>>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magmablas.h:12,
>>>>>>
>>>>>> from
>>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magma.h:15,
>>>>>> from /tmp/petsc-Z_wD6e/config.headers/conftest.c:3:
>>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magma_types.h:146:6:
>>>>>> error: #error "One of HAVE_CUBLAS, HAVE_clAmdBlas, or HAVE_MIC must be
>>>>>> defined. This typically happens in Makefile.internal."
>>>>>> ret = 256
>>>>>> error message = {In file included from
>>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magmablas_z.h:14:0,
>>>>>>
>>>>>> from
>>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magmablas.h:12,
>>>>>>
>>>>>> from
>>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magma.h:15,
>>>>>> from /tmp/petsc-Z_wD6e/config.headers/conftest.c:3:
>>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug/include/magma_types.h:146:6:
>>>>>> error: #error "One of HAVE_CUBLAS, HAVE_clAmdBlas, or HAVE_MIC must be
>>>>>> defined. This typically happens in Makefile.internal."
>>>>>>
>>>>>>
>>>>>> On Aug 15, 2013, at 2:43 PM, Harshad Sahasrabudhe < [email protected]
>>>>>> > wrote:
>>>>>>
>>>>>>> Attached is the diff and configure.log
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>> From: "Jed Brown" < [email protected] >
>>>>>>> To: "Matthew Knepley" < [email protected] >
>>>>>>> Cc: "Harshad Sahasrabudhe" < [email protected] >, "For users of the
>>>>>>> development version of PETSc" < [email protected] >
>>>>>>> Sent: Thursday, August 15, 2013 3:33:33 PM
>>>>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ... and send the changes you have made. One way to do that is to 'git
>>>>>>> add config/PETSc/packages/magma.py' and send the output if 'git diff
>>>>>>> master'.
>>>>>>> On Aug 15, 2013 2:28 PM, "Matthew Knepley" < [email protected] > wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 15, 2013 at 1:56 PM, Harshad Sahasrabudhe <
>>>>>>> [email protected] > wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Now MAGMA compiles correctly and libmagma.a is generated, however PETSc
>>>>>>> says
>>>>>>>
>>>>>>> Downloaded magma could not be used. Please check install in
>>>>>>> /home/harshad/workspace/petsc-hg/arch-linux2-c-debug
>>>>>>>
>>>>>>> I am copying the generated library and include files to the install
>>>>>>> directory using
>>>>>>>
>>>>>>> includeDir = os.path.join(self.installDir, self.includedir)
>>>>>>> output,err,ret = PETSc.package.NewPackage.executeShellCommand('cd
>>>>>>> '+self.packageDir+' && mv -f lib/*.* '+libDir+'/. && cp -f include/*.*
>>>>>>> '+includeDir+'/.', timeout=2500, log = self.framework.log)
>>>>>>>
>>>>>>> I need help in resolving this error.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Always send configure.log.
>>>>>>>
>>>>>>>
>>>>>>> Matt
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Harshad
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>> From: "Matthew Knepley" < [email protected] >
>>>>>>> To: "Harshad Sahasrabudhe" < [email protected] >
>>>>>>> Cc: "For users of the development version of PETSc" <
>>>>>>> [email protected] >
>>>>>>> Sent: Wednesday, August 14, 2013 6:27:04 PM
>>>>>>> Subject: Re: [petsc-dev] Error while building MAGMA from PETSc
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Aug 14, 2013 at 5:23 PM, Harshad Sahasrabudhe <
>>>>>>> [email protected] > wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm trying the write the package compiler for MAGMA using PETSc build
>>>>>>> system. I'm getting the following error while building MAGMA. With the
>>>>>>> error, I'm also getting a couple hundred warnings:
>>>>>>>
>>>>>>> sormqr_gpu_2stage.cpp: In function ‘magma_int_t
>>>>>>> magma_sormqr_gpu_2stages(char, char, magma_int_t, magma_int_t,
>>>>>>> magma_int_t, float*, magma_int_t, float*, magma_int_t, float*,
>>>>>>> magma_int_t, magma_int_t*)’:
>>>>>>> sormqr_gpu_2stage.cpp:98: warning: unused variable ‘c_one’
>>>>>>> sormqr_gpu_2stage.cpp:107: warning: unused variable ‘lwkopt’
>>>>>>> strsm_m.cpp: In function ‘magma_int_t magma_strsm_m(magma_int_t, char,
>>>>>>> char, char, char, magma_int_t, magma_int_t, float, float*, magma_int_t,
>>>>>>> float*, magma_int_t)’:
>>>>>>> strsm_m.cpp:181: warning: unused variable ‘jj’
>>>>>>> strsm_m.cpp:181: warning: unused variable ‘jjb’
>>>>>>> sgeev.cpp: In function ‘magma_int_t magma_sgeev(char, char,
>>>>>>> magma_int_t, float*, magma_int_t, float*, float*, float*, magma_int_t,
>>>>>>> float*, magma_int_t, float*, magma_int_t, magma_int_t*)’:
>>>>>>> sgeev.cpp:126: warning: unused variable ‘c_n1’
>>>>>>> nvcc fatal : redefinition of argument 'gpu-architecture'
>>>>>>>
>>>>>>>
>>>>>>> What is the easiest way to debug this kind of an error?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 1) Make the MAGMA build system print out the full compile line
>>>>>>>
>>>>>>>
>>>>>>> 2) Make the MAGMA people clean up their sloppy code, which would
>>>>>>> eliminate the warnings
>>>>>>>
>>>>>>>
>>>>>>> Matt
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Harshad
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> What most experimenters take for granted before they begin their
>>>>>>> experiments is infinitely more interesting than any results to which
>>>>>>> their experiments lead.
>>>>>>> -- Norbert Wiener
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> What most experimenters take for granted before they begin their
>>>>>>> experiments is infinitely more interesting than any results to which
>>>>>>> their experiments lead.
>>>>>>> -- Norbert Wiener
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>
>>>
>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>
>> py>
>
> <configure.log>
