Dear satish,
The make test after installing petsc works fine. The compile command that
produced the error is as follows.
mpicc -fPIC -c -g -Wshadow -fomit-frame-pointer -Wall -DSPACE_DIM=2
-DNUMBER_EQN=4 -DNUMBER_SPECIES=1 -DNUMBER_TEMPERATURES=1
-DNUMBER_ENERGYMODES=1 -D_idealgas_ -DEOS_PERFECT_GAS
-DHAVE_FORTRAN_UNDERSCORE -DPARALLEL -include
/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/flow_solver/Source/Include/grspetsc_c.h
-DPARFB -DMPI_Get_address=MPI_Address
-DMPI_Type_create_struct=MPI_Type_struct -I/opt/petsc-3.4.2/include
-I/opt/open-mpi/1.6/include -I/opt/metis/include -I/opt/parametis/include
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/blitz++
-I/opt/cgns/include
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Program/Main
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/mstlib
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/mstlib/EssIGPMlib
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/igpm_t_lib
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/fluid_t_lib/xrms/include
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/qigpmesh
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/parallel
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/flow_solver/Source/Include
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/flow_solver/Source/Main/grid_generation_dir/Flower_Converter
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/flow_solver/Source/Main/grid_generation_dir/Tau_Converter
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/flow_solver/Source/Main/reconstruction_dir
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/flow_solver/Source/Main/implicit_dir
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/flow_solver/Source/Main/implicit_dir/matvec_2ndorder
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/visual
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/eos
-I/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/Library/flow_solver/Source/Main/ANN_kd_tree
Main/time_integration_dir/compute_tau_w_mean.c -o
/Network/Servers/grsacx01.grs.rwth-aachen.de/Volumes/home/grsprabh/resmgmt/applications/Malleability/Quadflow/quadflow/install/tmp/Library/flow_solver/compute_tau_w_mean.o
Best,
Suraj
On Sep 4, 2013, at 5:14 AM, Satish Balay wrote:
> This is incomplete info.
>
> what compile command produced this error? And what do you get when you
> compile a petsc example with a petsc makefile? [complete output]
>
> Satish
>
> On Wed, 4 Sep 2013, Suraj Prabhakaran wrote:
>
>> Thanks a lot for the confirmation!!
>> I am now compiling it with 3.4.2, but I get a strange error and could be
>> great if someone can just point out whats wrong. (Might be something stupid,
>> but just in case...)
>>
>> Thanks in advance!
>>
>> Suraj
>>
>> -------------------------------ERROR-----------------------------
>>
>> In file included from /opt/petsc-3.4.2/include/petsc.h:8,
>> from Main/time_integration_dir/compute_tau_w_mean.c:91:
>> /opt/petsc-3.4.2/include/petscbt.h: In function ‘compute_tau_w_mean’:
>> /opt/petsc-3.4.2/include/petscbt.h:33: error: invalid storage class for
>> function ‘PetscBTLength’
>> /opt/petsc-3.4.2/include/petscbt.h:38: error: invalid storage class for
>> function ‘PetscBTMemzero’
>> /opt/petsc-3.4.2/include/petscbt.h:43: error: invalid storage class for
>> function ‘PetscBTDestroy’
>> /opt/petsc-3.4.2/include/petscbt.h:48: error: invalid storage class for
>> function ‘PetscBTLookup’
>> /opt/petsc-3.4.2/include/petscbt.h: In function ‘PetscBTLookup’:
>> /opt/petsc-3.4.2/include/petscbt.h:47: warning: declaration of ‘index’
>> shadows a global declaration
>> /usr/include/string.h:489: warning: shadowed declaration is here
>> /opt/petsc-3.4.2/include/petscbt.h: In function ‘compute_tau_w_mean’:
>> /opt/petsc-3.4.2/include/petscbt.h:59: error: invalid storage class for
>> function ‘PetscBTView’
>> /opt/petsc-3.4.2/include/petscbt.h:74: error: invalid storage class for
>> function ‘PetscBTCreate’
>> /opt/petsc-3.4.2/include/petscbt.h:79: error: invalid storage class for
>> function ‘PetscBTLookupSet’
>> /opt/petsc-3.4.2/include/petscbt.h: In function ‘PetscBTLookupSet’:
>> /opt/petsc-3.4.2/include/petscbt.h:78: warning: declaration of ‘index’
>> shadows a global declaration
>> /usr/include/string.h:489: warning: shadowed declaration is here
>> /opt/petsc-3.4.2/include/petscbt.h: In function ‘compute_tau_w_mean’:
>> /opt/petsc-3.4.2/include/petscbt.h:91: error: invalid storage class for
>> function ‘PetscBTSet’
>> /opt/petsc-3.4.2/include/petscbt.h: In function ‘PetscBTSet’:
>> /opt/petsc-3.4.2/include/petscbt.h:90: warning: declaration of ‘index’
>> shadows a global declaration
>> /usr/include/string.h:489: warning: shadowed declaration is here
>> /opt/petsc-3.4.2/include/petscbt.h: In function ‘compute_tau_w_mean’:
>> /opt/petsc-3.4.2/include/petscbt.h:103: error: invalid storage class for
>> function ‘PetscBTLookupClear’
>> /opt/petsc-3.4.2/include/petscbt.h: In function ‘PetscBTLookupClear’:
>> /opt/petsc-3.4.2/include/petscbt.h:102: warning: declaration of ‘index’
>> shadows a global declaration
>> /usr/include/string.h:489: warning: shadowed declaration is here
>> /opt/petsc-3.4.2/include/petscbt.h: In function ‘compute_tau_w_mean’:
>> /opt/petsc-3.4.2/include/petscbt.h:115: error: invalid storage class for
>> function ‘PetscBTClear’
>> /opt/petsc-3.4.2/include/petscbt.h: In function ‘PetscBTClear’:
>> /opt/petsc-3.4.2/include/petscbt.h:114: warning: declaration of ‘index’
>> shadows a global declaration
>> /usr/include/string.h:489: warning: shadowed declaration is here
>> In file included from /opt/petsc-3.4.2/include/petsc.h:9,
>> from Main/time_integration_dir/compute_tau_w_mean.c:91:
>> /opt/petsc-3.4.2/include/petscctable.h: In function ‘compute_tau_w_mean’:
>> /opt/petsc-3.4.2/include/petscctable.h:20: error: invalid storage class for
>> function ‘PetscHash’
>> /opt/petsc-3.4.2/include/petscctable.h:40: error: invalid storage class for
>> function ‘PetscTableAdd’
>> In file included from /opt/petsc-3.4.2/include/petsc.h:9,
>> from Main/time_integration_dir/compute_tau_w_mean.c:91:
>> /opt/petsc-3.4.2/include/petscctable.h:83: error: invalid storage class for
>> function ‘PetscTableAddCount’
>> /opt/petsc-3.4.2/include/petscctable.h:120: error: invalid storage class for
>> function ‘PetscTableFind’
>> In file included from /opt/petsc-3.4.2/include/petscsnesfas.h:3,
>> from /opt/petsc-3.4.2/include/petsc.h:27,
>> from Main/time_integration_dir/compute_tau_w_mean.c:91:
>> /opt/petsc-3.4.2/include/petscsnes.h:585: error: invalid storage class for
>> function ‘SNESGetSNESLineSearch’
>> /opt/petsc-3.4.2/include/petscsnes.h:586: error: invalid storage class for
>> function ‘SNESSetSNESLineSearch’
>> Main/time_integration_dir/compute_tau_w_mean.c:223: warning: implicit
>> declaration of function ‘wall_thermoquants’
>> Main/time_integration_dir/compute_tau_w_mean.c:155: warning: unused variable
>> ‘iface_plus_one’
>> Main/time_integration_dir/compute_tau_w_mean.c:330: warning: value computed
>> is not used
>> Main/time_integration_dir/compute_tau_w_mean.c:116: warning: unused variable
>> ‘ierr’
>> Main/time_integration_dir/compute_tau_w_mean.c:111: warning: unused variable
>> ‘icell’
>> Main/time_integration_dir/compute_tau_w_mean.c:105: warning: unused variable
>> ‘mach’
>> /opt/petsc-3.4.2/include/petscts.h:328: warning: unused variable
>> ‘TSRegisterAllCalled’
>> /opt/petsc-3.4.2/include/petscts.h:327: warning: unused variable ‘TSList’
>> /opt/petsc-3.4.2/include/petscts.h:168: warning: unused variable
>> ‘DMTS_CLASSID’
>> /opt/petsc-3.4.2/include/petscts.h:167: warning: unused variable ‘TS_CLASSID’
>> /opt/petsc-3.4.2/include/petscts.h:163: warning: unused variable
>> ‘TSExactFinalTimeOptions’
>> /opt/petsc-3.4.2/include/petscts.h:101: warning: unused variable
>> ‘TSConvergedReasons’
>> /opt/petsc-3.4.2/include/petscts.h:80: warning: unused variable
>> ‘TSEquationTypes’
>> /opt/petsc-3.4.2/include/petscsnesfas.h:19: warning: unused variable
>> ‘SNESFASTypes’
>> /opt/petsc-3.4.2/include/petscsnes.h:679: warning: unused variable
>> ‘SNESQNRestartTypes’
>> /opt/petsc-3.4.2/include/petscsnes.h:675: warning: unused variable
>> ‘SNESQNScaleTypes’
>> /opt/petsc-3.4.2/include/petscsnes.h:667: warning: unused variable
>> ‘SNESNCGTypes’
>> /opt/petsc-3.4.2/include/petscsnes.h:654: warning: unused variable
>> ‘SNESNGMRESSelectTypes’
>> /opt/petsc-3.4.2/include/petscsnes.h:648: warning: unused variable
>> ‘SNESNGMRESRestartTypes’
>> /opt/petsc-3.4.2/include/petscsnes.h:474: warning: unused variable
>> ‘SNESLineSearch_Apply’
>> /opt/petsc-3.4.2/include/petscsnes.h:473: warning: unused variable
>> ‘SNESLineSearchList’
>> /opt/petsc-3.4.2/include/petscsnes.h:472: warning: unused variable
>> ‘SNESLineSearchRegisterAllCalled’
>> /opt/petsc-3.4.2/include/petscsnes.h:471: warning: unused variable
>> ‘SNESLINESEARCH_CLASSID’
>> /opt/petsc-3.4.2/include/petscsnes.h:470: warning: unused variable ‘SNESList’
>> /opt/petsc-3.4.2/include/petscsnes.h:469: warning: unused variable
>> ‘SNESRegisterAllCalled’
>> /opt/petsc-3.4.2/include/petscsnes.h:356: warning: unused variable
>> ‘SNESNormTypes’
>> /opt/petsc-3.4.2/include/petscsnes.h:217: warning: unused variable
>> ‘SNESConvergedReasons’
>> /opt/petsc-3.4.2/include/petscsnes.h:50: warning: unused variable
>> ‘DMSNES_CLASSID’
>> /opt/petsc-3.4.2/include/petscsnes.h:49: warning: unused variable
>> ‘SNES_CLASSID’
>> /opt/petsc-3.4.2/include/petsccharacteristic.h:44: warning: unused variable
>> ‘CharacteristicList’
>> /opt/petsc-3.4.2/include/petsccharacteristic.h:43: warning: unused variable
>> ‘CharacteristicRegisterAllCalled’
>> /opt/petsc-3.4.2/include/petscdmda.h:27: warning: unused variable
>> ‘DMDABoundaryTypes’
>> /opt/petsc-3.4.2/include/petscpf.h:50: warning: unused variable
>> ‘PFRegisterAllCalled’
>> /opt/petsc-3.4.2/include/petscpf.h:40: warning: unused variable ‘PF_CLASSID’
>> /opt/petsc-3.4.2/include/petscpf.h:12: warning: unused variable ‘PFList’
>> /opt/petsc-3.4.2/include/petscdm.h:34: warning: unused variable
>> ‘DMRegisterAllCalled’
>> /opt/petsc-3.4.2/include/petscdm.h:33: warning: unused variable ‘DMList’
>> /opt/petsc-3.4.2/include/petscdm.h:11: warning: unused variable ‘DM_CLASSID’
>> /opt/petsc-3.4.2/include/petscdraw.h:239: warning: unused variable
>> ‘PETSC_DRAWHG_CLASSID’
>> /opt/petsc-3.4.2/include/petscdraw.h:225: warning: unused variable
>> ‘PETSC_DRAWSP_CLASSID’
>> /opt/petsc-3.4.2/include/petscdraw.h:206: warning: unused variable
>> ‘PETSC_DRAWLG_CLASSID’
>> /opt/petsc-3.4.2/include/petscdraw.h:195: warning: unused variable
>> ‘PETSC_DRAWAXIS_CLASSID’
>> /opt/petsc-3.4.2/include/petscdraw.h:26: warning: unused variable
>> ‘PetscDrawList’
>> /opt/petsc-3.4.2/include/petscdraw.h:9: warning: unused variable
>> ‘PETSC_DRAW_CLASSID’
>> /opt/petsc-3.4.2/include/petscmatlab.h:8: warning: unused variable
>> ‘MATLABENGINE_CLASSID’
>> /opt/petsc-3.4.2/include/petscthreadcomm.h:35: warning: unused variable
>> ‘PetscThreadCommReductionOps’
>> /opt/petsc-3.4.2/include/petscthreadcomm.h:32: warning: unused variable
>> ‘PetscThreadCommList’
>>
>>
>>
>> On Sep 3, 2013, at 9:03 PM, Satish Balay wrote:
>>
>>> Also you would have to call MPI_Init() before the first PetscInitialize()
>>> [and
>>> similarly MPI_Finalize() after the last call to PetscFinalize()]
>>>
>>> Satish
>>>
>>> On Tue, 3 Sep 2013, Barry Smith wrote:
>>>
>>>>
>>>> There is a problem with the 3.3 series when reinitializing PETSc. Can you
>>>> switch to PETSc 3.4 which properly supports calling PetscInitialize()
>>>> several times.
>>>>
>>>> Barry
>>>>
>>>> On Sep 3, 2013, at 1:58 PM, Suraj Prabhakaran
>>>> <suraj.prabhaka...@gmail.com> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> Again, this relates to finalizing and initializing petsc in my
>>>>> application. I basically want to finalize petsc and initialize it again
>>>>> with a different MPI communicator. A rough sketch of my quite huge
>>>>> application is as follows.
>>>>>
>>>>>
>>>>> main()
>>>>> {
>>>>> PetscInitialize()
>>>>> ...
>>>>> ...
>>>>> AOCreate()..
>>>>> AO operations..
>>>>> AODestroy()..
>>>>> ...
>>>>> ...
>>>>> PetscFinalize()
>>>>> PETSC_COMM_WORLD = A new communicator (which includes more processes)
>>>>> PetscInitialize()
>>>>> ...
>>>>> ...
>>>>> AOCreate()..
>>>>> AO operations..
>>>>> AODestroy()
>>>>> ...
>>>>> ...
>>>>> PetscFinalize()
>>>>> }
>>>>>
>>>>> So after finalizing and initializing it back, I always get an error with
>>>>> one of the AO calls. Given below is the complete error I get when running
>>>>> with 8 procs. Can anyone give me a hint what the problem could be? I was
>>>>> told earlier, that finalizing and initializing petsc should not be a
>>>>> problem!
>>>>>
>>>>> Best,
>>>>> Suraj
>>>>>
>>>>> -------------------------ERROR starts here-------------------------------
>>>>> [0]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
>>>>> probably memory access out of range
>>>>> [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
>>>>> [0]PETSC ERROR: or see
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC
>>>>> ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find
>>>>> memory corruption errors
>>>>> [0]PETSC ERROR: likely location of problem given in stack below
>>>>> [0]PETSC ERROR: --------------------- Stack Frames
>>>>> ------------------------------------
>>>>> [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>>>> available,
>>>>> [0]PETSC ERROR: INSTEAD the line number of the start of the function
>>>>> [0]PETSC ERROR: is given.
>>>>> [0]PETSC ERROR: [0] PetscStrcmp line 414
>>>>> /root/petsc/petsc-3.3-p7/src/sys/utils/str.c
>>>>> [0]PETSC ERROR: [0] PetscFListAdd line 202
>>>>> /root/petsc/petsc-3.3-p7/src/sys/dll/reg.c
>>>>> [0]PETSC ERROR: [0] AORegister line 99
>>>>> /root/petsc/petsc-3.3-p7/src/dm/ao/interface/aoreg.c
>>>>> [0]PETSC ERROR: [0] AORegisterAll line 28
>>>>> /root/petsc/petsc-3.3-p7/src/dm/ao/interface/aoregall.c
>>>>> [0]PETSC ERROR: [0] AOInitializePackage line 49
>>>>> /root/petsc/petsc-3.3-p7/src/dm/ao/interface/dlregisdm.c
>>>>> [0]PETSC ERROR: [0] AOCreate line 530
>>>>> /root/petsc/petsc-3.3-p7/src/dm/ao/interface/ao.c
>>>>> [0]PETSC ERROR: [0] AOCreateBasicIS line 381
>>>>> /root/petsc/petsc-3.3-p7/src/dm/ao/impls/basic/aobasic.c
>>>>> [0]PETSC ERROR: [0] AOCreateBasic line 336
>>>>> /root/petsc/petsc-3.3-p7/src/dm/ao/impls/basic/aobasic.c
>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>> ------------------------------------
>>>>> [0]PETSC ERROR: Signal received!
>>>>> [0]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [0]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24
>>>>> CDT 2013
>>>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>> [0]PETSC ERROR: See docs/index.html for manual pages.
>>>>> [0]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [0]PETSC ERROR: ./quadflow on a arch-linu named grsacc17 by grsprabh Tue
>>>>> Sep 3 20:56:21 2013
>>>>> [0]PETSC ERROR: Libraries linked from /opt/petsc3.3-1/lib
>>>>> [0]PETSC ERROR: Configure run at Mon Aug 19 13:56:50 2013
>>>>> [0]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90
>>>>> --with-cxx=mpiCC --download-f-blas-lapack --prefix=/opt/petsc3.3-1
>>>>> --with-debugging
>>>>> [0]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [0]PETSC ERROR: User provided function() line 0 in unknown directory
>>>>> unknown file
>>>>> --------------------------------------------------------------------------
>>>>> MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
>>>>> with errorcode 59.
>>>>>
>>>>> NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
>>>>> You may or may not see output from other processes, depending on
>>>>> exactly when Open MPI kills them.
>>>>> --------------------------------------------------------------------------
>>>>> [1]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [1]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
>>>>> batch system) has told this process to end
>>>>> [1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
>>>>> [1]PETSC ERROR: or see
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[1]PETSC
>>>>> ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find
>>>>> memory corruption errors
>>>>> [1]PETSC ERROR: likely location of problem given in stack below
>>>>> [1]PETSC ERROR: --------------------- Stack Frames
>>>>> ------------------------------------
>>>>> [1]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>>>> available,
>>>>> [1]PETSC ERROR: INSTEAD the line number of the start of the function
>>>>> [1]PETSC ERROR: is given.
>>>>> [1]PETSC ERROR: --------------------- Error Message
>>>>> ------------------------------------
>>>>> [1]PETSC ERROR: Signal received!
>>>>> [1]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [1]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24
>>>>> CDT 2013
>>>>> [1]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>> [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>> [1]PETSC ERROR: See docs/index.html for manual pages.
>>>>> [1]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [1]PETSC ERROR: ./quadflow on a arch-linu named grsacc17 by grsprabh Tue
>>>>> Sep 3 20:56:21 2013
>>>>> [1]PETSC ERROR: Libraries linked from /opt/petsc3.3-1/lib
>>>>> [1]PETSC ERROR: Configure run at Mon Aug 19 13:56:50 2013
>>>>> [1]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90
>>>>> --with-cxx=mpiCC --download-f-blas-lapack --prefix=/opt/petsc3.3-1
>>>>> --with-debugging
>>>>> [1]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [1]PETSC ERROR: User provided function() line 0 in unknown directory
>>>>> unknown file
>>>>> [2]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [2]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
>>>>> batch system) has told this process to end
>>>>> [2]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
>>>>> [2]PETSC ERROR: or see
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[2]PETSC
>>>>> ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find
>>>>> memory corruption errors
>>>>> [2]PETSC ERROR: likely location of problem given in stack below
>>>>> [2]PETSC ERROR: --------------------- Stack Frames
>>>>> ------------------------------------
>>>>> [2]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>>>> available,
>>>>> [2]PETSC ERROR: INSTEAD the line number of the start of the function
>>>>> [2]PETSC ERROR: is given.
>>>>> [2]PETSC ERROR: --------------------- Error Message
>>>>> ------------------------------------
>>>>> [2]PETSC ERROR: Signal received!
>>>>> [2]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [2]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24
>>>>> CDT 2013
>>>>> [2]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>> [2]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>> [2]PETSC ERROR: See docs/index.html for manual pages.
>>>>> [2]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [2]PETSC ERROR: ./quadflow on a arch-linu named grsacc17 by grsprabh Tue
>>>>> Sep 3 20:56:21 2013
>>>>> [2]PETSC ERROR: Libraries linked from /opt/petsc3.3-1/lib
>>>>> [2]PETSC ERROR: Configure run at Mon Aug 19 13:56:50 2013
>>>>> [2]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90
>>>>> --with-cxx=mpiCC --download-f-blas-lapack --prefix=/opt/petsc3.3-1
>>>>> --with-debugging
>>>>> [2]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [2]PETSC ERROR: User provided function() line 0 in unknown directory
>>>>> unknown file
>>>>> [3]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [3]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
>>>>> batch system) has told this process to end
>>>>> [3]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
>>>>> [3]PETSC ERROR: or see
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[3]PETSC
>>>>> ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find
>>>>> memory corruption errors
>>>>> [3]PETSC ERROR: likely location of problem given in stack below
>>>>> [3]PETSC ERROR: --------------------- Stack Frames
>>>>> ------------------------------------
>>>>> [3]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>>>> available,
>>>>> [3]PETSC ERROR: INSTEAD the line number of the start of the function
>>>>> [3]PETSC ERROR: is given.
>>>>> [3]PETSC ERROR: --------------------- Error Message
>>>>> ------------------------------------
>>>>> [3]PETSC ERROR: Signal received!
>>>>> [3]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [3]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24
>>>>> CDT 2013
>>>>> [3]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>> [3]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>> [3]PETSC ERROR: See docs/index.html for manual pages.
>>>>> [3]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [3]PETSC ERROR: ./quadflow on a arch-linu named grsacc17 by grsprabh Tue
>>>>> Sep 3 20:56:21 2013
>>>>> [3]PETSC ERROR: Libraries linked from /opt/petsc3.3-1/lib
>>>>> [3]PETSC ERROR: Configure run at Mon Aug 19 13:56:50 2013
>>>>> [3]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90
>>>>> --with-cxx=mpiCC --download-f-blas-lapack --prefix=/opt/petsc3.3-1
>>>>> --with-debugging
>>>>> [3]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [3]PETSC ERROR: User provided function() line 0 in unknown directory
>>>>> unknown file
>>>>> [4]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [4]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
>>>>> batch system) has told this process to end
>>>>> [4]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
>>>>> [4]PETSC ERROR: or see
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[4]PETSC
>>>>> ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find
>>>>> memory corruption errors
>>>>> [4]PETSC ERROR: likely location of problem given in stack below
>>>>> [4]PETSC ERROR: --------------------- Stack Frames
>>>>> ------------------------------------
>>>>> [4]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>>>> available,
>>>>> [4]PETSC ERROR: INSTEAD the line number of the start of the function
>>>>> [4]PETSC ERROR: is given.
>>>>> [4]PETSC ERROR: --------------------- Error Message
>>>>> ------------------------------------
>>>>> [4]PETSC ERROR: Signal received!
>>>>> [4]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [4]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24
>>>>> CDT 2013
>>>>> [4]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>> [4]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>> [4]PETSC ERROR: See docs/index.html for manual pages.
>>>>> [4]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [4]PETSC ERROR: ./quadflow on a arch-linu named grsacc17 by grsprabh Tue
>>>>> Sep 3 20:56:21 2013
>>>>> [4]PETSC ERROR: Libraries linked from /opt/petsc3.3-1/lib
>>>>> [4]PETSC ERROR: Configure run at Mon Aug 19 13:56:50 2013
>>>>> [4]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90
>>>>> --with-cxx=mpiCC --download-f-blas-lapack --prefix=/opt/petsc3.3-1
>>>>> --with-debugging
>>>>> [4]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [4]PETSC ERROR: User provided function() line 0 in unknown directory
>>>>> unknown file
>>>>> [5]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [5]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
>>>>> batch system) has told this process to end
>>>>> [5]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
>>>>> [5]PETSC ERROR: or see
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[5]PETSC
>>>>> ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find
>>>>> memory corruption errors
>>>>> [5]PETSC ERROR: likely location of problem given in stack below
>>>>> [5]PETSC ERROR: --------------------- Stack Frames
>>>>> ------------------------------------
>>>>> [5]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>>>> available,
>>>>> [5]PETSC ERROR: INSTEAD the line number of the start of the function
>>>>> [5]PETSC ERROR: is given.
>>>>> [5]PETSC ERROR: --------------------- Error Message
>>>>> ------------------------------------
>>>>> [5]PETSC ERROR: Signal received!
>>>>> [5]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [5]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24
>>>>> CDT 2013
>>>>> [5]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>> [5]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>> [5]PETSC ERROR: See docs/index.html for manual pages.
>>>>> [5]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [5]PETSC ERROR: ./quadflow on a arch-linu named grsacc17 by grsprabh Tue
>>>>> Sep 3 20:56:21 2013
>>>>> [5]PETSC ERROR: Libraries linked from /opt/petsc3.3-1/lib
>>>>> [5]PETSC ERROR: Configure run at Mon Aug 19 13:56:50 2013
>>>>> [5]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90
>>>>> --with-cxx=mpiCC --download-f-blas-lapack --prefix=/opt/petsc3.3-1
>>>>> --with-debugging
>>>>> [5]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [5]PETSC ERROR: User provided function() line 0 in unknown directory
>>>>> unknown file
>>>>> [6]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [6]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
>>>>> batch system) has told this process to end
>>>>> [6]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
>>>>> [6]PETSC ERROR: or see
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[6]PETSC
>>>>> ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find
>>>>> memory corruption errors
>>>>> [6]PETSC ERROR: likely location of problem given in stack below
>>>>> [6]PETSC ERROR: --------------------- Stack Frames
>>>>> ------------------------------------
>>>>> [6]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>>>> available,
>>>>> [6]PETSC ERROR: INSTEAD the line number of the start of the function
>>>>> [6]PETSC ERROR: is given.
>>>>> [6]PETSC ERROR: --------------------- Error Message
>>>>> ------------------------------------
>>>>> [6]PETSC ERROR: Signal received!
>>>>> [6]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [6]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24
>>>>> CDT 2013
>>>>> [6]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>> [6]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>> [6]PETSC ERROR: See docs/index.html for manual pages.
>>>>> [6]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [6]PETSC ERROR: ./quadflow on a arch-linu named grsacc17 by grsprabh Tue
>>>>> Sep 3 20:56:21 2013
>>>>> [6]PETSC ERROR: Libraries linked from /opt/petsc3.3-1/lib
>>>>> [6]PETSC ERROR: Configure run at Mon Aug 19 13:56:50 2013
>>>>> [6]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90
>>>>> --with-cxx=mpiCC --download-f-blas-lapack --prefix=/opt/petsc3.3-1
>>>>> --with-debugging
>>>>> [6]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [6]PETSC ERROR: User provided function() line 0 in unknown directory
>>>>> unknown file
>>>>> [7]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [7]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
>>>>> batch system) has told this process to end
>>>>> [7]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
>>>>> [7]PETSC ERROR: or see
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[7]PETSC
>>>>> ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find
>>>>> memory corruption errors
>>>>> [7]PETSC ERROR: likely location of problem given in stack below
>>>>> [7]PETSC ERROR: --------------------- Stack Frames
>>>>> ------------------------------------
>>>>> [7]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>>>> available,
>>>>> [7]PETSC ERROR: INSTEAD the line number of the start of the function
>>>>> [7]PETSC ERROR: is given.
>>>>> [7]PETSC ERROR: --------------------- Error Message
>>>>> ------------------------------------
>>>>> [7]PETSC ERROR: Signal received!
>>>>> [7]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [7]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24
>>>>> CDT 2013
>>>>> [7]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>> [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>> [7]PETSC ERROR: See docs/index.html for manual pages.
>>>>> [7]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [7]PETSC ERROR: ./quadflow on a arch-linu named grsacc17 by grsprabh Tue
>>>>> Sep 3 20:56:21 2013
>>>>> [7]PETSC ERROR: Libraries linked from /opt/petsc3.3-1/lib
>>>>> [7]PETSC ERROR: Configure run at Mon Aug 19 13:56:50 2013
>>>>> [7]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90
>>>>> --with-cxx=mpiCC --download-f-blas-lapack --prefix=/opt/petsc3.3-1
>>>>> --with-debugging
>>>>> [7]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [7]PETSC ERROR: User provided function() line 0 in unknown directory
>>>>> unknown file
>>>>> --------------------------------------------------------------------------
>>>>> mpiexec has exited due to process rank 0 with PID 2670 on
>>>>> node grsacc17 exiting improperly. There are two reasons this could occur:
>>>>>
>>>>> 1. this process did not call "init" before exiting, but others in
>>>>> the job did. This can cause a job to hang indefinitely while it waits
>>>>> for all processes to call "init". By rule, if one process calls "init",
>>>>> then ALL processes must call "init" prior to termination.
>>>>>
>>>>> 2. this process called "init", but exited without calling "finalize".
>>>>> By rule, all processes that call "init" MUST call "finalize" prior to
>>>>> exiting or it will be considered an "abnormal termination"
>>>>>
>>>>> This may have caused other processes in the application to be
>>>>> terminated by signals sent by mpiexec (as reported here).
>>>>
>>>>
>>>
>>
>>
