On 21/2/2018 9:00 AM, Matthew Knepley wrote:
On Tue, Feb 20, 2018 at 7:54 PM, TAY wee-beng <[email protected]
<mailto:[email protected]>> wrote:
Hi,
When I run my CFD code with a grid size of 1119x1119x499 ( total
grid size = 624828339 ), I got the error saying I need to
compile PETSc with 64-bit indices.
So I tried to compile PETSc again and then compile my CFD code
with the newly compiled PETSc. However, now I got segmentation error:
rm: cannot remove `log': No such file or directory
[409]PETSC ERROR:
------------------------------------------------------------------------
[409]PETSC ERROR: [535]PETSC ERROR: [410]PETSC ERROR:
------------------------------------------------------------------------
[410]PETSC ERROR: Caught signal number 11 SEGV: Segmentation
Violation, probably memory access out of range
[410]PETSC ERROR: Try option -start_in_debugger or
-on_error_attach_debugger
[410]PETSC ERROR: [536]PETSC ERROR:
------------------------------------------------------------------------
[536]PETSC ERROR: Caught signal number 11 SEGV: Segmentation
Violation, probably memory access out of range
[536]PETSC ERROR: Try option -start_in_debugger or
-on_error_attach_debugger
[536]PETSC ERROR: or see
http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
<http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind>
[536]PETSC ERROR: or try http://valgrind.org on GNU/linux and
Apple Mac OS X to find memory corruption errors
[536]PETSC ERROR: likely location of problem given in stack below
[536]PETSC ERROR: --------------------- Stack Frames
------------------------------------
[536]PETSC ERROR: Note: The EXACT line numbers in the stack are
not available,
[536]PETSC ERROR: INSTEAD the line number of the start of
the function
[536]PETSC ERROR: is given.
[536]PETSC ERROR: [536] DMDACheckOwnershipRanges_Private line 581
/home/users/nus/tsltaywb/source/petsc-3.7.6/src/dm/impls/da/da.c
[536]PETSC ERROR: or see
http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
<http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind>
[410]PETSC ERROR: or try http://valgrind.org on GNU/linux and
Apple Mac OS X to find memory corruption errors
[410]PETSC ERROR: likely location of problem given in stack below
[410]PETSC ERROR: --------------------- Stack Frames
------------------------------------
[410]PETSC ERROR: Note: The EXACT line numbers in the stack are
not available,
[897]PETSC ERROR: [536] DMDASetOwnershipRanges line 613
/home/users/nus/tsltaywb/source/petsc-3.7.6/src/dm/impls/da/da.c
[536]PETSC ERROR: [536] DMDACreate3d line 1434
/home/users/nus/tsltaywb/source/petsc-3.7.6/src/dm/impls/da/da3.c
[536]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
The CFD code worked previously but increasing the problem size
results in segmentation error. It seems to be related to
DMDACreate3d and DMDASetOwnershipRanges. Any idea where the
problem lies?
Besides, I want to know when and why do I have to use PETSc with
64-bit indices?
1) A 32-bit integer can hold numbers up to 2^32 = 4.2e9, so if you
have a 3D velocity, pressure, and energy, you already have 3e9 unknowns,
before you even start to count nonzero entries in the matrix.
64-bit integers allow you to handle these big sizes.
Also, can I use the 64-bit indices version with smaller sized
problems?
2) Yes
And is there a speed difference between using the 32-bit and
64-bit indices ver?
3) I have seen no evidence of this
4) My guess is that you have defines regular integers in your code and
passed them to PETSc, rather than using PetscInt as the type.
Oh that seems probable. So I am still using integer(4) when it should be
integer(8) for some values, is that so? If I use PetscInt, is it the
same as integer(8)? Or does it depend on the actual number?
I wonder if I replace all my integer to PetscInt, will there be a large
increase in memory usage, because all integer(4) now becomes integer(8)?
Thanks.
Thanks,
Matt
--
Thank you very much.
Yours sincerely,
================================================
TAY Wee-Beng (Zheng Weiming) 郑伟明
Personal research webpage: http://tayweebeng.wixsite.com/website
<http://tayweebeng.wixsite.com/website>
Youtube research showcase:
https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA
<https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA>
linkedin: www.linkedin.com/in/tay-weebeng
<http://www.linkedin.com/in/tay-weebeng>
================================================
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which
their experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/%7Emk51/>
