On Tue, Feb 20, 2018 at 8:08 PM, TAY wee-beng <[email protected]> wrote:
> > On 21/2/2018 9:00 AM, Matthew Knepley wrote: > > On Tue, Feb 20, 2018 at 7:54 PM, TAY wee-beng <[email protected]> wrote: > >> Hi, >> >> When I run my CFD code with a grid size of 1119x1119x499 ( total grid >> size = 624828339 ), I got the error saying I need to compile PETSc with >> 64-bit indices. >> >> So I tried to compile PETSc again and then compile my CFD code with the >> newly compiled PETSc. However, now I got segmentation error: >> >> rm: cannot remove `log': No such file or directory >> [409]PETSC ERROR: ------------------------------ >> ------------------------------------------ >> [409]PETSC ERROR: [535]PETSC ERROR: [410]PETSC ERROR: >> ------------------------------------------------------------------------ >> [410]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, >> probably memory access out of range >> [410]PETSC ERROR: Try option -start_in_debugger or >> -on_error_attach_debugger >> [410]PETSC ERROR: [536]PETSC ERROR: ------------------------------ >> ------------------------------------------ >> [536]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, >> probably memory access out of range >> [536]PETSC ERROR: Try option -start_in_debugger or >> -on_error_attach_debugger >> [536]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/d >> ocumentation/faq.html#valgrind >> [536]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac >> OS X to find memory corruption errors >> [536]PETSC ERROR: likely location of problem given in stack below >> [536]PETSC ERROR: --------------------- Stack Frames >> ------------------------------------ >> [536]PETSC ERROR: Note: The EXACT line numbers in the stack are not >> available, >> [536]PETSC ERROR: INSTEAD the line number of the start of the >> function >> [536]PETSC ERROR: is given. >> [536]PETSC ERROR: [536] DMDACheckOwnershipRanges_Private line 581 >> /home/users/nus/tsltaywb/source/petsc-3.7.6/src/dm/impls/da/da.c >> [536]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/d >> ocumentation/faq.html#valgrind >> [410]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac >> OS X to find memory corruption errors >> [410]PETSC ERROR: likely location of problem given in stack below >> [410]PETSC ERROR: --------------------- Stack Frames >> ------------------------------------ >> [410]PETSC ERROR: Note: The EXACT line numbers in the stack are not >> available, >> [897]PETSC ERROR: [536] DMDASetOwnershipRanges line 613 >> /home/users/nus/tsltaywb/source/petsc-3.7.6/src/dm/impls/da/da.c >> [536]PETSC ERROR: [536] DMDACreate3d line 1434 >> /home/users/nus/tsltaywb/source/petsc-3.7.6/src/dm/impls/da/da3.c >> [536]PETSC ERROR: --------------------- Error Message >> -------------------------------------------------------------- >> >> The CFD code worked previously but increasing the problem size results in >> segmentation error. It seems to be related to DMDACreate3d and >> DMDASetOwnershipRanges. Any idea where the problem lies? >> >> Besides, I want to know when and why do I have to use PETSc with 64-bit >> indices? >> > > 1) A 32-bit integer can hold numbers up to 2^32 = 4.2e9, so if you have a > 3D velocity, pressure, and energy, you already have 3e9 unknowns, > before you even start to count nonzero entries in the matrix. 64-bit > integers allow you to handle these big sizes. > > >> Also, can I use the 64-bit indices version with smaller sized problems? >> > > 2) Yes > > >> And is there a speed difference between using the 32-bit and 64-bit >> indices ver? > > > 3) I have seen no evidence of this > > 4) My guess is that you have defines regular integers in your code and > passed them to PETSc, rather than using PetscInt as the type. > > Oh that seems probable. So I am still using integer(4) when it should be > integer(8) for some values, is that so? If I use PetscInt, is it the same > as integer(8)? Or does it depend on the actual number? > PetscInt will be integer(4) if you configure with 32-bit ints, and integer(8) if you configure with 64-bit ints. If you use it consistently, you can avoid problems with matching the PETSc API. I wonder if I replace all my integer to PetscInt, will there be a large > increase in memory usage, because all integer(4) now becomes integer(8)? > Only if you have large integer storage. Most codes do not. Thanks, Matt > Thanks. > > > Thanks, > > Matt > > >> >> -- >> Thank you very much. >> >> Yours sincerely, >> >> ================================================ >> TAY Wee-Beng (Zheng Weiming) ιδΌζ >> Personal research webpage: http://tayweebeng.wixsite.com/website >> Youtube research showcase: https://www.youtube.com/channe >> l/UC72ZHtvQNMpNs2uRTSToiLA >> linkedin: www.linkedin.com/in/tay-weebeng >> ================================================ >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/%7Emk51/> > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
