Matt, Thanks for the quick reply. Users of our ALE3D hydro code at LLNL run very large problems , which I believe can pass in large amounts of local integer data to solver packages like HYPRE and PETSc for global solves. All of our local integer data is stored with 32-bit indices and switching to 64-bit indices would increase the memory footprint and data motion. This is a problem when running on our BGQ machines like Sequoia machine and potentially for our GPU machines.
Is this not a performance issue for most PETSc users who are running very large problems on 64-bit indices? And is there any interest in the future for allowing local ID’s to be in 32-bit and global ID’s be in 64-bit? Best, Brian From: Matthew Knepley <knep...@gmail.com> Date: Wednesday, January 9, 2019 at 2:56 PM To: "Weston, Brian Thomas" <west...@llnl.gov> Cc: PETSc <petsc-users@mcs.anl.gov> Subject: Re: [petsc-users] Data types for local ID's and global ID's for large problems On Wed, Jan 9, 2019 at 5:53 PM Weston, Brian Thomas via petsc-users <petsc-users@mcs.anl.gov<mailto:petsc-users@mcs.anl.gov>> wrote: Does PETSc have different data types for local ID’s and global ID’s? For example, if PETSc is configured for 32-bit indices on very large problems with say 10 billion degrees of freedoms (exceeding the limit of 32-bit indices for a global ID) would it crash and require us to re-compile with 64-bit indices or can it gracefully handle this? You would need to reconfigure for 64-bit indices. Keeping local indices 32-bit does not do much. I experimented with a local-index-only version several years ago, but it breaks a lot of things, and does not really deliver much benefit. Thanks, Matt Sincerely, Brian -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>
