Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

> On Apr 17, 2019, at 6:49 AM, Matthew Knepley wrote: > > On Wed, Apr 17, 2019 at 2:40 AM Smith, Barry F. via petsc-users > wrote: > > > > On Apr 17, 2019, at 1:35 AM, Balay, Satish wrote: > > > > On Wed, 17 Apr 2019, Smith, Barry F. via petsc-users wrote: > > > >> This is fine for

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

"Smith, Barry F." writes: > This is fine for "hacking" on PETSc but worthless for any other package. > Here is my concern, when someone > realizes there is a problem with a package they are using through a package > manager they think, crud I have to > > 1) find the git repository for this

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

> On Apr 17, 2019, at 1:35 AM, Balay, Satish wrote: > > On Wed, 17 Apr 2019, Smith, Barry F. via petsc-users wrote: > >> This is fine for "hacking" on PETSc but worthless for any other package. >> Here is my concern, when someone >> realizes there is a problem with a package they are

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

On Wed, 17 Apr 2019, Smith, Barry F. via petsc-users wrote: > This is fine for "hacking" on PETSc but worthless for any other package. > Here is my concern, when someone > realizes there is a problem with a package they are using through a package > manager they think, crud I have to > > 1)

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

On Wed, 17 Apr 2019, Satish Balay wrote: > Its not ideal - but having local changes in our spack clones (change > git url, add appropriate version lines to branches that one is > working on) is possible [for a group working in this mode]. [balay@pj03 petsc]$ pwd /home/balay/petsc [balay@pj03

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

> On Apr 17, 2019, at 12:56 AM, Jed Brown wrote: > > "Smith, Barry F. via petsc-users" writes: > >> So it sounds like spack is still mostly a "package manager" where people >> use "static" packages and don't hack the package's code. This is not >> unreasonable, no other package manager

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

"Smith, Barry F. via petsc-users" writes: > So it sounds like spack is still mostly a "package manager" where people > use "static" packages and don't hack the package's code. This is not > unreasonable, no other package manager supports hacking a package's code > easily, presumably. The

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

On Wed, 17 Apr 2019, Smith, Barry F. wrote: > > So it sounds like spack is still mostly a "package manager" where people > use "static" packages and don't hack the package's code. This is not > unreasonable, no other package manager supports hacking a package's code > easily, presumably.

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

>Perhaps if spack had an easier mechanism to allow the user to "point to" local git clones it could get closer to the best of both worlds. Maybe spack could support a list of local repositories and branches in the yaml file. I wonder if a local git clone of petsc can become a "mirror" for petsc

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

So it sounds like spack is still mostly a "package manager" where people use "static" packages and don't hack the package's code. This is not unreasonable, no other package manager supports hacking a package's code easily, presumably. The problem is that in the HPC world a "packaged" code

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

On Tue, 16 Apr 2019, Sajid Ali via petsc-users wrote: > > develop > 3.11.99 > 3.10.xx > maint (or other strings) > Just discovered this issue when trying to build with my fork of spack at [1 > ]. > > > So,

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

> develop > 3.11.99 > 3.10.xx > maint (or other strings) Just discovered this issue when trying to build with my fork of spack at [1 ]. So, ideally each developer has to have their develop point to the branch

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

On Wed, 17 Apr 2019, Balay, Satish via petsc-users wrote: > On Tue, 16 Apr 2019, Smith, Barry F. wrote: > > > > > > > > On Apr 16, 2019, at 10:30 PM, Sajid Ali > > > wrote: > > > > > > @Barry: Thanks for the bugfix! > > > > > > @Satish: Thanks for pointing out this method! > > > > > > My

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

On Tue, 16 Apr 2019, Smith, Barry F. wrote: > > > > On Apr 16, 2019, at 10:30 PM, Sajid Ali > > wrote: > > > > @Barry: Thanks for the bugfix! > > > > @Satish: Thanks for pointing out this method! > > > > My preferred way previously was to download the source code, unzip, edit, > > zip.

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

On Tue, 16 Apr 2019, Sajid Ali wrote: > Lately, spack has added git support and now I create a branch of > spack where I add my bugfix branch as the default build git repo instead of > master to now deal with checksum headaches. Some good and bad here.. version('develop', branch='master')

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

> On Apr 16, 2019, at 10:30 PM, Sajid Ali > wrote: > > @Barry: Thanks for the bugfix! > > @Satish: Thanks for pointing out this method! > > My preferred way previously was to download the source code, unzip, edit, > zip. Now ask spack to not checksum (because my edit has changed stuff)

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

@Barry: Thanks for the bugfix! @Satish: Thanks for pointing out this method! My preferred way previously was to download the source code, unzip, edit, zip. Now ask spack to not checksum (because my edit has changed stuff) and build. Lately, spack has added git support and now I create a branch

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

Satish, Thanks for the instructions. Since eventually we want everyone to use spack (but with the git repository of PETSc) is there a place we should document this? Perhaps at the bottom of the installation.html? Barry Note: of course we want them to be able to painlessly make pull

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

On Wed, 17 Apr 2019, Smith, Barry F. via petsc-users wrote: > > Funny you should ask, I just found the bug. > > > On Apr 16, 2019, at 9:47 PM, Sajid Ali > > wrote: > > > > Quick question : To drop a print statement at the required location, I need > > to modify the source code, build

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

https://bitbucket.org/petsc/petsc/pull-requests/1551/chunksize-could-overflow-and-become/diff With this fix I can run with your vector size on 1 process. With 2 processes I get $ petscmpiexec -n 2 ./ex1 Assertion failed in file adio/common/ad_write_coll.c at line 904: (curr_to_proc[p] + len -

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

Funny you should ask, I just found the bug. > On Apr 16, 2019, at 9:47 PM, Sajid Ali > wrote: > > Quick question : To drop a print statement at the required location, I need > to modify the source code, build petsc from source and compile with this new > version of petsc, right or is

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

Quick question : To drop a print statement at the required location, I need to modify the source code, build petsc from source and compile with this new version of petsc, right or is there an easier way? (Just to confirm before putting in the effort) On Tue, Apr 16, 2019 at 8:42 PM Smith, Barry

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

Dang, I ranted too soon. I built mpich using spack (master branch) and a very old Gnu C compiler and it produced valgrind clean code. Spack definitely is not passing the --enable-g=meminit to MPICH ./configure so this version of MPICH valgrind must be clean by default? MPICH's

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

You can run with -start_in_debugger -debugger_nodes 0 and in that one debugger window put a break point in the needed vector routine to print the chunk sizes at the appropriate line. Barry > On Apr 16, 2019, at 9:32 AM, Sajid Ali via petsc-users > wrote: > > Hi PETSc developers, >

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

So valgrind is printing all kinds of juicy information about uninitialized values but it is all worthless because MPICH was not built by spack to be valgrind clean. We can't know if any of the problems valgrind flags are real. MPICH needs to be configured with the option --enable-g=meminit

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

On Tue, Apr 16, 2019 at 3:44 PM Sajid Ali wrote: > So, I tried running the debug version with valgrind to see if I can find > the chunk size that's being set but I don't see it. Is there a better way > to do it ? > > `$ mpirun -np 32 valgrind ./ex_ms -prop_steps 1 -info &> out`. [The out > file

Re: [petsc-users] VecView to hdf5 broken for large (complex) vectors

Hi Matt, I tried running the same example with a smaller grid on a workstation and I see that for a grid size of 8192x8192 (vector write dims 67108864, 2), the output file has a chunk size of (16777215, 2). I can’t see HDF5_INT_MAX in the spack build-log (which includes configure). Is there a