Ordinarily, I’d agree. In fact, I don’t often run ‘make check’ on HDF5 due to the likelihood of tests themselves creating extra work (at least relative to the actual use cases for HDF5 callers I support).
This looks like a pretty vanilla build with GNU compilers. Are you using bleeding edge compiler versions? If not, seems like it could be a bug in HDF5 testing or library. You might wanna ping The HDF Group help line directly about it. I agree its an uneasy feeling to go ahead and install it with a known failure. Mark From: Hdf-forum <[email protected]<mailto:[email protected]>> on behalf of Prentice Bisbal <[email protected]<mailto:[email protected]>> Reply-To: HDF Users Discussion List <[email protected]<mailto:[email protected]>> Date: Monday, July 25, 2016 at 2:05 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: [Hdf-forum] How long should 'make check' take? You did say kill -9, which my brain wrongly interpreted as Control-C. Sorry. I configured HDF5 with minimal options: ./configure --prefix=/usr/pppl/gcc/5.4-pkgs/hdf5-serial-1.10.0-patch1 CC=gcc CXX=g++ FC=gfortran 2>&1 | tee configure.log When I got to the make check stage, I noted the time testing btree started. After 20 minutes I killed it (didn't even need the -9), and it spit out the output shown below. . As you can see at the bottom, the make check then exited with an error. Any suggestions on how to proceed? Should I run lt-bree2 manually and run a stack trace on it or do anything like that? Part of me is ready to install w/o the make check completing successfully, but I have a feeling that will just come back to haunt me in the near future. Testing btree2 ============================ btree2 Test Log ============================ Testing without reopening B-tree: Testing B-tree creation PASSED Testing B-tree iteration: empty B-tree PASSED Testing B-tree insert: first record PASSED Testing B-tree insert: several records PASSED Testing B-tree insert: split root PASSED Testing B-tree insert: redistribute 2 leaves in level 1 B-tree (l->r) PASSED Testing B-tree insert: redistribute 2 leaves in level 1 B-tree (r->l) PASSED Testing B-tree insert: split side leaf into 2 leaves in level 1 B-tree (l->r) PASSED Testing B-tree insert: split side leaf into 2 leaves in level 1 B-tree (r->l) PASSED Testing B-tree insert: redistribute 3 leaves in level 1 B-tree PASSED Testing B-tree insert: split middle leaf into 2 leaves in level 1 B-tree PASSED Testing B-tree insert: make level 2 B-tree PASSED Testing B-tree insert: redistrib right-most leaf in level 2 B-tree PASSED Testing B-tree insert: redistrib left-most leaf in level 2 B-tree PASSED Testing B-tree insert: redistrib middle leaf in level 2 B-tree PASSED Testing B-tree insert: split right-most leaf in level 2 B-tree PASSED Testing B-tree insert: split left-most leaf in level 2 B-tree PASSED Testing B-tree insert: split middle leaf in level 2 B-tree PASSED Testing B-tree insert: redist. 2 internal (r->l) in level 2 B-tree PASSED Testing B-tree insert: redist. 2 internal (l->r) in level 2 B-tree PASSED Testing B-tree insert: split side internal node to 2 in level 2 B-tree (r->l) PASSED Testing B-tree insert: split side internal node to 2 in level 2 B-tree (l->2) PASSED Testing B-tree insert: redistrib 3 internals in level 2 B-tree PASSED Testing B-tree insert: split 3 internals to 4 in level 2 B-tree PASSED Testing B-tree insert: create random level 4 B-tree PASSED Testing B-tree insert: attempt duplicate record in level 4 B-tree PASSED Testing B-tree update: inserting first record in empty B-tree PASSED Testing B-tree update: update only record in B-tree PASSED Testing B-tree update: insert several records PASSED Testing B-tree update: update several records PASSED Testing B-tree update: split root PASSED Testing B-tree update: redistribute 2 leaves in level 1 B-tree (l->r) PASSED Testing B-tree update: redistribute 2 leaves in level 1 B-tree (r->l) PASSED Testing B-tree update: split side leaf into 2 leaves in level 1 B-tree (l->r) PASSED Testing B-tree update: split side leaf into 2 leaves in level 1 B-tree (r->l) PASSED Testing B-tree update: redistribute 3 leaves in level 1 B-tree PASSED Testing B-tree update: split middle leaf into 2 leaves in level 1 B-tree PASSED Testing B-tree update: make level 2 B-tree (l->r) PASSED Testing B-tree update: make level 2 B-tree (r->l) PASSED Testing B-tree update: make level 2 B-tree (l+r->middle) PASSED Testing B-tree update: create random level 4 B-tree PASSED Testing B-tree update: update record in level 4 B-tree PASSED Testing B-tree remove: record from empty B-tree PASSED Testing B-tree remove: non-existant record from 1 record B-tree PASSED Testing B-tree remove: existant record from 1 record B-tree PASSED Testing B-tree remove: adding records to B-tree after removal PASSED Testing B-tree remove: non-existant record from level-0 B-tree PASSED Testing B-tree remove: mult. existant records from level-0 B-tree PASSED Testing B-tree remove: non-existant record from level-1 B-tree PASSED Testing B-tree remove: record from right leaf of level-1 B-tree PASSED Testing B-tree remove: record from left leaf of level-1 B-tree PASSED Testing B-tree remove: record from middle leaf of level-1 B-tree PASSED Testing B-tree remove: redistribute 2 leaves in level-1 B-tree (r->l) PASSED Testing B-tree remove: redistribute 2 leaves in level-1 B-tree (l->r) PASSED Testing B-tree remove: redistribute 3 leaves in level-1 B-tree PASSED Testing B-tree remove: merge 2 leaves to 1 in level-1 B-tree (r->l) PASSED Testing B-tree remove: merge 2 leaves to 1 in level-1 B-tree (l->r) PASSED Testing B-tree remove: merge 3 leaves to 2 in level-1 B-tree PASSED Testing B-tree remove: promote from right leaf of level-1 B-tree PASSED Testing B-tree remove: promote from left leaf of level-1 B-tree PASSED Testing B-tree remove: promote from middle leaf of level-1 B-tree PASSED Testing B-tree remove: promote from leaf of level-1 B-tree w/2 node redistrib PASSED Testing B-tree remove: promote from leaf of level-1 B-tree w/3 node redistrib PASSED Testing B-tree remove: promote from leaf of level-1 B-tree w/2->1 merge PASSED Testing B-tree remove: promote from leaf of level-1 B-tree w/3->2 merge PASSED Testing B-tree remove: collapse level-1 B-tree back to level-0 PASSED Testing B-tree remove: promote from right internal of level-2 B-tree PASSED Testing B-tree remove: promote from left internal of level-2 B-tree PASSED Testing B-tree remove: promote from middle internal of level-2 B-tree PASSED Testing B-tree remove: promote record from root of level-2 B-tree PASSED Testing B-tree remove: promote from right internal of level-2 B-tree w/redistrib PASSED Testing B-tree remove: promote from left internal of level-2 B-tree w/redistrib PASSED Testing B-tree remove: promote from right internal of level-2 B-tree w/merge PASSED Testing B-tree remove: promote from middle internal of level-2 B-tree w/merge PASSED Testing B-tree remove: merge 2 internal nodes to 1 in level-2 B-tree (l->r) PASSED Testing B-tree remove: merge 2 internal nodes to 1 in level-2 B-tree (r->l) PASSED Testing B-tree remove: merge 3 internal nodes to 2 in level-2 B-tree PASSED Testing B-tree remove: collapse level-2 B-tree back to level-1 (r->l) PASSED Testing B-tree remove: create random level 4 B-tree and delete all records in random order PASSED Testing B-tree remove: create random level 4 B-tree and delete all records by index, in random order PASSED Testing B-tree remove: create random level 4 B-tree and delete all records by index, in increasing order PASSED Testing B-tree remove: create random level 4 B-tree and delete all records by index, in decreasing order PASSED Testing B-tree find: nearest neighbor less than a value PASSED Testing B-tree find: nearest neighbor greater than a value PASSED Testing B-tree delete: delete empty B-tree PASSED Testing B-tree delete: delete level-0 B-tree PASSED Testing B-tree delete: delete level-1 B-tree PASSED Testing B-tree delete: delete level-2 B-tree PASSED Testing B-tree modify: attempt to modify non-existant record PASSED Testing B-tree modify: modify record in leaf node PASSED Testing B-tree modify: modify record in internal node PASSED Testing B-tree modify: modify record in root node PASSED Testing with reopening B-tree: Testing B-tree creation PASSED Testing B-tree iteration: empty B-tree PASSED Testing B-tree insert: first record PASSED Testing B-tree insert: several records PASSED Testing B-tree insert: split root PASSED Testing B-tree insert: redistribute 2 leaves in level 1 B-tree (l->r) PASSED Testing B-tree insert: redistribute 2 leaves in level 1 B-tree (r->l) PASSED Testing B-tree insert: split side leaf into 2 leaves in level 1 B-tree (l->r) PASSED Testing B-tree insert: split side leaf into 2 leaves in level 1 B-tree (r->l) PASSED Testing B-tree insert: redistribute 3 leaves in level 1 B-tree PASSED Testing B-tree insert: split middle leaf into 2 leaves in level 1 B-tree PASSED Testing B-tree insert: make level 2 B-tree PASSED Testing B-tree insert: redistrib right-most leaf in level 2 B-tree PASSED Testing B-tree insert: redistrib left-most leaf in level 2 B-tree PASSED Testing B-tree insert: redistrib middle leaf in level 2 B-tree PASSED Testing B-tree insert: split right-most leaf in level 2 B-tree PASSED Testing B-tree insert: split left-most leaf in level 2 B-tree PASSED Testing B-tree insert: split middle leaf in level 2 B-tree PASSED Testing B-tree insert: redist. 2 internal (r->l) in level 2 B-tree PASSED Testing B-tree insert: redist. 2 internal (l->r) in level 2 B-tree PASSED Testing B-tree insert: split side internal node to 2 in level 2 B-tree (r->l) PASSED Testing B-tree insert: split side internal node to 2 in level 2 B-tree (l->2) PASSED Testing B-tree insert: redistrib 3 internals in level 2 B-tree PASSED Testing B-tree insert: split 3 internals to 4 in level 2 B-tree PASSED Testing B-tree insert: create random level 4 B-tree PASSED Testing B-tree insert: attempt duplicate record in level 4 B-tree PASSED Testing B-tree update: inserting first record in empty B-tree PASSED Testing B-tree update: update only record in B-tree PASSED Testing B-tree update: insert several records PASSED Testing B-tree update: update several records PASSED Testing B-tree update: split root PASSED Testing B-tree update: redistribute 2 leaves in level 1 B-tree (l->r) PASSED Testing B-tree update: redistribute 2 leaves in level 1 B-tree (r->l) PASSED Testing B-tree update: split side leaf into 2 leaves in level 1 B-tree (l->r) PASSED Testing B-tree update: split side leaf into 2 leaves in level 1 B-tree (r->l) PASSED Testing B-tree update: redistribute 3 leaves in level 1 B-tree PASSED Testing B-tree update: split middle leaf into 2 leaves in level 1 B-tree PASSED Testing B-tree update: make level 2 B-tree (l->r) PASSED Testing B-tree update: make level 2 B-tree (r->l) PASSED Testing B-tree update: make level 2 B-tree (l+r->middle) PASSED Testing B-tree update: create random level 4 B-tree PASSED Testing B-tree update: update record in level 4 B-tree PASSED Testing B-tree remove: record from empty B-tree PASSED Testing B-tree remove: non-existant record from 1 record B-tree PASSED Testing B-tree remove: existant record from 1 record B-tree PASSED Testing B-tree remove: adding records to B-tree after removal PASSED Testing B-tree remove: non-existant record from level-0 B-tree PASSED Testing B-tree remove: mult. existant records from level-0 B-tree PASSED Testing B-tree remove: non-existant record from level-1 B-tree PASSED Testing B-tree remove: record from right leaf of level-1 B-tree PASSED Testing B-tree remove: record from left leaf of level-1 B-tree PASSED Testing B-tree remove: record from middle leaf of level-1 B-tree PASSED Testing B-tree remove: redistribute 2 leaves in level-1 B-tree (r->l) PASSED Testing B-tree remove: redistribute 2 leaves in level-1 B-tree (l->r) PASSED Testing B-tree remove: redistribute 3 leaves in level-1 B-tree PASSED Testing B-tree remove: merge 2 leaves to 1 in level-1 B-tree (r->l) PASSED Testing B-tree remove: merge 2 leaves to 1 in level-1 B-tree (l->r) PASSED Testing B-tree remove: merge 3 leaves to 2 in level-1 B-tree PASSED Testing B-tree remove: promote from right leaf of level-1 B-tree PASSED Testing B-tree remove: promote from left leaf of level-1 B-tree PASSED Testing B-tree remove: promote from middle leaf of level-1 B-tree PASSED Testing B-tree remove: promote from leaf of level-1 B-tree w/2 node redistrib PASSED Testing B-tree remove: promote from leaf of level-1 B-tree w/3 node redistrib PASSED Testing B-tree remove: promote from leaf of level-1 B-tree w/2->1 merge PASSED Testing B-tree remove: promote from leaf of level-1 B-tree w/3->2 merge PASSED Testing B-tree remove: collapse level-1 B-tree back to level-0 PASSED Testing B-tree remove: promote from right internal of level-2 B-tree PASSED Testing B-tree remove: promote from left internal of level-2 B-tree PASSED Testing B-tree remove: promote from middle internal of level-2 B-tree PASSED Testing B-tree remove: promote record from root of level-2 B-tree PASSED Testing B-tree remove: promote from right internal of level-2 B-tree w/redistrib PASSED Testing B-tree remove: promote from left internal of level-2 B-tree w/redistrib PASSED Testing B-tree remove: promote from right internal of level-2 B-tree w/merge PASSED Testing B-tree remove: promote from middle internal of level-2 B-tree w/merge PASSED Testing B-tree remove: merge 2 internal nodes to 1 in level-2 B-tree (l->r) PASSED Testing B-tree remove: merge 2 internal nodes to 1 in level-2 B-tree (r->l) PASSED Testing B-tree remove: merge 3 internal nodes to 2 in level-2 B-tree PASSED Testing B-tree remove: collapse level-2 B-tree back to level-1 (r->l) PASSED Command terminated by signal 15 150.13user 39.62system 22:25.95elapsed 14%CPU (0avgtext+0avgdata 494240maxresident)k 0inputs+0outputs (0major+54831minor)pagefaults 0swaps make[4]: *** [btree2.chkexe_] Error 1 make[4]: Leaving directory `/local/pbisbal/gcc-5.4.0/hdf5-1.10.0-patch1/test' make[3]: *** [build-check-s] Error 2 make[3]: Leaving directory `/local/pbisbal/gcc-5.4.0/hdf5-1.10.0-patch1/test' make[2]: *** [test] Error 2 make[2]: Leaving directory `/local/pbisbal/gcc-5.4.0/hdf5-1.10.0-patch1/test' make[1]: *** [check-am] Error 2 make[1]: Leaving directory `/local/pbisbal/gcc-5.4.0/hdf5-1.10.0-patch1/test' make: *** [check-recursive] Error 1 On 07/25/2016 04:31 PM, Miller, Mark C. wrote: Yeah, I meant ‘kill -9’ not ctrl-C. I would expect ctrl-C to just abort the current command (e.g. ‘make check’). By killing the running process (lt-btree) alone, make check would just interpret that as some kind of failure and then move on to the next test. Yes, you need a “true” parallel filesystem for any parallel tests. NFS will NOT work. Running on /local makes a lot of sense other than for parallel. Whenever I run on NFS, things take a lot longer due to all the contention for that shared resource. Mark From: Hdf-forum <[email protected]<mailto:[email protected]>> on behalf of Prentice Bisbal <[email protected]<mailto:[email protected]>> Reply-To: HDF Users Discussion List <[email protected]<mailto:[email protected]>> Date: Monday, July 25, 2016 at 1:22 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: [Hdf-forum] How long should 'make check' take? Where (e.g. on which kind of filesystem) are you trying to run ‘make check’? Can you kill -9 that one process and see if it proceeds to completion on remaining tests? Since I'm working on a serial build, right now, I'm using scratch disk local to the node I'm doing my builds on, mounted as '/local' When I do a control-C, the whole 'make check' process appears to end, it doesn't go on to the next test. I'm now doing a build with minimal options specified in the configure step to see if maybe combinations of options was causing the problem. For parallel tests, you *should* be using a parallel fileystem. I frequently forget about that. Do you mean a shared filesystem, like NFS, or a true parallel filesystem like Lustre or GPFS? After realizing my stupidity of trying to do the parallel make check on my local disk, I did move it to an NFS filesystem. Prentice On 07/25/2016 04:07 PM, Miller, Mark C. wrote: 30+mins *does* seem a bit long. Typically, a ‘make check’ for me completes *all* tests in a total of about 10-15 mins at the most. Where (e.g. on which kind of filesystem) are you trying to run ‘make check’? For parallel tests, you *should* be using a parallel fileystem. I frequently forget about that. If the parallel filesystem is *extremely* busy and/or not setup to deal with small I/Os and/or files (as the tests generate), I could see there being *some* additional delay in completion. The btree test I think is testing HDF5’s internal btree data structures for group objects. So, if its doing any I/O I think its a bunch of tiny HDF5 metadata stuff. Can you kill -9 that one process and see if it proceeds to completion on remaining tests? Mark From: Hdf-forum <[email protected]<mailto:[email protected]>> on behalf of Prentice Bisbal <[email protected]<mailto:[email protected]>> Reply-To: HDF Users Discussion List <[email protected]<mailto:[email protected]>> Date: Monday, July 25, 2016 at 12:59 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: [Hdf-forum] How long should 'make check' take? How long should make check normally take to run for a serial build? After 'make check' was hanging for me at testphdf5 when building HDF5 with parallel support, make check now seems to be hanging when testing btree2. It's been at the same spot for 30 minutes or more, but this time, I see the process (lt-btree2) using CPU and memory in top, just not much of it. Is this normal? I would expect file I/O tests to only take a few minutes, not 30+ minutes. -- Prentice _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected]<mailto:[email protected]> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5 _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected]<mailto:[email protected]>http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5 _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected]<mailto:[email protected]>http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
