Re: [Hdf-forum] How long should 'make check' take?

Prentice Bisbal Mon, 25 Jul 2016 14:08:00 -0700

You did say kill -9, which my brain wrongly interpreted as Control-C.Sorry. I configured HDF5 with minimal options:

./configure --prefix=/usr/pppl/gcc/5.4-pkgs/hdf5-serial-1.10.0-patch1CC=gcc CXX=g++ FC=gfortran 2>&1 | tee configure.log

When I got to the make check stage, I noted the time testing btreestarted. After 20 minutes I killed it (didn't even need the -9), and itspit out the output shown below. . As you can see at the bottom, themake check then exited with an error.

Any suggestions on how to proceed? Should I run lt-bree2 manually andrun a stack trace on it or do anything like that?

Part of me is ready to install w/o the make check completingsuccessfully, but I have a feeling that will just come back to haunt mein the near future.


Testing  btree2
============================
 btree2  Test Log
============================
Testing without reopening B-tree:

Testing B-tree creationPASSEDTesting B-tree iteration: empty B-treePASSEDTesting B-tree insert: first recordPASSEDTesting B-tree insert: several recordsPASSEDTesting B-tree insert: split rootPASSEDTesting B-tree insert: redistribute 2 leaves in level 1 B-tree (l->r)PASSEDTesting B-tree insert: redistribute 2 leaves in level 1 B-tree (r->l)PASSEDTesting B-tree insert: split side leaf into 2 leaves in level 1 B-tree(l->r) PASSEDTesting B-tree insert: split side leaf into 2 leaves in level 1 B-tree(r->l) PASSEDTesting B-tree insert: redistribute 3 leaves in level 1 B-treePASSEDTesting B-tree insert: split middle leaf into 2 leaves in level 1 B-treePASSEDTesting B-tree insert: make level 2 B-treePASSEDTesting B-tree insert: redistrib right-most leaf in level 2 B-treePASSEDTesting B-tree insert: redistrib left-most leaf in level 2 B-treePASSEDTesting B-tree insert: redistrib middle leaf in level 2 B-treePASSEDTesting B-tree insert: split right-most leaf in level 2 B-treePASSEDTesting B-tree insert: split left-most leaf in level 2 B-treePASSEDTesting B-tree insert: split middle leaf in level 2 B-treePASSEDTesting B-tree insert: redist. 2 internal (r->l) in level 2 B-treePASSEDTesting B-tree insert: redist. 2 internal (l->r) in level 2 B-treePASSEDTesting B-tree insert: split side internal node to 2 in level 2 B-tree(r->l) PASSEDTesting B-tree insert: split side internal node to 2 in level 2 B-tree(l->2) PASSEDTesting B-tree insert: redistrib 3 internals in level 2 B-treePASSEDTesting B-tree insert: split 3 internals to 4 in level 2 B-treePASSEDTesting B-tree insert: create random level 4 B-treePASSEDTesting B-tree insert: attempt duplicate record in level 4 B-treePASSEDTesting B-tree update: inserting first record in empty B-treePASSEDTesting B-tree update: update only record in B-treePASSEDTesting B-tree update: insert several recordsPASSEDTesting B-tree update: update several recordsPASSEDTesting B-tree update: split rootPASSEDTesting B-tree update: redistribute 2 leaves in level 1 B-tree (l->r)PASSEDTesting B-tree update: redistribute 2 leaves in level 1 B-tree (r->l)PASSEDTesting B-tree update: split side leaf into 2 leaves in level 1 B-tree(l->r) PASSEDTesting B-tree update: split side leaf into 2 leaves in level 1 B-tree(r->l) PASSEDTesting B-tree update: redistribute 3 leaves in level 1 B-treePASSEDTesting B-tree update: split middle leaf into 2 leaves in level 1 B-treePASSEDTesting B-tree update: make level 2 B-tree (l->r)PASSEDTesting B-tree update: make level 2 B-tree (r->l)PASSEDTesting B-tree update: make level 2 B-tree (l+r->middle)PASSEDTesting B-tree update: create random level 4 B-treePASSEDTesting B-tree update: update record in level 4 B-treePASSEDTesting B-tree remove: record from empty B-treePASSEDTesting B-tree remove: non-existant record from 1 record B-treePASSEDTesting B-tree remove: existant record from 1 record B-treePASSEDTesting B-tree remove: adding records to B-tree after removalPASSEDTesting B-tree remove: non-existant record from level-0 B-treePASSEDTesting B-tree remove: mult. existant records from level-0 B-treePASSEDTesting B-tree remove: non-existant record from level-1 B-treePASSEDTesting B-tree remove: record from right leaf of level-1 B-treePASSEDTesting B-tree remove: record from left leaf of level-1 B-treePASSEDTesting B-tree remove: record from middle leaf of level-1 B-treePASSEDTesting B-tree remove: redistribute 2 leaves in level-1 B-tree (r->l)PASSEDTesting B-tree remove: redistribute 2 leaves in level-1 B-tree (l->r)PASSEDTesting B-tree remove: redistribute 3 leaves in level-1 B-treePASSEDTesting B-tree remove: merge 2 leaves to 1 in level-1 B-tree (r->l)PASSEDTesting B-tree remove: merge 2 leaves to 1 in level-1 B-tree (l->r)PASSEDTesting B-tree remove: merge 3 leaves to 2 in level-1 B-treePASSEDTesting B-tree remove: promote from right leaf of level-1 B-treePASSEDTesting B-tree remove: promote from left leaf of level-1 B-treePASSEDTesting B-tree remove: promote from middle leaf of level-1 B-treePASSEDTesting B-tree remove: promote from leaf of level-1 B-tree w/2 noderedistrib PASSEDTesting B-tree remove: promote from leaf of level-1 B-tree w/3 noderedistrib PASSEDTesting B-tree remove: promote from leaf of level-1 B-tree w/2->1 mergePASSEDTesting B-tree remove: promote from leaf of level-1 B-tree w/3->2 mergePASSEDTesting B-tree remove: collapse level-1 B-tree back to level-0PASSEDTesting B-tree remove: promote from right internal of level-2 B-treePASSEDTesting B-tree remove: promote from left internal of level-2 B-treePASSEDTesting B-tree remove: promote from middle internal of level-2 B-treePASSEDTesting B-tree remove: promote record from root of level-2 B-treePASSEDTesting B-tree remove: promote from right internal of level-2 B-treew/redistrib PASSEDTesting B-tree remove: promote from left internal of level-2 B-treew/redistrib PASSEDTesting B-tree remove: promote from right internal of level-2 B-treew/merge PASSEDTesting B-tree remove: promote from middle internal of level-2 B-treew/merge PASSEDTesting B-tree remove: merge 2 internal nodes to 1 in level-2 B-tree(l->r) PASSEDTesting B-tree remove: merge 2 internal nodes to 1 in level-2 B-tree(r->l) PASSEDTesting B-tree remove: merge 3 internal nodes to 2 in level-2 B-treePASSEDTesting B-tree remove: collapse level-2 B-tree back to level-1 (r->l)PASSEDTesting B-tree remove: create random level 4 B-tree and delete allrecords in random order PASSEDTesting B-tree remove: create random level 4 B-tree and delete allrecords by index, in random order PASSEDTesting B-tree remove: create random level 4 B-tree and delete allrecords by index, in increasing order PASSEDTesting B-tree remove: create random level 4 B-tree and delete allrecords by index, in decreasing order PASSEDTesting B-tree find: nearest neighbor less than a valuePASSEDTesting B-tree find: nearest neighbor greater than a valuePASSEDTesting B-tree delete: delete empty B-treePASSEDTesting B-tree delete: delete level-0 B-treePASSEDTesting B-tree delete: delete level-1 B-treePASSEDTesting B-tree delete: delete level-2 B-treePASSEDTesting B-tree modify: attempt to modify non-existant recordPASSEDTesting B-tree modify: modify record in leaf nodePASSEDTesting B-tree modify: modify record in internal nodePASSEDTesting B-tree modify: modify record in root nodePASSED

Testing with reopening B-tree:

Command terminated by signal 15

150.13user 39.62system 22:25.95elapsed 14%CPU (0avgtext+0avgdata494240maxresident)k

0inputs+0outputs (0major+54831minor)pagefaults 0swaps
make[4]: *** [btree2.chkexe_] Error 1

make[4]: Leaving directory`/local/pbisbal/gcc-5.4.0/hdf5-1.10.0-patch1/test'

make[3]: *** [build-check-s] Error 2

make[3]: Leaving directory`/local/pbisbal/gcc-5.4.0/hdf5-1.10.0-patch1/test'

make[2]: *** [test] Error 2

make[2]: Leaving directory`/local/pbisbal/gcc-5.4.0/hdf5-1.10.0-patch1/test'

make[1]: *** [check-am] Error 2

make[1]: Leaving directory`/local/pbisbal/gcc-5.4.0/hdf5-1.10.0-patch1/test'

make: *** [check-recursive] Error 1

On 07/25/2016 04:31 PM, Miller, Mark C. wrote:

Yeah, I meant ‘kill -9’ not ctrl-C. I would expect ctrl-C to justabort the current command (e.g. ‘make check’). By killing the runningprocess (lt-btree) alone, make check would just interpret that as somekind of failure and then move on to the next test.

Yes, you need a “true” parallel filesystem for any parallel tests. NFSwill NOT work.

Running on /local makes a lot of sense other than for parallel.Whenever I run on NFS, things take a lot longer due to all thecontention for that shared resource.


Mark

From: Hdf-forum <[email protected]<mailto:[email protected]>> on behalf of PrenticeBisbal <[email protected] <mailto:[email protected]>>Reply-To: HDF Users Discussion List <[email protected]<mailto:[email protected]>>

Date: Monday, July 25, 2016 at 1:22 PM

To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>

Subject: Re: [Hdf-forum] How long should 'make check' take?


    Where (e.g. on which kind of filesystem) are you trying to run
    ‘make check’?
    Can you kill -9 that one process and see if it proceeds to
    completion on remaining tests?


    Since I'm working on a serial build, right now, I'm using scratch
    disk local to the node I'm doing my builds on, mounted as '/local'
    When I do a control-C, the whole 'make check' process appears to
    end, it doesn't go on to the next test.

    I'm now doing a build with minimal options specified in the
    configure step to see if maybe combinations of options was causing
    the problem.

    For parallel tests, you *should* be using a parallel fileystem. I
    frequently forget about that.


    Do you mean a shared filesystem, like NFS, or a true parallel
    filesystem like Lustre or GPFS? After realizing my stupidity of
    trying to do the parallel make check on my local disk, I did move
    it to an NFS filesystem.

    Prentice

    On 07/25/2016 04:07 PM, Miller, Mark C. wrote:

    30+mins *does* seem a bit long.

    Typically, a ‘make check’ for me completes *all* tests in a total
    of about 10-15 mins at the most.

    Where (e.g. on which kind of filesystem) are you trying to run
    ‘make check’?

    For parallel tests, you *should* be using a parallel fileystem. I
    frequently forget about that.

    If the parallel filesystem is *extremely* busy and/or not setup
    to deal with small I/Os and/or files (as the tests generate), I
    could see there being *some* additional delay in completion.

    The btree test I think is testing HDF5’s internal btree data
    structures for group objects. So, if its doing any I/O I think
    its a bunch of tiny HDF5 metadata stuff.

    Can you kill -9 that one process and see if it proceeds to
    completion on remaining tests?

    Mark

    From: Hdf-forum <[email protected]
    <mailto:[email protected]>> on behalf of
    Prentice Bisbal <[email protected] <mailto:[email protected]>>
    Reply-To: HDF Users Discussion List <[email protected]
    <mailto:[email protected]>>
    Date: Monday, July 25, 2016 at 12:59 PM
    To: "[email protected]
    <mailto:[email protected]>"
    <[email protected] <mailto:[email protected]>>
    Subject: [Hdf-forum] How long should 'make check' take?

        How long should make check normally take to run for a serial
        build?
        After 'make check' was hanging for me at testphdf5 when
        building HDF5
        with parallel support, make check now seems to be hanging
        when testing
        btree2. It's been at the same spot for 30 minutes or more,
        but this
        time, I see the process (lt-btree2) using CPU and memory in
        top, just
        not much of it. Is this normal? I would expect file I/O tests
        to only
        take a few minutes, not 30+ minutes.

--Prentice



        _______________________________________________
        Hdf-forum is for HDF software users discussion.
        [email protected]
        <mailto:[email protected]>
        http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
        Twitter: https://twitter.com/hdf5



    _______________________________________________
    Hdf-forum is for HDF software users discussion.
    
[email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
    Twitter:https://twitter.com/hdf5




_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

Re: [Hdf-forum] How long should 'make check' take?

Reply via email to