On 04/11/2017 12:08 PM, Tom Lane wrote: > Robert Haas <robertmh...@gmail.com> writes: >> On Tue, Apr 11, 2017 at 11:32 AM, Andrew Dunstan >> <andrew.duns...@2ndquadrant.com> wrote: >>> This buildfarm run as you can see takes 33m32s, and the Tap tests take a >>> combined 19m52s of that time. >> I don't think it's quite fair to complain about the TAP tests taking a >> long time to run as a general matter. Many people here have long >> wanted a separate test-suite for long running tests that can be run to >> really check everything possible, and apparently, TAP tests are it. >> What I think would be more useful is to break down where the time is >> getting spent. It may be that some of those tests are not adding >> value proportionate to their runtime.
Well, you can get a lot of information on timings in crake's latest builds for example, or nightjar's. Here's an interesting fact: almost all the time taken up in the scripts test set seems to be consumed in running initdb over and over. The actual tests take almost no time at all. Other test sets might have similar stats, I am still digging. +1 for doing some assessments on test value, and also for possibly creating several test categories, based on both speed and criticality. I could imaging having some very long running tests, using TAP or pg_regress, and optionally enabling them in a buildfarm critter. I'm not at all opposed to having TAP tests - I have just put in quite a bit of time extending and improving buildfarm coverage of them. And I'm probably about to do some more work to enable similar functionality for MSVC. > The other thing that might be useful here is to push on parallelizing > buildfarm runs. Admittedly that will do nothing for the oldest and > slowest buildfarm critters, but for reasonably modern hardware the > serialization of the tests really handicaps you. We seem to have > fixed that for manual application of "make check-world", at least > if you know the right magic incantations to parallelize it; but > AFAIK the buildfarm script is entirely serial. Yeah, it is, and changing that won't be simple. Say 3 steps run in parallel fail? What do we do? I guess we can just pick the first (by some definition) and discard the others. Parallelizing individual steps might be simpler, I'd have to take a look. Honestly, I'm not that concerned about how long it takes the buildfarm client per se to run, although I am working on facilities for it where timing will matter to some people. And anyone who is concerned about how long it takes on their machine can run without --enable-tap-tests. Not sure what the magic incantation is other than "make -j nn"? One reason I haven't supported "make -j nn" everywhere (it is supported for making core, contrib and test_modules) is that the output tends to get rather jumbled, and I didn't want to impose that extra burden in people trying to decipher the results. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
