Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
tfiala added a comment. I'm glad that worked for you, Dawn! Users: tfiala (Author) dawn (Auditor) http://reviews.llvm.org/rL246794 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
I agree with Zachary that we shouldn't mess around with options piecemeal, but let Todd get done with this first round of fixes then go take a hatchet to the options that dotest has, hopefully removing all the ones that nobody needs and rationalizing the others. Jim > On Sep 4, 2015, at 1:31 PM, via lldb-commits > wrote: > > Understood. I don't think a short option for --output-on-success is needed > if -v turns it on. A short option for --test-subdir would be useful > however. Looks like these short options are still available: > BHIJKLMOQUVWYZjoz. > -z seems as good a choice as any :) ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
dawn accepted this commit. dawn added a comment. Apparently now you must use: ./dotest.py --output-on-success -v --executable $INSTALLDIR/bin/lldb to see the same detailed output as you used to see with: ./dosep.py -s --options "-v --executable $INSTALLDIR/bin/lldb" or: ./dotest.py -v --executable $INSTALLDIR/bin/lldb So no info is lost, just have to add the extra option --output-on-success. Users: tfiala (Author) dawn (Auditor) http://reviews.llvm.org/rL246794 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
On Fri, Sep 04, 2015 at 12:44:36PM -0700, Todd Fiala wrote: > Please try: > > ./dotest.py --output-on-success -v --executable $INSTALLDIR/bin/lldb Yes, this appears to work. With this, I'm now back up to my old totals :) lldb_failures=0 lldb_errors=0 lldb_passes=1146 lldb_total=1378 Thank you!! I'd recommend turning --output-on-success on when -v is used so that the loss of information doesn't appear as a regression. That way, dotest.py -v will always show the same detailed info in both single and multithreaded modes. > I couldn't maintain the '-s' > mapping for the dosep option because it collided with an existing dotest.py > option for session directory specification. Understood. I don't think a short option for --output-on-success is needed if -v turns it on. A short option for --test-subdir would be useful however. Looks like these short options are still available: BHIJKLMOQUVWYZjoz. -z seems as good a choice as any :) ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
On Fri, Sep 04, 2015 at 12:40:12PM -0700, Todd Fiala wrote: > > Now there is no equivelent to "dosep.py -s", and: > >./dotest.py -v --executable $INSTALLDIR/bin/lldb > > All of dosep's args moved over to dotest. I think if you wanted dosep.py > -s behavior, you only need to use --output-on-success (long form of what > used to be -s in dosep I believe). The reason I couldn't maintain it as -s > is that dotest.py already had a -s abbreviation (for session dir > specification). Thanks! I'll give --output-on-success a try... ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
Yeah I'm actually hoping that once we move over to a test event stream that can be formatted with a customizable formatter, we can pretty much start deprecating the other output streams aside from the pure test run stdout/stderr. On Fri, Sep 4, 2015 at 12:45 PM, Zachary Turner wrote: > > > On Fri, Sep 4, 2015 at 12:40 PM Todd Fiala wrote: > >> > Now there is no equivelent to "dosep.py -s", and: >> >./dotest.py -v --executable $INSTALLDIR/bin/lldb >> >> All of dosep's args moved over to dotest. I think if you wanted dosep.py >> -s behavior, you only need to use --output-on-success (long form of what >> used to be -s in dosep I believe). The reason I couldn't maintain it as -s >> is that dotest.py already had a -s abbreviation (for session dir >> specification). >> > > One reason I would like to move towards fewer options in dotest, it's > crazy how many options there are. I'm willing to bet 80% of them are not > used by anyone and we could probably remove them. > > Maybe after you finish a lot of the cleanup work and we decide on a final > vision of how everything should be structured, we can do a pass over the > options and decide which ones we can remove. > -- -Todd ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
On Fri, Sep 4, 2015 at 12:40 PM Todd Fiala wrote: > > Now there is no equivelent to "dosep.py -s", and: > >./dotest.py -v --executable $INSTALLDIR/bin/lldb > > All of dosep's args moved over to dotest. I think if you wanted dosep.py > -s behavior, you only need to use --output-on-success (long form of what > used to be -s in dosep I believe). The reason I couldn't maintain it as -s > is that dotest.py already had a -s abbreviation (for session dir > specification). > One reason I would like to move towards fewer options in dotest, it's crazy how many options there are. I'm willing to bet 80% of them are not used by anyone and we could probably remove them. Maybe after you finish a lot of the cleanup work and we decide on a final vision of how everything should be structured, we can do a pass over the options and decide which ones we can remove. ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
Please try: ./dotest.py --output-on-success -v --executable $INSTALLDIR/bin/lldb I think you'll find that is identical output to what used to be this: ./dosep.py -s --options "-v --executable $INSTALLDIR/bin/lldb" If you find differently, please let me know. I couldn't maintain the '-s' mapping for the dosep option because it collided with an existing dotest.py option for session directory specification. I'll wait to hear if that works for you. (I tested it when I added it, and I didn't mess with the logic aside from the flag name, so if it is broken I definitely want to fix it --- you just have to use a different flag name). On Fri, Sep 4, 2015 at 12:35 PM, wrote: > On Fri, Sep 04, 2015 at 06:25:44PM +, Todd Fiala wrote: > > tfiala added a comment. > > > > Dawn, the output supported by the command shouldn't have been changed. > I'll have a peek at what may have caused that to happen. All this change > was trying to do was stop the need to call dosep and roll that behavior > into dotest. > > Can you add the 'dosep.py -s -o "-v"' behavior to 'dotest.py -v'? I > think that may be all that's needed in order to show the same output as > before with all the desired details. Note: before the -s option was > added to dosep.py, 'dosep.py -o "-v"' would only show the info that > 'dotest.py -v' shows now (after your change). Afer the -s option was > added to dosep.py, 'dosep.py -s -o "-v"' would report the same info as > 'dotest.py -v' did before your change. I.e. dosep.py's -s option may be > the key to fixing the regressions in 'dotest.py -v'. > > Thanks for looking into this! > -Dawn > -- -Todd ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
I see how that could have been confusing. I should have called that out more. On Fri, Sep 4, 2015 at 12:40 PM, Todd Fiala wrote: > > Now there is no equivelent to "dosep.py -s", and: > >./dotest.py -v --executable $INSTALLDIR/bin/lldb > > All of dosep's args moved over to dotest. I think if you wanted dosep.py > -s behavior, you only need to use --output-on-success (long form of what > used to be -s in dosep I believe). The reason I couldn't maintain it as -s > is that dotest.py already had a -s abbreviation (for session dir > specification). > > On Fri, Sep 4, 2015 at 12:02 PM, wrote: > >> On Fri, Sep 04, 2015 at 06:25:44PM +, Todd Fiala wrote: >> > tfiala added a comment. >> > >> > Dawn, the output supported by the command shouldn't have been changed. >> I'll have a peek at what may have caused that to happen. All this change >> was trying to do was stop the need to call dosep and roll that behavior >> into dotest. >> >> Both of these commands: >> ./dosep.py -s --options "-v --executable $INSTALLDIR/bin/lldb" >> ./dotest.py -v --executable $INSTALLDIR/bin/lldb >> >> used to print results like the following: >> Collected 6 tests >> >> 1: test_double_type_from_block_with_dsym >> (TestFloatTypes.FloatTypesTestCase) >> Test that double-type variables are displayed correctly from a >> block. ... ok >> 2: test_double_type_with_dsym (TestFloatTypes.FloatTypesTestCase) >> Test that double-type variables are displayed correctly. ... ok >> 3: test_double_type_with_dwarf (TestFloatTypes.FloatTypesTestCase) >> Test that double-type variables are displayed correctly. ... ok >> 4: test_float_type_from_block_with_dsym >> (TestFloatTypes.FloatTypesTestCase) >> Test that float-type variables are displayed correctly from a block. >> ... ok >> 5: test_float_type_with_dsym (TestFloatTypes.FloatTypesTestCase) >> Test that float-type variables are displayed correctly. ... ok >> 6: test_float_type_with_dwarf (TestFloatTypes.FloatTypesTestCase) >> Test that float-type variables are displayed correctly. ... ok >> >> -- >> Ran 6 tests in 6.844s >> >> RESULT: PASSED (6 passes, 0 failures, 0 errors, 0 skipped, 0 >> expected failures, 0 unexpected successes) >> >> [TestFloatTypes.py PASSED] >> >> Now there is no equivelent to "dosep.py -s", and: >> ./dotest.py -v --executable $INSTALLDIR/bin/lldb >> >> prints results like the following: >> Testing: 395 test suites, 8 threads >> >> 0 out of 395 test suites processed - ['./dotest.py', '-v', >> '--executable', >> >> '/Users/testuser/build/workspace/LLVM-Clang-LLDB_master_release_OSX/llvm/build_ninja/lldb_redist/usr/local/bin/lldb'] >> >> 1 out of 395 test suites processed - >> TestCompileRunToBreakpointTurnaround.py >> [...] >> 395 out of 395 test suites processed - TestRecursiveTypes.py >> Ran 395 test suites (0 failed) (0.00%) >> Ran 826 test cases (0 failed) (0.00%) >> >> So we no longer have info on what tests ran, can't figure out what >> tests were skipped, can't calculate the correct totals, etc. >> > > > > -- > -Todd > -- -Todd ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
> Now there is no equivelent to "dosep.py -s", and: >./dotest.py -v --executable $INSTALLDIR/bin/lldb All of dosep's args moved over to dotest. I think if you wanted dosep.py -s behavior, you only need to use --output-on-success (long form of what used to be -s in dosep I believe). The reason I couldn't maintain it as -s is that dotest.py already had a -s abbreviation (for session dir specification). On Fri, Sep 4, 2015 at 12:02 PM, wrote: > On Fri, Sep 04, 2015 at 06:25:44PM +, Todd Fiala wrote: > > tfiala added a comment. > > > > Dawn, the output supported by the command shouldn't have been changed. > I'll have a peek at what may have caused that to happen. All this change > was trying to do was stop the need to call dosep and roll that behavior > into dotest. > > Both of these commands: > ./dosep.py -s --options "-v --executable $INSTALLDIR/bin/lldb" > ./dotest.py -v --executable $INSTALLDIR/bin/lldb > > used to print results like the following: > Collected 6 tests > > 1: test_double_type_from_block_with_dsym > (TestFloatTypes.FloatTypesTestCase) > Test that double-type variables are displayed correctly from a > block. ... ok > 2: test_double_type_with_dsym (TestFloatTypes.FloatTypesTestCase) > Test that double-type variables are displayed correctly. ... ok > 3: test_double_type_with_dwarf (TestFloatTypes.FloatTypesTestCase) > Test that double-type variables are displayed correctly. ... ok > 4: test_float_type_from_block_with_dsym > (TestFloatTypes.FloatTypesTestCase) > Test that float-type variables are displayed correctly from a block. > ... ok > 5: test_float_type_with_dsym (TestFloatTypes.FloatTypesTestCase) > Test that float-type variables are displayed correctly. ... ok > 6: test_float_type_with_dwarf (TestFloatTypes.FloatTypesTestCase) > Test that float-type variables are displayed correctly. ... ok > > -- > Ran 6 tests in 6.844s > > RESULT: PASSED (6 passes, 0 failures, 0 errors, 0 skipped, 0 > expected failures, 0 unexpected successes) > > [TestFloatTypes.py PASSED] > > Now there is no equivelent to "dosep.py -s", and: > ./dotest.py -v --executable $INSTALLDIR/bin/lldb > > prints results like the following: > Testing: 395 test suites, 8 threads > > 0 out of 395 test suites processed - ['./dotest.py', '-v', > '--executable', > > '/Users/testuser/build/workspace/LLVM-Clang-LLDB_master_release_OSX/llvm/build_ninja/lldb_redist/usr/local/bin/lldb'] > > 1 out of 395 test suites processed - > TestCompileRunToBreakpointTurnaround.py > [...] > 395 out of 395 test suites processed - TestRecursiveTypes.py > Ran 395 test suites (0 failed) (0.00%) > Ran 826 test cases (0 failed) (0.00%) > > So we no longer have info on what tests ran, can't figure out what > tests were skipped, can't calculate the correct totals, etc. > -- -Todd ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
On Fri, Sep 04, 2015 at 06:25:44PM +, Todd Fiala wrote: > tfiala added a comment. > > Dawn, the output supported by the command shouldn't have been changed. I'll > have a peek at what may have caused that to happen. All this change was > trying to do was stop the need to call dosep and roll that behavior into > dotest. Can you add the 'dosep.py -s -o "-v"' behavior to 'dotest.py -v'? I think that may be all that's needed in order to show the same output as before with all the desired details. Note: before the -s option was added to dosep.py, 'dosep.py -o "-v"' would only show the info that 'dotest.py -v' shows now (after your change). Afer the -s option was added to dosep.py, 'dosep.py -s -o "-v"' would report the same info as 'dotest.py -v' did before your change. I.e. dosep.py's -s option may be the key to fixing the regressions in 'dotest.py -v'. Thanks for looking into this! -Dawn ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
On Fri, Sep 04, 2015 at 06:25:44PM +, Todd Fiala wrote: > tfiala added a comment. > > Dawn, the output supported by the command shouldn't have been changed. I'll > have a peek at what may have caused that to happen. All this change was > trying to do was stop the need to call dosep and roll that behavior into > dotest. Both of these commands: ./dosep.py -s --options "-v --executable $INSTALLDIR/bin/lldb" ./dotest.py -v --executable $INSTALLDIR/bin/lldb used to print results like the following: Collected 6 tests 1: test_double_type_from_block_with_dsym (TestFloatTypes.FloatTypesTestCase) Test that double-type variables are displayed correctly from a block. ... ok 2: test_double_type_with_dsym (TestFloatTypes.FloatTypesTestCase) Test that double-type variables are displayed correctly. ... ok 3: test_double_type_with_dwarf (TestFloatTypes.FloatTypesTestCase) Test that double-type variables are displayed correctly. ... ok 4: test_float_type_from_block_with_dsym (TestFloatTypes.FloatTypesTestCase) Test that float-type variables are displayed correctly from a block. ... ok 5: test_float_type_with_dsym (TestFloatTypes.FloatTypesTestCase) Test that float-type variables are displayed correctly. ... ok 6: test_float_type_with_dwarf (TestFloatTypes.FloatTypesTestCase) Test that float-type variables are displayed correctly. ... ok -- Ran 6 tests in 6.844s RESULT: PASSED (6 passes, 0 failures, 0 errors, 0 skipped, 0 expected failures, 0 unexpected successes) [TestFloatTypes.py PASSED] Now there is no equivelent to "dosep.py -s", and: ./dotest.py -v --executable $INSTALLDIR/bin/lldb prints results like the following: Testing: 395 test suites, 8 threads 0 out of 395 test suites processed - ['./dotest.py', '-v', '--executable', '/Users/testuser/build/workspace/LLVM-Clang-LLDB_master_release_OSX/llvm/build_ninja/lldb_redist/usr/local/bin/lldb'] 1 out of 395 test suites processed - TestCompileRunToBreakpointTurnaround.py [...] 395 out of 395 test suites processed - TestRecursiveTypes.py Ran 395 test suites (0 failed) (0.00%) Ran 826 test cases (0 failed) (0.00%) So we no longer have info on what tests ran, can't figure out what tests were skipped, can't calculate the correct totals, etc. ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
I guess the thing to do in that case would be to fix it. I know priorities and all, but I don't want to hold back progress supporting an external workflow. In any case, Todd says that the output shouldn't have changed, so it may still be that you can continue doing this (for now). But at some point you may have to jump in and fix the totals if the ability to scrape the logs goes away, and the reported total is still wrong. On Fri, Sep 4, 2015 at 11:41 AM wrote: > On Fri, Sep 04, 2015 at 06:17:03PM +, Zachary Turner wrote: > > To be honest I think grepping the output is the wrong way to go about > > counting the tests. > > [...] Log scraping is kind of a horrible thing to do, > > Agreed, but it was the only way to get correct totals. I would love for > dosep.py to have been able to count the results correctly so I wouldn't > have had to resort to "log scraping". dotest.py at least used to be > able to count the totals correctly, but no longer can since it now > inherrited the same bug that dosep.py had. > ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
On Fri, Sep 04, 2015 at 06:17:03PM +, Zachary Turner wrote: > To be honest I think grepping the output is the wrong way to go about > counting the tests. > [...] Log scraping is kind of a horrible thing to do, Agreed, but it was the only way to get correct totals. I would love for dosep.py to have been able to count the results correctly so I wouldn't have had to resort to "log scraping". dotest.py at least used to be able to count the totals correctly, but no longer can since it now inherrited the same bug that dosep.py had. ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
tfiala added a comment. Dawn, the output supported by the command shouldn't have been changed. I'll have a peek at what may have caused that to happen. All this change was trying to do was stop the need to call dosep and roll that behavior into dotest. Zachary, I'm in the process of add xUnit (and other pluggable) output support so we can flip a flag and have traditional build/test bots generate test result reports. Users: tfiala (Author) dawn (Auditor) http://reviews.llvm.org/rL246794 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
To be honest I think grepping the output is the wrong way to go about counting the tests. If there's a regression in that you no longer have access to a piece of information that you need, then we should fix it by printing the value you need. Log scraping is kind of a horrible thing to do, because it means you are going to be broken by changes to the format of the output, and it will hold back progress on the test suite making real improvements in maintainability and performance because all the different people doing their own custom log scraping will get upset if anything breaks. Anyone who has worked on the test suite can vouch for the fact that it is in need of some major love, and if that means a transition period where things are a little wonky, and we lose some things only to get them back later, I'm ok with that. On Fri, Sep 4, 2015 at 11:04 AM Dawn Perchik via lldb-commits < lldb-commits@lists.llvm.org> wrote: > dawn added a comment. > > I really liked that dosep.py -s and dotest.py would report the run of each > test case: > > Collected 6 tests > > 1: test_double_type_from_block_with_dsym > (TestFloatTypes.FloatTypesTestCase) > Test that double-type variables are displayed correctly from a block. > ... ok > 2: test_double_type_with_dsym (TestFloatTypes.FloatTypesTestCase) > Test that double-type variables are displayed correctly. ... ok > 3: test_double_type_with_dwarf (TestFloatTypes.FloatTypesTestCase) > Test that double-type variables are displayed correctly. ... ok > 4: test_float_type_from_block_with_dsym > (TestFloatTypes.FloatTypesTestCase) > Test that float-type variables are displayed correctly from a block. ... > ok > 5: test_float_type_with_dsym (TestFloatTypes.FloatTypesTestCase) > Test that float-type variables are displayed correctly. ... ok > 6: test_float_type_with_dwarf (TestFloatTypes.FloatTypesTestCase) > Test that float-type variables are displayed correctly. ... ok > > -- > Ran 6 tests in 6.844s > > RESULT: PASSED (6 passes, 0 failures, 0 errors, 0 skipped, 0 expected > failures, 0 unexpected successes) > > [TestFloatTypes.py PASSED] > > I would like to have that back please. > > > Users: > tfiala (Author) > dawn (Auditor) > > http://reviews.llvm.org/rL246794 > > > > ___ > lldb-commits mailing list > lldb-commits@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits > ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
dawn added a comment. I really liked that dosep.py -s and dotest.py would report the run of each test case: Collected 6 tests 1: test_double_type_from_block_with_dsym (TestFloatTypes.FloatTypesTestCase) Test that double-type variables are displayed correctly from a block. ... ok 2: test_double_type_with_dsym (TestFloatTypes.FloatTypesTestCase) Test that double-type variables are displayed correctly. ... ok 3: test_double_type_with_dwarf (TestFloatTypes.FloatTypesTestCase) Test that double-type variables are displayed correctly. ... ok 4: test_float_type_from_block_with_dsym (TestFloatTypes.FloatTypesTestCase) Test that float-type variables are displayed correctly from a block. ... ok 5: test_float_type_with_dsym (TestFloatTypes.FloatTypesTestCase) Test that float-type variables are displayed correctly. ... ok 6: test_float_type_with_dwarf (TestFloatTypes.FloatTypesTestCase) Test that float-type variables are displayed correctly. ... ok -- Ran 6 tests in 6.844s RESULT: PASSED (6 passes, 0 failures, 0 errors, 0 skipped, 0 expected failures, 0 unexpected successes) [TestFloatTypes.py PASSED] I would like to have that back please. Users: tfiala (Author) dawn (Auditor) http://reviews.llvm.org/rL246794 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
dawn added a subscriber: dawn.
dawn raised a concern with this commit.
dawn added a comment.
I'm very unhappy with this change. Before I could count up all the test
failures using:
./dotest.py -v --executable $INSTALLDIR/bin/lldb 2>&1 | tee
$INSTALLDIR/lldb_test_out.log || true
lldb_failures=`grep -E "^RESULT:" lldb_test_out.log | grep failures | awk
'{count+=$5} END {print count}'` || true
lldb_errors=`grep -E "^RESULT:" lldb_test_out.log | grep errors | awk
'{count+=$7} END {print count}'` || true
lldb_passes=`grep -E "^RESULT: " lldb_test_out.log | sed 's/(//' | awk
'{count+=$3} END {print count}'` || true
lldb_total=`grep -E "^Ran [0-9]+ tests? in" lldb_test_out.log | awk
'{count+=$2} END {print count}'`
And get the correct totals:
lldb_failures=2
lldb_errors=0
lldb_passes=1143
lldb_total=1376
Now it seems we are back to the broken behavior of dosep.py where there's no
way to count the correct totals since it seems to forget to count 25% of the
test results. Only now I have no workaround, because even dotest.py can't
report the changes correctly. Please revert this so I can use dosep.py as
before, or fix dotest.py to count correctly.
Thank you,
-Dawn
Users:
tfiala (Author)
dawn (Auditor)
http://reviews.llvm.org/rL246794
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [Diffusion] rL246794: Roll dosep.py parallel test runner into dotest.py command line
dawn added a subscriber: lldb-commits. Users: tfiala (Author) http://reviews.llvm.org/rL246794 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
