Den 2010-09-10 07:38 skrev Ralf Wildenhues: > Hi Peter, > > * Peter Rosin wrote on Fri, Sep 10, 2010 at 12:35:59AM CEST: >> Den 2010-09-09 22:05 skrev Charles Wilson: >>> On 9/9/2010 3:56 PM, Ralf Wildenhues wrote: >>>> Secondly, I can help with unsupervised git bisect if you need. In >>>> <http://lists.gnu.org/archive/html/bug-libtool/2010-09/msg00006.html>, > >> I adjusted both reconfdirs and TESTS. You tend to learn how to do that >> quickly in windows-land. The payoff for that learning curve is almost >> instantaneous. > > Hehe. > >>> I'll take a look, but it may actually be faster to do it old-school: >>> just trace thru the failing case in git master-NOW, figure out what's >>> going wrong, and fix it. >> >> In my current setup it's hard to automate, as I do git and bootstrap on >> Cygwin only, then test Cygwin and when that is done I switch to MSYS and >> test there too. > > Ok, so the payoff for changing that will be noticeable then, too. > Install msysgit (someplace else), make sure the git program is found > in PATH in your normal MSYS installation. Profit.
Well, only until you mess things up and you don't know what you have done. Running autotools from both left and right seems like a sure way to create a mess. And when I do foul things up, I would have to go back from the top and that's never fun. Also, I don't trust MSYS to not add dos style line endings somewhere, or to not foul up the permissions so that I accidentally start committing executable source files and what not. I just don't feel comfortable with that. But the above is not related to running a git bisect in general. Installing and setting up msysgit and current autotools and getting things to work would not have been faster than just doing it the way I did. At least I don't think so. Not for me anyway, remember that I'm not familiar with running various versions of these tools, and that I don't know exactly how they interact and how to control wrappers etc. I could easily have spent hours barking up the wrong tree. Anyway, the short term benefit was not obvious. >> I have version mismatches for automake and autoconf in >> MSYS and Cygwin, so if I'm not careful things go belly up. I should >> perhaps try to have matching autotools, but I'm used to not having them >> at all in MSYS, and I'm not sure if things will be screwed up with paths >> leaking in from the other posix world if I mix, so I'm just doing it in >> a way that I know works. > > Sure. I have current autotools in $HOME/local in MSYS, put that early > in PATH, profit (also because it executes a bit faster than Cygwin). > >> Watch out, rant coming up. You have been warned. >> >> My problem(?) is that libltdl issues are the hardest to fix, so there >> are multiple issues that are not fixed for MSVC. That, and that most >> libltdl tests are too wide for my liking, so the libltdl part of the >> testsuite tend to either be all failing or all failing even if you have >> made an improvement. Which makes it hard to use the testsuite to >> determine if you have made any progress when you are testing something. > > Fully agreed. I already mentioned that improving ltdl coverage would be > paramount for any rewriting in that area, this is just another reason. > >> It is also hard to deduce if you have caused some spectacular new fail >> with a patch, if the test in question fails before it got to the newly >> introduced badness. > > So are you saying we should have compile+link-only ltdl tests? > Which ones are likely to fail early? I honestly don't know. It's probably not fixable with only testsuite work. When looking up what goes belly up with libltdl, you will almost always need to dig into the C code, or in some symbol crap that's too low level for the testsuite to be able to reasonably know anything about. Unless the testsuite is taught internals such as the expected content of the *S.c file in various circumstances etc, but creating and maintaining such tests would be a nightmare. To test various parts of the C code of libltdl without doing a full test (that would only be able to say "fail"), you would probably need to add some tracing mechanism to see that the expected code paths are taken. But the problem is of course that this is mainly needed when porting to alien stuff like Windows, and in those alien cases the normally expected code paths are perhaps not always taken for various other reasons, voiding the whole scheme. Regarding other stuff, like the hardship with the stresstest.at looping galore, I think that instead of having the current loops, it would be useful to run all 2*3*2*3*3*2 variations regardless, even if one variation failed in the beginning. If the termination of the test could be delayed, and the result of the LT_AT_{EXEC_,}CHECK macros could be recorded and shown at the end of the stresstest log, I think that would have been extremely useful. IOW, a table something like this instead of only fail:282 (and the inevitable hunt for which 282 that would be, it's much better with the eval at line 263, but still not obvious): linkargs="-no-undefined " rel st selfsyms result 1 ok 2 -export-symbols dlselfsyms ok 3 -static ok 4 -static -export-symbols dlselfsyms ok 5 -no-install ok 6 -no-install -export-symbols dlselfsyms ok 7 ./ ok 8 ./ -export-symbols dlselfsyms ok 9 ./ -static ok 10 ./ -static -export-symbols dlselfsyms ok 11 ./ -no-install ok 12 ./ -no-install -export-symbols dlselfsyms ok 13 `pwd`/ ok 14 `pwd`/ -export-symbols dlselfsyms ok 15 `pwd`/ -static ok 16 `pwd`/ -static -export-symbols dlselfsyms ok 17 `pwd`/ -no-install ok 18 `pwd`/ -no-install -export-symbols dlselfsyms ok linkargs="-no-undefined -rpath /nonexistent" rel st selfsyms result 19 ok 20 -export-symbols dlselfsyms ok 21 -static ok 22 -static -export-symbols dlselfsyms ok 23 -no-install ok 24 -no-install -export-symbols dlselfsyms ok 25 ./ ok 26 ./ -export-symbols dlselfsyms ok 27 ./ -static ok 28 ./ -static -export-symbols dlselfsyms ok 29 ./ -no-install ok 30 ./ -no-install -export-symbols dlselfsyms ok 31 `pwd`/ ok 32 `pwd`/ -export-symbols dlselfsyms ok 33 `pwd`/ -static ok 34 `pwd`/ -static -export-symbols dlselfsyms ok 35 `pwd`/ -no-install ok 36 `pwd`/ -no-install -export-symbols dlselfsyms ok linkargs="-no-undefined -export-symbols-regex "v.*" " rel st selfsyms result 37 ok 38 -export-symbols dlselfsyms ok 39 -static ok 30 -static -export-symbols dlselfsyms ok 41 -no-install ok 42 -no-install -export-symbols dlselfsyms fail:282 43 ./ fail:263 44 ./ -export-symbols dlselfsyms fail:263 45 ./ -static fail:263 46 ./ -static -export-symbols dlselfsyms fail:263 47 ./ -no-install fail:263 48 ./ -no-install -export-symbols dlselfsyms fail:263 49 `pwd`/ ok 40 `pwd`/ -export-symbols dlselfsyms ok 51 `pwd`/ -static ok 52 `pwd`/ -static -export-symbols dlselfsyms ok 53 `pwd`/ -no-install ok 54 `pwd`/ -no-install -export-symbols dlselfsyms ok ... I.e., one line for each run of the innermost loop. If a test leading up to that innermost test failed, all affected entries in the innermost loop would be marked the same way (fail:263 above). Patterns in such a table would probably help pinpointing regressions. I'm having fuzzy dreams of course. >> Even if the the new fail happens before the old >> fail, it's still harder to discriminate fail:63 from fail:67 than it is >> to discriminate fail:63 from ok. It also doesn't really help that I'm >> not using libltdl for anything real: I'm not really familiar with what >> the expected behavior is and it tends to not be that important and >> "itchy" to me. > > What would help here, though? More descriptive comments in the test > sources? I don't know what to do about it, I was just ranting on about my pain. I think the fundamental problem is that I'm not using libltdl for anything real, so nothing to scratch. >> And then, as always, there's the wall clock issue. > > Yep. Dunno what to do about that. Not much to do about that, no. >> stresstest is now ok for MSVC, so it is finally really useful in >> detecting regressions, and no longer a "masking failure rat hole". > > Cool. Yes. I wanted to finish up with something positive. :-) Cheers, Peter