More information after another build iteration. The two tests that failed after updating to the unhinted images were subtests of tests that were failing earlier. If a single test function outputs multiple images, image comparison stops after the first mismatched image. So there's nothing peculiar about these tests, it's just that the system wasn't saying they were failing before since they were short-circuited by earlier failures. I wonder if it's possible to run through all the images and batch up all the failures together, so we don't have these "hidden" failures -- might mean fewer iterations with the buildbots down the road.
Good news is this does point to having the font problem licked. Mike On 09/08/2009 12:47 PM, Michael Droettboom wrote: > Interesting result. I pulled all of the new "actual" files from the 21 > failing tests on the buildbots to my local machine and all of those > tests now pass for me. Good. Interestingly, there are still two tests > failing on my machine which did not fail on the buildbots, so I can't > grab the buildbots' new output. Could this just be a thresholding issue > for the tolerance value? I'm a little wary of "polluting" the baseline > images with images from my machine which doesn't have our "standard" > version of Freetype, so I'll leave those out of SVN for now, but will go > ahead and commit the new baseline images from the buildbots. Assuming > these two mystery failures are resolved by pulling new images from the > buildbots, I think this experiment with turning of hinting is a success. > > As an aside, is there an easy way to update the baselines I'm missing? > At the moment, I'm copying each result file to the correct folder under > tests/baseline_images, but it takes me a while because I don't know the > heirarchy by heart and there are 22 failures. I was expecting to just > manually verify everything was ok and then "cp *.png" from my scratch > tests folder to baseline_images and let SVN take care of which files had > actually changed. This is just the naive feedback of a new set of eyes: > it's extremely useful and powerful what you've put together here. > > Mike > > On 09/08/2009 12:06 PM, Andrew Straw wrote: > >> Michael Droettboom wrote: >> >> >>> Doing so, my results are even *less* in agreement with the baseline, but >>> the real question is whether my results are in agreement with those on >>> the buildbot machines with this change to forcibly turn hinting off. I >>> should no pretty quickly when the buildbots start complaining in a few >>> minutes and I can look at the results ;) >>> >>> >>> >> Yes, even though the waterfall is showing green (for the next 2 minutes >> until my buildbot script bugfix gets run), it's pretty clear from the >> image failure page that disabling hinting introduced changes to the >> generated figure appearance. It will be interesting to see if, after >> checking in the newly generated actual images as the new baseline, the >> tests start passing on your machine with the newer freetype. >> >> In a footnote to myself, I think the ImageComparisonFailure exception >> should tell nose that the test failed, not that there was an error. >> >> -Andrew >> >> > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
