All I could add to what Ed and Traian have said is: - try to keep the tests neutral in all other respects: if you can, you want to minimize any indications of what tool the image came from, remove extraneous toolbars and scrollbars, etc -- anything subconsciously biasing or visually distracting
- the quality (and settings) of the display monitor are very important. If you can, do the tests on the same monitor with the same lighting in the room -mpg > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Traian Stanev > Sent: Wednesday, October 10, 2007 7:26 AM > To: OSGeo Discussions > Subject: RE: [OSGeo-Discuss] How to assess rendering quality? > > > Antialiasing, when done wrong is just "making the edges > intermediate colors". > > When done right it also involves subpixel positioning which > improves not only the visual appearance but also the relative > accuracy of lines -- i.e. visual weight is most correctly > distributed over exact position of the line given the > discrete sampling frequency (pixels). There is actually solid > scientific basis in that (Niquist-Shannon theorem). > "Sharpness of detail" in non-AA lines is fake accuracy. > > Unfortunately antialiasing is also hard to get right, because > it depends on gamma correction. For example compare the > antialiased output here > (http://www.realtimerendering.com/gamma10.png with gamma = > 1.0) and here with gamma 2.2 > (http://www.realtimerendering.com/gamma22.png). They are both > antialiased using the same algorithm, yet one looks better > than the other. > > Asking people to compare smooth versus sharp will be > sensitive not only to the subjective appeal of the picture, > but also by the ambient lighting conditions, the gamma used > by the AA algorithm versus the gamma of the monitor used for > the test, and by whether or not the AA algorithm used > subpixel positioning. Also, it depends on whether the image > reading application honors the gamma value stored in the > image. In addition, you will get "fanboy bias" where people > will prefer the output they have seen before. This comes up > with things like font glyph rasterization surveys, where > people who are more familiar with Macs prefer blurry but > correct glyphs while people who use predominantly Windows > prefer sharp but deformed glyphs, only because that's what > they are used to seeing. So you will need to have lots of > questions that ask for the same thing using screenshots that > do not say which product they are from. You should also > include output from a third party app like Adobe Acrobat. I > would also include test questions where the two images being > compared are identical. > > > > Traian > > > > > > > > -----Original Message----- > > From: [EMAIL PROTECTED] [mailto:discuss- > > [EMAIL PROTECTED] On Behalf Of Ed McNierney > > Sent: Wednesday, October 10, 2007 9:02 AM > > To: OSGeo Discussions > > Subject: RE: [OSGeo-Discuss] How to assess rendering quality? > > > > Gilles - > > > > Just keep in mind that subjective metrics are, after all, > subjective, > > and some of your metrics are mutually exclusive. > "Smoothness of lines" > > is normally accomplished by antialiasing those lines, > making the edges > > intermediate colors and a little "soft". This is a good > thing, but is > > incompatible with "sharpness of details", which is best accomplished > > without antialiasing but with more jagged artifacts on curves and > > diagonal lines. > > > > - Ed > > > > Ed McNierney > > Chief Mapmaker > > Demand Media / TopoZone.com > > 73 Princeton Street, Suite 305 > > North Chelmsford, MA 01863 > > Phone: 978-251-4242, Fax: 978-251-1396 > > [EMAIL PROTECTED] > > > > > > > > > > -----Original Message----- > > From: [EMAIL PROTECTED] [mailto:discuss- > > [EMAIL PROTECTED] On Behalf Of Gilles Bassière > > Sent: Wednesday, October 10, 2007 8:38 AM > > To: OSGeo Discussions > > Subject: Re: [OSGeo-Discuss] How to assess rendering quality? > > > > Hi Michael, > > > > I'm more concerned with subjective metrics, I actually plan > to survey > > some users. The questionnaire will include some map samples > and gather > > user preference for each criteria. My problem is to identify what > > questions/criteria should I ask to a user. > > > > Gilles > > > > > > Michael P. Gerlek wrote: > > > Gilles- > > > > > > Is your idea to measure the quality by having a human > look at outputs > > (subjective metrics) or automatically via some analysis routine > > (objective metrics)? > > > > > > -mpg > > > > > > > > > > > > > > >> -----Original Message----- > > >> From: [EMAIL PROTECTED] > > >> [mailto:[EMAIL PROTECTED] On Behalf Of Gilles > > Bassière > > >> Sent: Wednesday, October 10, 2007 3:01 AM > > >> To: discuss@lists.osgeo.org > > >> Subject: [OSGeo-Discuss] How to assess rendering quality? > > >> > > >> Hi list, > > >> > > >> I'm doing a comparative study of OpenSource cartographic servers > > >> (Mapserver, Geoserver and Mapnik). Beside raw performance and > > >> features, > > >> I'd like to assess the rendering quality, say how pretty > > >> produced maps > > >> are. Precisely, I'm interested in the quality of the > drawing work, > > my > > >> point is not about symbology, nor styling of maps. > > >> > > >> I have some problems to find a set of objective criteria I could > > >> benchmark my servers against. So far, I have already > identified the > > >> following: > > >> - sharpness of details > > >> - smoothness of lines > > >> - uniformity of colors > > >> > > >> I'm open to any comments. Do you think these criteria > are consistent > > >> regarding the purpose of my study? Does anyone have > other criteria > > to > > >> suggest? > > >> > > >> Regards > > >> > > >> -- > > >> Gilles Bassiere > > >> MAKINA CORPUS > > >> 30 rue des Jeuneurs > > >> FR-75011 PARIS > > >> http://www.makina-corpus.com > > >> > > >> _______________________________________________ > > >> Discuss mailing list > > >> Discuss@lists.osgeo.org > > >> http://lists.osgeo.org/mailman/listinfo/discuss > > >> > > >> > > > _______________________________________________ > > > Discuss mailing list > > > Discuss@lists.osgeo.org > > > http://lists.osgeo.org/mailman/listinfo/discuss > > > > > > > > > -- > > Gilles Bassiere > > MAKINA CORPUS > > 30 rue des Jeuneurs > > FR-75011 PARIS > > http://www.makina-corpus.com > > > > _______________________________________________ > > Discuss mailing list > > Discuss@lists.osgeo.org > > http://lists.osgeo.org/mailman/listinfo/discuss > > _______________________________________________ > > Discuss mailing list > > Discuss@lists.osgeo.org > > http://lists.osgeo.org/mailman/listinfo/discuss > > > _______________________________________________ > Discuss mailing list > Discuss@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/discuss > _______________________________________________ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
