On Mon, Apr 09, 2012 at 04:03:52AM -0400, Kragen Javier Sitaker wrote: > There was actually a 1997 SPIE paper, [Image Quality Assessment with a Gabor > Pyramid Model of the Human Visual System][8], by Taylor, Pizlo, Allebach, and > Bouman, which proposed doing exactly this in order to measure the fidelity of > image approximation methods, including, among other things, halftoning. This > hasn't been completely ignored (Google Scholar finds 34 citations) but you > still find even the best current halftoning work using totally boneheaded > blue-noise criteria instead. Maybe it's because Taylor was a grad student at > Purdue, where he'd actually just done a bunch of work on halftoning and then > went on to do his Ph.D. thesis on this model, and then went off to teach at a > vocational school in Milwaukee and stopped doing research. > > [8]: > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.2727&rep=rep1&type=pdf > > Taylor et al.'s algorithm doesn't let you invert the Gabor transform, but it > does produce for each pixel a probability that a human will notice a > difference > between the images at that pixel, which seems like it would be just as good > for > the case of halftoning, where it tells you which pixels you most need to flip. > From there, it's gradient descent and boom, Bob's your uncle.
I've been thinking about this a lot more. The 1998 dissertation expanding this work is called "IMAGE QUALITY ASSESSMENT BASED ON A HUMAN VISUAL SYSTEM MODEL"; he calls his algorithm the "Image Fidelity Assessor", and as he explains in the dissertation: > The IFA accepts two images and a few viewing parameters as input and produces > a probability map as output. The probability map indicates the probability of > seeing a difference between the two input images (White = 100% probability, > Black = 0% probability). That is, it tells you how perceptually different two images are, pixel by pixel. He validated his model with a bunch of psychovisual experiments on humans and supposedly found that it predicted well. What could you do given an efficient algorithm to produce such a map? In my previous post I suggested that you could do much better halftoning, but here's a list of additional possible applications, some of which were proposed by Taylor: - JPEG or JPEG2000 compression: improve the visual quality per bit by using bits on the parts of the image, and the frequencies, that really need it - generating line-art representations of images (e.g. pencil portraits). This requires that Taylor's algorithm continues to provide useful values for images that are very different indeed, which I haven't validated but which I will assume for the rest of this list. This includes both robot drawings with physical pencils and purely in-computer drawings with virtual pencils. - ASCII art - choice of palette values for dithering: far beyond Heckbert median-cut - pointillist versions of images - image approximation with, say, overlapping circles of solid colors - designing a mosaic to approximate an image, given a known set of available tiles, e.g. a pile of broken ceramic of different colors. This can be generalized to making images from found objects in general: fallen leaves, river rocks, etc. - mural planning for robotic airbrushing or for humans following a computer-generated plan - designing stencils or silkscreens to approximate a desired image as closely as possible, subject to some constraint (e.g. limited total edge length, small number of colors) - constructing images containing features that only amblyopic people can see, because they can't see the distractors that would be present for normal vision - guiding texture synthesis from a limited palette of textures to match, as closely as possible, hand-drawn line art or ASCII art - designing shiny three-dimensional shapes whose caustics approximate a desired image - optimizing the scratch holograms Bill Beaty rediscovered to produce the best image quality for a limited number of scratches - optimizing the "opacity 'holograms'" that I wrote about in 2000: <http://lists.canonical.org/pipermail/kragen-hacks/2000-August/000260.html>. Here the thing to optimize is the quality of the worst image in a set, for a given overall density. By making the sheets more and more opaque, with a smaller fraction transparent, you can make this optimization problem easier, converging in the limit to a vertical-slit version of lenticular 3D images. But if you can intelligently trade off pixel errors between the images, you should be able to do a dramatically better job. (This is also a video codec. Unfortunately Taylor's model does not attempt to model the perceptual fidelity of lossy moving images.) - robotic sculpting of sand to approximate scenes using the shadows of the sand. This, too, can be a multiple-image optimization problem, if you can change the direction of the light. - selection of pigments for e.g. silkscreening or sand painting. As you can tell, I've been wishing this algorithm existed for a very long time, perhaps nearly as long as it's existed. I don't understand why it hasn't had more impact. Google Scholar claims that the dissertation is cited by only two publications (!!), one of which is by Taylor's own professors and coauthors on the shorter SPIE paper version from 1997, which is only cited in 34. The second most-cited of those citing documents is [Modern Image Quality Assessment][0], a 2006 book which surveys the field of computational quality models like Taylor's; it can be downloaded from Morgan Claypool as a PDF for US$30. There appears to be a lot of work going on in the field. [0]: http://www.morganclaypool.com/doi/abs/10.2200/S00010ED1V01Y200508IVM003?prevSearch=allfield%3A(modern+image+quality) Oh brave new world, that has such publications in it! -- To unsubscribe: http://lists.canonical.org/mailman/listinfo/kragen-tol
