As others have already pointed out, the primary benefactor of eye-tracking studies is the coordinator of the studies. After many such studies, I gradually have absorbed what textbooks could not teach me effectively about this science. What draws the human eye is not quite unpredictable, but the wide variance does not lend itself to easy rule-making.
Madison Avenue has been using this technology for years for somewhat nefarious purposes. We need to embrace it and put the knowledge to good use across the discipline of interaction and experience design. How could we tolerate the idea that it would be okay for marketing and advertising folks to know more about this area than ourselves? Here's the rub though: I have always found it challenging to use the eye-tracking data to correct my own designs. When I tried to do this initially, I would never get it right: I would re-submit "corrected" designs for new eye-tracking and I'd get even worse results. Persistence was key. Eye-tracking studies have made me a better designer. And my clients love it. Dave -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Larry Tesler Sent: Tuesday, April 22, 2008 11:22 PM To: Jared M. Spool Cc: [EMAIL PROTECTED] Subject: Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated? Jared, If most readers think this has gone on too long, we should wind it down. On Apr 22, 2008, at 5:59 AM, Jared M. Spool wrote: > > On Apr 22, 2008, at 3:08 AM, Larry Tesler wrote: > >> The fact that different observers see different things in the same >> raw eye tracking data is of no more concern to me than the fact >> that different players count a different number of words on the >> same Boggle board. Some people see words that are hidden in plain >> sight; some do not. But noticed or not, the words are there. In the >> tea leaves, there are no hidden words. > > Larry, > > I have no doubt that the observations are of interest. > > My point is that the inferences drawn from those observations have > little-to-no validity, thus the tea leaf analogy. I consider them valid if they inspire us to make design changes that lead to improvements in objective metrics. In any study, with or without an eye tracker, I look for: - a well-designed experiment - clean data - appropriate, error-free analysis - perceptive observation (which may require several observers to point things out to each other and reach consensus, as radiologists often do when faced with difficult images) - generation of hypotheses consistent with the study observations and any other available observations - prioritization of those hypotheses - generation of design solutions that respond to the most likely hypothesis - implementation - bucket testing - if no improvement is seen, iterate with alternate hypotheses > If someone fixates on a link for a unusually large time, does that > mean they are confused by it? Or they aren't confused, but are > trying to decide if its what they want? Or they know whether they > want it or not but are considering something else? If the user's mental state matters to you, ask the user what it was. They may know. If they do not know, devise a more clever experiment. But sometimes, the user's mental state doesn't matter. We may have run the test because too few people were clicking on the link. We thought perhaps they didn't even look at the area of the page that contained the link. The tracker has refuted our hypothesis. We know that some people look straight at the link and still do not click it. Other data may be needed if we want to find out why. But the study was a success. It achieved its goal. > Different inferences will lead to completely different design > solutions. Are you saying it doesn't matter which inference (and > therefore, which design solution) the observers choose? If the radiologists call your malignant tumor benign, or vice versa, you may receive the wrong treatment, which could be a costly mistake. But design changes that eye tracking studies inspire often entail simple modifications to layout, color, size, typeface, etc., that help to steer attention. They are often cheap to implement. If there are two competing inferences, you can often try both implied solutions. Of course, if you had both designs in mind (and particularly the one that ultimately proved to be best) before you ran the eye tracking study, then the study was a waste of time. If that is the situation you are in, and you never want to run a study that may simply confirm that you were right, then your point is valid. But it is not always the situation. There may be no unrefuted theory about why users are not clicking the link. There may be no considered designs that would increase clicks. There may be too many credible designs--more than one has the time and staff to implement and test. Or you may simply want to confirm other data or hunches. > When you back an eye-tracking supporter into a corner about this, > they all say, "Well, you should only use eye tracking in conjunction > with other data collection tools and techniques to verify your > inferences." In almost all cases, the "other data collection tools > and techniques" would yield just as much value without the eye > tracking as with it, so what's the benefit? The eye tracking test may have been the source of the first clue as to what ails your interface. Without it, you may never have thought to try those other tools and techniques. Or the eye tracking test may help to rule out hypotheses that one has generated from the use of other tools and techniques. > Second, in almost all uses of eye tracking I've seen in the last 5 > years, it's in the form of twisting the meaning of the heatmap/plot > diagram/tea leaf reading into supporting whatever wacky inference > the specialist wants to support. "See that big red spot there. That > means the users are confused" v. "See that big red spot there, that > means we fixed the design." I agree that features of the heat map don't tell you mental states. But if they are inconsistent with hypotheses about mental states, they may call those hypotheses into question. "See that big red spot over there. Maybe they were looking at the link after all." A big red spot may also provide evidence that the design is improved-- if the problem with the design was that nobody was looking at that link. But there are usually more convincing methods of validating designs than looking at heat maps, e.g., bucket tests with well-chosen metrics. > If there really is something to this eye tracking thing, I'd think > you'd want your team members to all look at the same heat map and > come to somewhat similar design implications. I thought my previous email, with its counterexamples, refuted the assumption that independent observers of an image must come to the same separate conclusions for the methodology to be useful. I guess we'll have to agree to disagree about that. > Eyetracking equipment: $30,000 Amortized over N studies = $30,000/N. If your business is large, your benefit may be large, too, and you could be getting a very good ROI. > Ouija Board: $5 > Quality design based on solid inferences from rich, meaningful data: > Priceless If your clients' markets are too small to make eye tracking investments pay off, and they have other sources of data that give them useful insights, then I can see why you don't advocate eye tracking. But for a client selling a high volume product, especially on the web where switching costs are low, small changes can have large effects, and eye tracking can pay off. Larry Tesler ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... [EMAIL PROTECTED] Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... [EMAIL PROTECTED] Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help
