James, I love where you're going here, and recourse to rigor (and Popper) seems like a neat antidote to a lot of research methods (I'm thinking more about the field of marketing, than anything) that get more than a bit sloppy in their applied versions.
So I'm just nailing down some stuff at the end, cuz I wasn't sure exactly where you were going, so forgive me if I've misunderstood what you wrote. Might be good to tease it out a bit more, cuz that's where it gets interesting. Too often, reference to the "sciences" and properly rigorous research methods equates (in some people's minds) to overly foundationalist assumptions that require generalizable, quantitative data only. I don't believe you are going there, from what you reference at the very end, but that makes me want to push on where you are going. Descriptive, rich, qualitative methods are by definition NOT generalizable. That would be the whole point. One can inductively triangulate data, amass evidence that reinforces emerging categories of data, develop heuristics, and even conduct parallel studies and discover points of intersection between similar qualitative or ethnographic-type studies. So replicate to some extent, but generalize, never. True, people doing multi-modal studies are trying to work with qualitative and quantitative methods in tandem, so you may get some cross pollination there. Content analysis, linguistics, these are rich areas for combining methods, again, to triangulate, or to use emerging qualitative data to develop quantitative hypotheses. Quantitative heads tend toward more restrictive, or limited definitions of what is "real" research, which methods are most rigorous, yield the best data, and so on. They like to tout generalizability as some kind of Holy Grail that only they can claim, like it gives them some kind of foundationalist claim to capital T Truth. Blah. Not all of them think like this, however. You tend to get that kind of POV more often in journalists who write stories about "science," and bias their coverage toward methods they more readily grasp or can easily convert into sound bites. (you'd be amazed at how widespread this POV is among journalists at CNN, for instance--I can speak from experience) To this end, then, journalists unconsciously tend to reinforce misconceptions about real research, real science, which mushes up the whole pseudo-science problem more, as they often tout as authoritative quantitative studies that are so severely limited and short-sighted in their hypothesis development or baseline assumptions as to render their so-called valid and generalizable data utterly worthless. Which is worse: to have a method that is rigorously generalizable, but doesn't actually fit what we find in the world, or a rich data set which hews closely to the actual behaviors of actual people with deep insight and understanding, but should not ever be generalized beyond that level of detail? Which is closer to real small-t truths? Chris On Wed, Nov 19, 2008 at 9:49 AM, James Page <[EMAIL PROTECTED]> wrote: > Hi All, > I think it may help people here if I inject some theory into > this discussion. > > The first point is that people keep making claims that the method has some > scientific validity. For example Liz says that "Hopefully my quick > elucidation about the original persona creation methodology helps you to > see > that the mapping of individuals to dimensions of interest is a relatively > scientific method" > > Either a method is scientific or pseduscientific. There is no middle > ground. > Is the distinction only important as academic argument? The answer is no, > and it helps to understand a small bit of history to see why. > > The distinction between science or psedu science came about because there > where two political movements that claimed that they where scientific, and > by following them would lead to improvement to everybody in society. The > two > political movements where communism, and national socialism. > > Karl Poppers, who you could say his early life was upturned by both > movements, thought that it is important to qualify what is scientific and > what is pseduscientific. [DISCLAIMER] Some members of my family where > slightly put out by these movements as well] He came up with the idea that > unless a theory has a negative hypothesis and is replicable, then it is not > scientific. In one sweep he had disqualified both Marxism, > and communism from claim of being scientific. > > One of sciences that wiped out by this definition was Eugenics. Eugenics > was > one of the academic justifications of Nazism. Another science to disappear > was > biotype. This is the > idea that you could predict if somebody was a criminal by their body > measurement. > > Back to Persona's and Liz's presentation. She gives an example about Tom. > By > my count there are at least 21 bits of data points about Tom. Using the > example given by Chapman and Milham (which again uses at least 21 data > points. > > http://cnchapman.files.wordpress.com/2007/03/chapman-milham-personas-hfes2006-0139-0330.pdf > That would show that "Under those assumptions, the composite data for > "Patrick" would represent (0.5)21 * 100% = 0.000048% of the population, or > approximately 134 people in the > United States." [If your target population (lets say Car Mechanics) is > smaller than that then the number of people your Persona > could represent approaches zero] > > They go on further to say that :- > The key point is "there is essentially no way to generalize from > a well-specified persona to a population of interest, and thus no way to > say > anything about the users of interest. There is no way to distinguish > which characteristics of a given persona are indicative of users and which > are irrelevant > > The point is that unless you can show that you are designing for Users and > not something fictional then it is hard to call it User Centred Design > > Is there a way out of the theory Trap. I think yes there is. Idea one is to > treat a design as a Hypothesis and test it. Idea Two is to go back to the > Sciences that have contributed methods to UCD, like Anthropology, and see > how they overcome some of the Theory Challenges. For example many people on > the list complain about the time that it takes to go through the research, > and to distil the ideas. Ethnography was developed as a descriptive > language. > Or go to Activity Theory which is another descriptive process. If you use > either the language of Ethnography or the methods of AT it will save you > time. Forget about trying to get data to jump out at you. This is called > Grounded Theory and it is time consuming and very hard to follow correctly. > Also come up with some ideas before the research and then test them (in a > negative wayi.e.... my theory is not true if.....), again this will both > save you time and can be quite reliable. > > All the best > > James > > On Tue, Nov 18, 2008 at 6:05 AM, Jarod Tang <[EMAIL PROTECTED]> wrote: > > > Hi James, > > > > > We are told so many times not to use us for our designs, ourselves, or > > our > > > mothers as the target for a design. But surely this is better than > > something > > > that is purely fictional. > > Persona is/should be based on user research data underneath (at least > > for design). This is defined from early practitioners like Alan Cooper > > (and he proved why the instantiation of persona should based on > > concrete user research in his books). To say it's fictional, one may > > miss the point of persona usage for design . > > > > Regards, > > Jarod > > > > -- > > http://designforuse.blogspot.com/ > > > ________________________________________________________________ > Welcome to the Interaction Design Association (IxDA)! > To post to this list ....... [EMAIL PROTECTED] > Unsubscribe ................ http://www.ixda.org/unsubscribe > List Guidelines ............ http://www.ixda.org/guidelines > List Help .................. http://www.ixda.org/help > ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... [EMAIL PROTECTED] Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help
