darrenyeats wrote: > Dave, > The "facts" in this case are the results of rounds picking A or B as the > identity of X. > > A statistical analysis is needed on these results to generate a p-value > - the outcome of the experiment is a probability. >
Agreed\ > > The problem with this statistical angle is that a large number of > samples is needed, and so listening tests involving one or a few people > run into trouble with listening fatigue. > *False. * The above statement actually says nothing because of the reliance of vague terminology, such as "large number". Here, let me show you how to actually say something. For common statistical methods to work with reasonable accuracy, at least 5 and hopefully more like 10 or 15 trials (the technical word for the word rounds which sounds like a kind of song or adjustments to a mechanism). If one is actually hearing a difference then listening fatigue is usually not much of a problem. However, if you are trying to prove the existence of something like high sample rates or cable magic or audible differences between reasonably good amplifiers or DACs that other science pretty clearly indicates is impossible for humans to hear, then you can indeed get exhausted doing enough trials (50, 100, 200...) to show even vanishing statistical significance. If you approach a reliable listening test with a belief system that is based on placebo effects, the same problem(s) can exist. The role of listener training cannot be overemphasized. Early listener training often serves to provide people with evidence that they have indeed based their beliefs on something other than reliable facts, because the apparent reliability of their perceptions disappears when sight and other irrelevant cues are removed and only listening to music that varies only in terms of the effect or equipment beind studied is the only influence on the test. > > Note this is not physical fatigue. For example: we never "hear" the same > thing twice, because hearing changes the hearer, and hearing is a > combination of stimulus and experience; this has a long-term and > short-term aspect (e.g. you just get "fed up" of hearing the same thing > many times and A and B start to mush). There may be other mechanisms at > play, but I've done enough blind tests to come to believe fatigue is a > factor beyond 3-4 rounds of listening to the same sample for the same > particular difference, at least without a significant break. Usually > this sort of problem is not mentioned. > *False.* The most common kind of audio listening test for subtle differences is the ABX test described by Clark and others in several AES publications dating back to 1978-1983. One of the properties of ABX testing that critics such as the above *like to ignore or obfuscate* is the fact that while choices are forced to be of one kind or the other (usually labelled A or B), the circumstances in terms of time and effort that is used to obtain those results is up to the listener. The listener and/or test coordinator (ABX testing is designed so that they can be the same person) can use as many test sessions as he wishes, engage in whatever kind of listener training as is desired, take as many breaks as he wishes, listen as long as he wishes, adjust levels at will as long as the adjustment is uniformly applied to the audible samples, use whatever music he chooses, etc., etc. > > Another problem with listening tests in general is the number of > confounding factors. For example, driving hours to a bake-off affects > your hearing. You are more familiar with your own system at home, this > process of familiarisation can take a long time - I can pick out an > unusual squeak in my car, but a passenger can't, yet we're hearing the > same sounds. These confounding factors mean NOT hearing a difference > EVEN SIGHTED is not necessarily real evidence of non-audibility. (aside) OK folks,* this is what anti-science propaganda looks like*. Remember its contents and its source. It is pretty common because of the real and present economic danger that ABX testing presents to those who would use false evidence to sell audio gear. ------------------------------------------------------------------------ arnyk's Profile: http://forums.slimdevices.com/member.php?userid=64365 View this thread: http://forums.slimdevices.com/showthread.php?t=106914 _______________________________________________ audiophiles mailing list audiophiles@lists.slimdevices.com http://lists.slimdevices.com/mailman/listinfo/audiophiles
