On Wed, Feb 25, 2015 at 10:31 AM, Devon McCormick <[email protected]> wrote:
> Ian - setting up a framework for this sort of experiment could prove > valuable in a wide variety of fields, not just audio testing. > > However, you may be overshadowing the point of the example from the book: > it's an illustration of the relevance of prior knowledge. As I remember > it, not having the book in front of me, it goes something like this: > > There are these three examples of evidence supporting a hypothesis: > > 1) A lady claims to be able to distinguish, by tasting a cup of tea with > milk, whether the tea was added before the milk or the milk before the > tea. You test her ten times and she is correct every time. > > 2) Someone claims to be able to distinguish by ear a score written by > Mozart from one not written by Mozart. You test him ten times and he is > correct every time. > > 3) A drunken friend claims to be able to predict the result of a coin > toss. You test him ten times and he is correct every time. > > Since the empirical evidence in all three cases is identical, why would we > not believe all three hypotheses to be equally well-proved? > > Having thought about this and with the appropriate disclaimer: I am not a statistician, I would suggest that the third example is different than the first two examples. I would consider the first two a form of sensory discrimination and the third predicting an event with 50% probability (assuming a fair coin). The first two hypotheses can be tested using different experiment designs[1][2][3], which may assist on reducing or explaining the effect of guessing. For example, you could have a triangle test, which changes the likelihood of guessing to 1/3 in the first example The second example would be harder to design correctly without any bias. Would you play mozart 50% of the time and a random similar sounding classical piece? Clearly you couldn't play 50% mozart and 50% rock music. I didn't quite understand what you meant by "since the empirical evidence in all three cases is identical" since that's a bit overloaded. Is the empirical evidence zero because it hasn't been tested or are you getting at a prior probability along the lines of bayes? I think we an intuition on prior probability for a fair coin, but maybe not everyone does. Getting back to being able to distinguish audio files, I think there may need to be more consideration on the experiment design. [1 ]http://en.wikipedia.org/wiki/Discrimination_testing [2] http://sensory.byu.edu/Clients/DifferenceTesting.aspx [3] http://www.sensorysociety.org/knowledge/sspwiki/Pages/Triangle%20Test.aspx ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
