Fig. 4c is SUCH a pile crap and you know it.
On Feb 19, 2012, at 5:48 PM, Sam <sammyc...@gmail.com> wrote: > > The article Vul et el did was on studies of functional MRI (fMRI) > scans which use voxels and apply some sort of gaussian smoothing to > determine which ones to use. Same exact method Firth et al used. > > > http://www.edvul.com/pdf/VulHarrisWinkielmanPashler-PPS-2009.pdf > > These basic steps common to most fMRI data analyses yield > matrices consisting of tens or hundreds of thousands of numbers > indicating activation levels in different voxels. These can be > (and indeed generally are) displayed as images. However, to > obtain quantitative summaries of these results and do further > statistics on them (such as correlating them with behavioral > measuresââ¬âthe topic of the present article), an investigator must > somehow select a subset of voxels and aggregate measurements > across them. This can be done in various ways. A subset of voxels > in the whole brain image may be selected based on purely anatomical > constraints (e.g., all voxels in a region generally agreed to > represent the amygdala, or all voxels within a certain radius of > some a priori speciï¬Âed brain coordinates). Alternatively, > regions can be selected based on ââ¬Ëââ¬Ëfunctional > constraints,ââ¬â¢Ã¢â¬â¢ > meaning that voxels are selected based on their activity > pattern in functional scans. For example, one could select > all the voxels for a particular subject that responded more to > reading than to nonlinguistic stimuli. Finally, voxels could be > chosen based on some combination of anatomy and functional > response. > > In the articles we are focusing on here, the ï¬Ânal result, as we > have seen, was always a correlation valueââ¬âa correlation between each > personââ¬â¢s score on some behavioral measure and some summary statistic > of their brain activation. The latter summary > statistic reï¬âects the activation or activation contrast within a > certain set of voxels. In either case, the critical question is, > ââ¬Ëââ¬ËHow was this set of voxels selected?ââ¬â¢Ã¢â¬â¢ As we have > seen, voxels > may be selected based on anatomical criteria, functional criteria, or > both. Within these broad options, there are a number of additional > more ï¬Âne-grained choices. It is hardly surprising, > then, that brief method sections rarely sufï¬Âce to describe how > the analyses were done in adequate detail to really understand > what choices were being made. > > We display the raw results from our survey as the proportion of > studies that investigators described with a particular answer to > each question (see Fig. 2). As some questions only applied to a > subset of participants, we display only the proportion of the > relevant subset of studies. > The raw answers to our survey do not by themselves explain > how respondents arrived at the (implausibly high, or so we have > argued) correlations. The key, we believe, lies in the 53% of > respondents who said that ââ¬Ëââ¬Ëregression across > subjectsââ¬â¢Ã¢â¬â¢ was the > functional constraint used to select voxels, indicating that > voxels were selected because they correlated highly with the > behavioral measure of interest. > > Figure 3 shows very concretely the sequence of steps that > these respondents reported following when analyzing their data. > A separate correlation across subjects was performed for each > voxel within a speciï¬Âed brain region. Each correlation relates > some measure of brain activity in that voxel (which might be a > difference between responses in two tasks or in two conditions) > with the behavioral measure for that individual. Thus, thenumber of > correlations computed was equal to the number > of voxels, meaning that thousands of correlations were > computed in many cases. At the next stage, researchers selected > the set of voxels for which this correlation exceeded a > certain threshold, and reported the correlation within this set of > voxels. > > What are the implications of selecting voxels in this fashion? > Such an analysis will inï¬âate observed across-subject correlations and > can even produce signiï¬Âcant measures out of pure noise. The problem is > illustrated in the simple simulation displayed in Figure 4. First, the > investigator computes a separate correlation of the behavioral measure > of interest with each of the > voxels (Fig. 4a). Then, he or she selects those voxels that exhibited > a sufï¬Âciently high correlation (by passing a statistical threshold; > Fig. 4b). Finally, an ostensible measure of the > ââ¬Ëââ¬Ëtrueââ¬â¢Ã¢â¬â¢ > correlation is aggregated from the voxels that showed high > correlations (e.g., by taking the mean of the voxels over the > threshold). With enough voxels, such a biased analysis is > guaranteed to produce high correlations even if none are truly > present (Fig. 4). Moreover, this analysis will produce visually > pleasing scattergrams (e.g., Fig. 4c) that will provide (quite > meaningless) reassurance to the viewer that s/he is looking at a > result that is solid, is ââ¬Ëââ¬Ënot driven by > outliers,ââ¬â¢Ã¢â¬â¢ and so on. > > > . > > On Sun, Feb 19, 2012 at 3:09 PM, Larry C. Lyons <larrycly...@gmail.com> wrote: >> >> I've been stying out of this discussion mainly because I've been reading >> the article that Sam provided a link to. à I'm still going through the >> article, but first and foremost it has nothing to do with the topic at >> hand. Vui et al were looking at the unexpectedly high correlations between >> personality traits and vartious systems within the brain. Last I checked >> the psychological constructs known as personality traits are not political >> orientation. Even someone who has taken an intro to psychology course at >> the high school level should have picked up that one. Did ytou actually >> read the article Sam? >> >> Second the main thrust of the criticism was how these personality traits >> were correlated with problematic measures of brain activation. The study I >> mentioned looked at the differences between specific groups. In à other >> words a very different form of statistical analysis was used. This >> meta-analysis you cited does not address that. >> >> There are other criticisms I could make about the methodological approach >> the authors take. However everyone else would find it completely boring. I >> also think that this study is important for entirely different reasons than >> Sam gave and those reasons are not at all relevant to this discussion, >> mainly involving recent research in the neuophysiology of personality. >> >> In a nutshell Sam you need to actually read and attempt to understand the >> material you present in cases like these. That way you do not come across >> with so much egg on your face. >> >> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-community/message.cfm/messageid:347124 Subscription: http://www.houseoffusion.com/groups/cf-community/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-community/unsubscribe.cfm
