Re: [Freesurfer] How to do permutation for multiple comparison correction?

[Xiaojiang Yang]

External Email - Use Caution

Thank you! Now I see how permutation works. So given about 100-200 subjects
in the normal group, doing 1000 permutations is of no help.

In fact, in my cortical thickness abnormality test, no matter how many
times of permutations I take, the original non-permutated data is the only
one arrangement I can observe that has a "big" cluster area - all
permutated data has no cluster of big continuous area (given the
vertex-wise threshold). So, permutations cannot give me a reasonable
probabulity distribution of the sampling data, which means I cannot rely
on it to reduce False Positives. Very frustrated here. If you have more
comments here, please let me know. Thanks.

Best Regards,
Xiao


Douglas N. Greve

 Tue, 04 Aug 2020 08:27:47 -0700


There are actually only N-1 permutations that you can make:

Real matrix:
1 0
0 1
0 1
0 1
0 1


Permutation 1:

0 1
1 0
0 1
0 1
0 1


Permutation 2:

0 1
0 1
1 0
0 1
0 1




On 8/4/2020 10:41 AM, Xiaojiang Yang wrote:

External Email - Use Caution

Hi Doug,


I am still uncertain how to permutate the data for multiple comparisons.
Please help me with a little more details on this. For example, could you
give me 2-3 examples of the permutations for the matrix you gave to me?

In addition, if you can explain to me in a non-design-matrix way, that would
be great. For example, suppose I have 1 subject to be compared to n
subjects in a normal group. The number of comparisons is m. So I have m x
(1+n) data in total:

V_10   V_11 , V_12 , V_13 , … V_1n

V_20   V_21 , V_22 , V_23 , … V_2n

…… …… …… ……

V_m0  V_m1 , V_m2 , V_m3 , … V_mn

_


_Could you please explain to me how data should be permutated? Or give me
2-3 examples of the permutations of the above data that could be?

_Thank you a lot!

Xiao


Douglas N. Greve <
https://www.mail-archive.com/search?l=freesurfer@nmr.mgh.harvard.edu=from:%22Douglas+N.+Greve%22
>Thu, 23 Jul 2020 10:54:17 -0700 <
https://www.mail-archive.com/search?l=freesurfer@nmr.mgh.harvard.edu=date:20200723
>

You would have a design matrix with two columns and rows equal to n+1, eg
1  0
0 1
0 1
0 1
0 1
... n times
you would then permute the design matrix

On 7/23/2020 12:06 PM, Xiaojiang Yang wrote:

 External Email - Use Caution

Hi Doug,

For the first question, you answered "It is unusual, though it
should work". Could you please briefly describe the way FS used to
permute (based on my notation v0, v1, ... vn)? Or, the usual way
to permute?The way I described seems to be the only way I can
think of. Looking forward to your help here. Thanks a lot!

Xiao

Douglas N. Greve

Thu,
23 Jul 2020 07:37:13 -0700




On 7/21/2020 11:45 AM, Xiaojiang Yang wrote:

  External Email - Use Caution

 Hi Doug,
 For your questions:

 1. "Not sure what you mean by in the ROI. Are you trying to
 permute across space? That generally does not work because the
 points are not exchangeable across space."

 By "In the ROI", I mean for all vertices listed in the label file.
Here, I
am
 talking about a test subject and n control subjects, they are all
considered in the same
 reference subject space - fsaverage. A ROI is defined by a label file
for
the subject
 fsaverage. So, I am comparing the test subject and control group on the
same ROI region,
 vertex by vertex. I want to permute points in the test subject and
points
in all subjects
 in control groups.
 I am not talking about permuting vertex locations; I am talking about
permuting
 values (thickness in my case) from subjects on each vertex. For
example, for
 vertex i, I have one value (v0) from test subject, and n values (v1,
v2,...vn)
 from control subjects:
 test subject  control subjects
 v0v1, v2, .. vn
 One way of permutation would be:
 test subject  control subjects
 v1v0, v2, .. vn
 Is this a reasonable way to do permutation?

It is unusual, though it should work.

 2. "Not sure. You cannot discriminate between the groups when you
 are doing permutation"

 By doing the permutation many (say 1000) times, I want to get the
probability
 distribution of the sampling (observed or test) data inside the ROI
area, so
 that I can decide if I should 

Re: [Freesurfer] How to do permutation for multiple comparison correction?

[Douglas N. Greve]

There are actually only N-1 permutations that you can make:

Real matrix:
1 0
0 1
0 1
0 1
0 1


Permutation 1:

0 1
1 0
0 1
0 1
0 1


Permutation 2:

0 1
0 1
1 0
0 1
0 1




On 8/4/2020 10:41 AM, Xiaojiang Yang wrote:


External Email - Use Caution

Hi Doug,

I am still uncertain how to permutate the data for multiple 
comparisons. Please help me with a little more details on this. For 
example, could you give me 2-3 examples of the permutations for the 
matrix you gave to me?


In addition, if you can explain to me in a non-design-matrix way, that 
would be great. For example, suppose I have 1 subject to be compared 
to n subjects in a normal group. The number of comparisons is m. So I 
have m x (1+n) data in total:


V_10   V_11 , V_12 , V_13 , … V_1n

V_20   V_21 , V_22 , V_23 , … V_2n

…… …… …… ……

V_m0  V_m1 , V_m2 , V_m3 , … V_mn

_

_Could you please explain to me how data should be permutated? Or give 
me 2-3 examples of the permutations of the above data that could be?


_Thank you a lot!

Xiao

Douglas N. Greve 
Thu, 
23 Jul 2020 10:54:17 -0700 


You would have a design matrix with two columns and rows equal to n+1, eg
1  0
0 1
0 1
0 1
0 1
... n times
you would then permute the design matrix

On 7/23/2020 12:06 PM, Xiaojiang Yang wrote:

 External Email - Use Caution

Hi Doug,

For the first question, you answered "It is unusual, though it
should work". Could you please briefly describe the way FS used to
permute (based on my notation v0, v1, ... vn)? Or, the usual way
to permute?The way I described seems to be the only way I can
think of. Looking forward to your help here. Thanks a lot!

Xiao

Douglas N. Greve

Thu,
23 Jul 2020 07:37:13 -0700




On 7/21/2020 11:45 AM, Xiaojiang Yang wrote:

  External Email - Use Caution

 Hi Doug,
 For your questions:

 1. "Not sure what you mean by in the ROI. Are you trying to
 permute across space? That generally does not work because the
 points are not exchangeable across space."

 By "In the ROI", I mean for all vertices listed in the label file. 
Here, I
am
 talking about a test subject and n control subjects, they are all
considered in the same
 reference subject space - fsaverage. A ROI is defined by a label file 
for
the subject
 fsaverage. So, I am comparing the test subject and control group on the
same ROI region,
 vertex by vertex. I want to permute points in the test subject and 
points
in all subjects
 in control groups.
 I am not talking about permuting vertex locations; I am talking about
permuting
 values (thickness in my case) from subjects on each vertex. For 
example, for
 vertex i, I have one value (v0) from test subject, and n values (v1,
v2,...vn)
 from control subjects:
 test subject  control subjects
 v0v1, v2, .. vn
 One way of permutation would be:
 test subject  control subjects
 v1v0, v2, .. vn
 Is this a reasonable way to do permutation?

It is unusual, though it should work.

 2. "Not sure. You cannot discriminate between the groups when you
 are doing permutation"

 By doing the permutation many (say 1000) times, I want to get the
probability
 distribution of the sampling (observed or test) data inside the ROI 
area, so
 that I can decide if I should reject or accept null hypothesis based on
 cluster-wise significance level. What problems do you think I have in 
this
idea?

That should work. I'm not sure what my original concern was

 Thanks a lot!
 On 7/21/2020 1:12 AM, Xiaojiang Yang wrote:

   External Email - Use Caution

  Dear FS experts,

  Instead of using mri_glmfit-sim, I am trying to implement a
  customized multiple comparison correction algorithm using
  permutation. Before I implement my own, I want to make sure my
  permutation idea is correct. So I was looking at how
  mri_glmfit-sim does the permutation. The link here
  http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm has     

Re: [Freesurfer] How to do permutation for multiple comparison correction?

[Xiaojiang Yang]

External Email - Use Caution

Hi Doug,

I am still uncertain how to permutate the data for multiple comparisons.
Please help me with a little more details on this. For example, could you
give me 2-3 examples of the permutations for the matrix you gave to me?

In addition, if you can explain to me in a non-design-matrix way, that
would be great. For example, suppose I have 1 subject to be compared to n
subjects in a normal group. The number of comparisons is m. So I have m x
(1+n) data in total:

V10  V11, V12, V13, … V1n

V20  V21, V22, V23, … V2n

…… …… …… ……

Vm0 Vm1, Vm2, Vm3, … Vmn


Could you please explain to me how data should be permutated? Or give me
2-3 examples of the permutations of the above data that could be?

Thank you a lot!
Xiao

Douglas N. Greve

 Thu, 23 Jul 2020 10:54:17 -0700


You would have a design matrix with two columns and rows equal to n+1, eg
1  0
0 1
0 1
0 1
0 1
... n times

you would then permute the design matrix

On 7/23/2020 12:06 PM, Xiaojiang Yang wrote:

External Email - Use Caution

Hi Doug,


For the first question, you answered "It is unusual, though it should work".
Could you please briefly describe the way FS used to permute (based on my
notation v0, v1, ... vn)? Or, the usual way to permute?

The way I described seems to be the only way I can think of. Looking
forward to your help
here. Thanks a lot!

Xiao


Douglas N. Greve <
https://www.mail-archive.com/search?l=freesurfer@nmr.mgh.harvard.edu=from:%22Douglas+N.+Greve%22
>Thu, 23 Jul 2020 07:37:13 -0700 <
https://www.mail-archive.com/search?l=freesurfer@nmr.mgh.harvard.edu=date:20200723
>

On 7/21/2020 11:45 AM, Xiaojiang Yang wrote:

 External Email - Use Caution

Hi Doug,
For your questions:

1. "Not sure what you mean by in the ROI. Are you trying to
permute across space? That generally does not work because the
points are not exchangeable across space."

By "In the ROI", I mean for all vertices listed in the label file. Here, I
am
talking about a test subject and n control subjects, they are all
considered in the same
reference subject space - fsaverage. A ROI is defined by a label file for
the subject
fsaverage. So, I am comparing the test subject and control group on the
same ROI region,
vertex by vertex. I want to permute points in the test subject and points
in all subjects
in control groups.
I am not talking about permuting vertex locations; I am talking about
permuting
values (thickness in my case) from subjects on each vertex. For example, for
vertex i, I have one value (v0) from test subject, and n values (v1,
v2,...vn)
from control subjects:
test subject  control subjects
v0v1, v2, .. vn
One way of permutation would be:
test subject  control subjects
v1v0, v2, .. vn
Is this a reasonable way to do permutation?

It is unusual, though it should work.

2. "Not sure. You cannot discriminate between the groups when you
are doing permutation"

By doing the permutation many (say 1000) times, I want to get the
probability
distribution of the sampling (observed or test) data inside the ROI area, so
that I can decide if I should reject or accept null hypothesis based on
cluster-wise significance level. What problems do you think I have in this
idea?

That should work. I'm not sure what my original concern was

Thanks a lot!
On 7/21/2020 1:12 AM, Xiaojiang Yang wrote:

  External Email - Use Caution

 Dear FS experts,

 Instead of using mri_glmfit-sim, I am trying to implement a
 customized multiple comparison correction algorithm using
 permutation. Before I implement my own, I want to make sure my
 permutation idea is correct. So I was looking at how
 mri_glmfit-sim does the permutation. The link here
 http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm
has


<
http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm%C2%A0has
>

 a simple description for how to do permutation, but I don't quite
 understand the 1st step: Permute the design matrix. To me, permute
 the design matrix means permute the matrix rows here, but still
 hard to understand why permuting matrix rows does the trick.

This is pretty standard in permutation. I think Tom Nichols has
some basic tutorials on how permutations work.

 Anyway, I will not use any design matrix in my customized
 implementation, so it does not matter 

Re: [Freesurfer] How to do permutation for multiple comparison correction?

[Douglas N. Greve]

You would have a design matrix with two columns and rows equal to n+1, eg
1  0
0 1
0 1
0 1
0 1
... n times

you would then permute the design matrix

On 7/23/2020 12:06 PM, Xiaojiang Yang wrote:


External Email - Use Caution

Hi Doug,

For the first question, you answered "It is unusual, though it should 
work". Could you please briefly describe the way FS used to permute 
(based on my notation v0, v1, ... vn)? Or, the usual way to permute?


The way I described seems to be the only way I can think of. Looking 
forward to your help here. Thanks a lot!


Xiao

Douglas N. Greve 
Thu, 
23 Jul 2020 07:37:13 -0700 


On 7/21/2020 11:45 AM, Xiaojiang Yang wrote:

 External Email - Use Caution

Hi Doug,
For your questions:

1. "Not sure what you mean by in the ROI. Are you trying to
permute across space? That generally does not work because the
points are not exchangeable across space."

By "In the ROI", I mean for all vertices listed in the label file. Here, I 
am
talking about a test subject and n control subjects, they are all 
considered in the same
reference subject space - fsaverage. A ROI is defined by a label file for 
the subject
fsaverage. So, I am comparing the test subject and control group on the 
same ROI region,
vertex by vertex. I want to permute points in the test subject and points 
in all subjects
in control groups.
I am not talking about permuting vertex locations; I am talking about 
permuting
values (thickness in my case) from subjects on each vertex. For example, for
vertex i, I have one value (v0) from test subject, and n values (v1, 
v2,...vn)
from control subjects:
test subject  control subjects
v0v1, v2, .. vn
One way of permutation would be:
test subject  control subjects
v1v0, v2, .. vn
Is this a reasonable way to do permutation?

It is unusual, though it should work.

2. "Not sure. You cannot discriminate between the groups when you
are doing permutation"

By doing the permutation many (say 1000) times, I want to get the 
probability
distribution of the sampling (observed or test) data inside the ROI area, so
that I can decide if I should reject or accept null hypothesis based on
cluster-wise significance level. What problems do you think I have in this 
idea?

That should work. I'm not sure what my original concern was

Thanks a lot!
On 7/21/2020 1:12 AM, Xiaojiang Yang wrote:

  External Email - Use Caution

 Dear FS experts,

 Instead of using mri_glmfit-sim, I am trying to implement a
 customized multiple comparison correction algorithm using
 permutation. Before I implement my own, I want to make sure my
 permutation idea is correct. So I was looking at how
 mri_glmfit-sim does the permutation. The link here
 http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm has  

 


 a simple description for how to do permutation, but I don't quite
 understand the 1st step: Permute the design matrix. To me, permute
 the design matrix means permute the matrix rows here, but still
 hard to understand why permuting matrix rows does the trick.

This is pretty standard in permutation. I think Tom Nichols has
some basic tutorials on how permutations work.

 Anyway, I will not use any design matrix in my customized
 implementation, so it does not matter for now. My problem can be
 described as follows: if I have a ROI (a label file) on fsaverage,
 and I have a test subject and a group of control subjects whose
 thickness values on every vertex in this label are all known. (The
 test subject and control subjects are all using fsaverage
 reference space). I want to compare this test subject's thickness
 within the ROI to a control group of subjects (within the same
 ROI). This is a multiple-comparison problem, so I want to use
 permutation to get less FP rate. My question is: How do I permute
 the test subject's points and control subjects' points in ROI?

Not sure what you mean by in the ROI. Are you trying to permute
across space? That generally does not work because the points are
not exchangeable across space.

 My understanding is that: for each point in the label, I randomly
 re-assign all (1+n) values from (1+n) subjects to these (n+1)
 subjects (where n is the number of subjects in control group). And
 

Re: [Freesurfer] How to do permutation for multiple comparison correction?

[Xiaojiang Yang]

External Email - Use Caution

Hi Doug,

For the first question, you answered "It is unusual, though it should
work". Could you please briefly describe the way FS used to permute (based
on my notation v0, v1, ... vn)? Or, the usual way to permute?

The way I described seems to be the only way I can think of. Looking
forward to your help here. Thanks a lot!

Xiao

Douglas N. Greve

 Thu, 23 Jul 2020 07:37:13 -0700


On 7/21/2020 11:45 AM, Xiaojiang Yang wrote:

External Email - Use Caution

Hi Doug,
For your questions:

1. "Not sure what you mean by in the ROI. Are you trying to permute
across space?
That generally does not work because the points are not exchangeable across
space."

By "In the ROI", I mean for all vertices listed in the label file. Here, I am
talking about a test subject and n control subjects, they are all
considered in the same
reference subject space - fsaverage. A ROI is defined by a label file
for the subject
fsaverage. So, I am comparing the test subject and control group on
the same ROI region,
vertex by vertex. I want to permute points in the test subject and
points in all subjects
in control groups.
I am not talking about permuting vertex locations; I am talking about permuting
values (thickness in my case) from subjects on each vertex. For example, for
vertex i, I have one value (v0) from test subject, and n values (v1, v2,...vn)
from control subjects:
test subject  control subjects
v0v1, v2, .. vn
One way of permutation would be:
test subject  control subjects
v1v0, v2, .. vn
Is this a reasonable way to do permutation?

It is unusual, though it should work.

2. "Not sure. You cannot discriminate between the groups when you are doing
permutation"

By doing the permutation many (say 1000) times, I want to get the probability
distribution of the sampling (observed or test) data inside the ROI area, so
that I can decide if I should reject or accept null hypothesis based on
cluster-wise significance level. What problems do you think I have in this idea?

That should work. I'm not sure what my original concern was

Thanks a lot!
On 7/21/2020 1:12 AM, Xiaojiang Yang wrote:

 External Email - Use Caution

Dear FS experts,

Instead of using mri_glmfit-sim, I am trying to implement a
customized multiple comparison correction algorithm using
permutation. Before I implement my own, I want to make sure my
permutation idea is correct. So I was looking at how
mri_glmfit-sim does the permutation. The link here
http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm has


a simple description for how to do permutation, but I don't quite
understand the 1st step: Permute the design matrix. To me, permute
the design matrix means permute the matrix rows here, but still
hard to understand why permuting matrix rows does the trick.


This is pretty standard in permutation. I think Tom Nichols has some basic
tutorials on how permutations work.

Anyway, I will not use any design matrix in my customized
implementation, so it does not matter for now. My problem can be
described as follows: if I have a ROI (a label file) on fsaverage,
and I have a test subject and a group of control subjects whose
thickness values on every vertex in this label are all known. (The
test subject and control subjects are all using fsaverage
reference space). I want to compare this test subject's thickness
within the ROI to a control group of subjects (within the same
ROI). This is a multiple-comparison problem, so I want to use
permutation to get less FP rate. My question is: How do I permute
the test subject's points and control subjects' points in ROI?


Not sure what you mean by in the ROI. Are you trying to permute across space?
That generally does not work because the points are not exchangeable across
space.

My understanding is that: for each point in the label, I randomly
re-assign all (1+n) values from (1+n) subjects to these (n+1)
subjects (where n is the number of subjects in control group). And
when all points in the label are done, this is only 1 permutation.
I will need at least 1000 times of permutation to get the
comparison statistics.

Is my understanding right?


Not sure. You cannot discriminate between the groups when you are doing
permutation

Thank you!
Xiao


___
Freesurfer mailing
listfreesur...@nmr.mgh.harvard.eduhttps://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu

Re: [Freesurfer] How to do permutation for multiple comparison correction?

[Douglas N. Greve]

On 7/21/2020 11:45 AM, Xiaojiang Yang wrote:


External Email - Use Caution

Hi Doug,
For your questions:
1. "Not sure what you mean by in the ROI. Are you trying to permute across 
space? That generally does not work because the points are not 
exchangeable across space."

By "In the ROI", I mean for all vertices listed in the label file. Here, I am 
talking about a test subject and n control subjects, they are all considered in the same 
reference subject space - fsaverage. A ROI is defined by a label file for the subject 
fsaverage. So, I am comparing the test subject and control group on the same ROI region, 
vertex by vertex. I want to permute points in the test subject and points in all subjects 
in control groups.
I am not talking about permuting vertex locations; I am talking about permuting 
values (thickness in my case) from subjects on each vertex. For example, for 
vertex i, I have one value (v0) from test subject, and n values (v1, v2,...vn) 
from control subjects:
test subject  control subjects
v0v1, v2, .. vn
One way of permutation would be:
test subject  control subjects
v1v0, v2, .. vn
Is this a reasonable way to do permutation?

It is unusual, though it should work.
2. "Not sure. You cannot discriminate between the groups when you are 
doing permutation"

By doing the permutation many (say 1000) times, I want to get the probability 
distribution of the sampling (observed or test) data inside the ROI area, so 
that I can decide if I should reject or accept null hypothesis based on 
cluster-wise significance level. What problems do you think I have in this idea?

That should work. I'm not sure what my original concern was

Thanks a lot!
On 7/21/2020 1:12 AM, Xiaojiang Yang wrote:

 External Email - Use Caution

Dear FS experts,

Instead of using mri_glmfit-sim, I am trying to implement a
customized multiple comparison correction algorithm using
permutation. Before I implement my own, I want to make sure my
permutation idea is correct. So I was looking at how
mri_glmfit-sim does the permutation. The link here
http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm has


a simple description for how to do permutation, but I don't quite
understand the 1st step: Permute the design matrix. To me, permute
the design matrix means permute the matrix rows here, but still
hard to understand why permuting matrix rows does the trick.

This is pretty standard in permutation. I think Tom Nichols has some 
basic tutorials on how permutations work.


Anyway, I will not use any design matrix in my customized
implementation, so it does not matter for now. My problem can be
described as follows: if I have a ROI (a label file) on fsaverage,
and I have a test subject and a group of control subjects whose
thickness values on every vertex in this label are all known. (The
test subject and control subjects are all using fsaverage
reference space). I want to compare this test subject's thickness
within the ROI to a control group of subjects (within the same
ROI). This is a multiple-comparison problem, so I want to use
permutation to get less FP rate. My question is: How do I permute
the test subject's points and control subjects' points in ROI?

Not sure what you mean by in the ROI. Are you trying to permute across 
space? That generally does not work because the points are not 
exchangeable across space.


My understanding is that: for each point in the label, I randomly
re-assign all (1+n) values from (1+n) subjects to these (n+1)
subjects (where n is the number of subjects in control group). And
when all points in the label are done, this is only 1 permutation.
I will need at least 1000 times of permutation to get the
comparison statistics.

Is my understanding right?

Not sure. You cannot discriminate between the groups when you are 
doing permutation


Thank you!
Xiao


___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer


___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

Re: [Freesurfer] How to do permutation for multiple comparison correction?

[Xiaojiang Yang]

External Email - Use Caution

Hi Doug,


For your questions:


1. "Not sure what you mean by in the ROI. Are you trying to permute
across space? That generally does not work because the points are not
exchangeable across space."


By "In the ROI", I mean for all vertices listed in the label file.
Here, I am talking about a test subject and n control subjects, they
are all considered in the same reference subject space - fsaverage. A
ROI is defined by a label file for the subject fsaverage. So, I am
comparing the test subject and control group on the same ROI region,
vertex by vertex. I want to permute points in the test subject and
points in all subjects in control groups.


I am not talking about permuting vertex locations; I am talking about
permuting values (thickness in my case) from subjects on each vertex.
For example, for vertex i, I have one value (v0) from test subject,
and n values (v1, v2,...vn) from control subjects:


test subject  control subjects

v0v1, v2, .. vn


One way of permutation would be:

test subject  control subjects

v1v0, v2, .. vn


Is this a reasonable way to do permutation?

2. "Not sure. You cannot discriminate between the groups when you are
doing permutation"


By doing the permutation many (say 1000) times, I want to get the
probability distribution of the sampling (observed or test) data
inside the ROI area, so that I can decide if I should reject or accept
null hypothesis based on cluster-wise significance level. What
problems do you think I have in this idea?


Thanks a lot!


On 7/21/2020 1:12 AM, Xiaojiang Yang wrote:

External Email - Use Caution

Dear FS experts,


Instead of using mri_glmfit-sim, I am trying to implement a customized multiple
comparison correction algorithm using permutation. Before I implement my
own, I want to make sure my permutation idea is correct. So I was looking
at how mri_glmfit-sim does the permutation. The link here
http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm has a
simple description for how to do permutation, but I don't quite understand
the 1st step: Permute the design matrix. To me, permute the design matrix
means permute the matrix rows here, but still hard to understand why
permuting matrix rows does the trick.

This is pretty standard in permutation. I think Tom Nichols has some basic
tutorials on how permutations work.

Anyway, I will not use any design matrix in my customized implementation,
so it does not matter for now. My problem can be described as follows: if I
have a ROI (a label file) on fsaverage, and I have a test subject and a
group of control subjects whose thickness values on every vertex in this
label are all known. (The test subject and control subjects are all using
fsaverage reference space). I want to compare this test subject's
thickness within
the ROI to a control group of subjects (within the same ROI). This is a
multiple-comparison problem, so I want to use permutation to get less FP
rate. My question is: How do I permute the test subject's points and
control subjects' points in ROI?

Not sure what you mean by in the ROI. Are you trying to permute across space?
That generally does not work because the points are not exchangeable across
space.

My understanding is that: for each point in the label, I randomly re-assign
all (1+n) values from (1+n) subjects to these (n+1) subjects (where n is
the number of subjects in control group). And when all points in the label
are done, this is only 1 permutation. I will need at least 1000 times of
permutation to get the comparison statistics.

Is my understanding right?

Not sure. You cannot discriminate between the groups when you are doing
permutation

Thank you!
Xiao
___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

Re: [Freesurfer] How to do permutation for multiple comparison correction?

[Douglas N. Greve]

On 7/21/2020 1:12 AM, Xiaojiang Yang wrote:


External Email - Use Caution

Dear FS experts,

Instead of using mri_glmfit-sim, I am trying to implement a customized 
multiple comparison correction algorithm using permutation. Before I 
implement my own, I want to make sure my permutation idea is correct. 
So I was looking at how mri_glmfit-sim does the permutation.
The link here 
http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm has 
a simple description for how to do permutation, but I don't quite 
understand the 1st step: Permute the design matrix. To me, permute the 
design matrix means permute the matrix rows here, but still hard to 
understand why permuting matrix rows does the trick.
This is pretty standard in permutation. I think Tom Nichols has some 
basic tutorials on how permutations work.
Anyway, I will not use any design matrix in my customized 
implementation, so it does not matter for now.
My problem can be described as follows: if I have a ROI (a label file) 
on fsaverage, and I have a test subject and a group of control 
subjects whose thickness values on every vertex in this label are all 
known. (The test subject and control subjects are all using fsaverage 
reference space). I want to compare this test subject's thickness 
within the ROI to a control group of subjects (within the same ROI). 
This is a multiple-comparison problem, so I want to use permutation to 
get less FP rate.
My question is: How do I permute the test subject's points and control 
subjects' points in ROI?
Not sure what you mean by in the ROI. Are you trying to permute across 
space? That generally does not work because the points are not 
exchangeable across space.
My understanding is that: for each point in the label, I randomly 
re-assign all (1+n) values from (1+n) subjects to these (n+1) subjects 
(where n is the number of subjects in control group). And when all 
points in the label are done, this is only 1 permutation. I will need 
at least 1000 times of permutation to get the comparison statistics.

Is my understanding right?
Not sure. You cannot discriminate between the groups when you are doing 
permutation

Thank you!
Xiao


___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer


___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

[Freesurfer] How to do permutation for multiple comparison correction?

[Xiaojiang Yang]

External Email - Use Caution

Dear FS experts,

Instead of using mri_glmfit-sim, I am trying to implement a customized
multiple comparison correction algorithm using permutation. Before I
implement my own, I want to make sure my permutation idea is correct. So I
was looking at how mri_glmfit-sim does the permutation.
The link here
http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm has a
simple description for how to do permutation, but I don't quite understand
the 1st step: Permute the design matrix. To me, permute the design matrix
means permute the matrix rows here, but still hard to understand why
permuting matrix rows does the trick.
Anyway, I will not use any design matrix in my customized implementation,
so it does not matter for now.
My problem can be described as follows: if I have a ROI (a label file) on
fsaverage, and I have a test subject and a group of control subjects whose
thickness values on every vertex in this label are all known. (The test
subject and control subjects are all using fsaverage reference space). I
want to compare this test subject's thickness within the ROI to a control
group of subjects (within the same ROI). This is a multiple-comparison
problem, so I want to use permutation to get less FP rate.
My question is: How do I permute the test subject's points and control
subjects' points in ROI?
My understanding is that: for each point in the label, I randomly re-assign
all (1+n) values from (1+n) subjects to these (n+1) subjects (where n is
the number of subjects in control group). And when all points in the label
are done, this is only 1 permutation. I will need at least 1000 times of
permutation to get the comparison statistics.
Is my understanding right?
Thank you!
Xiao
___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer