On 31.5.2015 19:38, Tsung Fei Khang wrote:
Dear community,
I would like to share my experience with using some (really cool)
computational tools for phylogenetic signal and morphological
integration analysis.
I am using physignal (geomorph R package) and the Phylo.Morphol.PLS
function provided in the paper by Adams and Felice (2014; PLoS ONE,
9:e94335) in my work. I noticed that if the same analysis is rerun for
a particular number of iterations, the results may vary. Additionally,
I observed that increasing the number of iterations, up to some
critical point, may push down the p-value, depending on data set
(didn't happen with the plethspecies (9 species) data, but happened in
my data set - 13 species, not salamanders). I attach runs (10 times)
for both data sets for iterations of 100, 1000, 10000 and 100000 here
for Phylo.Morphol.PLS. Note that some kind of stable results is
attained after 1000 iterations (default) for the plethspecies data,
but for my case, which needs 10000.
I think the notion that p-values returned from a permutation method
are actually realizations of random variables with a certain mean and
variance may not be familiar to many biologists, who are accustomed to
expect a reproducible p-value when the same data set is rerun using
common statistical tests. Perhaps in a future version the authors of
the code can implement a checker within the functions that checks the
number of iterations for attaining "convergence", so that a more
stable p-value is returned?
Dear Tsung,
I understand that you stressed two questions:
1. how to make reproducible p-value
2. how to assess variability of p-value in non-exhaustive randomization
Answer for 1:
Technically speaking and if you're looking for a reproducible script,
you can obtain the same p value if you specify seed value via set.seed()
function prior statistical inference.
Add a bit for 2:
When all possible permutations for a given dataset exceed number of
permutatios used for statistical inference, it is likely that p value
will differ because p value is not point probability, but an interval
around the (real) point. An appropriate number of permutations as well
as its stability are important questions which are data specific.
However, I believe that experienced researchers here can elaborate on
that issue in more details.
Marko
--
Marko Djurakic,
Teaching assistant,
Faculty of Sciences
Department of Biology and Ecology
University of Novi Sad
Trg Dositeja Obradovića 2
21 000 Novi Sad
Serbia
PhD student at Faculty of Biology,
University of Belgrade
Studentski trg 16
11000 Belgrade
Serbia
e-mail: marko.djura...@dbe.uns.ac.rs
--
MORPHMET may be accessed via its webpage at http://www.morphometrics.org
To unsubscribe from this group and stop receiving emails from it, send an email
to morphmet+unsubscr...@morphometrics.org.
