Dear Tsung,
I guess that what you observed is relatively normal in common practice
and many people are quite used to it when using randomization methods
(i.e. most people don't expect to find always exactly the same p value
when using random permutations). If my memory doesn't fail me,
recommendations of checking at higher number of randomization runs
already exist in the literature.
Another thing to consider is that by using random permutations, one
tries to obtain an approssimation of the results obtained using all
possible permutations (which on real datasets is often an
impractically large number). So part of the "problem" (if ever there
is one) lie on the fact of using an approximation in the first place...
Just my two cents...
Best,
Carmelo
Tsung Fei Khang <tfkh...@um.edu.my> ha scritto:
Dear community,
I would like to share my experience with using some (really cool)
computational tools for phylogenetic signal and morphological integration
analysis.
I am using physignal (geomorph R package) and the Phylo.Morphol.PLS
function provided in the paper by Adams and Felice (2014; PLoS ONE,
9:e94335) in my work. I noticed that if the same analysis is rerun for a
particular number of iterations, the results may vary. Additionally, I
observed that increasing the number of iterations, up to some critical
point, may push down the p-value, depending on data set (didn't happen with
the plethspecies (9 species) data, but happened in my data set - 13
species, not salamanders). I attach runs (10 times) for both data sets for
iterations of 100, 1000, 10000 and 100000 here for Phylo.Morphol.PLS. Note
that some kind of stable results is attained after 1000 iterations
(default) for the plethspecies data, but for my case, which needs 10000.
I think the notion that p-values returned from a permutation method are
actually realizations of random variables with a certain mean and variance
may not be familiar to many biologists, who are accustomed to expect a
reproducible p-value when the same data set is rerun using common
statistical tests. Perhaps in a future version the authors of the code can
implement a checker within the functions that checks the number of
iterations for attaining "convergence", so that a more stable p-value is
returned?
--
" PENAFIAN: E-mel ini dan apa-apa fail yang dikepilkan bersamanya ("Mesej")
adalah ditujukan hanya untuk kegunaan penerima(-penerima) yang termaklum di
atas dan mungkin mengandungi maklumat sulit. Anda dengan ini dimaklumkan
bahawa mengambil apa jua tindakan bersandarkan kepada, membuat penilaian,
mengulang hantar, menghebah, mengedar, mencetak, atau menyalin Mesej ini
atau sebahagian daripadanya oleh sesiapa selain daripada
penerima(-penerima) yang termaklum di atas adalah dilarang. Jika anda telah
menerima Mesej ini kerana kesilapan, anda mesti menghapuskan Mesej ini
dengan segera dan memaklumkan kepada penghantar Mesej ini menerusi balasan
e-mel. Pendapat-pendapat, rumusan-rumusan, dan sebarang maklumat lain di
dalam Mesej ini yang tidak berkait dengan urusan rasmi Universiti Malaya
adalah difahami sebagai bukan dikeluar atau diperakui oleh mana-mana pihak
yang disebut.
DISCLAIMER: This e-mail and any files transmitted with it ("Message") is
intended only for the use of the recipient(s) named above and may contain
confidential information. You are hereby notified that the taking of any
action in reliance upon, or any review, retransmission, dissemination,
distribution, printing or copying of this Message or any part thereof by
anyone other than the intended recipient(s) is strictly prohibited. If you
have received this Message in error, you should delete this Message
immediately and advise the sender by return e-mail. Opinions, conclusions
and other information in this Message that do not relate to the official
business of University of Malaya shall be understood as neither given nor
endorsed by any of the forementioned. "
--
MORPHMET may be accessed via its webpage at http://www.morphometrics.org
To unsubscribe from this group and stop receiving emails from it,
send an email to morphmet+unsubscr...@morphometrics.org.
--
Carmelo Fruciano
Marie Curie Fellow - University of Konstanz - Konstanz, Germany
Honorary Fellow - University of Catania - Catania, Italy
e-mail c.fruci...@unict.it
http://www.fruciano.it/research/
--
MORPHMET may be accessed via its webpage at http://www.morphometrics.org
To unsubscribe from this group and stop receiving emails from it, send an email
to morphmet+unsubscr...@morphometrics.org.
