*Title:* Towards replicable and generalizable genomic prediction models Speaker: Levi Waldron, CUNY Friday, October 12, 4-5pm Building 40, Room 1201/1203
*Abstract:* Learning algorithms and the prediction models they generate are typically evaluated on the basis of cross-validation error estimates in a few exemplary datasets. However, the ultimate goal of most prediction modeling is to provide accurate predictions for independent samples obtained in different settings. In this talk I will discuss this disconnect and its practical implications for methods development and model selection. I discuss cross-study validation as an alternative in settings where multiple experimental studies employing comparable assays have been performed, and its application to assessing learning algorithms, heterogeneity between studies, and the impacts of heterogeneity on prediction accuracy. I will also introduce a series of curated databases and related R/Bioconductor software for cancer genomic and microbiome data that facilitate this area of research. *About the speaker:* Levi Waldron is an Associate Professor of biostatistics at the CUNY Graduate School of Public Health and Health Policy in New York City and a member of the technical advisory board of the Bioconductor project for open-source Bioinformatics. His lab develops methods, software, and databases that allow others to more efficiently leverage publicly available data. *References:* Zhang Y, Bernau C, Parmigiani G, Waldron L: *The impact of different sources of heterogeneity on loss of accuracy from genomic prediction models* . *Biostatistics* 2018. Trippa L, Waldron L, Huttenhower C, Parmigiani G: *Bayesian nonparametric cross-study validation of prediction methods*. *Ann. Appl. Stat.* 2015, *9* :402–428. Zhao SD, Parmigiani G, Huttenhower C, Waldron L: *Más-o-menos: a simple sign averaging method for discrimination in genomic data analysis*. *Bioinformatics* 2014, *30*:3062–3069. Bernau C, Riester M, Boulesteix A-L, Parmigiani G, Huttenhower C, Waldron L, Trippa L: *Cross-study validation for the assessment of prediction algorithms*. *Bioinformatics* 2014, *30*:i105–12. Pasolli E, Schiffer L, Manghi P, Renson A, Obenchain V, Truong DT, Beghini F, Malik F, Ramos M, Dowd JB, Huttenhower C, Morgan M, Segata N, Waldron L: *Accessible, curated metagenomic data through ExperimentHub*. *Nat. Methods* 2017, *14* :1023–1024. Ramos M, Schiffer L, Re A, Azhar R, Basunia A, Rodriguez C, Chan T, Chapman P, Davis SR, Gomez-Cabrero D, Culhane AC, Haibe-Kains B, Hansen KD, Kodali H, Louis MS, Mer AS, Riester M, Morgan M, Carey V, Waldron L: *Software for the Integration of Multiomics Experiments in Bioconductor*. *Cancer Res.* 2017, *77*:e39–e42. Ganzfried BF, Riester M, Haibe-Kains B, Risch T, Tyekucheva S, Jazic I, Wang XV, Ahmadifar M, Birrer MJ, Parmigiani G, Huttenhower C, Waldron L: *curatedOvarianData: clinically annotated data for the ovarian cancer transcriptome*. *Database * 2013, *2013*:bat013.
