Skip Navigation


Bioinformatics Advance Access originally published online on September 27, 2005
Bioinformatics 2005 21(23):4280-4288; doi:10.1093/bioinformatics/bti685
This Article
Right arrow Full Text Freely available
Right arrow FREE Full Text (Print PDF) Freely available
Right arrow All Versions of this Article:
21/23/4280    most recent
bti685v1
Right arrow Comments: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when Comments are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (28)
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Xie, Y.
Right arrow Articles by Khodursky, A. B.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Xie, Y.
Right arrow Articles by Khodursky, A. B.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2005. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions{at}oxfordjournals.org

A note on using permutation-based false discovery rate estimates to compare different analysis methods for microarray data

Yang Xie 1,*, Wei Pan 1 and Arkady B. Khodursky 2

1Division of Biostatistics, School of Public Health, University of Minnesota Minneapolis, MN 55455, USA
2Department of Biochemistry, Molecular Biology and Biophysics, University of Minnesota St Paul, MN 55108, USA

*To whom correspondence should be addressed.

Motivation: False discovery rate (FDR) is defined as the expected percentage of false positives among all the claimed positives. In practice, with the true FDR unknown, an estimated FDR can serve as a criterion to evaluate the performance of various statistical methods under the condition that the estimated FDR approximates the true FDR well, or at least, it does not improperly favor or disfavor any particular method. Permutation methods have become popular to estimate FDR in genomic studies. The purpose of this paper is 2-fold. First, we investigate theoretically and empirically whether the standard permutation-based FDR estimator is biased, and if so, whether the bias inappropriately favors or disfavors any method. Second, we propose a simple modification of the standard permutation to yield a better FDR estimator, which can in turn serve as a more fair criterion to evaluate various statistical methods.

Results: Both simulated and real data examples are used for illustration and comparison. Three commonly used test statistics, the sample mean, SAM statistic and Student's t-statistic, are considered. The results show that the standard permutation method overestimates FDR. The overestimation is the most severe for the sample mean statistic while the least for the t-statistic with the SAM-statistic lying between the two extremes, suggesting that one has to be cautious when using the standard permutation-based FDR estimates to evaluate various statistical methods. In addition, our proposed FDR estimation method is simple and outperforms the standard method.

Contact: yangxie{at}biostat.umn.ed

Received on June 30, 2005; revised on September 2, 2005; accepted on September 20, 2005

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 BioinformaticsHome page
K. F. Kerr
Comments on the analysis of unbalanced microarray data
Bioinformatics, August 15, 2009; 25(16): 2035 - 2041.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
M. Zhang, L. Zhang, J. Zou, C. Yao, H. Xiao, Q. Liu, J. Wang, D. Wang, C. Wang, and Z. Guo
Evaluating reproducibility of differential expression discoveries in microarray studies by considering correlated molecular changes
Bioinformatics, July 1, 2009; 25(13): 1662 - 1668.
[Abstract] [Full Text] [PDF]

Home page
 Brief BioinformHome page
R. M. Flight and P. D. Wentzell
Potential Bias in GO::TermFinder
Brief Bioinform, May 1, 2009; 10(3): 289 - 294.
[Abstract] [Full Text] [PDF]

Home page
 Proc. Natl. Acad. Sci. USAHome page
Y. Merbl and M. W. Kirschner
Large-scale detection of ubiquitination substrates using cell extracts and protein microarrays
PNAS, February 24, 2009; 106(8): 2543 - 2548.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
Y. Xie
Comments on 'On correcting the overestimation of the permutation-based false discovery rate estimator'
Bioinformatics, October 15, 2008; 24(20): 2420 - 2420.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
M. Zhang, C. Yao, Z. Guo, J. Zou, L. Zhang, H. Xiao, D. Wang, D. Yang, X. Gong, J. Zhu, et al.
Apparently low reproducibility of true differential expression discoveries in microarray studies
Bioinformatics, September 15, 2008; 24(18): 2057 - 2063.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
S. Jiao and S. Zhang
On correcting the overestimation of the permutation-based false discovery rate estimator
Bioinformatics, August 1, 2008; 24(15): 1655 - 1661.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
H. Qin, T. Feng, S. A. Harding, C.-J. Tsai, and S. Zhang
An efficient method to identify differentially expressed genes in microarray experiments
Bioinformatics, July 15, 2008; 24(14): 1583 - 1589.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
J. Xu and X. Cui
Robustified MANOVA with applications in detecting differentially expressed genes from oligonucleotide arrays
Bioinformatics, April 15, 2008; 24(8): 1056 - 1062.
[Abstract] [Full Text] [PDF]

Home page
 Brief Funct Genomic ProteomicHome page
F. Cordero, M. Botta, and R. A. Calogero
Microarray data analysis and mining approaches
Brief Funct Genomic Proteomic, January 22, 2008; (2008) elm034v1.
[Abstract] [Full Text] [PDF]

Home page
 Physiol. GenomicsHome page
R. W. Li, M. J. Meyer, Curtis. P. Van Tassell, T. S. Sonstegard, E. E. Connor, M. E. Van Amburgh, Y. R. Boisclair, and A. V. Capuco
Identification of estrogen-responsive genes in the parenchyma and fat pad of the bovine mammary gland by microarray analysis
Physiol Genomics, January 12, 2007; 27(1): 42 - 53.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
H. Yang and G. Churchill
Estimating p-values in small microarray experiments
Bioinformatics, January 1, 2007; 23(1): 38 - 43.
[Abstract] [Full Text] [PDF]

Home page
 Physiol. GenomicsHome page
T. S. Mehta, S. O. Zakharkin, G. L. Gadbury, and D. B. Allison
Epistemological issues in omics and high-dimensional biology: give the people what they want
Physiol Genomics, December 13, 2006; 28(1): 24 - 32.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
N. Rajicic, D. M. Finkelstein, D. A. Schoenfeld, and the Inflammation Host Response to Injury Research
Survival analysis of longitudinal microarrays
Bioinformatics, November 1, 2006; 22(21): 2643 - 2649.
[Abstract] [Full Text] [PDF]

Home page
 GeneticsHome page
X. Cui, J. Affourtit, K. R. Shockley, Y. Woo, and G. A. Churchill
Inheritance Patterns of Transcript Levels in F1 Hybrid Mice
Genetics, October 1, 2006; 174(2): 627 - 637.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
G.J. McLachlan, R.W. Bean, and L. B.-T. Jones
A simple implementation of a normal mixture approach to differential gene expression in multiclass microarrays
Bioinformatics, July 1, 2006; 22(13): 1608 - 1615.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
X. Gao
Construction of null statistics in permutation-based multiple testing for multi-factorial microarray experiments
Bioinformatics, June 15, 2006; 22(12): 1486 - 1494.
[Abstract] [Full Text] [PDF]


Disclaimer: Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.