Bioinformatics Vol. 19 no. 12 2003
Pages 1484-1491
© 2003 Oxford University Press
Class prediction and discovery using gene microarray and proteomics mass spectroscopy data: curses, caveats, cautions
Institute for Biodiagnostics, National Research Council Canada, Winnipeg, MB, Canada R3B 1Y6
Received on October 18, 2002
; revised on November 20, 2002
; accepted on February 16, 2003
Motivation: Two practical realities constrain the analysis of microarray data, mass spectra from proteomics, and biomedical infrared or magnetic resonance spectra. One is the ‘curse of dimensionality’: the number of features characterizing these data is in the thousands or tens of thousands. The other is the ‘curse of dataset sparsity’: the number of samples is limited. The consequences of these two curses are far-reaching when such data are used to classify the presence or absence of disease.
Results: Using very simple classifiers, we show for several publicly available microarray and proteomics datasets how these curses influence classification outcomes. In particular, even if the sample per feature ratio is increased to the recommended 5–10 by feature extraction/reduction methods, dataset sparsity can render any classification result statistically suspect. In addition, several ‘optimal’ feature sets are typically identifiable for sparse datasets, all producing perfect classification results, both for the training and independent validation sets. This non-uniqueness leads to interpretational difficulties and casts doubt on the biological relevance of any of these ‘optimal’ feature sets. We suggest an approach to assess the relative quality of apparently equally good classifiers.
Contact: Ray.Somorjai{at}nrc-cnrc.gc.ca
* To whom correspondence should be addressed.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  A. Alibes, A. Canada, and R. Diaz-Uriarte PaLS: filtering common literature, biological terms and pathway information Nucleic Acids Res., July 1, 2008; 36(suppl_2): W364 - W367. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Hilario and A. Kalousis Approaches to dimensionality reduction in proteomic biomarker studies Brief Bioinform, March 1, 2008; 9(2): 102 - 118. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Y. Saeys, I. Inza, and P. Larranaga A review of feature selection techniques in bioinformatics Bioinformatics, October 1, 2007; 23(19): 2507 - 2517. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Diaz-Uriarte, A. Alibes, E. R. Morrissey, A. Canada, O. M. Rueda, and M. L. Neves Asterias: integrated analysis of expression and aCGH data using an open-source, web-based, parallelized software suite Nucleic Acids Res., July 13, 2007; 35(suppl_2): W75 - W80. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
I. Fishel, A. Kaufman, and E. Ruppin Meta-analysis of gene expression data: a predictor-based approach Bioinformatics, July 1, 2007; 23(13): 1599 - 1606. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J.-x. Wang, P. Qin, Q.-l. Liu, H.-y. Yang, Y.-z. Fan, J.-k. Yu, and S. Zheng Detection and Significance of Serum Protein Marker of Hirschsprung Disease Pediatrics, July 1, 2007; 120(1): e56 - e60. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Berrar, I. Bradbury, and W. Dubitzky Avoiding model selection bias in small-sample genomic datasets Bioinformatics, May 15, 2006; 22(10): 1245 - 1250. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. J. Siddall, P. Stanwell, A. Woodhouse, R. L. Somorjai, B. Dolenko, A. Nikulin, R. Bourne, U. Himmelreich, C. Lean, M. J. Cousins, et al. Magnetic resonance spectroscopy detects biochemical changes in the brain associated with chronic low back pain: a preliminary report. Anesth. Analg., April 1, 2006; 102(4): 1164 - 1168. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y. Tan, L. Shi, S. M. Hussain, J. Xu, W. Tong, J. M. Frazier, and C. Wang Integrating time-course microarray gene expression profiles with cytotoxicity for identification of biomarkers in primary rat hepatocytes exposed to cadmium Bioinformatics, January 1, 2006; 22(1): 77 - 87. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. R. Fielden, B. P. Eynon, G. Natsoulis, K. Jarnagin, D. Banas, and K. L. Kolaja A Gene Expression Signature that Predicts the Future Onset of Drug-Induced Renal Tubular Toxicity Toxicol Pathol, October 1, 2005; 33(6): 675 - 683. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y. L. Yap, M. P. Wong, X. W. Zhang, D. Hernandez, R. Gras, D. K. Smith, and A. Danchin Conserved transcription factor binding sites of cancer markers derived from primary lung adenocarcinoma microarrays Nucleic Acids Res., January 14, 2005; 33(1): 409 - 421. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. Morrison, W. Farrar, J. Kneile, N. Williams, Y. Liu-Stratton, A. Bakaletz, M. A. Aldred, and C. Eng Molecular Classification of Parathyroid Neoplasia by Gene Expression Profiling Am. J. Pathol., August 1, 2004; 165(2): 565 - 576. [Abstract] [Full Text] [PDF]
|
|