Skip Navigation



Bioinformatics Advance Access published online on October 28, 2004
Bioinformatics, doi:10.1093/bioinformatics/bti081
Bioinformatics © Oxford University Press 2004; all rights reserved
This Article
Right arrow Advance Access manuscript (PDF) Freely available
Right arrow All Versions of this Article:
21/7/1046    most recent
bti081v1
Right arrow Comments: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when Comments are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Sima, C.
Right arrow Articles by Dougherty, E. R.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Sima, C.
Right arrow Articles by Dougherty, E. R.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

Received July 22, 2004
Revised September 25, 2004
Accepted September 30, 2004

Article

Superior feature-set ranking for small samples using bolstered error estimation

Chao Sima 1, Ulisses Braga-Neto 2, and Edward R. Dougherty 3*

1 Department of Electrical Engineering, Texas A&M University, College Station, TX
2 Department of Electrical Engineering, Texas A&M University, College Station, TX; Section of Clinical Cancer Genetics, University of Texas M. D. Anderson Cancer Center, Houston, TX
3 Department of Electrical Engineering, Texas A&M University, College Station, TX; Department of Pathology, University of Texas M. D. Anderson Cancer Center, Houston, TX

* To whom correspondence should be addressed.
Edward R. Dougherty, E-mail: edward{at}ee.tamu.edu


  Abstract

Motivation: Ranking feature sets is a key issue for classification, for instance, phenotype classification based on gene expression. Since ranking is often based on error estimation, and error estimators suffer to differing degrees of imprecision in small-sample settings, it is important to choose a computationally feasible error estimator that yields good feature-set ranking.

Results: This paper examines the feature-ranking performance of several kinds of error estimators: resubstitution, cross-validation, bootstrap, and bolstered error estimation. It does so for three classification rules: linear discriminant analysis, 3-nearest-neighbor classification, and classification trees. Two measures of performance are considered. One counts the number of the truly best feature sets appearing among the best feature sets discovered by the error estimator and the other computes the mean absolute error between the top ranks of the truly best feature sets and their ranks as given by the error estimator. Our results indicate that bolstering is superior to bootstrap, and bootstrap is better than cross-validation, for discovering top-performing feature sets for classification when using small samples. A key issue is that bolstered error estimation is tens of times faster than bootstrap, and faster than cross-validation, and is therefore feasible for feature-set ranking when the number of feature sets is extremely large.

Availability: We provide a companion web site, which contains the complete set of tables and plots regarding the simulation study, and a compilation of references on feature-set ranking with applications in Genomics. The companion web site can be accessed at the URL http://ee.tamu.edu/~edward/bolster_ranking.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 BioinformaticsHome page
Y. Saeys, I. Inza, and P. Larranaga
A review of feature selection techniques in bioinformatics
Bioinformatics, October 1, 2007; 23(19): 2507 - 2517.
[Abstract] [Full Text] [PDF]

Home page
 J. Mol. Diagn.Home page
M. D. Onken, L. A. Worley, R. M. Davila, D. H. Char, and J. W. Harbour
Prognostic Testing in Uveal Melanoma by Transcriptomic Profiling of Fine Needle Biopsy Specimens
J. Mol. Diagn., November 1, 2006; 8(5): 567 - 573.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
X. Zhou and K. Z. Mao
The ties problem resulting from counting-based error estimators and its impact on gene selection algorithms
Bioinformatics, October 15, 2006; 22(20): 2507 - 2515.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
A. Choudhary, M. Brun, J. Hua, J. Lowey, E. Suh, and E. R. Dougherty
Genetic test bed for feature selection
Bioinformatics, April 1, 2006; 22(7): 837 - 842.
[Abstract] [Full Text] [PDF]

Home page
 Brief BioinformHome page
P. Larranaga, B. Calvo, R. Santana, C. Bielza, J. Galdiano, I. Inza, J. A. Lozano, R. Armananzas, G. Santafe, A. Perez, et al.
Machine learning in bioinformatics
Brief Bioinform, March 1, 2006; 7(1): 86 - 112.
[Abstract] [Full Text] [PDF]


Disclaimer: Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.