Skip Navigation



Bioinformatics Advance Access published online on March 17, 2009
Bioinformatics, doi:10.1093/bioinformatics/btp150
This Article
Right arrow Advance Access manuscript (PDF) Freely available
Right arrow Supplementary Data
Right arrowOA All Versions of this Article:
25/10/1271    most recent
btp150v1
Right arrow Comments: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when Comments are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Google Scholar
Right arrow Articles by Handl, J.
Right arrow Articles by Lovell, S. C.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Handl, J.
Right arrow Articles by Lovell, S. C.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© 2009 The Author(s)
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/2.0/uk/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Artefacts and biases affecting the evaluation of scoring functions on decoy sets for protein structure prediction

Julia Handl 1, Joshua Knowles 2 and Simon C. Lovell 1,*

1 Faculty of Life Sciences, University of Manchester, UK
2 School of Computer Science, University of Manchester, UK

*To whom correspondence should be addressed. Simon C. Lovell, E-mail: simon.lovell{at}manchester.ac.uk


  Abstract

Motivation: Decoy data sets, consisting of a solved protein structure and numerous alternative native-like structures, are in common use for the evaluation of scoring functions in protein structure prediction. Several pitfalls with the use of these data sets have been identified in the literature, as well as useful guidelines for generating more effective decoy data sets. We contribute to this ongoing discussion an empirical assessment of several decoy data sets commonly used in experimental studies.

Results: : Results: We find that artefacts and sampling issues in the large majority of these data make it trivial to discriminate the native structure. This underlines that evaluation based on the rank / z-score of the native is a weak test of scoring function performance. Moreover, sampling biases present in the way decoy sets are generated or used can strongly affect other types of evaluation measures such as the correlation between score and RMSD to the native. We demonstrate how, depending on type of bias and evaluation context, sampling biases may lead to both over- or under-estimation of the quality of scoring terms, functions or methods.

Availability: Links to the software and data used in this study are available at http://dbkgroup.org/handl/decoy_sets.

Contact: simon.lovell{at}manchester.ac.uk

Associate Editor: Prof. Anna Tramontano

Received on October 29, 2008; revised on March 6, 2009; accepted on March 14, 2009

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?




Disclaimer: Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.