Skip Navigation

This Article
Right arrow FREE Full Text (Print PDF) Freely available
Right arrow FREE Full Text (Screen PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (14)
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Toivonen, H.
Right arrow Articles by Helma, C.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Toivonen, H.
Right arrow Articles by Helma, C.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

Bioinformatics Vol. 19 no. 10 2003
Pages 1183-1193
© 2003 Oxford University Press

Statistical evaluation of the Predictive Toxicology Challenge 2000–2001

Hannu Toivonen 1,*, Ashwin Srinivasan 2, Ross D. King 3, Stefan Kramer 4,5 and Christoph Helma 4

1 Department of Computer Science, PO Box 26 (Teollisuuskatu 23), FIN-00014 University of Helsinki, Finland
2 Oxford University Computing Laboratory, Wolfson Building, Parks Road, Oxford OX1 3QD, UK
3 Department of Computer Science, Penglais, Aberystwyth, Ceredigion SY23 3DB, Wales, UK
4 Institut Für Informatik, Albert-Ludwigs-Universität Freiburg, Georges-Köhler-Allee, Gebäude 079, D-79110 Freiburg i. Br. Germany
5 Institute for Computer Science, Technical University of Munich, Boltzmannstr. 3, D-85748 Garching b. Muenchen, Germany

Received on July 30, 2002 ; revised on November 7, 2002 ; accepted on November 11, 2002

Motivation: The development of in silico models to predict chemical carcinogenesis from molecular structure would help greatly to prevent environmentally caused cancers. The Predictive Toxicology Challenge (PTC) competition was organized to test the state-of-the-art in applying machine learning to form such predictive models.

Results: Fourteen machine learning groups generated 111 models. The use of Receiver Operating Characteristic (ROC) space allowed the models to be uniformly compared regardless of the error cost function. We developed a statistical method to test if a model performs significantly better than random in ROC space. Using this test as criteria five models performed better than random guessing at a significance level p of 0.05 (not corrected for multiple testing). Statistically the best predictor was the Viniti model for female mice, with p value below 0.002. The toxicologically most interesting models were Leuven2 for male mice, and Kwansei for female rats. These models performed well in the statistical analysis and they are in the middle of ROC space, i.e. distant from extreme cost assumptions. These predictive models were also independently judged by domain experts to be among the three most interesting, and are believed to include a small but significant amount of empirically learned toxicological knowledge.

Availability: PTC details and data can be found at: http://www.predictive-toxicology.org/ptc/

Contact: hannu.toivonen{at}cs.helsinki.fi

* To whom correspondence should be addressed.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?




Disclaimer:
Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.