Bioinformatics Advance Access originally published online on June 14, 2005
Bioinformatics 2005 21(17):3509-3515; doi:10.1093/bioinformatics/bti540
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Improving sequence-based fold recognition by using 3D model quality assessment
Bioinformatics Unit, Department of Computer Science, University College London Gower Street, WC1E 6BT, UK
*To whom correspondence should be addressed.
 |
Abstract |
|---|
 TOP Abstract INTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Motivation: The ability of a simple method (MODCHECK) to determine the sequence–structure compatibility of a set of structural models generated by fold recognition is tested in a thorough benchmark analysis. Four Model Quality Assessment Programs (MQAPs) were tested on 188 targets from the latest LiveBench-9 automated structure evaluation experiment. We systematically test and evaluate whether the MQAP methods can successfully detect native-likemodels.
Results: We show that compared with the other three methods tested MODCHECK is the most reliable method for consistently performing the best top model selection and for ranking the models. In addition, we show that the choice of model similarity score used to assess a model's similarity to the experimental structure can influence the overall performance of these tools. Although these MQAP methods fail to improve the model selection performance for methods that already incorporate protein three dimension (3D) structural information, an improvement is observed for methods that are purely sequence-based, including the best profile–profile methods. This suggests that even the best sequence-based fold recognition methods can still be improved by taking into account the 3D structural information.
Contact: d.jones{at}cs.ucl.ac.uk
|
INTRODUCTION |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Protein structure prediction methods have greatly improved in their ability to provide large sets of structural models for target protein sequences and as a result have become useful tools for in silico structural genomics. The CASP experiments (Moult et al., 2003) highlight the growing number of structure prediction servers in the bioinformatics community and a recent research shows a steady increase in the quality of the predictions made by these tools (Venclovas et al., 2003). Where these methods may benefit from most of the improvement, and which has proven to be a formidable task, these are in consistent selection of the most accurate structures from the generated set of structural models. In terms of fold recognition this corresponds to accurate selection of models that obtain the correct fold, whereas for comparative modeling and more generally, this problem can be viewed as the selection of the best or most native-like structures from a set of good models. Determining and accurately assessing the properties that govern the nativeness of a protein are of paramount importance to the production of an accurate tool for model selection, and a huge body of research is available. Traditionally, solutions to this problem have taken many forms such as the development of empirical or knowledge-based potentials (Sippl, 1995) that use databases of experimental structures to extract features of these ‘ correct’ models, and these are tested on decoy sets of protein structures to assess their efficacy. Other approaches to model selection range from clustering methods applied to sets of structural conformations (Zhang and Skolnick, 2004) to evaluation of structural energies using molecular mechanics force fields (Lee et al., 2001). The increase in the number of experimentally determined structures in the Protein Data Bank (PDB) (Sussman et al., 1998) may lead to improvements in the model selection methods that derive information about native structural features by sampling the PDB.
The recent CAFASP4 experiment has seen the addition of a new category, Model Quality Assessment Programs (MQAPs), that aims to tackle the problem of model selection. Here we describe MODCHECK, a method for assessing model quality, and perform a systematic benchmark analysis of four model quality methods using 188 targets from the latest LiveBench-9 experiment (Rychlewski et al., 2003).
|
MATERIALS AND METHODS |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
MODCHECK
In order to evaluate whether any extra value can be added from viewing the 3D structure of a particular fold recognition model, we implemented a very simple model quality assessment program (MODCHECK) that is based on classical threading potentials. The evaluation function used in MODCHECK is principally based on a set of pairwise potentials of mean force (Hendlich et al., 1990), determined by a statistical analysis of highly resolved protein X-ray crystal structures and the application of the inverse Boltzmann equation as described for the original THREADER program (Jones et al., 1992). In addition to the pairwise potentials, a solvation potential is also used (Jones et al., 1992).
For specified atoms (Cß 
Cß for example) in a pair of residues ab with sequence separation k and distance interval s, the potential is given by the following expression:
 | (1) |
is the weight given to each observation, fk(s) is the frequency of occurrence of all residue pairs at topological level k and separation distance s, 
is the equivalent frequency of occurrence of residue pair ab and RT is taken to be 0.582 kcal/mol. In this work, short (sequence separation, k 
11), medium (11 k 22) and long (k > 22) range potentials have been calculated between Cß atoms only.
In addition to the pairwise potentials, a solvation potential for an amino acid residue a is defined as follows:
 | (2) |
One of the problems in classical threading is finding a suitable way to combine pairwise and solvation energy terms, and also to correct for the effects of protein size. In our original threading work, this correction was achieved by calculating Z-scores using the energy values for each hit in the fold library. Here we use a similar approach, but in this case the Z-scores are obtained by carrying out extensive sequence shuffling trials. By comparing the native threading energies with those from sequence shuffled decoys, biases from both the folds, amino acid composition and sequence length can be reduced. For a given model, the amino acid sequence is shuffled up to 100 000 times and the pairwise and solvation energy terms are evaluated for each sequence shuffled model. To save computer time, the Z-scores of the model are assessed after 1000 shuffles and the calculation is terminated if both the pairwise and solvation Z-scores are <6. The final MODCHECK quality score is derived by simply summing the pairwise and solvation Z-scores.
Benchmarking model quality assessment methods
Data
All model data for this study were acquired from the fully automated LiveBench-9 experiment for 3D structure prediction (Bujnicki et al., 2001). Eight structure prediction servers were chosen and the models generated by these servers were used in the analysis. For each server, between 5 and 10 models were generated for each of the 188 selected targets providing 13 449 models, and from this set the late submissions to LiveBench-9, where the MaxSub score (Siew et al., 2000) was >0.95 (95%), were removed giving a total of 11 880 models. The targets consist of protein sequences between 100 and 500 residues in length, and are either single domain structures or single selected chains from a multimeric protein.
The 188 targets were additionally separated into two subsets and classified as ‘easy’ and ‘hard’ targets by the LiveBench standard. LiveBench defines easy targets as those that have a structurally similar fold to the target sequence and where the E-value of the first hit in a PDB-BLAST (Li et al., 2002) result set is <0.001. Hard targets are those sequences for which standard methods for homology detection fail to find any similar folds. A total of 77 easy targets and 111 hard targets were obtained using this categorization.
Model rebuilding
The models from the LiveBench server contain solely the C
atoms and in order to calculate scores for the model quality assessment methods, the backbone atoms were generated. CTrip (Petrey et al., 2003) was used from the Jackal protein structure modeling package (http://honiglab.cpmc.columbia.edu/programs/jackal/index.html) in order to generate the backbone atoms for each of the models.
Automated structure prediction servers
Eight structure prediction servers were selected from the LiveBench-9 by general categorization (sequence, threading, or consensus). FFAS, FFAS03 (Jaroszewski et al., 2000), 3D-PSSM (Kelley et al., 2000), ORFeus (Ginalski et al., 2003), GenTHREADER (Jones, 1999), mGenTHREADER (McGuffin and Jones, 2003), Pcons-4 (Lundstrom et al., 2001) and SAM-T02 (Karplus et al., 2003) were selected, and the models generated by each server for each target were obtained. The selection includes a combination of methods classified under LiveBench as sequence comparison servers, fold recognition and consensus methods. The 5–10 models generated for each target by these methods are submitted to LiveBench-9 in a ranked order determined by some selection scheme specific to each prediction server.
Model Quality Assessment Programs (MQAP)
Four model quality assessment programs that were available at the time of writing, MODCHECK, ProQ (Wallner and Elofsson, 2003), Solvex (Holm and Sander, 1992) and FRST (S. Tosatto, manuscript in preparation), were evaluated in this study and were selected from methods available at the CAFASP4 model quality assessment website (http://www.cs.bgu.ac.il/~dfischer/CAFASP4/).
Model similarity methods
Model similarity methods are used to determine how similar a model is to the native experimental structure. Often the term model quality is used to designate the class of programs that use the experimental structure to evaluate a generated model's native-like structural quality. It is important to make the distinction that the term ‘model similarity’ is used in this context to signify this particular group of methods, whereas, MQAP methods describe the set of tools that calculate a model quality score from the predicted structure alone. Model similarity is assessed here using MaxSub, which calculates the largest subset of C atoms that can be superimposed over the native structure at a given distance threshold, and the 3D-Score, a measure that combines a rigid-body superposition with the contact measure (http://bioinfo.pl/LiveBench/). A MaxSub distance threshold of 3.5 Å was used. In addition, the GDT_TS score (Zemla, 2003) was used as a comparator and this provided a score calculated as the average percentage of C atoms that can be superimposed to 1, 2, 4 and 8 Å.
Assessing top model selection accuracy
The procedure for assessing the ability of a method to select the best model for a target is achieved as follows. The predicted best model for a target is selected by re-ordering the models in the best to worst order using the calculated MQAP scores. The corresponding similarity score of the top model after the re-ordering is then selected. The final set of 188 similarity scores is then compared with the similarity scores of the top model predicted using the original server, and a one-sided Wilcoxon Matched Pairs Signed-Rank test is used to calculate whether the similarity scores composing each pair differ in size. The one-sided test ensures that only significant increases in scores using the MQAP program are recorded, and a significant P-value means that the MQAP selects a better top model than the original server. The significance level used in this test is 95%.
Assessing model quality ranking using mean Spearman's rank correlation
To test the ranking ability of a MQAP over the original server, we perform a Spearman's rank correlation analysis to measure the direction and strength of the relationship between two variables. In this case the analysis was performed for the similarity scores of the N models for each target and the MQAP scores for the same models. Our null hypothesis states that there is no relationship between the ranks of the similarity scores and the MQAP scores. The Spearman's rank correlation, R, allows the comparison of the rank order of two variables and takes values in the range of –1 to 1. An R score of 1 signifies perfect correlation between the similarity scores and the MQAP scores, asserting that the MQAP ranks the models in perfect order and that we must reject the null hypothesis. An R score of –1 shows perfect negative correlation, i.e. the MQAP orders the models in the reverse order (worst to best). A score of 0 means that there is no difference in order and that the MQAP provides no value over the original method's ranking. We calculate R values for all the 188 models and present the mean or average R value over the set for each MQAP method.
The overall ranking of the MQAP's is assessed using the combined set of 11 880 models. For each target, all the models generated by each of the eight servers are pooled and the Spearman's rank correlation R calculated over the set. This combined set was then split into ‘easy’ and ‘hard’ targets, and the Spearman's rank correlations calculated once more.
Analysis of method confidence
The method confidence, defined as the number of true positives that were given a similarity score above a threshold value, is calculated by showing the number of true positives against false positives detected by each method. The threshold values for defining a true positive in the similarity scores were a MaxSub 
0.3 (30%), a GDT_TS 25% and a 3D-Score 40. Models that achieve similarity scores below these values in most cases often adopt the wrong fold or topology, or are bad structural models.
|
RESULTS |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Improving top model selection
Table 1 shows the results of a Wilcoxon Matched Pair Sign-Rank analysis (see Materials and Methods section), which examines whether these MQAPs provide significant improvements in the similarity scores of the top model selected for each target over the top model selected by the original servers. Clearly, where there is a significant improvement in the top model selection ability, MODCHECK is able to provide this in most cases. The improvement is often dependent on the similarity score, though MODCHECK consistently improves FFAS and FFAS03 largely independent of the similarity measure. The use of the GDT_TS score generates more significant results overall, and ProQ-LG and FRST are able to provide significant top model selection improvements for ORFeus and Pcons-4, respectively. It is interesting to note that the top model selection for 3D-PSSM, GenTHREADER, mGenTHREADER and SAM-T02 is not improved by any MQAP method.
|
Improving the rank order of server models
The ability of these MQAPs to improve the overall rank of the top N submitted models is shown in Figure 1, where the results are displayed for each of the servers with rank comparisons for all the three similarity scoring methods. First, examining the ranking using the MaxSub score shows only FFAS and FFAS03 rankings, which could be improved using these MQAP methods and both the improvements are achieved by MODCHECK. Notably for FFAS, the mean Spearman's correlation value is high (R
0.6) using MODCHECK.
|
The results of the analysis using the GDT_TS score are less clear. MODCHECK, ProQ-LG and FRST improve the FFAS ranking, and all the methods improve the FFAS03 and ORFeus ranking. For Pcons-4 and SAM-T02, improvements are seen with the use of the ProQ-LG score and FRST, and GenTHREADER rankings benefit from the use of MODCHECK and ProQ. These mixed results show that improvements can be gained although the correlation values are often lower than those seen when the MaxSub score is used. Very few improvements in rank are seen with the use of the 3D-Score but for the two cases FFAS and FFAS03, MODCHECK, ProQ and Solvex can provide improvements in the former, and MODCHECK alone in the latter.
Assessing overall performance by combining LiveBench-9 models
The assessment of results for the combined model set (Materials and Methods section) shows the general performance of the MQAP methods on a large dataset. The Spearman's rank correlation coefficient for each model quality assessment method describes the general ability of that method to rank the models. Figure 2 shows the comparison of MQAP methods coefficients for each similarity score. Overall, MODCHECK and ProQ show good ranking abilities with less clear results for Solvex and FRST. The methods show much larger correlation values on the ‘easy’ targets than ‘hard’ targets as expected, and there is a noticeable variability between the Solvex and FRST scores in these two cases. The lower ranking ability of MQAPs in the case of ‘hard’ targets is most likely a reflection of the lack of completeness of the generated models in many cases.
|
Examining the confidence in these methods
It is important to quantify the level of confidence that can be expected from these model quality assessment predictions if these tools are to be used in automated structure prediction procedures. Figure 3 shows a representation of the confidence level of each method prediction over the whole set of 11 880 models. The level at which we can be confident that a model is ‘good’ can be specified by thresholding the similarity scores. Threshold values for defining a true positive (TP) were a MaxSub score
0.3 (30%), a GDT_TS score 25% and a 3D-Score 40. The results of the analysis using the MaxSub score show that we can be most confident that the MODCHECK and ProQ methods will provide a true model more often than the other methods tested at this MaxSub threshold. Solvex generates a large number of false positives, and hence, we can assume a low confidence in its predictions. For the GDT_TS score, shown in Figure 3b, the ProQ-LG score, MODCHECK and FRST provide an acceptable confidence level, though the overall number of true positives is reduced compared with the MaxSub score.
|
|
DISCUSSION |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
In this study, we have examined the ability of different MQAP methods to select the best models from a set of fold recognition models, and their ability to correctly rank the structures. We have also shown how confident we can be with the results provided by these methods. Additionally, we have shown that a detailed comparison of methods is affected by the similarity score chosen to provide a base measure of structural similarity to native structures. Based on this analysis we have tried to deduce, from a practical perspective, whether these methods add value to existing server-based fold recognition methods. The results of this study indicate that MODCHECK is the most reliable method for improving the top model selection and re-ranking sets of models though we note that all methods failed in many of the cases examined. ProQ also provides confident predictions, though at a slightly lower level of accuracy than that of MODCHECK by using the set of models obtained from the LiveBench-9 experiment. We also find FRST is able to improve many servers' model ranking when using the GDT_TS score, but notice that the overall level of improvement is less than that of MODCHECK and ProQ.
Which methods can we rely on?
The results in Table 1 show that in many cases, where improvements in the top model selection ability can be gained over the original server methods, MODCHECK clearly improves the cumulative scores more often. In addition, where MODCHECK provides improvements, for example in the cases of FFAS and FFAS03, the improvements are seen over all the three similarity scores. We find a number of less clear cases where an MQAP method improves structure prediction servers using one similarity score but fails to provide significant improvements using the other similarity measures. Also worth noting are the cases of 3D-PSSM, GenTHREADER, mGenTHREADER and SAM-T02 where no method can significantly improve the top model selection.
Given that we cannot gain insight from the top model selection results alone, we then assess the ability of these MQAP methods to rank the set of models. A method that can produce an optimal ranking will be able to rank the top scoring model as the first model and therefore, we can consider a MQAP method that consistently achieves this to fulfil the first criteria of a model selection method implicitly. Figure 2 provides the mean rank correlation coefficients for a Spearman's rank correlation analysis of each method for all servers. The Spearman's rank test will provide an appropriate statistical result in this application only if we can assume that the true rank order is accurately reflected in the results provided by the similarity measure scores. A full quantitative assessment of these tools is beyond the scope of this paper and we refer the reader to a number of publications on the subject (Wallin et al., 2003; Cristobal et al., 2001). For this reason we have included three similarity scoring methods in this analysis to gauge whether the ‘standard of truth’ affects the benchmark results. Comparisons can then also be made between results produced for different similarity scores. We would expect to find the rank order of an MQAP method to agree across similarity scores if these measures provide the same assessment of a model's similarity to a native conformation. Instead, there is a lack of agreement between the order of the MQAP methods and also in the level of correlation. This makes the task of choosing the most appropriate method more difficult. MODCHECK is best able to improve model ranking in two cases when the MaxSub score is used, and the same two cases (FFAS and FFAS03) where the 3D-Score is used. In contrast, when the GDT_TS score is used, many of the MQAP methods are able to achieve better re-ranking over the original methods. However, we have to ask whether these results are significant given that the GDT_TS score enables all methods to be improved, while using the MaxSub score only results in two cases, or whether this phenomenon is a result of the similarity scores’ ability to compare models and native structures effectively. Solvex performs worst in these tests and this may be in part, due to its application to models lacking full atomic coordinates.
Given these results, we examined how confident we can be in the predictions made by these MQAP methods. We plotted the number of true positives against the number of false positives predicted by each MQAP method using the whole dataset, where the threshold values define the boundary for a true or ‘good’ predicted model. We find a large false positive rate among all MQAP methods though the largest number of true positives was obtained using MODCHECK and ProQ across all similarity scores.
Implications for improving fold recognition methods
The earliest fold recognition methods (so-called ‘classical threading’ methods) made use of 3D structural information without any reference to the sequence of the template proteins. Early results from threading were promising, as fold similarities could be detected which were far beyond the abilities of sequence-based methods at the time. However, although this feature also allowed the possibility of recognizing evolutionarily unrelated proteins with similar folds (analogous folds), the quality of threading models (i.e. the sequence–structure alignment accuracy) was found to be poor. Subsequent development of sensitive sequence comparison methods such as PSI-BLAST (Altschul et al., 1997), hidden Markov models (Karplus et al., 1998) and the more recent profile–profile comparison methods (Rychlewski et al., 2000) has caused a shift in the fold recognition field away from structure-based approaches. Although some fold recognition programs (e.g. GenTHREADER) incorporate both structure and sequence information in their algorithms, in benchmarking studies these methods have not been shown to be better than the best sequence-based methods that employ profile–profile alignment (e.g. FFAS03). This has given rise to the belief that 3D structural information is in some way redundant in fold recognition, but this belief rather defies logic.
Our results here show quite clearly that by adding 3D structural information by means of a number of MQAPs, even the very best profile–profile fold recognition methods can be improved in terms of their ability to correctly identify the best models. Interestingly, no real improvements are observed for methods such as GenTHREADER or 3D-PSSM, which already make use of 3D information. This suggests that further improvements in fold recognition will be achieved by combining a state-of-the-art profile–profile alignment algorithm with a state-of-the-art MQAP. Or, in other words, that 3D structural information is still a valuable resource in carrying out accurate protein fold recognition.
|
CONCLUSION |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
At the current level and accuracy of structure prediction methods, building multiple models for a target sequence is often beneficial given the difficulty in guaranteeing high-quality single predictions. Structure prediction methods often generate large numbers of models for a target sequence and to be able to assess the properties of these conformations is extremely important for detecting ‘native-like’ conformations. When assessing the ability of an MQAP method to perform this task, the questions that should be asked are first, whether the method can fulfil the criteria for selecting the most native-like model and ranking the remaining models, and second, how consistently the method can do this.
MODCHECK is able to improve the top model quality selection ability for structure prediction servers that do not already attempt to incorporate information from the 3D structure of the template protein, and both the MODCHECK and ProQ are consistent in improving model rankings in these cases. Although these results indicate that some of the methods tested may be useful for the incorporation into new automated procedures, it is important to note the high false positive rates for all four of the MQAPs. This suggests that better approaches to fold recognition will need to treat the outputs from MQAPs as additional features in a machine learning method, for example, rather than as a simple post-filter.
|
Acknowledgments |
|---|
This study was supported by the UK Biotechnology and Biological Sciences Research Council (C.S.P. and L.J.M.) and the UK Department of Trade and Industry (L.J.M.). The work was also supported by the BioSapiens Network of Excellence funded by the European Commission FP6 Programme, contract number LHSG-CT-2003-503265.
Received on January 31, 2005; revised on May 11, 2005; accepted on June 13, 2005
|
REFERENCES |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Altschul, S.F., et al. (1997) Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Res., 25, 3389–3402
Bujnicki, J.M., et al. (2001) LiveBench-1: continuous benchmarking of protein structure prediction servers. Protein Sci., 10, 352–361
Cristobal, S., et al. (2001) A study of quality measures for protein threading models. BMC Bioinformatics, 2, 2–5[CrossRef][Medline].
Ginalski, K., et al. (2003) ORFeus: detection of distant homology using sequence profiles and predicted secondary structure. Nucleic Acids Res., 31, 3804–3807
Hendlich, M., et al. (1990) Identification of native protein folds amongst a large number of incorrect models—the calculation of low-energy conformations from potentials of mean force. J. Mol. Biol., 216, 167–180[CrossRef][ISI][Medline].
Holm, L. and Sander, C. (1992) Evaluation of protein models by atomic solvation preferences. J. Mol. Biol., 225, 93–105[CrossRef][ISI][Medline].
Jaroszewski, L., et al. (2000) Improving the quality of twilight-zone alignments. Protein Sci., 9, 1487–1496[Abstract].
Jones, D.T. (1999) GenTHREADER: an efficient and reliable protein fold recognition method for genomic sequences. J. Mol. Biol., 287, 797–815[CrossRef][ISI][Medline].
Jones, D.T., et al. (1992) A new approach to protein fold recognition. Nature, 358, 86–89[CrossRef][Medline].
Karplus, K., et al. (1998) Hidden Markov models for detecting remote protein homologies. Bioinformatics, 14, 846–856
Karplus, K., et al. (2003) Combining local-structure, fold-recognition and new-fold methods for protein structure prediction. Proteins, 53, 491–496.
Kelley, L.A., et al. (2000) Enhanced genome annotation using structural profiles in the program 3D-PSSM. J. Mol. Biol., 299, 499–520[ISI][Medline].
Lee, M.R., et al. (2001) Molecular dynamics in the endgame of protein structure prediction. J. Mol. Biol., 313, 417–430[CrossRef][ISI][Medline].
Li, W., et al. (2002) Sequence clustering strategies improve remote homology recognitions while reducing search times. Protein Eng., 15, 643–649
Lundstrom, J., et al. (2001) Pcons: a neural-network-based consensus predictor that improves fold recognition. Protein Sci., 10, 2354–2362
McGuffin, L.J. and Jones, D.T. (2003) Improvement of the GenTHREADER method for genomic fold recognition. Bioinformatics, 19, 874–881
Moult, J., et al. (2003) Critical assessment of methods of protein structure prediction (CASP)-round V. Proteins, 53, 334–349.
Petrey, D., et al. (2003) Using multiple structure alignments, fast model building and energetic analysis in fold recognition and homology modeling. Proteins, 53, 430–435.
Rychlewski, L., et al. (2000) Comparison of sequence profiles. Strategies for structural predictions using sequence information. Protein Sci., 9, 776–785[Abstract].
Rychlewski, L., et al. (2003) LiveBench-6: large-scale automated evaluation of protein structure prediction servers. Proteins, 53, 524–527.
Siew, N., et al. (2000) MaxSub: an automated measure for the assessment of protein structure prediction quality. Bioinformatics, 16, 776–785
Sippl, M.J. (1995) Knowledge based potentials for proteins. Curr. Opin. Struct. Biol., 5, 229–235[CrossRef][ISI][Medline].
Sussman, J.L., et al. (1998) Protein Data Bank (PDB): database of three-dimensional structural information of biological macromolecules Acta Crystallogr. D Biol Crystallogr., 1, 1078–1084.
Venclovas, C., et al. (2003) Assessment of progress over the CASP experiments. Proteins, 53, 585–595.
Wallin, S., et al. (2003) Testing similarity measures with continuous and discrete protein models. Proteins, 50, 144–157[CrossRef][Medline].
Wallner, B. and Elofsson, A. (2003) Can correct protein models be identified? Protein Sci., 12, 1073–1086
Zemla, A. (2003) Lga: a method for finding 3D similarities in protein structures. Nucleic Acids Res., 31, 3370–3374
Zhang, Y. and Skolnick, J. (2004) SPICKER: a clustering approach to identify near-native protein folds. J. Comput. Chem., 25, 865–871[CrossRef][ISI][Medline].
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  L. J. McGuffin The ModFOLD server for the quality assessment of protein structural models Bioinformatics, February 15, 2008; 24(4): 586 - 587. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Fasnacht, J. Zhu, and B. Honig Local quality assessment in homology models using statistical potentials and support vector machines Protein Sci., August 1, 2007; 16(8): 1557 - 1568. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Cheng and P. Baldi A machine learning information retrieval approach to protein fold recognition Bioinformatics, June 15, 2006; 22(12): 1456 - 1463. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||