Bioinformatics Advance Access originally published online on March 23, 2007
Bioinformatics 2007 23(10):1243-1250; doi:10.1093/bioinformatics/btm103
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
A mixture model approach to the tests of concordance and discordance between two large-scale experiments with two-sample groups
1Department of Statistics and Biostatistics Center, The George Washington University, 2140 Pennsylvania Avenue, N.W. Washington, DC 20052, USA and 2Center for Biotechnology and Genomic Medicine, Medical College of Georgia, 1120 15th street, CA4098, GA 30912, USA
*To whom correspondence should be addressed.
 |
ABSTRACT |
|---|
 TOP ABSTRACT 1 INTRODUCTION2 METHODS3 RESULTS4 DISCUSSIONACKNOWLEDGEMENTSREFERENCES |
|---|
Motivation: Due to advances in experimental technologies, such as microarray, mass spectrometry and nuclear magnetic resonance, it is feasible to obtain large-scale data sets, in which measurements for a large number of features can be simultaneously collected. However, the sample sizes of these data sets are usually small due to their relatively high costs, which leads to the issue of concordance among different data sets collected for the same study: features should have consistent behavior in different data sets. There is a lack of rigorous statistical methods for evaluating this concordance or discordance.
Methods: Based on a three-component normal-mixture model, we propose two likelihood ratio tests for evaluating the concordance and discordance between two large-scale data sets with two sample groups. The parameter estimation is achieved through the expectation-maximization (E-M) algorithm. A normal-distribution-quantile-based method is used for data transformation.
Results: To evaluate the proposed tests, we conducted some simulation studies, which suggested their satisfactory performances. As applications, the proposed tests were applied to three SELDI-MS data sets with replicates. One data set has replicates from different platforms and the other two have replicates from the same platform. We found that data generated by SELDI-MS showed satisfactory concordance between replicates from the same platform but unsatisfactory concordance between replicates from different platforms.
Availability: The R codes are freely available at http://home.gwu.edu/~ylai/research/Concordance
Contact: ylai{at}gwu.edu
|
1 INTRODUCTION |
|---|
|
TOPABSTRACT1 INTRODUCTION2 METHODS3 RESULTS4 DISCUSSIONACKNOWLEDGEMENTSREFERENCES |
|---|
Recent advances in experimental technologies enable us to obtain large-scale data sets, such as microarray gene expression data (Golub et al., 1999), mass spectrometry (MS) data (Petricoin III et al., 2002) and nuclear magnetic resonance data (Glunde et al., 2004). These data sets contain measurements collected simultaneously for a large number of features (variables). However, the sample sizes of these data sets are usually small due to their relatively high costs, which lead to the issue of concordance among different data sets collected for the same study: features should have consistent behavior in different data sets.
The issue of concordance was recently considered in microarray studies (Miron et al., 2006). For example, when different microarray data sets, which are generated from different laboratories, are used to study the same disease, genes with (truly) up- or down-regulated expressions in one experiment should also be up- or down-regulated in another experiment. Cahan et al. (2005) studied different gene lists identified from different data sets. A recent study by Ein-Dor et al. (2006) showed that thousands of samples may be necessary for generating a robust gene list for disease prediction. However, there is still a lack of rigorous statistical methods for evaluating concordance or discordance among different experiments. Because of experimental noises, a feature highly ranked in one data set may have a (much) lower rank in another data set. It is necessary to consider this randomness when we develop statistical measures or tests for concordance or discordance of all features under study.
Understanding the concordance among different data sets is important: it provides the information of reproducibility of the study. If different data sets have satisfactory concordance, then the integration of these data sets can be considered. Data integration can increase the sample size of study and therefore increase the power of discovery. Recent studies have showed that the integration of different microarray data sets can improve the identification of disease-related genes (Choi et al., 2003; Xu et al., 2005). However, if these data sets are not concordant, then we may obtain misleading results. Therefore, before data integration, it is necessary to perform a concordance or discordance analysis.
One may simply consider the correlation approach (Miron et al., 2006) for measuring concordance (calculate the correlation between two lists of test statistics from two data sets). However, this approach does not consider that features in the data set are a mixture of different components. For example, in a two-sample normal/disease microarray study, there will be thousands of differentially expressed genes as well as thousands of non-differentially expressed genes. The correlations in different components are usually different but we have no component information for any genes. Taking the overall correlation may provide misleading results (see Fig. 1 for examples). In this situation, it is more appropriate to consider mixture-model-based approaches.
|
Mixture models have been used for microarray-gene-expression data-based clustering (Ghosh and Chinnaiyan, 2002; Ng et al., 2006) and differential expression identification (McLachlan et al., 2006; Pan et al., 2002). The expectation-maximization (E-M) algorithm is usually used for the optimization of mixture models (McLachlan and Krishman, 1997). To our knowledge, this approach has not been applied to the aforementioned concordance/discordance analysis. Motivated by a recent study (McLachlan et al., 2006), we consider a three-component normal-mixture model for analyzing the concordance and discordance between two large-scale data sets with two sample groups. Based on this model, we propose two likelihood ratio tests for evaluating concordance and discordance. We conduct some simulations to understand the performance of these tests. As applications, the proposed tests are applied to three MS data sets.
Recently, MS has been widely used for various disease studies (Adam et al., 2002; Petricoin III et al., 2002; Roesch-Ely et al., 2007; Ward et al., 2006). Most of them are case-control studies. MS technology allows us to collect data at the proteomic level and to discover novel biomarkers for disease diagnoses and treatments. However, the reproducibility of MS data, especially surface-enhanced laser desorption and ionization (SELDI) MS data, has been questioned and extensively discussed (Baggerly et al., 2004; Ressom et al., 2005). In this study, we choose to apply the proposed methods to three SELDI-MS data sets so that we can further understand the reproducibility of this technology.
|
2 METHODS |
|---|
|
TOPABSTRACT1 INTRODUCTION2 METHODS3 RESULTS4 DISCUSSIONACKNOWLEDGEMENTSREFERENCES |
|---|
2.1 Motivation
We first describe three motivating examples from simulations. z-Scores (see Section 2.7 for details) are simulated from normal distributions for two data sets with 500 features. Each simulated feature has a pair of z-scores. A z-score is used to evaluate the differentiability of a feature in a two-sample data set. A feature can be up-regulated, down-regulated or null; we use normal distributions N(2,1), N(–2,1) and N(0,1) to simulate these components. If two data sets are completely concordant, each feature will belong to the same component in both data sets.
The first example is simulated from 0.6 x [N(0,1), N(0,1)] + 0.2 x [N(–2,1), N(–2,1)] + 0.2 x [N(2,1), N(2,1)], the second example is simulated from 0.6 x [N(0,1), N(0,1)] + 0.15 x [N(–2,1), N(–2,1)] + 0.15 x [N(2,1), N(2,1)] + 0.05 x [N(2,1), N(–2,1)] + 0.05 x [N(–2,1), N(2,1)] and third example is simulated from 0.9 x [N(0,1), N(0,1)] + 0.05 x [N(–2,1), N(–2,1)] + 0.05 x [N(2,1), N(2,1)]. Both the first and the third examples are completely concordant. The second example is not completely concordant. Figure 1 shows the scatter plots of these simulated paired z-scores. If the Pearson correlation coefficient is considered for measuring concordance, then it is 0.60 for the first example. However, when there are some discordant features in the second example, the correlation decreases to 0.28. Furthermore, even under complete concordance, the correlation is only 0.17 for the third example when the proportion of null component is relatively high.
2.2 A mixture model
We consider the situation that m features (variables) have been measured in two separate experiments for the same study. The purpose of these two experiments is to identify features that can significantly distinguish a disease group from a normal group. Statistically, we can test the hypothesis for each feature H0: it has the same population mean in two groups versus H1: it has different population means in two groups. After a normal-distribution quantile-based transformation of P-values (see Section 2.7 for details), we can obtain z-scores (McLachlan et al., 2006) for all m features in each data set. Then, there are m paired z-scores: 
.
Generally, the number of components in a mixture model should be restricted because of the issues of identifiability and computational burden. In general, a two-component or three-component mixture model can provide satisfactory performance (McLachlan et al., 2006). In this study, we consider a three-component normal-mixture model for the joint distribution of paired z-scores. As we describe below, it is actually an extended two-component model because one component is fixed (but its proportion is unknown):
|
| (1) |
µ, 
2 is the normal probability distribution function (p.d.f.) with mean µ and variance 2. We use the first component (index 0) to represent the null (no population mean difference) feature component. Therefore, µ 0 = 
0 = 0 and 
. (P-values under the null hypothesis are uniformly distributed. After the normal-distribution-quantile-based transformation, z-scores under the null hypothesis follow the standard normal distribution.) The second and third components (indices 1 and 2) are used to represent a down-regulated (negative population mean difference) and an up-regulated (positive population mean difference) feature components. Their corresponding parameters (means and variances) are estimated from the data with constraints: µ 1, 1 
0 and µ 2, 2 
0. 
ij is the proportion of features consistent with component i in the first study and component j in the second study with constraint 
ij ij = 1.
This general three-component normal-mixture model can be reduced to the models for evaluating complete concordance or complete discordance between the results from two experiments. We will refer to model (1) as partial concordance/discordance (PCD) model. In the following, we define ·j = i ij and i · = j ij. Notice that the marginal distributions of z(1) and z(2) are 
and 
, respectively.
2.3 Complete concordance model
If the results from two experiments are completely concordant, then each feature will belong to the same component in both data sets. Therefore, we expect ij = 0 for i
j and i · = ·i = i i. The PCD model is reduced to the complete concordance (CC) model:
|
| (2) |
2.4 Complete discordance model
If the results from two experiments are completely discordant, then each feature can be randomly assigned to any component in both studies. Therefore, any pair of z-scores is independent. The PCD model is reduced to the complete discordance (CD) model:
|
| (3) |
If this model can be established, then it may not be appropriate to integrate two data sets. The data quality of one or both of two data sets may be questionable and it is necessary to check data collection and processing procedures.
2.5 Parameter estimation
The parameters in the PCD, CC and CD models can be estimated through the well-known EM algorithm (McLachlan and Krishnan, 1997), which is briefly described as follows. The convergence of likelihood is defined as the difference between the log-likelihoods from two consecutive M-steps is less than 10–3. There are some differences among the estimation procedures for different models. Therefore, we describe them separately as follows.
2.5.1 PCD model
For the general PCD model, we introduce an additional unobserved variable w(k) for each feature in data set k, k = 1, 2, to represent its component group. If their values for all variables are given, then a simplified likelihood function can be obtained and it is straightforward to derive maximum likelihood estimation formulas for the unknown parameters (M-step); If the values of parameters are given, then a formula for the expected value of w(k) can be derived and it can be used to estimate w(k) (E-step). We begin with M-step with the following ad hoc initialization of w(k) : given a threshold c > 0, w(k) will be set to 2 or 3 if its corresponding z(k) is less than –c or greater than c, respectively; Otherwise, w(k) will be set to 1. With this initialization, the M- and E- steps are iterated until the convergence of likelihood.
2.5.2 CC model
For the CC model, the above EM algorithm is modified as follows. For each feature, instead of introducing two unobserved variables w(1) and w(2) in data sets one and two, respectively, we introduce only one unobserved variable w to represent its component group because of the special structure of CC model. We begin with M-step with the following ad hoc initialization of w: given a threshold c > 0, w will be set to 2 or 3 if its corresponding z(1) and z(2) are both less than –c or both greater than c, respectively; Otherwise, w will be set to 1. With this initialization, the M- and E- steps are iterated until the convergence of likelihood.
2.5.3 CD model
For the CD model, the parameters in each data set can be estimated separately because of the product structure of CC model. For each data set, we introduce an additional unobserved variable w for each feature to represent its component group. We begin with M-step with the following ad hoc initialization of w: given a threshold c > 0, w will be set to 2 or 3 if its corresponding z is less than –c or greater than c, respectively; Otherwise, w will be set to 1. With this initialization, the M- and E-steps are iterated until the convergence of likelihood in each data set.
2.6 Likelihood ratio test and parametric bootstrap
With the assumption of independence among different features, the likelihoods of PCD, CC and CD models are
|
|
|
|
|
|
which tests PCD (H1) against CD (H0) models. With these two tests, we can establish CC or CD model if we fail to reject the corresponding null hypothesis H0.
To assess the significance of an observed likelihood ratio test value, we can use parametric bootstrap approach, which is particularly suitable for mixture model analyses (McLachlan, 1987). For either TCC or TCD, we simulate data with the parameters estimated from the original data for the respective CC or CD model. These simulated data are generated from the corresponding null hypothesis. Then, we re-estimate the parameters and re-calculate the test statistics. A resample null distribution can be obtained after this procedure is repeated for many times (1000 in this study).
2.7 Normal-distribution-quantile-based transformation
As we proposed in Section 2.2, z-scores from a normal distribution quantile transformation are used for the analysis. These z-scores have been shown to be suitable for mixture model analyses with large-scale data (McLachlan et al., 2006). Here, we briefly describe the procedure to obtain z-scores. For hypothesis testing, we use the Student's t-test for all m features in each data set. Since these m features may not be normally distributed, we consider the permutation procedure (Dudoit et al., 2003) to assess the P-values of these observed test scores: sample group labels are first randomly permuted and then test statistics are re-calculated. After 1000 random permutations, there are 1000 permuted test scores for each of m features. We pool these 1000 x m permuted test scores {t*} and assess the P-value of an observed test score t as [(number of t*s greater than t) + 0.5] / [(number of total t*s) + 1]. (0.5 and 1 are added to avoid 0 or 1 for a P-value). This P-value is one-sided and upper-tail, which is used so that t-test scores and their corresponding z-scores have consistent signs.
With a given P-value P, its corresponding z-score z can be calculated by the following normal-distribution-quantile-based transformation:
|
|
–1(·) is the inverse function of the standard normal cumulative distribution function (c.d.f.). Because of limited computation resources, we can only perform a limited number of permutations (1000 for each feature). Therefore, some tiny P-values are under-evaluated. It is necessary to adjust their corresponding z-scores. For the three applications analyzed in this study, from the scatter plots of the t-test scores against their corresponding z-scores, it seems that the data within –4, <, z,<, –1 and 1 < z < 4 can be well fitted by different straight lines (Fig. 3). Therefore, these z-scores less than –4 are adjusted by their corresponding t-test scores using the fitted line within –4, <, z,<, –1 and these z-scores greater than 4 are adjusted by their corresponding t-test scores using the fitted line within 1, <, z <, 4. Although this fitting is not absolutely necessary for the third application (Fig. 3), we still perform it to adjust large z-scores.
|
3 RESULTS |
|---|
|
TOPABSTRACT1 INTRODUCTION2 METHODS3 RESULTS4 DISCUSSIONACKNOWLEDGEMENTSREFERENCES |
|---|
3.1 Simulation Studies
Simulations can be used to understand the performance of the proposed tests since we know the truth. We simulate z-scores for two data sets with 500 features. Each simulated feature has a pair of z-scores. We have considered three components in the proposed mixture models. However, in practice, the experimental data may be a more complicated mixture. Therefore, we simulate 0.6 x [N(0,1), N(0,1)] + 0.2 x [N(µ 1,1), N(µ 2,1)]+ 0.2 x [N(
1,1), N( 2,1)] for the CC model, [0.6 x N(0,1) + 0.2 x N(µ 1,1) + 0.2 x N( 1,1), 0.6 x N(0,1) + 0.2 x N(µ 2,1) + 0.2 x N( 2,1)] for the CD model and 0.6 x [N(0,1), N(0,1)] + 0.15 x [N(µ 1,1), N(µ 2 ,1)] + 0.15 x [N( 1,1), N( 2,1)] + 0.05 x [N(µ 3,1), N( 3,1)] + 0.05 x [N( 4,1), N(µ 4,1)] for the PCD model, in which µs are randomly simulated from a uniform distribution U[–c–0.5, –c+0.5] and s are randomly simulated from a uniform distribution U[c–0.5, c+0.5]. Three different values, 1, 1.25 and 1.5, are considered for c so that we can understand the performance of the tests TCC and TCD better. Notice that c is also the threshold value used for the initialization of the EM algorithm (see Section 2.5 for details). Among the above models, TCC is used to test the data sets simulated from the CC and PCD model, and TCD is used to test the data sets simulated from the CD and PCD model. For TCC, a rejection of CC is a true positive or a false positive when the data are generated from the PCD or CC model, respectively; Similarly for TCD, a rejection of CD is a true positive or a false positive when the data are generated from the PCD or CD model, respectively. We use the receiver operating characteristic (ROC) curves to evaluate the performances of different test statistics.
Figure 2 shows the ROC curves of TCC and TCD based on the simulations with different c values. When c = 1, the difference among different components is relatively small. The test TCC can achieve about 90% sensitivity when the specificity is 90%. The test TCD has worse performance: it achieves about 70% sensitivity when the specificity is 90% and about 85% sensitivity when the specificity is 80%. When c is increased to 1.25, the ROCs are significantly improved: both tests can achieve more than 95% sensitivity when the specificity is 95%. When c = 1.5, both tests can achieve almost perfect performances. These ROC curves suggest satisfactory performances of the proposed tests.
|
|
3.2 Applications to experimental data
We consider three applications to SELDI-MS data. The first application is to study the concordance between two replicates from different platforms (chips). Since different chips can bind to different proteins, we expect to observe a high discordance for this application. The second application is to study the concordance between two replicates from the same platform without fractionation and the third one is to study the concordance between two replicates from the same platform with fractionation. Although these replicates are from the same platform for both applications, data from different fractions may complicate mixture modeling for the third application. Therefore, we expect to observe a relatively high concordance for the second application but a relatively low concordance for the third one.
For each application, we obtain two sets of z-scores through the procedure described in Section 2.7. We perform the concordance and discordance tests TCC and TCD, and the parametric bootstrap procedure (Section 2.6). The threshold used for the EM algorithm initialization is set to 1.96 (see Section 2.5 for details). In addition to the tests, we also give surface plots for the 2D distributions of paired z-scores to compare different models. (Like histograms for describing 1D distributions, surface plots give the descriptions of 2D distributions.) Among the aforementioned models, the PCD model gives the best approximation to the observed distribution since the CC and CD models are just the two special cases of the PCD model. If the surface plot based on the CC or CD model is close to the surface plot based on the PCD model, then we may fail to reject the CC or CD model, respectively.
3.2.1 Ovarian cancer data sets
For the first application, we consider an ovarian cancer study (Petricion III et al., 2002). Currently, there are two SELDI-MS data sets: one is based on the H4 chip and the other is based the WCX2 chip. Here, 200 cases (100 normal and 100 cancerous) were studied in both data sets. The original data sets contain 15 154 features (m/z ratios) and have not been pre-processed. We use both data sets for pre-processing with the R caMassClass package, which is publicly available at http://ncicb.nci.nih.gov/download/. The following pre-processing functions are sequentially considered:
- Baseline subtraction;
- Removal of low mass portion;
- Normalization and mass drift adjustment;
- Peak identification;
- Peak alignment; and
- Imputation of empty peaks.
We consider this study to understand the concordance between the two replicates generated from different chips: H4 and WCX2. Figure 4 gives a scatter plot with histograms of the paired z-scores. The results of two replicates are quite discordant. Table 1 summarizes the test results. The P-value of TCC is almost 0. Therefore, the complete concordance model is clearly rejected. The P-value of TCD is slightly less than 0.01. Therefore, according to different significance criterion, the complete discordance model may be either rejected or not rejected. The surface plots (Fig. 5) further confirm the test results: compared to the CC model, the CD model gives a fitted distribution more similar to the fitted distribution given by the PCD model (see the upper right portion of these surface plots). Therefore, we may conclude that two data sets are completely discordant.
|
|
|
3.2.2 Prostate cancer data sets
For the second application, we consider a prostate cancer study (Adam et al., 2002). This data set has been pre-processed and there are 779 identified peaks. It contains 326 cases (159 normal/benign and 167 cancerous) and each case has been measured twice. After excluding non-informative markers with zero variance, we obtain 522 markers.
We consider this study to understand the concordance between the two replicates generated from the same platform. Figure 4 gives a scatter plot with histograms of the paired z-scores. The results from two replicates are very concordant. Table 1 summarizes the test results. The P-value of TCD is almost zero. Therefore, the complete discordance model is clearly rejected. The P-value of TCC is almost 1. Therefore, the complete concordance model is clearly not rejected. The surface plots (Fig. 5) further confirm the test results: compared to the CD model, the CC model gives a fitted distribution much more similar to the fitted distribution given by the PCD model. Therefore, we can conclude that two data sets are completely concordant.
3.2.3 Type 1 diabetes data sets
For the third application, we consider a type 1 diabetes (T1D) study (Purohit et al., 2006). This data set has been pre-processed and there are 373 peaks identified from different fractions. It contains 102 cases (69 normal and 33 T1D) and each case has been measured twice on different dates.
We consider this study to understand the concordance between the two replicates generated from the same platform with different fractions. Figure 4 gives a scatter plot with histograms of the paired z-scores. The results from two replicates are quite concordant. Table 1 summarizes the test results. The P-value of TCD is almost zero and the complete discordance model is clearly rejected. The P-value of TCC is between 0.5 and 0.75, which implies that the complete concordance is acceptable. The surface plots (Fig. 5) further confirm the test results: compared to the CD model, the CC model gives a fitted distribution much more similar to the fitted distribution given by the PCD model. Therefore, we can conclude that two data sets are completely concordant.
|
4 DISCUSSION |
|---|
|
TOPABSTRACT1 INTRODUCTION2 METHODS3 RESULTS4 DISCUSSIONACKNOWLEDGEMENTSREFERENCES |
|---|
In this study, based on a three-component normal-mixture model, we proposed two likelihood ratio tests for evaluating the concordance and the discordance between two large-scale data sets with two sample groups. We first conducted some simulations to understand the performance of these tests and then considered three applications to SELDI-MS data. The proposed methods are certainly applicable to other types of large-scale data sets, such as microarray data and nuclear magnetic resonance data.
We conducted some simulation studies to evaluate the proposed tests. The results suggested satisfactory performances of these tests. More simulation studies are certainly necessary so that these tests can be further understood, such as the power and robustness under complicated situations. Furthermore, it is necessary to develop better statistical methods to accommodate more complicated situations.
One advantage of the proposed approach is that different feature components are considered. The correlation approach (Miron et al., 2006) does not consider this. Due to data randomness, there are always certain differences between two experimental studies and therefore there are always certain differences between the test scores from the two studies for any features. Two completely concordant data sets may result in a relatively low correlation. We addressed this issue through a mixture-model-based hypothesis testing approach.
There are certain disadvantages of the proposed approach: First, it assumes independence among different features. The dependence structures in large-scale data are usually complicated. There may be considerable impact on the analysis results from these structures. Second, the proposed three-component normal-mixture model is an approximation. Although the transformed z-scores are suggested, it is still possible that the distribution model is incorrectly specified and there are more than three feature components (it seems the case from the scatter plot based on the prostate data sets in Figure 4).
The proposed method is designed for two data sets with two-sample groups. It is necessary to develop statistical methods for evaluating concordance and discordance among multiple data sets with multi-sample groups. The proposed method is a hypothesis-testing-based approach. It is necessary to develop other statistical measures so that the concordance and discordance can be quantified to distinguish different degrees of concordance.
|
ACKNOWLEDGEMENTS |
|---|
|
TOPABSTRACT1 INTRODUCTION2 METHODS3 RESULTS4 DISCUSSIONACKNOWLEDGEMENTSREFERENCES |
|---|
We thank the associate editor and two anonymous reviewers for their valuable comments. This work was partially supported by NIH grants DK-75004 (Y.L.), HD-37800 (J-X.S.) and HD-50196 (J-X.S.).
Conflict of Interest: none declared.
|
FOOTNOTES |
|---|
Associate Editor: David Rocke
Received on December 5, 2006; revised on March 3, 2007; accepted on March 10, 2007
|
REFERENCES |
|---|
|
TOPABSTRACT1 INTRODUCTION2 METHODS3 RESULTS4 DISCUSSIONACKNOWLEDGEMENTSREFERENCES |
|---|
Adam BL, et al. Serum protein fingerprinting coupled with a pattern-matching algorithm distinguishes prostate cancer from benign prostate hyperplasia and healthy men. Cancer Res (2002) 62:3609–3614.
Baggerly KA, et al. Reproducibility of SELDI-TOF protein patterns in serum: comparing datasets from different experiments. Bioinformatics (2004) 20:777–785.
Cahan P, et al. List of lists-annotated (LOLA): a database for annotation and comparison of published microarray gene lists. Gene (2005) 360:78–82.[CrossRef][Web of Science][Medline]
Choi JK, et al. Combining multiple microarray studies and modeling interstudy variation. Bioinformatics (2003) 19(Suppl. 1):i84–i90.[Abstract]
Dudoit S, et al. Multiple hypothesis testing in microarray experiments. Stat. Sci (2003) 18:71–103.[CrossRef][Web of Science]
Ein-Dor L, et al. Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer. Proc. Natl Acad. Sci. USA (2006) 103:5923–5928.
Ghosh D, Chinnaiyan AM. Mixture modelling of gene expression data from microarray experiments. Bioinformatics (2002) 18:275–286.
Glunde K, et al. Molecular causes of the aberrant choline phospholipid metabolism in breast cancer. Cancer Res (2004) 64:4270–4276.
Golub TR, et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science (1999) 286:531–537.
McLachlan GJ. On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture. Appl. Stat (1987) 36:318–324.[CrossRef]
McLachlan GJ, Krishnan T. The EM Algorithm and Extensions (1997) New York: John Wiley & Sons.
McLachlan GJ, et al. A simple implementation of a normal mixture approach to differential gene expression in multiclass microarrays. Bioinformatics (2006) 22:1608–1615.
Miron M, et al. A methodology for global validation of microarray experiments. BMC Bioinformatics (2006) 7:333.[CrossRef][Medline]
Ng SK, et al. A Mixture model with random-effects components for clustering correlated gene-expression profiles. Bioinformatics (2006) 22:1745–1752.
Pan W, et al. Model-based cluster analysis of microarray gene-expression data. Genome Biol (2002) 3:0009.1–0009.8. research.
Petricoin EF III, et al. Use of proteomic patterns in serum to identify ovarian cancer. Lancet (2002) 359:572–577.[CrossRef][Web of Science][Medline]
Purohit S, et al. Assessing the utility of SELDI-TOF and model averaging for serum proteomic biomarker discovery. Proteomics (2006) 6:6405–6415.[CrossRef][Web of Science][Medline]
Ressom HW, et al. Analysis of mass spectral serum profiles for biomarker selection. Bioinformatics (2005) 21:4039–4045.
Roesch-Ely M, et al. Proteomic analysis reveals successive aberrations in protein expression from healthy mucosa to invasive head and neck cancer. Oncogene (2007) 26:54–64.[CrossRef][Web of Science][Medline]
Ward DG, et al. Identification of serum biomarkers for colon cancer by proteomic analysis. Br. J. Cancer (2006) 94:1898–1905.[CrossRef][Web of Science][Medline]
Xu L, et al. Robust prostate cancer marker genes emerge from direct integration of inter-study microarray data. Bioinformatics (2005) 21:3905–3911.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||