Bioinformatics Advance Access originally published online on December 21, 2004
Bioinformatics 2005 21(8):1572-1578; doi:10.1093/bioinformatics/bti238
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Quick and simple: quality control of microarray data
ARC Seibersdorf research GmbH, Department Bio-resources A-2444 Seibersdorf, Austria
*To whom correspondence should be addressed.
 |
Abstract |
|---|
 TOP Abstract INTRODUCTIONMATERIALS AND METHODSRESULTS AND DISCUSSIONREFERENCES |
|---|
Motivation: Microarrays are high-throughput tools for parallel miniaturized detection of biomolecules. In contrast to experiments using ratios of signals in two channels, experiments with only one fluorescent dye cause special problems for data analysis. The present work compares algorithms for quality filtering on spot level as well as array/slide level.
Results: Methods for quantitative spot filtering are discussed and new sets of quality scores for data preprocessing are designed. As measures of spot quality also reflect the quality of protocols, they were employed to find the optimal print buffer in an optimization experiment. In order to determine problematic arrays within a set of replicates we tested methods of outlier detection which can suitably replace the visual inspection of slides.
Contact: Ursula.Sauer{at}arcs.ac.at
|
INTRODUCTION |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTS AND DISCUSSIONREFERENCES |
|---|
Data collected from microarray experiments are random snapshots with errors, inherently noisy and incomplete. The problem of making microarray data more reproducible is tackled from various directions. Variability caused by several factors in the fields of experimental design, experimental setup, image analysis and data analysis, disguises actual differences in signal intensities. In previous papers we focused on optimization of technical factors such as quality of support (glass) and coating, print buffer, probe immobilization, arrayer (pins), blocking and probe concentration (Preininger and Sauer, 2003a,b). Proper choice of probe sequences with optimal specificity and probe length is another important step toward good data quality (Bodrossy, 2003). But even after optimizing the experimental approach and the image processing of an array, variability of the signal intensities remains high and data interpretation may be misleading. Many statisitcal approaches are based on the assumption that one part of variation is systematic and the other is random, which may be accounted for through biological and technical replicates (Lee et al., 2000; Kerr et al., 2002) and error models (Huber et al., 2003).
Data preprocessing, i.e. data filtering, transformation and normalization, may improve the situation by correcting for systematic variation. Until now the primary attention has been focused on methods of channel normalization and transformation, while now data quality and filtering techniques are gaining more and more interest (the MGED Data Transformation and Normalization Working Group, http://www.mged.org). Data filtering is the assignment of pass/fail criteria to the quality control measures applied. Algorithms for this purpose have to meet problems such as overlapping spots, contamination (dust, surplus dye), strong background and varying spot size, saturated measurement, scratches, donuts or spots that are not at the expected location. Furthermore, they have to be usable for automated detection of poor quality spots, as visual inspection is time-consuming and prone to error. Simple filtering methods employ a fixed threshold, which the background corrected signal intensities must exceed or use a cut-off value for signal-to-noise ratios (Jenssen et al., 2002). Choosing these thresholds is a non-trivial task, as noise varies from array to array. A method for automatic preprocessing of raw data uses the distance to the least-mean square line in a plot of duplicate probes (Kadota et al., 2001). The composite quality score introduced by Wang et al. (2001) provides a very comprehensive quality assessment of microarray data and was also integrated in a normalization procedure (Wang et al., 2003). Tran et al. (2002) suggested a correlation between mean and median of pixel intensities as a measure for spot quality.
The present work evaluates three methods for spot filtering, namely the spot-related coefficient of variation (CVspot), the mean–median correlation (Tran et al., 2002) and the composite quality score (Wang et al., 2001), by comparing the reproducibility of raw data and quality filtered data. The second part describes the quantitative evaluation of technical factors by means of extended sets of quality scores. This is shown for probes printed in various buffers. Producing large batches of printed oligonucleotide microarrays on our proprietary surfaces requires a routine method for assessing quality of arrays, replacing the visual inspection of slides. The third task of the work presented here is to introduce a new algorithm for detecting problematic arrays within a set of replicates and compare the output to the batch quality control method, employing a ratio constant of selected probes, introduced earlier (Preininger et al., 2004).
|
MATERIALS AND METHODS |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTS AND DISCUSSIONREFERENCES |
|---|
All chips and arrays were produced and processed in-house.
Chip manufacture. The so-called ARChip UVTM was manufactured by coating Silane PrepTM slides (25 x 75 mm) (Sigma, S4651) with 2% poly(styrene-co-4-vinylbenzyl thiocyanate) (PST-co-VBT) (Preininger et al., 2004) in chloroform using a quadrupol film applicator (Erichsen, model 360). The wet-layer thickness was 15 µm.
Assay. The 16S rRNA oligonuceotide Alf 1b (5'-CGTTCGYTCTGAGCCAG-3', 33T spacer, NH2-link) was ordered from VBC-Genomics. The oligo is specific for various groups of methanotrophic bacteria. The gene targeted is the pmoA gene, which is present in all the known methanotrophs. The probe was printed in eight different print buffers: H2O, 3 x SSC (saline–sodium citrate); 0.01 N phosphate buffer, pH 8.0; 0.1 N phosphate buffer, pH 8.0; 0.01 N carbonate buffer, pH 9.5; 0.1 N carbonate buffer, pH 9.5; 50% dimethyl sulfoxide (DMSO) in 0.1 N phosphate buffer, pH 8.0 and 50% DMSO in H2O. Two random probes and two buffer blanks were added as negative controls.
Probe concentration was 20 µM. Probes were arrayed in triplicate onto the chips by using a piezoelectric biochip arrayer from Packard BioScience.
Hybridization. Total 16S rDNA sequences were amplified using an initial denaturing step of 5 min at 95°C followed by 30 cycles of 30 s at 95°C, 1 min annealing at 52°C and 1 min extension at 74°C. The PCR reaction mixtures (50 µl) contained 5 µl reaction buffer (Gibco, BRL), 200 µM dNTP's, 0.15 µM primers 1520r and Dy633-labelled fD1 (Dyomics), 3 mM MgCl2 and 2.5 U Taq DNA polymerase (Gibco, BRL) and 1 µl extracted DNA. The amplification products were confirmed by gel electrophoresis on 1% agarose gels. Aliquots of 2 ng/µl Dy633-labelled Rhizobium fredii in 20 mM Tris, pH 7.4, 0.9 M NaCl and 0.01% sodium dodecyl sulfate (SDS) was denatured at 95°C for 5 min, snap cooled on ice and hybridized to the chip at 50°C for 3 h.
Read out. Binding was detected by measuring the signal intensity (
ex = 635) with a non-confocal fluorescence scanner (Genepix 4000, Axon Instruments). Spot detection and signal segmentation were carried out with GenePix Pro 4.0. Scanner settings were chosen with maximum scope (25–300%) of feature size, and without a minimum feature threshold, meaning that all spots except blanks, which are flagged ‘not found’, were included in further analyses. No additional flagging was performed by the operator. The resulting data output from GenePix Pro is called raw data in the following paragraph.
Data analyses. Spot quality was evaluated by applying quality metrics, described in the following paragraph and in the Results section:
The correlation of mean and median signal intensity was calculated for each spot in an array by dividing the smaller of the mean or median by the larger (Tran et al., 2002) and the compared with a threshold.
The combined quality index proposed by Wang et al. (2001) consists of
 | (1) |
 | (2) |
 | (3) |
 | (4) |
 | (5) |
 | (6) |
All quality measures are values between 0 and 1. Briefly, qsize assesses the irregularities of spot size, q is a measure for the signal-to-noise ratio, qbkg1 quantifies the variability in local background and qbkg2 scores the level of local background.
For automated analysis all algorithms were implemented in an ACCESS database, formulating the algorithms in SQL.
|
RESULTS AND DISCUSSION |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTS AND DISCUSSIONREFERENCES |
|---|
Evaluation of spot filtering methods
We chose four replicate oligonucleotide arrays of varying quality to evaluate the efficiency of quality filtering, based on two algorithms proposed before (Wang et al., 2001; Tran et al., 2002) and the CVspot. Array no. 4 included an evident outlier, i.e. one spot with 42% saturated pixels out of 216 spots with 0% saturated pixels.
Data generated from microarray hybridizations were neither transformed nor normalized, nor corrected for background previously. For each spot the composite quality score (1), the mean–median correlation and the CVspot were calculated and used for data filtering with 3–4 thresholds each.
 | (7) |
The outcome of spot filtering was evaluated by comparing replicate arrays by means of correlation coefficients based on the mean values of replicate spots, as these were the data used for further analysis (Table 1). Furthermore, the percentage of spots which are deemed bad by the quality measure used, was determined. To get an initial idea of the power, the threshold values were set to 0.35, 0.45 and 0.55 for the composite quality score, 0.85, 0.75 and 0.65 for the mean–median correlation and 0.5, 0.6 and 0.7 for the CVspot filter. Coefficients of correlation of raw data ranged from 0.59 to 0.85. The composite quality score rejected 3–31% of the spots, but increased the correlation up to r = 0.91, whereas the mean–median correlation filter increased it up to 0.88, paying the high prize of rejecting 13–37% of the features. Regarding CVspot even the lowest cut-off (0.7) turned out to improve correlation coefficients sufficiently (0.88–0.91). More stringent values tremendously increased the number of spots, which did not fulfill the respective criteria, but without increasing reproducibility significantly. In contrast to the composite quality score and the CVspot filter, the mean–median correlation did not detect the outlier unless it was combined with a saturation-filter that rejects all spots with >10% of saturated pixels. Saturation happens when the intensity values exceed the detection range of the detector. The instrument settings have to be adjusted to minimize the amount of saturated spots. In spite of that saturated pixels will occur in contaminated spots. Usually, spots with >10% saturated pixels are rejected if mean intensity values are used, for median intensity values, which are less sensitive to variations, a cut-off of 50% is suggested (Wang et al., 2001).
|
The optimum threshold for each quality score is the one resulting in a maximum true positive (%) and a minimum false positive rate (%). This procedure resulted in 0.45 for the composite quality score and 0.55 for the mean–median correlation, resulting in 91 and 76% correctly identified positive controls, and 5 and 7% false positives. As the characteristics of spots valuated by quality scores change with technical factors and processing protocols (e.g. surface chemistry, probes, print buffer, hybridization conditions), optimal stringency has to be determined anew if one of these parameters is changed. Too high stringency means loss of information and a too small number of features included in further analysis, which may decrease statistical robustness. For diagnostic one-color arrays, the question whether a signal is detectable or not is vital. It depends on accurate signal quantification, thresholds and stringency, which therefore should be chosen carefully. While rejecting spots with signals near background is quite common for gene expression experiments (Finkelstein et al., 2002), low signals are a part of important information in diagnostic one-color experiments and also for detecting rarely expressed genes in expression profiling experiments, and therefore should not be filtered. Low signals are not lost by quality filtering with the scores tested. On the contrary, signals near to background (e.g. negative controls) usually show excellent mean–median correlations and low CVspot.
We agree with Wang et al. (2001) that poor correlations between replicates are caused by spots of poor quality, that quality filtering and definition of cut-off values are critical for the reproducibility of the experiments and therefore become key issues of data mining. This is especially true for one-color experiments, e.g. diagnostic and protein arrays, where no channel normalization is done and instead of ratios or log ratios raw intensities are the basis for further analysis. Using ratios of signals yielded a 4-fold reduction in the coefficient of variation (Finkelstein et al., 2002). Nevertheless, all of the proposed algorithms can easily be applied for both, one- and two-color experiments. All filtering methods tested improved correlation between replicate slides, CVspot, and mean–median correlation are powerful and at the same time straightforward methods with minimal computing requirements. Calculating the composite quality score needs more effort, but by including more spot characteristics, is applicable more generally.
Optimizing experimental conditions employing quality scores
The quality metrics focus on different spot characteristics and therefore were employed to study impact of print buffer on spot quality. Quality scores of features were compared directly and to spot morphology, employing the three-dimensional (3D) graph function of IconoClustTM (Clondiag). The choice of print buffer can affect the hybridization efficiency drastically. The optimal print buffer leads to high quality spots with good spot morphology, i.e. with an even distribution of fluorescence, regular shape and low local background. According to Tran et al. (2002) high quality spots show a high correlation between mean and median signal intensities. But as shown in Figure 1a and c neither mean–median correlation nor composite quality score reveal donut-shaped spots. On the other hand, features with good spot morphology may have a low mean–median correlation (Fig. 1b and d). The reason for this and for the high CVspot in Figure 1d may be inaccurate spot segmentation (e.g. including background pixels). There are several algorithms for spot segmentation, the process defined as the extraction of signal intensities from microarray images, and a critical step toward accurate signal quantification (Yang et al., 2002). Donut-shaped spots show bimodal density curves, which results in misleading mean–median correlations. The formation of donut-shaped spots may be avoided by using printing solutions with low evaporation, by keeping the relative humidity high during printing or by rehydrating after spotting. We suggest the coefficient of variation of pixel intensities (CVspot) within an individual spot as a straightforward measure for good spot morphology. We found that spots with irregular shape and donut-shaped spots have a CVspot > 0.5, while CV of spots with good morphology range 
0.3. In contrast to biological assays, where all intensities within the dynamic range of the scanner are possible, not only the signal-to-noise ratio but also the signal intensity is an important feature for optimizing experimental conditions. Consequenly, extended quality measures qcom2 and qcom3 (8 and 10) were introduced. Following the model of Equation (1) two versions of the combined quality score were defined.
|
The extended versions of the combined quality index include raw signal intensities
 | (8) |
 | (9) |
 | (10) |
 | (11) |
The effect of a signal equal to the background intensity was tested by simulating low signal intensities: altering the foreground intensity from 5028 to 370 lowered the composite quality score from 0.71 to 0.61 which does not represent a significant decrease, while the extended quality measure [Equation (8)] dropped from 0.75 to 0.41, due to the effect of the new introduced score qF. Figure 2 compares mean quality scores (8) of spots printed in different buffers, revealing that printing probes in 0.1 N carbonate buffer, pH 9.5 leads to spots of highest quality. What are the reasons for the poor quality of spots printed in 3 x SSC? The extended composite quality scores (8) and (10) provide a very comprehensive quality assessment of microarray data. They are composed of six scores, grading size, signal-to-noise ratio, background uniformity, saturation status and signal or the spot related coefficient of variation. Studying the components of the quality measures in detail allows us to trace back the effect of print buffer to certain spot characteristics. Figure 3 shows the components of the extended composite quality score for four printing buffers. In most cases a low composite quality score was caused by high variation in local background noise (qbkg1) or low signal (qF) or both. With 3 x SSC the score for spot size (qsize) contributed as well to a low composite score. This special problem may be overcome by adding SDS to the buffer. The score for signal-to-noise ratios was excellent with all buffers tested. With this method various technical factors influencing the quality of slides and individual spots, e.g. coating, cross linking, washing, blocking and hybridization procedures, may be studied in detail.
|
|
Array filter and batch control
The aim was to find a straightforward algorithm for detecting outliers in a set of hybridized microarrays, which is an imperative quality control step for either detecting problematic arrays in an experiment or for routine batch control. Nine arrays with identical assays on three slides were compared. In order to give a first impression of slide performance and hybridization efficiency, average spot diameter, signal and background were calculated. Spot diameter appeared to be quite consistent with a deviation between slides <10%. Whereas overall signals (mean of total signal) and signal-to-noise ratios varied around 30%. Eight out of nine arrays passed the criteria of a minimum mean signal-to-noise ratio of 7, or maximum mean local background of 15%—a threshold which we defined for oligonucleotide arrays on in-house PST-co-VBT coated slides from experience. Other criteria tested as a tool for batch control were the ratio constants (Preininger et al., 2004) and the array quality filter (AQF).
Ratio constant for batch control
 | (12) |
Four ratio constants from different sets of positive control probes were calculated for each array. When the fluorescence intensity scale is arbitrary and intervals are meaningful, the quotients of replicate arrays would be constant in an ideal case. The mean and standard deviation of a ratio constant of nine arrays were calculated. Ratio constants lying outside an interval of mean±SD were flagged ‘bad’. As shown in Figure 4, two arrays had two outliers; two arrays had one outlier each.
|
AQF
To filter arrays of low quality, we employed an iterative processing of a subset of n positive controls Cik (i = 1,2,...,n) on k arrays (k = 1,2,...,m). For each i of control probe Ci we calculated the mean Ci,(1,2,...,m) and the difference
 | (13) |
Controls which did not satisfy
 | (14) |
|
|
We approached data processing from a practical point of view and thus the question of whether or not something works is crucial. In our department, biochip experiments have been performed for years, and during this time we have amassed a great deal of experience with the problems and issues involved. Furthermore, the underlying theoretical consideration was that variability caused by technical factors may predominate over biological variability—our current investigation. Filtering out spots/arrays displaying problems that may well have been caused by experimental factors reduces this sort of variability, allowing the biological differences to come to the fore. These considerations have been checked against real data—quality filtering reduces variability among replicates, as higher correlation coefficients demonstrate. Consequenly, theory and practice corroborated each other.
Received on June 2, 2004; revised on November 29, 2004; accepted on December 15, 2004
|
REFERENCES |
|---|
|
TOPAbstractINTRODUCTIONMATERIALS AND METHODSRESULTS AND DISCUSSIONREFERENCES |
|---|
Bodrossy, L. (2003) Diagnostic oligonucleotide microarrays for microbiology. In Blalock, E. (Ed.). A Beginner's Guide to Microarrays, , New York ISBN 1-4020-7472-7 Kluwer Academic Publishers, pp. 43–92.
Finkelstein, D., Ewing, R., Gollub, J., Sterky, F., Cherry, J.M., Somerville, S. (2002) Microarray data quality analysis: lessons from the AFGC project. Plant Mol. Biol., 48, 119–131[CrossRef][Web of Science][Medline].
Huber, W., von Heydebreck, A., Vingron, M. (2003) Analysis of microarray gene expression data. Handbook of Statistical Genetics, 2nd edn , Chichester, UK Wiley.
Jenssen, T.-K., Langaas, M., Kuo, W.P., Smith-Sorensen, B., Myklebost, O., Hovig, E. (2002) Analysis of repeatability in spotted cDNA microarrays. Nucleic Acids Res., 30, 3235–3244
Kadota, K., Miki, R., Bono, H., Shimizu, K., Okazaki, Y., Hayashizaki, Y. (2001) Preprocessing implementation for microarray (PRIM): an efficient method for processing cDNA microarray data. Physiol. Genomics, 4, 183–188
Kerr, M.K., Afshari, C.A., Bennett, L., Bushel, P., Martinez, J., Walker, N.J., Churchill, G.A. (2002) Statistical analysis of a gene expression microarray experiment with replication. Stat. Sin., 12, 203–218.
Lee, M.L., Kuo, F.C., Whitmore, G.A., Sklar, J. (2000) Importance of replication in microarray gene expression studies: statistical methods and evidence from repetitive cDNA hybridizations. Proc. Natl Acad. Sci. USA, 97, 9834–9839
Preininger, C. and Sauer, U. (2003a) Quality control of chip manufacture and chip analysis using epoxy-chips as a model. Sens. Actuators B, 90, 98–103[CrossRef].
Preininger, C. and Sauer, U. (2003b) Design, quality control and normalization of biosensor chips. In Wolfbeis, O.S. and Narayanaswamy, R. (Eds.). Optical Sensors for Industrial and Environmental Applications, , Berlin Springer, pp. 67–90.
Preininger, C., Bodrossy, L., Sauer, U., Pichler, R., Weilharter, A. (2004) ARChip Epoxy and Archip UV for covalent on-chip immobilization of pmoA gene specific oligonucleotides. Anal. Biochem., 330, 29–36[CrossRef][Web of Science][Medline].
Tran, P.H., Pfeiffer, D.A., Shin, Y., Meek, L.M., Brody, J.P., Cho, K.W.Y. (2002) Microarray optimizations: increasing spot accuracy and automated identification of true microarray signals. Nucleic Acids Res., 30, e54
Wang, X., Ghosh, S., Guo, S.-W. (2001) Quantitative quality control in microarray image processing and data acquisition. Nucleic Acids Res., 29, e75
Wang, X., Hessner, M.J., Wu, Y., Pati, N., Gosh, S. (2003) Quantitative quality control in microarray experiments and the application in data filtering, normalization and false positive rate prediction. Bioinformatics, 19, 1341–1347
Yang, M.C.K., Ruan, Q.G., Yang, J.J., Eckenrode, S., Wu, S., McIndoe, R.A., She, J.X. (2001) A statisitical method for flagging weak spots improves normalization and ratio estimates in microarrays. Physiol. Genomics, 7, 45–53
Yang, Y.H., Buckley, M.J., Dudoit, S., Speed, T. (2002) Comparison of methods for image analysis on cDNA microarray data. J. Comput. Graph. Stat., 11, 108–136[CrossRef].
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  G. V. Cohen Freue, Z. Hollander, E. Shen, R. H. Zamar, R. Balshaw, A. Scherer, B. McManus, P. Keown, W. R. McMaster, and R. T. Ng MDQC: a new quality assessment method for microarrays based on quality control reports Bioinformatics, December 1, 2007; 23(23): 3162 - 3169. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. S. Song, K. Maghsoudi, W. Li, E. Fox, J. Quackenbush, and X. Shirley Liu Microarray blob-defect removal improves array analysis Bioinformatics, April 15, 2007; 23(8): 966 - 971. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. J. Martyniuk, H. Xiong, K. Crump, S. Chiu, R. Sardana, A. Nadler, E. R. Gerrie, X. Xia, and V. L. Trudeau Gene expression profiling in the neuroendocrine brain of male goldfish (Carassius auratus) exposed to 17{alpha}-ethinylestradiol Physiol Genomics, November 21, 2006; 27(3): 328 - 336. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Bylesjo, A. Sjodin, D. Eriksson, H. Antti, T. Moritz, S. Jansson, and J. Trygg MASQOT-GUI: spot quality assessment for the two-channel microarray platform Bioinformatics, October 15, 2006; 22(20): 2554 - 2555. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Jourdren and S. Le Crom Doelan: a solution for quality control monitoring of microarray production Bioinformatics, November 15, 2005; 21(22): 4194 - 4195. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||