Physiol. Genomics Add DOIs to your references at manuscript stage!
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Physiol. Genomics 22: 346-355, 2005. First published May 24, 2005; doi:10.1152/physiolgenomics.00260.2004
1094-8341/05 $8.00
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Supplemental Material
Right arrow All Versions of this Article:
22/3/346    most recent
00260.2004v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (18)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Jolly, R. A.
Right arrow Articles by Estrem, S. T.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Jolly, R. A.
Right arrow Articles by Estrem, S. T.
Received 3 November 2004; accepted in final form 13 May 2005.
Physiological Genomics 22:346-355 (2005)
1094-8341/05 $8.00 © 2005 American Physiological Society

Pooling samples within microarray studies: a comparative analysis of rat liver transcription response to prototypical toxicants

Robert A. Jolly1, Keith M. Goldstein1, Tao Wei1, Hong Gao1, Peining Chen2, Shuguang Huang2, Jean-Marie Colet3, Timothy P. Ryan1, Craig E. Thomas1 and Shawn T. Estrem1

1 Lilly Research Laboratories, Greenfield, Indiana
2 Lilly Research Laboratories, Indianapolis, Indiana
3 Lilly Research Laboratories, Mont-Saint-Guibert, Belgium, A Division of Eli Lilly and Company


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Combining or pooling individual samples when carrying out transcript profiling using microarrays is a fairly common means to reduce both the cost and complexity of data analysis. However, pooling does not allow for statistical comparison of changes between samples and can result in a loss of information. Because a rigorous comparison of the identified expression changes from the two approaches has not been reported, we compared the results for hepatic transcript profiles from pooled vs. individual samples. Hepatic transcript profiles from a single-dose time-course rat study in response to the prototypical toxicants clofibrate, diethylhexylphthalate, and valproic acid were evaluated. Approximately 50% more transcript expression changes were observed in the individual (statistical) analysis compared with the pooled analysis. While the majority of these changes were less than twofold in magnitude (~80%), a substantial number were greater than twofold (~20%). Transcript changes unique to the individual analysis were confirmed by quantitative RT-PCR, while all the changes unique to the pooled analysis did not confirm. The individual analysis identified more hits per biological pathway than the pooled approach. Many of the transcripts identified by the individual analysis were novel findings and may contribute to a better understanding of molecular mechanisms of these compounds. Furthermore, having individual animal data provided the opportunity to correlate changes in transcript expression to phenotypes (i.e., histology) observed in toxicology studies. The two approaches were similar when clustering methods were used despite the large difference in the absolute number of transcripts changed. In summary, pooling reduced resource requirements substantially, but the individual approach enabled statistical analysis that identified more gene expression changes to evaluate mechanisms of toxicity. An individual animal approach becomes more valuable when the overall expression response is subtle and/or when associating expression data to variable phenotypic responses.

Affymetrix microarray; statistical; hepatotoxicants


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
MICROARRAY TECHNOLOGY has enabled the measurement of mRNA abundance on a genomic scale, providing deeper insight into cellular physiology. Microarrays have been applied to many types of biological inquiry such as studying responses to environmental changes (15), classification of tumors (37), classification of compound therapeutic mechanism (6), and prediction of cancer prognosis to name only a few. In toxicology, there has been substantial effort in applying microarray technology as a tool to identify biomarkers (35), develop surrogate toxicity assays (22), determine mechanism (42), and predict toxicity (18). There have been a number of reviews covering the application of microarray technology to toxicology more extensively (26, 34).

Despite demonstrated utility, there are many challenges in using microarrays. Among them is the task of extracting useful biological information from the enormous amount of data generated. This challenge has been aided by the development of many new tools over the last few years that facilitate data reduction, visualization, and analysis (13, 49). Another prominent, yet largely unaddressed, challenge is that appropriately powered experimental designs using microarrays can be costly and time consuming (52). This cost has led many scientists to the practice of pooling replicate biological samples, which not only decreases the number of microarrays needed but also reduces sample preparation for a given study. However, with the practice of pooling comes a loss of opportunity to apply statistical analysis methods that enable the measurement of individual animal variation within the experiment. Furthermore, the ability to correlate individual transcript expression changes back to individual animal phenotypes, which often vary within treatment groups, is lost. This correlation is critical for identification of biomarkers and modeling response. Ultimately, the effect of pooling samples as opposed to a statistical analysis of individual samples relates to the balancing of costs vs. a tolerance for lack of statistical confidence in the data.

Literature on the topic of pooling samples for microarray experiments tends to be technical reports of formulas for calculating the number of biological samples (pooled or individual) needed for a properly powered analysis (23, 24, 28, 32, 44). This study examines the impact of pooling samples from both a qualitative or "biological" perspective and a quantitative viewpoint. We used the same single-dose time-course toxicology samples and pooled them or processed them individually. Three well-characterized prototypical toxicants, clofibrate (CLO), diethylhexylphthalate (DEHP), and valproic acid (VPA) were selected based on 1) knowledge of their mechanism(s) in the scientific literature, 2) the robust expression response in liver following treatment, and 3) their relative effect on peroxisome proliferator-activated receptor (PPAR) (3, 16, 33, 38, 48). The analysis methods employed for analyzing data from pooled or individual sample designs were those typical for microarray users when utilizing the respective designs. This analysis allows researchers to make informed decisions as to appropriate experimental design and sample processing based on their needs and resource constraints.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
 DISCUSSION
 REFERENCES
 
Experimental.
Animal protocols were approved by our Institutional Animal Care and Use Committee. Male Sprague-Dawley rats (n = 5) were given a single dose of test article (oral administration) and killed 48 and 168 h postexposure, as previously described (29). Additional groups of animals were killed at 4 and 24 h (n = 3) after exposure (see Supplemental Materials; available at the Physiological Genomics web site).1 Doses were selected based on what was needed to obtain acute hepatotoxicity as determined by range-finding studies (data not shown). Clofibrate was dosed at 1,000 mg/kg using 0.9% (wt/vol) saline as a vehicle (10 ml/kg). DEHP was dosed at 20 g/kg (20 ml/kg) and VPA at 2,000 mg/kg (10 ml/kg) using distilled water as a vehicle. One animal in the VPA group died before euthanasia. Animals were euthanized, and toxicity was evaluated by clinical chemistry, measuring liver weights, and histological examination of the liver after staining with hematoxylin and eosin.



View larger version (21K):
[in this window]
[in a new window]
  		Fig. 2. A: clofibrate (CLO). B: DEHP. C: VPA. Overlap of probe sets changed between approaches at 48 h. The no. of probe sets unique to the individual analysis that have absolute fold changes >2-fold are listed. For filtering criteria for both approaches, see MATERIALS AND METHODS. All changes within the overlapping section of the Venn diagrams are concordant as to the direction of change.

 


View larger version (30K):
[in this window]
[in a new window]
  		Fig. 5. Effect of VPA on mitochondrial fatty acid ß-oxidation. The mitochondrial fatty acid ß-oxidation pathway was adapted from GenMapp and shaded to demonstrate what transcripts are changing at 48 h (in this case, all of the transcripts are induced by treatment), as detected by both pooling and individual methods (darker shading) or only by individual samples (lighter shading). Additional diagrams of the mitochondrial fatty acid ß-oxidation pathway for DEHP and CLO can be found in the Supplemental Materials.

 
RNA isolation and microarray analysis.
Total RNA from liver was isolated with RNA STAT-60 (Tel-Test) according to the manufacturer's protocol. For the pooled samples, equal amounts of total RNA from each sample within a given treatment group were combined. Ten micrograms of total RNA were labeled and hybridized to Affymetrix RG_U34A arrays according to the Affymetrix protocol (1). Affymetrix fluidics station 400 and Agilent GeneArray scanner 3000 were used. Each sample was hybridized to a single microarray.

Sample quality.
All samples were assessed for RNA quality such as microarray scaling factors, background levels, percent present calls, ß-actin and GAPDH 3'/5' ratios, etc. Samples with high 3'/5' ratios would usually be excluded from further analysis; however, the CLO samples were pooled before the results for the individual animal microarrays were completed. For the remaining two compounds, samples were pooled after the quality of the individual samples was assessed, thus ensuring that the pools for VPA and DEHP contained only high-quality samples. This design allowed us to compare the two analysis methods as a worst-case scenario (CLO, with 7 of 26 samples being of suspect quality RNA) and a best-case scenario (VPA and DEHP, where all of the samples are high quality). The data discussed in this publication have been deposited in the National Center for Biotechnology Information's Gene Expression Omnibus (GEO; http://www.ncbi.nlm.nih.gov/geo/) and are accessible through GEO series accession no. GSE-2303.

Data analysis of individual samples.
Signal intensities were generated from Microarray Suite version 5.0 (MAS5) using the default settings, and a global scaling set to 1500. To apply statistical analysis to the experiment, the fold change was calculated as the ratio of the two group means based on the observed signal values from MAS5 for the two treatment groups.

To test whether a gene is differentially expressed, an ANOVA model was fitted on each of the 8,799 probe sets on the chip. The intensity value, Affymetrix MAS5 signal, of a particular gene was modeled as

where Yki is the signal of animal i from treatment group k, µk is the group mean of treatment k, and {varepsilon}ki is the measurement error (combination of sample variation and chip-to-chip variation) following distribution N(0,{sigma}2).

Because thousands of hypotheses were tested simultaneously, the issue of multiplicity is a concern. To estimate the false positives, Benjamini and Hochberg's false discovery rate (FDR) was used to adjust the P values derived from the above ANOVA model. Because FDR estimates the proportion of false positives among the tests called "significant," the cutoff value of FDR in a particular study is flexible depending on the goal of the experiment. For instance, an FDR of 0.1 represents a tolerance of 10% of false positives among the selected significant genes.

Data analysis for pooled samples.
Signal intensities were generated using MAS5 as described above, and fold-change values were obtained by running a comparison analysis in MAS5 using the default settings. The MAS5 fold-change output is the signal log ratio (SLR), which is the fold change, presented in log2. To simplify comparison with the individual analysis, the SLR was converted to a standard, nonlogarithmic scale.

Filtering criteria.
All changes for the individual analysis had an FDR ≤0.1, a minimum signal of 500 for either the vehicle or treated samples, and an absolute fold change of ≥1.4. For the pooled analysis, changes had an MAS5 call of increase or decrease, an absolute fold change of at least 2, and a minimum signal of 500 for either the vehicle or treated samples and were not called absent in both vehicle and treated samples. Analysis of these data showed that signal values below ~500 had a high coefficient of variation (data not shown). The 1.4-fold minimum change for the statistical analysis was arbitrarily chosen to remove small changes not consistently validated using quantitative RT-PCR (QRT-PCR; data not shown). A lower fold-change cutoff for the statistically analyzed data was implemented as a result of the additional confidence gained from a 10% FDR. The twofold minimum for MAS5 changes was based on guidance from Affymetrix. Implementing a twofold cutoff for both analyses resulted in a greater overlap between the two methods, but there were still numerous statistically significant changes missed by pooling, and the false positives remained (see Supplemental Materials).

Principle component analysis.
Principle component analysis (PCA) was performed as described by Yeung and Ruzzo (53) using the R statistical package (http://www.R-project.org), and data were standardized using a z-score calculation before PCA. The PCA scores corresponding to the first three principal components were imported into Spotfire Decision Suite 7.2 for visualization. To include all observable variation in the experiment and to avoid sample grouping caused by transcript selection, all probe sets on the chip were used (2, 36).

Hierarchical clustering.
Heirarchical clustering (HCS) was performed using the Pearson correlation as the distance metric and average linkage clustering in Spotfire Decision Suite. The data were clustered based on the MAS5 signal for all 8,799 probe sets.

Pathway mapping of expression data.
Mapping of expression data was performed using GenMapp 2.0 and MappFinder 2.0 (11, 12). MappFinder identified the number of genes changing for a number of pathways as well as provided a statistical evaluation of the overrepresentation of modulated genes within the various pathways. Because not all of the genes in a given pathway have been annotated to the RG_U34A chip, the data were formatted so that the pathway content of the RG_U34A chip was used for the background in the statistical calculations. MappFinder positive z-scores indicate that the number of modulated genes in the pathway is greater than expected by chance. Larger z-scores represent greater significance. Once modulated pathways were identified using MappFinder, the data were overlaid onto pathway visualization maps and colored based on whether a transcript was detected by pooled, individual, or both approaches.

QRT-PCR.
We used QRT-PCR to validate the array data. Total RNA was treated with DNase (DNA-free, Ambion), and subsequently 1.5 µg were used for first-strand cDNA synthesis (Superscript II, Invitrogen) followed by QRT-PCR using SYBR Green (Applied Biosystems). Transcripts for confirmation were picked arbitrarily, representing a range of fold change and signal change from the different subsets in the data. Because the goal of these experiments was to validate the technical results from the two approaches, the same preparation of RNA as hybridized to the arrays was used. The QRT-PCR results represent the mean values from each dosing group. Aldehyde reductase was used as an internal control. As shown in Table 1, aldehyde reductase expression levels remained consistent across all treatments. Fold changes were calculated by dividing the gene of interest/internal control for the treatment by GOI/ctrl for the vehicle samples (GOI/ctrl = signal for the gene of interest divided by the signal for the control gene). Statistical analysis (t-test) was performed on the QRT-PCR results from individual animals for each transcript using the JMP statistical package (version 4.0.4). Additional details on this method, including primer sequences, can be found in the Supplemental Materials.


View this table:
[in this window]
[in a new window]
  		Table 1. QRT-PCR confirmation

 

   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 REFERENCES
 
The transcriptional response to the toxicants was dynamic over the time course (Fig. 1). Although the time dependence of the transcriptional response within the pooled and individual analyses was similar, there were more significant transcript changes observed in the individual data set. Both analyses revealed maximal responses in gene expression at 24 or 48 h. The 48-h data were focused on because they had the largest transcription response (no. of changes), and the most robust change in liver phenotype (liver weight increase as well as minimal changes in morphology) occurred at this time point.



View larger version (16K):
[in this window]
[in a new window]
  		Fig. 1. No. of transcripts changed when samples were pooled or run individually. Graph represents the no. of probe sets on the chip that pass the filtering criteria for each time point postadministration of the toxicant. For filtering criteria for both approaches, see MATERIALS AND METHODS. DEHP, diethylhexylphthalate; VPA, valproic acid.

 
We compared the probe sets identified as differentially expressed relative to control in the pooled and individual analysis to determine whether probe sets in the pooled observations were a subset of those in the individual data set. The percent overlap of the two approaches was ~30% for all compounds (Fig. 2, A–C). Most of the pooled changes were also identified within the individual set of significant changes as demonstrated by the small number of transcripts unique to the pooled analysis (5 and 3% for DEHP and VPA, respectively, and 20% of the total for CLO). In contrast, a large percentage of changes from the individual analysis (49–67%) were not found in the pooled data set. The majority of these probe sets had expression changes of less than twofold, but there were numerous transcripts (14–20%) with expression changes greater than twofold that were also not identified by the pooled analysis (Fig. 2, A–C). Similar results were obtained with data at the 4- and 24-h time points, showing that a significant fraction of data was lost even where there were fewer total transcript changes (see Supplemental Materials).

QRT-PCR analysis of transcript changes falling in the various regions of the Venn diagram, generated from the data set, highlights additional differences between the approaches. Five or six transcripts from each of the three regions of the Venn diagram representing unique and overlapping areas of the VPA data set (Fig. 2C) were arbitrarily chosen for technical confirmation by QRT-PCR of the VPA samples. The results from the QRT-PCR analysis are shown in Table 1. It is striking to note that none of the transcripts unique to the VPA pooled analysis was confirmed as being statistically significant (P value ≤ 0.05) when the individual samples were evaluated by QRT-PCR, suggesting a high false-positive rate for this subset of transcripts upon pooling. The changes for both the overlap and the transcripts detected uniquely in the individual samples were confirmed by QRT-PCR as being statistically significant changes. Therefore, all changes identified by the statistical microarray analysis and tested by QRT-PCR were confirmed. Among these transcripts, the individual analysis identified three transcripts (sodium channel-ß1, selenoprotein W, and lanosterol-14-demethylase) called "not changed" by the MAS5 algorithm and therefore missed by the pooled approach. Each of these transcripts was strongly regulated (5.7-, 2.3-, and –5.3-fold, respectively) when detected by the individual analysis and significantly regulated when checked by QRT-PCR.

To determine the impact of a pooled vs. individual analysis on sample classification, all 48-h array data were analyzed using PCA and HCS, as shown in Fig. 3. Treatment with the three toxicants separated from the vehicle along the first and second principal components, while differences between compounds separated along the third principal component. A clear compound-dependent effect relative to vehicle control was observed using PCA. As expected, the pooled samples for each compound consistently clustered tightly with the individual samples from their respective groups in the PCA.



View larger version (31K):
[in this window]
[in a new window]
  		Fig. 3. Principle component analysis (PCA) and hierarchical clustering (HCS) analysis of all pooled and individual 48-h microarray data. Circles represent the pooled samples and triangles the individual samples, while color delineates treatment by compound: red = CLO, blue = DEHP, yellow = VPA, and green = vehicle. PCA captured 65% of the variation observed in the experiment in the first 3 principal components.

 
The VPA and DEHP samples clustered more closely together than with CLO. In addition, the spread between the individual animals in the CLO group was larger in magnitude than that observed for the other compounds. CLO samples that were distanced from the center of the clusters in PCA were lower-quality samples. Three of the five vehicle-treated animals (methylcellulose in the CLO study) had ß-actin 3'/5' ratios of >3 and two of five CLO-treated animals had ratios of >3, indicating that the RNA was partially degraded before labeling of the samples. While these parameters were outside of normal ranges, other quality control parameters for these samples were within normal limits. All other CLO samples including pooled samples had ß-actin 3'/5' ratios of <3. Samples from the VPA and DEHP experiments all had ß-actin 3'/5' ratios of <2, indicating higher-quality RNA. When the data were analyzed by HCS (Fig. 3B), another common method for microarray analysis and visualization, the different treatments were clearly separated from their respective vehicles and each other. The data separated into three distinct clusters: VPA and DEHP formed one cluster, CLO formed another, and the vehicle treatments formed a third cluster. The vehicle for VPA and DEHP (water) clustered in one branch, while the vehicle for CLO (methylcellulose) clustered in another. For all three treatments and the vehicles, the pooled samples clustered with the individual samples from their respective groups, as expected. Thus, even though the individual approach identified more significant changes in expression, both approaches resulted in similar separation of treatments when the data were analyzed by HCS.

Minimal changes in pathology were observed in these studies, and all changes were reversed by 168 h after treatment. Some vacuolation, hypertrophy, and increases in mitotic figures were noted; however, the most dramatic response to the compounds was an increase in absolute liver weight noted at 24 and 48 h. An advantage of the individual approach is the opportunity to correlate the transcript profiles to the phenotype, sometimes called "phenotypic anchoring." For example, a subtle increase in liver weight was observed in the CLO-treated animals at 48 h. The Spearman correlation was used to identify specific transcripts that correlate with increased liver weight (Fig. 4, A–C). Hydratase dehydrogenase, a known PPAR{alpha}-regulated transcript, was clearly upregulated by CLO treatment (FDR <0.001) but was not highly correlated to increased liver weight. Retinol dehydrogenase, although not associated with PPAR{alpha} activation (FDR >0.1), was highly correlated to liver weight increase. Lastly, acyl-CoA-synthetase-4 was significantly regulated by treatment (FDR = 0.03) and correlated to increased liver weight. Thus the individual approach enables both statistical analysis and correlation to phenotype, allowing for deeper investigation of the data.



View larger version (11K):
[in this window]
[in a new window]
  		Fig. 4. Correlation of transcript changes to liver weight. Gene expression data collected 48 h after CLO treatment were correlated to liver-to-brain weight ratios using the Spearman rho. For each probe set, the correlation of expression change to liver weight is listed. Shaded boxes represent vehicle control, and solid boxes represent treatment with CLO. A: K03249_at, hydratase dehydrogenase, example where there is a treatment effect but a lower correlation to liver weight. B: U33500_at, retinol dehydrogenase, an example of no treatment effect but strong correlation to liver weight. C: RC1236284_at, acyl-CoA-synthetase-4, an example of a treatment effect and correlation to liver weight.

 
The pooled and individual data from each compound were also mapped to metabolic pathways using GenMapp (11, 12) to compare pathways identified by each analytic approach. The mitochondrial long-chain fatty acid ß-oxidation pathway is illustrated in Fig. 5 for VPA (figures for all three compounds can be found in the Supplemental Materials). As might be expected, all three compounds modulated multiple transcripts that code for components of the ß-oxidation pathway (Table 2). The individual analysis identified more components of the pathway than did the pooled analysis, yet both approaches identified this pathway as having more of its genes modulated than would be expected by random chance. Table 2 shows the results when this pathway analysis was generalized to all GenMapp pathways using the MappFinder tool. Again, it was apparent that the pooled analysis generally resulted in fewer transcript changes per pathway relative to the individual method; however, both methods identified major pathways affected by these compounds. Notable pathways missed by the pooled method include proteasome degradation, ribosomal proteins, oxidative stress, cell cycle, and apoptosis and the glycine, serine, and threonine metabolism pathway in response to DEHP or VPA. One pathway, the arginine and proline metabolism pathway, was identified by the CLO pooled, not individual, analysis.


View this table:
[in this window]
[in a new window]
  		Table 2. Impact of approaches on compound-related effects assessed by pathway analysis

 

   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
REFERENCES
 
There is considerable anecdotal data and opinion on the value of using an individual vs. pooled microarray experimental design; however, there is a paucity of rigorous comparative analysis in the scientific literature. Lee et al. (28) emphasized the importance of statistics in array studies in the classification of transcripts among tumor samples. In support of pooling, Waring et al. (44) showed that pooled samples cluster with the individual samples from which they were derived (44). Recently, papers by Peng et al. (32) and Kendziorski and colleagues (23, 24) have found that subpooling is a statistically viable method for analysis of array results, particularly where the biological variation is expected to be much larger than the technical variation of the microarray. In subpooling, multiple animals from within a treatment group are pooled into multiple, evenly distributed subgroups, thus reducing the variability as well as the number of microarrays that are processed. Two of these papers are based primarily on an in silico analysis, and each suggests that to achieve the same level of precision, subpooling requires substantially more biological samples than are needed for individual samples. This is not cost effective for many in vivo experiments. Although these studies have examined the pros and cons of both designs to some degree, none has directly compared the expression changes observed using typical analytic methods commonly associated with each design.

The rationale for pooling samples tends to be nontechnical (limited sample RNA) and generally involves savings in time and reagent costs. In our experience, time and cost can be reduced up to fivefold by pooling samples. The benefits of an individual animal approach are many and include 1) estimation of biological variation, 2) robust statistical testing as opposed to heuristic filtering, 3) association with phenotypic data when available, and 4) more information to apply to mechanistic analysis. Statistics provide a quantitative measure of the probability of false positives in the data based on signal relative to noise and observed sample variation. Therefore, statistical measures of confidence, such as FDR, provide a mechanism for rational data filtering that incorporates knowledge of false-positive rates. In our comparison of pooled and individual designs, changes unique to the pooled analysis did not validate when evaluated with QRT-PCR (n = 5) using individual sample RNA, indicating that they are likely to be false positives (Table 1). This loss in confirmation of transcripts by QRT-PCR was a result of large variation in signal between samples within the treatment group as opposed to a technical problem related to the probe sets representing these transcripts on the microarray. This is supported by the observation that we could confirm the unique-to-pooled changes by QRT-PCR using pooled sample RNA (data not shown). The changes identified from the pooled data set that failed to validate can impact the interpretation of the data, as demonstrated by the fact that, at 168 h postdosing, CLO still regulated a number of changes (see Supplemental Materials) in the pooled samples, but none of these changes was statistically significant.

The expression changes uniquely identified by the individual analysis enhanced biological interpretation of the experiment. Three regulated transcripts identified in the individual analysis, sodium channel-ß1, selenoprotein W, and lanosterol-14-demethylase, were called not changed by MAS5 and therefore were not called changed in the pooled approach. These transcripts were strongly regulated (>5-fold) and were confirmed by QRT-PCR (with primers designed independently of the probe sequences on the arrays). Each of the three transcripts had a high level of hybridization to the Affymetrix mismatch probes, providing an explanation for why MAS5 did not pick up the change. Interestingly, VPA has been shown to modulate sodium channel-ß1, and this is likely associated with its pharmacological mechanism of action (51). The increase in lanosterol-14-demethylase transcript after VPA treatment has not been reported previously. This enzyme is involved in cholesterol biosynthesis and trafficking of sterols. Downregulation of this transcript and the cholesterol esterase (Table 1; a subtle –1.5-fold change also only detected in the individual analysis) by VPA may contribute to the steatotic effect in liver observed after VPA treatment (21, 39). Other transcripts involved in lipid metabolism that were strongly regulated by VPA and determined only in the individual analysis include acyl-CoA-synthetase-4 (11-fold increase) and the low-density lipoprotein receptor transcript (13-fold increase). Additionally, glycerol-3-phosphate dehydrogenase, a mitochondrial protein integral in intermediate shuttling and bioenergetics, was regulated by VPA (8-fold increase). These are just a few examples of biologically relevant changes that analysis of pooled samples did not identify.

Statistical analysis increases assay sensitivity, enabling confident identification of subtle changes and detection of changes that may be characterized by greater variability. Approximately one-half of the transcriptional changes detected by individual design were not detected in the pooled design (Figs. 1 and 2), a result similar to that reported by others (14). While most of the transcript changes detected by the individual design had fold changes of less than two, 20% of these changes were greater than two (Fig. 2). The identification of additional differentially expressed transcripts, even if subtle, increases the sensitivity in identifying modulated pathways that can impact biological interpretation of data. Here, gene expression changes were mapped to the fatty acid ß-oxidation pathway, a pathway that all of these compounds are known to affect via their interactions with the PPAR{alpha} receptor (54). It was clear from this exercise that the pooled vs. individual analysis gave the same result from a high-level view; i.e., expected effects on fatty acid oxidation were evident for both approaches. Still, some important differences were evident using this comparative analysis. For example, the individual analysis of VPA detected a change in the long-chain acyl-CoA-dehydrogenase mRNA where the pooled analysis did not. This enzyme is inhibited by both VPA and its active metabolite (25); thus an important piece of mechanistic information about VPA was missed by pooling. Further discrepancies between the methods were apparent upon analysis of additional pathways. Individual analysis identified more pathways and had greater representation within pathways than the pooled analysis. Ribosomal (DEHP), apoptosis (DEHP), proteosomal degradation (DEHP), oxidative stress (VPA), and cell cycle (VPA) transcripts were identified only in the individual analysis. Ribosomal protein transcripts are modulated in stress response and tissue regeneration (27, 31, 50). Although only DEHP modulated more ribosomal protein transcripts than expected by chance, individual analysis of VPA and CLO also detected changes in 12 and 3 of these transcripts, respectively. Activation of the ribosomal transcripts modulated by these three compounds have been associated with hepatocarcinogenesis in the rat (8, 9), an interesting finding given that PPAR{alpha} agonists are nongenotoxic carcinogens in rodents. The implications of elevated proteasomal transcripts are not clear; however, regulation by all compounds of a family of transcripts indicates a treatment-related effect. DEHP toxicity has been shown to involve apoptosis(20), while VPA has been shown to induce oxidative stress in rats (41). Clearly, additional insight into these toxicants' mechanisms of action was gained utilizing the individual approach.

Subtle changes in transcript abundance identified by statistical analysis may also be biologically interesting, since a number of laboratories have demonstrated discordance between mRNA and protein levels (5, 10, 17, 19, 45). Subtle changes in mRNA abundance may relate to a dramatic impact on protein activity that may be missed without the sensitivity enabled by statistical methods. Transcription factors such as nuclear hormone receptors represent such a class of genes. Another key benefit of individual analysis is that it enables the correlation of each animal's microarray results to phenotypes measured within an experiment. Variability in phenotype severity across multiple animals treated identically often complicates correlation analysis, and pooling samples does not allow discrimination between responders and nonresponders. In this study, several transcript changes correlated to the PPAR{alpha}-induced increase in liver weight (Fig. 4). The high correlation observed for retinol dehydrogenase was more a result of change in expression in vehicle animals rather than in treated animals, indicating that this was not a compound effect. This was not the case for acyl-CoA-synthetase, where the high correlation supports a treatment-related effect. Regardless of the interpretation, this analysis illustrated the utility of correlating changes in a given transcript with a given phenotype. Correlation between transcript abundance and phenotype can be useful to identify cause-and-effect relationships or in the identification of novel biomarkers. For example, within standard toxicology live phase experiments, serum clinical chemistry parameters are measured and histopathology observations are routinely captured. Understanding the correlation between clinical chemistry and gene expression could lead to additional markers that provide mechanistic information regarding the type of injury.

Clustering algorithms are often used to classify expression patterns as similar or different. In both PCA and HCS, the pooled samples fell into clusters with their corresponding individual animal samples. Waring et al. (44) also demonstrated that pooling samples did not alter their ability to use unsupervised clustering methods to compare overall sample similarity. Given that significant numbers of transcripts were missed with the pooled approach, it would appear that clustering methods are insensitive to subtle changes in a large population of transcript changes. However, there were apparently enough significant changes in the pooled data to drive the algorithm to a similar clustering of treatments. Interestingly, VPA and DEHP clustered more tightly together than with CLO regardless of the analysis. There is no evidence that DEHP causes microvesicular steatosis in the literature, but there is data indicating that VPA causes a slight PPAR-type response (16). These data show that clustering does not separate an inhibition of ß-oxidation from upregulation of this pathway. The quality of samples can impact array data, as demonstrated by the larger spread on the PCA chart for some of the CLO samples (treatment and vehicles). Some of these samples had an unacceptable amount of degradation, as assessed from control probes on the microarrays, and could have been excluded from analysis. This difference in sample integrity between CLO and the other compounds may be evident in Figs. 1 and 2, where CLO had fewer transcripts unique to individual analysis relative to the pooled analysis, since a statistical analysis is more sensitive to variability between samples. These observations underscore a potential risk of pooling samples. Just one degraded or contaminated sample can negatively impact the quality of data for a particular pool and potentially confound results.

In summary, while the two approaches to running microarray chips were comparable, pooling saves considerable time and resources at the expense of statistical confidence in the data. The ability to correlate expression data to other data sets and phenotypes can be critical for effective modeling, the building of training sets, or biomarker discovery (7, 30). As has been done with the classification of cancers, the analysis of individual samples is required to provide appropriate power for predictive modeling (4, 40, 46, 47). This is an important issue in toxicology, since microarrays have been heralded for their potential use in building a predictive database of prototypical toxicants (26, 43). As we have seen, the individual analysis revealed subtle changes that affect interpretation of the experiment that were lost in the pooled analysis and important for mechanistic understanding. Although there are dissenting opinions as to the value of pooled or individual experimental designs, an individual approach becomes more valuable where the variation in phenotypic response is large and/or the overall response is subtle. A pooled approach is better employed where the expected response or phenotype is robust and its variation in that response is minimal.


   ACKNOWLEDGMENTS
 
These studies were done as part of the Consortium on Metabonomic Toxicology (COMET).


   FOOTNOTES
 
Address for reprint requests and other correspondence: S. T. Estrem, Lilly Research Laboratories, Greenfield, IN 46140 (e-mail: stestrem{at}lilly.com)

Article published online before print. See web site for date of publication (http://physiolgenomics.physiology.org).

10.1152/physiolgenomics.00260.2004

1 The Supplemental Material for this article is available online at http://physiolgenomics.physiology.org/cgi/content/full/00260.2004/DC1. The Supplemental Material contains 5 files that are available for downloading. An experimental design summary table is provided, DesignTable.doc. The raw microarray data for this manuscript is contained in the file, "Fold-change data.zip." This file contains the complete statistical analysis of the individual data for all probe sets with a P value < 0.05 as well as the MAS5 output for the pooled samples. A more detailed protocol for the QRT-PCR analysis is included in the file, "Detailed QRT-PCR protocol.doc;" the primers that were used for these reactions can be found in the file, "QRT-PCR primer sequences.xls." Additional figures for Fig. 2 containing Venn diagrams at all 4 time points can be found in the file, "Figure 2 Venn diagrams.PDF." There are also additional figures for Fig. 5 overlaying the effect of all 3 compounds on mitochondrial fatty acid ß-oxidation, contained in the file, "Figure 5.pdf." Back


   REFERENCES
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 

  1. Affymetrix. GeneChip Expression Analysis Technical Manual, http://www.affymetrix.com/support/technical/manual/expression_manual.affx (accessed Sept. 10, 2002).
  2. Ambroise C and McLachlan GJ. Selection bias in gene extraction on the basis of microarray gene-expression data. Proc Natl Acad Sci USA 99: 6562–6566, 2002.
  3. Baker VA, Harries HM, Waring JF, Duggan CM, Ni HA, Jolly RA, Yoon LW, De Souza AT, Schmid JE, Brown RH, Ulrich RG, and Rockett JC. Clofibrate-induced gene expression changes in rat liver: a cross-laboratory analysis using membrane cDNA arrays. Environ Health Perspect 112: 428–438, 2004.
  4. Califano A, Stolovitzky G, and Tu Y. Analysis of gene expression microarrays for phenotype classification. Proc Int Conf Intell Syst Mol Biol 8: 75–85, 2000.
  5. Chen G, Gharib TG, Huang CC, Taylor JM, Misek DE, Kardia SL, Giordano TJ, Iannettoni MD, Orringer MB, Hanash SM, and Beer DG. Discordant protein and mRNA expression in lung adenocarcinomas. Mol Cell Proteomics 1: 304–313, 2002.
  6. Chen WF, Huang MH, Tzang CH, Yang M, and Wong MS. Inhibitory actions of genistein in human breast cancer (MCF-7) cells. Biochim Biophys Acta 1638: 187–196, 2003.
  7. Clarke PA, te Poele R, Wooster R, and Workman P. Gene expression microarray analysis in cancer biology, pharmacology, and drug development: progress and potential. Biochem Pharmacol 62: 1311–1336, 2001.
  8. Comolli R, Alberti P, and Frigerio M. Activation of a ribosomal protein S6 kinase during the 4-dimethylaminoazobenzene-induced rat hepatocarcinogenesis. Cancer Lett 41: 281–285, 1988.
  9. Comolli R, Frigerio M, and Alberti P. Heat shock, protein synthesis and ribosomal protein S6 phosphorylation in vitro in Yoshida AH 130 ascites hepatoma cells. Cell Biol Int Rep 12: 907–917, 1988.
  10. Corbin RW, Paliy O, Yang F, Shabanowitz J, Platt M, Lyons CE Jr, Root K, McAuliffe J, Jordan MI, Kustu S, Soupene E, and Hunt DF. Toward a protein profile of Escherichia coli: comparison to its transcription profile. Proc Natl Acad Sci USA 100: 9232–9237, 2003.
  11. Dahlquist KD, Salomonis N, Vranizan K, Lawlor SC, and Conklin BR. GenMAPP, a new tool for viewing and analyzing microarray data on biological pathways. Nat Genet 31: 19–20, 2002.
  12. Doniger SW, Salomonis N, Dahlquist KD, Vranizan K, Lawlor SC, and Conklin BR. MAPPFinder: using Gene Ontology and GenMAPP to create a global gene-expression profile from microarray data. Genome Biol 4: R7, 2003.
  13. Duyk GM. Sharper tools and simpler methods. Nat Genet 32,Suppl: 465–468, 2002.
  14. Finkelstein D, Retief J, Grass P, and Staedtler F. The effects of pooling RNA samples in a DNA micro array toxicogenomics study with Clofibric acid in mice. Toxicol Sci 72: S1, 2003.
  15. Gasch AP, Spellman PT, Kao CM, Carmel-Harel O, Eisen MB, Storz G, Botstein D, and Brown PO. Genomic expression programs in the response of yeast cells to environmental changes. Mol Biol Cell 11: 4241–4257, 2000.
  16. Gonzalez FJ, Peters JM, and Cattley RC. Mechanism of action of the nongenotoxic peroxisome proliferators: role of the peroxisome proliferator-activator receptor alpha. J Natl Cancer Inst 90: 1702–1709, 1998.
  17. Griffin TJ, Gygi SP, Ideker T, Rist B, Eng J, Hood L, and Aebersold R. Complementary profiling of gene expression at the transcriptome and proteome levels in Saccharomyces cerevisiae. Mol Cell Proteomics 1: 323–333, 2002.
  18. Gunther EC, Stone DJ, Gerwien RW, Bento P, and Heyes MP. Prediction of clinical drug efficacy by classification of drug-induced genomic expression profiles in vitro. Proc Natl Acad Sci USA 100: 9608–9613, 2003.
  19. Gygi SP, Rochon Y, Franza BR, and Aebersold R. Correlation between protein and mRNA abundance in yeast. Mol Cell Biol 19: 1720–1730, 1999.
  20. Hasmall S, Orphanides G, James N, Pennie W, Hedley K, Soames A, Kimber I, and Roberts R. Downregulation of lactoferrin by PPAR{alpha} ligands: role in perturbation of hepatocyte proliferation and apoptosis. Toxicol Sci 68: 304–313, 2002.
  21. Jimenez-Rodriguezvila M, Caro-Paton A, Duenas-Laita A, Conde M, Coca MC, Martin-Lorente JL, Velasco A, and Maranon A. Histological, ultrastructural and mitochondrial oxidative phosphorylation studies in liver of rats chronically treated with oral valproic acid. J Hepatol 1: 453–465, 1985.
  22. Jolly RA, Waring JF, Ciurlionis R, Ulrich RG, Morfitt D, Helgren M, and Patterson R. Microvesicular steatosis induced by a short chain fatty acid: effects on mitochondrial function and correlation with gene expression. Toxicol Pathol 32: 1–7, 2004.
  23. Kendziorski C, Irizarry RA, Chen KS, Haag JD, and Gould MN. On the utility of pooling biological samples in microarray experiments. Proc Natl Acad Sci USA 102: 4252–4257, 2005.
  24. Kendziorski CM, Zhang Y, Lan H, and Attie AD. The efficiency of pooling mRNA in microarray experiments. Biostatistics 4: 465–477, 2003.
  25. Kibayashi M, Nagao M, and Chiba S. Influence of valproic acid on the expression of various acyl-CoA dehydrogenases in rats. Pediatr Int 41: 52–60, 1999.
  26. Kramer JA and Kolaja KL. Toxicogenomics: an opportunity to optimise drug development and safety evaluation. Expert Opin Drug Saf 1: 275–286, 2002.
  27. Laine RO, Shay NF, and Kilberg MS. Nuclear retention of the induced mRNA following amino acid-dependent transcriptional regulation of mammalian ribosomal proteins L17 and S25. J Biol Chem 269: 9693–9697, 1994.
  28. Lee ML, Kuo FC, Whitmore GA, and Sklar J. Importance of replication in microarray gene expression studies: statistical methods and evidence from repetitive cDNA hybridizations. Proc Natl Acad Sci USA 97: 9834–9839, 2000.
  29. Lindon JC, Nicholson JK, Holmes E, Antti H, Bollard ME, Keun H, Beckonert O, Ebbels TM, Reily MD, Robertson D, Stevens GJ, Luke P, Breau AP, Cantor GH, Bible RH, Niederhauser U, Senn H, Schlotterbeck G, Sidelmann UG, Laursen SM, Tymiak A, Car BD, Lehman-McKeeman L, Colet JM, Loukaci A, and Thomas C. Contemporary issues in toxicology the role of metabonomics in toxicology and its evaluation by the COMET project. Toxicol Appl Pharmacol 187: 137–146, 2003.
  30. Macgregor PF. Gene expression in cancer: the application of microarrays. Expert Rev Mol Diagn 3: 185–200, 2003.
  31. Natarajan K, Meyer MR, Jackson BM, Slade D, Roberts C, Hinnebusch AG, and Marton MJ. Transcriptional profiling shows that Gcn4p is a master regulator of gene expression during amino acid starvation in yeast. Mol Cell Biol 21: 4347–4368, 2001.
  32. Peng X, Wood CL, Blalock EM, Chen KC, Landfield PW, and Stromberg AJ. Statistical implications of pooling NA samples for microarray experiments. BMC Bioinformatics 4: 26, 2003.
  33. Ponchaut S, Draye JP, Veitch K, and Van Hoof F. Influence of chronic administration of valproate on ultrastructure and enzyme content of peroxisomes in rat liver and kidney. Oxidation of valproate by liver peroxisomes. Biochem Pharmacol 41: 1419–1428, 1991.
  34. Ryan TP and Searfoss GH. Finding toxicology biomarkers using functional genomics. Am Pharmaceut Rev.In press.
  35. Searfoss GH, Jordan WH, Calligaro DO, Galbreath EJ, Schirtzinger LM, Berridge BR, Gao H, Higgins MA, May PC, and Ryan TP. Adipsin, a biomarker of gastrointestinal toxicity mediated by a functional {gamma}-secretase inhibitor. J Biol Chem 278: 46107–46116, 2003.
  36. Simon R, Radmacher MD, Dobbin K, and McShane LM. Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J Natl Cancer Inst 95: 14–18, 2003.
  37. Smith MW, Yue ZN, Geiss GK, Sadovnikova NY, Carter VS, Boix L, Lazaro CA, Rosenberg GB, Bumgarner RE, Fausto N, Bruix J, and Katze MG. Identification of novel tumor markers in hepatitis C virus-associated hepatocellular carcinoma. Cancer Res 63: 859–864, 2003.
  38. Sobaniec-Lotowska ME. Effects of long-term administration of the antiepileptic drug–sodium valproate upon the ultrastructure of hepatocytes in rats. Exp Toxicol Pathol 49: 225–232, 1997.
  39. Stiemer B. Morphological evaluation of steatosis in monolayer cultures (MDCK cells) after treatment with gentamicin and valproic acid. Histol Histopathol 4: 421–428, 1989.
  40. Tay ST, Leong SH, Yu K, Aggarwal A, Tan SY, Lee CH, Wong K, Visvanathan J, Lim D, Wong WK, Soo KC, Kon OL, and Tan P. A combined comparative genomic hybridization and expression microarray analysis of gastric cancer reveals novel molecular subtypes. Cancer Res 63: 3309–3316, 2003.
  41. Tong V, Chang TK, Chen J, and Abbott FS. The effect of valproic acid on hepatic and plasma levels of 15-F2t-isoprostane in rats. Free Radic Biol Med 34: 1435–1446, 2003.
  42. Waring JF, Gum R, Morfitt D, Jolly RA, Ciurlionis R, Heindel M, Gallenberg L, Buratto B, and Ulrich RG. Identifying toxic mechanisms using DNA microarrays: evidence that an experimental inhibitor of cell adhesion molecule expression signals through the aryl hydrocarbon nuclear receptor. Toxicology 181–182: 537–550, 2002.
  43. Waring JF and Halbert DN. The promise of toxicogenomics. Curr Opin Mol Ther 4: 229–235, 2002.
  44. Waring JF, Jolly RA, Ciurlionis R, Lum PY, Praestgaard JT, Morfitt DC, Buratto B, Roberts C, Schadt E, and Ulrich RG. Clustering of hepatotoxins based on mechanism of toxicity using gene expression profiles. Toxicol Appl Pharmacol 175: 28–42, 2001.
  45. Washburn MP, Koller A, Oshiro G, Ulaszek RR, Plouffe D, Deciu C, Winzeler E, and Yates JR 3rd. Protein pathway and complex clustering of correlated mRNA and protein expression analyses in Saccharomyces cerevisiae. Proc Natl Acad Sci USA 100: 3107–3112, 2003.
  46. West M, Blanchette C, Dressman H, Huang E, Ishida S, Spang R, Zuzan H, Olson JA Jr, Marks JR, and Nevins JR. Predicting the clinical status of human breast cancer by using gene expression profiles. Proc Natl Acad Sci USA 98: 11462–11467, 2001.
  47. Wilson KS, Roberts H, Leek R, Harris AL, and Geradts J. Differential gene expression patterns in HER2/neu-positive and -negative breast cancer cell lines and tissues. Am J Pathol 161: 1171–1185, 2002.
  48. Wong JS and Gill SS. Gene expression changes induced in mouse liver by di(2-ethylhexyl) phthalate. Toxicol Appl Pharmacol 185: 180–196, 2002.
  49. Xiang Z, Yang Y, Ma X, and Ding W. Microarray expression profiling: analysis and applications. Curr Opin Drug Discov Devel 6: 384–395, 2003.
  50. Xu W, Wang S, Wang G, Wei H, He F, and Yang X. Identification and characterization of differentially expressed genes in the early response phase during liver regeneration. Biochem Biophys Res Commun 278: 318–325, 2000.
  51. Yamamoto R, Yanagita T, Kobayashi H, Yokoo H, and Wada A. Up-regulation of sodium channel subunit mRNAs and their cell surface expression by antiepileptic valproic acid: activation of calcium channel and catecholamine secretion in adrenal chromaffin cells. J Neurochem 68: 1655–1662, 1997.
  52. Yang YH and Speed T. Design issues for cDNA microarray experiments. Nat Rev Genet 3: 579–588, 2002.
  53. Yeung KY and Ruzzo WL. Principal component analysis for clustering gene expression data. Bioinformatics 17: 763–774, 2001.
  54. Yu S, Rao S, and Reddy JK. Peroxisome proliferator-activated receptors, fatty acid oxidation, steatohepatitis and hepatocarcinogenesis. Curr Mol Med 3: 561–572, 2003.



This article has been cited by other articles:


Home page
 Brief Funct Genomic ProteomicHome page
J. N. McGuire, J. Overgaard, and F. Pociot
Mass spectrometry is only one piece of the puzzle in clinical proteomics
Brief Funct Genomic Proteomic, February 28, 2008; (2008) eln005v1.
[Abstract] [Full Text] [PDF]

Home page
 Toxicol SciHome page
Y. Guo, R. A. Jolly, B. W. Halstead, T. K. Baker, J. P. Stutz, M. Huffman, J. N. Calley, A. West, H. Gao, G. H. Searfoss, et al.
Underlying Mechanisms of Pharmacology and Toxicity of a Novel PPAR Agonist Revealed Using Rodent and Canine Hepatocytes
Toxicol. Sci., April 1, 2007; 96(2): 294 - 309.
[Abstract] [Full Text] [PDF]

Home page
 Toxicol SciHome page
H. Wintz, L. J. Yoo, A. Loguinov, Y.-Y. Wu, J. A. Steevens, R. D. Holland, R. D. Beger, E. J. Perkins, O. Hughes, and C. D. Vulpe
Gene Expression Profiles in Fathead Minnow Exposed to 2,4-DNT: Correlation with Toxicity in Mammals
Toxicol. Sci., November 1, 2006; 94(1): 71 - 82.
[Abstract] [Full Text] [PDF]

Home page
 Physiol. GenomicsHome page
E. van Lunteren, M. Moyer, and P. Leahy
Gene expression profiling of diaphragm muscle in {alpha}2-laminin (merosin)-deficient dy/dy dystrophic mice
Physiol Genomics, March 13, 2006; 25(1): 85 - 95.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
S.-D. Zhang and T. W. Gant
Effect of pooling samples on the efficiency of comparative studies using microarrays
Bioinformatics, December 15, 2005; 21(24): 4378 - 4383.
[Abstract] [Full Text] [PDF]

Home page
 Physiol. GenomicsHome page
M. Liang and B. Ventura
Physiological genomics in PG and beyond: July to September 2005
Physiol Genomics, October 17, 2005; 23(2): 119 - 124.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Supplemental Material
Right arrow All Versions of this Article:
22/3/346    most recent
00260.2004v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (18)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Jolly, R. A.
Right arrow Articles by Estrem, S. T.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Jolly, R. A.
Right arrow Articles by Estrem, S. T.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
Visit Other APS Journals Online
Copyright © 2005 by the American Physiological Society.