QUALITY-ASSURANCE RESULTS FOR ROUTINE WATER ANALYSES IN
U.S. GEOLOGICAL SURVEY LABORATORIES, WATER YEAR 1997
by Amy S. Ludtke, Mark T. Woodworth, and Philip S. Marsh link to Department of the Interior homepage
U.S. GEOLOGICAL SURVEY
Water-Resources Investigations Report 99-4057
CONTENTS LIST OF FIGURES

TABLES

ABBREVIATIONS

ABSTRACT

The U.S. Geological Survey operates a quality-assurance program based on the analyses of reference samples for the National Water Quality Laboratory in Arvada, Colorado, and the Quality of Water Service Unit in Ocala, Florida. Reference samples that contain selected inorganic, nutrient, and low ionic-strength constituents are prepared and submitted to the laboratory as disguised routine samples. The program goal is to estimate precision and bias for as many analytical methods offered by the participating laboratories as possible. Blind reference samples typically are submitted at a rate of 2 to 5 percent of the annual environmental-sample load for each constituent. The samples are distributed to the laboratories throughout the year. The reference samples are subject to the identical laboratory handling, processing, and analytical procedures as those applied to environmental samples and, therefore, have been used as an independent source to verify bias and precision of laboratory analytical methods and ambient water-quality measurements. The results are stored permanently in the National Water Information System and the Blind Sample Projects data base. During water year 1997, 95 analytical procedures were evaluated at the National Water Quality Laboratory and 51 analytical procedures were evaluated at the Quality of Water Service Unit.

An overall evaluation of the inorganic and low ionic-strength constituent data for water year 1997 indicated analytical imprecision in the National Water Quality Laboratory for 6 of 80 analytical procedures. These procedures were arsenic (dissolved, hydride generation-atomic absorption spectrophotometry); arsenic (whole-water recoverable, hydride generation-atomic absorption spectrophotometry); cadmium (dissolved, inductively coupled plasma-atomic emission spectrometry); lead (dissolved, inductively coupled plasma-atomic emission spectrometry); mercury (whole-water recoverable, cold vapor-atomic absorption spectrophotometry); and silver (dissolved, inductively coupled plasma-atomic emission spectrometry). Two of 80 analytical procedures showed bias throughout the range of reference samples: aluminum (dissolved, inductively coupled plasma-atomic emission spectrometry) and vanadium (dissolved, colorimetric).

At the National Water Quality Laboratory during water year 1997, lack of precision was indicated for 2 of 15 nutrient procedures: nitrate plus nitrite as nitrogen (dissolved, colorimetric, low ionic-strength) and phosphorus (whole-water recoverable, micro-Kjeldahl digestion, automated segmented flow, colorimetric). Bias was indicated on very few nutrient samples.

One of the 51 analytical procedures tested at the Quality of Water Service Unit during water year 1997 indicated analytical imprecision for strontium (dissolved, inductively coupled plasma-atomic emission spectrometry, trace).

Positive analytical bias throughout the reference sample range was indicated for aluminum (whole-water recoverable, inductively coupled plasma-atomic emission spectrometry), and a negative analytical bias throughout the reference sample range was indicated for barium (whole-water recoverable, inductively coupled plasma-atomic emission spectrometry). As with the National Water Quality Laboratory, the Quality of Water Service Unit had very few nutrient sample biases.

INTRODUCTION

The Water Resources Division (WRD) of the U.S. Geological Survey (USGS) performs numerous hydrologic investigations that require analyses of water for inorganic, nutrient, and low ionic- strength constituents. The National Water Quality Laboratory (NWQL) in Arvada, Colorado, and the Quality of Water Service Unit (QWSU) laboratory in Ocala, Florida, are the primary sources of analytical services for many of these hydrologic investigations. The NWQL provides analytical services for all national programs conducted by the USGS and is also used by district offices throughout the Nation for local and regional programs. The QWSU provides analytical services to USGS district offices in the Southeastern United States.

This report describes the results of a quality-assurance program used to monitor the quality of analytical inorganic, nutrient, and low ionic-strength procedures at the NWQL and inorganic and nutrient procedures at the QWSU. Previous reports (Peart and Thomas, 1983a, 1983b, 1984; Peart and Sutphin, 1987; Lucey and Peart, 1988, 1989a, 1989b; Lucey, 1989; Maloney and others, 1992, 1993, 1994) document results from February 1981 through September 1991. Some of the previous reports contain quality-assurance information for organic determinations.

The water year used by the USGS is the 12 months from October 1 through September 30 and is identified by the calendar year in which the water year ends. During water year 1997, 95 analytical procedures were evaluated for the NWQL by the Blind Sample Project (BSP). This represents about 80 percent of the inorganic and nutrient analytical procedures offered by the laboratory for water-matrix determinations. The remaining 20 percent of the water-matrix determinations offered were either infrequently requested, so that it was not feasible to include them in the BSP, or the constituents did not have a readily available, stable source of reference material. In addition, the NWQL offers analytical determinations for sample matrices other than water, such as biological tissues and sediments.

The BSP evaluated 51 analytical procedures for the QWSU during water year 1997. Although this represents only about 70 percent of the total number of water-matrix constituents offered for analysis, the remaining 30 percent were considered to be custom determinations that were infrequently requested. In addition, some of the constituents did not have a readily available, stable source of reference material.

The analytes included in the 1997 NWQL and QWSU BSP are listed below by constituent categories. Inorganic constituents determined by methods adapted to detect low-level concentrations are included in the low ionic-strength category. Laboratory users who require analyses for water-matrix constituents not offered through the BSP, or any nonwater-matrix determinations, should consider alternative procedures to measure analytical quality.

Nutrient constituents: (NWQL and QWSU BSPs) ammonia as nitrogen, ammonia plus organic nitrogen as nitrogen, nitrate plus nitrite as nitrogen, orthophosphate as phosphorus, and phosphorus.

Inorganic constituents: (NWQL and QWSU BSPs) alkalinity, aluminum, arsenic, barium, cadmium, calcium, chloride, chromium, copper, dissolved solids (residue on evaporation at 180oC), fluoride, iron, lead, magnesium, manganese, mercury, nickel, potassium, selenium, silica, silver, sodium, strontium, sulfate, and zinc. Constituents included in the NWQL BSP only were antimony, beryllium, boron, cobalt, lithium, molybdenum, and vanadium.

Low ionic-strength constituents: (NWQL BSP only) ammonia as nitrogen, chloride, fluoride, magnesium, nitrate plus nitrite as nitrogen, orthophosphate as phosphorus, potassium, sodium, specific conductance, and sulfate.

The authors of this report would like to acknowledge several individuals whose contributions greatly benefited this report. Edward J. Gilroy, statistician, USGS-retired, and Thomas J. Maloney, NWQL, were both instrumental in developing the statistical test used for bias throughout this report.

PROGRAM DESCRIPTION

Standard reference water samples (SRWS) (Skougstad and Fishman, 1975; Schroder and others, 1980; Janzer, 1985; Farrar, 1998) are used to prepare samples for the BSP. The SRWS are used undiluted, diluted with deionized water, or mixed in varying proportions with other SRWS. This sample-mixing procedure produces a large number of unique samples available for quality-assurance purposes.

Reference samples for the BSP are made to appear as much like environmental samples as possible to reduce the possibility that analysts will recognize them as quality-assurance samples. Analytical request forms are completed to ensure that appropriate analyses have been requested for the samples. For the NWQL program, samples are delivered by BSP personnel directly to the sample login unit of the laboratory. Because of the great number of samples delivered to the NWQL and because the login unit is separated from the analytical operations, there is little chance that the origin of these quality-assurance samples will be detected. Because the QWSU operation is much smaller and the analysts have direct contact with the sample login operations, however, the samples and forms are sent to selected USGS offices that use that laboratory. Throughout the year, these offices send the quality-assurance samples for the BSP to the QWSU with their regular environmental samples.

The BSP samples are subjected to identical laboratory handling, processing, and analytical procedures as the environmental samples. After laboratory analysis, BSP personnel compile and review the analytical results. The resulting data are stored in a National Water Information System (NWIS) data base and the BSPs SASã data base.

The SRWS are filtered during preparation; therefore, all constituents in the Blind Reference Samples are in the dissolved phase. Constituents that are designated as whole-water recoverable in this report are from filtered reference samples that have undergone a digestion process (Fishman and Friedman, 1989, p. 87-88) rather than from unfiltered or whole-water samples. Differences that appear in this report between the dissolved (DIS) analyses and the whole-water recoverable (WWR) analyses will be due to the digestion process rather than any difference in the sample phase.

Estimation of Sample Loads

The number of quality-assurance determinations requested for each analytical procedure is proportional to the number of requests for the procedure from all environmental samples submitted. Because both laboratories have active quality-control programs, the BSP followed the guidelines of Friedman and Erdmann (1982) to set submission of these external quality-assurance samples at a rate of about 2 to 5 percent of the laboratory work for each analytical procedure. The annual workload for each analytical procedure is estimated from sample login records for the previous year. The estimate is determined by taking the difference of the total number of login records for the procedure minus the number of samples submitted by the BSP and the laboratories’ quality-control program.

Comparison of Results with the National Water Information System Water-Quality Data Bases

Analytical results for environmental samples are stored in the USGS National Water Information System (NWIS) data base. The NWIS is a national standardized data-base system that is maintained by each WRD office nationwide. The NWIS allows users in those offices to enter, review, update, and retrieve analytical results that pertain to the field area the office serves.

The assessments presented in this report are based on analytical results released from the laboratories that have the same level of quality-control review as the data released to each WRD office. The results presented in the report, however, provide a conservative estimate of the quality of the data stored in individual NWIS data bases because water-quality specialists and project chiefs are expected to scrutinize analytical results for discrepancies, request reruns for questionable results, and update analytical results in the NWIS data base as appropriate.

Factors that need to be considered for interpretation of results stored in NWIS data bases with relation to the results presented in this report include the following:

  1. No effort was made to correct nonanalytical errors, even when it was obvious which corrective measures were appropriate. The data are presented as originally produced by the laboratories. Nonanalytical errors include sample login errors, transcription errors by the analyst, data-transmission errors by laboratory instruments, and manual data-entry errors. Therefore, a data reviewer that detects nonanalytical errors can make corrections to improve the quality of the NWIS data base. For example, two samples from different sites are submitted to a laboratory on the same day and are misidentified by the laboratory in a way that the analytical data reported for one actually belongs to the other. A data reviewer familiar with one of the sites or its historical data usually could detect the problem and make the necessary corrections.
  2. Dilution factors that were incorrectly applied account for some analytical errors. Sample dilutions are routinely made in the laboratory to bring sample concentrations into analytical calibration range. If the dilution factor is not applied or is applied incorrectly, the reported value will be in error. For example, if a nutrient sample has a phosphorus concentration of 1.6 mg/L and an analysis is reported at 0.16 mg/L, a tenfold dilution may have been used and not applied to the final result. These errors are difficult to identify unless historical data for a sampling site are available for comparison.
  3. Determinations that exceed control limits, set at ± 2 standard deviations of the reference-sample concentration, are typically submitted for reanalysis by the BSP. The purpose of these rerun requests is to identify reference samples that may be deteriorating or that may have been incorrectly bottled. If the BSP determines that the source of error was deterioration of the sample or a sample mixup that occurred before the sample was delivered to the laboratory, then the data are purged from the quality-assurance data base. The majority of analysis reruns indicate the source of error to be problems associated with laboratory operations, such as analytical errors, internal bottle mixups, or data-transmission errors. If a laboratory problem is indicated by the analysis rerun, the original data remain in the quality-assurance data base. If the quality-control section of the laboratory independently identifies problems with an analytical determination and requires that updated analyses be released, then the quality-assurance data base is updated with the new values.
  4. Control charts included in this report may be used to determine analytical conditions at any given time for water year 1997. A chart may show an analytical process to be out of statistical control for a short period of time, but in statistical control for most of the year. The data for the short period may affect the statistical tests for the entire year such that they would indicate analytical imprecision or significant bias. The data for the period when the analytical process was in statistical control can be considered separately to evaluate precision and bias. An interactive quality-assurance data base is available for the retrieval and assessment of blind quality-control sample analytical results. Currently, the BSP data-base system contains more than 170,000 analyses dating from October 1984 to present (1999). New analytical data released from the laboratories are added weekly to the data base. Data retrievals can be customized to document the laboratories’ analytical bias and variability relative to the time period, analytical procedures, and concentration ranges of individual water-quality programs. You can find step-by-step instructions for accessing the BSP data base on the internet at: https://bqs.usgs.gov/bsp/qadatanew.htm.
STATISTICAL EVALUATION

Control Chart Development and Evaluation

The SRWS used in the BSP are analyzed through a round-robin evaluation program described by Farrar (1998). A statistical summary of the round-robin results is prepared for each set of samples. The SRWS Project uses median and F-pseudosigma in the summary reports as a measure of the known value and variability about the known value. The median data reported in the SRWS summaries have been referred to as most probable value (MPV). The F-pseudosigma ( Fs) is defined by Hoaglin and others (1983) as:

Formula 1(1)
where, the Fourth-spread is analogous to the interquartile range of the data. In addition, they show that the F-pseudosigma yields an unbiased estimate of standard deviation when the data distribution is Gaussian. However, it is recognized that some of the SRWS Project data sets may not have Gaussian data distributions. For non-Gaussian distributions, the F-pseudosigma is still a good estimate of spread and is not unduly influenced by outliers.

The SRWS Project MPVs are used to estimate the BSP mix concentrations that are based on the proportion of the SRWS used. Likewise, if deionized water is used in the preparation of a blind sample mix, then a sample concentration of zero is applied to estimate the resultant MPV, that is based on the proportion used. The resulting MPVs are used in the BSP to compare with analytical results.

For each SRWS constituent, a regression equation was used to estimate the Fs over a continuous range of concentrations. The equations were derived by using ordinary least-squares and regressing the F-pseudosigma against the MPV. The ordinary least-squares regression equation derived for sodium is displayed in figure 1.

Helsel and Hirsch (1992) provide a general model for estimation of ordinary least-squares regression,

Fsi = b0 + b1MPVi + eii = 1,2,…,n(2)
whereFsi=the ith observation of the response variable, Fs,
MPVi=the ith observation of the explanatory variable, Most Probable Value,
b0=the intercept,
b1=the slope,
ei=the random error or residual for the ith observation, and
n=the number of samples.

The SRWS Project summary data for semiannual round-robin sample studies conducted during the last 7 years were used to derive regression equations for each analyte. The concentration range of SRWS samples used to derive the regression equations, the units of measurement, the derived Fs equation, the number of samples used to derive the equation, and the p-value indicating the level of significance associated with the derived equation are presented in table 1. P-values greater than 0.05 indicate there is less than a 5-percent chance that such a relationship would exist.

    Table 1 -- Ordinary least-squared equations for determining the most probable deviation
The assessment of whether an analytical measurement meets control specifications is based on the number of standard deviations (NSD) that the measured concentration differs from the MPV. In this report, the term standard deviations will be used when comparing individual determinations against the MPV. The NSD is determined by taking the difference between the analytical result and the MPV and dividing by the Fs estimated by the appropriate regression equation using the MPV concentration. Taylor (1987, p. 33-34) provides a general equation that was modified to determine the NSD values for this assessment:
Formula 3(3)
whereNSD=number of standard deviations,
Xq=value in question,
MPV=most probable value, and
Fs=regression estimate of F-pseudosigma.

Analytical results that are within two NSDs of the MPV concentration are considered acceptable; however, there have been instances where the assessment criteria, as defined above, resulted in unrealistic tolerances not related to analytical capability but rather to data-reporting criteria set for computer storage of environmental data. This primarily occurred for results near the reporting limits. The numerical precision for computer storage of environmental data is often much less than that used to calculate the MPVs of the SRWS. For example, the criteria for NWIS data entry for many metals determined by flame-atomic absorption spectrophotometry (F-AAS) is to report values to the nearest 10 mg/L for determinations below 100 mg/L. For the SRWS project round-robin study, however, laboratories typically report data to the nearest 1 mg/L, and the standard deviation for samples with concentrations below 100 mg/L may be only a few micrograms per liter. Because the regression equations used to estimate Fs are based on SRWS data that are reported with more significant figures, especially at lower concentrations.

The rounding differences between the SRWS summary results and the NWIS computer-storage criteria make the assessments at lower sample concentrations less sensitive. As a means to make the assessments more sensitive at lower concentration ranges, a correction factor has been applied to allow at least one reportable value to be within one standard deviation of the MPV. The correction is accomplished by setting a minimum Fs at three-fourths of the minimum reporting limit. For example, a sample may be assigned an MPV for beryllium of 25.2 mg/L; however, the laboratories can only report environmental data for beryllium (WWR, F-AAS) to the nearest 10 mg/L in the NWIS data base. For the reference sample in question, a reported value of 20 or 30 would generally be expected. The regression equation for beryllium [Fs = 0.042MPV + 0.581] provides an estimate that reported Fs values within ± 1.64 mg/L of a sample with an MPV of 25.2 mg/L will be considered within one standard deviation. The laboratories’ closest reportable values, 20 and 30 mg/L, would be –3.17 and +2.93 standard deviations, respectively, from the MPV. When a minimum Fs value of 7.5 mg/L is assigned (three-fourths of the reporting limit), then values reported at 20 and 30 mg/L would be –0.69 and +0.64 standard deviations from the MPV, respectively.

For some determinations, laboratories report data as "less than" the reporting limit. This most frequently happens for samples that have MPVs near the reporting limit. For the purposes of the BSP assessments, these "less than" values have been replaced, using simple substitution, by the reporting limit.

For each constituent, the NSD values were plotted against the date that the samples were logged into the laboratory to prepare control charts. The results for each constituent are presented as control charts, as shown in part A of figures 2 through 96 for the NWQL and in figures 97 through 147 for the QWSU (OCALA) in the Supplemental Data section of this report.

Control charts for nutrient constituents are shown in figures 2 through 16 for the NWQL and figures 97 through 103 for the QWSU. Control charts for inorganic constituent-reference samples are presented in figures 17 through 96 for the NWQL and figures 104 through 147 for the QWSU. The data on control charts have been separated into quartiles on the basis of environmental sample concentrations. All environmental data released during 1997 by the NWQL and QWSU were obtained and used to determine the quartiles. The quartiles and calibration range of each analytical method are identified in the explanation of the associated figure. The MPVs of the reference materials were examined to determine which quartile group they represented. A different symbol was used to represent data points in each of the quartile ranges. For reference samples with MPVs in quartile 1 (up through 25 percent), the (l) symbol has been used; reference samples in quartile 2 (25.1 through 50 percent) are represented by the (l) symbol; quartile 3 (50.1 through 75 percent) by the (l) symbol; and quartile 4 (75.1 through 100 percent) by the (l) symbol.

Points on the control charts that are greater than 6 standard deviations or less than -6 standard deviations have been rounded to 6 and -6, respectively, and are plotted at the top or bottom edge of the figure.

Precision Chart Development and Evaluation

Replicate determinations of reference samples were used to estimate precision. For each sample mixture having at least three determinations, the mean, standard deviation, and relative standard deviation (RSD) were calculated for each constituent. Taylor (1987, p. 20) defines RSD as the coefficient of variation multiplied by 100 (percent). The equation to represent this is:

Formula 4(4)
whereRSD=relative standard deviation;
s=standard deviation; and
X=mean of replicate values reported by the laboratory.

The RSD provides an estimate of error relative to the mean of replicate values reported by the laboratory for each reference sample.

The RSD data are presented graphically as precision data charts in part B of figures 2 through 96 for the NWQL and figures 97 through 147 for the QWSU (OCALA) in the Supplemental Data section of this report. These charts were prepared by plotting the RSD for inorganic, nutrient, and low ionic-strength constituents against the mean concentration of the reference samples reported by the individual laboratories. These charts allow a data reviewer to estimate precision at any concentration shown for a constituent. For example, figure 2B shows precision data for dissolved ammonia determinations from the NWQL. This plot shows a distribution of approximately 42 to 9 percent RSD for concentrations that range from 0.05 to 0.275 mg/L.

To allow the precision charts to be used to estimate expected error of analytical results, outliers were rejected from the data set. Outliers are a rare occurrence in the data reported by the laboratories, accounting for much less than 1 percent of all data. Outliers produce erroneous results in the use of parametric statistics such as RSD. Taylor (1987, p. 33-34), suggested NSD ?|4| as a criterion for rejection of data for a large sample set and NSD ?|6| for smaller data sets or when the standard deviation is not well established. In this report, an outlier was defined as a value greater than 6 standard deviations from the MPV. There were 27 outliers deleted from the entire data set of 10,387 analyses for the RSD procedure. Table 2 for the NWQL and table 3 for the QWSU (OCALA) present the total number of determinations for each constituent processed during the water year, the number of determinations that lie outside the ± 2 standard deviation limits, and the number of determinations that lie outside the ± 6 standard deviation limits.

The precision chart data for inorganic, nutrient, and low ionic-strength constituents have been separated into groups that are based on environmental-sample data quartiles in the same manner as explained earlier in this report for the development of the control charts.

    Table 2 -- Total number of analyses from quality-assurance samples during water year 1997 with number greater than two and six standard deviations from the most probable value for the National Water Quality Laboratory

    Table 3 -- Total number of analyses from quality-assurance samples during water year 1997 with number greater than two and six standard deviations from the most probable value for the Quality of Water Service Unit Laboratory

Binomial-Probability-Distribution Technique To Assess Precision

Measures of precision were determined from the control chart data by applying binomial- probability-distribution procedures described by Friedman, Bradford, and Peart (1983) and by Peart and Thomas (1983a). The precision evaluation is based on whether or not an analytical method could produce results within ± 2 standard deviations of the MPV. The binomial equation identifies the maximum number of determinations that could exceed the control limit at a 0.01 significance level. A comparison is then made between the number of analytical determinations that exceed control limits and the results of the binomial-probability-distribution equation for the total number of analytical determinations.

Formula 5(5)
whereP(x)=probability of having x or more points greater than two standard deviations,
N=number of successive points, and
i=number of points greater than two standard deviations.

Analytical procedures exhibit imprecision if they have more determinations outside the control limits than the result predicted by the binomial equation. The binomial-probability-distribution procedure to measure precision allows tracking of annual variations in the precision of analytical measurements. The binomial test can be used to evaluate analytical results for short periods that appear to indicate imprecision, but the test loses power as fewer total analytical determinations are used in the evaluation.

Wilcoxon Signed-Rank Procedure To Assess Bias

The Wilcoxon signed-rank test (Helsel and Hirsch, 1992) was used to determine whether the median difference between the laboratory determinations and the MPV of the reference samples equaled zero. An assessment of bias was made for each analyte and mix combination that had three or more observations. A bias condition was assigned if the p-value of the test was less than 0.05 and the median difference was greater than the reporting level used for the analyte concentration in the mix. Whether bias is negative or positive was determined on the basis of the sign of the median difference. Data users are urged to consider the concentration level and review the magnitude of bias, which is based on the median differences, to evaluate whether a bias is of any practical concern for their particular situation.

There are two important factors to consider when analyzing analytical results for bias: the first is the number of observations in the subsample, and the second is the magnitude of the bias. If there are few observations in the subsample, the median difference has to be large to attain a p-value that indicates a biased condition. On the other hand, if there are many observations, a small median difference may result in a p-value that indicates bias.

The second factor used for the bias assessment is the bias magnitude. The magnitude of the bias is defined as the difference in concentration between the reported values from the laboratory and the MPV. The magnitude must be greater than one reporting unit for that MPV for the results to be considered biased. For example, 34 barium (DIS, ICP-AES) determinations of a sample having an MPV of 66.4 mg/L resulted in a median difference of -0.6 mg/L from the MPV. It is important to consider that at a concentration of 66.4 mg/L, a laboratory would typically report data only to the nearest 1 mg/L. The signed-rank test provided a p-value of 0.00143, which is considerably less than the 0.05 value. If the reporting unit was not considered, this result would have been considered biased negative, even though the laboratory was reporting the sample to the best of its reporting ability. Because the median difference of -0.6 mg/L is less than the reporting level of 1 mg/L, the magnitude of the bias is less than the implied accuracy of the analysis; therefore, analytical bias that is based strictly on the p-value has no practical meaning.

QUALITY-ASSURANCE DATA FOR NUTRIENT-CONSTITUENT SAMPLES

Precision

The results of statistical testing for analytical precision for each nutrient constituent are presented in table 4 for the NWQL and table 5 for the QWSU. These tables show either acceptable results (indicated by --) or a statistically significant lack of precision (LOP) at a significance level of 0.01 for each constituent. NWQL data from water year 1997 for nitrate plus nitrite as nitrogen (DIS,COL,LIS) and phosphorus (WWR,ASF) indicated lack of precision. All seven nutrient procedures at the QWSU displayed acceptable precision for water year 1997.

Bias

Analytical bias for the nutrient analyses are presented in table 6 for the NWQL and in table 7 for the QWSU. These tables show the results of the Wilcoxon signed-rank test for each reference sample mix by constituent. Strong evidence of bias was considered to be greater than 50 percent of the samples found to have a positive or negative bias. Frequently one or two of the reference mixes for a constituent indicated analytical bias. In many of these cases, the median bias was only slightly greater or less than the reporting unit, or data for other reference samples in the same analytical range did not indicate analytical bias. The following discussion presents an evaluation of those analytes that showed strong evidence of bias through part or all of the range of reference samples submitted by the BSP.

At the NWQL and the QWSU, analytical bias was indicated for very few nutrient samples; in most cases, the median bias was close to the nearest reporting unit. Negative bias was indicated for two of four mixes for the NWQL nitrate plus nitrite as nitrogen (DIS, USEPA) method. This bias was most likely due to the sample preservation process rather than an analytical error. The U.S. Environmental Protection Agency (USEPA) method requires acidification of the sample to a pH of less than 2 with sulfuric acid. Literature, including Patton and Gilroy (1998), has shown that acid preservation in some samples may result in a low estimate of the nitrate plus nitrite concentrations.

QUALITY-ASSURANCE DATA FOR INORGANIC-CONSTITUENT SAMPLES

Precision

The results of statistical testing for analytical precision for each inorganic constituent are presented in table 4 for the NWQL and in table 5 for the QWSU. These tables show either acceptable results (indicated by --) or a statistically significant lack of precision (LOP) at a significance level of 0.01 for each constituent.

The NWQL data from water year 1997 for arsenic (DIS, HG-AAS), arsenic (WWR, HG-AAS), cadmium (DIS, ICP-AES), lead (DIS, ICP-AES), mercury (WWR, CV-AAS), and silver (DIS, ICP-AES) indicated lack of precision. The high variability seen in the arsenic (DIS,HG-AAS) and arsenic (WWR,HG-AAS) appear to be caused by an interference present in an SRWS used by the BSP (G.E. Brown, U.S. Geological Survey, National Water Quality Laboratory, written commun., 1997). All of the inductively coupled plasma (ICP-AES) analyses noted above had their minimum reporting limits (MRL) raised in December 1997 (U.S. Geological Survey National Water Quality Laboratory Technical Memorandum No. 98.05, 1998). Most of the BSP MPVs will fall below the new MRLs for the ICP-AES determinations noted above. By raising the MRLs, the data will be reported as "less than the reporting limit", which result in a censoring of the BSP data to the new MRL. The BSP anticipates the high variability to decrease for these determinations once the new MRL is in effect. The lack of precision noted for mercury (WWR, CV-AAS) was due to analytical transcription errors. All seven outliers were false negatives, reported as less than detects, and should have been manually updated with the correct values by the analyst.

The QWSU data from water year 1997 for strontium (DIS, ICP-AES/T) indicated a lack of precision. The high variability was due to an error in the rounding algorithm used on the laboratory computer system. The variance improved mid-January 1997 after the rounding algorithm was corrected.

    Table 4 -- Results of statistical testing for analytical precision in constituent data for the National Water Quality Laboratory

    Table 5 -- Results of statistical testing for analytical precision in constituent data for the Quality of Water Service Unit Laboratory

Bias

Analytical bias for the inorganic analyses are presented in table 6 for the NWQL and in table 7 for the QWSU. These tables show the results of the Wilcoxon signed-rank test for each reference sample mix by constituent. The following discussion presents an evaluation of those analytes that show strong evidence of bias through part or all of the concentration range of reference samples submitted by the BSP. Strong evidence of bias was considered to be greater than 50 percent of the samples found to have a positive or negative bias. Frequently only one or two of the reference mixes for a constituent indicated analytical bias. In many of these cases, the median bias was only slightly greater than the reporting unit, or the data for other reference samples in the same analytical range did not indicate analytical bias.

At the NWQL, positive analytical bias was indicated throughout the range of reference materials for aluminum (DIS,ICP-AES). Negative bias was indicated throughout the range of reference samples for vanadium (DIS,COL). Positive bias was indicated for several higher concentration mixes for antimony (DIS,ICP-MS) and lead (DIS,GF-AAS). Mixes showing a positive bias for lead (DIS, GF-AAS) had concentrations above the upper analytical range, which might be attributed to dilution errors. The bias indicated for arsenic (DIS,HG-AAS) and arsenic (WWR,HG-AAS) occurred on only those mixes containing a particular SRWS, Trace-137 (G.E. Brown, U.S. Geological Survey, National Water Quality Laboratory, written commun., 1997). Trace-137 appears to have a matrix interference with the NWQL arsenic hydride methods. Negative analytical bias was indicated in the 200-300 mg/L concentration range for manganese (DIS,ICP-MS). Less than 5 percent of the environmental data analyzed for manganese (DIS, ICP-MS) at the NWQL was above 155 mg/L. Negative bias was also indicated for the lower level samples for sulfate (DIS,IC,LIS).

At the QWSU, positive analytical bias was indicated throughout the range of reference materials for aluminum (WWR,ICP-AES). Positive bias was indicated for 70 percent of the mixes analyzed for chloride (DIS,IC) and 67 percent of the mixes analyzed for dissolved solids (DIS,GRAV). Negative analytical bias throughout the reference sample range was indicated for barium (WWR,ICP-AES). Negative bias also was indicated for concentration greater than 39 mg/L for nickel (WWR,ICP-AES) and for 56 percent of the mixes analyzed for sodium (AA).

Alkalinity (WWR,TITR) at both the NWQL and the QWSU indicated positive and negative bias on several of the same mixes, indicating BSP mix biases rather than analytical biases.

The statistical power of the Wilcoxon signed-ranks tests for analytical bias from QWSU are dependent on sample size. Tests on analytical data from the QWSU are less powerful than the same tests used at the NWQL due to the smaller sample sizes associated with the QWSU data.

    Table 6 -- Results of Wilcoxon signed-rank test for bias in constituent data for the National Water Quality Laboratory

    Table 7 -- Results of Wilcoxon signed-rank test for bias in constituent data for the Quality of Water Service Unit Laboratory

SUMMARY

A quality-assurance program was operated during water year 1997 to evaluate the quality of analytical work for inorganic, nutrient, and low ionic-strength constituents at two USGS laboratories. Reference water samples with established most probable values were disguised and submitted as routine environmental water samples to the National Water Quality Laboratory in Arvada, Colorado, and the Quality of Water Service Unit Laboratory in Ocala, Florida. Reference samples were submitted at a rate of 2 to 5 percent of the laboratory work for each analytical procedure. Resulting analytical data were stored in National Water Information System and the Blind Sample Projects data base.

For each constituent, control charts were prepared on the basis of the difference between the analytical results and the most probable values of the reference samples. To allow the data for all reference mixes to be plotted on the same chart, the difference from the above calculations was divided by each sample’s F-pseudosigma, which was determined from a linear-regression technique. Replicate sample determinations allowed the preparation of precision charts for each constituent. Data for inorganic, nutrient, and low ionic-strength constituent samples were then evaluated statistically for precision by using a binomial-probability-distribution test. The Wilcoxon signed-rank test was used in the bias assessment for each constituent by quantifying the median difference between the reported values and the most probable values.

An overall evaluation of the National Water Quality Laboratory inorganic and low ionic-strength constituent data for water year 1997 indicated imprecision for the following procedures: arsenic (dissolved, hydride generation-atomic absorption spectrophotometry); arsenic (whole-water recoverable, hydride generation-atomic absorption spectrophotometry); cadmium (dissolved, inductively coupled plasma-atomic emission spectrometry); lead (dissolved, inductively coupled plasma-atomic emission spectrometry); mercury (whole-water recoverable, cold vapor-atomic absorption spectrophotometry); and silver (dissolved, inductively coupled plasma-atomic emission spectrometry). Aluminum (dissolved, inductively coupled plasma-atomic emission spectrometry) and vanadium (dissolved, colorimetric) showed an analytical bias throughout the range of reference samples.

An overall evaluation of the Quality of Water Service Unit inorganic-constituent data for water year 1997 indicated imprecision for strontium (dissolved, inductively coupled plasma-atomic emission spectrometry, trace). Positive bias was indicated throughout the reference sample range for aluminum (whole-water recoverable, inductively coupled plasma-atomic emission spectrometry) and a negative bias was indicated throughout the reference sample range for barium (whole-water recoverable, inductively coupled plasma-atomic emission spectrometry).

Statistical evaluations of the National Water Quality Laboratory and the Quality of Water Service Unit nutrient methods showed few nutrient sample biases but indicated a lack of precision for nitrate plus nitrite as nitrogen (dissolved, colorimetric, low ionic-stength) and phosphorus (whole-water recoverable, micro-Kjeldahl digestion, automated segmented flow, colorimetric) at the National Water Quality Laboratory.

REFERENCES

  • Farrar, J.W., 1998, Results of the U.S. Geological Survey’s analytical evaluation program for standard reference samples: T-153 (trace constituents), M-146 (major constituents), N-57 (nutrient constituents), N-58 (nutrient constituents), P-30 (low ionic-strength constituents), GWT-3 (ground-water trace constituents), GWM-3 (ground-water major constituents), and Hg-26 (mercury) distributed in April 1998: U.S. Geological Survey Open-File Report 98-391, 160 p.
  • Fishman, M.J., and Friedman, L.C., 1989, Methods for determination of inorganic substances in water and fluvial sediments: Techniques of Water-Resources Investigations of the U.S. Geological Survey, book 5, chap. A1, 545 p.
  • Friedman, L.C., Bradford, W.L., and Peart, D.B., 1983, Application of binomial distributions to quality assurance of quantitative chemical analyses: Journal of Environmental Science and Health, v. A18, no. 4, p. 561-570.
  • Friedman, L.C., and Erdmann, D.E., 1982, Quality assurance practices for the chemical and biological analyses of water and sediments: U.S. Geological Survey Techniques of Water-Resources Investigations, book 5, chap. A6, 181 p.
  • Helsel, D.R., and Hirsch, R.M., 1992, Statistical methods in water resources - Studies in environmental science 49: Amsterdam, The Netherlands, Elsevier Science Publishers, 522 p.
  • Hoaglin, D.C., Mosteller, Frederick, and Tukey, J.W. 1983, Understanding robust and exploratory data analysis: New York, John Wiley, 447 p.
  • Janzer, V.J., 1985, The use of natural waters as U.S. Geological Survey reference samples, in Taylor, J.K., and Stanley, T.W., eds., Quality assurance for environmental measurements, ASTM STP 867: Philadelphia, American Society for Testing and Materials, p. 319-333.
  • Lucey, K.J., 1989, Quality-assurance data for routine water analysis in the National Water Quality Laboratory of the U.S. Geological Survey for water year 1988: U.S. Geological Survey Water- Resources Investigations Report 89-4166, 96 p.
  • Lucey, K.J., and Peart, D.B., 1988, Quality-assurance data for routine water analysis in the laboratories of the U.S. Geological Survey for water year 1985: U.S. Geological Survey Water-Resources Investigations Report 88-4109, 121 p.
  • ¾¾ 1989a, Quality-assurance data for routine water analysis in the laboratories of the U.S. Geological Survey for water year 1986: U.S. Geological Survey Water-Resources Investigations Report 89-4009, 145 p.
  • ¾¾ 1989b, Quality-assurance data for routine water analysis in the laboratories of the U.S. Geological Survey for water year 1987: U.S. Geological Survey Water-Resources Investigations Report 89-4049, 90 p.
  • Maloney, T.J., Ludtke, A.S, and Krizman, T.L., 1992, Quality-assurance data for routine water analysis in the National Water Quality Laboratory of the U.S. Geological Survey for water year 1989: U.S. Geological Survey Water-Resources Investigations Report 92-4075, 99 p.
  • ¾¾ 1993, Quality-assurance data for the routine water analysis in the laboratories of the U.S. Geological Survey for water year 1990: U.S. Geological Survey Water-Resources Investigations Report 93-4082, 145 p.
  • ¾¾ 1994, Quality-assurance results for routine water analysis in U.S. Geological Survey laboratories, water year 1991: U.S. Geological Survey Water-Resources Investigations Report 94-4046, 144 p.
  • Patton, C.J., and Gilroy, E.J. 1998, U.S. Geological Survey Nutrient Preservation Experiment - Experimental design, statistical analysis, and interpretation of analytical results: U.S. Geological Survey Water-Resources Investigations Report 98-4118, 84 p.
  • Peart, D.B., and Sutphin, H.B. 1987, Quality-assurance data for routine water analysis in the laboratories of the U.S. Geological Survey for water year 1984: U.S. Geological Survey Water-Resources Investigations Report 87-4077, 125 p.
  • Peart, D.B., and Thomas, Nancy, 1983a, Quality-assurance data for routine water analysis in the laboratories of the U.S. Geological Survey 1981 annual report: U.S. Geological Survey Water-Resources Investigations Report 83-4090, 112 p.
  • ¾¾ 1983b, Quality-assurance data for routine water analysis in the laboratories of the U.S. Geological Survey for water year 1982: U.S. Geological Survey Water-Resources Investigations Report 83-4264, 112 p.
  • ¾¾ 1984, Quality-assurance data for routine water analysis in the laboratories of the U.S. Geological Survey for water year 1983: U.S. Geological Survey Water-Resources Investigations Report 84-4234, 112 p.
  • Schroder, L.J., Fishman, M.J., Friedman, L.C., and Darlington, G.W., 1980, The use of standard reference water samples by the U.S. Geological Survey: U.S. Geological Survey Open-File Report 80-738, 11 p.
  • Skougstad, M.W., and Fishman, M.J., 1975, Standard reference water samples: American Water Works Association Water Quality Technology Conference, Dallas, 1974, Proceedings, p. XIX-1-XIX-6.
  • Taylor, J.K., 1987, Quality assurance of chemical measurements: Chelsea, Mich., Lewis Publishers, 328 p.
INTERNAL DOCUMENTS SUPPLEMENTAL DATA

Go to the List of Figures

The use of firm, trade, or brand names in this report is for identification purposes only and does not constitute endorsement by the U.S. Government.

For additional information write to: Copies of this report can be purchased from:
Chief, Branch of Quality Systems
U.S. Geological Survey
Water Resources Division
Box 25046, Mail Stop 401
Denver Federal Center
Denver, CO 80225-0046		 U.S. Geological Survey
Branch of Information Services
Box 25286
Denver Federal Center
Denver, CO 80225-0286