Loading (50 kb)...'

(continued)

c. If the source is not isolated, it may be necessary to use a multi-source model to establish the impact of nearby sources. Since sources don't typically operate at their maximum allowable capacity (which may include the use of “dirtier” fuels), modeling is necessary to express the potential contribution of background sources, and this impact would not be captured via monitoring. Background concentrations should be determined for each critical (concentration) averaging time.

8.2.2 Recommendations (Isolated Single Source)

a. Two options (paragraph (b) or (c) of this section) are available to determine the background concentration near isolated sources.

b. Use air quality data collected in the vicinity of the source to determine the background concentration for the averaging times of concern. Determine the mean background concentration at each monitor by excluding values when the source in question is impacting the monitor. The mean annual background is the average of the annual concentrations so determined at each monitor. For shorter averaging periods, the meteorological conditions accompanying the concentrations of concern should be identified. Concentrations for meteorological conditions of concern, at monitors not impacted by the source in question, should be averaged for each separate averaging time to determine the average background value. Monitoring sites inside a 90° sector downwind of the source may be used to determine the area of impact. One hour concentrations may be added and averaged to determine longer averaging periods.

c. If there are no monitors located in the vicinity of the source, a “regional site” may be used to determine background. A “regional site” is one that is located away from the area of interest but is impacted by similar natural and distant man-made sources.

8.2.3 Recommendations (Multi-Source Areas)

a. In multi-source areas, two components of background should be determined: contributions from nearby sources and contributions from other sources.

b. Nearby Sources: All sources expected to cause a significant concentration gradient in the vicinity of the source or sources under consideration for emission limit(s) should be explicitly modeled. The number of such sources is expected to be small except in unusual situations. Owing to both the uniqueness of each modeling situation and the large number of variables involved in identifying nearby sources, no attempt is made here to comprehensively define this term. Rather, identification of nearby sources calls for the exercise of professional judgement by the appropriate reviewing authority (paragraph 3.0(b)). This guidance is not intended to alter the exercise of that judgement or to comprehensively define which sources are nearby sources.

c. For compliance with the short-term and annual ambient standards, the nearby sources as well as the primary source(s) should be evaluated using an appropriate Appendix A model with the emission input data shown in Table 8–1 or 8–2. When modeling a nearby source that does not have a permit and the emission limit contained in the SIP for a particular source category is greater than the emissions possible given the source's maximum physical capacity to emit, the “maximum allowable emission limit” for such a nearby source may be calculated as the emission rate representative of the nearby source's maximum physical capacity to emit, considering its design specifications and allowable fuels and process materials. However, the burden is on the permit applicant to sufficiently document what the maximum physical capacity to emit is for such a nearby source.

d. It is appropriate to model nearby sources only during those times when they, by their nature, operate at the same time as the primary source(s) being modeled. Where a primary source believes that a nearby source does not, by its nature, operate at the same time as the primary source being modeled, the burden is on the primary source to demonstrate to the satisfaction of the appropriate reviewing authority (paragraph 3.0(b)) that this is, in fact, the case. Whether or not the primary source has adequately demonstrated that fact is a matter of professional judgement left to the discretion of the appropriate reviewing authority. The following examples illustrate two cases in which a nearby source may be shown not to operate at the same time as the primary source(s) being modeled. Some sources are only used during certain seasons of the year. Those sources would not be modeled as nearby sources during times in which they do not operate. Similarly, emergency backup generators that never operate simultaneously with the sources that they back up would not be modeled as nearby sources. To reiterate, in these examples and other appropriate cases, the burden is on the primary source being modeled to make the appropriate demonstration to the satisfaction of the appropriate reviewing authority.

e. The impact of the nearby sources should be examined at locations where interactions between the plume of the point source under consideration and those of nearby sources (plus natural background) can occur. Significant locations include: (1) the area of maximum impact of the point source; (2) the area of maximum impact of nearby sources; and (3) the area where all sources combine to cause maximum impact. These locations may be identified through trial and error analyses.

f. Other Sources: That portion of the background attributable to all other sources (e.g., natural sources, minor sources and distant major sources) should be determined by the procedures found in subsection 89.2.2 or by application of a model using Table 8–1 or 8–2.

8.3 Meteorological Input Data

a. The meteorological data used as input to a dispersion model should be selected on the basis of spatial and climatological (temporal) representativeness as well as the ability of the individual parameters selected to characterize the transport and dispersion conditions in the area of concern. The representativeness of the data is dependent on: (1) The proximity of the meteorological monitoring site to the area under consideration; (2) the complexity of the terrain; (3) the exposure of the meteorological monitoring site; and (4) the period of time during which data are collected. The spatial representativeness of the data can be adversely affected by large distances between the source and receptors of interest and the complex topographic characteristics of the area. Temporal representativeness is a function of the year-to-year variations in weather conditions. Where appropriate, data representativeness should be viewed in terms of the appropriateness of the data for constructing realistic boundary layer profiles and three dimensional meteorological fields, as described in paragraphs (c) and (d) below.

b. Model input data are normally obtained either from the National Weather Service or as part of a site specific measurement program. Local universities, Federal Aviation Administration (FAA), military stations, industry and pollution control agencies may also be sources of such data. Some recommendations for the use of each type of data are included in this subsection.

c. Regulatory application of AERMOD requires careful consideration of minimum data for input to AERMET. Data representativeness, in the case of AERMOD, means utilizing data of an appropriate type for constructing realistic boundary layer profiles. Of paramount importance is the requirement that all meteorological data used as input to AERMOD must be both laterally and vertically representative of the transport and dispersion within the analysis domain. Where surface conditions vary significantly over the analysis domain, the emphasis in assessing representativeness should be given to adequate characterization of transport and dispersion between the source(s) of concern and areas where maximum design concentrations are anticipated to occur. The representativeness of data that were collected off-site should be judged, in part, by comparing the surface characteristics in the vicinity of the meteorological monitoring site with the surface characteristics that generally describe the analysis domain. The surface characteristics input to AERMET should be based on the topographic conditions in the vicinity of the meteorological tower. Furthermore, since the spatial scope of each variable could be different, representativeness should be judged for each variable separately. For example, for a variable such as wind direction, the data may need to be collected very near plume height to be adequately representative, whereas, for a variable such as temperature, data from a station several kilometers away from the source may in some cases be considered to be adequately representative.

d. For long range transport modeling assessments (subsection 6.2.3) or for assessments where the transport winds are complex and the application involves a non-steady-state dispersion model (subsection 7.2.8), use of output from prognostic mesoscale meteorological models is encouraged. 84,85,86 Some diagnostic meteorological processors are designed to appropriately blend available NWS comparable meteorological observations, local site specific meteorological observations, and prognostic mesoscale meteorological data, using empirical relationships, to diagnostically adjust the wind field for mesoscale and local-scale effects. These diagnostic adjustments can sometimes be improved through the use of strategically placed site specific meteorological observations. The placement of these special meteorological observations (often more than one location is needed) involves expert judgement, and is specific to the terrain and land use of the modeling domain. Acceptance for use of output from prognostic mesoscale meteorological models is contingent on concurrence by the appropriate reviewing authorities (paragraph 3.0(b)) that the data are of acceptable quality, which can be demonstrated through statistical comparisons with observations of winds aloft and at the surface at several appropriate locations.

8.3.1 Length of Record of Meteorological Data

8.3.1.1 Discussion

a. The model user should acquire enough meteorological data to ensure that worst-case meteorological conditions are adequately represented in the model results. The trend toward statistically based standards suggests a need for all meteorological conditions to be adequately represented in the data set selected for model input. The number of years of record needed to obtain a stable distribution of conditions depends on the variable being measured and has been estimated by Landsberg and Jacobs 87 for various parameters. Although that study indicates in excess of 10 years may be required to achieve stability in the frequency distributions of some meteorological variables, such long periods are not reasonable for model input data. This is due in part to the fact that hourly data in model input format are frequently not available for such periods and that hourly calculations of concentration for long periods may be prohibitively expensive. Another study 88 compared various periods from a 17-year data set to determine the minimum number of years of data needed to approximate the concentrations modeled with a 17-year period of meteorological data from one station. This study indicated that the variability of model estimates due to the meteorological data input was adequately reduced if a 5-year period of record of meteorological input was used.

8.3.1.2 Recommendations

a. Five years of representative meteorological data should be used when estimating concentrations with an air quality model. Consecutive years from the most recent, readily available 5-year period are preferred. The meteorological data should be adequately representative, and may be site specific or from a nearby NWS station. Where professional judgment indicates NWS-collected ASOS (automated surface observing stations) data are inadequate {for cloud cover observations}, the most recent 5 years of NWS data that are observer-based may be considered for use.

b. The use of 5 years of NWS meteorological data or at least l year of site specific data is required. If one year or more (including partial years), up to five years, of site specific data is available, these data are preferred for use in air quality analyses. Such data should have been subjected to quality assurance procedures as described in subsection 8.3.3.2.

c. For permitted sources whose emission limitations are based on a specific year of meteorological data, that year should be added to any longer period being used (e.g., 5 years of NWS data) when modeling the facility at a later time.

d. For LRT situations (subsection 6.2.3) and for complex wind situations (paragraph 7.2.8(a)), if only NWS or comparable standard meteorological observations are employed, five years of meteorological data (within and near the modeling domain) should be used. Consecutive years from the most recent, readily available 5-year period are preferred. Less than five, but at least three, years of meteorological data (need not be consecutive) may be used if mesoscale meteorological fields are available, as discussed in paragraph 8.3(d). These mesoscale meteorological fields should be used in conjunction with available standard NWS or comparable meteorological observations within and near the modeling domain.

e. For solely LRT applications (subsection 6.2.3), if site specific meteorological data are available, these data may be helpful when used in conjunction with available standard NWS or comparable observations and mesoscale meteorological fields as described in paragraph 8.3.1.2(d).

f. For complex wind situations (paragraph 7.2.8(a)) where site specific meteorological data are being relied upon as the basis for characterizing the meteorological conditions, a data base of at least 1 full-year of meteorological data is required. If more data are available, they should be used. Site specific meteorological data may have to be collected at multiple locations. Such data should have been subjected to quality assurance procedures as described in paragraph 8.3.3.2(a), and should be reviewed for spatial and temporal representativeness.

8.3.2 National Weather Service Data

8.3.2.1 Discussion

a. The NWS meteorological data are routinely available and familiar to most model users. Although the NWS does not provide direct measurements of all the needed dispersion model input variables, methods have been developed and successfully used to translate the basic NWS data to the needed model input. Site specific measurements of model input parameters have been made for many modeling studies, and those methods and techniques are becoming more widely applied, especially in situations such as complex terrain applications, where available NWS data are not adequately representative. However, there are many model applications where NWS data are adequately representative, and the applications still rely heavily on the NWS data.

b. Many models use the standard hourly weather observations available from the National Climatic Data Center (NCDC). These observations are then preprocessed before they can be used in the models.

8.3.2.2 Recommendations

a. The preferred models listed in Appendix A all accept as input the NWS meteorological data preprocessed into model compatible form. If NWS data are judged to be adequately representative for a particular modeling application, they may be used. NCDC makes available surface 89,90 and upper air 91 meteorological data in CD–ROM format.

b. Although most NWS measurements are made at a standard height of 10 meters, the actual anemometer height should be used as input to the preferred model. Note that AERMOD at a minimum requires wind observations at a height above ground between seven times the local surface roughness height and 100 meters.

c. Wind directions observed by the National Weather Service are reported to the nearest 10 degrees. A specific set of randomly generated numbers has been developed for use with the preferred EPA models and should be used with NWS data to ensure a lack of bias in wind direction assignments within the models.

d. Data from universities, FAA, military stations, industry and pollution control agencies may be used if such data are equivalent in accuracy and detail to the NWS data, and they are judged to be adequately representative for the particular application.

8.3.3 Site Specific Data

8.3.3.1 Discussion

a. Spatial or geographical representativeness is best achieved by collection of all of the needed model input data in close proximity to the actual site of the source(s). Site specific measured data are therefore preferred as model input, provided that appropriate instrumentation and quality assurance procedures are followed and that the data collected are adequately representative (free from inappropriate local or microscale influences) and compatible with the input requirements of the model to be used. It should be noted that, while site specific measurements are frequently made “on-property” (i.e., on the source's premises), acquisition of adequately representative site specific data does not preclude collection of data from a location off property. Conversely, collection of meteorological data on a source's property does not of itself guarantee adequate representativeness. For help in determining representativeness of site specific measurements, technical guidance 92 is available. Site specific data should always be reviewed for representativeness and consistency by a qualified meteorologist.

8.3.3.2 Recommendations

a. EPA guidance 92 provides recommendations on the collection and use of site specific meteorological data. Recommendations on characteristics, siting, and exposure of meteorological instruments and on data recording, processing, completeness requirements, reporting, and archiving are also included. This publication should be used as a supplement to other limited guidance on these subjects. 83,93,94 Detailed information on quality assurance is also available. 95 As a minimum, site specific measurements of ambient air temperature, transport wind speed and direction, and the variables necessary to estimate atmospheric dispersion should be available in meteorological data sets to be used in modeling. Care should be taken to ensure that meteorological instruments are located to provide representative characterization of pollutant transport between sources and receptors of interest. The appropriate reviewing authority (paragraph 3.0(b)) is available to help determine the appropriateness of the measurement locations.

b. All site specific data should be reduced to hourly averages. Table 8–3 lists the wind related parameters and the averaging time requirements.

c. Missing Data Substitution. After valid data retrieval requirements have been met, 92 hours in the record having missing data should be treated according to an established data substitution protocol provided that data from an adequately representative alternative site are available. Such protocols are usually part of the approved monitoring program plan. Data substitution guidance is provided in Section 5.3 of reference 92. If no representative alternative data are available for substitution, the absent data should be coded as missing using missing data codes appropriate to the applicable meteorological pre-processor. Appropriate model options for treating missing data, if available in the model, should be employed.

d. Solar Radiation Measurements. Total solar radiation or net radiation should be measured with a reliable pyranometer or net radiometer, sited and operated in accordance with established site specific meteorological guidance. 92,95

e. Temperature Measurements. Temperature measurements should be made at standard shelter height (2m) in accordance with established site specific meteorological guidance. 92

f. Temperature Difference Measurements. Temperature difference (?T) measurements should be obtained using matched thermometers or a reliable thermocouple system to achieve adequate accuracy. Siting, probe placement, and operation of ?T systems should be based on guidance found in Chapter 3 of reference 92, and such guidance should be followed when obtaining vertical temperature gradient data. AERMET employs the Bulk Richardson scheme which requires measurements of temperature difference. To ensure correct application and acceptance, AERMOD users should consult with the appropriate Reviewing Authority before using the Bulk Richardson scheme for their analysis.

g. Winds Aloft. For simulation of plume rise and dispersion of a plume emitted from a stack, characterization of the wind profile up through the layer in which the plume disperses is required. This is especially important in complex terrain and/or complex wind situations where wind measurements at heights up to hundreds of meters above stack base may be required in some circumstances. For tall stacks when site specific data are needed, these winds have been obtained traditionally using meteorological sensors mounted on tall towers. A feasible alternative to tall towers is the use of meteorological remote sensing instruments (e.g., acoustic sounders or radar wind profilers) to provide winds aloft, coupled with 10-meter towers to provide the near-surface winds. (For specific requirements for AERMOD and CTDMPLUS, see Appendix A.) Specifications for wind measuring instruments and systems are contained in reference 92.

h. Turbulence. There are several dispersion models that are capable of using direct measurements of turbulence (wind fluctuations) in the characterization of the vertical and lateral dispersion (e.g., CTDMPLUS, AERMOD, and CALPUFF). For specific requirements for CTDMPLUS, AERMOD, and CALPUFF, see Appendix A. For technical guidance on measurement and processing of turbulence parameters, see reference 92. When turbulence data are used in this manner to directly characterize the vertical and lateral dispersion, the averaging time for the turbulence measurements should be one hour (Table 8–3). There are other dispersion models (e.g., BLP, and CALINE3) that employ P–G stability categories for the characterization of the vertical and lateral dispersion. Methods for using site specific turbulence data for the characterization of P–G stability categories are discussed in reference 92. When turbulence data are used in this manner to determine the P–G stability category, the averaging time for the turbulence measurements should be 15 minutes.

i. Stability Categories. For dispersion models that employ P–G stability categories for the characterization of the vertical and lateral dispersion, the P–G stability categories, as originally defined, couple near-surface measurements of wind speed with subjectively determined insolation assessments based on hourly cloud cover and ceiling height observations. The wind speed measurements are made at or near 10m. The insolation rate is typically assessed using observations of cloud cover and ceiling height based on criteria outlined by Turner. 70 It is recommended that the P–G stability category be estimated using the Turner method with site specific wind speed measured at or near 10m and representative cloud cover and ceiling height. Implementation of the Turner method, as well as considerations in determining representativeness of cloud cover and ceiling height in cases for which site specific cloud observations are unavailable, may be found in Section 6 of reference 92. In the absence of requisite data to implement the Turner method, the SRDT method or wind fluctuation statistics (i.e., the sE and sA methods) may be used.

j. The SRDT method, described in Section 6.4.4.2 of reference 92, is modified slightly from that published from earlier work 96 and has been evaluated with three site specific data bases. 97 The two methods of stability classification which use wind fluctuation statistics, the sE and sA methods, are also described in detail in Section 6.4.4 of reference 92 (note applicable tables in Section 6). For additional information on the wind fluctuation methods, several references are available. 98,99,100,101

k. Meteorological Data Preprocessors. The following meteorological preprocessors are recommended by EPA: AERMET, 102 PCRAMMET, 103 MPRM, 104 METPRO, 105 and CALMET 106 AERMET, which is patterned after MPRM, should be used to preprocess all data for use with AERMOD. Except for applications that employ AERMOD, PCRAMMET is the recommended meteorological preprocessor for use in applications employing hourly NWS data. MPRM is a general purpose meteorological data preprocessor which supports regulatory models requiring PCRAMMET formatted (NWS) data. MPRM is available for use in applications employing site specific meteorological data. The latest version (MPRM 1.3) has been configured to implement the SRDT method for estimating P–G stability categories. METPRO is the required meteorological data preprocessor for use with CTDMPLUS. CALMET is available for use with applications of CALPUFF. All of the above mentioned data preprocessors are available for downloading from EPA's Internet SCRAM Web site (subsection 2.3).


Table 8-3._Averaging Times for Site Specific Wind and Turbulence
Measurements
------------------------------------------------------------------------
Averaging
Parameter time
(hour)
------------------------------------------------------------------------
Surface wind speed (for use in stability determinations)... 1
Transport direction........................................ 1
Dilution wind speed........................................ 1
Turbulence measurements (sE and sA) for use in 1 \1\
stability determinations..................................
Turbulence measurements for direct input to dispersion 1
models....................................................
------------------------------------------------------------------------
\1\ To minimize meander effects in sA when wind conditions are
light and/or variable, determine the hourly average s value from
four sequential 15-minute s's according to the following
formula:



8.3.4 Treatment of Near-Calms and Calms

8.3.4.1 Discussion

a. Treatment of calm or light and variable wind poses a special problem in model applications since steady-state Gaussian plume models assume that concentration is inversely proportional to wind speed. Furthermore, concentrations may become unrealistically large when wind speeds less than 1 m/s are input to the model. Procedures have been developed to prevent the occurrence of overly conservative concentration estimates during periods of calms. These procedures acknowledge that a steady-state Gaussian plume model does not apply during calm conditions, and that our knowledge of wind patterns and plume behavior during these conditions does not, at present, permit the development of a better technique. Therefore, the procedures disregard hours which are identified as calm. The hour is treated as missing and a convention for handling missing hours is recommended.

b. AERMOD, while fundamentally a steady-state Gaussian plume model, contains algorithms for dealing with low wind speed (near calm) conditions. As a result, AERMOD can produce model estimates for conditions when the wind speed may be less than 1 m/s, but still greater than the instrument threshold. Required input to AERMET, the meteorological processor for AERMOD, includes a threshold wind speed and a reference wind speed. The threshold wind speed is typically the threshold of the instrument used to collect the wind speed data. The reference wind speed is selected by the model as the lowest level of non-missing wind speed and direction data where the speed is greater than the wind speed threshold, and the height of the measurement is between seven times the local surface roughness and 100 meters. If the only valid observation of the reference wind speed between these heights is less than the threshold, the hour is considered calm, and no concentration is calculated. None of the observed wind speeds in a measured wind profile that are less than the threshold speed are used in construction of the modeled wind speed profile in AERMOD.

8.3.4.2 Recommendations

a. Hourly concentrations calculated with steady-state Gaussian plume models using calms should not be considered valid; the wind and concentration estimates for these hours should be disregarded and considered to be missing. Critical concentrations for 3-, 8-, and 24-hour averages should be calculated by dividing the sum of the hourly concentrations for the period by the number of valid or non-missing hours. If the total number of valid hours is less than 18 for 24-hour averages, less than 6 for 8-hour averages or less than 3 for 3-hour averages, the total concentration should be divided by 18 for the 24-hour average, 6 for the 8-hour average and 3 for the 3-hour average. For annual averages, the sum of all valid hourly concentrations is divided by the number of non-calm hours during the year. AERMOD has been coded to implement these instructions. For models listed in Appendix A, a post-processor computer program, CALMPRO 107 has been prepared, is available on the SCRAM Internet Web site (subsection 2.3), and should be used.

b. Stagnant conditions that include extended periods of calms often produce high concentrations over wide areas for relatively long averaging periods. The standard steady-state Gaussian plume models are often not applicable to such situations. When stagnation conditions are of concern, other modeling techniques should be considered on a case-by-case basis (see also subsection 7.2.8).

c. When used in steady-state Gaussian plume models, measured site specific wind speeds of less than 1 m/s but higher than the response threshold of the instrument should be input as 1 m/s; the corresponding wind direction should also be input. Wind observations below the response threshold of the instrument should be set to zero, with the input file in ASCII format. For input to AERMOD, no adjustment should be made to the site specific wind data. In all cases involving steady-state Gaussian plume models, calm hours should be treated as missing, and concentrations should be calculated as in paragraph (a) of this subsection.

9.0 Accuracy and Uncertainty of Models

9.1 Discussion

a. Increasing reliance has been placed on concentration estimates from models as the primary basis for regulatory decisions concerning source permits and emission control requirements. In many situations, such as review of a proposed source, no practical alternative exists. Therefore, there is an obvious need to know how accurate models really are and how any uncertainty in the estimates affects regulatory decisions. During the 1980's, attempts were made to encourage development of standardized evaluation methods. 11,108 EPA recognized the need for incorporating such information and has sponsored workshops 109 on model accuracy, the possible ways to quantify accuracy, and on considerations in the incorporation of model accuracy and uncertainty in the regulatory process. The Second (EPA) Conference on Air Quality Modeling, August 1982 110 , was devoted to that subject.

b. To better deduce the statistical significance of differences seen in model performance in the face of unaccounted for uncertainties and variations, investigators have more recently explored the use of bootstrap techniques. 111,112 Work is underway to develop a new generation of evaluation metrics 16 that takes into account the statistical differences (in error distributions) between model predictions and observations. 113 Even though the procedures and measures are still evolving to describe performance of models that characterize atmospheric fate, transport and diffusion, 114,115,116 there has been general acceptance of a need to address the uncertainties inherent in atmospheric processes.

9.1.1 Overview of Model Uncertainty

a. Dispersion models generally attempt to estimate concentrations at specific sites that really represent an ensemble average of numerous repetitions of the same event. 16 The event is characterized by measured or “known” conditions that are input to the models, e.g., wind speed, mixed layer height, surface heat flux, emission characteristics, etc. However, in addition to the known conditions, there are unmeasured or unknown variations in the conditions of this event, e.g., unresolved details of the atmospheric flow such as the turbulent velocity field. These unknown conditions, may vary among repetitions of the event. As a result, deviations in observed concentrations from their ensemble average, and from the concentrations estimated by the model, are likely to occur even though the known conditions are fixed. Even with a perfect model that predicts the correct ensemble average, there are likely to be deviations from the observed concentrations in individual repetitions of the event, due to variations in the unknown conditions. The statistics of these concentration residuals are termed “inherent” uncertainty. Available evidence suggests that this source of uncertainty alone may be responsible for a typical range of variation in concentrations of as much as ±50 percent. 117

b. Moreover, there is “reducible” uncertainty 108 associated with the model and its input conditions; neither models nor data bases are perfect. Reducible uncertainties are caused by: (1) Uncertainties in the input values of the known conditions (i.e., emission characteristics and meteorological data); (2) errors in the measured concentrations which are used to compute the concentration residuals; and (3) inadequate model physics and formulation. The “reducible” uncertainties can be minimized through better (more accurate and more representative) measurements and better model physics.

c. To use the terminology correctly, reference to model accuracy should be limited to that portion of reducible uncertainty which deals with the physics and the formulation of the model. The accuracy of the model is normally determined by an evaluation procedure which involves the comparison of model concentration estimates with measured air quality data. 118 The statement of accuracy is based on statistical tests or performance measures such as bias, noise, correlation, etc. 11 However, information that allows a distinction between contributions of the various elements of inherent and reducible uncertainty is only now beginning to emerge. 16 As a result most discussions of the accuracy of models make no quantitative distinction between (1) limitations of the model versus (2) limitations of the data base and of knowledge concerning atmospheric variability. The reader should be aware that statements on model accuracy and uncertainty may imply the need for improvements in model performance that even the “perfect” model could not satisfy.

9.1.2 Studies of Model Accuracy

a. A number of studies 119,120 have been conducted to examine model accuracy, particularly with respect to the reliability of short-term concentrations required for ambient standard and increment evaluations. The results of these studies are not surprising. Basically, they confirm what expert atmospheric scientists have said for some time: (1) Models are more reliable for estimating longer time-averaged concentrations than for estimating short-term concentrations at specific locations; and (2) the models are reasonably reliable in estimating the magnitude of highest concentrations occurring sometime, somewhere within an area. For example, errors in highest estimated concentrations of ± 10 to 40 percent are found to be typical, 121,122 i.e., certainly well within the often quoted factor-of-two accuracy that has long been recognized for these models. However, estimates of concentrations that occur at a specific time and site, are poorly correlated with actually observed concentrations and are much less reliable.

b. As noted above, poor correlations between paired concentrations at fixed stations may be due to “reducible” uncertainties in knowledge of the precise plume location and to unquantified inherent uncertainties. For example, Pasquill 123 estimates that, apart from data input errors, maximum ground-level concentrations at a given hour for a point source in flat terrain could be in error by 50 percent due to these uncertainties. Uncertainty of five to 10 degrees in the measured wind direction, which transports the plume, can result in concentration errors of 20 to 70 percent for a particular time and location, depending on stability and station location. Such uncertainties do not indicate that an estimated concentration does not occur, only that the precise time and locations are in doubt.

9.1.3 Use of Uncertainty in Decision-Making

a. The accuracy of model estimates varies with the model used, the type of application, and site specific characteristics. Thus, it is desirable to quantify the accuracy or uncertainty associated with concentration estimates used in decision-making. Communications between modelers and decision-makers must be fostered and further developed. Communications concerning concentration estimates currently exist in most cases, but the communications dealing with the accuracy of models and its meaning to the decision-maker are limited by the lack of a technical basis for quantifying and directly including uncertainty in decisions. Procedures for quantifying and interpreting uncertainty in the practical application of such concepts are only beginning to evolve; much study is still required. 108,109,110,124,125

b. In all applications of models an effort is encouraged to identify the reliability of the model estimates for that particular area and to determine the magnitude and sources of error associated with the use of the model. The analyst is responsible for recognizing and quantifying limitations in the accuracy, precision and sensitivity of the procedure. Information that might be useful to the decision-maker in recognizing the seriousness of potential air quality violations includes such model accuracy estimates as accuracy of peak predictions, bias, noise, correlation, frequency distribution, spatial extent of high concentration, etc. Both space/time pairing of estimates and measurements and unpaired comparisons are recommended. Emphasis should be on the highest concentrations and the averaging times of the standards or increments of concern. Where possible, confidence intervals about the statistical values should be provided. However, while such information can be provided by the modeler to the decision-maker, it is unclear how this information should be used to make an air pollution control decision. Given a range of possible outcomes, it is easiest and tends to ensure consistency if the decision-maker confines his judgement to use of the “best estimate” provided by the modeler (i.e., the design concentration estimated by a model recommended in the Guideline or an alternate model of known accuracy). This is an indication of the practical limitations imposed by current abilities of the technical community.

c. To improve the basis for decision-making, EPA has developed and is continuing to study procedures for determining the accuracy of models, quantifying the uncertainty, and expressing confidence levels in decisions that are made concerning emissions controls. 126,127 However, work in this area involves “breaking new ground” with slow and sporadic progress likely. As a result, it may be necessary to continue using the “best estimate” until sufficient technical progress has been made to meaningfully implement such concepts dealing with uncertainty.

9.1.4 Evaluation of Models

a. A number of actions have been taken to ensure that the best model is used correctly for each regulatory application and that a model is not arbitrarily imposed. First, the Guideline clearly recommends the most appropriate model be used in each case. Preferred models, based on a number of factors, are identified for many uses. General guidance on using alternatives to the preferred models is also provided. Second, the models have been subjected to a systematic performance evaluation and a peer scientific review. Statistical performance measures, including measures of difference (or residuals) such as bias, variance of difference and gross variability of the difference, and measures of correlation such as time, space, and time and space combined as recommended by the AMS Woods Hole Workshop, 11 were generally followed. Third, more specific information has been provided for justifying the site specific use of alternative models in previously cited EPA guidance, 15 and new models are under consideration and review. 16 Together these documents provide methods that allow a judgement to be made as to what models are most appropriate for a specific application. For the present, performance and the theoretical evaluation of models are being used as an indirect means to quantify one element of uncertainty in air pollution regulatory decisions.

b. EPA has participated in a series of conferences entitled, “Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes.” 128 for the purpose of promoting the development of improved methods for the characterization of model performance. There is a consensus developing on what should be considered in the evaluation of air quality models 129 , namely quality assurance planning, documentation and scrutiny should be consistent with the intended use, and should include:

• Scientific peer review;

• Supportive analyses (diagnostic evaluations, code verification, sensitivity and uncertainty analyses);

• Diagnostic and performance evaluations with data obtained in trial locations, and

• Statistical performance evaluations in the circumstances of the intended applications.

Performance evaluations and diagnostic evaluations assess different qualities of how well a model is performing, and both are needed to establish credibility within the client and scientific community. Performance evaluations allow us to decide how well the model simulates the average temporal and spatial patterns seen in the observations, and employ large spatial/temporal scale data sets (e.g., national data sets). Performance evaluations also allow determination of relative performance of a model in comparison with alternative modeling systems. Diagnostic evaluations allow determination of a model capability to simulate individual processes that affect the results, and usually employ smaller spatial/temporal scale date sets (e.g., field studies). Diagnostic evaluations allow us to decide if we get the right answer for the right reason. The objective comparison of modeled concentrations with observed field data provides only a partial means for assessing model performance. Due to the limited supply of evaluation data sets, there are severe practical limits in assessing model performance. For this reason, the conclusions reached in the science peer reviews and the supportive analyses have particular relevance in deciding whether a model will be useful for its intended purposes.

c. To extend information from diagnostic and performance evaluations, sensitivity and uncertainty analyses are encouraged since they can provide additional information on the effect of inaccuracies in the data bases and on the uncertainty in model estimates. Sensitivity analyses can aid in determining the effect of inaccuracies of variations or uncertainties in the data bases on the range of likely concentrations. Uncertainty analyses can aid in determining the range of likely concentration values, resulting from uncertainties in the model inputs, the model formulations, and parameterizations. Such information may be used to determine source impact and to evaluate control strategies. Where possible, information from such sensitivity analyses should be made available to the decision-maker with an appropriate interpretation of the effect on the critical concentrations.

9.2 Recommendations

a. No specific guidance on the quantification of model uncertainty for use in decision-making is being given at this time. As procedures for considering uncertainty develop and become implementable, this guidance will be changed and expanded. For the present, continued use of the “best estimate” is acceptable; however, in specific circumstances for O3, PM–2.5 and regional haze, additional information and/or procedures may be appropriate. 32,33

10.0 Regulatory Application of Models

10.1 Discussion

a. Procedures with respect to the review and analysis of air quality modeling and data analyses in support of SIP revisions, PSD permitting or other regulatory requirements need a certain amount of standardization to ensure consistency in the depth and comprehensiveness of both the review and the analysis itself. This section recommends procedures that permit some degree of standardization while at the same time allowing the flexibility needed to assure the technically best analysis for each regulatory application.

b. Dispersion model estimates, especially with the support of measured air quality data, are the preferred basis for air quality demonstrations. Nevertheless, there are instances where the performance of recommended dispersion modeling techniques, by comparison with observed air quality data, may be shown to be less than acceptable. Also, there may be no recommended modeling procedure suitable for the situation. In these instances, emission limitations may be established solely on the basis of observed air quality data as would be applied to a modeling analysis. The same care should be given to the analyses of the air quality data as would be applied to a modeling analysis.

c. The current NAAQS for SO2 and CO are both stated in terms of a concentration not to be exceeded more than once a year. There is only an annual standard for NO2 and a quarterly standard for Pb. Standards for fine particulate matter (PM–2.5) are expressed in terms of both long-term (annual) and short-term (daily) averages. The long-term standard is calculated using the three year average of the annual averages while the short-term standard is calculated using the three year average of the 98th percentile of the daily average concentration. For PM–10, the convention is to compare the arithmetic mean, averaged over 3 consecutive years, with the concentration specified in the NAAQS (50 µg/m 3 ). The 24-hour NAAQS (150 µg/m 3 ) is met if, over a 3-year period, there is (on average) no more than one exceedance per year. As noted in subsection 7.2.1.1, the modeled compliance for this NAAQS is based on the highest 6th highest concentration over 5 years. For ozone the short term 1-hour standard is expressed in terms of an expected exceedance limit while the short term 8-hour standard is expressed in terms of a three year average of the annual fourth highest daily maximum 8-hour value. The NAAQS are subjected to extensive review and possible revision every 5 years.

d. This section discusses general requirements for concentration estimates and identifies the relationship to emission limits. The following recommendations apply to: (1) Revisions of State Implementation Plans and (2) the review of new sources and the prevention of significant deterioration (PSD).

10.2 Recommendations

10.2.1 Analysis Requirements

a. Every effort should be made by the Regional Office to meet with all parties involved in either a SIP revision or a PSD permit application prior to the start of any work on such a project. During this meeting, a protocol should be established between the preparing and reviewing parties to define the procedures to be followed, the data to be collected, the model to be used, and the analysis of the source and concentration data. An example of requirements for such an effort is contained in the Air Quality Analysis Checklist posted on EPA's Internet SCRAM Web site (subsection 2.3). This checklist suggests the level of detail required to assess the air quality resulting from the proposed action. Special cases may require additional data collection or analysis and this should be determined and agreed upon at this preapplication meeting. The protocol should be written and agreed upon by the parties concerned, although a formal legal document is not intended. Changes in such a protocol are often required as the data collection and analysis progresses. However, the protocol establishes a common understanding of the requirements.

b. An air quality analysis should begin with a screening model to determine the potential of the proposed source or control strategy to violate the PSD increment or NAAQS. For traditional stationary sources, EPA guidance 24 should be followed. Guidance is also available for mobile sources. 48

c. If the concentration estimates from screening techniques indicate a significant impact or that the PSD increment or NAAQS may be approached or exceeded, then a more refined modeling analysis is appropriate and the model user should select a model according to recommendations in Sections 4–8. In some instances, no refined technique may be specified in this guide for the situation. The model user is then encouraged to submit a model developed specifically for the case at hand. If that is not possible, a screening technique may supply the needed results. (continued)