Assessing Performance Using BMP Database Protocols

By Jane Clary, Jonathan Jones, Eric Strecker, and Marcus Quigley

Understanding stormwater-related pollution has become increasingly important to both water suppliers and stormwater/wastewater dischargers as Source Water Assessment Program (SWAP) and Total Maximum Daily Load (TMDL) requirements (expressed as new National Pollutant Discharge Elimination System [NPDES] permit conditions) affect more communities throughout the U.S. More specifically, water suppliers want to know how well stormwater best management practices (BMPs) protect water supplies, and those receiving wasteload allocations under TMDLs want to know what load reductions can be expected from BMP implementation.

Although stormwater BMPs rely on many of the same “unit processes” (e.g., filtration, sedimentation) used in wastewater and drinking water treatment, there are many more variables involved in BMP performance that must be taken into account in order to understand and enhance stormwater BMP performance. The International Stormwater BMP Database Project is now over a decade-long effort intended to help facilitate better understanding of BMP performance so that BMPs can be better designed and implemented.

The project is sponsored by a coalition of partners led by the Water Environment Research Foundation and includes these components:

1. Recommendations for data reporting requirements for BMP monitoring.
2. Development of an international database to store BMP performance data and making the base performance data available for others to evaluate.
3. Detailed guidance for designing, implementing and reporting on BMP performance monitoring and evaluation programs.
4. Recommendations for measures to assess BMP performance.
5. Interpretive reports that provide performance data for individual BMP studies, as well as for overall BMP types (e.g., detention basins).

An overview of each component follows, with more detailed information available on all of these topics at the project website (www.bmpdatabase.org).

Monitoring/Reporting Protocols

Through the work of many members of the Urban Water Resources Research Council of the American Society of Civil Engineers and other invited experts in the mid-1990s, a detailed list of monitoring and reporting protocols for various BMP types was developed. These protocols laid the foundation for the database, which is the cornerstone of the overall project. The database encompasses a broad range of parameters including test site location, watershed characteristics, climate data, BMP design and layout characteristics, monitoring instrumentation, and monitoring data for precipitation, flow and water quality. The database has roughly tripled in size since its initial release in 1999 and now contains nearly 280 studies, summarized in Table 1.

Work during 2007 has focused on making the database easier to use through development of data entry spreadsheets in Excel, along with the User’s Guide, both of which are available for download from the project website. An exciting aspect of the project is that large data providers in different parts of the country are adopting the basic database structure and using it to track their own BMP data. Examples include Washington State Department of Transportation, the Harris County Flood Control District in Houston, and the State of Florida. In several cases, the project team is working with these data providers to assist them in adopting or modifying the database to meet the user’s specific needs.

Monitoring Guidance

Early in the project, the project team identified the need for better guidance on proper monitoring and reporting of stormwater BMP performance studies and developed the U.S. Environmental Protection Agency (EPA) guidance document “Urban Stormwater BMP Performance Monitoring,” also downloadable from the project website. The manual is intended to help achieve stormwater BMP monitoring project goals through the collection of more useful and representative rainfall, flow, and water quality information. Many of the recommended protocols (particularly those for reporting monitoring, watershed, and design information) are directly related to requirements of the Stormwater BMP Database. The manual focuses primarily on the collection, reporting, and analysis of water quantity and quality measurements at the heart of quantitative BMP efficiency projects.

BMP Performance Assessment Approaches

To develop standardized BMP performance protocols, the project team identified a wide variety of measures that had been used historically to assess BMP performance, resulting in wide variations in reported BMP effectiveness. Additionally, the commonly used, simplistic approach of assessing BMP performance based only on the percent difference between mean concentrations (i.e., percent pollutant removal) was determined to be misleading under many, if not most, conditions.

As a result, the project team selected a BMP performance assessment approach that fully accounts for the statistical characteristics inherent to stormwater quality data. For example, the project team determined that the lognormal distribution was most representative for describing the majority of water quality data examined. Using the log-transformed data as a starting point, the project team selected a variety of statistical plots to describe the data based on their ability to quickly and accurately depict BMP efficiency and convey information that is statistically relevant (e.g., central tendency, confidence in mean values, and variability).

Methods were selected for application to individual BMP studies, as well as for application to groups of BMPs. The results are summarized in technical reports downloadable from the project website, including “Development of BMP Performance Measures” (1999) and periodic BMP analysis summaries, which were last updated in May 2007.

Ultimately, the BMP database team recommends that BMP performance be described as:

• The amount of runoff that is “prevented” (reduced) in the BMP.
• The amount of runoff that is treated (vs. bypassed or relatively untreated).
• The concentration characteristics of the treated effluent.

Where hydromodification is an issue, a fourth measure is the ability of the BMP to reduce downstream erosion impacts. These measures form the basis of a much more robust way to evaluate performance, as well as to be able to predict the potential effectiveness of BMPs in watershed planning efforts.

Data Interpretation/Findings

Data entered in the International BMP database are analyzed on an individual BMP basis, as well as according to BMP categories. Downloadable outputs from the database include:

• Individual BMP summaries (PDFs): Analysis results for individual BMP studies can be downloaded in PDF format through online searches of the Database.
• Queries for various data subsets (Excel spreadsheets): Most requests that the project team receives focus on water quality and flow data. These outputs are conducive to an Excel spreadsheet-type output. Currently, two spreadsheet formats are available for download from the website: 1) a table containing all storm events for BMPs meeting analysis criteria, and 2) a table containing the statistical analysis results for the BMPs. Users can download these tables and use Excel’s auto-filter feature to focus on analyses of interest or BMP types of interest, for example. The spreadsheets are simple and easy to use and do not require extensive knowledge of database usage.
• Overall database in Microsoft Access: This format is appropriate for individuals with database usage skills, who are able to create their own custom queries. The primary audience for the overall database has been researchers at universities.

In order to draw scientifically defensible conclusions regarding BMP performance, statistically valid sample populations are needed. As a result, conclusions drawn to date from analysis of the database have been carefully limited. The “Data Evaluation Report” (2000) contains detailed descriptions of analysis and findings conducted through 2000, with key preliminary findings regarding BMP performance, including:

• BMP performance should not be based on comparisons using percent removal alone. It may be appropriate to use this measure only on carefully chosen data subsets such as “dirty” sites.
• The chosen performance evaluation method can significantly affect reported pollutant removal efficiencies. For example, some BMP types may have been historically mischaracterized as less effective because of cleaner influent. Most BMPs and biochemical processes will exhibit lower percent removals when the concentrations are low in the influent.
• Retention (wet) ponds and wetland basins are not well represented by storm-by-storm comparisons because paired inflow and outflow samples often are not from the same event when the permanent pool size is relatively large compared to the storm events entering the facility.
• Effluent quality is useful for characterizing the effectiveness of the BMP; however, it is still important to determine whether the BMP had a statistically significant effect on water quality at each test site.
• Hydrologic and hydraulic performance that accounts for changes in runoff volumes within the BMP and bypass/ineffective treatment flows is critical for assessing overall BMP performance. These factors are relevant not only from a water quality perspective, but also with regard to downstream channel stability.
• Downstream response and biological/habitat assessment may be a better gauge of long-term BMP effectiveness for an area than pollutant removal efficiencies alone, but there is insufficient information at this time to suggest which BMPs provide the best mitigation of urbanization effects on the downstream receiving waters.
• More data is needed for sound statistical analysis, particularly for BMPs other than ponds and wetlands.

Since 2000, the database has grown considerably, allowing addition of several preliminary conclusions that will be integrated into future reports. The cumulative results for the database findings through 2006 were being completed at the time this article was written and will be available to the public as of May 2007. In the interim, these findings have been disseminated at conferences and include:

• Results of the analyses of the now expanded database have reinforced the initial findings that BMPs are best described by the degree to which they reduce runoff volumes; the degree to which they treat runoff (e.g., how much runoff bypasses the BMP); and the quality of effluent achieved.
• The BMP performance descriptions recommended by the BMP database team can be used to 1) assess the effluent concentrations that BMPs are able to achieve; 2) assess effects of BMPs on total loadings (TMDLs); and 3) identify the frequency of potential exceedances of water quality criteria or other targets.
• The analysis results show that the effluent quality of various BMP types can be statistically characterized as being different from one another for many pollutant types.
• Some design parameters (e.g., sizing relative to incoming measured storm volumes) may be statistically significant with regard to performance of BMPs.
• Additional work is needed to link BMP performance to the physical, chemical and biological impacts of stormwater on receiving waters. The current database provides little information relative to physical/biological impacts.
• Monitoring of BMPs located in cold regions is needed. The majority of the database contains BMPs in warm climates and data collected during warm periods in cold climates.

Conclusion

The International Stormwater BMP Database project provides tools useful for those desiring a better understanding of BMP monitoring and performance, those in need of tools to store and manage BMP data, and those in need of guidance regarding BMP performance data interpretation. The project team operates a clearinghouse that provides assistance to the technical community on these topics and continues to actively seek new BMP studies for inclusion in the database. Feel free to contact any of the authors for more information or visit the project website (www.bmpdatabase.org).

About the Authors:

Jane Clary is a Senior Water Resources Scientist for Wright Water Engineers in Denver and has served as the Database Project Coordinator for the past decade ([email protected]). Jonathan Jones, P.E., is CEO of Wright Water Engineers in Denver and Co-principal Investigator for the Database Project ([email protected]). Eric Strecker, P.E., is a Principal with GeoSyntec Consultants in Portland, OR, and a Co-principal Investigator for the Database Project ([email protected]). Marcus, Quigley, P.E., is an Associate with GeoSyntec Consultants in Boston, MA, and manages the BMP performance analysis for the Database Project ([email protected]).