Setting Appropriate Performance Measures

List of Project Performance Measures

Printable Version of Performance Measures

EPA's Proposed Indicators for 2006 Report on the Environment (ROE 2006)

Performance Measure Requirements

All projects should identify their baseline measures, the defined community that they will reach, their improvement target(s), how far their project participants have moved along an established IPM transition gradient as a result of the project, and the performance measures they will use to indicate environmental improvement as a result of increased IPM adoption.

Baseline Data All projects need baseline data to show the degree of environmental improvement achieved as a result of the project. This data can either be provided in the proposal or early in the reporting cycles. For example, projects could compare baseline pesticide use data to post-project pesticide use data, summarized over project participants. Defined Community Projects should identify the population whose behavior they seek to change and provide estimates on how many will adopt those changes (for example, producers, consumers, food processing companies, etc.) IPM Transition Gradient Most projects should report how far their project participants have moved along an established IPM transition gradient as a result of the project. This is a measure that all regions can use for most projects to help evaluate progress. Performance Measures Each project should identify three or more performance measures to track. However, if the project chooses a direct environmental measure, then only one other surrogate measure is required. Direct environmental measurements provide the strongest evidence of impact(s) but may be expensive. In contrast, surrogate measures may have broader-based applications but require assumptions to be made about environmental impacts so are not as strong. If grantees locate projects in areas where environmental monitoring is already underway, they may be able to use that data. Projects should include a diversity of measures. For example, if the project includes a direct measure to track water quality, then the second measure should provide information about effects on a different media such as soil quality. Grant applicants should also use at least one project tracking measure. These measures help ensure that a project is progressing on schedule. Grantees should collect economic data whenever possible. If they demonstrate that growers can save money by adopting a practice or program, then it is much more likely that behavioral change by growers will be sustainable and not just transient during the life of the project. Grantees should be encouraged to collect efficacy data for the same reason. Also see Baseline Data Sources

Other IPM Performance Measures

IPM Roadmap: Sets performance measures for improving economic benefits, reducing human health risks and reducing adverse environmental effects. IPM Performance Planning and Reporting System The web-based performance planning and reporting system (PPRS) collects information about IPM implementation at land grants and lists indicators of progress towards national IPM goals.

Environmental Indicator Models:
Overview
This fact sheet describes publications/webpages/email listservs that review, compare or discuss "Pesticide RIsk Indicators" and other "RIsk Ranking Tools" that could be applied to pesticides.
Models tested with field data: These models predict residue levels from pesticides based on pesticide, weather, crop, location and soils data and be used to compare conventional pesticide use to IPM programs.
WIN-PST:

WIN-PST homepage Pesticide environmental risk screening tool that NRCS field office conversvationists and others can use to evaluate the potential for pesticides to move with water and eroded soil/organic matter and affect non-target organisms. WIN-PST and other models: How model used by NRCS to predict risk to the environment by pesticides compares to other environmental indicator models.

Conveying Requirements to Grantees
In the Request for Proposals, ask grantees to identify their baseline measures, the defined community that they will reach, their improvement target(s) and the performance measures they will use to indicate environmental improvement as a result of increased IPM adoption. They should be asked to report baseline data in the first or second reporting cycle. You can include this requirement in Terms and Conditions, controlling the funding flow to ensure baseline data is collected and reported.

Quality Assurance Project Plans (QAPPS)
A QAPP is a written document that describes the quality assurance procedures, quality control specifications, and other technical activities that must be implemented to ensure that the results of the project or task to be performed will meet project specifications. EPA requires approved QAPPS (or equivalent documents) for all applicable projects and tasks involving environmental data to ensure the project and task is documented and reviewed before the work is started.
QAPPS Guidance and Examples From Region 1
EPA Quality Manual for Environmental Programs
Draft QAPPS for EPA regional grants.
Overview of QAPPS (from EPA’s Office of Water for volunteer water quality programs)

Measuring the Sustainability of Success
You may consider placing time delimiters on some of the measures we list to try to document that these changes occurred and were sustainable over time, especially in light of stressors like invasive species, pest resistance, etc. If you can include follow-up measures to evaluate the impacts of projects beyond their original funding cycle, you can use what you learn to craft new RFP's for better success.

Evaluating IPM Grants Programs:
The Center for Agricultural Partnerships analyzed the effectiveness of California EPA's Department of Pesticide Regulation Pest Management Alliance (PMA) program. The Appendix includes the survey and forms they used.

Reporting on Project Portfolios:
Example of final report on portfolio of projects from EPA Region 10