It's impossible to estimate the impact of many nutrient reduction efforts because of inconsistent and - often poor - efforts to track their implementation, a new report says.

A panel of scientists said existing programs undercount some nutrient control actions, double-count others and frequently count them differently from state-to-state.

The issue is of "paramount importance," the panel said, because the Bay Program relies on that data to estimate progress toward meeting its nutrient reduction goals. "The current accounting cannot on the whole be viewed as accurate," said the report from the National Research Council.

The nine-member panel did credit the Bay Program for starting to address the problem. And it expressed optimism that recent actions, including setting two-year cleanup milestones, would boost cleanup progress.

The panel warned that the cleanup efforts would be difficult and expensive, and likely take far longer to produce tangible water quality improvements than most people recognize. "If the public expects visible, tangible evidence of local and Bay water quality improvements in fairly short order, they will almost certainly become frustrated," the report said. It said the Bay Program needed to better explain, and attempt to address, uncertainties that surround its cleanup program.

The Bay Program, which had requested the review, pledged a written response within 90 days.

"While supporting the program's current efforts, the report also points out some critical challenges to consider in making decisions moving forward," EPA Regional Administrator Shawn Garvin said in a statement. Garvin chairs the Bay Program's Principals Staff Committee that includes the leaders of state environmental agencies and senior federal officials.

The report stems from a 2008 decision by the Chesapeake Executive Council which, in the wake of several critical evaluations of Bay Program cleanup efforts, called for an independent evaluation of key aspects of its nutrient reduction efforts. The council is the top policy-making body for the Bay cleanup effort, and includes the governors of Maryland, Virginia and Pennsylvania; the EPA administrator; the District of Columbia mayor; and the chair of the Chesapeake Bay Commission, which represents state legislatures.

The National Research Council, an arm of the National Academy of Sciences, an independent nonprofit that serves as an adviser on national scientific issues, was selected to conduct the two-year review, which was released May 4.

One criticism of the Bay Program has been its repeated failure to meet long-term goals. That led to a commitment to establish two-year cleanup "milestones," so states would be held accountable for near-term objectives.

The study panel said, "In general, the two-year milestone strategy should improve accountability and encourage re-evaluations and adjustments for Bay jurisdictions that are not achieving their goals."

But the panel cautioned that actions included in the first milestone goals, which were set in 2009 and are to be implemented by the end of this year, are "low-hanging fruit" and likely to be the least expensive or easiest reductions to achieve.

The panel concluded that significant improvements were needed in the ways that states track actions to control runoff, known as best management practices. The states annually report the number, and types, of BMPs implemented to the EPA, which feeds the information into a computer model, which then estimates the amount of nutrient reductions achieved.

Tracking is pivotal to the Bay cleanup process. The EPA at the end of last year established a Bay cleanup plan, known as a Total Maximum Daily Load, which established nutrient and sediment limits that would restore Bay water quality.

States are required to make plans to meet those goals, with cleanup objectives broken into two-year milestones. Accurate BMP tracking is fundamental to ensuring those plans are on pace to meet goals.

The BMP tracking issue is timely: Agricultural groups have charged that the EPA has dramatically underestimated the amount of nutrient reduction actions taken by farmers.

The report agreed that efforts by farmers, who sometimes don't report their actions to agencies, were sometimes undercounted. But it also said the current tracking systems sometimes double-count BMPs as multiple agencies often report the same project.

Other problems contribute to uncertainties. Many BMPs deteriorate, yet only Maryland has a process to identify and remove from tracking databases those practices that are no longer working or - in some cases - no longer exist.

The committee said it was unable to determine the magnitude of those errors, or even whether implementation of pollution control practices, taken as a whole, are overestimated or underestimated. "Some of these errors will likely cancel each other out, but there is substantial room for improvement," the report said.

States also differ in how they define BMPs. Most allow manure application to cover crops, while the Bay Program assumes no manure is applied. That can skew the effectiveness of the BMPs being reported.

The report noted that some states don't verify BMPs, although Virginia inspects up to 5 percent of BMPs annually, Maryland up to 7 percent, and New York inspects all reported practices.

Because of such problems, the panel said it was "unable to determine whether the actual data reported by each jurisdiction are reliable and accurate."

The report praised Bay Program efforts to update estimates of the nutrient control effectiveness of various BMPs based on available information. It said those refinements should continue as new information is gained.

Yet great uncertainties remain about real-world effectiveness of BMPs on water quality, and the report called for more monitoring in small watersheds to determine if nutrient control actions have had their anticipated impacts.

The panel said the region should explore the use of independent third parties to track and verify BMPs, and should collect greater detail about where they are located. Practices close to the water are generally most effective, but most data provided to the Bay Program merely lists the county where a BMP was installed. The panel said the Bay Program should consider incentives to encourage farmers to better report BMP implementation.

The panel acknowledged that all states are making efforts to improve tracking and accounting, and that the Bay Program is working to standardize reporting procedures. The EPA is also developing a more rigorous tracking and verification system for BMPs.

The amount of BMP implementation, and its impact on water quality, is only one uncertainty surrounding Bay cleanup efforts.

The report listed others.

Cleanup plans assume that farmers and others will respond to financial incentives or other tools intended to promote the implementation of pollution control efforts, even though other factors may be more important to their decision making.

Climate change looms as one of the largest uncertainties. Alterations in weather patterns could make attaining Bay goals more difficult, or even impossible, and they could dramatically change the effectiveness of various BMPs.

Another major uncertainty is how long it would take to clean up the Bay. Many on-the-ground nutrient control actions have significant "lag times" - often years or decades - between when they are installed and when they impact downstream water quality.

In addition, the panel cautioned that the Bay has so fundamentally changed over the years that it is unclear when - or if - it would attain desired water quality conditions even when nutrient goals are met.

"Responses by ecosystems, either terrestrial or aquatic, are just not that predictable," said Kenneth Reckhow, of RTI International, a nonprofit research organization in North Carolina, who chaired the panel. "They are too complex. These systems are far more complex than our most complicated models."

As a result, the panel said the Bay Program needs a more systematic approach to understand and reduce uncertainties related to cleanup efforts so it could make necessary course corrections. The Bay Program is in the process of developing a formal adaptive management framework that would allow it to alter its actions if necessary.

But, the report expressed skepticism that either the EPA or the states clearly understand adaptive management and its implications for water quality goals. "Truly embracing adaptive management requires recognition that the TMDL, load allocations, and possibly even water quality standards, might need to be modified based on what is learned," the panel said.

In fact, some aspects of the EPA's rigorous TMDL framework could discourage adaptive management. Because states may face sanctions from the EPA if they fail to meet two-year milestones, the report said there is a disincentive to experiment with alternative management techniques.

Instead, the report said, the EPA should explicitly support, without fear of penalty, "carefully designed" experiments with adequate monitoring and evaluation to explore management alternatives that could help meet goals.

The Bay Program relies heavily on complex computer models to determine where nutrients come from, how they affect Chesapeake water quality and how various management actions affect the amount of nutrient reaching the Bay.

The committee did not review the models, but called for an overhaul of the Bay Program's modeling program. Its models are complex and understood by only a few individuals who have such heavy workloads they have limited opportunities to work with the scientific community.

As a result, few outside scientists use the Bay Program's models, and are therefore unable to weigh in when disputes arise over the veracity of model results, such as a recent challenge by farm organizations to model credibility.

"Credibility of the models is essential if the [Bay Program] goals and strategies are to be accepted and have widespread support," the report said. The committee called for creating a Chesapeake Bay Modeling Laboratory to integrate scientists from research institutions that would develop multiple models.

Such a laboratory could also develop smaller-scale models to better understand issues that have been vexing, such as lag times, or how the Bay might respond to reduced nutrient levels.

Reaching nutrient and sediment goals will require a huge commitment from everyone in the watershed, and may be made more difficult by increases in human population, greater concentrations of livestock and farm animals, development and climate change, the report said.

The report went through a list of potential new cleanup actions, many of which are already being explored. They ranged from calling for additional air pollution controls to reduce airborne nutrient deposition to encouraging people to eat less meat, which takes large amounts of fertilizer to produce.

But the report raises great concern that support for cleanup efforts - both among the general public and elected officials - may wane in the face of steep cleanup costs and lack of tangible results.

Water quality improvements will almost certainly lag well behind the implementation of pollution control measures. "If these lags and uncertainties are not adequately explained, [Bay Program] partners will need to anticipate and be prepared to respond to the potential ramifications of an impatient or disillusioned public," the report said.

If the Bay isn't improving and the Bay Program is unable to present realistic explanations to the public, it could trigger loss of support and create pressure to change the goals - actions that could effectively undo some recent developments.

"Adoption of the two-year milestone approach was intended to overcome uncertainties associated with electoral cycles and leadership changes, but elections will continue to introduce new questions about commitment to Bay priorities," the committee said.

The report, "Achieving Nutrient and Sediment Reduction Goals in the Chesapeake Bay: An Evaluation of Program Strategies and Implementation," is available at the NAS website: