NAS Says OMB Risk Assessment Document "Fundamentally Flawed"

Jan 11: A draft bulletin issued by the White House Office of Management and Budget (OMB) prescribing technical standards for Federal risk assessments is "fundamentally flawed" and should be withdrawn, according to a new National Academy of Sciences' (NAS), National Research Council report. OMB requested that the Research Council review the bulletin.

Last January OMB issued the draft bulletin, which included a new definition of risk assessment and proposed standards aimed at improving Federal risk assessments [See WIMS 1/10/06]. Risk assessments are often used by the Federal government to estimate the risk the public may face from such things as exposure to a chemical or the potential failure of an engineered structure, and they underlie many regulatory decisions.

John F. Ahearne, chair of the committee that wrote the report said, "We began our review of the draft bulletin thinking we would only be recommending changes, but the more we dug into it, the more we realized that from a scientific and technical standpoint, it should be withdrawn altogether." The committee agreed with OMB that there is room for improvement in Federal risk assessments and that additional guidance would help. However, it concluded that the bulletin would not accomplish its stated goal of enhancing the technical quality and objectivity of Federal risk assessments. The Committee said, "OMB should instead issue a different type of bulletin that outlines goals and general principles for risk assessments, but that directs federal agencies to develop their own technical guidelines to meet those goals and principles. The new bulletin should draw on the risk assessment expertise that exists in federal agencies and the organizations that advise them."

In a release the Committee indicated that although the general thrust of the draft bulletin appears to be consistent with past expert recommendations on risk assessments, a number of specific proposals are inconsistent. It added that the bulletin attempts to move standards for risk assessment into "territory beyond what previous reports have recommended and beyond the current state of the science." Also, OMB's definition of risk assessment is "too broad and in conflict with long-established concepts and practices."

Many of the standards proposed in the bulletin are unclear, the report adds. In particular, OMB's proposal of separate standards for general risk assessments and "influential" ones is problematic because agencies may not know at the outset whether a risk assessment will be considered influential. The committee also took issue with the bulletin's definition of an "adverse health effect" because it implies that only clinically apparent effects should be considered adverse. They said, "This ignores a fundamental public health goal to control exposures well before they cause functional impairment."

The Committee said the bulletin also omits several topics, further limiting its usefulness. For example, OMB erred in focusing mainly on human health risk assessments while neglecting risk assessments of technology and engineered structures. The bulletin's incomplete and unbalanced approach to engineering, ecological, and other types of risk assessments contradicts its stated objective of improving the quality of risk assessment throughout the Federal government. The bulletin also gives little attention to the integral role of risk communication, the importance of default assumptions in conducting risk assessments, and the risks faced by sensitive populations, such as children and pregnant women.

The Committee said, "OMB has not established a baseline of each agency's proficiency at conducting risk assessments, nor estimated the cost of implementing the bulletin. However, the committee determined -- based on comments from the agencies and its own knowledge of risk assessment practices -- that some aspects of the bulletin's implementation could be beneficial but that the costs are likely to be substantial. Overall, the committee concluded that the potential for negative impacts on the practice of risk assessment in the federal government would be very high." The Committee noted that risk assessment is not a monolithic process or single method, adding that "one size does not fit all."

House Chairmen Bart Gordon (D-TN, Science & Technology), John D. Dingell (D-MI, Energy & Commerce), Henry A. Waxman (D-CA, Oversight & Government Reform), and James Oberstar (D-MN, Transportation & Infrastructure) issued a joint release saying they agreed with the NAS conclusions that the document was fundamentally flawed. The Members, who have been critical of the document, expressed their concerns in a letter to NAS last May when they initiated their review of the Risk Assessment Bulletin.

Access a release from NAS (click here). Access the complete report on-line and a summary report (click here). Access the complete 26-page OMB Proposed Risk Assessment Bulletin document (click here). Access the Members joint release (click here). [*All]