Earlier this month, the White House Office of Science and Technology Policy (OSTP) Scientific Integrity Task Force released a report titled, “Protecting the Integrity of Government Science.” Although general and overarching, the report draws attention to the particular challenges associated with artificial intelligence and machine learning.
The OSTP Report is a response to President Biden’s request directive ensure that lawmakers make decisions based on scientific evidence and data with the aim of restoring public trust in government.
The document identifies principles of scientific integrity and best practices aimed at ensuring that “science is conducted, managed, communicated and used in a way that preserves its accuracy and objectivity and protects it from suppression, manipulation and improper influence”.— including political interference.
The foreword to the report – signed by OSTP Director Eric Lander, OSTP Deputy Director for Science and Society Alondra Nelson and OSTP Deputy Director for Climate and Environment Jane Lubchenco – lists the five new principles that will guide the agency’s scientific integrity policy:
- Contestation. Science takes advantage of dissent within the scientific community to sharpen ideas and thinking. The ability of scientists to freely express legitimate disagreement that improves science should not be limited.
- Whole of government. Because evidence-based policymaking happens across government, science integrity policies should apply not just to “science agencies,” but to all federal agencies and departments engaged in science. the generation, analysis, communication and use of evidence, science and technology. These policies must also apply to all career employees, contractors and political appointments.
- Science at the policy table. For science to inform policy and management decisions, it must be understood and actively taken into account when making decisions. This requires that scientists actively participate in policy-making.
- Transparency in science sharing. Transparency underpins sound knowledge generation and promotes accountability to the American public. Federal scientists should be able to speak freely, if they wish, about their unclassified research, including to members of the press.
- Responsibility. Attacks on scientific integrity must be considered in the same way as attacks on government ethics, with comparable consequences.
These are based on the six original principles established by the obama administration[i] in 2009.
The report cites other key themes that scientific integrity policies should address, such as improving diversity, equity, inclusion and accessibility (DEIA) and the improvement of potential biases in artificial intelligence and machine learning.
The report acknowledges, “AI and ML algorithms can amplify inherent biases in the underlying data source and may contain their own inherent biases that lead to inaccurate findings, conclusions, and policy decisions. Lack of transparency in ML algorithms can undermine trust in the results generated and ultimately in science and government. The concentration of AI data and capabilities in the hands of the federal government and private sector organizations can create inequities in who can conduct cutting-edge research and who can access and use the results of that work.
The task force recommends that scientific integrity policies help ensure that:
- AI and ML do not amplify the biases inherent in the data they analyze or are trained on.
- Transparency is provided in ML algorithms.
- Quality of data used for AI and ML, including in its generation, sharing and use.
- Privacy considerations are built into AI and ML processes and privacy risks are mitigated.
- Transparency and access are provided to data and AI capabilities.
The OSTP plans to use the data from the report to create a near-term plan to evaluate and improve scientific integrity policies that agency leaders can deploy within their organizations.
The Scientific Integrity Working Group was formed in May 2021 and includes 57 members from 29 agencies. The task force “addresses short-term, high-priority actions to strengthen scientific integrity and also lays the foundation for longer-term coordination of federal agencies’ scientific integrity efforts.”
[i] The original six principles are:
- The selection and retention of candidates for science and technology positions in the executive branch should be based on the knowledge, credentials, experience and integrity of the candidate.
- Each agency should have appropriate rules and procedures to ensure the integrity of the scientific process within the agency.
- When scientific or technological information is taken into account in policy decisions, the information must be subject to well-established scientific processes, including peer review where appropriate, and each agency must appropriately and accurately reflect this information. in compliance with and application of the relevant legal standards.
- Except for information the disclosure of which is duly restricted under procedures established pursuant to law, regulation, executive order or presidential memorandum, each agency shall make available to the public scientific or technological discoveries or conclusions made taken into account or on which political decisions are based.
- Each agency should have procedures in place to identify and address instances in which the scientific process or the integrity of scientific and technological information may be compromised.
- Each agency should adopt such additional procedures, including any appropriate whistleblower protections, that are necessary to ensure the integrity of the scientific and technological information and processes on which the agency relies in its decision-making or uses or otherwise prepare.
