Close
Close

Rigorous evidence transforms federal decisions

RFP
Businessman discussing charts on a monitor Before the current COVID-19 calamity and since, federal agencies have been quietly building the foundation for expanding the collection and use of evidence to improve their performance. This includes appointing evidence officers, developing learning agendas and assessing agency evidence-building capacity. While some treated issues like this as a compliance exercise in the past, a global pandemic underscores the fact that evidence and data have never been so critical to our ability to move forward effectively.

That is where the government has an important role to play, by gathering data and looking at the evidence before making decisions about policies that must often shift in order to adequately address an evolving situation. We are all flooded with information in the news media and on the internet about COVID-19. It’s overwhelming, and it’s hard to know what to believe. Is a vaccine really around the corner, as some companies claim? Is a required mask the best way to safely re-open the economy? Is more federal stimulus needed, and for whom? That is where the government has an important role to play, by gathering data and looking at the evidence before making decisions about policies that must often shift in order to adequately address an evolving situation.

As we can all attest, COVID-19 has impacted every aspect of our lives — our health care system, economy, food supply, employment practices and unemployment rates, education, transportation, and the list goes on. We need to adjust policies regarding all of these areas and more. Therefore, we need reliable data and evidence.

We need to know about infection, treatment and mortality rates. We need to know how the federal stimulus is working to help the economy. We need to know about food and financial insecurity among our citizens, as well as mental health impacts and social equity concerns. And we need to know if the things we are doing to slow the spread are working — are there enough tests, medical-grade PPE and citizens wearing masks and social-distancing to get this under control? These questions must be considered while monitoring the progress of the scientific community that is working feverishly on vaccines and treatments.

How can government help us get our arms around all these needs? Government agencies must step up their arbitration of evidence about effective practices in the current crisis and in every other domain in which they have expertise. It is the perfect time to add questions about what works to agency learning agendas. What works in slowing the spread of COVID-19? What works in potentially curing COVID-19? What works in helping our nation recover from COVID-19’s economic impact? What works in ensuring a culture of justice and equity in law enforcement?



Shining examples of injecting reliable evidence of effectiveness into the debate do exist. The Department of Education’s Institute for Education Sciences collected and posted helpful results of research that evaluated the effectiveness of distance education practices and products. On its What Works Clearinghouse website, you can find 22 evaluations that meet the Institute’s guidelines “without reservations.”

The Psychonomic Society formed the Behavioral Science Response to COVID-19 Working Group to “disseminate evidence-based recommendations in areas where behavioral science can make a positive contribution” to the COVID-19 pandemic.

The Data Foundation launched the COVID Impact Survey, a statistically valid survey that provides national and regional statistics about physical health, mental health, economic security and social dynamics in the United States.



And evidence advocates like Project Evident, the Pew Charitable Trusts, and Results for America have quickly created websites to direct their stakeholders to badly needed sources of evidence-based information.

But most would agree that there is a scarcity of rigorous evidence to consult when deciding what to do in response to the many challenges caused by the pandemic. While it may be difficult not to jump to conclusions when early data indicates a positive result, or even when rigorous evaluations demonstrate significant impact, without solid data-backed evidence, nothing is ever as certain as we would hope.

Those recommendations are still relevant — some would say urgently so — today, like establishing a National Secure Data Service to facilitate access to data, finalizing rules concerning the Evidence Act, and funding the act’s implementation. The bipartisan Commission on Evidence-Based Policymaking’s The Promise of Evidence-Based Policymaking in 2017 outlined a series of recommendations for how the federal government can provide the infrastructure for secure access to data, the mechanisms to improve privacy protections and transparency about the usage of data for evidence-building, and the institutional capacity to support evidence-building. Those recommendations are still relevant — some would say urgently so — today, like establishing a National Secure Data Service to facilitate access to data, finalizing rules concerning the Evidence Act, and funding the act’s implementation.

There are practical actions that leaders, program managers and others interested in finding what works can take in the short term.

  • Review available evidence addressing the problem you are trying to solve — what data exists showing that something worked?
  • Find expert evaluators or, better yet, the agency’s evaluation officer, and ask them to assess the rigor of available evidence.
  • Consult others trying to address the same or similar problem and ask about the evidence they’re relying upon.

These are the first steps to take when transforming into an evidence-based culture.

Government has an important role to rapidly, credibly report reliable data and results that can help address the massive challenges we are facing as a nation. Agencies are at the early stages of transforming their cultures into evidence-based ones. Government must support those efforts and take the necessary steps to make gathering and analyzing data — the evidence we seek for the decisions that need to be made — a top priority.

Originally published on July 14, 2020, in Federal News Network

Contact:

Robert SheaRobert Shea
National Managing Principal, Public Policy
T +1 703 637 2780