Many funders around the world are working to make their grant awards more equitable, diverse, and inclusive in order to create a level playing field – one where research is funded based on its merits rather than, for example, the Matthew Effect. Systemic and structural bias been shown to disadvantage people across a whole range of demographics and behaviors – race, ethnicity, and gender, of course, but also career stage, working pattern, neurotype, parenthood, and more. Eliminating this bias helps to ensure a fair and transparent assessment of the merits of an application, so that good ideas are taken forward no matter whose ideas they are. Efforts to do so are now all the more critical given the current crackdown on DEI in US federal agencies. So I’m especially happy to share this update on a previous post on the need for both institutions and funding organizations to improve DEI in the pre-award funding process – the beginning of the grant lifecycle (announcing opportunities, submitting, and reviewing applications), which, until recently, has been mostly overlooked. In that post I described the outcomes of a Wellcome-funded project at the Elizabeth Blackwell Institute (EBI) of the University of Bristol, for which MoreBrains Cooperative (full disclosure, my company) provided support, to identify opportunities to improve Equality, Diversity, and Inclusion (EDI) in pre-award funding processes. This led to the publication of a report containing 11 concrete recommendations. EBI and MoreBrains then worked with the Swiss National Science Foundation and the San Francisco Declaration on Open Research Assessment (DORA) to run a series of online engagement events with DORA’s funder discussion group. They identified three key areas of intervention in pre-award funding processes that would be both feasible and impactful:

1. Simplification of funding call structures
2. Changes to application processes to reduce likelihood of bias in outcomes
3. Improvements in training for reviewers and evaluators

Thanks to additional funding from the University of Bristol, during 2024 we were delighted to once again work with EBI and DORA on a follow-up project to develop concrete ways for funders to influence the design and delivery of their processes at the pre-application stage, to improve fairness and create more equitable outcomes. Our focus this time was on how these three interventions could be implemented, how practical barriers could be reduced, and how progress could be benchmarked.

We took a two-pronged approach to this work. First, we designed and delivered two virtual workshops for the DORA funder discussion groups, structured around each of the three interventions, to enable us to develop a roadmap for delivering the interventions. In parallel, we invited DORA funder discussion group members whose organizations are already working to address one or more of the interventions to help us develop case studies that illustrate some of the real-life changes being implemented to make funding processes fairer.

The main output of this work is an openly available, evidence-based toolkit for research funders to implement evidence-based changes to their funding call processes. It includes case studies from the European Research Council (ERC), the Luxembourg National Research Fund (Fonds National de la Recherche – FNR), and the UK’s National Institute for Health and Care Research (NIHR); as well as checklists to support each of the three interventions – simplifying call structures; implementing changes to reduce the likelihood of bias; and implementing improvements in training for reviewers and evaluators.

Many of the findings in the case studies are equally applicable for other types of organizations seeking to improve DEI — they are summarized below.

ERC Case Study

Having decided to take a structured look at research assessment in general, establish their own position, and consider possible changes to their evaluation processes, ERC’s Scientific Council then defined three tasks:

1. Decide which characteristics and qualities of the proposed project and the applicant should be considered
2. Decide how to evaluate those characteristics and qualities
3. Assign relative weights to the different characteristics and qualities.

This led ERC to place a stronger emphasis on the groundbreaking nature of the research ideas contained in the proposal, while also ensuring that the applicants have the expertise and capacity to successfully execute the project. To reflect this change, the ERC application template was substantially revised so, for example it now allows for any type of research output to be included, not just publications. Scientific excellence continues to be the prime factor in awarding grants, but applicants can now provide a more holistic and fuller account of their research career and contributions.

FNR Case Study

As part of its efforts to reduce biases during the proposal evaluation process, FNR has implemented a narrative-style curriculum vitae (CV), which allows applicants to include a broader range of their research outputs and outcomes for evaluation. This has had implications for reviewers and evaluators, as well as applicants, so FNR has updated their guidelines and also provides more training resources. FNR sees this as a long-term initiative, with the ultimate goal of helping the current generation of early career researchers to develop evaluation skills that are compatible with responsible research assessment in the future. Feedback from both applicants and evaluators already shows a general and sustained acceptance of these changes, and FNR will continue to collect data. Already, by shifting the focus from pure citation metrics, proposals are now focused on the proposed work, not the CV.

NIHR Case Study

NIHR’s Research Inclusion strategy represents a formal commitment to promote equality and diversity within the health and social care research landscape. One of the organization’s first steps was to understand stakeholders’ demographics, and to embed what they learned in the strategy. Establishing this initial data diversity baseline was fundamental to understanding the current makeup of the various groups of award-holders, applicants, and committees. Once this was accomplished, NIHR started making changes to the application process, for example, by making inclusion in applications a condition for funding. Like FNR, one of the ways that NIHR is supporting access to funding opportunities is through the introduction of narrative CVs for applicants – initially just for fellowships (which are individual awards), while conducting pilots for more complex funding calls (which involve teams, making narrative CVs more challenging).

Checklists

As well as creating specific checklists for each intervention type, we also developed a general checklist, which I’m sharing here (lightly edited), which can easily be adapted for a variety of applications and settings, not just funding.

• Clarify the challenge; what exactly are you trying to solve?
• How will solving this challenge improve the workings, mission, or culture of your organization? Identify an internal stakeholder or team to be responsible and accountable, and to drive the project. A champion at the leadership level will be especially helpful
• Is the challenge specific to your organization, or is it part of a wider problem within your sector? (If the latter, consider communicating with another organization or a group to gain a sense of where the general thinking is and increase the likelihood of success/acceptance within your sector)
• Benchmark the current situation (collect data, conduct surveys or focus groups)
• What internal resources are available? (e.g. people, technology, financial)
• Are any other internal resources available? For example, are there existing initiatives or groups within your organization that you could engage with to expedite the initiative?
• What external resources are available? For example, toolkits on the DORA or Research on Research Institute (RoRI) or other websites, examples of  other organizations working in this area, who could be approached for advice?
• What intervention options are available and feasible?
• Decide how to measure the impact of intervention(s) — would quantitative, qualitative, or both methods work best?
• Are there any potential downsides to implementing changes?
• Decide how to follow through the process, including checking workflows
• Is additional training required, and what form should this take — online, in-person, via interactive sessions or asynchronously?
• Do you have examples of best practices and case studies?
• Will other changes be needed,  e.g. to deadlines, to grant applications or other systems, to templates?
• What follow up actions will be needed, and what is the time frame for review?
• How will you address any issues that arise?
• What  opportunities will there be for users to provide feedback outside of the consultation processes? (e.g. feedback forms on websites, regular surveys, etc.)
• Develop a strong communications plan, which includes listening as well as disseminating)

If you and/or your organization want some inspiration on ways to make concrete improvements to DEI, I hope you’ll look at these materials in more detail. And please share your feedback in the comments, including other examples of good work in this space.