Editor’s Note: Today’s guest post is by Katie Einhorn and Steph Pollock, respectively Peer Review Manager and Associate Publisher at the American Psychological Association, and Nick Paolini, Product Manager at Aries Systems.

Lack of representation among editorial leadership, editorial boards, and authors remains a key barrier to promoting an inclusive and equitable scholarly publishing industry (see also: Hanson, et al., 2020; Roberts, et al., 2020; and Roh, et al., 2020). The systemic exclusion from scholarly communications of groups including Black, Indigenous, and People of Color (BIPOC), women, and gender non-conforming people, members of the LGBTQ+ community, and members of the disability community is harmful to both the integrity of the scientific review and publication process and to those individuals’ academic careers. As Charlotte Roh and colleagues astutely note in their recent book chapter on social justice in scholarly communications, “lack of publication causes an erasure of voices from our academic institutions, our scholarly record, and our culture and knowledge at large.”

An important facet of the American Psychological Association (APA) Publishing’s equity, diversity, and inclusion framework is to promote a diverse community of authors, editors, and reviewers by dismantling the systemic and institutional barriers that historically excluded scholars and researchers face. But to do so, we first must be able to accurately measure how our authors, reviewers, and editors are represented in our scholarly journals.

We’ve therefore been working with Aries Systems to roll out an improved mechanism for collecting demographic data in our peer review system, Editorial Manager® (EM). In this interview, we share what we did, why, how, and what this means for other publishing organizations.

EM has had the capacity to collect demographic information from registered users for about a year – what does this entail?

Nick Paolini: We have recently expanded the EM Additional People Details (APD) functionality. APD fields are customizable metadata fields that can be configured to store people-related information in the system. These fields have been enhanced to allow for more customization in response types, allowing journals to learn more about their key audiences.

Katie Einhorn: We’ve worked with Aries on this feature over the last few years to ensure we could capture the full breadth of a user’s identity. We wanted to be certain that users could select multiple options in each category.

Previously, our authors, reviewers, and editors could only select “male”, “female”, or “other” for gender, as well as affirmative responses for minority or disability status, thus giving them little to no way of providing true demographic information. With this EM feature update, we are now able to ask users about their racial and ethnic identity, gender identity, sexual orientation, and disability status, as well as level of degree and year of degree earned to help us find early career researchers. Most importantly, this update allows us to use checkboxes instead of radio buttons – so our respondents can select more than one answer per question. The screenshot below is a summary of the questions (and response options) now available in our peer review system.

Can you talk a bit about how APA decided on these specific questions and format?

Katie: In 2017, we developed a diversity and inclusion survey to better understand the demographics of APA Journals participants, and to identify and improve any gaps in representation across the network of authors, reviewers, and editors until the EM release. The survey was deployed once in spring 2018 and again in spring 2021, reflecting user demographics in 2017 and 2020, respectively.

The survey was composed of questions on racial and ethnic categories, gender identity, sexual orientation, disability status, country of residence, and field/career stage. The structuring of these questions was reviewed and edited by APA staff liaisons from the Committee on Ethnic Minority Affairs, the Committee on Sexual Orientation and Gender Diversity, and the Committee on Disability Issues in Psychology in fall 2017, with updates made in spring 2021.

Steph: Recently, APA’s own Equity, Diversity, and Inclusion (EDI) Office created an even further revised template based on discussions with above groups, which will allow us to collect consistent information from our membership and journal audiences. As Katie mentioned, we updated our own questions to reflect our EDI office’s format earlier this year. Our goal is to create a fully inclusive field of options for our users, so they know they are being accurately and fully represented.

We also recognize that “demographics” can take on different meanings in different countries and communities. As APA’s members are primarily based in the United States, our demographic questions are U.S.-centric. The French-based Organisation for Economic Cooperation and Development (OECD) published a terrific white paper on how different countries collect diversity statistics, and we welcome comments on this post from folks at organizations outside the U.S. about your organization’s approach and/or how you might use or adapt these kinds of questions for your communities.

Are the questions required? If they’re not required, how will you encourage uptake?

Katie: The questions are entirely optional, so we’ll encourage uptake in a few ways. For example, our editorial teams have been involved in this process from the start and are committed to our broader EDI initiatives throughout the journals program, so they asked their own editorial boards to complete this information as soon as it was rolled out.

Steph: We’ll also be doing a broader marketing campaign to all contributing users in the fall, focusing on how we must first accurately measure how our authors, reviewers, and editors are represented in our journals, a key part of APA’s equity, diversity, and inclusion framework.

Understandably, many people who identify as members of historically excluded groups can be hesitant to share their personal demographic information for a variety of reasons: the fear of backlash, harm, or misuse of their personal information; concern of being overburdened with requests to contribute to the journal; or because they want to protect their personally identifiable information. It’s important that when collecting this kind of information, organizations are transparent about how they plan to protect and use the data to support – rather than harm – members from historically excluded groups.

Demographic information is, of course, sensitive and protected data. Can you tell us a bit about the privacy and data protection policies in place?

Nick: Editorial Manager has been carefully designed to limit and control access to data, going beyond what is required to be compliant with international standards. In addition to these security measures, access to personal data is limited to those needing to perform the processing. By leveraging RoleManager permissions in the System Administrator functions, journals can restrict access to key personnel only.

Katie: To comply with the General Data Protection Regulation (GDPR, Europe’s data privacy and security law) and APA’s commitment to privacy of user information, the responses to these questions are only visible to the person completing their own profile, and to the EM System Administrators, which at present includes only three managers in the Journals department (myself included). We will only ever share this demographic data in the aggregate, and no specific information will be linked to a specific user. We’ve also updated the instructions on the user profile itself to make it clear how this information will be accessed and shared.

What will you do with this data? How will it inform future EDI efforts?

Katie: In the near future, APA aims to get a clear picture of the makeup of our editorial boards and the authors who contribute to our journals, and use the anonymized, aggregate data to help our journal editors set annual goals to develop more representative editorial boards and author pools.

Steph: Another long-term goal is to be able to identify potential biases in the peer review process by assessing things like acceptance and rejection rates for authors from different demographic groups. For instance: for journals with unmasked review policies, are authors more likely to get favorable reviews from reviewers who identify with similar demographic groups? Are research articles on BIPOC or other historically excluded populations more or less likely to be accepted, and are they being sufficiently reviewed by scholars who represent those communities? How has the COVID-19 pandemic affected the long-term research output from authors who identify as members of historically excluded groups? We hope we can answer these and other questions by looking at the patterns in the peer review data, itself.

What does this update mean for other organizations?

Steph: One of the most exciting things about this update is that any organization using Editorial Manager now has access to these improvements. We also know demographic data collection is a priority for many groups, already. For example, the Royal Society of Chemistry’s joint commitment for action on inclusion and diversity in publishing says that collecting and reporting on diversity data will be one of several actions for the group’s 44 publishing organization signatories. The American Speech-Language-Hearing Association (ASHA) also just announced an exciting new effort to use these EM updates to improve their own data-collection efforts, too. Our hope is that by improving how we collect this information, we can promote a more equitable and representative scientific process for our fellow members of the scholarly publishing community.

We recognize that having a representative editorial board and pool of authors is one of the best ways to ensure integrity in the scientific review and publication process. Journal content should reflect the full composition of the communities the journal looks to serve, and a diverse editorial board and author pool can help assure those outcomes. We’re excited to see how other publishing organizations will use and build on these efforts, too.