Editor’s Note: Today’s guest post is by Katie Einhorn and Steph Pollock, respectively Peer Review Manager and Associate Publisher at the American Psychological Association, and Nick Paolini, Product Manager at Aries Systems.

Lack of representation among editorial leadership, editorial boards, and authors remains a key barrier to promoting an inclusive and equitable scholarly publishing industry (see also: Hanson, et al., 2020; Roberts, et al., 2020; and Roh, et al., 2020). The systemic exclusion from scholarly communications of groups including Black, Indigenous, and People of Color (BIPOC), women, and gender non-conforming people, members of the LGBTQ+ community, and members of the disability community is harmful to both the integrity of the scientific review and publication process and to those individuals’ academic careers. As Charlotte Roh and colleagues astutely note in their recent book chapter on social justice in scholarly communications, “lack of publication causes an erasure of voices from our academic institutions, our scholarly record, and our culture and knowledge at large.”

An important facet of the American Psychological Association (APA) Publishing’s equity, diversity, and inclusion framework is to promote a diverse community of authors, editors, and reviewers by dismantling the systemic and institutional barriers that historically excluded scholars and researchers face. But to do so, we first must be able to accurately measure how our authors, reviewers, and editors are represented in our scholarly journals.

We’ve therefore been working with Aries Systems to roll out an improved mechanism for collecting demographic data in our peer review system, Editorial Manager® (EM). In this interview, we share what we did, why, how, and what this means for other publishing organizations.

illustration of pie chart composed of people

EM has had the capacity to collect demographic information from registered users for about a year – what does this entail?

Nick Paolini: We have recently expanded the EM Additional People Details (APD) functionality. APD fields are customizable metadata fields that can be configured to store people-related information in the system. These fields have been enhanced to allow for more customization in response types, allowing journals to learn more about their key audiences.

Katie Einhorn: We’ve worked with Aries on this feature over the last few years to ensure we could capture the full breadth of a user’s identity. We wanted to be certain that users could select multiple options in each category.

Previously, our authors, reviewers, and editors could only select “male”, “female”, or “other” for gender, as well as affirmative responses for minority or disability status, thus giving them little to no way of providing true demographic information. With this EM feature update, we are now able to ask users about their racial and ethnic identity, gender identity, sexual orientation, and disability status, as well as level of degree and year of degree earned to help us find early career researchers. Most importantly, this update allows us to use checkboxes instead of radio buttons – so our respondents can select more than one answer per question. The screenshot below is a summary of the questions (and response options) now available in our peer review system.

screenshot of author questionnaire

Can you talk a bit about how APA decided on these specific questions and format?

Katie: In 2017, we developed a diversity and inclusion survey to better understand the demographics of APA Journals participants, and to identify and improve any gaps in representation across the network of authors, reviewers, and editors until the EM release. The survey was deployed once in spring 2018 and again in spring 2021, reflecting user demographics in 2017 and 2020, respectively.

The survey was composed of questions on racial and ethnic categories, gender identity, sexual orientation, disability status, country of residence, and field/career stage. The structuring of these questions was reviewed and edited by APA staff liaisons from the Committee on Ethnic Minority Affairs, the Committee on Sexual Orientation and Gender Diversity, and the Committee on Disability Issues in Psychology in fall 2017, with updates made in spring 2021.

Steph: Recently, APA’s own Equity, Diversity, and Inclusion (EDI) Office created an even further revised template based on discussions with above groups, which will allow us to collect consistent information from our membership and journal audiences. As Katie mentioned, we updated our own questions to reflect our EDI office’s format earlier this year. Our goal is to create a fully inclusive field of options for our users, so they know they are being accurately and fully represented.

We also recognize that “demographics” can take on different meanings in different countries and communities. As APA’s members are primarily based in the United States, our demographic questions are U.S.-centric. The French-based Organisation for Economic Cooperation and Development (OECD) published a terrific white paper on how different countries collect diversity statistics, and we welcome comments on this post from folks at organizations outside the U.S. about your organization’s approach and/or how you might use or adapt these kinds of questions for your communities.

Are the questions required? If they’re not required, how will you encourage uptake?

Katie: The questions are entirely optional, so we’ll encourage uptake in a few ways. For example, our editorial teams have been involved in this process from the start and are committed to our broader EDI initiatives throughout the journals program, so they asked their own editorial boards to complete this information as soon as it was rolled out.

Steph: We’ll also be doing a broader marketing campaign to all contributing users in the fall, focusing on how we must first accurately measure how our authors, reviewers, and editors are represented in our journals, a key part of APA’s equity, diversity, and inclusion framework.

Understandably, many people who identify as members of historically excluded groups can be hesitant to share their personal demographic information for a variety of reasons: the fear of backlash, harm, or misuse of their personal information; concern of being overburdened with requests to contribute to the journal; or because they want to protect their personally identifiable information. It’s important that when collecting this kind of information, organizations are transparent about how they plan to protect and use the data to support – rather than harm – members from historically excluded groups.

Demographic information is, of course, sensitive and protected data. Can you tell us a bit about the privacy and data protection policies in place?

Nick: Editorial Manager has been carefully designed to limit and control access to data, going beyond what is required to be compliant with international standards. In addition to these security measures, access to personal data is limited to those needing to perform the processing. By leveraging RoleManager permissions in the System Administrator functions, journals can restrict access to key personnel only.

Katie: To comply with the General Data Protection Regulation (GDPR, Europe’s data privacy and security law) and APA’s commitment to privacy of user information, the responses to these questions are only visible to the person completing their own profile, and to the EM System Administrators, which at present includes only three managers in the Journals department (myself included). We will only ever share this demographic data in the aggregate, and no specific information will be linked to a specific user. We’ve also updated the instructions on the user profile itself to make it clear how this information will be accessed and shared.

What will you do with this data? How will it inform future EDI efforts?

Katie: In the near future, APA aims to get a clear picture of the makeup of our editorial boards and the authors who contribute to our journals, and use the anonymized, aggregate data to help our journal editors set annual goals to develop more representative editorial boards and author pools.

Steph: Another long-term goal is to be able to identify potential biases in the peer review process by assessing things like acceptance and rejection rates for authors from different demographic groups. For instance: for journals with unmasked review policies, are authors more likely to get favorable reviews from reviewers who identify with similar demographic groups? Are research articles on BIPOC or other historically excluded populations more or less likely to be accepted, and are they being sufficiently reviewed by scholars who represent those communities? How has the COVID-19 pandemic affected the long-term research output from authors who identify as members of historically excluded groups? We hope we can answer these and other questions by looking at the patterns in the peer review data, itself.

What does this update mean for other organizations?

Steph: One of the most exciting things about this update is that any organization using Editorial Manager now has access to these improvements. We also know demographic data collection is a priority for many groups, already. For example, the Royal Society of Chemistry’s joint commitment for action on inclusion and diversity in publishing says that collecting and reporting on diversity data will be one of several actions for the group’s 44 publishing organization signatories. The American Speech-Language-Hearing Association (ASHA) also just announced an exciting new effort to use these EM updates to improve their own data-collection efforts, too. Our hope is that by improving how we collect this information, we can promote a more equitable and representative scientific process for our fellow members of the scholarly publishing community.

We recognize that having a representative editorial board and pool of authors is one of the best ways to ensure integrity in the scientific review and publication process. Journal content should reflect the full composition of the communities the journal looks to serve, and a diverse editorial board and author pool can help assure those outcomes. We’re excited to see how other publishing organizations will use and build on these efforts, too.

Katie Einhorn

At the time of this writing, Katie Einhorn (pronouns: she/her) was a Peer Review Manager at the American Psychological Association (APA), where she served as a Journals program operations and project manager, guiding the strategy for peer review and implementing best practices for both open science and equity, diversity, and inclusion (EDI) in peer review. Katie worked closely with Aries to implement process improvements that streamline APA journals based on individualized needs, drive strategic goals of the organization, and sustainably scale the peer review program. Katie now works in a knowledge management role for GetUpside, a startup that personalizes brick-and-mortar commerce to help communities thrive.

Steph Pollock

Steph Pollock (pronouns: she/her) is the Associate Publisher, Community Initiatives at the American Psychological Association (APA), where she drives strategy and publishing partner engagement for the APA Journals program and serves as the Journals program lead for equity, diversity, and inclusion (EDI) initiatives. With more than ten years of experience in scholarly publishing and a master's degree in publishing from The George Washington University, Steph is committed to improving equity in both the field of psychology and in the scholarly publishing industry. She is a co-author on the C4DISC Antiracism Toolkit for Organizations, a member of the Association for Women in Psychology, and has served on SSP’s Diversity, Equity, and Inclusion Committee since 2018.

Nick Paolini

Nick Paolini (pronouns: he/him) is a Product Manager at Aries Systems where he implements third party integrations and user experience improvements within Editorial Manager. In addition, he develops accessibility improvements to ensure the system is inclusive and usable for all users.

Discussion

7 Thoughts on "Guest Post — Introducing Demographic Questions during Manuscript Submission at the American Psychological Association"

Congratulations to the authors of this post. This article speaks to an innovation that we all should consider adopting/adapting/embracing! For our journals, we’ve been discussing adding “Mx” in addition to male/female in ScholarOne but this article suggestions so many more options to better know and understand our authors, editors, reviewers. Thank you TSK for sharing this post and trailblazing in such an important aspect of scholarly publishing!

Thank you, Phaedra, for your support and enthusiasm! It’s great to hear that your team is thinking about this important work, too.

I was a little horrified when I heard you were doing this, but your reasons and details of implementation seem to be a positive starting point for publishers to reflect on where they are on DEI. If it will help you start giving an academic platform to hear the research of BIPOCs, then I think you will have made a small but important step. More important might be to fund these researchers as an ongoing priority.

You make a good and important point, Scott, that collecting demographic data should only be a starting point, not the end game. Our vision is that you can’t understand where you need to go until you know where you’ve been; measuring representation in our journals is an important first step, but it won’t fix the systemic issues (including funding inequities, as you note) alone.

Thank you for this post, and for the details about your methodology and implementation. I appreciate that you’re willing to share information about this project with the wider publishing community.

Thank you for this important and informative post. I hosted a session yesterday at the #ISMTEGlobal virtual event on this topic: “Using Data to Promote DEI: You Can’t Improve What You Don’t Measure.” Teo Pulvirenti, of the American Chemical Society, discussed the survey they used with their community and Anna Jester, of eJournalPress, described the capabilities of their submission and peer review system to collect and securely store demographic data. There are so many considerations to ensure these data are properly collected, stored, and used and do not result in any harm to the individuals providing the information.

Hi Kristen, Thats my concern here too – as a person of colour -I’d be concerned with how this data was used by publishers- it might indicate where there are disparities in how peer review is conducted as mentioned, but if for example; aggregate data was used train a machine learning model -to predict “likely outcomes” for manuscripts, based on my under represented demographic- without equity, providing this, can be inherently harmful. – Michaela

Leave a Comment