Scholarly information discovery is a broad topic, going well beyond search, and it varies tremendously across different user types. Some students are looking for three references to meet the requirements of a (poorly designed) undergraduate assignment. Most researchers are striving to maintain current awareness of the new scholarship in their fields. Whatever the research task at hand, scholarly publishers strive to support the discoverability of their works through all such workflows, while libraries strive to support the researcher’s discovery across all relevant works.

I have been tracking one kind of discovery – what I will call the quest for comprehensiveness – that is widespread among researchers but seems comparatively quiescent in professional discussion about supporting researcher needs. Researchers need to be confident that not only have they identified many sources on a given topic, but actually that they have located all of the sources that might be of conceivable relevance. This quest for comprehensiveness is expressed in a number of different types of projects that we do not always see as related to one another.

Scholarly papers and monographs typically provide at least some review of the related literature. These literature reviews are conducted in the course of research projects and grant applications. In addition to situating the present work in its intellectual context, they often have signaling purposes to potential reviewers, which is the topic for a discussion all its own.

There is also the standalone review article. In some fields, these are more typically written by a graduate student, often as a dissertation chapter, while in other fields these activities are taken on by more advanced researchers or with the extensive support of a librarian. Regardless of the exact contributors, preparing a review article is yeoman’s work for one’s field, providing bibliographic infrastructure to help others navigate a given topic. Review articles are so important that they are at the center of stand-alone periodicals in many fields.

The quest for comprehensiveness is not without its anxiety. For example, in a project about their research practices, historians told Ithaka S+R researchers again and again of a paradox.  Given vast digitization and new search tools, it was becoming easier for them to discover items on any given topic. But the same digital abundance was making it harder for them to feel confident that they had found everything they would need to find in order to reach and defend their research conclusions.

In another project several years ago, I recall interviewing a political scientist, who shared his approach to conducting a literature review. He ran all his searches through the current version of the citation management tool EndNote, through which he could connect to many popular content platforms. By using EndNote as the front-end for search, he was able to track not only the searches he had conducted but also their complete result sets. Subsequent searches could be filtered against previous searches’ result sets, allowing him to quickly look at only the newly discovered items. When sufficient search iterations successively retrieved no new results (and all relevant references in relevant articles had been pursued), he was able to satisfy himself that his work was done. While this workaround helped him, it is obviously not the optimal way to provide for this type of much-needed functionality.

I share these examples not because they will seem surprising to those who are familiar with research habits. Rather, I offer them up to raise the question that for me connects them: Would it be possible, I wonder, to develop a discovery tool that is designed not to find the best items but rather to provide some assurance that you hadn’t missed something?

There could be a variety of approaches. The basic feature of the EndNote method described above is to allow result sets to be filtered against a list of all the items that a researcher has already reviewed and deemed to be relevant, and irrelevant, for the project. This could ideally work across platforms, so that an item that had previously been discovered via PubMed would be available to one’s filter when searching at Scopus.

I also wonder if bringing co-citation analyses into discovery environments would be helpful. In such a case, a researcher could somehow upload to a discovery tool a list of the references seen as most relevant on a topic. An automated analysis of what is most likely to be cited with those works would be returned – not unlike the way that a social network can suggest additional friends.

I offer these two approaches as more concrete examples of how discovery systems could be designed to help scholars in their quest for comprehensiveness, as they conduct literature reviews as well as other types of discovery processes. Are there opportunities for scholarly publishers, their platform providers, and perhaps other kinds of organizations as well, to support more effectively this type of research work?

See Part 2: Repackaging the Review Article