Public policy is created for a public purpose. The purpose at the heart of the Nelson Memo is unambiguous: the intended beneficiary of expanded public access is “the American public,” and the goal of such policies is to “allow all Americans to benefit from the returns on our research and development investments without delay.” Today, funding agencies, policy makers, publishers, universities, and other stakeholders are gripped with uncertainty about how best to translate the Nelson Memo’s call for expanded public access into coherent policy and workable business and service models. I wish to address a different question: Will we adequately achieve the public purpose of this policy guidance? And if, as I believe, we may not, then can we take this opportunity to address some of the most significant actual needs of the American public? Today, I focus on a use case familiar to many: the patient trying to learn about, and consider among treatment options for, a disease or medical condition.
This piece is drawn from a talk that I gave last week at a National Academies workshop. That workshop was intended to advise the Department of Health and Human Services about the public access policies that its constituent agencies will adopt as part of the Nelson Memo. These agencies — for example the NIH and FDA — include the principal biomedical research and regulatory bodies in the United States, so their policies will impact access for the American public to significant volumes of scientific, clinical, and public health research. My focus in this piece is therefore on biomedicine.
It is my contention that the outputs of biomedical research are not equally usable by all members of the American public. With the laudable increase in access to journal articles, it is now fairly simple to discover and access much of the biomedical literature. A simple search of Google or Google Scholar is enormously valuable for discovery. But that doesn’t make the biomedical literature usable to all. There are at least two reasons for this. For one, not everyone can establish the trustworthiness and authority of information resources, particularly given that we are awash in a sea of misinformation including on biomedical topics. And second, not everyone can distill or synthesize what they find into usable knowledge.
In using the biomedical literature, academic and professional researchers have numerous advantages that are worth stating plainly. They are the intended audience among authors for scientific writing and publications. They have knowledge of the field and of key research institutions and authors. They have knowledge of which publishers and journals are more trustworthy, selective, and high quality. They have expertise in critically reading the literature in their fields, the result of scientific and statistical literacies, among others. They have years of training in synthesizing what they find in order to apply it. And they have access to numerous specialized discovery and analysis tools. It is easy to take these advantages and others like them for granted, but they deeply shape the abilities of academic and professional researchers to use the biomedical literature.
Yet academic and professional researchers are not the intended beneficiaries of public access. As mentioned above, the Nelson Memo plainly states that the intended beneficiary is “the American public.” The American public is, understandably, typically not the intended audience of biomedical papers and publications. When members of the American public seek access to this literature, it is for other purposes, and unraveling the various particular use cases that exist is foundational to addressing them. One common use case, particularly with respect to biomedical literature, is as, or to advocate for, a patient trying to learn about, and consider among treatment options for, a disease or medical condition.
So many adults have had personal experience with this use case. Perhaps they have faced a complex medical decision, or someone they love has faced such a decision. Today, a common early step is to search the literature — through Google or Google Scholar — for information on the disease or condition, then try to read through the resulting articles from the biomedical literature.
If you have done this — as I have several times in recent years — it would be no surprise if you discovered dozens of scholarly articles on your disease or condition, and many more depending on the breadth of your search and the funding for research on that topic. And, having done this, it would also be little surprise if the items in this result set suggested a bevy of contradictory findings and recommendations. This is the biomedical literature: a conversation among scientists and clinicians, intended to build upon one another’s work and designed to stimulate future work.
Digging in, you might even find, as I did, a highly optimistic take on the least interventional approach possible, in the form of an article from a surgeon practicing in western China, presenting a case report drawn from half a dozen examples. No doubt such an article can be highly useful in presenting evidence and hypotheses from which future research studies and interventions could be developed. Without question such an article is utterly useless as guidance for a patient. Even worse than useless, it can lead to unrealistic expectations or undue doubts, undermining trust in the advice being offered by a medical professional. Stated plainly: The biomedical literature is not always directly useful to the American public. As I myself have experienced, access alone can cause confusion and even risk poor outcomes without interpretation or translation from someone who is medically and statistically literate.
My observations here are not intended to serve as an argument opposing open access. Some have argued that public access to the scientific literature will drive greater efficiencies into the research process, enabling more scientific progress to take place more quickly. Others find that public access influences patenting and therefore presumably can promote economic development. There can be many forms of return to the American public from public access.
Still, when the Department of Health and Human Services and its constituent agencies implement policy to achieve the public purpose to “allow all Americans to benefit from the returns on our research and development investments without delay,” they face a choice. Will they focus exclusively on enabling greater public access to the scientific literature they fund, with a mixture of benefits, limitations, and even risks? This is the call for action in the Nelson Memo, and it is the common policy action that can be taken across all federal funding agencies. But for different parts of the scientific literature, there are different use cases for the American public. Perhaps there is an opportunity, as agencies implement this policy guidance, to consider the needs that are specific to their own portions of the literature. In the case of agencies like the NIH and FDA, will they consider the use case of a patient trying to learn about, and consider among treatment options for, a disease or medical condition?
Perhaps we have seen comparatively little interest in this use case because it is the more complicated one to address. Many of the ideas that have taken hold are generally about what scientific publishers themselves can do, for example to enable researchers to provide annotation and translation of scientific articles, often through plain language summaries. But as I’ve argued, the very structure of the biomedical literature is not fit for purpose for patients and their advocates. What is needed is distillation and synthesis into formats that will be useful and situated within workflows that will be convenient, incorporating the medical literature and addressing misinformation, through whatever combination of expert human creation, curation, and artificial intelligence can be most effective. Addressing this need calls for action by those with broader mandates for or interests in supporting the needs of the American public, not least of which is the federal government.
Federal policy makers have spent several generations focusing on public access, showing the transformative impact of the policy making process. But to address the use case for patients and their advocates among the American public, and others that are specific to discrete portions of the federally funded literature, we would need not merely policy guidance and mandates but perhaps more importantly new service models and product categories, as well as the incentives necessary to motivate their development. How many more generations of policy making will it take to address these needs?
Discussion
17 Thoughts on "The Problem at the Heart of Public Access"
Putting the right ingredients together is not à guarantee for a good meal. We need experienced cooks who understand digestion.
You have made an excellent point. The public often gets led down the wrong path starting with authors spinning their results. Peer review does have the effect of toning down spin but the rise of pre-print servers and open peer review can lead to overstatement or misinformation within the public sphere. Pre-prints have not undergone peer review and both pre-prints and open peer review suffer in that the public may not read or be able to understand the comments.
The public needs a lay abstract that explains results and conclusions and how they may use the information when consulting physicians. Access to the paper does not help if they do not have the knowledge to understand the data.
Isn’t that what AI is actually pretty good at, lay summaries of more complex text?
I think currently it very much depends on what AI platform is used. Results vary widely – from good enough through to downright misleading. Though of course we can feel confident that the accuracy, ‘completeness’ and usefulness of these AI-produced summaries will improve.
It’s beyond a translational issue though, right? One paper is one paper, which can only be read in dialogue with the other literature – a feat the layperson simply cannot hope to accomplish. We clearly see examples of this in science reporting – where a single study gets translated without an understanding of context which would at least require a comprehensive, long-form article – something that begins to get away from popular journalism due to to length, etc. that most people are not interested in reading. My worry is an inflated sense conveyed to the public of an unearned expertise. We can’t al be experts in everything!
One thing worth considering is the gap between research and available treatments. Reading the latest cell biology investigations into cancer will not help patients understand available treatments, which are likely at least 10 years of clinical trials and approval away.
I would also emphasize that by the “general public”, the government means companies and private industry. One of the main reasons governments fund research is to drive economic development, meaning jobs and tax revenue. The benefit to the public of such funding is presumably the treatments and products that result.
This is a great point I haven’t thought much about before, but the pandemic brought very much into focus how much harm can be done by people who don’t understand what they are reading trying to read the scholarly biomed literature, especially in an area that is still developing.
What the public really needs is systematic reviews like Cochrane, to distill the info for them. In Atlantic Canada, we academic libraries have a deal with Wiley that as long as we maintain our paid subscriptions, our entire communities (based on their IP addresses) can access Cochrane for free without any authentication system. Maybe if the US govt spent a few hundred million to make a US-wide Cochrane deal, that would be a better use of funds than [insert your favorite waste of federal tax dollars here].
This is a great article on an area that doesn’t get the attention it should do. We will miss the opportunity to harness the full potential of public access or Open Access without expanding the way we communicate science. But also, saying to a non-specialist audience, “Here, we’ve made it all publicly available – go read,” and then simply pointing them to a technical, scientific journal article will likely be counterproductive in improving public trust in science. The language is completely inaccessible, the format is totally unengaging, and as you say, the depth of reading required on a topic is too much. The potential for misunderstanding is too great.
One approach to solving this problem is, for example, explanatory animations or podcasts, or adapting articles to more engaging articles adapted or non-specialists. These also need to be delivered through channels and platforms aimed at these audiences. This is something we’ve been working on at Karger, as are some other organizations. I’m pleased to see this spotlighted more, as it’s genuinely important not only for individuals in the public to be able to understand their own conditions, but for also society to have a common understanding of facts.
Medical librarians (of which I am one) wrestle with this constantly. As far back as the beginnings of my own career, working the reference desk at NLM (as a post-grad fellow) in the early 80s, I learned how to provide loads of caveats with every encounter I had with a member of the public seeking information on their’s or a loved one’s rare medical condition. The move towards open access has made getting to the literature quantum leaps easier, but the avalanche of information has made the problems of interpretation exponentially worse. There have been attempts to address this — PNAS has experimented with requiring summaries of articles directed toward a general audience; I dimly recall a project by Elsevier (and some other publishers) to distill recent research into usable formats, but I don’t think it lasted long. Melissa mentions Cochrane, which is good, but very slow and doesn’t have wide coverage. Products like UpToDate or Dynamed do a good job of distilling the literature into clinical decision support, but of course they’re directed toward clinicians and, as proprietary products, are not widely accessible to the general public anyway. NLM’s MedlinePlus is an excellent resource for the general public, and the big healthcare systems like Mayo and Johns Hopkins do a good job of distilling information (although they’re terrible at citing the literature). But the rarer the condition, the more difficult it is to find useful distillates (in no small part because the rarer the condition the less research being done on it). Making the literature itself more accessible is an important (possibly essential) step, but what these various efforts show is that making the scientific literature truly useful to the public at large is extremely difficult and expensive. Medical librarians and patient educators (who are typically, although not exclusively, nurses) have been aware of these problems since, well, forever, and we remain frustrated by how difficult it remains to marshal the resources required to address them.
Excellent article. Thank you. I’m one of a group of consumer health informatics researchers who has been concerned about the intersection of science literacy and health literacy for some time – for just the reasons you cite. As a patient myself I will always vote for *informed* access over *confused* access and wish more resources could be devoted to helping consumers and patients understand what they’re reading.
I agree with those above who find this post to be helpful guide to the issue of public access to the research literature. There is one aspect among the important points raised that we, at the Public Knowledge Project, have started to address. This concerns how the public can “establish the trustworthiness and authority of information resources.” This will also involve, I would add, helping the public learn about what distinguishes scholarly publishing from other types of publication.
Our approach has been to build on the success of the Nutrition Facts label in both informing and educating the public about food and health, which has been shown to positively change behaviors. We are developing a Publication Facts label with a similar design, which is intended to accompany every research article, while presenting eight facts that bear on its adherence to scholarly publishing standards with educational pop-ups for each. The label and data are generated for each article by the journal’s publishing platform, with third party verification (although any system can be hacked). As with the Nutrition Facts label, such an approach will need to be supported by public education efforts in science classes, which only makes sense in an Age of Public Access. To learn more about and view a prototype of the Publication Facts label, which we are currently testing with science journalists, high school students and their communities, and journal editors and authors, see our preprint here: https://doi.org/10.1590/SciELOPreprints.6799
I wish you luck in your endeavor. Considering that:
How many biomedical articles are published each year?
Most of the landmark studies that have changed clinical medicine have been published on journals. Over the years, the number of active, peer-reviewed journals has expanded to approximately 28,000, collectively publishing more than 1.8 million articles every year.
Build infrastructure in publishing scientific journals to benefit medical …
nih.gov
https://www.ncbi.nlm.nih.gov › articles › PMC3937756
I think you will need millions of dollars to summarize and rewrite the literature into the common vernacular.
Thanks Roger for another great thought piece! Having founded several consumer libraries within hospitals when I worked, and also always encouraging medical librarians to promote and advance health literacy, I share your concerns. I believe librarians can help to bridge the knowledge gaps and translate the science in understandable ways to patients. Librarians are skilled at this and can play an important role in helping patients understand their health.
I echo my friend and colleague Jean Shipman’s suggestion about information professionals such as librarians and informationists being in the mix (my iPad spell checker has an issue with the term informationists). It would probably be a straight forward project to add an “emerging research” link to a MedlinePlus topic resource page, but how to curate and make a systematic review article understandable to the reading level of 8th grade and still be engaging is a conundrum. And building relevant extenders to deeper understanding for patients at higher reading levels. Five NLM Associate Fellows arrive each year to take on health information problems. Please involve the NLM and health information professionals.
I became the publisher of the Critical Review journals published by CRC Press. I learned that they were often the number one cited Journal in the Citation Index. I was curious as to why. I spoke with the journal editors and with other leaders in their respective fields. I learned that researchers preamble their grant proposals with a literature review and the most reliable reviews were those published in the series. The review article topics were chosen in consultation with the editor-in-chief and section editors. From there, invitations went out to experts in the field and articles were penned. I next asked the authors of the review articles how they selected which articles to include. They said by seeing which articles were most cited. Thus, the willowing process was accomplished, and the researcher was provided a base from which to write their preamble.
To replicate the above would take hundreds of review journals and the accompanying infrastructure which would be formidable.
To bring the above to the public would be daunting, especially in light of the ever-changing nature of scientific discovery on any given topic, and the finding of authors who could break down complex ideas to the general populace.
Although OA may be a noble cause, I do not think it is practical when attempting to bring very complex ideas to those who are not versed in the subject.
These are all brilliant comments. Thanks for spotlighting this issue, Roger. From my perspective, having focused on various offshoots of science sharing challenges for the past 12 years through the Science Communication Institute (www.sci.institute), the solution we need is probably layered (as you can tell from the comments so far on your post):
1. At the bottom, foundational level, for patients and their families, we need more thoughtful and carefully curated resources like what the Michael J. Fox Foundation provides for Parkinson’s research (https://www.michaeljfox.org). Better lay summaries of current research would also help to the extent families feel the need to look beyond what’s available in trusted resources. AI (with human oversight) will probably have a big role to play in the future development of this level.
2. At the “middle” level where more detail is needed, maybe for citizen scientists and policymakers, we need these better portals and lay summaries from level 1, AS WELL AS more complete access to the raw research literature and data, PLUS much more attention to organizing and curating all this information and trying to see the patterns and big pictures—understanding which work is promising, which is fake, where the holes are, and where the connections are to similar research teams and research work. It’s important to note that at this level, there has been significant concern from the research community that people with an inadequate understanding and/or an agenda will misinterpret the raw data and churn out bad research and scientific misinformation (which is why many research sharing portals restrict access to qualified researchers); and finally,
3. At the pinnacle of sharing and engagement, for researchers, we need these better portals, lay summaries, access, and basic curation from levels 1 and 2, PLUS a whole ecosystem of tools and processes that focus on the actual nuts and bolts of data usability such as data standards, integration, management and high-speed computing. This step is enormously complex and labor intensive; only a few fields have really made significant and sustainable headway into figuring out how to create robust research sharing networks like this. They aren’t going to magically appear without significant investment and commitment, and even when they do appear, they aren’t going to magically create the kind of value we hope for. This pinnacle level hasn’t been widely explored yet, but there’s a lot we can learn from the fields that have been at this bleeding edge of research sharing and collaboration for many years already.
So in summary, yes—just trumpeting the promise and potential of public access (or open access) isn’t enough, akin to saying that growing more trees will solve the housing crisis. We need to also pay attention to what comes next, and start building out these next steps so we can begin truly fulfilling the promise and potential of open.
I would argue public good is also knowing not only the article but the underlying research data is available for others to build on the research, and help collectively find a cure. For someone with Parkinson’s for example, knowing key federally funded research is not locked behind a paywall, or only stored on a researchers laptop, is somewhat reassuring that even if a cure isn’t found in one’s own lifetime, the system is in place when technologies like AI advance, that the underlying research and datasets can be very useful. It’s not just about being understand the literature, but knowing we have a long term system for the greater public good.