Last month, Springer Nature announced the publication of their first machine-generated book — an experimental proof of the efficacy and impacts of algorithmically curated scholarly resources. In the age of “robot reporters” and auto-generated novels, Springer intends to lead the way in seriously examining the value of machine learning to aid readers at all levels with comprehending vast volumes of academic literature. This publication — a book synthesizing 150 other Springer books that address the topic of lithium ion batteries — has the hefty goal of piloting the ability of such machine-learning technology to save us the time of reading dozens of resources in order to grasp a new topic.

This news has me wondering: Will students adequately learn complex concepts from these resources, with the necessary depth of understanding we have come to expect from many years of rigorous reading and study? Will lay-readers of such books sufficiently comprehend the relevant context in which new concepts fit? How do we avoid perpetuating human biases, power structures, and assumptions that inevitably become baked into both scientific research and the information technologies that support it?

I recently had the opportunity to address these questions with two leading forces that made this new machine-generated ebook possible: Henning Schoenenberger, Director Product Data & Metadata Management at Springer Nature, and Prof. Dr. Christian Chiarcos, Applied Computational Linguistics lab at Goethe University Frankfurt/Main.

Robot's hand holding a pencil.

What was the goal or inspiration for this project?

Henning Schoenenberger:  Springer Nature is aiming to shape the future of book publishing and reading. Progress in natural language processing is advancing fast, and new technologies around Artificial Intelligence offer promising opportunities for generating scientific content automatically with the help of algorithms. Hence, we decided to develop and publish our first machine-generated research book. This prototype is designed for all interested audiences, such as researchers, master and PhD students, reviewers, academic writers and decision-makers in science education. By providing a structured excerpt from a potentially huge set of papers, it is supposed to deliver an overview of a specific subject area or topic, saving time and effort. With this prototype we would also like to initiate a public debate on the opportunities, implications and potential risks of machine-generated content in scholarly publishing. The risk is growing with the complexity of the technology. The more we continue to look into Deep Learning approaches the more we also have to refine the reviewing process by subject matter experts in order to ensure the accuracy of the content.

So, how would you characterize the problem you’re looking to solve with this machine-generated book?

Henning Schoenenberger:  Our first machine-generated book proposes a potential solution to the problem of managing information overload efficiently. It allows for readers to gain an overview of a given field of research in a short amount of time, instead of reading through hundreds of published articles. At the same time, if needed, readers are always able to identify and click through to the underlying original source in order to dig deeper and further explore the subject. Instead of using results from search engines which may often be hard to qualify, readers can rely on qualified information published on Springer Nature’s content platform SpringerLink which stands up to scientific scrutiny. Machine-generated books, such as our prototype, can assist anyone who, for example, has to write a literature survey or requires a quick and focused start into a certain topic.

What metrics will be used to assess how well this book solves that problem? Are you looking at learning outcomes or other ways to measure researchers’ ability to consume / understand these topics more quickly or effectively?

Henning Schoenenberger:  We are evaluating the success of this project through ongoing user research as well as analyzing the feedback from the research community. At the moment, our focus lies especially on finding the most useful parameter settings for the algorithm pipeline, to allow for an optimized consumption process. Parameters we use are for example, page counts, the number of clusters and sections, the length of summaries, the number of word features etc. Of course, the parameter settings vary from discipline to discipline, and are also dependent on the scope and size of a given topic. Our expectation so far as learning outcomes for readers is a pragmatic one: It should speed up the literature digestion process. At the same time it should support readers identifying the underlying original sources and click through and be able to further explore the subject where necessary.

What about peer review? How did you validate the end results were accurate and of sufficient quality for publication?

Henning Schoenenberger:  We are aware that the quality of machine-generated content can only be as good as the underlying sources which have been used to curate it. Hence we have decided to only use peer-reviewed, robust research from our content platform SpringerLink for this prototype. Through referencing all source documents with hyperlinks, readers are at any time able to identify the underlying source. We decided not to manually polish or copy-edit any of the texts, as we want to highlight the current status and remaining boundaries of machine-generated content. Springer Nature editors and experts in the field of Chemistry supervised the iterative content creation process and provided guidance and feedback regarding the content output on a regular basis.

How did you avoid human bias built into the algorithm?

Christian Chiarcos:  The current implementation consists of several components, each of which have their own characteristics. These components include, for example, the generation of the table of contents, respectively, the overall structuring of the book in chapters, sections, and so on. The underlying algorithm in this case is fully unsupervised: Based on a given set of parameters such as number of chapters and sections, and a particular similarity metric, similar papers are grouped together and clustered. From each cluster, the most representative publications are selected according to predefined parameters.

Is there a scalable business model in machine-generated book publishing?

Henning Schoenenberger:  I do expect that machine-generated content will become a scalable business model at some point. However, as with many technological innovations, we also acknowledge that machine-generated research texts may become an entirely new kind of content with specific features not yet fully foreseeable. As a global publisher, it is our responsibility to take potential implications of machine-generated content into consideration and work on providing a framework for machine-generated research content. That being said, it would be highly presumptuous to claim we knew exactly where this journey would take us in the future.

Would you expect composition, editing, or other publishing costs to be remarkably more or less for machine-generated books production vs. traditional book publishing costs?

Henning Schoenenberger:  At a first glance, you should expect cost reduction when it comes to producing machine-generated content. However, this topic is more complex than it might seem, and in fact we are just starting to explore this field. It might be the case that the review process and the iterative quality checks which have to be built into the publication cycle, eventually eat up the time that we gained through facilitating a faster content generation process. Therefore, it is too early for us to provide an informed answer to this question. As a global publisher, we are always looking for options to optimize our publication processes, to make them faster and more efficient and hence more valuable for our customers.

What has been the biggest surprise in this project so far? Has anything come out of this experiment that you didn’t expect?

Christian Chiarcos:  When we started this project, we had a range of well-understood technologies at hand, but it was completely unclear how to evaluate a machine-generated book. In a machine learning context, people usually have a certain amount of manually created gold data to test whether the system performs as expected. For book generation, nothing similar is in place, and I doubt that we could find the resources to manually create, let’s say, ten 250-page overview books about lithium-ion batteries or any particular topic that we could possibly use as gold or training data. While we chose the technologies for the different modules according to our intuitions about the expectations of the audience, it was thus largely unclear how the researchers in this field, and the general public, would perceive such a prototype. So far, the reactions have actually been way more positive than I personally anticipated – or feared, if you will.

Whether the machine-generated book on lithium-ion batteries is a good read is very hard to judge. Most likely not, as its prose is certainly not better nor more readable than that of the original text – and much of the content is very technical. But people are attracted by the convenience of structuring and compressing a large body of literature into a single book, and the technology is mostly seen as a fruitful endeavor – despite some fears about the future of scientific authors, which I personally don’t see endangered by machine-generated publications of this kind any time soon, but rather supported by this type of technology.

Another thing that really surprised me was the granularity of the feedback we got from subject matter experts consulted during the generation process. This indicates that users would like to have a more direct interaction with the system in order to explore the effect of the parameters it provides, and possibly, to revise some of the decisions the system made with respect to chapter structuring, style or degree of compression. This may be a promising direction for future research in this fascinating field.

Researchers play a crucial role in the scholarly publishing ecosystem. With Springer Nature’s first machine-generated book, do you want to introduce a new book format, and does this mean that human authors can be made redundant?

Henning Schoenenberger:  If the technology turns out to be reliable, we plan to increase the use and creation of machine-generated content. However, it is not our intention to disregard the high value of human-created content. Research articles and books written by researchers and authors will continue to play a crucial role in scientific publishing. Artificial Intelligence is not yet able to generate anything similar to a full-scope and meaningful research article. Algorithms still have a hard time rememberinf what was said three pages before – due to a lack of contextual understanding – and to build a storyline that appeals to readers, although the latest research in this field is quite promising.

We foresee that in future there will be a wide range of options to create content – from entirely human-created content, a variety of blended man-machine text generation to entirely machine-generated text.

What comes next in this project?

Henning Schoenenberger:  For the first prototype, we decided to focus on a current chemistry topic. We are planning to publish prototypes in other subject areas as well, including the Humanities and Social Sciences, with special emphasis on an interdisciplinary approach, acknowledging how difficult it often is to keep an overview across the disciplines. The current implementation will be subject to ongoing refinement – based on user research and advances of the technology – and we will use the prototype on Lithium-Ion Batteries as a basis to explore further development of the product.

Lettie Y. Conrad

Lettie Y. Conrad

Lettie Y. Conrad is a publishing and product development consultant, working as a senior associate with Maverick Publishing Specialists, as well as with a portfolio of independent global clients. When she's not bringing a user-centered approach to scholarly content discovery and accessibility, Lettie serves as North American Editor for Learned Publishing and is a part-time information science doctoral student via a remote program at Queensland University of Technology.

View All Posts by Lettie Y. Conrad

Discussion

9 Thoughts on "The Robots are Writing: Will Machine-Generated Books Accelerate our Consumption of Scholarly Literature?"

It’s another Turing test: If the human user is unaware, does it matter. I wonder if we worry as much that students don’t really need cursive or long division when other tools fill the gap. I remember my math teacher saying, you won’t always have a calculator, but now I have one on my phone that never leaves my reach.

Seriously, if you have 6 apples and you want to split them evenly with Mary and Tim, you need to use your phone? Sometimes there’s value in learning how something works, even if you’re not going to be performing that action yourself. Understanding division as a process is fairly crucial to understanding anything in math higher than a very early level. Sometimes you have to do the work to get that understanding, otherwise you’re just inputting numbers into a machine and getting a mysterious number in response.

It’s interesting that in the area where I live, the local businesses that offer after school classes for kids (often science and art classes to replace those subjects that have been largely removed from schools as unnecessary) have now begun teaching wildly popular classes and camps on learning how to write in cursive. This is seen as a fun activity by the participants. And I’ll add that as a left-hander, cursive was extremely valuable in training letter formation, as it requires each letter to connect, which prevents us lefties from drawing them backwards as many do instinctively.

And of course, relying on technology has its limits. At some point, your batteries will fail. A recent venture to the wilds of Iceland where the phone GPS lost all signal was a good reminder of how valuable map-reading skills are. And then there’s this:
https://www.washingtonpost.com/opinions/ditch-the-gps-its-ruining-your-brain/2019/06/05/29a3170e-87af-11e9-98c1-e945ae5db8fb_story.html

While giving readers overviews and abstracts might be helpful, the real value of a publication oftens lies in the details, variants,and exceptions that would be overlooked or worse, over-emphasized. Then in liberal arts, theology and philosophy in particular, generalizations would do violence to the argument. I wonder what AI would do to theologian’s Ed Farley’s book, “Theologia”? But the same holds for scientific papers, particularly in the interpretation of data.

This book only synthesizes Springer resources as described in the first paragraph. Do the links provide resources from elsewhere? Is copyright infringement an issue? This type of digital resource sends the message that you can’t do this yourself or a good librarian, but AI can provide you with all the information you need. Even with algorithms, how accurate is this? Think google with 1,548 hits with only a handful of applicable web sites.

A colleague and I wrote a “review” of this book a few weeks ago. We remarked a few things:
1. The human-written preface is excellent and should be read if you’re interested in NLP research for the purpose of generating scientific content.
2. It’s not really a book because it’s a bit light on narrative – as the authors noted in thisinterview.
3. Garbage in, garbage out: some of the prose was less than stellar and read like a lot of the first draft manuscripts I get to read before they’re cleaned up by reviewers and editors.

For better or worse though, there is no putting this genie back in the bottle and we should all better understand these technologies and think carefully how to incorporate them into our industry.

The review: https://www.advancedsciencenews.com/betas-draft-a-human-review-of-a-machine-generated-book

It seems to me that machine generated books can be useful for maintaining an up-to-date knowledge base which should provide value for experts (think systematic reviews), and the public (think Wikipedia) but what fascinates me most is the potential implications of machine-written books for machines (think standards, nomenclature, interopability).

Fascinating article. However, what worries me is how the question “How did you avoid human bias built into the algorithm?” was either avoided or misunderstood. My concern with machine-generated books, and artificial intelligence in general, is the propensity to continue, or even exacerbate, human bias. These machines are, at first, only as intelligent as the content/data they are originally provided (before the machine-learning takes place). We need to do our best, and behave responsibly, to ensure all potential human bias is eliminated before putting machine-generated books or content, videos or tools, etc, into the world. Who is being consulted on this? And how do we control the proliferation of bias, before it’s too late? This is why it is so important the sciences, humanities and other fields must work together now before it’s too late, and not just leave the future in the hands of software developers.

If machines can generate prose that humans have in the past done manually, then presumably that part of the scientist’s job will be reduced or eliminated. And where the creation of books and articles is more and more easily done, the volume of these outputs will undoubtedly increase (as if we needed that). An increase in the output (and with it the number of citations to one’s works) would seem to devalue publications and their citations as a measure of research productivity.

Which leads me to believe that data collection will be where the real heavy-lifting of scientific research is done and where research value is increasingly recognized. Having one’s research data cited may one day be immensely more valuable than having papers cited, especially when faced with a flood of scientific papers (generated by machines) .

I tried to summarize this here if my comment above is unintelligible: https://www.digital-science.com/blog/guest/robotics-takes-scientific-publishing/

Leave a Comment