The story of Jack Andraka is a seductive everyman’s story — a 15-year-old wunderkind surreptitiously reading in his high school science class a scientific article he downloaded thanks to free online content initiatives, combining this with other free sources like Wikipedia and YouTube, and, using these free materials and his own ideas, he invents a cheap, effective, and novel way of testing for pancreatic cancer.
With a story like this, Andraka won a science prize and was embraced by the media — giving a TED talk, appearing on the Colbert Report, and receiving glowing coverage around the world, including accolades at the White House, the Vatican, and the Max Planck Society. He has also become a folk hero of sorts for advocates of open access (OA), who point to his story to dispute “elitist” claims that lay people can’t benefit from free online science publications, and to demonstrate clearly that society benefits immensely from OA policies. After all, here is a boy who used free online information to create a cheap, effective, and novel test for the fourth-deadliest cancer in the world.
It sounds too good to be true. And, if you’ve lived a while, you already suspect what’s coming next.
Jack Andraka comes from a well-educated, relatively affluent, and scientifically-oriented family, one that has produced another science standout (Jack’s older brother) and that could afford to buy a few articles or subscriptions. His parents also started a business in 2000, which went under a few years later. They are clearly an active, inventive, and entrepreneurial family. Given the parents’ stable careers in good fields and with good organizations, and the family’s deep science backgrounds, the economic and “layperson” aspects of his story seem less clear-cut than some would have us believe.
Let’s move on to his claims, which were startling. Using carbon nanotubes in a novel way, he sought to derive the level of serum mesothelin using a dipstick on a few drops of blood or urine, and thereby detect early signs of pancreatic cancer. The test was purported to be 90% effective at detecting mesothelin, a soluble protein thought to be associated with some early stage cancers.
But, as the saying goes, extraordinary claims require extraordinary evidence. Does Andraka have extraordinary evidence?
One paper he relied on to develop his hypothesis apparently came from the Proceedings of the National Academies of Science (PNAS), and was published in 1999. The paper was clearly speculative, stating at the end:
Soluble molecules of the mesothelin/MPF family may provide useful new marker(s) for diagnosis of ovarian carcinoma and/or monitoring its response to therapy.
These researchers speculated that serum mesothelin might have possible relevance to screening for ovarian cancer or monitoring therapy. In the paper itself, the only mention of pancreatic tissue is as a normal tissue that did not respond to the screening test the authors used. The cancerous tissues they tested included ovarian, testicular, cervical, breast, bladder, lung, and prostate, but not pancreatic. And of these, only ovarian showed a strong correlation. In some press clippings, Andraka boasts of his test being able to detect early-stage lung or ovarian cancer, but his focus is on pancreatic cancer in most instances. It’s the kind of cancer that took a close family member, and also the type of cancer that killed Steve Jobs, whom Andraka identifies as a particular hero of his.
Another paper Andraka apparently relied on is a 2001 paper published in Clinical Cancer Research. It is an analysis of a gene library, where the authors found “the tag for the mesothelin mRNA transcript . . . in seven of eight SAGE [serial analysis of gene expression] libraries derived from pancreatic carcinomas but not in the two SAGE libraries derived from normal pancreatic duct epithelial cells.” From this association study, they speculate that mesothelin overexpression may be a potential disease marker. However, they also state that because “mesothelin is known to be overexpressed in other cancers, detection of a soluble form would be unlikely to be specific for pancreatic cancer.”
A commentary on another study in Clinical Cancer Research from 2011, which it’s not clear Andraka integrated into his thinking, outlines the vast array of markers associated with pancreatic ductal adenocarcinoma, then discusses the importance of first finding high-risk individuals, along with the potential and comparative utility of imaging studies. It ends with the much more cautious statement: “Because screening brings with it the risk of overtreatment, more controlled trials are needed to better determine the risks, benefits, and optimal approaches to pancreatic screening.”
Suffice to say that there is a vast amount of scientific information in this field — the area is complex, information is emerging, associations are unclear, and claims appropriately cautious. But one piece of research is missing — Jack Andraka’s published results. In fact, this is one of the major complaints about the Andraka story — he has never published a peer-reviewed paper of his findings, despite being an advocate of open science and free information. Instead, he seems to be focused on exploiting his claims — taking more than $100,000 in prize money for his unpublished claims, working with Intel and other corporations to develop a commercial over-the-counter test based on his approach, and possibly even filing a patent for the invention.
In December 2012, Andraka started a company called Andraka Technologies, LLC. On his LinkedIn page, he lists himself as “Founder And President.” Because he’s a minor, the resident agent for the LLC is Jane Andraka, his mother. The LLC application was filed as a walk-in application on December 21, 2012. She paid the $100 filing fee and an additional $50 fee to expedite the application. The LLC’s purpose is to “develope [sic] medical technologies.” The LLC currently has a status of “Not in Good Standing” with the state of Maryland, a status given when “the entity has not filed the current year’s Annual Report/Personal Property Return or it owes a late filing penalty.” But the fact remains that mere months after making his claims about a cancer screening test, Andraka established a company to commercially exploit what he thought he’d found.
In a fundamental way, these strategies are the antithesis of “open” — making claims that other scientists can’t examine fully, then taking rewards for the claims and trying to exploit them personally and commercially. These attempts include a rumored patent, a partnership with Intel and another commercial entity, and an LLC started to develop medical technologies.
There seems to be a good deal of self-promotion involved with the Andraka story, as well, including his own vanity Web site and some deliberate Wikipedia editing revealed in criticisms of his Wikipedia page from BioMed Critical Commentary:
The references in the Wikipedia “Jack Andraka” page were (as of accession October 13-17, 2013) largely self-references to statements and research conclusions made by Mr. Andraka, as had appeared in talks and news articles. Independent scientific sources were not cited for the research claims.
Assertions made in the Wikipedia page about Andraka patenting the test were expanded to be more fulsome and detailed, but these edits were deleted. Other references to a paper refuting mesothelin as a cancer biomarker in the sera of pacreatic and biliary cancer patients were also deleted, as was the text summarizing these findings in Wikipedia:
Pancreatic cancer patients representing multiple stages of disease did not have serum mesothelin levels higher than was normal.
As other scientists attempted to confirm the logic of Andraka’s methods and reproduce his findings from the scant information available — findings that, once again, have never been published in full — problems have emerged.
For instance, the authors of the paper contested in edits to his Wikipedia page found no association between serum mesothelin levels and pancreatic tumor stage or differentiation grade. A major problem with using mesothelin levels from serum appears to be that normal adults can have high levels and no disease. There are plenty of opportunities for false-positives, and making the test more sensitive would only increase the rate of false-positives. Sensitivity is not specificity. In addition, his claims about his test’s increased sensitivity don’t seem to matter, as currently available commercial tests for serum mesothelin levels are quite sensitive, making any claimed improvements clinically meaningless.
There are also problems with the scant and unpublished evidence backing up the claims, which as the BioMed Critical Commentary piece states:
As time has passed since May 2012, when Mr. Andraka’s project won the Intel ISEF award, inconsistency has emerged among accounts relating the sample types studied in the project.
These inconsistencies include mentioning patients with pancreatic intraepithelial neoplasia in one summary of his work then removing these in later summaries (possibly because patients with this pre-clinical version of pacreatic cancer are nearly impossible to identify); presenting data without labels on the axes of the charts shown in poster sessions and not correcting these oversights; listing drug therapies for purported patients that were non-standard for pancreatic cancer treatments; and failure to clearly describe control groups other than to say “healthy” controls.
While the media version of the Andraka story is nothing but glowing, a more complete version of the Andraka story is more chastening:
- An open science and free information paragon doesn’t publish his claimed positive results, yet these unproven positive claims are embraced unquestioningly by some in the OA community and by more in the mass media because they fit a particular folk narrative about open science and its utility among lay people.
- Someone benefiting from taxpayer-funded science did not feel compelled to share his results, and instead took prize money, worked with commercial funders, started his own company, and may or may not have filed a patent on the claimed invention.
- Findings that have never been published in a peer-reviewed publication have been widely broadcast in the mass media across many continents, as if they were vetted and validated by experts. In the meantime, doubts about their veracity or reproducibility are not finding the same welcoming spotlight. This puts us right back into the sad old problem of positive claims overshadowing negative studies. More worrying this time, while negative studies have been published, the positive claims aren’t based on any published results.
- The logic and facts on which these claims seem to be based may not even have face validity — pancreatic cancer may not be associated with increased mesothelin levels in sera, while increasing the sensitivity of serum mesothelin testing wouldn’t add specificity and would only yield more false-positives.
The list above is a mixed bag of questionable science, low accountability, attempts at personal enrichment, and limitations of public peer review.
It also may be possible this the Andraka story is collapsing under its own weight. The LLC appears to have been abandoned. The patent was only rumored but never filed. The paper we’re owed may never be seen. But as long as the accolades keep coming and accountability is not demanded, fame could continue to provide plenty of rewards for outsized claims while continuing to mislead the public and provide a source of false hopes.
So, what is the Jack Andraka story? We can’t know because he hasn’t published any paper clearly outlining his methods or findings. But his claims are looking less compelling by the day, as is his status as an open science and free information poster child. Here you have a person who could afford to buy information to complete his inquiries becoming a symbol for free information. You have claims that are outsized, but these claims have yet to be validated by strong evidence. You have a boy who has perhaps been overwhelmed by big science and big business and big fame. You have a young aspiring scientist who is not following normal scientific protocols regarding publication and peer review. You have a parent helping her son create an LLC to exploit his inventions. You have the mass media and some enthusiasts getting ahead of the facts.
Whatever the final disposition of the Andraka tale ends up being, it’s clear that the story is far from settled. In that regard, we should stop throwing the Jack Andraka story around as if we know what it means. In the final analysis, it could all be for naught.