Twenty-one researchers from the University of California San Diego were involved in a study performed on babies in China that has been called unethical, risky and misleading. Experts say the experiment likely would not have passed an ethics review in the United States.
The experiment was a new surgical treatment for infants with cataracts and involved an eye incision in the hopes that the lens would regrow and work properly. Dr. Kang Zhang, the former UCSD chief of eye genetics who resigned this month, helped design the study. The results of the research were published three years ago in Nature, one of the premier scientific journals in the world.
Why this matters
Volunteers in research studies accept potential health risks in order to contribute to a growing bank of scientific knowledge.
As China becomes a growing force in the world’s medical and technological development, some scientists in the U.S. worry the country has not properly protected research participants from harm.
Since then, 26 eye doctors from around the globe have decried the study in two open letters to Nature, and three of the physicians explained their concerns to inewsource.
One of the most troubling offenses, they said, is that the surgeries were tested on infants in both of their eyes, rather than just one. That means if the experiment went wrong, the researchers could have caused the babies to lose their vision. And despite the researchers’ claims that the new surgeries were a success, these eye doctors say the results were no better than treatments already in existence.
“If this had happened in the U.S., I would have sued,” said Demetrios Vavvas, an associate professor and ophthalmology chair at Harvard Medical School. “I would have sued on behalf of the families.”
Craig Klugman, a bioethicist at DePaul University, said this kind of reaction to a research paper is unusual.
“When you have that many people jumping at the same time to be on there, they are concerned and they are worried that people will be harmed by following this,” he said.
Nature says all articles it publishes go through a rigorous review process, but Vavvas and other ophthalmologists have questioned how thorough that process was for this study. They’ve also been frustrated by the journal’s slow response to their complaints — it took more than 20 months to publish their letters. Nature has not retracted the article, which has been cited at least 24 times by other researchers studying lens regrowth and the potential of stem cells.
The study by Zhang and others calls attention to several ongoing problems affecting human research, including gaps in the way scientific articles are reviewed before publishing, conflicting ethical principles around the world and the role universities play in regulating their researchers’ work outside of school.
Zhang resigned from UCSD on July 4 as inewsource was reporting on his undisclosed connections to foreign businesses and a Chinese government program. He had been on leave since April, when our investigation found he had put research subjects in harm’s way for years.
UCSD would not comment for this story.
Zhang told inewsource in an email last week the study was not unethical or misleading, followed international research standards and underwent a proper ethics review in China.
“It is not unusual for groundbreaking research involving innovative treatments to invite criticism, especially from persons who are wed to the pre-existing treatments,” Zhang said, adding that the surgery “revealed an immense potential for alleviating suffering and improving the quality of life for countless people well beyond the field of ophthalmology.”
Blind hope
Vavvas became an ophthalmologist 16 years ago because his father was born with congenital cataracts, the eye problem Zhang and the team were studying.
“It was a paper on a subject that is near and dear to my heart,” Vavvas said.
The condition is present from birth and creates a cloudy lens — the part of the eye that focuses light so a person can see clearly. If the cataract is not removed, the child can develop amblyopia, also known as “lazy eye,” and lose vision.
The standard treatment in the U.S. for children under 2 years old is to remove the lens while leaving tissue behind that allows it to regrow naturally. It’s not a flawless procedure – lenses that regenerate don’t always stay clear, and the children often require additional surgeries and thick glasses.
The researchers, including those from UCSD and China, wanted to try a different approach. They made a smaller incision in a different part of the eye, hoping the lens would regrow properly.
They tested the new surgery on rabbits and monkeys — and then a dozen babies up to the age of 2.
Their resulting article in Nature claimed every child regrew clear, functioning lenses with the new treatment and the babies had fewer complications than the 25 children who received the standard treatment for comparison.
“Therefore, our new study provides a novel approach to lens regeneration,” the research paper says, “and results in improved outcomes.”
When he first saw the results, Vavvas said he was excited and hopeful that scientists had found a better solution. But as he scrutinized the details, he realized the study was “both scientifically not sound and ethically unjustifiable in the U.S. and the world.”
“I could not believe what I was reading,” he said. “Immediately I contacted other colleagues, preeminent people in the field, chairmen of departments in the U.S. and Europe and Japan, and everybody was also appalled to the same level I was that this thing was done and that it was even published in Nature.”
One of his biggest concerns was that the surgeries were tested on infants in both of their eyes. Although the study didn’t report any major complications from the new treatment, making incisions in both eyes meant any negative effects could have occurred twice in each child and led to a loss of vision.
The researchers have defended their procedure. Zhang told inewsource surgeries were performed in both eyes “to prevent an imbalance of vision and increased risk” of complications, including vision loss. Yizhi Liu, a researcher at Sun Yat-sen University in China who helped oversee the study, said in an email that “a smaller group of patients underwent the procedure to establish acceptable safety and efficacy, before proceeding to a full study.”
But Vavvas believes that doesn’t eliminate or outweigh the risk of possibly blinding children. And he’s not the only one with those concerns. Vavvas and his Harvard University colleague Deborah VanderVeen told inewsource they believe this experiment would not have been approved by an ethics review board in the U.S.
“Perhaps this is acceptable in China, but it’s not here,” in the U.S., said VanderVeen, an associate professor at Harvard Medical School and a full-time associate in the ophthalmology department at Boston Children’s Hospital. “I don’t think it was ethical.”
Statements to inewsource
Read emails sent to inewsource by researchers Kang Zhang and Yizhi Liu, as well as comments by Nature and its publisher, in response to our questions.
VanderVeen noticed inconsistencies between what the researchers said they accomplished and what the details actually showed.
Images of the infants’ eyes show their lenses were not clear and normal like the study claimed, VanderVeen said, and in one case, a photo of a baby’s eye showed no lens regrowth at all. Plus, the study didn’t take place over a long enough time to see how many of the children would need more surgeries. Overall, the results were no better than what one would expect from existing treatments in the U.S., she said.
The experimental surgeries seemed successful, VanderVeen said, because the infants who received the standard treatment had much worse outcomes than normal. Almost half had complications including inflammation, swelling, pressure and lens cloudiness — a much higher rate than she typically sees.
In one of the responses written to Nature, three ophthalmologists from the United Kingdom wrote that the average baby’s vision after surgery was “the threshold for the legal definition of blindness, an outcome that would lead most ophthalmologists, and probably most parents, to question the value of this new proposed intervention.”
Even so, since the paper was published, other eye doctors have begun testing the new treatment. A group of three ophthalmologists from Russia studied how the procedure affected a 1-year-old girl with cataracts. Their results have not been published yet.
Despite the concerns, Liu said the study was following the principle of “first do no harm” by developing a new technique to improve treatments for babies with cataracts. He and the other researchers are currently conducting a follow-up study to learn more about the long-term safety of the treatment, which he said will show the babies have a “very good prognosis.”
Zhang added that the babies’ eyesight was close to normal for their age and the researchers’ results were not inaccurate.
“This is wishful thinking at its best. We all wish their findings were true,” Vavvas said. “If true, we would be the first to adopt it.”
Ethics in practice
Over the past four decades, the U.S. and other Western countries have struggled to come to an understanding with China — a growing force in the world’s scientific and technological development — on the best standards and principles to follow in research.
The Chinese government spends more than $250 billion a year on research, and that number is rising. More than 800 new drugs enter human trials in the country annually, which are tested on about 500,000 research participants.
The country is quickly catching up to the U.S., which spent 2.8% of its gross domestic product on research and development in 2016, compared to 2.1% in China.
Western countries have been pressuring China to create policies around research ethics and set up institutional review boards, which approve studies in advance and can shut them down if they are not properly protecting participants.
The first review board at a Chinese university was established in 1996, when Peking University wanted to work with the U.S. Centers for Disease Control and Prevention to examine the effects of folic acid on pregnancy. Since then, the Ministry of Health and the Food and Drug Administration in China have established standards for how the increasing number of boards around the country should review research.
But in 2016, three Chinese scientists found that the country’s review boards are not consistent, efficient or well organized. That followed a study by a Chinese researcher who revealed nearly a third of articles published in one of the country’s key academic journals were plagiarized. China’s standards for research ethics also continue to draw criticism from the West.
“I don’t think (China is) necessarily adopting Western values when it comes to research,” said Klugman, the bioethicist at DePaul. “I think what they’re doing is adopting the Western process for research. But when you’re doing that without the underlying philosophy of the importance of the individual making decisions, then it’s kind of a meaningless system.”
Chinese researcher Dr. He Jiankui shocked the world in November when he proudly announced that he had taken embryos from an HIV-positive parent and made them resistant to the virus, then implanted them in a female volunteer.
He claims his experiment was a success — that the woman gave birth to healthy twins without HIV — but the incident sparked international outrage. Scientists feared the procedure had not been studied enough to understand the potential long-term health effects for the children, and they worried about the possibility of “designer babies,” which would be gene edited for non-medical traits like eye and hair color.
“There’s a concern that some of the research we’re seeing out of China, it’s happening fast,” Klugman said. “They move from computer to animal to people models very quickly. And that makes sense from an economic standpoint, because if you get something that’s marketable which you can sell, you can make a lot of money.”
Differing ethical expectations around the world can create complications during international projects. The infant cataract study was performed in China, but it involved researchers from San Diego, Boston and Dallas.
The published article in Nature said the study was approved by a review board at Sun Yat-sen University. UCSD policy also requires “clinical trials and other human subject activity” conducted by its researchers be approved by the school’s institutional review board, or IRB. That’s true even when the studies are not performed on campus, “unless UCSD has entered into an agreement with an ‘outside’ IRB to provide review.”
inewsource could find no evidence in UCSD documents that the cataract study was ever brought before the university’s review board. UCSD wouldn’t answer questions about whether that violated procedures, but Zhang said he followed the university’s rules.
UCSD put out a news release the day the Nature article published. It quoted Zhang describing the new procedure as “a paradigm shift in cataract surgery” that offers “a safer and better treatment” for patients.
Lax ethics review boards can affect what becomes published research. Scientific journals don’t typically do their own ethics reviews of studies before publishing them. Instead, they require studies be approved in advance by these boards, which are supposed to guarantee the researchers protected the rights of participants.
Because a Chinese university’s review board approved the cataract study, Nature considered the study ethical and the research was published in one of the world’s foremost scientific journals, which reaches more than half a million people.
Research and review
About 6,000 miles away from where the cataract study was performed, eye researcher Chris Hammond picked up a copy of Nature and read about the new treatment. He was astonished and fascinated.
Hammond is the Frost chair of ophthalmology at King’s College London and holds degrees from Cambridge and Oxford universities. He told inewsource he was so interested in what he was reading that he brought it up during a lab meeting the next week.
But as he and his colleagues discussed the findings, they developed some of the same concerns as Vavvas and VanderVeen. Hammond worried that someone superficially reading the study might conclude that the new surgery was as good as existing treatments, which may not be the case.
How, Hammond wondered, could this not have been caught by Nature’s review process before it was published?
“We are used to the concept that only the best research is published in Nature,” Hammond said, and therefore it must have been properly reviewed by the journal. “But on reading the manuscript, there are questions about who reviewed it from the clinical side, because those are the areas that we questioned.
“Clearly in our view, peer review of those aspects wasn’t thorough enough.”
Nature publishes less than 8% of the 200 papers submitted to the journal each week. If the paper isn’t immediately rejected, it is sent to one or more independent reviewers familiar with the subject to provide feedback. Nature relies on tens of thousands of these peer reviewers, or “referees,” each year.
“Referees are expected to identify flaws, suggest improvements and assess novelty,” Nature’s peer review policy states. “If the manuscript is deemed important enough to be published in a high visibility journal, referees ensure that it is internally consistent, thereby ferreting out spurious conclusions or clumsy frauds.”
Although Nature offers guidelines for reviewers, training is not mandatory. The peer review process scientific journals use has been the subject of many complaints and news stories questioning its efficacy.
“Peer review can be quite a toothless watchdog,” said Ivan Oransky, vice president of editorial at Medscape and co-founder of Retraction Watch, a website that covers scientific integrity and fraud.
“It doesn’t actually do a lot of the things that journal editors like us to think it does,” Oransky said. “Peer reviewers are volunteers — it’s something you’re expected and have to do as part of your academic service. There’s never enough time, and more and more papers are being published. Why are we surprised that things slip through?”
When articles are published in journals like Nature, they are more likely to be picked up by news outlets and considered high-quality science, Oransky said.
In 2014, Nature Neuroscience, a related journal, published a study showing cocoa supplements could improve cognition in old age, which faced criticism from other scientists on many aspects of the work. Even so, news sites reported it as evidence that chocolate could someday cure Alzheimer’s disease.
Ophthalmologists worried they were witnessing a similar problem with the cataract study in 2016. The results were picked up by The Guardian, WebMD and scientific websites such as STAT and Science Daily. They were also touted by UCSD, one of the top research universities in the world.
Hammond had never written a letter with concerns about a study before, but in this case, he wanted to make his doubts known. He said he believes the surgery was novel and is worthy of more research, but it should still be considered experimental until the long-term impacts have been tested.
Hammond partnered with two other eye doctors in the United Kingdom to draft and submit a response to Nature. At the same time, Vavvas at Harvard University led a group of 22 other eye doctors in drafting a separate letter.
The two letters were submitted one month after the eye study appeared in Nature. It took more than 20 months for the journal to publish them, along with a response from the original research team led by Zhang.
“There is a forum for such criticism of peer-reviewed articles, and the response, in the academic journals in which the research is published,” Zhang told inewsource. “Please note the comments published by these two ophthalmologist groups and our response have been rigorously adjudicated and approved by a panel of the world’s top experts in this field invited by Nature, who also reviewed and approved our initial publication.”
As he waited for Nature to review his letter, Vavvas said other doctors reached out asking for his help — parents were coming to them, hoping their children with cataracts could receive the new surgery. The doctors wanted to know when Vavvas’ letter would be published, so they could explain concerns about the new treatment to the parents.
Vavvas said his response to Nature was “delayed unnecessarily.”
When asked about the delay, a spokeswoman for Springer Nature, Nature’s publisher, told inewsource in an email that the letters needed to be closely reviewed.
“Ensuring that the scientific record is accurate is always our highest priority and we will update the literature when appropriate,” said Rebecca Walton, head of communications for Springer Nature. “We take all concerns raised about papers we publish seriously and look into them carefully. It is important that all material is scrutinised by experts in the field — the process of peer review — before any conclusions are drawn and any comments made.”
A spokesperson for Nature said that “for confidentiality reasons, we cannot discuss specific cases.”
Days before this article published, Zhang sent an email to inewsource that said he “was not involved in the human subject study in any way,” and his role was only to analyze the data after the surgeries were completed. Zhang said he was the front man because he was a better English speaker than Liu, the researcher in China who oversaw the experiment.
The Nature article specifically states Zhang and two others “designed the study and wrote the paper” and, according to Nature’s policies, he confirmed the study’s integrity. Plus, the Guardian, WebMD and at least two other news organizations describe Zhang as the study’s lead author and quote him explaining the new treatment’s success and potential.
Zhang is also the lead author on the response addressing other eye doctor’s concerns about the study. That response includes a note stating authors who contributed to it were “involved in clinical investigation of lens regeneration,” meaning they were involved in the experiments and not just data analysis. Zhang told inewsource “the entire clinical trial, including design, execution, and follow-up, was performed by Dr. Liu’s clinical team” in China.
