Many Concerned About Safety, True Value of Sequencing Tests

Richard L. Schilsky, MD

Next-generation sequencing—the rapid analysis of genetic biomarkers that may be triggers for cancer—is an advance in medicine that has many physicians and researchers excited, but substantial concern remains about the usefulness of the data generated.

A big question mark hovers over the clinical utility of the information that comes back from the testing laboratory, and there is also concern about physicians’ ability to interpret these reports accurately and act appropriately when treating their patients.

Lack of clinical evidence, lack of standards in reporting, lack of physician education, the danger of acting on genetic variants of unknown significance, industry hype, and the variety of the tests on the market were all cited as hurdles by oncologists and others interviewed by OncLive.

“There are many concerns, and also great excitement,” says Richard L. Schilsky, MD, chief medical officer of ASCO. “NGS can reveal the entire portrait of a tumor, and in doing so identify treatment options that might not have been immediately obvious from other forms of testing.”

With little standardization and huge volumes of data, the reports that come back from labs are confusing for doctors to understand, Schilsky said. It’s unclear how reliable the information is, and because tumor characteristics can vary inside the same patient, a single biopsy sent to a lab for NGS testing may not yield a true picture of a patient’s disease, he says.

Nevertheless, the US genomics testing market was estimated at $5.9 billion in 2011 by research firm Booz Allen Hamilton, with nearly 2900 different tests available in that year.1 A subsequent study released in June by Grand View Research predicted growth would continue and that the US genomics testing market would measure $27.87 billion by 2022.2

With this scale of growth and marketing in mind, the FDA has been working to develop a set of standards and processes to ensure that the tests are reliable. The agency is also working also to develop clinical databases that will help physicians to make better decisions about what to do based on the information they get from these tests, says Elizabeth Mansfield, PhD, director of personalized medicine for the FDA.

“There are a lot of ways to make errors in sequencing, and our concept is to develop a set of process and material standards that will, we hope, allow us to essentially be confident that a lab that develops a test will do it in a manner that generates a test that is accurate and reliable,” Mansfield told OncLive.

She says the FDA aims to publish a report on these activities by the end of the year.

In a public forum held early this year by the FDA to discuss the issues involved in this process, David Litwack of the FDA’s Office of In Vitro Diagnostics and Radiological Health talked about the importance of producing tests that measure what they purport to measure and do it accurately; and further, of the need to ensure that what manufacturers and labs say about their tests is true.3 “We recognize that NGS is incredibly important.…We want to allow it to continue while we still perform our mission of protecting patients, making sure that it is used in a safe and effective way.”

The standards the FDA is developing will help to inform manufacturers and laboratories that the tests they are offering work right and are accurate and reliable, Mansfield says. “We’re not necessarily attempting to standardize the results,” she says.

Mansfield says doctors are very much in the catch-up mode as they try to understand how to use these tests and what to make of the information they produce, adding that “We’re trying to address the interpretation part through the clinical validity database building side of our project that will provide clinical evidence for given variants.”

From that point on, doctors will be on their own, she says. “We can’t tell doctors what to order. That’s their practice of medicine. We hope that people over time will catch up to the technology and understand what they should order.”

What doctors know already is that NGS tests offer far more information than is useful, says Schilsky. “Roughly 40% of the time, NGS tests reveal an abnormality that’s defined as being actionable. That’s a definition in the eye of the beholder—something that the doctor might be able to act on. Only in half of those cases can the doctor get access to therapy that is thought to be beneficial, and in less than half of those cases is there evidence that the patient actually benefits from the treatment, so we’re still at a point where NGS testing is more of a promissory note than it is a highly accurate predictive test,” he says.

“What’s fueling the hope is the remarkable, periodic responder—the patient who has a dramatic improvement after receiving some therapy predicted by the test that wouldn’t have been considered—but those cases are quite rare.”

Each NGS laboratory appears to have its own language for reporting results, and doctors often have to rely on the quality of the interpretation and recommendations that come back from the laboratory. The degree of clinical evidence that backs up the findings is also important, because it is a measure of how reliable the recommendations are. But there is huge variation.

“It’s a very confusing time right now, largely because of lack of standards and because of lack of transparent information—in terms of how the variants that are detected are associated with different clinical outcomes and different therapies,” says Schilsky.

Two oncologists who role-played point-counterpoint on the subject at the Miami Breast Cancer Conference this year spoke to OncLive about the positions they took in the debate and why.

Pat Whitworth, MD

One of those physicians, Pat Whitworth, MD, director of the Nashville Breast Center in Tennessee, says that despite the concerns that many have expressed, germline NGS testing remains extremely valuable and useful, and Whitworth says that there is a need for doctors to learn what they need to know about this technology quickly, especially if they intend to continue practicing in the oncology field for the next 5 to 10 years or longer.

“Originally it was BRCA1 and BRCA2, and now we have a good bit longer list of genes that are more or less penetrant as risk-modifiers for breast cancer. Now we have panel testing, and what I was advocating in that debate and what I am advocating in real life is, if the woman has a family history that suggests she may have a Mendelian pattern, a genetic pattern of cancer in her family, then that family needs to be evaluated.”

Whitworth is a co-founder of Targeted Medical Education, a company that seeks to promote access to and knowledge of advanced medicine among physicians. He expresses confidence in the quality of the genetic tests that are available. He says that for the veteran companies in this space to have survived, their technology has to be reasonably sophisticated and practical. “I have not really run into companies that had a presence over a year or two that were in this space that were doing bad work.”

Physicians are so busy with their general tasks that they have tended to delay learning all that they need to know about genetic testing, Whitworth says. “They are anticipating this a little like they are anticipating ICD-10: they know they are going to need to learn it.”

Even so, Whitworth says doctors are liable to react with appropriate caution and moderation to the information they receive, particularly in cases of genetic variants of unknown significance, which some other physicians worry can tempt doctors and patients to make drastic and unwarranted decisions about treatment.

Whitworth’s opponent in the Miami debate, Mike Dixon, MD, professor of surgery and consultant surgeon at the Edinburgh Breast Unit in the United Kingdom, brought up exactly that point, saying that doctors are under pressure to do something for women, especially when the wait-and-see method can lead to further trouble.

“I think that it’s very difficult for patients to tell them we don’t know what it is and then bring them in 2 years later and say, ‘You know what? We worked out that gene, so that mutation is pathogenic.’ In the interim, if the patient develops breast cancer, they may not be too happy. Given the propensity for people in the United States to plug for bilateral mastectomy, even with small breast cancers, I’m concerned that the hype of testing will result in some women having unnecessary mastectomies. There is certainly evidence of that already within the literature,” Dixon says.

While not necessarily the fault of an inaccurate NGS test, a poor quality biopsy, or an interpretation error, a prophylactic mastectomy that is not necessary can lead to substantial problems for some patients, Dixon says.

“If you look at studies of prophylactic mastectomies, one study showed that 69% of women having a prophylactic mastectomy experienced pain, for 36% the pain affected their sleep, for 22% it had an effect on their daily activities, and 75% reported less enjoyment of sexual activities. These are women undergoing prophylactic mastectomies, so this is not an operation without significant morbidity and therefore we have to be sure that it’s worthwhile.”

Dixon says that there remains a substantial knowledge vacuum in terms of what the mutations identified by panel testing of genes really signify. Until that is cleared up, physicians may be liable to act on insufficient knowledge.

“You need to know in the population what these mutations in different genes mean. You need to know how frequent some of these mutations are, but much more importantly, how frequent they are in patients with breast cancer. That will give us some insight into what these different mutations mean and that will allow us to be in a much better position to inform patients, if they have a mutation, what that means. We’re currently not in that position, so for me the technology is ahead of the knowledge, and we’re doing tests on women and we really don’t know yet how to interpret the results,” Dixon says.

Mansfield says that guidelines for physicians on how they should use NGS test information would be a good idea, but aren’t within the FDA’s scope. “I think a lot of physicians are so incredibly busy that they haven’t really had the time to sit down and study the technology and read all the papers. A set of guidelines could be pretty helpful for them.…When doctors get information from a test, maybe even information they weren’t expecting, we tend to think it should come along with an explanation of what the test provider thinks that result means and how much confidence they have in that interpretation. If it’s something they think might be a pathogenic mutation but they’re not sure, our philosophy is the test provider should be truthful about that.”

Mike Dixon, MD

Schilsky says that limited panel tests remain the most useful for doctors who have a disease culprit in mind, whereas larger panel tests have the potential to become “fishing expeditions” that yield some comfort in terms of knowledge, but not much more.

“It’s generally the case when you have patients with a far-advanced cancer and no other known treatment options where the NGS sequencing is being done,” Schilsky says. “Generally speaking you don’t have a lot of treatment options to offer the patient anyway. You could pull something out of the air, or you could say, ‘Let’s get this test done,’ and even though the test is not perfect and may be confusing, it may still be informative and better than having no information whatsoever, which is the alternative, and I think many patients would rather have some information that the doctor could potentially act on than have no information at all.”

References

1. Leslie T, Agar D, Fielding S, Miller S. Market trends in genetic services: impacting clinical care through better prediction, detection and care selection. Booz Allen Hamilton. http://www.boozallen.com/media/file/GeneticTesting_VP.pdf. Published 2013. Accessed July 20, 2015.
2. Grand View Research. Next generation sequencing market is anticipated to grow to $27.87 billion in 2022. http://globenewswire.com/news-release/2015/06/13/744356/10138308/en/Next-Generation-Sequencing-Market-Is-Anticipated-To-Grow-To-27-87-Billion-In-2022-New-Report-By-Grand-View-Research-Inc.html#sthash.VC45AXAc.dpuf. Published June 13, 2015. July 20, 20156.
3. US Department of Health and Human Services, US Food and Drug Administration. Public Workshop - optimizing FDA’s regulatory oversight of next generation sequencing diagnostic tests public workshop, February 20, 2015. http://www.fda.gov/MedicalDevices/NewsEvents/WorkshopsConferences/ucm427296.htm. Accessed July 20, 2015.