Findings from a study published in Future Science OA showed that ChatGPT’s responses in terms of general questions regarding hematologic malignancies received a higher average score (3.38) compared with questions on novel therapies and specific mutations (3.06), according to 4 human reviewers who were hematologic oncologists.
“With artificial intelligence [AI] tools being readily accessible, patients are turning to it for information,” Taylor, an associate professor in the Division of Hematology at the University of Miami Miller School of Medicine in Florida, said in an interview with OncLive®. “We should be interested in what kind of information our patients are getting online. The reason that we wanted to do this study was to test the accuracy of what these AI tools are telling patients if they’re asking for information from them.”
In the interview, Miller, who coauthored the study, discussed the background of the research, how AI-based tools are shaping information gathering for patients, and how these tools can best be designed to offer accurate and relevant information.
OncLive: How should you respond to patients using AI tools to gather information?
Taylor: I personally don’t get upset about it. It’s good that patients want more information about their disease. I would recommend that we should be discussing [answers from AI tools] with patients, because some of the information may not be completely accurate. However, you can use it as a launching site for having an open discussion, and I find that it tells you what the patients really care about. If you ask them about what kinds of questions they’re asking the AI tool or what they’re using it for, it can key you into what they care the most about [with their treatment goals].
What was the rationale for conducting this study?
Before starting our study, we did a review of the literature, and there were data published on this [topic] already, but most of them dealt with either non-oncology [fields], or they were within oncology but geared more toward different cancers [besides] hematologic cancers, which is what I treat. We thought there was a gap in the literature addressing [hematologic malignancies].
Also, in a lot of the literature, the questions that we found that were being asked were more generalized to the overall oncology population. We’re not hearing our patients ask those questions too much. We’re hearing a lot of specific questions their treatments. We wanted to challenge the AI by asking harder questions that were more specific.
How should the findings from the study be interpreted in the context of patient education?
We saw, even amongst experts, there was a bit of difference in opinions about some of the specific questions. We did find that the experts disagreed more with [ChatGPT] on the specific questions as opposed to the general questions, where most of the experts agreed that the information was pretty accurate.
We have to meet patients where they are and where they’re finding the information. One of the studies that we looked at that had been previously published about AI was looking at patient forums online. A lot of patients will join groups, and sometimes those are moderated by physicians, and they get to ask the physician a question. This study had looked at patient forums that were moderated by an AI agent and compared them with rooms moderated by a physician, and the patients preferred the responses from the AI agent, likely because it was giving them more lay language that they could understand and longer responses. Physicians are often in a rush because of how busy the medical field is, or they may use jargon that is not understandable to patients. It could potentially be helpful to integrate AI into health information, and, as long as we’re checking the accuracy of it and making sure that it’s not the wrong information, it could be helpful and give patients more information to bring to the table for a discussion and the informed decision-making process.
How significant was the limitation of responses from ChatGPT regarding newer therapies and mutations scoring the lowest?
New studies are coming out all the time, and these AI systems are trained on available information. As new data emerge, the question is how quickly [the AI systems] are retrained and updated. We specifically tried to use newer studies, such as those involving FLT3 inhibitors, and ChatGPT struggled with those more specific questions. That highlights a limitation. Of course, AI tools are constantly improving, and that limitation may not apply to newer versions or other chatbots. We only tested ChatGPT because it’s widely used and free, but paid or medically specialized versions may overcome that issue. Still, it’s a reminder to be cautious when using AI for medical information. It’s generally accurate, but for newer topics, it’s best to double-check with your doctor.
How can an AI tool be best designed to support conversations with patients?
I tend to see these tools as positive, though opinions differ. Regardless, they’re becoming part of daily life, and patients are using them to prepare for appointments. ChatGPT can provide references to medical literature, which not everyone knows how to search for. It serves as a useful starting point—you might bring a paper to your doctor, even if it’s outdated or not quite right, and that can spark a good discussion.
Some patients even input their lab results to see what’s normal or abnormal, then review that with their doctor. The doctor can clarify or correct as needed.
At my institution, AI is also being used for note-taking. During patient visits, the AI records the conversation so the clinician can focus on the patient. Afterwards, the clinician reviews the note for accuracy. That’s one behind-the-scenes use, but I can imagine more creative ways AI could help with patient interactions in the future.
As patients increasingly use AI tools for health information, what should these tools include to ensure they support safe and informed decision-making?
There are specific medical AI chatbots being developed with these issues in mind. We focused on ChatGPT because it’s familiar and publicly available, but medically designed tools are incorporating safety features.
For instance, future AI could recognize signs of a medical emergency—if a user reports chest pain, it could prompt them to call 911 instead of continuing the chat. Those safeguards are essential.
That was beyond our study’s scope—we focused mainly on accuracy—but it’s also up to the medical community to evaluate these tools and understand what information patients are receiving. Hopefully, our work encourages others to study different AI systems and newer versions. This is just the beginning of more research to come.
I also teach medical and PhD students, and AI tools are increasingly used in classrooms. It’s important to teach appropriate use—generating ideas, checking references—rather than copying or plagiarizing. Similarly in medicine, we need oversight to ensure AI is used correctly and benefits both patient care and education.
References
- Nong T, Britton S, Bhanderi V, Taylor J. ChatGPT’s role in the rapidly evolving hematologic cancer landscape. Future Sci OA. 2025;11(1):2546259. doi:10.1080/20565623.2025.2546259
- Introducing GPT-5. Open AI. August 7, 2025. Accessed November 5, 2025. https://openai.com/index/introducing-gpt-5/
