Pharmacists play an essential role in managing patients’ healthcare needs. Their job requires precision, organization, and specialized knowledge to ensure patients receive the correct medications in the proper doses and are sufficiently instructed on how to administer them.
Although essential, many aspects of this job can be tedious and time-consuming, prompting researchers to explore ways to make the process more efficient using artificial intelligence. Advancements in AI have made it a valuable tool in many fields with seemingly infinite capabilities; however, its unprecedented use in areas like medicine can have disastrous consequences when the technology yields inaccurate results. The risks associated with AI medical advice raise major questions about whether the public and the medical community are willing to shift to AI prescription renewal programs.
Over the last few years, AI chatbots like ChatGPT have become increasingly popular for answering people’s everyday medical questions. Despite the responses sounding legitimate to many users, it often provides inaccurate advice, which can have severe consequences, especially relating to drug dosage and use. One complaint in a wrongful death suit detailed how a nineteen-year-old boy used ChatGPT to “coach” him through the use of illicit drugs without advising him about the potential effects. Among other misleading and dangerous responses, the complaint listed that ChatGPT routinely advised the boy to take high doses of drugs without listing the possible interactions or consequences. The boy tragically died during an incident where the chatbot coached him through the use of Kratom to get high and improperly directed him to take Xanax to alleviate the nausea caused by Kratom. The combination of these drugs recommended by ChatGPT, along with alcohol, was determined to have likely caused the boy’s death by asphyxiation.
Earlier in 2026, Utah launched a pilot program that allows autonomous AI to renew prescriptions for certain drugs used to treat chronic conditions. The program is meant to test the waters and find out whether the public and the industry are ready to handle pharmaceuticals. In general, AI prescribing involves AI-generated recommendations that physicians review to renew prescriptions. Current Utah policy allows AI to renew prescriptions under specific circumstances. Parameters of the program run through a partnership with Doctronic include that patients must be adults taking medications used to treat common health conditions such as depression, birth control, hypertension, diabetes, glaucoma, and asthma.
Although Utah and Doctronic urge that the program will close the “access gap” and streamline processes by delegating administrative work that slows clinics, many remain skeptical about the risks of allowing AI to perform such a sensitive task. Physicians fear that using AI in this manner poses risks to both patients and physicians, namely concerning misuse or abuse. One article on Utah’s program noted that a “concern is missing subtle clinical red flags or drug interactions that a doctor would catch.” The risks of Utah’s program are low due to the nature of the prescriptions, some of which are available over the counter; however, software companies have already been discussing AI’s expansion into all parts of the medical care experience.
Currently, AI use in medicine is regulated by state laws and existing federal healthcare and privacy regulations such as HIPAA and the FDA, but there is no single federal AI healthcare law. When a physician intermediates in AI-assisted prescribing, the physician can be held liable over the AI developer; however, that liability shifts to the developer when the advice comes directly from AI. Because AI is rapidly evolving and being used autonomously, many scholars and professionals are pushing for its further regulation in medicine through mandatory disclosures, risk assessment, and content and functionality audits.
Although Utah’s program is in its early stages, researchers warn that these programs should not expand to autonomous medical practice without high standards. Cases involving AI medical advice have shown that there is little room for error when it comes to giving drug use instructions and recommending doses. Utah’s program has demonstrated AI’s current and potential future role in pharmaceuticals, but because it can give misleading or improper medical advice, the allure of AI mustn’t distract from the risks of missteps in a sensitive and intricate field.