The U.S. Food and Drug Administration (FDA) is nearing the creation of a regulatory framework for using generative artificial intelligence in medical devices, at a time when no medical device using this technology is yet subject to FDA regulation. Medical-device companies are watching which direction the agency will take, particularly regarding how to demonstrate safety and effectiveness and ensure that models continue performing after being launched on the market.
In August, the agency published a discussion paper seeking stakeholder views on how to ensure the safety of devices that use artificial intelligence before and after entering the market. Grace Davis Jamison, a spokesperson for the U.S. Department of Health and Human Services, said the goal is to understand the challenges specific to devices powered by generative artificial intelligence and help prepare future guidance.
Technology Present in Trials, Not Yet in Approved Products
The article notes that the FDA has so far authorized more than 1,500 devices incorporating an artificial-intelligence component, but has not authorized a device that uses generative artificial intelligence. Nevertheless, the agency has granted breakthrough designation to a limited number of tools under development, including two separate features from Aidoc and a Radiology Partners subsidiary for analyzing chest X-ray images and preparing draft radiology reports, along with a tool from Modella AI for analyzing pathology images and clinical data.
The FDA has also included devices based on generative artificial intelligence in a digital-health pilot program called Technology-Enabled Meaningful Patient Outcomes, or TEMPO. The program allows companies to collect real-world data while exempting them from premarket authorization requirements. Participants include Limbic, which provides cognitive behavioral therapy by telephone calls in which an AI-powered voice agent interacts with the user.
By contrast, some current features are not subject to FDA review. In 2024, Dexcom added a generative feature to its over-the-counter glucose sensors to analyze users’ data and provide personalized wellness recommendations, and the company said the feature did not require a premarket regulatory submission.
From Product Testing to Competency-Based Evaluation
The discussion paper acknowledges that generative artificial intelligence does not fit easily within existing device frameworks. Systems may produce different texts or images in response to changing inputs, making it difficult to test every possibility and measure safety and effectiveness. Hallucinations—that is, the production of incorrect information that appears reliable—also raise additional concerns, along with the possibility that a model’s performance may decline over time.
The FDA proposes the concept of “competency-based evaluation,” drawing on how physicians are evaluated through examinations, supervision, and public reporting. Rather than attempting to examine every possible input and output, premarket review may rely more heavily on benchmarking and tests measuring the required level of competency.
The agency also raises the idea of creating optional master files for foundation models. Under this concept, model developers or foundational platforms could submit confidential information to the FDA, allowing the regulator to understand the technology used inside a medical device without requiring every device developer to disclose all of the trade secrets of the model on which it relies.
Post-Market Monitoring Will Be Part of the Equation
The paper emphasizes the importance of monitoring devices after they are launched and considers whether it would be acceptable to allow a greater degree of uncertainty before marketing in exchange for relying more heavily on subsequent data and monitoring. Kayla Cristales, an attorney at Haynes Boone, believes the agency may rely heavily on companies themselves to provide updates and track how devices evolve.
This point addresses a problem that initial authorization testing alone cannot solve: a model may change, usage data may shift, or failure patterns may emerge that were not clear during trials. However, the source does not yet specify monitoring standards, the frequency of updates, or the detailed responsibility of developers and healthcare institutions.
Why Does This News Matter?
Companies are awaiting the guidance with varying degrees of interest. Erez Kaminski, CEO of Ketryx, said some developers are moving forward, while others prefer to wait, depending on the company’s size and its appetite for risk. Experts cited in the article believe that many current tools focus on administrative uses, workflow improvement, or applying established guidelines—areas that may not require the product to be considered a regulated medical device.
Companies that have already accepted entering the regulated space, such as Aidoc, view specialized guidance as useful, though not a decisive requirement for moving forward. Sharif Vakili, CEO of UpDoc, said the absence of regulatory leadership could push the market toward a mixture of different rules or delayed reactions to harmful uses.
From an editorial perspective, the real change is not yet the issuance of a binding rule, but the FDA’s shift from a general discussion of artificial intelligence to testing specific regulatory tools: competency-based evaluation, foundation-model files, and continuous monitoring. This gives companies initial signals, but it does not yet resolve how changing models will be authorized or how responsibility will be assigned when performance declines.
The questions extend beyond device companies. Kellie Owens, an assistant professor of medical ethics at NYU Grossman School of Medicine, called for stronger post-market oversight, while Luis Gil Abinader of Generation Patient called for greater supervision of digital companions and chatbots, whether marketed for health purposes or not. Commercial chatbots currently remain outside the FDA’s scope, despite concerns about their use for mental-health support, including some falsely claiming to provide psychotherapy. The FDA’s device center also included clinical-evidence guidance for digital mental-health devices on its list of work in progress for the current fiscal year.