ACCME supports accredited providers in exploring new approaches to continuing education, including the use of artificial intelligence (AI) to help clinicians answer questions that arise in practice. As with all accredited education, however, providers remain responsible for ensuring that clinical content is valid, scientifically grounded, independent, and free from commercial influence.
Many uses of AI in accredited education do not require independent content-validity assessment. For example, providers may use AI to help develop educational materials that are subsequently reviewed and approved by qualified individuals before learners receive them. Similarly, an AI tool may generate responses using a defined body of clinical content whose validity has been established by the accredited provider. In these circumstances, the provider can use its established processes to fulfill its responsibility for content validity.
A different approach is necessary when an accredited activity allows learners to receive dynamically generated clinical answers, recommendations, or guidance developed solely by a large language model (LLM) or an AI agent.
This may occur when an AI-enabled system dynamically searches, selects, retrieves, synthesizes, or generates clinical recommendations from sources such as the medical literature, journals, databases, or other external information sources.
Because the particular clinical content presented to each learner is generated at the time of use, conventional approaches to ensuring validity cannot be implemented. AI systems can introduce errors through the ways they retrieve, select, interpret, combine, prioritize, or present otherwise valid source material.
ACCME is implementing a system of independent platform assessment to evaluate the performance of AI-enabled educational activities.
When an Independent Assessment May be Required
An independent assessment of CE activities in which AI systems dynamically provide clinical answers or recommendations without human oversight may be required to undergo an independent content validity assessment when:
- An accredited provider is the subject of a complaint or inquiry;
- An accredited provider is in the process of reaccreditation; and
- An organization that intends to offer this type of activity is applying for initial accreditation.
ACCME will determine whether an independent content validity assessment is required based on the nature of the proposed activity and how its clinical content is generated.
Any provider offering or considering offering this type of activity who wishes to proactively participate in the independent review process may do so.
Who Performs the Assessment?
Assessments are performed by qualified organizations approved by ACCME based on their demonstrated expertise in evaluating AI-enabled clinical systems, rigorous and transparent assessment methods, and ability to conduct reviews independently. Approved organizations must identify and manage potential conflicts of interest, remain free from commercial or other influence that could compromise their judgments, and base their findings on objective evidence. Their role is to provide an impartial evaluation of an activity’s content validity and performance; they do not develop or operate the activity being assessed or make accreditation decisions.
What Is Assessed?
The assessment applies to the specific educational activity and AI-enabled system as configured and used by learners. It is not a general certification or endorsement of the AI platform, its developer, or all possible uses of the technology.
Approved assessors are expected to examine whether the activity reliably produces clinical information and recommendations that are accurate, sufficiently complete, scientifically valid, appropriately balanced, and free from commercial bias. The assessment should also examine important failure modes such as clinically significant errors or omissions, hallucinated or unsupported claims, inappropriate or misinterpreted citations, outdated or inadequately supported recommendations, inappropriate agreement with flawed clinical premises, and commercial or promotional bias.
Because an assessment applies to a particular activity, system configuration, clinical scope, workflow, and period of testing, material changes may require periodic reassessment.
Independent Assessment and ACCME Decision Making
The CE provider will arrange and pay for the assessment, and the independent assessor will provide findings regarding the content validity and performance of the proposed activity to the provider and to ACCME. A list of approved assessment organizations is coming soon.
There is no single numerical score or predetermined threshold that determines whether an activity is acceptable for accredited education. ACCME will consider the independent assessment findings, together with other relevant information, in making its accreditation and oversight decisions.
The assessment organization does not accredit, approve, certify, or endorse the activity or AI system. ACCME retains responsibility for determining whether the activity meets the requirements for accredited continuing education.
Related ACCME Resources
For additional resources, please see Artificial Intelligence in Accredited CE.
