Article

AI Governance for Clinical Systems

Bill
Authored by: Bill Trembley
Senior Vice President, Chief Technology Officer

Every clinical research organization we interact with is using AI in some form. In many cases, that use is informal, and that alone deserves attention. Team members are drafting documents with generative AI tools, project managers are summarizing meeting notes, and statistical programmers are exploring code generation. The technology arrived faster than most governance frameworks anticipated, and in a regulated industry, that gap between adoption and oversight matters.

This is not an article about what you should do or what you should have already done. Every organization operates in a different regulatory context, with different risk tolerances, contractual obligations, and levels of technical maturity. What I can offer is a set of considerations drawn from our own experience building and operationalizing an AI governance framework within a clinical research environment. Some will be immediately relevant to your organization. Others may not yet apply. All of them are worth considering.

There is a natural tendency to start with the tools: evaluate platforms, run pilots, measure outputs. In our experience, the governance framework needs to come first. Without a clear policy that defines what is permitted, what is prohibited, and what the data handling expectations are, every AI use case becomes an ad hoc judgment call. In clinical research, ad hoc judgment calls have a way of becoming audit findings, data integrity issues, or have the potential to negatively impact patient safety.

The considerations here go beyond the obvious prohibition against placing patient or confidential data into a consumer AI chat tool.

A well-constructed policy should address who has authority to approve new tools and what that review process looks like.

It should define the boundary between sanctioned and unsanctioned use, not only for standalone AI platforms, but also for AI features embedded within tools your organization already uses. It should establish data classification requirements specific to AI interactions, including redaction expectations for sponsor names, investigational products, and other confidential information before it reaches a model. And it should define the roles responsible for maintaining and enforcing those standards.

For sponsors, the existence of a published AI policy at your CRO partner is not the finish line. Rather, it is the starting point for a deeper conversation. How that policy is operationalized, trained on, and enforced matters.

Deciding which AI tools are approved for use may sound like a procurement exercise, but it is really a governance one. The considerations extend well beyond licensing. Does the vendor’s enterprise agreement establish your organization as the data controller? Are user inputs excluded from model training? Has the platform been evaluated against the data classification requirements relevant to your work, including GDPR, 21 CFR Part 11, and sponsor contractual obligations?

Embedded AI deserves particular attention. As major software vendors integrate AI capabilities into existing business applications—productivity suites, project management tools, CRM platforms, and even EDC and CTMS systems—the question of what constitutes an “AI tool” becomes less clear. A feature that auto-summarizes meeting notes or suggests email responses is still an AI interaction, even if no one explicitly chose to “use AI.” Organizations should consider whether their governance framework accounts for these ambient AI capabilities, particularly in applications that handle confidential sponsor or patient data.

The vendor management dimension is also worth considering. If your CRO uses AI tools on your behalf, the same scrutiny you would apply to any clinical technology vendor applies here. Security posture, data residency, breach notification, and subprocessor transparency are all relevant. The fact that the vendor is an AI platform rather than an EDC system does not change the diligence expectations. If anything, the novelty of the technology argues for more scrutiny, not less.

The regulatory landscape around AI in clinical research has moved significantly in a short period. Good Clinical Practice guidelines have evolved to accommodate risk-based quality management, technology integration, and more flexible oversight models. The final ICH E6(R3) guideline was adopted on January 6, 2025, and expects sponsors and their partners to maintain documented quality management systems and apply proportionate controls. Annex 2 of E6(R3) remains in draft and under public consultation.

The FDA’s January 2025 draft guidance on the use of AI to support regulatory decision-making for drugs and biological products introduced a risk-based credibility framework for AI-generated information used in submissions. In a related announcement, the agency said it has experience with more than 500 drug and biological product submissions with AI components since 2016.

In January 2026, the FDA and EMA jointly published ten guiding principles for good AI practice in drug development across the medicines lifecycle, covering evidence generation and monitoring from research and clinical trials through manufacturing and safety monitoring. This is a meaningful signal of regulatory convergence for sponsors operating across jurisdictions.

For Clinical Operations and Regulatory leaders, the practical implication is that AI governance is becoming a compliance topic, not just an IT topic. Organizations building their frameworks now will be better positioned than those waiting for prescriptive enforcement to define the operating standard for them.

Two professionals at a table reviewing a document containing regulatory information.

Human-in-the-loop is one of the most frequently invoked concepts in AI governance, and one of the least consistently defined. It is worth thinking carefully about what meaningful human oversight actually looks like in a clinical research context. The answer is not simply that “someone reviewed it.”

The considerations here are straightforward, but often overlooked.

There is also a subtler consideration around automation bias: the tendency to accept AI-generated output with less scrutiny than work produced from scratch. This is a well-documented cognitive phenomenon, and it is particularly relevant when AI outputs are polished, well formatted, and confident in tone. The quality of human oversight depends not only on having the right people in the review loop, but on those people approaching AI outputs with appropriate professional skepticism. Training and organizational culture matter as much as the workflow itself.

One area that deserves more attention is how AI governance is addressed in the contractual relationship between sponsors and CROs. As large pharmaceutical and medtech companies begin adding AI-specific exhibits and provisions to their master service agreements, both parties need to engage with this language thoughtfully.

There are several dimensions worth considering. How broadly is “AI System” defined? A definition that captures every piece of software with a machine learning component could inadvertently sweep in email security filters, validated clinical systems, and grammar-checking tools. That can create bottlenecks without serving a governance purpose. Scoping the definition to focus on systems that materially contribute to deliverables, while carving out incidental embedded functionality, tends to produce a more workable framework.

IP assignment and warranty provisions also deserve close attention. If a CRO is using third-party foundation models, as most are, it is worth asking whether the contract requires warranties about how those models were trained. It is safe to say that most CROs lack visibility into the training data practices of foundation model providers at a level that would support a blanket compliance warranty. Similarly, IP assignment language that triggers on any AI involvement in a deliverable, regardless of how minor, may not reflect the commercial intent of either party.

Termination provisions tied to global AI regulatory developments are another area to watch. If a regulatory action anywhere in the world against any AI system can trigger immediate SOW termination, both parties should consider whether that is the right risk allocation given the pace of global AI regulation. Notice-and-cure provisions, materiality thresholds, and wind-down periods tend to produce more durable frameworks than hair-trigger termination rights.

The broader point is that these conversations are better had early, ideally during MSA negotiation rather than after a study is underway. Sponsors and CROs that align on AI governance expectations up front are far more likely to avoid the surprises that erode trust and delay execution.

It is worth acknowledging that AI in clinical research is not speculative. Organizations are using it today to draft clinical documents, generate analysis code, summarize regulatory intelligence, build bid proposals, and automate administrative workflows. Some are going further, building systems that chain multiple tasks together, pull from external data sources, and produce structured deliverables with minimal manual intervention.

When properly governed, AI becomes a productivity multiplier for clinical teams. When improperly governed, it becomes a risk multiplier.

The common thread across these applications is that the value is not in replacing human expertise. It is in accelerating the first draft, reducing time spent on repetitive formatting and research tasks, and allowing skilled professionals to focus on the judgment-intensive work that actually requires their training and experience.

For sponsors evaluating where AI fits into their clinical operations strategy, the question is not only whether the technology works. It is whether the governance around it is robust enough to support responsible use in a regulated environment. That is a question about policy, process, and organizational discipline, not about the capabilities of the model.

I will close with an observation that may seem counterintuitive: the organizations that invest most heavily in AI governance are often the ones that move fastest with AI adoption. That is not a paradox. A clear governance framework removes the ambiguity that slows decision-making. When everyone knows what is permitted, what is prohibited, how data is handled, and who approves what, the path from “can we use AI for this?” to “yes, under these conditions” becomes much shorter.

The regulatory landscape is still taking shape. FDA’s January 2025 AI guidance in this area remains draft guidance, while the January 2026 FDA/EMA principles and the final ICH E6(R3) guideline show that formal expectations around governance, transparency, and oversight are continuing to mature. Annex 2 remains a draft.

But the core considerations are not. They are the same ones that have always mattered in regulated clinical research: protect patient data, maintain data integrity, ensure qualified human oversight, document what you are doing and why, and be transparent with your partners and regulators. AI does not change these principles. It gives us new and better tools to uphold them, if we are thoughtful about how we govern the process.

References

1. U.S. Food and Drug Administration. Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products. Draft Guidance. January 2025. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-use-artificial-intelligence-support-regulatory-decision-making-drug-and-biological

2. U.S. Food and Drug Administration. FDA Proposes Framework to Advance Credibility of AI Models Used for Drug and Biological Product Submissions. January 7, 2025. https://www.fda.gov/news-events/press-announcements/fda-proposes-framework-advance-credibility-ai-models-used-drug-and-biological-product-submissions

3. European Medicines Agency. EMA and FDA set common principles for AI in medicine development. January 14, 2026. https://www.ema.europa.eu/en/news/ema-fda-set-common-principles-ai-medicine-development-0


View all news & insights