Back to articles🏢Enterprise AI

Bridging the AI and Security Team Gap in Healthcare

Two teams, two languages, one catastrophic blind spot: healthcare's AI security gap isn't a tech problem—it's a translation problem nobody scheduled a meeting to solve.

Paul Lopez
··12 min read
Your Threat Model Is Written in a Language Half the Room Cannot Read

Your Threat Model Is Written in a Language Half the Room Cannot Read

How the AI team and the corporate security team should actually split the work of protecting a healthcare AI system, and why frontier model capability just moved the deadline up.


In 1799, French soldiers digging fortifications near Rosetta, Egypt pulled a stone from the ground that carried the same royal decree in three scripts: Ancient Egyptian hieroglyphics, Demotic, and Ancient Greek. The stone sat in European hands for decades before scholars realized the key was alignment. Once they placed the scripts side by side and read them against each other, a language that had been sealed for centuries opened up completely. The content was never the problem. The isolation was.

Most healthcare organizations are running the same failure right now, just with worse consequences than a misfiled artifact. In one room, the security team maps adversarial techniques, debates controls, and produces threat models fluent in MITRE ATLAS notation. In another room, clinical and domain leaders document the patient harms they fear and the regulatory exposure they cannot afford. Both rooms are describing the same risk. Neither can read the other's script. And nobody has put the stone on the table where both groups can see it at once.

This article is about the operating model that fixes that. Not the frameworks, which already exist. The operating structure: who owns what, where the seam runs, what the joint session actually produces, and why the urgency just increased.


There Is No Healthcare ATLAS, and That Is the Point

There is no exact healthcare equivalent to MITRE ATLAS, and looking for one misses how the problem actually decomposes. ATLAS is the adversarial catalog: a living knowledge base of the tactics and techniques used against AI-enabled systems. Healthcare did not fork it. Healthcare built an overlay on top of it.

Three Framework Layers of Healthcare AI Risk

The 2026 HSCC AI Cybersecurity Governance Framework is the closest current healthcare companion to ATLAS, and it is explicit that it does not replace ATLAS. It tells healthcare organizations to use ATLAS as the shared threat vocabulary, then add the healthcare-specific layer: the clinical scenario, the likelihood, the patient impact, the controls, and the residual risk. That is a profile, not a competitor. The catalog says how the attack works. The overlay says what it means when the attacked system is deciding whether a patient gets care.

A third layer completes the picture. OWASP's Top 10 for Large Language Model Applications, now on version 2.0, covers the application risks that sit below the model and above the infrastructure: prompt injection, insecure tool use, excessive agency, and the vulnerabilities specific to retrieval-augmented and agentic systems. ATLAS tells you the adversarial technique. HSCC tells you the clinical impact. OWASP tells you where it enters your application.

Three complementary sources. Three different layers of the same risk. None of them alone is an operating model. They are frameworks, and a framework tells you what to think about, not who does the thinking or when the two rooms finally share a table.


The Ownership Question Everyone Gets Wrong

The reflexive answer is that the CISO owns AI security and the domain team owns clinical risk. That answer is clean on an org chart and wrong in practice. The security team can map a scenario to AML.T0020 data poisoning with precision, and still not know that poisoned prior-authorization criteria could delay an oncology treatment or trigger a wave of inappropriate denials. The domain expert can see the clinical catastrophe coming and have no idea which attack technique produces it or which control stops it.

Neither person can do the full analysis alone, and that is not a soft observation. It is the design constraint that should shape the entire operating model. The correct framing is not division. It is paired ownership with a joint seam down the middle.

The split looks like this in practice. The healthcare AI domain expert owns the healthcare implementation of the HSCC framework, use-case risk assessment, clinical and operational impact analysis, human-oversight design, healthcare-specific failure scenarios, and the clinical and patient-safety acceptance criteria that define what a tolerable residual risk looks like. The CISO organization owns enterprise adoption of ATLAS, AI security control standards, threat intelligence, security architecture, red-team methodology, vulnerability and incident management, and security risk acceptance at the organizational level.

AI Security Ownership: The Joint Responsibility Model

The joint zone is where the actual work lives. AI threat modeling, risk scoring, control selection, test scenario design, third-party assessments, go-live approval, incident classification, and post-deployment monitoring all belong to both owners simultaneously. That joint zone is not a handoff. It is a shared table, and the decisions made there require both scripts to be in the room.


What the Joint Session Actually Produces

The abstraction is useful only as far as a concrete example takes it. Consider a prior-authorization AI agent that reviews clinical documentation and recommends coverage decisions. It is exactly the kind of system that sounds like a productivity win and operates at the precise intersection of clinical consequence and adversarial risk.

The domain expert brings the consequence map. A poisoned payer policy entering the retrieval index could produce a cascade of inappropriate denials. A prompt-injection payload hidden inside a clinical document could redirect the agent's reasoning without triggering any model-layer safeguard. A hallucinated medical criterion could delay care that should have been approved immediately. Excessive agent autonomy could let an incorrect determination reach a provider without any human review. And pattern analysis of model responses could allow an attacker to infer protected health information from what the system does and does not output. That is five distinct failure modes, each with a patient on the other end.

The security specialist then reads the same scenario in a different script. AML.T0020 maps to the poisoning risk. OWASP LLM01 maps to the prompt-injection entry points, both direct and indirect. AML.T0015 maps to the adversarial input scenarios. Model-extraction methodology maps to the inference-attack risk on PHI. Controls follow: input isolation, provenance requirements, least-privilege access for agent tools, logging configured to surface injection patterns, and detection logic tied to those logs.

The joint output is what matters most. Only validated policy sources may enter the retrieval index. Every retrieved criterion carries its source, version, and effective date. External documents are treated as untrusted input regardless of origin. The model cannot directly issue an adverse determination. Low-confidence, conflicting, or high-impact cases require human review before any decision reaches a provider. Policy-source changes trigger revalidation. Security incidents are classified on the same scale as clinical incidents, not a parallel track that never intersects.

That output is the Rosetta Stone made operational. Two experts, two scripts, one aligned artifact that the whole organization can read.


Structure It So It Scales: Three Lines of Defense

A joint table sounds expensive if you imagine every decision going through a committee. It does not have to. A mature healthcare organization can run this through a three-lines structure that keeps the paired work bounded. The first line is the AI product, engineering, and healthcare domain teams who build and operate the system, document risks, and implement controls. The second line is the CISO, privacy, clinical safety, compliance, and enterprise AI governance functions that set standards, challenge the design, and approve risk treatment. The third line is internal audit or independent model validation that verifies the first two lines are actually working.

NIST's AI Risk Management Framework recommends the same instincts: differentiated roles across design, development, testing, and oversight; independent red-team and test functions; and clear separation between the team that builds and the team that evaluates. The paired model is not an improvised structure. It is what the national framework already expects, which means adopting it now positions an organization ahead of the eventual audit question, not behind it.


Why the Deadline Just Moved Up

Google DeepMind's Big Sleep agent found an exploitable memory flaw in SQLite, one of the most widely deployed databases in the world, before any human researcher had flagged it. Google Project Zero described it as the first public example of an AI agent discovering a previously unknown, exploitable memory-safety vulnerability in widely used real-world software. The SQLite maintainers received the report and patched it the same day. DARPA's AI Cyber Challenge, run at DEF CON 32 in August 2024 with Anthropic, Google, Microsoft, and OpenAI as technical advisors, went further: multiple AI systems identified real vulnerabilities in critical infrastructure software including the Linux kernel. DARPA designed the entire program around the premise that AI is approaching the capability to discover vulnerabilities faster than human teams can review and patch them.

That capability is dual-use by definition. The tooling that helps a skilled defender find a flaw in SQLite before an attacker does is the same tooling that helps a less scrupulous researcher probe a production prior-authorization agent for prompt-injection chains, poisoning entry points, or model-extraction opportunities. Anthropic's Responsible Scaling Policy acknowledges this directly. Under their published framework, ASL-3 is triggered when a model can provide meaningful uplift toward weapons capable of mass casualties or can autonomously conduct sophisticated cyberattacks. Anthropic commits to implementing specific safeguards before deploying any model that crosses an ASL threshold. The framework exists precisely because the underlying capabilities are not neutral.

Export controls are moving in the same direction, though they currently target hardware rather than model API access directly. The Commerce Department's AI diffusion rule, updated in January 2025, restricts advanced AI semiconductors, and rulemaking around dual-use AI capabilities is ongoing. The regulatory perimeter is being drawn. Where it lands on frontier model access is an open question, but the direction is not.

The practical implication is simple. Adversarial tooling for probing deployed AI systems is getting cheaper, more capable, and more accessible at exactly the moment healthcare organizations are deploying AI into clinical workflows. A paired operating model stops being a maturity nicety at that point. It becomes the thing standing between an adversary and a decision system that determines whether a patient gets care.


The Honest Counterargument

The fair objection is that this adds overhead. A joint threat-modeling session, a shared artifact, and a three-lines structure all cost time, and a single accountable owner would move faster. That objection is correct about the cost and wrong about the trade.

The coordination cost is real. A single owner does move faster. The question is what a single owner misses. A security team working alone maps, controls, and signs off a threat in ATLAS language while the clinical consequence goes unread. A domain team working alone escalates a clinical risk with no attack technique, no control, and no detection logic attached. In healthcare, either failure has a cost that exceeds the joint session that would have caught it. A mis-classified threat means delayed care, protected health information exposure, or a regulatory finding. None of those are recoverable on a budget that was protecting against the wrong thing.

The three-lines structure keeps the overhead bounded. The paired work is not every operational decision. It is the threat-modeling sessions where scope and risk are defined, and the approval gates where go-live, risk treatment, and incident classification require both scripts. Routine operations run in the first line without a committee. The joint table appears at the seams where the two languages must align, which is exactly where the Rosetta Stone analogy earns its keep.


The Alignment Was Always the Point

The value of the Rosetta Stone was never in the hieroglyphics, and it was never in the Greek. It was in placing both scripts against each other so the meaning locked into place. That is exactly what a paired AI and security team produces: not two separate risk assessments filed in two separate systems, but a single aligned artifact that the clinician who accepts the risk and the security engineer who monitors the system can both read.

The organizations that build that paired table now, before adversarial tooling reaches the capability level where probing a clinical decision system is a weekend project, are the ones whose threat models will still be readable by the people who have to act on them. The scripts exist. The frameworks exist. What most organizations are still missing is the shared table. Build it first.


For the governance-layer architecture that sits above this operating model, see "AI Governance's Missing Pillar" at paullopez.ai.


References

[1] Google Project Zero and Google DeepMind. "From Naptime to Big Sleep: Using Large Language Models To Catch Vulnerabilities In Real-World Code." Google Project Zero Blog, November 2024. https://googleprojectzero.blogspot.com/2024/11/from-naptime-to-big-sleep-using-large.html

[2] Defense Advanced Research Projects Agency (DARPA). "AI Cyber Challenge (AIxCC): Competition Overview and DEF CON 32 Results." 2024. https://aicyberchallenge.com/

[3] Anthropic. "Anthropic's Responsible Scaling Policy, Version 1.1." 2024. https://www.anthropic.com/news/anthropics-responsible-scaling-policy

[4] OWASP Foundation. "OWASP Top 10 for Large Language Model Applications, Version 2.0." 2025. https://owasp.org/www-project-top-10-for-large-language-model-applications/

[5] NIST. "AI Risk Management Framework (AI RMF 1.0)." National Institute of Standards and Technology, January 2023. https://doi.org/10.6028/NIST.AI.100-1

[6] NIST. "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-4)." National Institute of Standards and Technology, 2024. https://doi.org/10.6028/NIST.AI.100-4

[7] U.S. Department of Commerce, Bureau of Industry and Security. "Export Controls on Advanced Computing Integrated Circuits and Related Items: Implementation of Additional Safeguards." Federal Register, January 2025. https://www.federalregister.gov/documents/2025/01/15/2025-00636/export-controls-on-advanced-computing-integrated-circuits

#enterprise-ai#healthcare-ai#ai-security#threat-modeling#cross-team-collaboration