Safety architecture

Safety begins with what an agent is allowed to do.

Healthcare agents do more than generate text. They speak to patients, access information and take actions. We design safety around that operational reality.

Our principle

Control capability—not only conversation.

A careful prompt is useful, but it is not a sufficient safety boundary. We also govern access, available actions, confirmation requirements and the point at which human judgement takes over.

An agent should be able to explain its limits because the system itself enforces them.

A clinician and healthcare AI engineer reviewing an agent workflow together
Human authoritySafety is strongest when clinical judgement and system evidence meet in the same review.

Permitted actions

Each agent receives only the capabilities required for its approved use case.

Escalation rules

Conditions for clarification, refusal and human handover are defined before deployment.

Data boundaries

Access, retention and disclosure follow provider policy and the needs of the task.

Human authority

Clinical judgement and higher-risk actions remain with appropriately authorised people.

Research signal

1 in 10

Patients experience harm in healthcare.

WHO estimates that more than half of this harm is preventable. The figure concerns healthcare overall—not AI—but it sets the standard any agent must respect.

WHO · Patient safety

In practice

When the system should stop

If identity, consent, scope or confidence is insufficient, the agent does not improvise. It pauses or hands the interaction to an authorised person with context intact.

Safety through the lifecycle

Designed in. Observed in production.

Safety is a continuous operating discipline—from defining scope through monitoring real interactions and approving each expansion of capability.

01

Before deployment

Map protocols, define prohibited actions, agree escalation thresholds and test representative failure cases.

02

During every interaction

Apply identity, consent, data and action controls in the context of the current patient and task.

03

After every interaction

Retain the evidence needed to review what the agent understood, decided, did and handed over.

04

Across every release

Re-evaluate behaviour before expanding scope or changing models, tools, policies or integrations.

Evidence, not assurance

Every important decision should be reviewable.

Teams need to understand not only the final response, but the context, policy and action path that produced it.

Interaction evidence

Review the conversation, relevant inputs, decisions, tool use and escalation path.

Policy adherence

Evaluate whether the agent remained within approved clinical and operational boundaries.

Outcome monitoring

Connect agent behaviour to patient access, care-team capacity and continuity outcomes.

Make safety part of the system—not a claim around it.

We can map your protocols and show how they translate into agent behaviour.