Agent Infrastructure for Regulated Buyers: The Twelve Pieces Auditors Will Ask About

The plumbing is what the audit asks for A regulated mid-market buyer who has signed a BAA on a frontier model and a SOC 2-attested orchestration vendor has done the procurement work the contract asks for. The work the audit asks for is different. The audit asks what happened on a specific Thursday in March when a clinician initiated an AI workflow that retrieved data on twenty-three patients. It asks for the prompt content, the completion content, the tool calls, the permission decisions, the data sources retrieved, the user identity, the agent identity, the session state at the time of the call, the budget consumed, the stop reason, and the artifacts produced. It asks for those things across a six-year retention horizon. It expects to be able to replay the request from a single queryable surface. The model vendor does not produce that artifact. The orchestration vendor produces some of it. The buyer's engineering team is responsible for the rest, and most of it is not built. The accidental publication of a major frontier vendor's full agent source code in early April 2026, through an configuration error that shipped 1,902 files and over half a million lines of internal code through an npm package, is, in retrospect, the cleanest reference catalog of what a production agent actually is. The model call is roughly twenty percent of the codebase. The other eighty percent is the plumbing: session persistence, permission pipelines, context budget management, tool registries, security stacks, error recovery, memory subsystems, observability scaffolding. The leak was an operational failure for the vendor. For everyone else it is a confirmation that the agent primitives are universal, structural, and not Anthropic-specific. The same primitives are required to build a defensible agent on Bedrock, on Foundry, on a self-hosted Llama deployment, on a Cowork-style enterprise agent, on a custom orchestration on top of the OpenAI API. The buyers who win the next two years of regulated-AI investments are the ones who can name which primitives are present in their deployment and which are not. This is the buyer-side reference for the twelve pieces. It is organized in three tiers, day-one non-negotiables, week-one operational maturity, month-one scale, and every piece is mapped to the HIPAA control or BAA chain link that an auditor will tie it to. Where the vendor's BAA covers a piece, we say so. Where the buyer has to build, we say so. The procurement teams we work with use this list as the explicit checklist on every Adopt-AI-Safely engagement. Tier one, day-one non-negotiables These are the four pieces an agent has to have before PHI touches the request path. They are not optional. A deployment that lacks any of the four is, in our experience, a deployment that produces an audit finding within twelve months. 1. Tool registry with metadata-first design A defensible agent maintains a tool registry separate from the model context. The registry is a structured document, typically a list of objects, each with a name, a description, an , a field, and a (read-only, mutating, destructive). The model receives the registry's metadata; the orchestration layer holds the registry's implementation. Tools are loaded on demand at the orchestration layer, not embedded in the model context indiscriminately. For a regulated buyer this is the pre-requisite for minimum-necessary tool exposure (§164.502(b)) and scoped access control (§164.312(a)(1)). A workflow that does documentation summarization should never have the EHR write API in its tool surface. The registry is what makes that scoping enforceable rather than aspirational. Procurement question: does the orchestration layer expose a tool registry the buyer's security team can audit, scope, and version? 2. Permission system with trust tiers Every tool call should pass through a permission gate before execution. The frontier-vendor source revealed that the simplest of these, a bash execution tool, required eighteen distinct security-evaluation modules, each capable of independently blocking the call: pre-classification of the command, pre-approved patterns, destructive-command detection, domain-specific safety (git operations, file deletions, network egress), sandbox determination, permission logging on both grant and deny. Eighteen modules for one tool. The buyer's environment may not need eighteen. It needs more than zero, and zero is the most common implementation we audit. The HIPAA mapping is direct. Permission decisions are part of the audit log under §164.312(b). A denial is as consequential as a grant, it shows the architecture working. Permission systems that log only granted actions are a structural finding. Procurement question: what is the permission pipeline architecture for tool calls in the agent, and does the audit log capture both grants and denials with the structured reason? 3. Session persistence that survives crashes A session in a regulated agent is a recoverable state object: the conversation, the usage telemetry, the permission decisions, the tool-call outcomes, the configuration. Persisted on disk, indexable, fully reconstructable. A operation reproduces the full state, not just the transcript. The control linkage is contingency planning (§164.308(a)(7)). A regulated workflow that crashes mid-session and cannot reconstruct state is a workflow that has lost the audit trail for the in-flight transaction. Worse, an architecture where session state is held in memory only, with no on-disk persistence, produces audit findings the moment a process restarts. We have seen agent deployments where a single rolling restart of the orchestration container destroyed the session state for forty-three in-flight workflows. The auditor's next question writes itself. Procurement question: is session state persisted before each side-effecting step, and is full state, not transcript only, recoverable on demand? 4. Workflow state and idempotency A session is not the same thing as a workflow. A session is the conversation; a workflow is the work. A defensible agent has explicit workflow states ( , , , , , ) and persists checkpoints after every side-effecting step. Mutating tool calls carry idempotency keys so a retry after a transient failure does not produce duplicate writes. For a regulated buyer this is the integrity control (§164.312(c)(1)). An agent that retries a record write on the EHR and produces two records is an agent that has corrupted the integrity of the underlying data. Idempotency is not a nice-to-have; it is the structural defense against that failure. Procurement question: does the agent maintain explicit workflow state, with idempotency keys on every mutating call? Tier two, week-one operational maturity These four pieces are the scaffolding that turns a working agent into an operable one. They are what allow the buyer's security team to do their job without rebuilding the agent. 5. Token budget tracking with pre-turn checks The frontier-vendor leak surfaced an embarrassing detail: a single session in production was observed to retry indefinitely on failure for 3,272 consecutive iterations, silently consuming tokens and producing no useful output. The fix was three lines. A pre-turn budget check, a hard maximum on consecutive auto-compact failures, and a graceful stop with a reason. The vendor, at the scale of a multi-billion-dollar product, had not implemented it. The buyer-side equivalent risk is twofold. The first is cost: an agent that retries silently is a budget surprise. The second, in a regulated context, is the audit log surface: a failure loop that does not produce a clear stop reason becomes a 3,272-line block of identical entries that the auditor reads as a control gap. Pre-turn budget checks, hard limits, and a clean stop reason are the structural defense. Procurement question: does the agent track and enforce a budget per session, halt before the budget is exceeded, and produce a structured stop reason? 6. Structured streaming events A defensible agent emits typed events during execution: , , , , , . The final event carries usage and stop reason. The buyer's security team is able to subscribe to the stream and intervene mid-thought when an event indicates the agent is heading somewhere it should not. The HIPAA mapping is real-time access monitoring (§164.308(a)(1)(ii)(D)). Audit logs reviewed quarterly do not catch a workflow that produces an unauthorized disclosure in real time. Structured streaming events are how a security team builds the kind of automated alerting that catches the disclosure before it lands. Procurement question: does the orchestration layer emit a structured event stream the security team can subscribe to and alert on? 7. System event logging Separate from the conversation transcript: a system-event log capturing context loading, registry initialization, routing decisions, tool execution counts, permission denials, session persistence. This is the human-readable scan surface the on-call engineer reads at three in the morning. The audit linkage is again §164.312(b), specifically the part of the audit-log requirement that is not captured by the prompt-and-completion log. A regulator's first request includes "the audit trail of the system events surrounding the user's session." If that log does not exist, the answer is "we don't have one," which is not a defensible answer. Procurement question: is there a separate human-readable system event log, with retention that meets the buyer's HIPAA retention obligation? 8. Verification harness Observability is not verification. The agent should ship with an invariant test harness that asserts, on demand, that destructive tools always require approval, that structured outputs validate against their schema, that denied tools never execute, and that budget exhaustion produces a graceful stop. The harness runs whenever a prompt, a model, a tool, or a routing rule changes. For a regulated buyer, the verification harness is the artifact that makes the architecture's claims testable. The auditor's question, "how do you know the destructive-tool-approval gate is still working", has a structured answer instead of an attestation. Procurement question: does the deployment include a verification harness, and is the harness part of the change-management gate (§164.308(a)(8))? Tier three, month-one scale The final four pieces are the layer that distinguishes an operable agent from one that scales across a regulated organization. Buyers do not need them all on day one. They need to know which are missing, and they need a roadmap for adding them. 9. Tool pool assembly A session-specific tool pool, selected from mode flags, the active permission context, and exact-or-prefix deny-lists. The agent's tool surface contracts to the minimum required for the active workflow. Smaller prompts are faster, cheaper, and safer; the audit surface contracts proportionally. This is the operational expression of minimum-necessary at the per-workflow level rather than the per-agent level. A documentation summarization agent and a clinical-decision agent may share an identity but should not share a tool pool. 10. Transcript compaction with provenance Conversation history is a managed resource. Auto-compact after a configurable turn threshold; preserve transcript-store persistence with a join key back to the original entries. Summarization is not truncation. The compaction algorithm has to produce a summary that the auditor can map back to the original prompt-completion pairs. The audit-trail requirement does not relax when the transcript is compacted. A six-year retention obligation on the prompt content is the same retention obligation whether the in-context view of the conversation is the original or a summary. Transcript-store persistence is the structural answer. 11. Permission audit trail Every denial recorded as a structured pair. Three permission handlers: interactive (human-in-the-loop), coordinator (multi-agent), swarm worker (autonomous). Each produces a different audit-log surface; all three feed the same retention. For a regulated environment the swarm-worker pathway is the one that demands the most discipline. An agent acting autonomously without an interactive approval surface is an agent whose every permission denial is the audit's only evidence that the architecture is doing its job. Treat the denial log as a primary control surface rather than a developer convenience. 12. Doctor pattern, staged boot, stop-reason taxonomy, provenance-aware context, agent-type system, memory subsystem, skills/extensibility The remaining pieces, and the ones most often missing from custom agent deployments, are the boot-time, end-of-session, and retrieval-side primitives that distinguish a robust agent from a fragile one. A startup endpoint that validates credentials, external connections, configuration integrity, and tool availability. A staged boot sequence that establishes situational awareness before the user's first prompt. A stop-reason taxonomy with a finite enumerated set ( , , , , , ). Provenance-aware context assembly so every retrieved fragment carries a source, an age, a trustworthiness score, and an instruction-vs-evidence flag. An agent-type system that constrains roles when work is split (an "explore" agent that cannot write files; a "plan" agent that cannot execute). A memory subsystem with relevance scoring, aging, type categorization, and scoping (personal, team, project). A skills layer that is the middle ground between rebuilding the workflow each session and hard-coding it into the application. For a regulated buyer the most consequential of these is provenance-aware context assembly. Every retrieved fragment in a RAG pipeline that touches PHI has to carry source attribution. Without it, memory and retrieval become another prompt-injection surface, a place where an attacker can inject content that the agent treats as instruction. We have audited deployments where the RAG pipeline pulled clinical notes, public-source news articles, and engineering team Slack messages into the same context window with no source distinction. The model treated all of it as instruction. The audit trail had no answer to "what content was treated as authoritative." How to read this list as a buyer The twelve-piece list is not a build manifest for the buyer's engineering team. It is a procurement checklist. For each piece, the buyer's question is: who is responsible, the model vendor, the orchestration vendor, our engineering team, or unowned? "Unowned" is the answer that produces audit findings. The four-architecture frame from the HIPAA AI Architecture Field Guide, Architecture A (hosted SaaS), Architecture B (cloud-native, self-managed), Architecture C (on-prem), Architecture D (hybrid, no PHI to model), distributes the twelve pieces differently. Architecture A pushes most of the twelve to the SaaS vendor, with the buyer responsible for procurement-level checks. Architecture B pushes the bottom of the stack (registry, permissions, sessions, idempotency, budget) to the cloud's agent platform; the upper tier (verification, audit trail, provenance) sits squarely with the buyer's engineering team. Architecture C and D push everything to the buyer. The procurement-time deliverable is a twelve-row table, one row per piece, with a column for the responsible party, a column for the BAA scope that covers it, a column for the audit log surface where the piece's output lands, and a column for the gap. The gap column is the working list of architectural findings the buyer is committing to close. We run this table in week one of every Adopt-AI-Safely Diagnostic, and we list it as a deliverable in every written assessment. The buyers who keep the table updated are the buyers whose audit posture compounds. The buyers who treat it as a one-off exercise are the buyers we end up re-engaging two years later because the next AI tool is being procured against a different list. The model is twenty percent of the agent. The other eighty percent is the plumbing, and the plumbing is the artifact a regulator's first request will ask you for. Build the list. Assign the rows. Close the gaps.