Code-Execution AI in Regulated Workloads: The Audit-Trail Surface Most Buyers Did Not See Coming

Code-writing vs code-running, different audit problems The procurement question that has changed over the last twelve months is not "are your developers using AI to write code." That question is settled; they are, and the controls are well understood. A developer pulls a Copilot suggestion, reads it, and commits it. The human is the actor on the audit log. Code review, secret scanning, SAST, and the existing change-management trail are the audit surface. The model sits upstream of a human gate, and the gate is where the regulator looks. The question that is not settled is what happens when the AI itself executes code against your data. Code Interpreter inside an OpenAI Assistant. Anthropic's Computer Use and the newer code-execution tool. A Bedrock Agent invoking a Lambda action group against an RDS instance with PHI in it. In each of these patterns, no human committed the code. The model wrote it, ran it, interpreted the result, and decided what to surface back to the user. The audit log the partners would want to see in a HIPAA breach investigation does not exist by default in any of these stacks, and the six things it has to capture are not what most observability tools were built to capture. We are seeing this gap show up across healthcare and regulated SaaS procurement: buyers with a defensible story for AI-assisted developer workflows are signing AI agent products with execution capability and treating the controls as the same conversation. They are not. The six things the audit log has to capture When a model gets execution capability against regulated data, the audit log has to capture six events for every invocation. These map to the HIPAA Security Rule controls the partners would expect to see evidence of in a Type II audit or a post-incident review. 1. The code the model wrote and executed. Verbatim. Not a summary, not a hash, the actual source of every script, query, or command the model generated and ran. This is the equivalent of the change record a human committer would have produced, and it maps to the Audit Controls standard (45 CFR 164.312(b)). 2. The data the code accessed. Every read, table, file, S3 prefix, API endpoint, vector index, with timestamp, identity, and the query or filter that defined the scope. This is where the Minimum Necessary standard (164.502(b)) gets tested. A model that runs on a PHI-bearing table when it only needed three columns is a minimum-necessary violation, and the audit log has to be specific enough to prove the difference. 3. The data the code returned to the model context. Distinct from #2. The model may have read 10,000 rows and returned 14 to its own context window. Both numbers matter, and only #3 shapes the next model action. If a downstream completion is later determined to have leaked, the partners need to know what was in the context window when it did. 4. The side effects. Writes, deletes, external network calls, file-system mutations inside the sandbox, and any tool-call chains the execution triggered. This maps to Integrity (164.312(c)(1)). "The agent did it" is not an answer; "the agent did it under this identity, against this resource, with this diff" is. 5. The identity the code ran under. The IAM principal, the database role, the API token, the sandbox container's effective scope. Most implementations default to a single shared service identity, collapsing every per-user action into one undifferentiated actor. That is a Person or Entity Authentication failure (164.312(d)) waiting to be cited. 6. The result the model interpreted. What the model "saw" when the code returned, and what it then surfaced to the user. The same SQL result can be summarized faithfully or hallucinated against; both versions have to be reconstructable after the fact. Six events, every invocation, retained for the HIPAA six-year window, queryable by patient, user, and time range. None of the major code-execution products produce all six by default. Vendor implementations and what they capture OpenAI's Code Interpreter (the sandbox attached to the Assistants and Responses APIs) captures the executed code and the resulting model message in the run record. It does not, by default, expose container-level filesystem diffs, the data returned to context distinct from the data accessed, or per-end-user identity attribution. Enterprise tier with Zero Data Retention closes part of the retention gap; the granularity gap requires a customer-side wrapper at the API edge. Anthropic's Computer Use and the newer code-execution tool surface the tool calls and the model's interpretation of results. Customer-side capture of the executed command is straightforward; reconciling those events to a single user session and a HIPAA-grade identity record is the customer's responsibility under any BAA we have reviewed in the last quarter. Bedrock Agents with action groups inherit CloudTrail and CloudWatch coverage of the underlying Lambda or Knowledge Base call, the strongest default of the three for events #4 and #5. The model-side events still require explicit logging on the orchestration layer. The default Bedrock invocation log captures prompt and completion; it does not capture the tool-call rationale that produced #1 and #6. Across all three: the sandboxed code itself is captured. The data-access scope, the post-filter context payload, the per-user identity, and the model's interpretation are partial-to-absent. This is a shared-responsibility line that has not yet been clearly drawn in most BAAs we have reviewed. Assume nothing on the customer side of that line until it is in writing. Where the architecture decision shifts Code-execution capability changes the calculus across the three architectures in our HIPAA AI Architecture Field Guide. Architecture A (vendor SaaS, vendor BAA) gets simpler if and only if the vendor has explicitly scoped code-execution under a BAA that names the sandbox, the retention behavior, and the customer's audit-export rights. Today, that is a small subset of available products. Most "we have a BAA" claims do not extend cleanly to the code-execution surface; the partners read the addenda before assuming they do. Architecture B (cloud-native inside the customer's account) shifts from a build to a heavier build. The customer owns the six-event capture pipeline, the identity propagation from end user to sandbox principal, and the retention store. Achievable on AWS, Azure, and GCP, and the architecture most regulated SaaS buyers we work with end up choosing once the audit-trail requirements are spelled out. Architecture C (on-prem or VPC-isolated) is where most code-execution-on-PHI deployments end up for hospital systems and behavioral health groups with a "PHI never leaves the trust boundary" posture. The model, the sandbox, the data, and the audit log all sit inside the same boundary. The lift is real; the defensibility is the highest of the three. Where the Diagnostic Fits, and three actions this week If your team is shipping a code-execution AI workflow against any regulated dataset, or shipped one in the last two quarters and has not yet gone back through the audit-trail review, that is the question the Adopt-AI-Safely Diagnostic answers. Two to three weeks, fixed-scope, fixed-price written report. We document the architecture, walk the six audit events against the implementation, score the BAA chain for explicit code-execution coverage, and produce a sequenced fix plan with cost ranges. The report stands on its own; you keep it whether or not you engage us further on a vCISO retainer. Three actions any CIO, CTO, or compliance lead can take inside the next five business days: 1. Inventory every AI agent or assistant with execution capability. Code Interpreter, Computer Use, Bedrock Agents, custom tool-calling stacks, anything that can run a script the model wrote. Most orgs we audit find one to three more than they expected. 2. Score each one against the six audit events. Where is the gap, who owns closing it, what is the BAA position. A one-page matrix is the artifact you want before the next vendor call, not after. 3. Walk the Audit-Trail Checklist against the workload that worries you most. It is the six-event frame the Diagnostic uses, in a one-pager any compliance lead can drive. The HIPAA AI Architecture Field Guide is at . The AI Governance one-page policy is at . The live AI Watch coverage sits at . The healthcare and regulated-SaaS engagement shapes are at and . The Diagnostic is at and the productized engagement at . The code-execution surface is moving fastest of the three this quarter.