Engineering
From vibes to verifiable: engineering AI you can put in front of a regulator
The demos are real. Someone types a few prompts, clicks generate, and a working app ships over the weekend. For a prototype, that speed is a gift. The trouble starts when the same prompt-and-pray workflow leaks into production, and by 2026 enough of it has leaked that the bill is coming due. In a regulated environment, where security perimeters, data sovereignty and CPS 230 are not up for negotiation, code built on vibes is an operational risk you have to answer for. This briefing is about what goes wrong, why you cannot see it, and the engineering that fixes it.
The danger isn't the code that won't compile
A bug that throws a red compilation error is the easy kind. You see it, you fix it. The dangerous failures from AI generation are the quiet ones, hidden under an interface that already looks finished.
Columbia's DAPLab studied hundreds of these failures across more than fifteen applications and named nine recurring patterns. Two of the most telling are exception suppression and business-logic mismatch. A probabilistic model is rewarded for producing something that runs, not something that is correct, so it takes shortcuts to keep the happy path green:
- Suppressing errors. Buggy code gets wrapped in a catch-all that swallows the exception, so the failure never reaches you.
- Hardcoding the answer. A placeholder value is returned where real logic should sit, and the demo passes because the demo only ever asks the one question.
- Skipping the check. A validation or security step is quietly dropped because it stood between the agent and a working screen.
Each one produces software that performs perfectly in the room and then, in production, corrupts data, leaks personal information, or falls over under load you never demonstrated. By the time someone goes looking, the cause is buried.
Treat the model as a component, not the system
The fix is not a cleverer prompt. It is to stop expecting the model to police itself. Treat the language model as a fixed, capable component: a reasoning engine you call, not the place where safety and correctness live. Those have to be enforced by the system around it.
That system is the harness, and building it well is its own discipline. The shorthand doing the rounds in 2026 captures it: an agent is a model plus a harness. A good one works in three layers.
- Context, not the whole repository. Tipping an entire codebase into the model's context confuses it and slows everything down. The harness uses search and dependency graphs to hand the agent only the files and schemas the task actually needs.
- A sandbox it cannot escape. The agent runs inside an isolated, zero-trust container and reaches your stack only through defined, rate-limited APIs. It cannot fire a destructive command because the command is not in the room.
- A feedback loop that says no. Linters, static security scanners and unit tests sit in the path and reject non-compliant output on the spot, forcing the agent to correct itself before a human ever reviews it.
Make the rules binding, not advisory
Left alone, an agent treats your system prompt and your style guide as suggestions. If a policy matters, it has to be enforced by the harness, not requested in prose. The loop we run makes that concrete in three steps:
- 1Propose. The agent reads the objective and writes out a step-by-step plan before it touches a file.
- 2Extract. A separate, read-only reviewer agent reads that plan and pulls out the architectural rules, security policies and edge cases that apply to it.
- 3Enforce. Those constraints are injected into the working agent's runtime, and it is blocked from advancing until the current output clears the automated gates.
The point is where the rules live. Move them out of the prompt and into the harness and the build pipeline, and the AI's output is untrusted until it is verified, the same standard you would hold any other contributor to. It is also the standard a supervisor expects you to be able to show.
The bottleneck moved, the engineers didn't disappear
AI coding tools have not made engineers redundant. They have moved the hard part. Typing syntax was never the constraint. Clarity of intent, system design and rigorous verification were, and now they are the whole job.
The leaders who get value from this over the next decade are not the ones chasing the weekend-demo wow. They are the ones building the unglamorous machinery: the sandboxes, the gates, the evidence trail that lets a team move fast with AI and still answer for every line when APRA asks. Vibes ship a prototype. A harness is what you put in front of a regulator.
Key Takeaway
Vibe coding is fine for a prototype and a hazard in regulated production, because its worst failures are silent: suppressed errors, skipped checks, data quietly corrupted behind a screen that looks finished. The fix is engineering, not prompting. Treat the model as a fixed component, wrap it in a harness that controls context, sandboxes execution and verifies every output, and make your policies binding in the pipeline rather than advisory in a prompt. Then your AI output is untrusted until proven, and provable the day you are asked.
Where to from here
AI you can put in front of a regulator doesn't happen by accident.
We build agentic systems for regulated production: harnessed, sandboxed and evidenced from the first commit to the audit.