An AI agent that can query your ERP, draft a vendor email, and also approve a $40,000 purchase order is not one system; it's three systems wearing a trench coat. The first is fine to let loose.
The second needs a human to hit send. The third should never have been the model's call in the first place.
Most agent failures we get called in to fix trace back to that confusion: an architecture that let a language model do arithmetic, code, or sign off on something it had no business deciding. Getting this right from the start is why teams increasingly bring in AI strategy consulting before writing a single line of the agent loop.
Why This Split Matters Right Now
By 2026, artificial intelligence agents have reached beyond answering prompts and begun to plan and execute multi-step actions across an ecosystem of connected systems with only limited human oversight, and over 57% of enterprises were already running agents in production.
Gartner forecasts that 40% of enterprise applications will embed task-specific agents by the end of 2026, up from under 5% in 2025. However, high-stakes decisions, such as those requiring a signature or involving a large amount of money, still require human approval rather than autonomous action by an agent.
Where Agent Projects Actually Fail
Most incident reviews we run land on one of three root causes, and none of them is "the model wasn't smart enough."
| Failure mode | What actually happened | Fixed by |
|---|---|---|
| Silent calculator errors | LLM computed totals, discounts, or unit conversions inline instead of calling a function | Routing all math to fixed-rule code |
| Unreviewed high-stakes action | Agent had write access to a system with no approval gate | Human decision checkpoint before execution |
| Failure that cannot be traced | No log of which layer (model, code, human) made a given call | Structured tracing per agent step |
Enterprise agent architecture is often described as five layers from top to bottom: intelligence (LLM), decision (planning/RAG), execution (integrations handled through solid API development), action (orchestration), and finally a learned layer for memory and system monitoring, but this stack says nothing about who gets to decide what within it.
Models vs. Code vs. Humans: Who Should Own the Task
The pattern is simple to state, but can be exceptionally difficult to follow: models are often good at ambiguity, code excels at certainty, and humans do best at consequence. Route by which of those three properties the task actually has.
| Task type | Right owner | Wrong owner (common mistake) |
|---|---|---|
| Interpreting a vague customer request | Model | Code (rigid rules break on phrasing) |
| Calculating tax, totals, or date math | Code | Model (LLMs approximate arithmetic; they don't compute it) |
| Approving a refund over $500 | Human | Model (no accountability if it's wrong) |
| Classifying a document type | Model | Human (too slow, too repetitive) |
| Executing an irreversible database write | Human (with model-prepared context) | Model (unsupervised) |
The mistake is rarely choosing the wrong owner for a very obvious case (no one lets their models wire direct payments, which is exactly the kind of exposure teams building fintech software have to design around from day one). It's the middle cases, where a model double-checks its own math, or a human approves 200 transactions a day because the system made it easier this way than not.
Presence in the loop is not the same as practicing oversight. Most organizations will put someone "in the loop" but not train them on what to do when they find automation lacks attention, or when to even escalate. A human review step that doesn't serve as a control is just theater.
6 Signs Your Agent Has the Wrong Component Doing the Job
An AI agent can have the right tools and still be poorly designed if the wrong component is handling an important task. In a reliable agent system, each job should be handled by the component that is best suited for it.
Here are six signs that your agent may have the wrong component doing the work.
1. The AI Model Is Checking Its Own Math
If an AI model is asked to perform a calculation and then use another LLM call to check the answer, there is probably a better way to handle it.
Important calculations should be handled by code or a dedicated function that returns a fixed and predictable result. This reduces the chance of errors and avoids using an LLM for a task that does not require language reasoning.
2. Almost Every Human Approval Gets Accepted
Human approval can be useful for sensitive or high-risk actions. But if more than 95% of requests are approved without changes, the approval process may not be adding much value.
A high approval rate could mean that the approval threshold is too low or that the task does not need human review in the first place.
The goal is not to remove humans from the system. It is to make sure human review is used where it can actually reduce risk.
3. There Is No Clear Audit Trail
A reliable AI agent should make it possible to understand how an important decision or action was produced.
If you cannot tell whether an output came from the AI model, application code, an external tool, or a human reviewer, troubleshooting becomes difficult.
An audit trail should record important steps, such as which component made the decision, which tool was used, what action was taken, and whether a human approved it. Building this kind of visibility is a core part of the DevOps consulting work that supports agent systems in production.
4. The Agent Can Take an Irreversible Action Without a Checkpoint
AI agents can sometimes perform actions that are difficult or impossible to undo. This could include deleting data, sending a large payment, changing an important setting, or publishing information.
High-impact actions should have safety controls such as approval steps, validation checks, transaction limits, or rollback options.
A useful rule is simple: the greater the impact of an action, the stronger the checkpoint should be.
If an agent can make a serious change without any opportunity to review or reverse it, the architecture may not be ready for an enterprise environment.
5. Business Rules Are Hidden Inside the Prompt
Rules such as pricing limits, rules for eligibility, approval thresholds, and compliance checks are better handled by application code or dedicated rule systems.
Prompts can change as models, instructions, and agent workflows are updated. Putting important business rules in code makes them easier to test, review, and maintain.
For example, instead of telling an agent in a prompt that "orders above $10,000 require approval," that rule can be enforced directly in the application.
6. Escalation Rules Are Too Vague
An instruction such as "If you're unsure, ask a human" sounds useful, but it is difficult to measure and apply consistently.
A better system uses clear escalation rules. For example, an agent could request human approval when its confidence score falls below a defined threshold, when a transaction exceeds $10,000, or when a request matches a specific exception list.
How Models, Code, and Humans Work Together in AI Agents
A reliable AI agent does not need to use an AI model for every task. The strongest AI agent architectures give different responsibilities to the model, application code, and humans.
1. Models Handle Judgment
AI models are useful when a task requires understanding language, dealing with incomplete information, or making a contextual judgment.
For example, an AI agent can interpret a customer support ticket, create a first draft of a response through a well-tuned ChatGPT customization layer, or identify which document is most relevant to a user's request.
However, model-based decisions are not guaranteed to be correct. An LLM produces an answer based on patterns and available context. Its output should therefore be treated as a judgment or best estimate, rather than a fixed fact.
2. Code Handles Calculation
This includes calculations such as totals, taxes, currency conversions, date calculations, eligibility rules, and data validation.
Unlike an LLM, deterministic code should return the same result when it receives the same input.
A simple rule is useful here: if a number can be wrong in a way that a unit test could detect, use a function instead of asking the AI model to calculate it.
For example, an AI agent can understand a customer's request and collect the required information, while a function calculates the final price.
3. Humans Handle Important Decisions
Human involvement is most valuable when a decision has serious consequences or is difficult to reverse.
This does not mean that humans need to review every action taken by an AI agent. Instead, human review should be used for decisions where an error could cause financial loss, damage a customer relationship, or create a compliance or policy problem.
The human is not repeating the agent's work. The human is making the final decision where judgment and accountability still matter.
Making Human Oversight Part of the Architecture
Human approval should not depend only on a written policy or an instruction in the agent's prompt.
The system should technically enforce who can approve an action and what they are allowed to approve. This requires controls such as authentication, authorization, and audit logs.
For example, if an AI agent needs approval before making a high-value transaction, the system should verify the identity of the person approving it and record that approval.
When these responsibilities are clearly separated, AI agent systems become easier to test, monitor, and control.
What's Included in a Properly Architected Agent System
- Task classification layer: routes each sub-task to model, code, or human before execution, not after
- Deterministic function library: calculations, validations, and rule checks that never touch a completion
- Confidence-scored model outputs: every judgment call carries a score the routing layer can act on, often built on top of machine learning development work
- Human checkpoint UI: built for actual review (diffs, source context, one-click escalate) through thoughtful UI/UX design, not a blind approve button
- Structured tracing: every LLM call, function call, and human action logged with which layer owned it
- Rollback/reversal paths: for any action a human or model can trigger, a defined way to undo it
- Escalation rules as configuration: thresholds a compliance team can audit without reading model instructions
Pricing and Engagement Models
The mistake is rarely choosing the wrong owner for a blatantly obvious case (no one lets their models wire direct payments). It's the middle cases, where a model double-checks its own math, or a human approves 200 transactions a day because the system made it easier this way than not.
Presence in the loop is not the same as practicing oversight. Most organizations will put someone "in the loop" but not train them on what to do when they find automation complacency, or when to even escalate. A human checkpoint that doesn't serve as a control is just theater.
| Engagement | Typical cost | Timeline | Best for |
|---|---|---|---|
| Architecture review & pilot | $15,000–$35,000 | 3–5 weeks | Teams with an existing agent prototype that needs the judgment/code/human split retrofitted |
| Single-agent production build | $40,000–$100,000 | 6–12 weeks | One well-defined workflow, approvals, triage, document processing |
| Multi-agent orchestration | $100,000–$300,000+ | 3–6 months | Coordinated workflows across CRM, ERP, and finance with layered HITL gates |
| Ongoing governance retainer | $2,000–$8,000/month | Continuous | Monitoring, threshold tuning, audit reporting, model version control |
These are planning ranges, not quotes. The critical factor is the number of existing systems that the agent will have to connect to and the amount of human-review tooling that the compliance function will need, not the model itself. This is also where digital transformation planning tends to overlap with the agent build itself.
Where This Fits in the Landscape
Low-code automation platforms are the correct choice for a narrow, low-stakes workflow (a FAQ bot, an internal document classifier), for which the cost of an erroneous output is low and easily reversed.
They're the wrong call the moment a workflow touches money, customer-facing commitments, or a system of record, because most of them do not give you a routing layer between model, code, and human; they give you a prompt and a webhook.
Custom builds (RejoiceHub included) cost more up front because that routing layer, the confidence scoring, and the audit trail are the actual engineering work, not the agent loop itself, which is now a commodity. The reasoning loop itself is what most generative AI development teams can now assemble quickly.
The agent reasoning loop is well-documented and not the hard part of a build anymore, even fast, prompt-driven approaches like vibe coding can stand one up in an afternoon. The control layer around it is the hard part.
If your workflow is truly low-stakes, do not over-architect it. A platform tool is the right, cheaper answer, and we'll tell you that in a scoping call rather than sell you a build you don't need.
How to Make AI Agent Routing Enforceable
Defining which tasks belong to the model, code, or a human is only the first step. These rules also need to be enforced by the system.
If the judgment, code, and human split exists only in a design document, the agent may not follow it consistently. A reliable architecture needs clear technical controls at each stage.
1. Use Confidence Thresholds as System Rules
Confidence thresholds should be part of the system configuration, not just instructions in the AI prompt.
When a model produces an output that will trigger another action, the system can assign a confidence score to that result. If the score falls below a predefined threshold, the request can automatically move to human review.
For example, a classification task might require a confidence score of 0.75 or higher before the agent can continue automatically.
The important part is that the routing rule is enforced by the application. The AI agent should not be able to simply decide to ignore the threshold.
2. Use Function Calls for Computable Tasks
If a task can be handled using a formula or fixed set of rules, it should usually be handled by code.
For example, calculations, data validation, pricing rules, and eligibility checks can be implemented as functions with defined inputs and outputs, the same discipline behind a well-built AgentKit builder workflow. These functions can then be tested with unit tests before they are connected to the agent.
The model can provide the required information and use the result, but it should not generate the calculation itself.
Memory systems also have different purposes. Short-term conversation context can help the agent understand the current interaction, while long-term or vector-store memory can help retrieve relevant information from previous data. However, these memory layers should not replace the code layer for deterministic calculations.
3. Use Identity-Based Approval Gates
High-impact actions should require approval from an authorized person, not just a generic "Approve?" response.
A stronger approval process can ask the reviewer to confirm several important points, such as:
- What action is being requested?
- Where did the supporting data come from?
- Does the person have permission to approve it?
- What could happen if the action goes wrong?
- Can the action be reversed?
This gives the human reviewer enough context to make an informed decision instead of simply approving an action because similar requests were usually safe, which is exactly the kind of workflow good user research helps get right before it ships. The system should also verify the identity and permissions of the person providing the approval.
4. Trace Every Step Across the System
A reliable AI agent should provide a complete record of how an important action was produced.
This means tracing model calls, function calls, tool usage, agent decisions, and human approvals from beginning to end.
It is also useful to record which layer was responsible for each step: the model, code, or human. The same rigor applies to the surrounding web development that exposes these traces to the compliance team.
Why Enforceable Routing Matters
The goal is not simply to tell an AI agent which component should handle a task. The system should make those boundaries difficult to bypass.
Confidence thresholds control uncertain model outputs. Function calls keep deterministic work in the code layer. Approval gates keep high-impact decisions with authorized humans. Tracing creates a record of what happened across the entire workflow.
Together, these controls turn an architectural principle into something the system can actually enforce and verify, the same principle behind good generative AI solutions more broadly.
Conclusion
The teams getting agentic AI right in 2026 are not simply using the most capable models. They are building systems that give each component the right job.
Judgment, calculation, and decision-making have different strengths and failure modes, so they should not all depend on the model. Start with one workflow, define where AI should act, where rules should take over, and where humans need to review the result.
Most importantly, make these boundaries enforceable in the system, not just documented in a prompt.
If you are planning an agentic AI workflow, talk to RejoiceHub for an architecture review before you start the full build.
Frequently Asked Questions
What is AI agent architecture?
AI agent architecture is how you split work between three parts: the AI model, plain code, and a human. The model handles judgment calls, code handles math and rules, and a person signs off on anything risky or hard to undo. This split keeps agents safe and predictable.
Why do AI agents make calculation mistakes?
AI models guess at math instead of computing it exactly, so totals, taxes, and date math can come out wrong. The fix is simple: let the model gather the details, but hand the actual calculation to a small piece of code that gives the same answer every time.
What tasks should AI models not handle?
AI models should stay away from exact math, fixed business rules like approval limits, and any high-stakes decision that needs real accountability. Those jobs belong to code or a human reviewer. Models are best used for reading messy text, spotting patterns, and drafting first-pass answers.
What does human in the loop mean for AI agents?
Human in the loop means a real person checks or approves an action before it happens, not just watches from the side. It only works if that person has the context to say no and the system actually stops and waits for their answer before moving forward.
How much does it cost to build an AI agent system?
Costs vary a lot based on the work involved. A review of an existing setup can run $15,000 to $35,000, a single-agent build often costs $40,000 to $100,000, and a full multi-agent system can pass $300,000. The number of connected systems drives the price more than the model.
What is the difference between low-code AI agent platforms and custom builds?
Low-code platforms work fine for small, low-risk jobs like a FAQ bot or a simple document sorter. Custom builds cost more because they add a real routing layer, confidence scoring, and an audit trail. Once money, customers, or core systems are involved, custom architecture is worth it.
How do confidence thresholds work in AI agents?
A confidence threshold is a number built into the system, not just a line in a prompt. If the model's score on a task drops below that number, say 0.75, the request gets sent to a human automatically. This keeps shaky answers from moving forward on their own.
What is an audit trail in AI agent systems?
An audit trail is a clear record showing which part of the system made each move, the AI model, the code, or a person. Without it, fixing a mistake means guessing. With it, you can trace any action back to exactly where it started and who signed off.
When should a human approve an AI agent's action?
A person should step in whenever an action is hard to reverse or touches money, a customer relationship, or compliance risk. Things like refunds over a set amount, database changes, or big payments need a real sign-off, not a system that just clicks approve out of habit.
What is the biggest reason AI agent projects fail?
Most agent failures do not come from a weak model. They come from giving the wrong job to the wrong part of the system, like letting a model do exact math or letting an agent take a risky action with no checkpoint or clear record of what happened.
