How to Build an AI Agent for Your Business: Architecture, Costs, and Production Challenges

An AI agent is useful when software must interpret a request, choose from a small set of actions, and adapt its next step to what a tool returns. That can help with work such as triaging support requests, assembling account research, or preparing a draft from several internal systems. It does not make every business process a good agent problem.

Before choosing a framework, write down the task, the permitted actions, the cost of a wrong result, and the point where a person must take over. Those decisions determine the architecture more than the word “agent” does.

What counts as a business AI agent?

A useful distinction is whether the model can choose what to do next. A fixed workflow that always runs steps A, B, and C is conventional automation, even if one step calls a language model. An agent has bounded discretion: it can select among tools or repeat a step based on the result, within an explicit limit.

Good candidates have clear inputs and outcomes, enough variation that a fixed decision tree becomes awkward, and actions that can be checked or reversed. Examples include categorizing an inbound request and preparing a CRM update for review, or searching a knowledge base and drafting a cited response.

Avoid an agent when a rule is more accurate, cheaper, and easier to audit. Tax calculations, entitlement checks, payment settlement, and deterministic routing generally belong in ordinary code. The model may explain a result, but should not become the authority for the underlying business rule.

Define the boundary before selecting a model

Write the task as a contract:

This also reveals whether an agent is feasible. If the source systems have no stable API, inconsistent identifiers, or permissions that cannot be represented safely, that integration work may dominate the model work.

A small architecture is easier to govern

For many first releases, one orchestrator with narrow tools is enough. Multiple agents add handoffs, shared-state questions, additional model calls, and more failure paths. Add them only when a measured separation of responsibility improves the workflow.

Authenticated user
      │ request + tenant scope
      ▼
FastAPI endpoint ──► policy and input validation
      │                         │
      ▼                         ▼
bounded agent run          scoped tool registry
      │                    ├── read CRM record
      │                    ├── search approved knowledge
      │                    └── propose pending update
      ▼
run state + tool results + usage trace
      │
      ├── low-risk result ──► validate ──► return with sources
      └── consequential write ──► human approval ──► execute once

The model proposes tool arguments. The server authenticates the caller, determines the tenant and allowed actions, validates arguments, and executes the operation. Never use a model-generated tenant ID, user ID, or permission claim as authorization. The model is a decision component inside the application boundary, not an identity provider.

Single agent or several?

Start with one model-driven controller and a small tool set when the task shares one context and one policy boundary. A multi-agent design may help when work divides into independently scoped stages, such as research followed by a separate verifier, but it also multiplies prompts, state transitions, and places where the system can lose evidence. It does not inherently make a workflow more accurate.

Before adding another agent, check whether a typed function, deterministic branch, or separate workflow activity would be simpler. If two roles need different permissions, separate service boundaries may be more important than separate prompts. Measure the extra latency and model usage, and ensure the handoff carries citations and a defined output schema.

Choosing an LLM

Choose a model against a task-specific evaluation set. Check structured-output behavior, tool-call quality, language coverage, latency, provider availability, data-processing terms, and usage price for the expected input and output sizes. A model leaderboard or a successful demo does not tell you how it handles your workflow's ambiguous examples.

Keep the provider adapter narrow: application code should request a task such as classify_support_case, rather than assembling provider-specific request details throughout the codebase. This makes model changes possible, but does not promise that models are interchangeable. Re-run the same evaluation and inspect quality and cost tradeoffs after a swap.

A FastAPI boundary for a proposed action

The following sketch shows where trusted identity and policy checks belong. current_user, the agent call, and repositories are application dependencies; they are not implemented by this snippet.

from fastapi import Depends, FastAPI, HTTPException
from pydantic import BaseModel, Field

app = FastAPI()

class AgentRequest(BaseModel):
    question: str = Field(min_length=1, max_length=4000)

@app.post("/workspaces/{workspace_id}/assist")
async def assist(
    workspace_id: str,
    request: AgentRequest,
    user = Depends(current_user),
):
    membership = await memberships.get(user.id, workspace_id)
    if membership is None or not membership.can_use_assistant:
        raise HTTPException(status_code=403, detail="Not allowed")

    result = await agent.run(
        question=request.question,
        scope=TrustedScope(user_id=user.id, workspace_id=workspace_id),
        max_steps=4,
    )
    return await validate_and_record(result, membership)

The route should verify membership before it calls the agent. The tool implementation must repeat the relevant authorization check at the resource boundary, especially if a job can continue after the request has ended. A model-provided workspace ID must not replace TrustedScope.

Connect tools as production integrations

A tool is an API surface with a business contract. Its description should state what it does, when it must not be used, required arguments, side effects, and expected errors. Keep read-only tools separate from write tools. Return compact structured results with stable identifiers and source timestamps rather than long prose.

Real integrations require more than an HTTP request. Plan for OAuth consent and token refresh, per-user or per-workspace access, rate limits, pagination, changed schemas, timeout behavior, webhook duplication, and provider outages. Confirm that the connected account is allowed to access each record. A successful API response does not imply the end user may see its contents.

For a write, create a proposed action record first. Apply deterministic validation and business rules; show the intended change to a person when the impact warrants it; then execute with an idempotency key. If a timeout occurs after the remote service applied a write, a retry must reconcile the outcome instead of creating a duplicate.

Use state for continuity, not as a substitute for truth

An agent run needs temporary state: the user goal, prior tool observations, and step count. Durable product facts belong in the system of record. Retrieved documents are evidence with provenance, not automatically trusted memory. Store a run ID, tool inputs and outputs with sensitive fields redacted, model and prompt versions, and the final outcome according to a retention policy.

If the task spans minutes or hours, use a durable job or workflow system so a process restart does not erase the work. FastAPI's in-process BackgroundTasks is intended for small after-response tasks; its documentation points to a queue or worker system for heavier work that may need multiple processes (FastAPI background tasks).

Build recovery and approval into the workflow

Classify errors before retrying. A timeout or rate limit may be transient; invalid credentials and schema errors usually need intervention. Use bounded retries with backoff, a maximum run duration, and a terminal state that an operator can inspect. Make each side-effecting step idempotent or provide a reconciliation query.

Human review should show the proposed action, the evidence used, and the consequences. “Approve” should authorize one specific action, not grant the agent a broader permission for the rest of the session. Provide cancel, retry, and escalation paths. Keep an audit trail that answers who requested the run, which policy allowed the tool, and what actually changed.

Keep approval state durable if a person may respond later. Record the exact normalized payload and a hash or version for the source record it was based on. At approval time, re-check that the user can still approve the action and that the target record has not changed. If it has, regenerate or ask the reviewer to confirm the updated proposal. This avoids applying stale intent to a newer business record.

Evaluate the complete job, not just the model response

Create a small representative set of tasks, including missing data, ambiguous instructions, stale records, provider errors, and requests that should be refused. Measure task completion, correct tool choice, field-level accuracy, evidence coverage, human edit rate, latency, and cost per accepted outcome. Track severe errors separately; a high average score must not hide an unsafe write.

Run evaluations when prompts, models, tool schemas, retrieval, or policies change. In production, sample traces, monitor tool error rates and approval overrides, and provide a way to disable writes. The agent implementation guide from OpenAI also recommends starting with the simplest architecture that meets the use case and increasing complexity only as needed.

What drives implementation and operating cost?

There is no meaningful price from the phrase “build an AI agent” alone. Scope depends on the workflow count, integration quality, identity model, data cleanup, expected volume, review interface, audit needs, evaluation coverage, and deployment requirements. Runtime costs include model input/output, retrieval and storage, integration APIs, background workers, observability, and human review.

Treat cost as a measured unit of work. Record provider, model, token usage, retries, tool calls, and accepted outcomes. Add per-workspace budgets and alerts before broad rollout. A cheaper model can still increase total cost if it needs repeated retries or creates more review work.

For a basic estimate, separate the unknowns instead of multiplying an arbitrary hourly rate by a guessed number of weeks. Count workflow paths and exceptions; list each integration and whether it has a usable sandbox; define the review UI and audit needs; then estimate build, evaluation, deployment, and support work independently. Spike the riskiest dependency first. The result is a range attached to stated assumptions, with a clear list of discoveries that would change it.

Maintenance continues when a provider changes a response format, an OAuth grant expires, a business field is renamed, or users find a new edge case. Assign an owner for credentials, failed runs, evaluation fixtures, and model configuration. An agent without operational ownership becomes a fragile integration even when the model itself has not changed.

For an MVP, choose one workflow, one or two read integrations, a draft or pending-write output, and a manual review step. A production rollout adds scoped authorization, durable state where needed, failure recovery, auditability, evaluation, operational alerts, and a support owner. Expand autonomy only when evidence shows the constrained path is working.

Make the rollout reversible. Begin with internal users or a narrow cohort, compare the agent's proposed decisions with the existing process, and inspect disagreements. Add customer-facing or write actions only after the system meets the agreed quality floor and operators know how to pause it. A feature flag should be able to disable model execution or writes without taking down the rest of the product.

Keep one human owner for the workflow's policy and one technical owner for its runtime. That makes it clear who can approve a new action class, update evaluation cases, and respond when an integration stops working.

A practical planning checklist

• Name the user, trigger, and business outcome.
• Show the current manual process and its exceptions.
• List source systems, API access, account ownership, and required scopes.
• Mark each action read-only, reversible, or consequential.
• Identify personal, customer, or regulated data and retention needs.
• Define what the agent must refuse or send to a person.
• Agree on representative test cases and unacceptable errors.
• Identify who reviews, monitors, and supports failed runs.

You do not need complete answers before discussing a project. A description of the workflow and the systems involved is enough to start; the contact page explicitly supports early conversations.

Further reading

• How I build bounded AI agents
• Recovering agent workflows after failure
• Evaluations and human review
• ProfMe engineering project
• Google's people-first content guidance