Create an AI agent: method, guardrails and acceptance criteria

This page is for marketing, operations and IT directors who want to create an AI agent that is useful in production, and not just an assistant that answers. It sets out the order of decisions: what is written before a tool is chosen, how far the agent is allowed to act, which building blocks are assembled, and how you can tell it is ready.

It assumes the general frame of our overview of AI agents is already in place, and reuses its four permission levels and the seven families of agents.

What has to be written before you build an AI agent

An agent only creates value if it executes in your environment (data, tools, processes) and if you can prove what it has done. Before connecting a single API, define a measurable steering frame: limits on action, indicators tracked, complete traceability. Without that, you are “automating” without making the autonomy acceptable.

Set these prerequisites down on one page, the one you will reread in six months to decide between the agent's rights and what it is being asked to do.

• Objective: an observable result (cut the time spent triaging technical anomalies, speed up updates to high-impact pages).
• Data: permitted sources, update frequency, minimum quality.
• Access rights: read-only or write, separate environments (dev/staging/prod), least privilege.
• Success criteria: before/after indicators (turnaround, error rate, volume of recommendations accepted).

Two decisions are taken at that moment, and no later: the agent reads everything but writes little, within a controlled frame; its rights on your publishing tools stay limited to drafts for as long as the measurement loop has not been validated. These rules cost little at the start and become almost impossible to impose after a first success.

Choosing the level of agency, and where the agent stops

Most failures come from a badly chosen “level of agency”: too autonomous too early, or not tooled enough to act. An agent perceives a context, reasons according to an objective and acts through tools (APIs, files, tickets); what separates it from an assistant is the orchestration of traceable actions. Reduce the ambition, then industrialize: the right level depends on three factors.

• Risk: brand image, compliance, irreversibility of the actions.
• Complexity: number of steps and sources to reconcile.
• Access to tools: what the agent can actually read or change.

Four levels, from diagnosis to bounded autonomy

Autonomy is not binary: it is set, and set in writing. Each level commits you to a different control set-up, and it is that set-up, not the capability of the model, that decides what you can deploy. These four tiers match the four permission levels of an AI agent in the overview.

Moving from one level to the next is decided on the execution logs of the previous level. An agent that has held three weeks under supervision without an automatic stop has demonstrated something; an agent that is “pretty good” has demonstrated nothing.

Stop thresholds, and why the chain has to stay short

Reliability does not degrade in proportion to the number of steps: it collapses as the chain lengthens, each step inheriting the approximations of the one before. So cut the number of steps, and move the checks out into code and rules rather than into the model.

Your design also has to say where the agent stops and what happens then: a clean stop, then a handover to a human with the context, never a blind retry. The stop cases are always the same.

• Insufficient evidence: the signal exists, but the measurement that justifies it is missing.
• Inconsistency: two sources contradict each other on the same data point.
• Irreversible action: the effect could not be undone.

An agent that does not know how to stop is not an autonomous agent: it is an agent nobody is watching.

The four building blocks of an AI agent: model, tools, memory, orchestration

Building a robust agent comes down to separating “thinking” from “acting”, then instrumenting each step. The model interprets, plans and produces structured outputs; your business rules are written in code and thresholds, where they stay readable, testable and changeable without touching the model. This split makes the behaviour reproducible.

The toolbox and the memory: what the agent reads, what it keeps

An agent only executes if it can call tools, and each tool exposes limited, documented functions controlled by permissions: a catalogue of narrow, auditable functions, in three categories.

• Data connectors: extraction of measurements, inventory of pages, editorial templates.
• Action functions: create a ticket, propose a fix, create a draft, request a validation.
• Validators: output schemas, consistency rules, risk thresholds.

On the memory side, short-term memory holds the thread of the task in progress; persistent memory builds things up (decisions, exceptions, preferences), but it raises the compliance and security stakes. When the agent decides from internal documents, controlled grounding on a corpus is the subject of the RAG AI agent.

The plan → action → observation → decision loop

A robust agentic loop looks like a control system: it observes, decides, acts, then observes again. Separate a “plan” loop, which proposes a list of actions, from an “execution” loop, which applies one action at a time, checking the preconditions and logging the effect.

• Plan: select a candidate priority action and produce a strictly structured output.
• Action: execute through a connector, with idempotency.
• Observation: collect the metrics and the state, before and after.
• Decision: carry on, escalate to a human, or stop (threshold reached, risk).

This separation reduces drift and makes supervision possible. If you code this loop yourself, structure, patterns and tests belong to the Python AI agent. And when the problem becomes several agents, AI agent orchestration takes over.

Writing the specification and the decision protocol

The quality of an agent depends less on the model than on your specifications. In B2B, traceability is a prerequisite for acceptance, not a bonus. Nothing requires writing code: the same specification holds in a visual builder, and it is its content that protects you. An agent becomes unpredictable as soon as it does not know three things.

• What to produce: the expected output format.
• When to stop: the thresholds that trigger a stop or an escalation.
• How to prove: the logs that link the action to its decision.

The specification, a contract between the model, your tools and your team

An agent specification is a contract between the LLM, your tools and your team. It reduces ambiguity, and therefore drift, and it makes automated tests possible. Write it before automating, under four headings.

A specification that fits on one page is worth more than an exhaustive document nobody will reread. The test: a colleague from outside the project should be able to say, by reading it, whether an output complies.

The decision protocol: scoring, business rules and guardrails

A useful agent does not “recommend” at random: it arbitrates. Your protocol has to make the decision explainable and stable, even when the model varies, through three mechanisms.

• Scoring: estimated impact, effort, risk, dependencies, urgency.
• Business rules: prohibitions (legal pages), thresholds (minimum traffic), windows (release freeze).
• Guardrails: stop if the evidence is insufficient, if the data is inconsistent, or if the action is irreversible.

Written this way, the protocol becomes contestable: a decision can be replayed, discussed, then corrected by changing a rule, not by rewriting a prompt and hoping the behaviour changes.

Handling error and proving what was done

Errors are not “exceptions”: interfaces unavailable, quotas reached, incomplete content and unexpected formats are normal operation. A robust architecture formalizes recovery instead of blindly retrying; drawing those branches step by step is the subject of the AI workflow agent. Five mechanisms need to be planned.

• Timeouts: cut cleanly, then escalate.
• Retries: only if the error is transient, and a limited number of times.
• Idempotency: a unique operation identifier, to avoid duplicates.
• Rollback: undo, restore, or create a correction ticket.
• Handover: pass control to a human with the context, the logs and a proposed resolution.

Logging the decision, not only the action

Without logs, you can neither debug nor convince. Log the action, but also the decision and its evidence, and version prompts and rules: otherwise there is no way to say what changed between a correct run and a failed one.

• Run: identifier, timestamp, input and output, model used.
• Decision: scores, rules applied, thresholds triggered.
• Evidence: state before and after, the original measurement, the page concerned.
• Versioning: prompts, schemas, connectors, rules, configuration.

The first two objects serve whoever has to fix the agent; the last two, whoever has to authorize a wider scope. Purely technical logs never get past the committee stage.

Classifying the data, and closing what the agent has no business opening

66% of users rely on AI outputs without checking their accuracy (Squid Impact, 2025). Verification therefore has to be carried by the system, not by goodwill: a whitelist of functions, strict validation of inputs against booby-trapped content, secrets outside the code. Service accounts, rights per environment and secret rotation are detailed in AI agent integration. Finally, classify the data into three categories before integrating it.

• Acceptable with conditions: aggregated metrics, public URLs, non-sensitive extracts.
• Forbidden: personal data, secrets, credentials, internal documents that have not been anonymized.
• To be validated: detailed logs, content before publication, customer data.

Acceptance testing before letting go, then monitoring

56% of users say they have made mistakes because of AI (Squid Impact, 2025). It is the test set that reveals the residual error rate, never the demonstration, which shows only the nominal path. Test like software: the question is not “does it work”, but “what happens when it does not”.

Test sets, acceptance criteria and non-regression

Acceptance testing for an agent is prepared like that of a software component, in three parts. They fit in a single document, reread at every change of prompt, rule or connector.

• Test sets: “normal” pages, sensitive pages, missing data, quotas reached.
• Acceptance criteria: rate of valid outputs, escalation rate, rate of actions correctly refused.
• Non-regression: same input → same decision, within an acceptable range.

The third part is the one people forget, and the most discriminating: an agent whose decision varies on an identical input is not ready. The correct refusal of a forbidden action is a test result just as much as a success.

What you monitor once in production

Without metrics, steering is impossible. Monitor three families of indicators, and draw a clear line between inference cost and operational cost (maintenance, incidents, human validation), because the second decides the real profitability.

• Agent metrics: success rate per step, latency, escalation rate to a human.
• Business metrics: volume handled, backlog cleared, effect measured over a consistent period.
• Costs: model calls, API calls, storage, supervision.

An agent ages fast: interfaces, rules and templates move. Plan for maintenance as a recurring load, not as an incident, and deploy a small scope first, with a rollback plan.

A worked example: from detection to an executable backlog

Take a content team that wants to hand an agent the detection of pages to rework. The hard part is not “detecting”: it is prioritizing without bias and connecting the decision to a real stake. Start from a map of signals, each connected to measurable evidence and to an action.

A signal without evidence remains an intuition, which does not enter a backlog. Prioritization then has to survive reality: simple, replicable and contestable scoring, with no “magic” score. The risk factor protects critical pages from an optimization that is technically correct but commercially absurd.

Finally, the order of execution counts: sequence by dependencies and by low-risk quick wins, so as to validate the measurement loop before widening the reach, batch by batch.

• Batch 1: low-risk technical fixes, with metrics that are easy to follow.
• Batch 2: content optimizations with systematic human validation.
• Batch 3: production of drafts and controlled publication on non-critical scopes.

Continue reading

The method varies with your team and tooling context.

• You want to validate the method without mobilizing a technical team: the full approach, from first test to scaling up, is that of the no-code AI agent.
• Your context calls for auditability or controlled hosting: licences, sovereignty and operational load are compared on the open source AI agent.
• Your teams already work in the editor and want the agent to act there: what it does on an open repository and what it must be forbidden belong to the VS Code AI agent.
• You have to judge an existing agent project or trigger it from your continuous integration: that is the ground of the GitHub AI agent.
• The method is settled and what remains is choosing what to build with: models, vendors and automation tools are compared on an AI agent platform.

And if the point blocking you is the first bullet of your prerequisites, permitted, up-to-date and consistent data sources, an GEO and SEO steering platform handles that task.

FAQ: frequently asked questions about creating an AI agent

Which types of AI agent can be created, depending on the use?

The types of agent you can create are the seven families of agents described in our overview of AI agents, from simple reflex agents to multi-agent systems. A marketing or content team project almost always calls for a task-oriented agent that uses tools, whose autonomy is then set with the four permission levels.

Is it possible to create your own AI?

Yes, you can design your own agent by assembling a language model, an orchestration layer and tools, with your data and your rules. Training a foundation model from scratch is a very different project, expensive and rarely relevant for a marketing team: the realistic route builds around an existing model.

How do you create a simple AI agent?

A simple agent is created on a non-critical task and a very short chain, because reliability degrades as steps pile up. Define one input, one structured output and a single tooled action, have a human validate every output, then measure the time saved and the errors before adding a step.

Which steps should you follow to create an AI agent end to end?

Creating an agent end to end follows seven steps: steering prerequisites, level of agency and stop thresholds, building blocks, specification and decision protocol, error handling and logging, acceptance testing on test sets, then deployment on small scopes. Acceptance testing is the step most often skipped, and it is what separates a prototype from an agent in service.

What does an AI agent cost?

The cost of an agent breaks down into model calls, calls to third-party interfaces, storage, human supervision and maintenance of the connectors and rules. That last item is underestimated, because an agent ages with every change of API or template: start from a narrow scope and widen once the operational cost is known.

Can you create an AI agent locally without exposing your data?

Yes, an agent can run locally, but “local” does not mean “risk-free”. Running locally reduces data exposure if access is secured and the hosting compliant, at the price of heavier maintenance. Minimizing the data sent to the model and logging remain the same.

Which architecture should you choose between persistent memory and “stateless” execution?

“Stateless” execution suits short, highly testable tasks, when you want to reduce data exposure. Persistent memory is needed for personalization, history and operational learning, at the price of stronger security. A frequent compromise keeps a “business” memory (decisions, tickets, rules) and limits conversational memory.

How do you assess an agent's reliability before letting it execute autonomously?

An agent's reliability is assessed on multiple scenarios, with error rates measured per step and a human in the loop at the start. Check that the output format is respected, that the evidence is present and that a forbidden action is blocked, then widen autonomy only to low-risk, repetitive actions, on the strength of the logs.