How to turn a prototype into a Python AI agent your team can ship

You have a Python file that calls a model, works one time in three, and that nobody but you can read. This page is for the technical, product or IT lead who has to turn that prototype into software a team can review, test, ship and audit. It helps you decide what shape to give the code of a Python AI agent: a tidy project, an explicit state, deterministic tools and tests that stop a regression.

Framing, level of autonomy and specification are decided upstream; the code that follows is their implementation.

From script to agent: what changes in your code

An AI agent is not an improved “chat”. It is a system designed to decide and act in a given environment, often through external tools, database, web search or programming interface, where a conversational agent stays confined to text. The autonomy you grant it is set according to the four permission levels of an AI agent, from read-only to supervised autonomy: that grid is not redefined here, the code puts it into practice.

Your practical criterion is simple: if your code does anything other than return an answer, create a report, run a query, trigger an action, and it evaluates itself, you are already in agent territory. A script runs a sequence of instructions; an agent maintains a state, chooses its actions and checks its results before deciding what comes next.

Before writing a single line, set down a shared vocabulary: in production, confusion between model, tool, agent and orchestration is paid for in bugs, in drift and in invoices.

• The language model: it generates, classifies, rephrases. It is the only building block you accept as probabilistic.
• The tool: it carries out a deterministic action, verifiable and replayable identically.
• The agent: it plans, chooses the tools, checks the result obtained and decides what comes next.
• Orchestration: it paces, triggers and supervises the runs.

This split carries the whole argument: the model takes the probabilistic part, planning and rephrasing, and everything else, data, calculations, exports, stays verifiable. A rule that always has to give the same result is written in Python, not in a prompt.

Three shapes come back almost every time, with the same building blocks.

• The research and synthesis agent: it collects and delivers a structured summary with its sources.
• The analysis agent: it turns raw exports into prioritized decisions without guessing; it computes, then explains, and keeps the traceability.
• The controlled production agent: it chains brief, writing, quality checks and delivery; its core is the pipeline of checks, not the model.

These three shapes belong to the same category, the task-oriented agent using tools, among the seven families of agents described in the pillar; they differ only in the tools plugged in and the evidence expected.

Structuring the project before writing the loop

An agent you cannot test is not shipped: it is demonstrated, which is not the same thing. Testability is decided from the first folders, with one rule: nothing that changes from one environment to another, threshold, identifier, path, quota, lives in the code that decides.

Four folders, and what each one holds

The minimum structure has four directories, and this is not a matter of style: each one corresponds to a distinct reason to modify the project. A file that could fall into two of them signals a badly split responsibility.

• Configuration: environments, thresholds, business rules, mappings. What changes without shipping code again.
• Connectors: the clients for external interfaces, with each provider’s quotas, timeouts and errors.
• Agent: the planner, the executor, the validators and the action policy. It is the only folder where decisions are taken.
• Tests: fixed cases, non-regressions and simulated errors from external providers.

Separate the business code from the connectors: that boundary lets you test the logic with no network, a frozen dataset standing in for the connector. An agent whose loop does not run offline has no tests: it has real calls that somebody reruns by hand.

What does not live in the repository: secrets, configuration, test sets

A hard-coded identifier looks like an ordinary variable, survives reviews and ends up in the version control history, where it is never really erased. Three rules prevent that kind of leak.

• Secrets: in a dedicated manager, never in the repository, separated by environment.
• Configuration: loaded at start-up, with an immediate failure when a mandatory value is missing.
• Test sets: in the repository, and therefore synthetic; a real export embedded “for testing” brings customer data into a project cloned onto several machines.

If your editor itself hosts an agent that reads that repository, the same rules apply to it: what a VS Code AI agent is allowed to touch is decided at the same moment, and is not to be confused with the agent you are writing.

Writing the agentic loop

A solid agent looks less like a demo than like a controlled loop: explicit state, deterministic actions, evidence, then a decision. The classic trap is the model, tool, model chain that never stops, because no step can say whether it has made progress. You get out of it through a structured state and explicit transitions, not through one more instruction in the prompt.

The four stages: plan, action, observation, decision

Each iteration goes through the four stages, plan, action, observation, decision, and each one produces an inspectable object: you know where the agent stands without rereading its outputs. In Python, that means a serializable state, a unique operation identifier per action against duplicates on replay, assertions on observed values rather than the model’s opinion, and a versioned exchange contract that rejects any off-schema output. The logic of the loop is detailed in the AI agent workflow.

The limits that stop the loop from spinning

A Python agent becomes dangerous when it acts without limits. Frame its autonomy like a budget, with ceilings written in the configuration and stop rules that apply without arbitration.

• Human supervision: mandatory validation before any write, publishing, deletion, sending or updating a third-party system, which is the pillar’s controlled-write level.
• Stop criteria: no evidence, no action; insufficient confidence, escalate; three iterations with no measured progress, stop.
• Action budget: cap network calls, planning loops and the size of the context passed on.

These three limits are checked in code, not in an intention. A stop criterion based on the confidence the model grants itself is not one: write it as a condition on an observable value, evidence collected, gap between two sources, missing field, and the stop becomes reproducible.

Tools: making deterministic whatever can be

Your tools have to be more reliable than your model: everything touching the truth, data, calculations, extraction, formatting, goes through deterministic functions, the only way to get the same result twice on the same input. A tool is designed as a narrow function, and four properties define it.

• One responsibility: a tool does one thing, and its name says which.
• Typed parameters: the domain of each input is declared, not guessed.
• An output validated by a schema: what comes out conforms before it is passed on.
• An explicit error: when something is missing, the tool fails, it does not fill in.

The Python ecosystem covers the essentials without asking the model for anything: the standard library or your engine’s drivers for data access, a tabular computing library (pandas, for example) for analysis, a charting library and a template engine (Jinja, for example) for output. That last point is not cosmetic: a report whose formatting is generated at every run cannot be compared from one version to the next, and therefore cannot be tested.

Before authorizing an action, three rules are set in the tool itself, not in the prompt that calls it.

When the agent has to answer or decide from internal documents, the tool is no longer a simple access path: chunking the corpus, the retrieval strategy and the citing of sources then belong to the RAG AI agent, and that building block is designed in its own right.

Logging, tracing, versioning

In a company, an agent with no observability is unusable, whatever the quality of its outputs. You have to be able to answer three questions, months after the run: what did it do, with which data, and on which rule did it decide? Those answers cost a few lines around each tool call.

• Structured logs: action, input, output, duration, error, estimated cost, in a machine-readable format.
• Traces: the call chain and a unique correlation identifier per run, propagated all the way into the connectors.
• Versioned prompts: fingerprint, date, author, change log. A prompt is code, it is reviewed and compared.
• Decision log: stops, escalations, refused actions, with the rule that triggered them.

The fourth line is the one that gets forgotten, and the only one an auditor cares about: actions show what happened, decisions show why. The sufficiency criterion: replay a complete run from the logs alone. What they become once the agent is connected to the company’s systems belongs to AI agent integration.

Framework or in-house implementation

An assembly library has to speed things up without hiding the decisions. What costs you most is neither writing time nor the licence: it is unreadability. A “magic” architecture, which makes you lose the plan, action, evidence, decision path, becomes impossible to debug, to test and to secure.

The criteria that decide, and their warning signals

Five criteria settle it, and each is checked on a real case rather than on a documentation example. The table gives, for each, the signal that should make you step back and its consequence in the code.

When an assembly library helps, and when it costs

The decision rests on the shape of the problem, never on a tool’s popularity. Three situations cover almost every case.

• Document retrieval, tool routing, modular pipelines: an assembly library is justified, it saves you rewriting known plumbing.
• A problem split into roles: research, analysis, writing, quality control. A role-oriented structure avoids the monolithic agent that does everything and becomes impossible to make reliable.
• Three tools and two rules: a simple implementation will be more robust and easier to prove than one more dependency.

In the roles case, the right pattern separates an analysis agent, which handles data and calculations, from an editing agent, with a validator that blocks any claim without evidence. The third case is the most frequent and the most often wrongly set aside: writing it in house wins on readability, control and proof, and the cost of a dependency shows up at the first version bump that breaks an abstraction you depended on without knowing it.

Acceptance testing before shipping

Test it like a product, not like a notebook. The question is not “does it work”, a demo only shows the nominal path, but “what happens when it does not”. The team receiving the agent is not judging your loop, but they know how to read an acceptance criterion and refuse a delivery that does not meet it.

Case sets, acceptance criteria and non-regression

The three parts of acceptance testing, case sets, acceptance criteria and non-regression, are those of the method to create an AI agent; here, they translate into test families run at every change of prompt, rule or connector. The most discriminating part remains non-regression: an agent whose decision varies on an identical input is not ready. The table gives, for each family, what blocks the delivery.

Packaging, environments and reproducibility

Reproducibility is an obligation, not a convenience: without it, no test proves anything, since nothing guarantees the run environment will be the same tomorrow. Three moves guarantee it.

• Isolate execution: a virtual environment dedicated to the project, and nothing installed outside it.
• Freeze the dependencies: a lock file (pip, Poetry or uv), indirect dependencies included.
• Move the configuration out: the same artefact runs in development, in staging and in production without a line of code changing.

Add a minimal install check: an entry point that imports the key libraries, checks read-only access to the connectors and fails explicitly if a mandatory variable is missing. Ten lines turn a half-day diagnosis into a readable message.

Latency, context and cost per run

Your cost is not only financial: it is also latency, which decides what your users will agree to wait for. Three levers keep it in check.

• Reduce the context passed on: cap its size and cache the retrievals.
• Limit iterations: set budgets for number of calls and duration, defined in the configuration.
• Prefer a deterministic tool to one more question put to the model: a check written in code gives a stable answer, where rephrasing costs one more call and guarantees nothing.

The third lever is the most profitable and the least applied. Measure the cost per run and cut the loops beyond a logged threshold: that is the only way to know whether an agent costs more because it handles more, or because it is going round in circles.

Continue reading

Depending on where you stand, three articles take this one further.

• Your tests pass and now have to be replayed at every change, or you have to judge an agent project somebody is proposing: that is the ground of the GitHub AI agent.
• You have picked a library and have to check what its licence allows, especially if the agent goes out inside a product: licences, sovereignty and operating load are compared on the open source AI agent.
• You do not want to write everything and are looking for what already exists: models, vendors and automation tools are compared on an AI agent platform.

And if the point is not to write the code yourself but to get an agent built on your data, your rules and your evidence, that is the scope of personalized AI for your business.

FAQ on Python AI agents

How do you create an AI agent with Python?

You create a Python AI agent by first writing a controlled loop: serializable state, a short plan, execution through deterministic tools, then verification and decision. The project is laid out in four folders, configuration, connectors, agent and tests, before the first function. The model takes only the probabilistic part; data, calculations and exports stay verifiable.

Which frameworks should be used?

The framework is chosen according to the shape of the problem, not a tool’s reputation. An assembly library is justified for document retrieval, tool routing or modular pipelines; a role-oriented structure, when the work splits into research, analysis, writing and control. For three tools and two rules, an in-house implementation is more robust and easier to audit.

Which are the best tools?

The best tools are the ones that make the agent deterministic where it has to be: the standard library or your engine’s drivers for data, a tabular computing library for analysis, a template engine for output. The criterion is not feature richness, but the ability to replay the same call and obtain the same result.

What is the difference between a Python agent and a simple script?

A script runs a sequence of instructions, whereas an agent maintains a state, chooses its actions according to an objective, relies on external tools and checks its results before deciding what comes next. The tipping criterion: if your code produces a deliverable or triggers an action, and it evaluates itself, it is already an agent.

How do you avoid hallucinations and enforce verifiable outputs?

Hallucinations are avoided through a single rule: no claim without evidence. Tools extract and compute instead of the model, the output format imposes citations, dates and extracts, and every output is validated against a schema before the next step. With no source available, the agent asks for the missing data instead of filling in.

When should persistent memory or RAG be added?

Persistent memory becomes necessary when the agent follows a case across several runs: history, decisions taken, exceptions granted. Document retrieval is called for when quality depends on precise, located information, internal procedures or product documentation, and sources have to be cited. In both cases, the compliance load goes up.

Which guardrails should be in place before authorizing actions (write, publish, delete)?

Least privilege applies first: read-only by default and write functions declared as such. Human validation precedes destructive actions, with strict stop criteria on action budget, time, cost and evidence. Every operation is idempotent, and the log keeps who asked for what, what was done and on which rule.

Which tests should be automated to make an agent reliable before production?

Five test families are automated before production: unit tests on the tools, integration tests on the exchange contracts, reference outputs with a controlled tolerance, security tests on permissions and secrets, and business scenarios validating concrete acceptance criteria. Non-regression, same input and same decision, completes the set; a correct refusal counts as a success.