What is the difference between an AI model and an AI agent?

How models, assistants, and workflows fit together

These terms describe different parts of a system. A single product can include all of them.

A workflow can contain an agent, and an agent can call a workflow as a tool. Product labels alone do not establish those capabilities: check the available information, tools, stopping conditions, and permissions. Microsoft lists its model catalog and agent services separately.

What makes a system agentic

The following capabilities explain why an agent is more than a model call.

Instructions and objective

Instructions tell the model the role, scope, constraints, output format, and when to ask for help. An objective gives the run a result to work toward, such as "prepare a draft answer grounded in approved policy documents" or "find the order status and explain the next customer action." Good instructions narrow behaviour. They are not a security boundary, because a model can misunderstand them or receive adversarial content from an external source.

Tools

Tools let the agent obtain information or request an operation outside the model. Common examples are a search index, file retrieval, a calculator, code execution, a calendar API, or a function that retrieves an order status. The model normally selects from tool descriptions, but software must validate the proposed call, authenticate it, enforce its parameters, and handle the result. The tool, not the model, performs the real-world operation.

Microsoft Foundry lists web search, file search, code interpreter, memory, MCP servers, and custom functions as tool options, with toolboxes providing shared authentication, governance, and versioning. Its documentation also describes how a model may choose a tool, receive its output, and decide whether another tool call is needed. See Foundry's agent and toolbox overview. This is the practical source of an agent's ability to do more than write an answer.

Context and memory

Context is the information available for the current model call. It can include the user's request, retrieved documents, tool results, earlier messages, and system instructions. A conversation history is one way to carry context across turns, but it is not the same as a model permanently remembering a person or a fact.

Memory is deliberately stored information that can be retrieved later. It might be a conversation summary, a user preference, a project fact, or a task state. It needs data governance: decide what may be retained, where it is stored, who can read it, when it expires, how it can be corrected or deleted, and how it is separated between users. In Foundry, persistent memory is an added capability, not an automatic property of every agent. The documented memory quickstart says that without a memory store each conversation starts from scratch, and describes a separate store backed by retrieval. See the persistent-memory quickstart.

Planning and control loops

An agent often follows a control loop: assess the current task, select the next allowed step, run a model or tool, inspect the result, and either continue, stop, or request a person. Planning can be as simple as choosing whether to retrieve a policy document before drafting a response. It can also be explicit task decomposition in application code. It does not require exposing private model reasoning, and it should not be confused with a guarantee that the plan is sound.

The loop needs limits. Useful controls include maximum tool calls, timeouts, token or cost budgets, an allowlist of tools, structured outputs, retry rules, and an explicit stop condition. Without limits, an agent can cycle through failed attempts, repeat a tool call, accumulate irrelevant context, or make its behaviour hard to audit. Microsoft Foundry's runtime model reflects this operational distinction: an agent supplies the reusable configuration, a response is one execution, and a conversation supplies stored history for later responses. See the component lifecycle.

Permissions and human control

An agent may be able to propose an action, but it should not receive a blanket right to perform it. Give each tool the least privilege it needs. Prefer read-only access by default, narrow resource scopes, validate input server-side, set transaction or deletion limits, and require an explicit human approval before a consequential action such as sending an external message, changing production data, spending money, or disclosing sensitive records.

Identity belongs to the surrounding platform and application. In Foundry, agent identity, Microsoft Entra authentication, role-based access control, private networking, content filters, and tool connection configuration are service features. They are examples of the controls that turn a model's suggested tool call into an auditable system operation. Microsoft documents these identity and security capabilities in the Agent Service overview. They reduce risk but do not remove the need for application-level authorisation and review.

Example of an expense-query assistant

Imagine an employee asks, "Why is my expense claim pending?" In a simple version, the application supplies the claim status and policy text to a model. The model explains that a receipt is missing. The application gathered the information; the model wrote the explanation.

In an agent version, the model can select a permitted get_my_claim_status tool, inspect the result, and request a policy lookup if needed. The application checks the employee's identity before returning any record. Once the agent has enough information, it explains the missing-receipt requirement and may prepare a message asking for the receipt.

Preparing that message and sending it are separate capabilities. If sending requires approval, application code must enforce it. A model-generated instruction to send is not approval. This example shows why an agent can do more work across several steps while still depending on the surrounding application for access and execution.

Choosing the simplest reliable design

Start with the smallest architecture that can meet the outcome and risk requirements. More agentic capability introduces more possible paths, more data movement, more cost, and more security work. That extra complexity is justified when it creates a material benefit, not merely because an agent is fashionable.

For example, a payroll system should not ask an agent to "handle compensation." Its workflow should calculate pay using approved rules and systems of record. A model can explain a pay statement in plain language, and a tightly constrained agent can retrieve a worker's own records and draft a support case. Neither should decide pay, access another worker's records, or execute a payment without the existing controls.

How this maps to Microsoft Foundry and Azure

Microsoft product names and service paths evolve, but the architecture remains useful. In the current Foundry design, the model catalog provides the models. An application can call a model directly for one inference. A prompt agent is a managed, reusable definition in which you configure instructions, a model, and tools. A hosted agent is application code and an agent framework packaged for Foundry to run with a managed endpoint, scaling, identity, session state, and observability. Microsoft's Agent Service documentation compares prompt and hosted agents.

Foundry also separates three runtime components that learners often collapse into the word "agent":

• An agent supplies reusable model, instruction, and tool configuration.
• A conversation preserves input and output items when a later turn needs history.
• A response is an execution that runs a model or agent against input and yields output items, including tool calls and tool results when configured.

This separation means a direct response can use a model without creating a persistent agent. An application can add a conversation only when it needs server-side history. It can add an agent when it wants reusable behaviour and tool configuration. Microsoft explains these components and their lifecycle here.

The service can manage a great deal, but it does not make governance optional. Foundry says its agent tools can include web search, file search, code interpreter, memory, MCP servers, and custom functions. It also provides identity, role-based access control, content safety, tracing, evaluations, and managed toolboxes. A team still needs to define which tool scopes are appropriate, choose trustworthy data sources, test prompt-injection resistance, validate outputs, monitor costs and failures, and review actions with real-world impact. See the Foundry overview of tools and security.