What format should a tool use when returning data to an LLM?

Choose the representation by the next operation

A tool result has two consumers: the model that needs to reason over it and the application that may need to validate or display it. Start by asking what the next step must do with the result.

JSON is a good canonical representation when later code or tools depend on the result. It gives a stable place for values such as an order ID, a status, a timestamp, and a numeric amount. A concise sentence is often better for a tool whose only useful result is “No current policy applies to this request.” A table is useful for a small comparison, but it is a weak primary contract when a later step must reliably identify a specific row or value.

Do not assume one representation always uses fewer tokens. Repeated JSON keys can cost more than a compact table or sentence. A table can be economical for repeated columns, while a structured object can avoid prose that the model would otherwise have to interpret. Actual cost depends on the data shape, the tokenizer, the model, and whether the result will be repeated in later context. Measure the representative workload if token cost is material.

The Model Context Protocol reflects this split. Its tools can return unstructured content and an optional structured result object, with an output schema for validation. The protocol recommends that a tool returning structured content also provide serialized JSON text for backwards compatibility. MCP Tools specification, version 2025-06-18

Make structured results a real contract

If a result will drive behavior, define an output schema and validate the tool response before it reaches the next programmatic step. JSON Schema provides a standard way to describe and validate the expected structure of JSON data. JSON Schema specification

The schema should express the distinctions the next step needs. For example:

• Use separate fields for an amount and its currency or unit.
• Make nullable fields genuinely nullable rather than using an empty string for several meanings.
• Use stable identifiers, not display labels, when another tool must select a resource.
• Constrain enumerated states such as “open”, “closed”, or “unknown”.
• Declare whether unknown fields are permitted, especially for high-impact workflows.

Schema validation checks shape. It does not prove that an order belongs to the right customer, a balance is fresh, a source is authoritative, or a model should execute an action. Keep those checks in the tool or downstream policy service. A model should not convert an unverified text field into a permission, an API target, or a financial instruction merely because the JSON parses.

The same principle applies to tabular output. Use explicit column names and include a machine-readable ID column when a model may act on a row. Avoid tables with mixed units in one column, decorative footnotes that change a value’s meaning, or rows that collapse missing, zero, and unavailable into the same symbol.

Return failures, limits, and provenance explicitly

An empty list can mean that there were no matches. It can also mean that the query timed out, the caller lacked access, the index is stale, or the result was truncated. A model cannot make a reliable next decision if the tool hides those states behind the same output.

Use an explicit success indicator and a stable error code. Include whether a failure is retryable, a safe short explanation, and a correlation ID for support. For HTTP APIs, RFC 9457 defines a machine-readable problem-details format because status codes alone often do not say enough about an error. RFC 9457, Problem Details for HTTP APIs

For successful but incomplete results, say what was returned and what remains:

• returned count and total count when the total is known
• a boolean such as “truncated” or “partial”
• a cursor or next-page reference when another request can continue
• timeout or freshness limits that affected the result
• the retrieval time and source-system version where material

Provenance should be proportionate to the task. A policy-answer tool might return the policy ID, revision, effective date, and source URL. A database lookup might return the system name, record ID, last-modified timestamp, and query time. These fields let the model cite the source or identify information that needs to be checked again.

Separate returned text from instructions

Tool output can contain user-submitted notes, web pages, email bodies, document excerpts, and other text that was not written by the tool developer. Preserve useful text, but label its origin and keep it separate from trusted control fields.

For example, return an untrusted customer note as a named content field with its source and retrieval time. Do not place it inside a status string, error instruction, tool description, or a field that the application treats as a command. The model may summarize the note, but the application must validate any resulting tool request against its own authorization and business rules.

OWASP identifies retrieved context, tool output, web pages, and email bodies as possible prompt-injection inputs. Its guidance treats a model-based guardrail as one layer rather than a replacement for deterministic controls. OWASP LLM Prompt Injection Prevention Cheat Sheet A format label such as “untrusted” helps the model and reviewer understand provenance, but it cannot by itself stop the model from being influenced. The next tool must still enforce its own schema, access control, and argument checks.

A compact result for a billing lookup

This hypothetical lookup returns one open order. JSON is the canonical result because a later tool may need the order ID and amount. The note is retained as data, not as a command. The short text summary is optional display help, not the value a downstream tool should parse.

{
  "ok": true,
  "as_of": "2026-09-05T10:30:00Z",
  "result": {
    "orders": [
      {
        "order_id": "ord_1842",
        "status": "open",
        "amount": { "value": 19.99, "currency": "USD" },
        "created_at": "2026-09-03T14:12:00Z",
        "customer_note": {
          "trust": "untrusted",
          "text": "Customer-supplied note"
        }
      }
    ]
  },
  "page": {
    "returned_count": 1,
    "total_count": 1,
    "truncated": false,
    "next_cursor": null
  },
  "provenance": {
    "system": "billing",
    "query_time": "2026-09-05T10:30:00Z",
    "record_version": "42"
  },
  "summary": "One open order was found."
}

The amount carries a currency instead of leaving the unit implicit. The result says it is complete. A later action tool can accept only the stable order ID and independently check the caller’s rights and the order’s current state. If this lookup had timed out, it should return an explicit error object rather than an empty orders array.

Keep the first contract small

Start with the fields the next operation genuinely needs. Add a field when a real decision, display, audit, or recovery step requires it. Large, loosely typed payloads force the model to guess which values matter and make schema changes harder to reason about.

Version the output contract when a consumer relies on it. A renamed field, altered unit, new pagination rule, or changed meaning of null can break an agent as surely as an API code change. Test representative successes, empty results, permission denials, timeouts, malformed upstream data, and truncated pages. A small fixture set will reveal more than a single attractive tool demo.