Should AI API specifications live in Git with the model code?

Treat the API as part of the release contract

An API contract is the observable agreement between a service and its callers: paths, authentication scheme, request shape, response shape, error behavior, streaming behavior, and the meaning of fields. OpenAPI is a good canonical form for HTTP APIs because it can describe operations, components, request bodies, responses, security schemes, and schemas in one portable document. At verification time, the OpenAPI Initiative's published current specification is 3.2.0. OpenAPI Specification v3.2.0

For an AI endpoint, the HTTP schema is necessary but incomplete. Two responses can both validate against { answer: string } while one becomes unsafe or unusable because it stops returning citations, selects a different tool, changes a refusal policy, or silently switches to another model. Treat these behavior-shaping inputs as a model contract. It is not a claim that every output will be identical. It describes the behavior reviewers should expect and test in that release.

Keep the API specification, implementation, and model contract in the same pull request when they change together. That makes a reviewer able to answer a basic operational question: "What did we promise clients, what will run in production, and what evidence says they still agree?" A wiki page or an API design workspace can still be useful for discovery and collaborative drafting, but it should not become a second, manually synchronized source of truth.

What to version together

The table does not mean one must encode hidden model reasoning, customer prompts, or raw production traces in Git. Do not request or retain chain-of-thought for this purpose. Record only the configuration, controlled tests, and operational evidence needed to understand the service contract.

A minimal model contract

A concise manifest is usually enough. It can declare an API version, the serving image digest, the selected model and pinned revision, prompt revision, tool-schema revision, retrieval collection or index snapshot, output-schema version, safety-policy revision, fallback sequence, and evaluation report identifier. For a retrieval system, include the document corpus snapshot and embedding-model revision as well, because either can alter answers without changing application code.

Use immutable identifiers when the platform provides them. A tag such as latest or a mutable model alias is useful for exploration, but it is not a reproducible production dependency. Put human-readable names in the manifest too, then resolve them to a revision, digest, or provider request identifier during a release. This is an engineering control, not a promise that third-party hosted inference will be bit-for-bit deterministic.

Model documentation should travel with the release decision. Hugging Face's model-card guidance calls for intended uses and limitations, training information, datasets, and evaluation results, and supports explicit metadata such as the model license and base model. That is useful evidence to link from a model contract when selecting a third-party or internal model. Hugging Face model cards

A repository layout that supports review

Use a layout that keeps the release inputs visible without forcing binary artifacts into the application repository.

The precise directories matter less than the rule that a release can be reconstructed from a commit plus referenced immutable artifacts. A model registry, object store, container registry, or model hub may own the large binary. Git owns the declaration that says which approved artifact is allowed to run.

Prompts and provider-managed prompt systems

Keep prompt templates alongside the service when the template is maintained and deployed with code. Store the source, a stable prompt ID or revision, fixtures, and tests. If a provider hosts prompt versions, use the repository to declare the approved immutable provider revision rather than copying text between systems by hand. A pull request should make it evident whether the application changed a template, switched to a different hosted revision, or only changed a model.

For prompts that contain regulated instructions, proprietary terminology, or security-sensitive policy, repository access should match the sensitivity. That may mean a restricted repository or a controlled prompt service. It does not mean the prompt becomes unversioned. Record its immutable revision and the approver in the release manifest.

Version the public API and the AI behavior separately

Use one release record, but keep two versioning questions distinct. The public API version tells callers whether their integration will continue to work. The model-contract version tells operators and reviewers which behavior-affecting configuration ran. A model or prompt change can require a model-contract bump even when the HTTP API is compatible.

Semantic Versioning offers a familiar convention for public APIs: increment MAJOR for incompatible API changes, MINOR for backward-compatible functionality, and PATCH for backward-compatible fixes. Use it only after defining what "compatible" means for your clients. Semantic Versioning 2.0.0

For probabilistic outputs, avoid promising a particular sentence. Promise verifiable invariants instead: valid structured output, a citation identifier for retrieved claims, a refusal category when policy requires one, maximum tool-call count, or an explicit uncertainty status when evidence is insufficient. Those invariants can be tested even though wording varies.

Block incompatible releases in CI

A spec stored in Git is useful only if the pipeline uses it. Validate the document syntactically, then compare it with the last supported version for breaking changes. Run consumer contract tests against the service, not only schema validation. Contract testing checks that applications sending and receiving messages conform to their shared agreement, and it is particularly valuable when clients and services release separately. Pact documentation

An AI API needs an additional layer of release checks:

1. Schema and client checks. Lint and bundle the API description, generate the supported SDKs with a pinned generator, compile them, and run request and response fixtures through both the service and client.

2. Compatibility checks. Compare the candidate schema with the last released schema. Require an explicit approval and migration plan for a breaking change. Test at least one currently supported client version against the candidate service.

3. Model-contract checks. Resolve every model, retrieval, prompt, tool, and image reference. Reject mutable production aliases unless an approved exception is recorded. Verify that runtime configuration matches the manifest.

4. Behavior and safety evals. Run labeled cases for correctness, required fields, tool selection, citation behavior, refusal behavior, latency, and cost. Keep a protected holdout set that cannot be tuned against casually. Review failures by category instead of averaging them into a single reassuring score.

5. Deployment checks. Build once, deploy the pinned image digest to an isolated environment, run integration smoke tests with least-privilege credentials, and roll out gradually. Capture the deployed commit, manifest digest, and evaluation report with the release.

Generated clients deserve a deliberate policy. Commit the generated SDK source if it is an artifact customers or internal services import and you want reviewers to see its published diff. Otherwise, commit the API spec, generator version, generator configuration, and a CI check that regeneration creates no uncommitted change. Do not silently hand-edit generated code. If a patch is unavoidable, make it a checked-in post-processing step with tests.

Keep secrets and large artifacts out of the repository

Never put provider keys, database passwords, signing keys, real certificates, or customer data in the specification, prompt files, deployment manifests, fixtures, or examples. Git history is durable, and deletion from the latest commit does not reliably remove a leaked credential from clones, forks, caches, or logs. Secret scanning helps find hardcoded credentials across repository history and branches, but it is a detection control, not permission to commit a secret. GitHub secret scanning

In Git, store a secret reference such as a workload identity name or secret-manager path, plus the permission boundary that may read it. Resolve the value only at runtime. Require separate review for a change that expands the service identity's access, and rotate a credential that is exposed even briefly.

Avoid placing multi-gigabyte model weights, training datasets, or operational logs in a normal application repository. Git LFS can technically manage large files by storing pointer files in the repository and content elsewhere, but that does not solve artifact retention, access control, provenance, promotion, or cost. GitHub Docs on Git LFS Use a model or artifact registry instead, and have the release manifest record its immutable digest, license, scanner result, and approval state. Hugging Face's release guidance, for example, recommends documenting model license and can link related quantized variants through metadata. Hugging Face model release checklist

Release ownership makes co-location effective

Co-location works when the same release process has clear owners. It does not mean one engineer personally approves every discipline. Set branch protections so an API contract change requires the people accountable for compatibility, runtime behavior, and security to see the same evidence.

A CODEOWNERS rule or equivalent review policy can enforce this without slowing routine fixes. The threshold should reflect risk. Adding an optional metadata field may need ordinary API review. Changing authorization claims, tenant filtering, a payment action, or an autonomous tool's permissions should require deeper security and product review, expanded evals, and a safer rollout.

When a separate repository is the better design

Separate repositories are justified by organizational boundaries, not by a desire to avoid synchronization work. The tradeoff is that every producer and consumer now needs an explicit dependency and compatibility process.

Do not solve independent ownership with periodic copying. Publish a versioned contract artifact, open a change request against it, and test both directions. Providers must show that the candidate service implements the new contract. Consumers must show that they understand it. This is more work than a shared repository, but it is the correct work when teams cannot release as one unit.

Example of a reviewable AI API change

Example

Hypothetical setup: a support assistant exposes POST /v1/answers. Its current response contains answer, status, and request_id. A product team wants the assistant to return source citations, use a new retrieval index, and allow a web client to display an explicit needs_human_review outcome. The service uses a hosted model, so the application does not own model weights.

The change is one pull request. It adds an optional citations array and needs_human_review status to the OpenAPI response schema, changes the typed response validator, and updates examples. It pins a new retrieval-index snapshot and prompt revision in model-contract.yaml, keeps the hosted model revision unchanged, adds labeled evaluation cases for grounded answers and no-evidence requests, regenerates the TypeScript client, and updates the deployment manifest to use a new image digest. The key name for the retrieval credential is declared, but its value never appears in the diff.

CI rejects the change if the output fails the published schema, the old client cannot parse a candidate response, a required citation is unsupported by the indexed document, the no-evidence cases are answered confidently rather than marked for review, or the running container does not report the manifest's model-contract revision. In staging, the team sends a small percentage of eligible traffic through the candidate, monitors parse errors, citation-validation failures, latency, tool calls, and human-review rate, and can return traffic to the previous image and manifest. The release record brings the changed interface, configuration, test results, and rollback target together for review.

Common failure modes