How should teams share and maintain AI-agent setups internally?

Treat a shared setup as an internal product

An AI-agent setup is the complete, repeatable arrangement that turns an approved input into a useful outcome. It may include instructions, model selection, tools, permissions, retrieval sources, environment assumptions, review gates, evaluation data, and user-facing documentation. A prompt is only one component.

This distinction matters because a setup can fail even if its instructions are unchanged. A model can be retired, a tool schema can change, a knowledge source can become stale, a new repository rule can conflict with an old team rule, or an environment variable can be absent. If none of those dependencies is recorded, users will experience a failure that nobody can reproduce or own.

NIST’s Generative AI Profile calls out the difficulty of attributing system behavior when generative AI systems depend on many third-party components and data sources. Its AI Risk Management Framework also emphasizes documented risks, repeatable test and evaluation practices, and monitoring of pre-trained models in use. NIST AI RMF Generative AI Profile and NIST AI RMF core resources

The goal is not bureaucracy around every sentence someone types. The goal is to apply engineering discipline once a workflow is reused by others, touches organization data, uses tools, informs a consequential decision, or becomes relied upon in a delivery process.

Separate supported workflows from personal snippets

Both kinds of sharing are useful, but they require different expectations.

An informal collection can be valuable for discovery, but call it an “idea shelf” and give it a separate visual label from supported setups. Users should be able to tell in one glance whether a card is personal, team-supported, experimental, deprecated, or production-approved.

Design the configuration as a contract

Store the configuration in version control alongside its documentation and tests. A manifest should be concise enough to review and structured enough to validate automatically. Do not put credentials, raw production exports, or long unversioned instructions inside it.

At a minimum, each supported setup should declare the following:

Use a published schema to validate these fields in continuous integration. A team does not need a perfect schema on day one. Start with identity, owner, purpose, data classification, tool list, and tests. Add fields once people need to make recurring decisions about them.

Software provenance practices offer a useful analogy. The SLSA provenance specification records build inputs and resolved dependencies so consumers can understand how an artifact was produced. An agent manifest need not implement SLSA, but it should similarly make material inputs and dependency versions inspectable. SLSA provenance specification

Use layered rules with explicit precedence

One giant prompt file becomes an unreviewable mixture of organization policy, domain guidance, project details, and personal preferences. It is hard to discover which instruction controls a decision, and changing it risks unrelated workflows.

Use four rule layers instead:

Document the merge order and conflict behavior. Higher-risk rules should be non-overridable. If two lower-level rules conflict, make the setup fail validation or require an explicit choice in the manifest. Silent precedence is a source of unsafe and inconsistent behavior.

The resulting runtime instruction set can be assembled from small referenced files, but the catalog should show a plain-language summary and links to the exact versions. This makes a code reviewer, security reviewer, or new maintainer able to answer “what rules did this agent run under?” without reading an enormous prompt.

Build a catalog people can actually use

The catalog is the front door. Generate it from manifests so that the information users see is tied to the configuration they run. A manually maintained wiki will drift unless it has the same release discipline as the setup itself.

For each catalog entry, show:

• name, one-sentence purpose, owning team, support contact, and current status
• suitable tasks and explicit non-goals
• required access and approved data classification
• whether it reads data, drafts content, recommends an action, or performs an action
• declared models, tools, data sources, and environment requirements
• quality status, last evaluation date, known limitations, and user feedback route
• latest release notes, compatibility promise, deprecation information, and replacement when applicable

Search should work by task, data source, team, tool, status, and risk tier, not only by name. Add examples of good inputs and expected output shape. Avoid publishing a raw “copy this prompt” button for a tool-bearing workflow because users may duplicate instructions without the necessary access controls, test fixtures, or logging.

A concrete starter repository tree

This structure is intentionally small. It supports a catalog, several owned workflows, shared policies, evaluation fixtures, and local development without turning the repository into a full platform.

ai-setups/
  README.md
  CONTRIBUTING.md
  SECURITY.md
  CHANGELOG.md
  catalog/
    index.yaml
    generated-catalog.md
  schemas/
    setup-manifest.schema.json
    tool-contract.schema.json
  policies/
    organization-baseline.md
    data-classification.md
    threat-review.md
  shared/
    instruction-modules/
      output-quality.md
      citation-policy.md
    model-profiles/
      approved-general.yaml
    tool-contracts/
      search-readonly.yaml
  teams/
    support/
      team-rules.md
      glossary.md
  setups/
    support-case-summary/
      manifest.yaml
      README.md
      instructions/
        workflow.md
      evals/
        fixtures.jsonl
        golden-examples.md
        rubric.md
      tests/
        smoke-test.yaml
        safety-cases.yaml
      releases/
        1.0.0.md
      threat-review.md
    engineering-change-review/
      manifest.yaml
      README.md
      instructions/
      evals/
      tests/
      releases/
  scripts/
    validate-manifests
    run-evals
    build-catalog
  docs/
    compatibility-policy.md
    deprecation-policy.md
    incident-and-rollback.md

Keep personal experiments outside the central supported repository by default. A team may maintain a separate, clearly marked, low-risk idea shelf, but it should not inherit production credentials or approval. When an experiment is promoted, its durable parts move into a setup directory with an owner, manifest, tests, and release history.

Promote an experiment through small gates

Promotion should be quick enough that useful work is not trapped in private chats, but strict enough that a copied prompt cannot gain invisible access or organizational authority.

Example

Hypothetical example: a support analyst creates a personal prompt that turns a redacted support ticket into a case summary. It becomes popular because colleagues save time copying the same fields into a report. The analyst should not share the raw prompt as a “support agent” and ask everyone to paste tickets into it.

Instead, the analyst creates a candidate template. The manifest declares that it accepts redacted ticket text, returns a draft summary and unresolved questions, has no action tools, and cannot be used for customer communication without human review. The team adds five golden examples: a straightforward case, an ambiguous case, a case with missing details, a case that must be escalated, and a case containing instructions that should be treated only as ticket content. It also declares the approved data source, retention rule, model profile, and owner.

After review, the template is published as version 1.0.0. A later addition of a tool that posts the draft into a customer system is not a minor prompt edit. It changes the side-effect and permission contract, so it requires a fresh threat review, stronger tests, staged rollout, and a new major version under the team’s stated compatibility policy.

Make security and access boundaries part of the setup

Agents combine natural-language inputs with tools and data, so the configuration must be reviewed as an access path, not merely as text. The OWASP GenAI project identifies prompt injection, sensitive information disclosure, improper output handling, excessive agency, and supply-chain issues among the important risks for LLM applications. OWASP GenAI LLM Top 10 This is why a policy line that says “do not reveal secrets” is not sufficient by itself.

Every setup with data or tools should have a short threat review. It should identify the assets being protected, input sources that are untrusted, model and tool failure modes, permission boundaries, plausible harm, controls, residual risk, and the escalation owner. Focus on the actual capability:

Reference secrets by name or path in a secrets manager, never by value in a manifest, example fixture, prompt, or repository history. Keep development credentials separate from production credentials. Use role-based access to the setup, data connector, tool invocation, and release workflow rather than granting every catalog user the same rights.

NIST’s Secure Software Development Framework recommends version control with accountable changes and code-owner review as examples of protecting software integrity. The same discipline is appropriate for shared agent configurations, tool contracts, and evaluation fixtures. NIST SP 800-218 Secure Software Development Framework

Test behavior before publishing a change

Each setup needs a small evaluation bundle. It is not enough to test that it produces a pleasant example once. At minimum, include:

• golden examples that represent correct expected behavior
• negative and boundary examples that should be refused, escalated, or ask for clarification
• tool fixtures or stubs so tests do not make uncontrolled live changes
• a quality rubric with observable pass criteria
• a regression case for every material past incident
• a safety case for untrusted content in retrieved documents, uploaded files, or tool output
• model and tool compatibility checks when a dependency changes

Golden examples are not necessarily single exact strings. For a summarization workflow, the rubric may require all material facts, uncertainty labels, no invented commitments, and a specified output structure. For a data-retrieval workflow, it may require the correct access-boundary outcome and citations to the current source. For an action workflow, it may require a valid plan and a human confirmation, not completion of the action.

Run the checks in continuous integration for every manifest, instruction, model-profile, tool-contract, or shared-policy change. Run a staging test before changing models, tool versions, data sources, or permissions. NIST’s AI RMF resources call for documented, repeatable test, evaluation, verification, and validation processes with methods, metrics, and reporting. NIST AI RMF core resources

Version, release, deprecate, and migrate deliberately

Give every supported setup a visible version and a compatibility policy. Semantic Versioning can be a useful familiar convention only if the team defines what the setup’s public contract is. For example, the contract may include input schema, output fields, access requirements, permitted actions, citation behavior, and supported model profiles. The Semantic Versioning specification requires a declared public API and recommends documenting a deprecation in a new minor release. Semantic Versioning 2.0.0

One workable policy is:

• Patch version for a correction that does not change documented input, output, permissions, or expected behavior.
• Minor version for a backward-compatible capability, a new optional output field, or an announced deprecation.
• Major version for a changed input or output contract, permission scope, side-effect capability, data classification, model behavior with material quality impact, or rule precedence.

Every release note should say what changed, why, who approved it, what tests passed, what users need to do, and how to roll back. For model upgrades, record the old and new model profile, the evaluation comparison, known differences, and fallback availability. Treat provider or framework model aliases as changeable dependencies unless a provider contract says otherwise.

Deprecation is part of maintenance. Mark the catalog entry as deprecated, name a replacement where one exists, provide a migration guide, set a retirement date, and notify identified users. Remove credentials and tool access at retirement. Do not leave an abandoned “temporary” workflow available merely because it still sometimes works.

Measure usefulness without turning adoption into a vanity metric

Measure usage and quality together. A popular setup might be saving time, or it might be creating rework that users do not report. Use privacy-conscious, aggregated metrics where possible and collect no more content than the support process needs.

Review metrics with the responsible owner on a defined cadence. A small team might review monthly. A production agent with external actions may need much closer operational monitoring. Usage should never be the sole reason to promote or retain a setup.

Assign clear responsibilities

The owner is accountable for purpose, documentation, releases, support triage, and deprecation. Ownership should be a named person or role with a backup, not a team name that eventually means nobody.

For higher-risk setups, add a data owner who approves data dependencies, a tool owner who approves tool contracts and scopes, and a security or privacy reviewer who approves material risk changes. The platform team owns shared schemas, catalog tooling, base policies, and the release path. Contributors can propose templates and tests, but they should not be able to broaden data access or tool permissions by editing an instruction file alone.

NIST’s governance guidance emphasizes documenting risks and potential impacts and communicating them across the organization. This supports a lightweight responsibility model in which people know who can change, approve, suspend, and retire each setup. NIST AI RMF core resources

Common failure modes

• The giant prompt file. Policy, domain knowledge, personal preferences, and tool instructions become inseparable. Split the layers and version their references.

• The catalog without ownership. Users can find setups, but nobody can answer a question, review a model change, or address an incident. Require an owner and backup before publishing.

• The popular snippet with hidden access. A copied workflow begins using restricted data or a privileged tool. Keep snippets explicitly unsupported until promotion adds a manifest, review, and access boundary.

• The silent model swap. Output quality shifts after a model, provider, or alias change. Declare model profiles, run comparisons, publish release notes, and preserve a rollback option.

• The untested action tool. A draft workflow gains the ability to update records or send messages. Treat side effects as a capability change, not as a small convenience feature.

• The metric that rewards volume. Teams celebrate runs or tokens while users correct bad outputs. Pair adoption with quality, rework, and safety signals.