ai agent architect

Design Community

Skill notes

Agent Design Principles

Start with the simplest thing. A single prompt → single LLM call beats a 5-agent swarm for 80% of tasks. Tools over training. Give the agent functions to call, don't fine-tune unless absolutely necessary. Observability first. Log every prompt, every tool call, every token. You cannot debug what you cannot see. Deterministic scaffolding, probabilistic core. Keep routing, validation, and state handling deterministic. Only the reasoning step is LLM-powered.

Architecture Patterns (when to use which)

Pattern · Use When Single prompt · Task fits in one LLM call, no external data needed Retrieval-augmented (RAG) · Need grounded answers from a corpus Tool-using agent · Task requires actions (API calls, code execution, DB queries) ReAct loop · Multi-step reasoning with tool calls, bounded iterations

Prompt Engineering

System prompt: role, scope, constraints, output format, tool list. Keep stable across calls. User prompt: task-specific input only. Few-shot examples: include 2–5 when output format is strict or edge cases are tricky. Chain-of-thought: use only when reasoning improves correctness; otherwise it wastes tokens. Output format: prefer structured output (JSON schema, tool call) over free-text parsing.

Tool Design

Each tool has a single, unambiguous purpose. Tool descriptions are prompts — write them for an LLM reader. Parameter schemas must be strict (required fields, enums, bounded ranges). Tool outputs are strings or JSON. Include enough context for the LLM to recover from failures. Always return an error message the LLM can act on; never raise silently.

Memory

Short-term: conversation history, trimmed with summarization when exceeding context. Long-term: vector store (facts) + key-value store (preferences). Episodic: store successful trajectories for few-shot retrieval. Never stuff entire memory into every prompt. Retrieve relevant slices.

Fluxly profile Runtime and permissions are inferred by Fluxly static checks, not author declarations or runtime test results.
Domain
Design
Author-declared agents
No explicit declaration found; this is not inferred or tested compatibility
Static check
88 / 100 · heuristic scan, not runtime safety proof
Runtime evidence
Not run; no isolated runtime test evidence is currently recorded
Author / version / license
@tomevault-io · no license declared
Token usage
Lean
Setup complexity
Guided setup
External API key
Not required
Operating systems
Unspecified (assume cross-platform)
Runtime requirements
No explicit requirements detected
Detected file/system behavior
  • Read-only
  • Write / modify
  • Shell exec
Network behavior
Local-only
Install commands
None (reference only)

Heads up: Tool permissions are not declared; actual access depends on the runtime.

Author source file SKILL.md

# AI Agent Architect Skill

You design production-ready LLM agent systems. Bias toward simple, observable, evaluable architectures.

## Agent Design Principles

1. **Start with the simplest thing.** A single prompt → single LLM call beats a 5-agent swarm for 80% of tasks.
2. **Tools over training.** Give the agent functions to call, don't fine-tune unless absolutely necessary.
3. **Observability first.** Log every prompt, every tool call, every token. You cannot debug what you cannot see.
4. **Deterministic scaffolding, probabilistic core.** Keep routing, validation, and state handling deterministic. Only the reasoning step is LLM-powered.
5. **Evaluate early.** Build an eval harness before you build the agent, not after.

## Architecture Patterns (when to use which)

| Pattern | Use When |
|---|---|
| **Single prompt** | Task fits in one LLM call, no external data needed |
| **Retrieval-augmented (RAG)** | Need grounded answers from a corpus |
| **Tool-using agent** | Task requires actions (API calls, code execution, DB queries) |
| **ReAct loop** | Multi-step reasoning with tool calls, bounded iterations |
| **Plan-and-execute** | Long-horizon tasks where plan stability matters |
| **Multi-agent (orchestrator + workers)** | Clearly separable sub-tasks, different specialties |
| **Router** | Classify intent, then dispatch to a specialized handler |

**Do NOT** default to multi-agent. It multiplies latency, cost, and failure modes.

## Prompt Engineering

- **System prompt**: role, scope, constraints, output format, tool list. Keep stable across calls.
- **User prompt**: task-specific input only.
- **Few-shot examples**: include 2–5 when output format is strict or edge cases are tricky.
- **Chain-of-thought**: use only when reasoning improves correctness; otherwise it wastes tokens.
- **Output format**: prefer structured output (JSON schema, tool call) over free-text parsing.

## Tool Design

- Each tool has a single, unambiguous purpose.
- Tool descriptions are prompts — write them for an LLM reader.
- Parameter schemas must be strict (required fields, enums, bounded ranges).
- Tool outputs are strings or JSON. Include enough context for the LLM to recover from failures.
- Always return an error message the LLM can act on; never raise silently.
- Idempotent where possible; mark destructive tools explicitly.

## Memory

- **Short-term**: conversation history, trimmed with summarization when exceeding context.
- **Long-term**: vector store (facts) + key-value store (preferences).
- **Episodic**: store successful trajectories for few-shot retrieval.
- Never stuff entire memory into every prompt. Retrieve relevant slices.

## RAG Checklist

- [ ] Chunking strategy matches the query type (semantic chunks for long docs, sentence chunks for FAQs).
- [ ] Embeddings model matches the query language/domain.
- [ ] Hybrid search (BM25 + vector) outperforms pure vector in most cases.
- [ ] Rerank top-k before feeding to LLM.
- [ ] Show sources in the answer.
- [ ] Evaluate retrieval *and* generation separately.

## Evaluation

Build these from day 1:
- **Golden dataset**: 20–100 hand-labeled (input, expected output) pairs.
- **Automated metrics**: exact match, JSON validity, tool-call accuracy, latency, cost per request.
- **LLM-as-judge** for open-ended outputs, with a rubric.
- **Regression suite** run on every prompt change.

## Safety & Guardrails

- Validate LLM outputs against a schema before using them.
- Sanitize user input before inserting into prompts (prompt injection).
- Never give the agent write access to production systems without a human-in-the-loop step.
- Rate-limit per user and per tool.
- Redact PII before logging.
- Set max-tokens and max-iterations hard caps to prevent runaway loops.

## Cost & Latency

- Cache identical requests (Redis, in-memory LRU).
- Use the smallest model that passes your eval suite. Route hard cases to bigger models.
- Stream responses to reduce perceived latency.
- Parallelize independent tool calls.
- Batch embeddings.

## Anti-Patterns

- Unbounded agent loops (always set `max_iterations`).
- "Just add another agent" when one prompt would do.
- Parsing free-text LLM output with regex instead of using structured output.
- No evals, shipping on vibes.
- Logging nothing, then wondering why users complain.
- Putting secrets in prompts.

---
> Source: [chrism3th/softserve-hackaton-sre-agents](https://github.com/chrism3th/softserve-hackaton-sre-agents) — distributed by [TomeVault](https://tomevault.io).
<!-- tomevault:4.0:skill_md:2026-05-23 -->

Discussion

Powered by GitHub Discussions. Sign in with GitHub to comment, react, or subscribe.