General AI

Agents, tools and evaluation

Reference only. Project progress and credentials come from assessed work.

An assistant mainly helps a person create or understand. An agent can pursue a goal through multiple steps and use permitted tools.

Components of an agentic system

  • Goal and instructions.
  • Current state and context.
  • A loop that chooses the next step.
  • Tools with defined input and output schemas.
  • Permissions controlling each tool and data source.
  • Memory or stored state when the work spans steps or sessions.
  • Stop conditions, approvals and recovery.

MCP, the Model Context Protocol, is one way applications expose tools and resources through a common interface. It does not remove the need to trust, permission and monitor the connected server.

Evaluation closes the map

An evaluation, often called an eval, is a repeatable test of the system against examples and a rubric. Evals can measure answer quality, retrieval, safety, latency, cost or tool behaviour.

Without evaluation, teams have impressions. With a representative test set, they can compare versions and see regressions.

Agents, tools and evaluation | Mariete Academy