Validators - MCPJam Inspector

The SDK provides 9 boolean validator functions for asserting tool call behavior in tests, plus evaluateToolCalls — a richer, configurable matcher that returns structured results.

Import

import {
  matchToolCalls,
  matchToolCallsSubset,
  matchAnyToolCall,
  matchToolCallCount,
  matchNoToolCalls,
  matchToolCallWithArgs,
  matchToolCallWithPartialArgs,
  matchToolArgument,
  matchToolArgumentWith,
  // Structured matcher
  evaluateToolCalls,
} from "@mcpjam/sdk";

The matchers module is also available as a standalone browser-safe entry point:

import { evaluateToolCalls } from "@mcpjam/sdk/matchers";

Tool Name Validators

matchToolCalls()

Exact match - all tools in exact order.

matchToolCalls(expected: string[], actual: string[]): boolean

Parameters

Parameter	Type	Description
`expected`	`string[]`	Expected tool names in order
`actual`	`string[]`	Actual tool names (from `result.toolsCalled()`)

Returns

boolean - true if exact match.

Example

const result = await agent.run("Add 2+3, then multiply by 4");
// toolsCalled() = ["add", "multiply"]

matchToolCalls(["add", "multiply"], result.toolsCalled()); // true
matchToolCalls(["multiply", "add"], result.toolsCalled()); // false (wrong order)
matchToolCalls(["add"], result.toolsCalled());             // false (missing multiply)

matchToolCallsSubset()

Checks if all expected tools were called (any order, extras allowed).

matchToolCallsSubset(expected: string[], actual: string[]): boolean

Parameters

Parameter	Type	Description
`expected`	`string[]`	Tools that must be present
`actual`	`string[]`	Actual tool names

Returns

boolean - true if all expected tools are present.

Example

// toolsCalled() = ["add", "multiply"]

matchToolCallsSubset(["add"], result.toolsCalled());             // true
matchToolCallsSubset(["multiply", "add"], result.toolsCalled()); // true (order doesn't matter)
matchToolCallsSubset(["divide"], result.toolsCalled());          // false

matchAnyToolCall()

Checks if at least one expected tool was called.

matchAnyToolCall(expected: string[], actual: string[]): boolean

Parameters

Parameter	Type	Description
`expected`	`string[]`	Any of these tools should be called
`actual`	`string[]`	Actual tool names

Returns

boolean - true if at least one match.

Example

// toolsCalled() = ["add"]

matchAnyToolCall(["add", "subtract"], result.toolsCalled()); // true
matchAnyToolCall(["multiply", "divide"], result.toolsCalled()); // false

matchToolCallCount()

Checks if a tool was called exactly N times.

matchToolCallCount(
  toolName: string,
  actual: string[],
  count: number
): boolean

Parameters

Parameter	Type	Description
`toolName`	`string`	Tool to count
`actual`	`string[]`	Actual tool names
`count`	`number`	Expected count

Returns

boolean - true if count matches.

Example

// toolsCalled() = ["add", "add", "add"]

matchToolCallCount("add", result.toolsCalled(), 3); // true
matchToolCallCount("add", result.toolsCalled(), 2); // false

matchNoToolCalls()

Checks that no tools were called.

matchNoToolCalls(actual: string[]): boolean

Parameters

Parameter	Type	Description
`actual`	`string[]`	Actual tool names

Returns

boolean - true if empty.

Example

const result = await agent.run("What is the capital of France?");
matchNoToolCalls(result.toolsCalled()); // true (if no tools called)

Argument Validators

matchToolCallWithArgs()

Checks if a tool was called with exactly the specified arguments.

matchToolCallWithArgs(
  toolName: string,
  expectedArgs: Record<string, unknown>,
  toolCalls: ToolCall[]
): boolean

Parameters

Parameter	Type	Description
`toolName`	`string`	Tool to check
`expectedArgs`	`Record<string, unknown>`	Expected arguments (exact match)
`toolCalls`	`ToolCall[]`	From `result.getToolCalls()`

Returns

boolean - true if exact argument match.

Example

const result = await agent.run("Add 2 and 3");

matchToolCallWithArgs("add", { a: 2, b: 3 }, result.getToolCalls()); // true
matchToolCallWithArgs("add", { a: 2 }, result.getToolCalls());       // false (missing b)
matchToolCallWithArgs("add", { a: 3, b: 2 }, result.getToolCalls()); // false (wrong values)

matchToolCallWithPartialArgs()

Checks if a tool was called with arguments containing the expected subset.

matchToolCallWithPartialArgs(
  toolName: string,
  expectedArgs: Record<string, unknown>,
  toolCalls: ToolCall[]
): boolean

Parameters

Parameter	Type	Description
`toolName`	`string`	Tool to check
`expectedArgs`	`Record<string, unknown>`	Arguments to match (subset)
`toolCalls`	`ToolCall[]`	From `result.getToolCalls()`

Returns

boolean - true if all expected args present with matching values.

Example

// Tool called with { name: "Task", priority: "high", project: "default" }

matchToolCallWithPartialArgs("createTask", { name: "Task" }, calls);     // true
matchToolCallWithPartialArgs("createTask", { priority: "high" }, calls); // true
matchToolCallWithPartialArgs("createTask", { status: "open" }, calls);   // false

matchToolArgument()

Checks if a specific argument has the expected value.

matchToolArgument(
  toolName: string,
  argName: string,
  expectedValue: unknown,
  toolCalls: ToolCall[]
): boolean

Parameters

Parameter	Type	Description
`toolName`	`string`	Tool to check
`argName`	`string`	Argument name
`expectedValue`	`unknown`	Expected value
`toolCalls`	`ToolCall[]`	From `result.getToolCalls()`

Returns

boolean - true if argument matches.

Example

matchToolArgument("add", "a", 5, result.getToolCalls());  // true if a=5
matchToolArgument("add", "b", 10, result.getToolCalls()); // true if b=10

matchToolArgumentWith()

Checks if an argument passes a custom predicate.

matchToolArgumentWith(
  toolName: string,
  argName: string,
  predicate: (value: unknown) => boolean,
  toolCalls: ToolCall[]
): boolean

Parameters

Parameter	Type	Description
`toolName`	`string`	Tool to check
`argName`	`string`	Argument name
`predicate`	`(value: unknown) => boolean`	Custom validation function
`toolCalls`	`ToolCall[]`	From `result.getToolCalls()`

Returns

boolean - true if predicate returns true.

Example

// Type check
matchToolArgumentWith(
  "echo",
  "message",
  (v) => typeof v === "string",
  result.getToolCalls()
);

// Content check
matchToolArgumentWith(
  "echo",
  "message",
  (v) => String(v).includes("hello"),
  result.getToolCalls()
);

// Range check
matchToolArgumentWith(
  "add",
  "a",
  (v) => typeof v === "number" && v > 0 && v < 100,
  result.getToolCalls()
);

// Email validation
matchToolArgumentWith(
  "sendEmail",
  "to",
  (v) => /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(String(v)),
  result.getToolCalls()
);

Usage with Test Frameworks

Jest / Vitest

import { describe, it, expect } from "vitest";
import { matchToolCalls, matchToolCallWithArgs } from "@mcpjam/sdk";

describe("Calculator", () => {
  it("calls add for addition", async () => {
    const result = await agent.run("Add 5 and 3");
    expect(matchToolCalls(["add"], result.toolsCalled())).toBe(true);
  });

  it("passes correct arguments", async () => {
    const result = await agent.run("Add 10 and 20");
    expect(
      matchToolCallWithArgs("add", { a: 10, b: 20 }, result.getToolCalls())
    ).toBe(true);
  });
});

With EvalTest

import { EvalTest, matchToolCallsSubset } from "@mcpjam/sdk";

const test = new EvalTest({
  name: "multi-step",
  test: async (agent) => {
    const result = await agent.run("Add 2+3, then multiply by 4");
    return matchToolCallsSubset(["add", "multiply"], result.toolsCalled());
  },
});

evaluateToolCalls()

A configurable matcher that returns a structured result instead of a plain boolean. Use this when you need to know why a match failed (missing tools, argument mismatches, wrong order, extra calls) or when you want to control matching behavior per test case.

evaluateToolCalls(
  expected: EvalToolCall[],
  actual: EvalToolCall[],
  options?: EvalMatchOptions & { isNegativeTest?: boolean }
): EvalToolCallMatchResult

Parameters

Parameter	Type	Description
`expected`	`EvalToolCall[]`	Expected tool calls (each has `toolName` and optional `arguments`)
`actual`	`EvalToolCall[]`	Actual tool calls from the trace
`options`	`EvalMatchOptions`	Matching behavior overrides (see below)

EvalMatchOptions

Option	Type	Default	Description
`toolCallOrder`	`"ignore" \| "strict"`	`"ignore"`	`"strict"` requires matched calls to appear in the same relative order as `expected`.
`allowExtraToolCalls`	`boolean`	`true`	When `false`, any actual call beyond what’s expected fails the test.
`argumentMatching`	`"partial" \| "exact" \| "ignore"`	`"partial"`	`"partial"` checks only expected keys and supports type placeholders (`"string"`, `"number"`, `"any"`, etc.). `"exact"` requires deep equality. `"ignore"` skips argument comparison.

EvalToolCallMatchResult

Property	Type	Description
`passed`	`boolean`	Whether the match succeeded.
`missing`	`EvalToolCall[]`	Expected calls that were not found in actual.
`extra`	`EvalToolCall[]`	Actual calls beyond what was expected.
`outOfOrder`	`EvalOutOfOrderToolCall[]`	Matched calls that appeared in the wrong order (only populated when `toolCallOrder: "strict"`).
`argumentMismatches`	`EvalArgumentMismatch[]`	Calls matched by name but with incompatible arguments.

Example

import { evaluateToolCalls } from "@mcpjam/sdk";

const result = await agent.run("Create a task called 'Deploy' with high priority");

const match = evaluateToolCalls(
  [{ toolName: "createTask", arguments: { priority: "high" } }],
  result.getToolCalls(),
  { argumentMatching: "partial" },
);

if (!match.passed) {
  console.log("Missing:", match.missing);
  console.log("Arg mismatches:", match.argumentMismatches);
}

Argument type placeholders

When argumentMatching is "partial" (the default), string values in expected arguments are interpreted as type checks:

Placeholder	Matches
`"any"`	Any non-undefined value
`"string"`	`typeof value === "string"`
`"number"`	`typeof value === "number"`
`"boolean"`	`typeof value === "boolean"`
`"object"`	Non-null, non-array object
`"array"`	Array
`"null"`	`null`

evaluateToolCalls(
  [{ toolName: "sendEmail", arguments: { to: "string", subject: "any" } }],
  result.getToolCalls(),
);

Layered match options

resolveMatchOptions merges suite → case → run-override layers on top of defaults. Import it from @mcpjam/sdk/matchers when building custom eval pipelines:

import { resolveMatchOptions } from "@mcpjam/sdk/matchers";

const resolved = resolveMatchOptions(
  { allowExtraToolCalls: false },   // suite-level
  { argumentMatching: "exact" },    // case-level
  undefined,                        // run-level override
);
// { toolCallOrder: "ignore", allowExtraToolCalls: false, argumentMatching: "exact" }

Quick Reference

Validator	Use Case
`evaluateToolCalls`	Structured match with detailed diagnostics
`matchToolCalls`	Exact sequence of tools
`matchToolCallsSubset`	Required tools (any order)
`matchAnyToolCall`	At least one of these tools
`matchToolCallCount`	Tool called N times
`matchNoToolCalls`	No tools called
`matchToolCallWithArgs`	Exact argument match
`matchToolCallWithPartialArgs`	Subset of arguments
`matchToolArgument`	Single argument value
`matchToolArgumentWith`	Custom validation

Testing with LLMs - Conceptual guide
PromptResult Reference - Get tool calls
EvalTest Reference - Use in evaluations
Saving Eval Results - Per-result matchOptions on EvalResultInput

​Import

​Tool Name Validators

​matchToolCalls()

​Parameters

​Returns

​Example

​matchToolCallsSubset()

​Parameters

​Returns

​Example

​matchAnyToolCall()

​Parameters

​Returns

​Example

​matchToolCallCount()

​Parameters

​Returns

​Example

​matchNoToolCalls()

​Parameters

​Returns

​Example

​Argument Validators

​matchToolCallWithArgs()

​Parameters

​Returns

​Example

​matchToolCallWithPartialArgs()

​Parameters

​Returns

​Example

​matchToolArgument()

​Parameters

​Returns

​Example

​matchToolArgumentWith()

​Parameters

​Returns

​Example

​Usage with Test Frameworks

​Jest / Vitest

​With EvalTest

​evaluateToolCalls()

​Parameters

​EvalMatchOptions

​EvalToolCallMatchResult

​Example

​Argument type placeholders

​Layered match options

​Quick Reference

​Related

Import

Tool Name Validators

matchToolCalls()

Parameters

Returns

Example

matchToolCallsSubset()

Parameters

Returns

Example

matchAnyToolCall()

Parameters

Returns

Example

matchToolCallCount()

Parameters

Returns

Example

matchNoToolCalls()

Parameters

Returns

Example

Argument Validators

matchToolCallWithArgs()

Parameters

Returns

Example

matchToolCallWithPartialArgs()

Parameters

Returns

Example

matchToolArgument()

Parameters

Returns

Example

matchToolArgumentWith()

Parameters

Returns

Example

Usage with Test Frameworks

Jest / Vitest

With EvalTest

evaluateToolCalls()

Parameters

EvalMatchOptions

EvalToolCallMatchResult

Example

Argument type placeholders

Layered match options

Quick Reference

Related