wingetCollections
  • Auto
  • Search
  • Awesome Collections
  • Awesome Programs
  • My Favorite Collections
  • My Collections
  • Account / Settings
  • Terms of Service
  • Privacy
  • About

© 2026 winget.ragerworks.com

Loading...
Install mcp-assert using Winget - wingetCollections
  1. Go back
  2. Packages
  3. mcp-assert
mcp-assert logo

mcp-assert Blackwell Systems

cli developer-tools mcp model-context-protocol testing
Use this command to install mcp-assert:
winget install --id=BlackwellSystems.mcp-assert -e

mcp-assert: Ensuring Robust MCP Server Compatibility

Primary Purpose:
mcp-assert is a tool designed to test Model Context Protocol (MCP) servers against real-world protocols. It ensures compatibility with various clients like Claude or Cursor, providing reliable testing without mocks.

Key Features:

  • Cross-Language Support: Works seamlessly with servers written in Go, TypeScript, Python, Rust, Java, and more.
  • YAML-Based Assertions: Define tests easily in YAML and integrate them into Continuous Integration (CI) pipelines for consistent results.
  • Real Transport Testing: Mimics actual client connections using stdio, SSE, or HTTP to ensure accurate testing environments.
  • CI Tool Integration: Compatible with popular CI tools like GitHub Actions, pytest, Jest, and more, enabling seamless setup without additional toolchains.

Audience & Benefits:
Tailored for developers and teams building MCP servers, mcp-assert ensures robustness and compatibility across all MCP clients. By testing against real protocols, it helps identify issues that mocks might miss, leading to fewer errors and improved reliability. This tool is essential for ensuring your MCP server functions correctly and consistently across different platforms.

Installation:
mcp-assert can be installed via winget on Windows, making setup straightforward and efficient.

README

Test your MCP server against the real protocol. No mocks. No imports. No language lock-in.

mcp-assert connects to your server exactly like Claude, Cursor, or any MCP client would: real stdio/SSE/HTTP transport, full initialize handshake, actual tool calls. It checks responses against expectations you define in YAML. If it passes mcp-assert, it works with every MCP client.

> [!WARNING] > We scanned 102 MCP servers and found 4,794 schema issues (2,239 errors) across 55 servers including AWS, Serena, and Grafana. The most common failure: parameters missing type definitions cause agents to send wrong value types. See the scorecard.

Your YAML        ──→  mcp-assert  ──→  MCP Server
(inputs + assertions)    (client)        (any language)
                            │
                        Pass / Fail

Your server can't tell the difference

mcp-assert speaks the full MCP protocol: initialize handshake, tools/list discovery, tools/call with real arguments. It finds bugs that unit tests miss because it tests over the wire, not in-process.

Adopted in production

  • Wyre Technology: 25 MCP servers tested via shared baseline workflow using mcp-assert-action
  • Ant Group (AntV): integrated into CI within 3 days of launch
  • Vera: recommended test harness on project roadmap (#529)
  • Fix PRs merged: Google, Grafana, LangChain, official MCP SDKs
Versions
0.6.0
Website
github.com
License
Last updated
4/28/2026
Download latest version

The testing standard for MCP, like pytest for Python or Jest for JavaScript.

Add it to any MCP server project in one line:

- uses: blackwell-systems/mcp-assert-action@v1
  with:
    suite: evals/

> [!NOTE] > LLMs are for subjective outputs. Assertions are for deterministic ones. Most MCP tools are deterministic. mcp-assert covers them.

Install

# npm (no Go required)
npx @blackwell-systems/mcp-assert

# pip (no Go required)
pip install mcp-assert

# Go
go install github.com/blackwell-systems/mcp-assert/cmd/mcp-assert@latest

# Homebrew
brew install blackwell-systems/tap/mcp-assert

# Docker
docker run blackwellsystems/mcp-assert audit --server "npx my-server"

# Snap (Linux)
sudo snap install mcp-assert --classic

# Scoop (Windows)
scoop bucket add blackwell-systems https://github.com/blackwell-systems/scoop-bucket
scoop install mcp-assert

# Winget (Windows)
winget install BlackwellSystems.mcp-assert

# curl | sh (macOS / Linux)
curl -fsSL https://raw.githubusercontent.com/blackwell-systems/mcp-assert/main/install.sh | sh

Quick Start

Audit any MCP server in seconds. No setup.

Point it at any server:

mcp-assert audit --server "npx my-mcp-server"

  Server: my-server
  Transport: stdio
  Score: 83%

  ✓ read_query      1ms  [E000] responds, returns content
  ✗ create_table    0ms  [E201] internal error: panic: nil pointer...
  ✓ list_tables     1ms  [E000] responds, returns content

  3 tools tested, 2 healthy, 1 crashed

Structured error codes classify issues instantly. See Error Reference for all 24 codes.

> [!TIP] > The audit connects, discovers every tool via tools/list, calls each one with schema-generated inputs, and reports which tools crash vs. handle errors properly. No YAML needed. To go deeper, generate assertion files and customize them:

# Audit + generate starter YAML for CI
mcp-assert audit --server "npx my-mcp-server" --output evals/

# Edit the generated YAMLs: add expected content, setup steps, multi-step flows

# Run in CI with regression detection
mcp-assert ci --suite evals/ --threshold 95

Write assertions from scratch

# Scaffold your first assertion
mcp-assert init evals                   # Or: init evals --server "my-server" for auto-generation

# Run it
mcp-assert run --suite evals/ --fixture evals/fixtures

See the Getting Started guide for a full walkthrough.

Already using Vitest, Jest, Bun, PHPUnit, or pytest?

# Vitest
npm install -D @blackwell-systems/vitest-mcp-assert

// mcp.test.ts
import { describeMcpSuite } from '@blackwell-systems/vitest-mcp-assert'
describeMcpSuite('mcp server', 'evals/')

# pytest
pip install pytest-mcp-assert
pytest --mcp-suite evals/

> [!IMPORTANT] > Same YAML files work across the CLI, Vitest, Jest, Bun, PHPUnit, pytest, and Go test. No migration needed. Write once, run anywhere.

Everything you can do

CommandWhat it doesSetup required
audit --server "..."Scan any server, classify every tool as healthy/crashed/timed outNone
fuzz --server "..."Throw adversarial inputs at every tool, find crashes and hangsNone
init --server "..."Generate a complete test suite from tools/list + capture snapshotsNone
run --suite evals/Run YAML assertions, report pass/failYAML files
ci --suite evals/Run with thresholds, baselines, JUnit XML, GitHub Step SummaryYAML files
coverage --suite evals/ --server "..."Report which tools have assertions and which don'tYAML files
snapshot --suite evals/ --updateCapture responses as golden files for regression detectionYAML files
watch --suite evals/Re-run on YAML changes, show diffs when status flipsYAML files
matrix --languages go:gopls,ts:tsserverSame suite across multiple language serversYAML files
intercept --server "..." --trajectory t.yamlProxy between agent and server, capture live tool call traceTrajectory YAML
lint --server "..."24 static analysis rules for agent usability; --fix auto-generates schema improvementsNone

Start with audit (zero setup), then fuzz (adversarial testing), then init (generates everything), then customize the YAML for your specific assertions.

Zero-Effort Coverage

# Generate stub assertions for every tool the server exposes
mcp-assert generate --server "my-mcp-server" --output evals/ --fixture ./fixtures

# Capture actual outputs as snapshots
mcp-assert snapshot --suite evals/ --server "my-mcp-server" --update

# Assert nothing changed
mcp-assert run --suite evals/ --server "my-mcp-server"

Lint + Auto-Fix

Static analysis catches schema issues without executing tools. 24 rules detect problems that cause agents to fail:

mcp-assert lint --server "npx my-mcp-server"

  E  E103   create_entities       Required parameter "entities" has no description
  W  W114   generate_chart        Input schema is 5 levels deep. LLMs struggle with nesting
  W  W112   (server)              Server exposes 27 tools. LLM accuracy degrades beyond 20

5 error(s), 11 warning(s)

Auto-generate fixes:

mcp-assert lint --server "npx my-mcp-server" --fix

memory-server: 9 tools, 25 findings, 23 auto-fixable

  E103   create_entities   Add description: "The entities value (array)"
  W109   search_nodes      Add examples to "query": [search term]
  W116   read_graph        Append: "Returns the graph data as JSON."

23 fixes generated.

Use --strict in CI to fail on warnings:

mcp-assert lint --server "..." --strict --threshold 0

How It Differs From LLM-as-Judge Frameworks

For deterministic tools, mcp-assert is the better fit. For subjective outputs, LLM-as-judge frameworks remain the right choice. Use both if your server mixes tool types.

DimensionLLM-as-judge eval frameworksmcp-assert
Best forSubjective outputs (prose, creative content)Deterministic outputs (data, state, validation)
GradingLanguage model scoring (flexible, costly)Assertion-based (exact, free)
SpeedSeconds per test (LLM round-trip)Milliseconds per test (no LLM)
CI costAPI calls on every runZero external dependencies
ReliabilityNot measuredpass@k / pass^k per assertion
RegressionNot supportedBaseline comparison, fail on backslide
Multi-languageNot supportedSame assertion across N language servers

Why not just write tests?

You'd need MCP protocol bootstrapping, a server-agnostic runner (your Go tests can't test your TypeScript server), and eval features (regression detection, Docker isolation, JUnit output). mcp-assert handles all of that. One YAML file, any server, any language.

CI Integration

Use the mcp-assert GitHub Action for zero-setup CI:

- uses: blackwell-systems/mcp-assert-action@v1
  with:
    suite: evals/
    threshold: 95

Downloads the binary, runs assertions, uploads JUnit XML results, writes GitHub Step Summary. No Go toolchain required on your runners.

Or run directly:

mcp-assert ci --suite evals/ --threshold 95 --junit results.xml

See the CI Integration guide for JUnit XML, markdown summaries, badges, and regression detection.

pytest Integration

Run mcp-assert assertions as pytest test items:

pip install pytest-mcp-assert
pytest --mcp-suite evals/

Each YAML file becomes a pytest Item with pass/fail/skip semantics. Configure via pyproject.toml:

[tool.pytest.ini_options]
mcp_suite = "evals/"
mcp_fixture = "fixtures/"

Then just run pytest. See pytest-plugin/README.md for all options.

Vitest Integration

Run mcp-assert assertions as Vitest tests:

npm install -D @blackwell-systems/vitest-mcp-assert

Auto-discover all YAML files in a directory:

// mcp.test.ts
import { describeMcpSuite } from '@blackwell-systems/vitest-mcp-assert'
describeMcpSuite('mcp server', 'evals/')

Or run individual assertions:

import { test } from 'vitest'
import { runMcpAssert } from '@blackwell-systems/vitest-mcp-assert'
test('echo tool', () => runMcpAssert('evals/echo.yaml'))

Same YAML files work across Vitest, pytest, and the CLI. See vitest-plugin/README.md for all options.

Documentation

Full documentation is available at blackwell-systems.github.io/mcp-assert:

  • Getting Started: install, scaffold, first run
  • Writing Assertions: YAML format, all 18 assertion types + 4 trajectory types, 8 block types, 6 test framework plugins (pytest, Vitest, Jest, Bun, PHPUnit, Go test), setup steps, capture, fixtures
  • CLI Reference: full command reference with flags and examples
  • Examples: 65 example suites across 8 languages (606 assertions)
  • CI Integration: GitHub Action, JUnit XML, regression detection
  • Badge: add the "Works with mcp-assert" badge to your server README
  • Architecture: internals and design decisions
  • Roadmap: what's shipped and what's next
  • Scorecard: 32 bugs found across 13 servers, 9 fix PRs submitted, 58 servers scanned

  <img src="assets/star-cta-light.png" width="600" />
</picture>

License

MIT