wingetCollections
  • Auto
  • Search
  • Awesome Collections
  • Awesome Programs
  • My Favorite Collections
  • My Collections
  • Account / Settings
  • Terms of Service
  • Privacy
  • About

© 2026 winget.ragerworks.com

Loading...
Install AgentClash using Winget - wingetCollections
  1. Go back
  2. Packages
  3. AgentClash
AgentClash logo

AgentClash AgentClash

ai cli evaluation
Use this command to install AgentClash:
winget install --id=AgentClash.AgentClash -e

AgentClash is an AI agent evaluation platform designed for real-task races. It enables users to compare AI agents under identical conditions, scoring them live based on completion, speed, token efficiency, and tool strategy. The platform provides detailed step-by-step replays to understand performance differences.

Key Features:

  • Define challenges using broken code or build tasks.
  • Utilize various models including OpenAI, Anthropic, Gemini, OpenRouter, Mistral.
  • Run races with composite scoring metrics.
  • Access full replays for transparent performance analysis.
  • Implement failure-to-eval detection for comprehensive evaluation.

Audience & Benefit: Ideal for AI researchers, developers, and teams focused on evaluating or benchmarking agents. AgentClash offers objective comparisons, identifies inefficiencies, tracks improvements over time, and supports CI/CD gates for automated testing.

The platform includes a CLI for managing workspaces, infrastructure, challenge packs, deployments, runs, and authentication directly from the terminal. It can be installed via winget, making it accessible with ease.

README

AgentClash

AI agent evaluation platform for real-task races. Compare agents with the same tools, same constraints, live scorecards, replay, and CI regression gates.

agentclash.dev

What is this?

AgentClash puts AI agents on the same real task, at the same time. Scored live on completion, speed, token efficiency, and tool strategy. Step-by-step replays show exactly why one agent won and another didn't.

  • Head-to-head races
  • Composite scoring
  • Full replays
  • Failure-to-eval flywheel

How it works

  1. Define a challenge (broken code, a build task, etc.)
  2. Drop in your models (OpenAI, Anthropic, Gemini, OpenRouter, Mistral)
  3. Run the race — same tools, same constraints
  4. See scored results with full step-by-step replays

Architecture

AgentClash is a monorepo with three main components:

ComponentTechLocation
API ServerGo / chibackend/cmd/api-server
WorkerGo / Temporal SDKbackend/cmd/worker
CLIGo / Cobracli/
WebNext.js 16 / React 19web/

Infrastructure dependencies:

ServicePurpose
PostgreSQL 17Source of truth for all state
TemporalDurable workflow orchestration for run execution
Redis (optional)WebSocket fanout, rate limiting
E2B (optional)Sandboxed code execution for native agent runs
S3-compatible storage (optional)Artifact storage (filesystem fallback for dev)

CLI

The agentclash CLI lets you manage everything from your terminal — runs, builds, deployments, comparisons, and infrastructure.

Versions
0.4.3
0.4.1
0.4.0
0.3.0
0.2.1
Website
github.com
License
Last updated
4/24/2026
Download latest version

Install

Fastest for JavaScript-ecosystem users — grabs the right prebuilt binary for your platform from npm, no postinstall downloads:

npm i -g agentclash
# or: npx agentclash --help

macOS or Linux with Homebrew, after the tap is populated by a release:

brew install --cask agentclash/tap/agentclash

Linux/macOS fallback script:

curl -fsSL https://raw.githubusercontent.com/agentclash/agentclash/main/scripts/install/install.sh | sh

Windows PowerShell fallback script:

irm https://raw.githubusercontent.com/agentclash/agentclash/main/scripts/install/install.ps1 | iex

Direct downloads are available from GitHub Releases. The installer scripts verify checksums.txt before installing.

More install, uninstall, and release-channel details are in CLI Distribution.

Uninstall script-installed binaries:

rm -f /usr/local/bin/agentclash ~/.local/bin/agentclash

Remove-Item "$env:LOCALAPPDATA\agentclash\bin\agentclash.exe"

Build from source:

cd cli && make build

Use a local CLI build against the hosted backend

If you're only changing the CLI, you do not need to run the API server or worker locally. Point the local binary at a hosted API with AGENTCLASH_API_URL or --api-url.

export AGENTCLASH_API_URL="https://api.agentclash.dev"

cd cli
go run . auth login --device
go run . link
go run . run list
go run . eval start --help
# When the workspace already has challenge packs and deployments:
go run . eval start --follow

Resolution order is --api-url > AGENTCLASH_API_URL > saved user config > default. Source builds (go run ., make build) default to http://localhost:8080; released binaries default to https://api.agentclash.dev.

Quick start

agentclash auth login                           # Authenticate
agentclash link                                 # Pick and save your default workspace
agentclash challenge-pack init support-eval.yaml
agentclash challenge-pack validate support-eval.yaml
agentclash challenge-pack publish support-eval.yaml
agentclash eval start --follow                  # Start an evaluation with guided selection
agentclash baseline set                         # Bookmark a baseline run
agentclash eval scorecard                       # View scorecard + regression verdict

If your workspace is already seeded with challenge packs and deployments, you can skip the authoring commands and start at agentclash eval start --follow.

CI/CD

AgentClash CI/CD gates the candidate agent revision against a repeatable workload. Start by committing an explicit manifest that maps repo changes to the candidate build, deployment settings, challenge pack, baseline, and release gate:

agentclash ci init .agentclash/ci.yaml
agentclash ci validate .agentclash/ci.yaml
export AGENTCLASH_WORKSPACE="your-workspace-id" # or pass -w
agentclash ci validate .agentclash/ci.yaml --remote --json
agentclash ci run --manifest .agentclash/ci.yaml --json --artifact-dir agentclash-artifacts
agentclash ci baseline --manifest .agentclash/ci.yaml --json
agentclash ci should-run --changed-file prompts/system.md --json

The current CLI validates that manifest locally by default, can optionally check referenced resource IDs against the selected workspace with --remote, runs the manifest workflow with ci run, and resolves the exact baseline run that the gate will compare against. In GitHub Actions, ci run attaches repository, pull request, branch, default branch, commit, workflow, and run URL metadata automatically and appends a Markdown gate summary when $GITHUB_STEP_SUMMARY is set. Use --artifact-dir to write uploadable result.json, run.json, scorecard.json, comparison.json, and gate.json files; use the --ci-* flags when another CI provider needs explicit source metadata. For pull request gates, prefer a locked baseline.run_id; update it only after a successful mainline run in a reviewed, auditable change.

Set regressions.promote_failures: proposed when failing CI gates should create reviewable regression candidates in the manifest's evaluation.regression_suites. The CLI deduplicates against existing non-archived/non-rejected cases and reports created, existing, skipped, blocked, and error outcomes in JSON and GitHub summaries. Use disabled to report only, or auto_on_main for protected default-branch workflows that may create active cases after refusing pull request and non-default-branch events.

For GitHub Actions, use the reusable agentclash-ci action to install the CLI, validate the manifest, skip unrelated changes, run the gate, and expose artifact paths:

- id: agentclash
  uses: agentclash/agentclash/.github/actions/agentclash-ci@main
  with:
    token: ${{ secrets.AGENTCLASH_TOKEN }}
    workspace: ${{ secrets.AGENTCLASH_WORKSPACE }}

Existing commands also work non-interactively with environment variables and explicit IDs:

export AGENTCLASH_API_URL="https://api.agentclash.dev"
export AGENTCLASH_TOKEN="your-token"
export AGENTCLASH_WORKSPACE="your-workspace-id"
agentclash run create \
  --challenge-pack-version  \
  --deployments ,
agentclash run list --json
agentclash compare gate --baseline $BASE --candidate $CAND  # exit 1 = regression

See CI/CD Agent Gates for the GitHub Actions sketch and manifest contract, and CI/CD Workload Recipes for realistic agent workload patterns.

Run agentclash --help for the full command reference.

Test the CLI before release

Start with the fast local checks:

cd cli
go build ./...
go vet ./...
go test -short -race -count=1 ./...
go run github.com/goreleaser/goreleaser/v2@latest check
go run github.com/goreleaser/goreleaser/v2@latest release --snapshot --clean
cd ../web && pnpm build
cd ..
bash testing/cli-e2e-suite.sh --help

If you changed packaging or install behavior, rehearse the npm packages locally from the snapshot artifacts:

node scripts/publish-npm/assemble.mjs v0.0.0-rehearse cli/dist
for p in npm-out/platforms/*/ npm-out/cli; do
  (cd "$p" && npm pack --dry-run)
done

For a real local install smoke test, pack the platform package for your host plus the root wrapper, then install both into a scratch directory:

(cd npm-out/platforms/ && npm pack --pack-destination /tmp)
(cd npm-out/cli && npm pack --pack-destination /tmp)
mkdir -p /tmp/agentclash-smoke && cd /tmp/agentclash-smoke
npm init -y
npm i /tmp/agentclash-cli--*.tgz /tmp/agentclash-*.tgz
./node_modules/.bin/agentclash version

Typical triples are darwin-arm64, darwin-x64, linux-arm64, linux-x64, win32-arm64, and win32-x64.

Release the CLI to npm

Routine CLI releases should go through Release Please rather than manual npm publish.

  1. Make a releasable CLI change under cli/ and validate it locally.
  2. Use a conventional commit that matches the desired version bump: fix: for patch, feat: for minor, feat!: for major.
  3. Merge to main.
  4. Release Please opens chore(main): release x.y.z when releasable fix:, feat:, or feat!: commits have touched cli/.
  5. Merge that release PR.
  6. The tag-triggered .github/workflows/release-cli.yml workflow builds GitHub release assets, publishes npm, and runs smoke installs on Ubuntu, macOS, and Windows.

The one-time npm Trusted Publishing bootstrap is already documented in CLI Distribution. Normal day-to-day releases should not need manual npm website work.

Local development

Prerequisites

  • Go 1.25+ — go.dev/dl
  • Docker — for PostgreSQL
  • Temporal CLI — brew install temporal or docs.temporal.io/cli
  • Node.js 20+ and pnpm — for the web frontend (optional)
  • psql — PostgreSQL client for running migrations

1. Start everything (one command)

The quickest way to get the full stack running locally:

./scripts/dev/start-local-stack.sh

This starts PostgreSQL, applies migrations, launches the Temporal dev server, API server, and worker. Logs are written to /tmp/agentclash-local-stack/.

2. Start services individually

If you prefer more control, start each component separately:

Database

# Start PostgreSQL (Docker)
make db-up

# Apply schema migrations
make db-migrate

# (Optional) Seed development data
make db-seed

The default connection string is postgres://agentclash:agentclash@localhost:5432/agentclash?sslmode=disable. Override it with the DATABASE_URL environment variable.

Temporal

Start the Temporal dev server on the default port:

temporal server start-dev --namespace default

The API server and worker connect to localhost:7233 by default. Override with TEMPORAL_HOST_PORT.

API Server

make api-server

The server starts on :8080. Verify with:

curl http://localhost:8080/healthz

Worker

make worker

The worker connects to both PostgreSQL and Temporal to execute run workflows.

3. Web frontend (optional)

cd web
pnpm install
pnpm dev

The dev server starts at http://localhost:3000.

Environment variables

Copy the example and fill in any keys you need:

cp backend/.env.example backend/.env
VariableDefaultDescription
DATABASE_URLpostgres://agentclash:agentclash@localhost:5432/agentclash?sslmode=disablePostgreSQL connection string
API_SERVER_BIND_ADDRESS:8080API server listen address
TEMPORAL_HOST_PORTlocalhost:7233Temporal server address
TEMPORAL_NAMESPACEdefaultTemporal namespace
HOSTED_RUN_CALLBACK_BASE_URLhttp://localhost:8080Base URL for hosted agent callbacks
HOSTED_RUN_CALLBACK_SECRETdev defaultSecret for callback auth
WORKER_IDENTITYhostname-basedWorker instance identifier
SANDBOX_PROVIDERunconfiguredunconfigured or e2b
E2B_API_KEY—Required if SANDBOX_PROVIDER=e2b
E2B_TEMPLATE_ID—Required if SANDBOX_PROVIDER=e2b
ARTIFACT_STORAGE_BACKENDfilesystemfilesystem or s3
ARTIFACT_SIGNING_SECRETauto-generated in devRequired in production (min 32 bytes)
APP_ENVdevelopmentdevelopment or production

Provider API keys (set whichever you need):

VariableProvider
OPENAI_API_KEYOpenAI
ANTHROPIC_API_KEYAnthropic
GEMINI_API_KEYGoogle Gemini
XAI_API_KEYxAI
OPENROUTER_API_KEYOpenRouter
MISTRAL_API_KEYMistral

Smoke tests

After the local stack is running:

# Seed fixture data for a test run
./scripts/dev/seed-local-run-fixture.sh

# Create a run via curl
./scripts/dev/curl-create-run.sh

> Note: Without a real sandbox provider (e.g. E2B), native runs will be queued but won't execute the model-backed path.

Deploying to Railway

AgentClash uses a multi-service Railway project for production services.

Services overview

You need to deploy these Railway services:

Railway ServiceWhat it runsBuild arg
api-serverREST API + WebSocketTARGET=api-server
workerTemporal workerTARGET=worker
PostgreSQLDatabase (Railway plugin)—

External services (not on Railway):

ServiceNotes
Temporal CloudUse cloud.temporal.io for production orchestration. Self-hosting Temporal on Railway is not recommended for production.
VercelDeploy the web/ frontend on Vercel.
E2BSign up at e2b.dev if you need sandboxed execution.
S3Any S3-compatible provider (AWS S3, Cloudflare R2, etc.) for artifact storage.

Step-by-step setup

1. Create the Railway project

# Install the Railway CLI
brew install railwayapp/tap/railway

# Login
railway login

# Create a new project
railway init

2. Create the production environment

In the Railway dashboard, create or select the production environment for your project.

3. Add PostgreSQL

In the Railway dashboard, click + New → Database → PostgreSQL. Railway provisions the database and exposes a DATABASE_URL variable automatically.

4. Deploy the API server

Create a new service in Railway:

  • Source: your GitHub repo
  • Root directory: backend
  • Build: Dockerfile
  • Build args: TARGET=api-server

Set these environment variables:

APP_ENV=production
DATABASE_URL=${{Postgres.DATABASE_URL}}  # Railway variable reference
TEMPORAL_HOST_PORT=:7233
TEMPORAL_NAMESPACE=
ARTIFACT_SIGNING_SECRET=
ARTIFACT_STORAGE_BACKEND=s3
ARTIFACT_STORAGE_BUCKET=
ARTIFACT_STORAGE_S3_REGION=
ARTIFACT_STORAGE_S3_ACCESS_KEY_ID=
ARTIFACT_STORAGE_S3_SECRET_ACCESS_KEY=
HOSTED_RUN_CALLBACK_BASE_URL=https://
HOSTED_RUN_CALLBACK_SECRET=

Set the deploy command to run migrations before starting:

/migrate.sh && /app

Or set it as the Start command in the Railway service settings.

5. Deploy the Worker

Create another service in Railway from the same repo:

  • Root directory: backend
  • Build args: TARGET=worker

Set environment variables:

DATABASE_URL=${{Postgres.DATABASE_URL}}
TEMPORAL_HOST_PORT=:7233
TEMPORAL_NAMESPACE=
HOSTED_RUN_CALLBACK_BASE_URL=https://
HOSTED_RUN_CALLBACK_SECRET=
SANDBOX_PROVIDER=e2b                     # or "unconfigured"
E2B_API_KEY=
E2B_TEMPLATE_ID=
OPENAI_API_KEY=
ANTHROPIC_API_KEY=
GEMINI_API_KEY=

6. Deploy the Web frontend

The Next.js frontend (web/) is best deployed on Vercel:

cd web
vercel --prod

Set NEXT_PUBLIC_API_URL to point to your Railway API server domain.

Production Deployment

ConcernProduction
APP_ENVproduction
TemporalTemporal Cloud production namespace
ArtifactsS3 production bucket
Sandboxe2b
Domainapi.agentclash.dev
Signing secretUnique per environment

Running migrations

Migrations run automatically if you set the start command to /migrate.sh && /app on the api-server service. To run them manually:

railway run --service api-server -- /migrate.sh

Project structure

backend/
  cmd/
    api-server/          — HTTP API entrypoint
    worker/              — Temporal worker entrypoint
  db/
    migrations/          — SQL schema migrations
    queries/             — sqlc query definitions
  internal/
    api/                 — HTTP handlers, managers, auth
    repository/          — Database access (sqlc generated)
    provider/            — LLM provider adapters
    engine/              — Execution loop, tool orchestration
    workflow/            — Temporal workflows and activities
    sandbox/             — Sandbox abstraction (E2B)
    storage/             — Artifact storage (S3/filesystem)
    domain/              — Core domain models
    scoring/             — Scorecard generation
    runevents/           — Event normalization, replay assembly
    worker/              — Worker runtime and config
web/                     — Next.js frontend
scripts/
  db/                    — Database migration scripts
  dev/                   — Local development helpers
  smoke/                 — Smoke test scripts

License

AgentClash is released under FSL-1.1-MIT — the Functional Source License with an MIT Future License clause. See LICENSE for the full text.

The short version:

  • You can use, modify, fork, self-host, and embed AgentClash for essentially any purpose — internal use, commercial product development, consulting, research, education — with one exception:
  • You can't offer AgentClash (or something "substantially similar") as a commercial product or service that competes with agentclash.dev.
  • Every released version auto-converts to MIT on its second anniversary, so anything released 2+ years ago is fully permissive open source.

If you want to do something this license doesn't obviously cover, email us before you build.