Anyray install
Self-host Anyray, the AI-spend optimizer gateway, from
published images. This repo is the install path for every target: Docker, Kubernetes,
AWS, Google Cloud, and Railway.
> Full docs and platform recipes: https://docs.anyray.ai
Quick start with a coding agent
Create a deployment at app.anyray.ai. Under Install
with your coding agent, copy the generated prompt and paste it into Claude
Code, Codex, or another coding agent with access to your infrastructure.
Install Anyray for my organization. Open , follow the agent instructions exactly, and continue until that URL reports status `ready`. Do not ask me for a deployment token or provider API keys, and never print, echo, or log credentials.
The portal fills in a one-hour, single-use aic_ URL. It contains no durable
deployment credential. The URL gives the agent the current Docker or Kubernetes
runbook and reports:
pending -> claimed -> preflight -> configured -> gateway_connected -> ready
ready means the local health check passed and Billing received a heartbeat
from the newly issued deployment credential.
| What | Where |
|---|
| Console | http://:3000 — sign in with the admin key setup.sh printed, then run the ~3-min in-console setup |
| Gateway | http://:8787/v1/… — point AI tools and SDKs here |
> Keep ports 3000 and 8787 reachable from your org network only. Never expose
> them publicly.
Manual Docker install
In the portal, select Use manual installation. This disables the live agent
link before the one-time deployment token appears. Then run:
git clone https://github.com/anyrayHQ/install anyray && cd anyray
./setup.sh --connect adt_XXXX
docker compose up -d
Within a minute the deployment shows as connected at app.anyray.ai.
Add a provider key (required)
The gateway can't serve requests until it has an LLM provider key. Add it in the
console (provider-keys settings), or in .env then re-run docker compose up -d:
ANYRAY_PROVIDER_KEY_OPENAI=… # or _ANTHROPIC / _AZURE_OPENAI
Developers never see this key — their tools use a placeholder.
Connect developers — zero install
Hand your team the setup link (https://app.anyray.ai/setup/dvl_…) from the
portal. They run one line — no Node, no npm, nothing to install. The installer
fetches a checksum-verified anyray-connect binary for their OS and points their
AI tools (Claude Code, Codex, …) at the gateway.
# macOS / Linux
curl -fsSL https://app.anyray.ai/connect.sh | sh -s --
# Windows (PowerShell)
& ([scriptblock]::Create((irm https://app.anyray.ai/connect.ps1))) ""
Flags pass straight through (e.g. --subscription; subscription-org links set it
automatically). Prefer a package manager? npx anyray-connect still
works (needs Node).
Deployment options
Docker Compose (above) is the default. Each target has its own recipe:
| Target | Command / artifact | Docs |
|---|
| Kubernetes (Helm / ArgoCD) | GitOps-native OCI chart: oci://public.ecr.aws/anyray/anyray (or ./setup.sh --k8s --connect adt_… + helm install from source) | helm/README.md |
| AWS (CloudFormation) | One-click ECS/Fargate + RDS + EFS stack; secrets via Secrets Manager | aws/ |
| Google Cloud (GKE Autopilot) | One-click Cloud Shell deploy of the bundled Helm chart | gcp/README.md |
| Google Cloud (Compute Engine VM) | One-click Cloud Shell deploy; docker compose on one VM (no Kubernetes, no managed DB) | gcp/gce/README.md |
| Railway | One-click; Railway generates every secret | railway/README.md |
| LiteLLM attach | Runs only the optimizer + console as a LiteLLM hook (no Anyray gateway) | attach/litellm/README.md |
For Helm, use an existing namespace unless your cluster policy says otherwise.
Other setups
Remote Docker host (EC2 / GCP / any VM with SSH + Docker):
./setup.sh --host --connect adt_XXXX
DOCKER_HOST=ssh://user@ docker compose up -d
Front an existing gateway (LiteLLM / OpenRouter / any OpenAI-compatible) — run
Anyray on top with --upstream. Tools point at the Anyray gateway and send normal
requests; no per-request x-anyray-config header needed.
./setup.sh --connect adt_XXXX --upstream http://host.docker.internal:4000
docker compose up -d
> Inside Docker, localhost is the gateway container — use host.docker.internal
> (or the host LAN IP) for an upstream on the host. Re-run ./setup.sh --upstream
> to change it later. --upstream also clears the proxy allowlist for that host.
Public exposure (remote devs / Cursor / Copilot BYOK) — terminate TLS with the
bundled Caddy. In .env set ANYRAY_PUBLIC_DOMAIN (DNS A record → host),
ANYRAY_GATEWAY_BIND=127.0.0.1, and ANYRAY_TRUST_PROXY=true, then:
docker compose --profile public up -d
Caddy serves HTTPS on 443 with automatic Let's Encrypt certs and 403s the admin
surface, so the console and /admin stay in-network only. This HTTPS URL is what
editor BYOK endpoints (Copilot Chat, Cursor) point at —
anyray-connect --tools copilot prints the exact VS Code steps.
The gateway is always restricted to enrolled developers — secure-by-default,
no toggle. Passwordless, offline hardening: only devices whose anyray-connect
run enrolled a per-device key against the gateway's pinned trust anchor get through;
everyone else gets a 403. Enroll developers (anyray-connect --enroll / --sso)
before sending traffic.
Upgrade
git fetch && git reset --keep origin/main && docker compose pull && docker compose up -d
Use the full command for releases reported as a hard/template update; do not
pull and restart only the application images. Use policy-stable for
compatible soft updates (or keep an immutable vX.Y.Z pin). Existing explicit
overrides are preserved until the operator changes them during a template
refresh.
The AWS Quick Launch template pins ImageTag to an immutable release. When
upgrading an older stack that stored ImageTag=latest, replace that parameter
with the candidate template's pinned default instead of preserving latest.
New optimizer default profiles ship in the image but seed config only on first run,
so an existing deployment keeps its saved config. To adopt new defaults, update the
optimizer config from the console — your .env and secrets are untouched.
Repair / status
docker compose ps # service health
docker compose logs # diagnose a failing service
./setup.sh # safe to re-run after the initial claim or --connect
# (--connect rewrites only the connect vars)
Security
setup.sh generates every secret locally — nothing leaves your machine.- Prompt/response content is encrypted at rest (AES-256-GCM) by default.
- The admin key (
ANYRAY_ADMIN_TOKEN in .env) gates the console and all admin
APIs. Rotate by editing .env and running docker compose up -d.
- The Billing URL and lease verification key are pinned in the gateway image.
Gateway installs send only content-free usage totals; the pseudonym salt
stays on your machine.
Testing policy — every artifact gets a LIVE lane
Static validation (cfn-lint, helm template, jq, docker compose config) proves an
artifact parses — it cannot prove it behaves. The July 2026 console-500 shipped green through
every static check and was only observable on a real deployment. Rule: an install artifact
is not "supported" until CI deploys it for real, probes the golden paths (console sign-in,
authenticated console, /admin/health through the proxy, scripts/verify-deploy.sh), tests
the UPDATE path from the currently published version, and tears it down.
| Artifact | Static check | Live lane |
|---|
docker-compose.yml | validate-artifacts | smoke.yml ✓ |
docker-compose.attach.yml | validate-artifacts | smoke-attach.yml ✓ |
aws/anyray-quicklaunch.template.yaml | validate-artifacts (cfn-lint) | smoke-aws-live.yml ✓ (fresh + customer-upgrade lanes, manual workflow_dispatch — run before a template change) |
helm/ | validate-artifacts (render) | ✗ TODO — kind-based lane (#126) |
railway/railway.template.json | validate-artifacts (jq) | ✗ TODO (#127) |
gcp/ GCE one-click | — | ✗ TODO (#128) |
Adding or materially changing an artifact? Add/extend its live lane in the same PR.
The agent install reference documents the prompt, progress states,
and credential boundary.
Releasing (maintainers)
The anyray-connect binary release + code-signing pipeline (and the signing
secrets an admin must provision) is documented in RELEASING.md.