llm-ports

🔌

Two-file rule

Only the adapter and the registry import from any LLM SDK. Your business code stays SDK-agnostic forever.

💵

USD-denominated cost gating

Set per-hour, per-day, per-month dollar caps per provider. Fallback chains pick the next provider when the cap trips.

🧱

17 capability factories

classify, score, draft, summarize, extract, plan, analyze (v0.1) — configure once, call many times. 10 more in v0.2.

🌐

4 adapters, 16+ providers

Anthropic, OpenAI (+12 compatible providers via baseURL — Groq, Together, Fireworks, Cerebras, Clarifai, SambaNova, and others), Ollama (local + model management), Vercel AI SDK migration helper.

🛡️

Tool-use security primitives

Mark tools destructive, requireConfirmation, set output size limits. Safety isn't an afterthought.

⚡

Negligible overhead

Mean p50 0.04 ms, max p99 0.47 ms framework overhead. 10x under the 5 ms target.

60 seconds

1. Configure providers in .env:

bash

LLM_PROVIDER_FAST=anthropic|claude-haiku-4-5|cost:5/day
LLM_PROVIDER_SMART=anthropic|claude-sonnet-4-6-20250514|cost:50/day
LLM_TASK_ROUTE_TRIAGE=fast,smart

2. Create the port once at app start:

import { createRegistryFromEnv } from "@llm-ports/core";
import { createAnthropicAdapter } from "@llm-ports/adapter-anthropic";

export const llm = createRegistryFromEnv({
  adapters: {
    anthropic: createAnthropicAdapter({ apiKey: process.env.ANTHROPIC_API_KEY! }),
  },
}).getPort();

3. Use anywhere in your codebase, no SDK imports:

const result = await llm.generateText({
  taskType: "triage",
  prompt: "Classify this email: ...",
});

The registry picks the model per task, enforces the cost cap, walks to the next provider when a budget is exhausted, and reports per-call latency and USD cost. (In v0.1, fallback walks the chain on budget gating today — runtime-error fallback ships in v0.2; see the multi-provider guide, or the v0.1 status page for the full inventory of what's stable and what's not.)

How it relates to other tools

Tool	How `llm-ports` relates
Vercel AI SDK	Vercel unifies provider calls. `llm-ports` adds registry, fallback chains, USD cost gating, validation recovery, and capability factories on top. Use `@llm-ports/adapter-vercel` to keep your existing setup.
LiteLLM	Python-first HTTP proxy. `llm-ports` is TypeScript in-process — zero network hop, no extra service to deploy. Talks to LiteLLM via the OpenAI adapter with `baseURL`.
Portkey	Commercial hosted gateway. `llm-ports` is MIT, in-process, no vendor lock-in. The tradeoff: Portkey ships a hosted UI; `llm-ports` does not.
LangChain.js	LangChain is a framework. `llm-ports` is a utility. Wrap LangChain LLM calls with a port for budget gating without adopting the whole framework.
LlamaIndex.TS	LlamaIndex is retrieval-first. `llm-ports` handles LLM invocation; bring your own retrieval. They compose cleanly.
Mastra	Mastra is opinionated agent-first with built-in memory. `llm-ports` is unopinionated primitives beneath that layer.

Tool

How llm-ports relates

Vercel AI SDK

Vercel unifies provider calls. llm-ports adds registry, fallback chains, USD cost gating, validation recovery, and capability factories on top. Use @llm-ports/adapter-vercel to keep your existing setup.

LiteLLM

Python-first HTTP proxy. llm-ports is TypeScript in-process — zero network hop, no extra service to deploy. Talks to LiteLLM via the OpenAI adapter with baseURL.

Portkey

Commercial hosted gateway. llm-ports is MIT, in-process, no vendor lock-in. The tradeoff: Portkey ships a hosted UI; llm-ports does not.

LangChain.js

LangChain is a framework. llm-ports is a utility. Wrap LangChain LLM calls with a port for budget gating without adopting the whole framework.

LlamaIndex.TS

LlamaIndex is retrieval-first. llm-ports handles LLM invocation; bring your own retrieval. They compose cleanly.

Mastra

Mastra is opinionated agent-first with built-in memory. llm-ports is unopinionated primitives beneath that layer.

llm-portsProvider-agnostic LLM architecture for TypeScript