llmops-build/llmops
Docs

LLM infrastructureinfrastructure that
grows with your

One SDK that gives you observability, prompt management, and evals for every LLM call. Install it with your first provider. The rest shows up when you need it.

Star on GitHubDocs
01

Start small!

One package. That's all you require to get started.

Add the SDK to your project.

$

Create a client — zero config needed.

import { Hono } from 'hono';
import { stream } from 'hono/streaming';
import { streamText } from 'ai';
import { createOpenAI } from '@ai-sdk/openai';
import { llmops } from '@llmops/sdk';
const llmopsClient = llmops();
const openai = createOpenAI(llmopsClient.provider());
const app = new Hono();

app.get('/', async (c) => {
  const result = streamText({
    model: openai.chat('@google/gemini-2.5-flash'),
    prompt: 'What model are you?',
  });

  return stream(c, async (stream) => {
    for await (const part of result.textStream) {
      await stream.write(part);
    }
  });
});

Route to any model through a unified interface.

http://localhost:3001

Prompt: What model are you?

Any provider, any model — one SDK.

02

Scale your providers

Organize multiple LLM providers with custom slugs. One config, many models.

Register providers with custom slugs.

import { llmops } from '@llmops/sdk';

const ops = llmops({
  providers: {
    'openai-prod': {
      type: 'openai',
      apiKey: process.env.OPENAI_API_KEY,
    },
    'anthropic-dev': {
      type: 'anthropic',
      apiKey: process.env.ANTHROPIC_API_KEY,
    },
  },
});
OpenAIAnthropicGoogleMistralCohereAWS BedrockAzureGroqDeepSeek+68 more

One config. Any model. Any provider.

03

Explore Visually

Connect a database, mount the middleware, and a full dashboard appears at /llmops.

Connect a Postgres database to store everything.

import { llmops } from '@llmops/sdk';
import { Pool } from 'pg';

export default llmops({
  database: new Pool({
    connectionString: process.env.DATABASE_URL,  }),
  // ...
});

Mount the middleware — the dashboard is served automatically.

import { Hono } from 'hono';
import { createLLMOpsMiddleware } from '@llmops/sdk/hono';
import ops from './llmops';

const app = new Hono();

app.use('/llmops/*', createLLMOpsMiddleware(ops));

export default app;
http://localhost:3000/llmops
LLMOps dashboard
04

See Everything

Every request is logged automatically. Costs, latency, tokens — all tracked without extra code.

Total Cost

$12.84

Input

$8.21

1.2M tokens
Output

$4.63

340K tokens
Requests

847

Cost by Model
gpt-4o: $0.042 (39%)
claude-sonnet: $0.031 (29%)
gemini-flash: $0.022 (21%)
kimi-k2: $0.012 (11%)
200moonshot/kimi-k21220ms
2026-02-12 21:46:39
200google/gemini-2.5-flash3163ms
2026-02-12 21:46:27
200gpt-4.1-nano7760ms
2026-02-09 17:31:41
200claude-sonnet-4-5-20250929876ms
2026-02-09 13:00:16
05

Version Your Prompts

Manage prompts from the UI and iterate without redeploying. Reference them by name in your API calls.

v3
GPT 4oactive

You are a helpful assistant. Greet {{userName}} warmly and ask how you can help today.

v2
Gemini 2.5 Flash

Assist {{userName}} with their query. Be concise and friendly.

v1
Claude Sonnet 4.5

You are a chatbot. Help the user.

import { streamText } from 'ai';
import { openai } from './providers';

const result = await streamText({
  model: openai.chat('@openai/gpt-4o'),
  headers: {
    'x-llmops-prompt': 'my-chatbot-prompt',
  },
  variables: { userName: 'Alice' },
});