Introduction

LLMOps is a framework-agnostic, universal LLM operations toolkit for TypeScript. It provides a comprehensive set of features out of the box. Whether you need prompt versioning or multi-environment deployments, it lets you focus on building your application instead of reinventing the wheel.

Features

LLMOps aims to be the most comprehensive LLMOps toolkit. It provides a wide range of features out of the box and allows you to extend it with plugins. Here are some of the features:

• AI Gateway - OpenAI-compatible API to access 1600+ LLMs from 70+ providers
• Prompt Management - Version, test, and deploy prompts with a visual editor
• Environment Management - Production, staging, development with secure secrets
• Framework Agnostic - Works with Express, Hono, and more
• TypeScript-First - Strict type safe SDK
• Self-Hosted - Full control over your data and infrastructure

...and much more!

Quick Setup

1. Install

npm install @llmops/sdk

2. Add your provider keys

# Add your provider keys
OPENAI_API_KEY=sk-...

3. Use with the AI SDK

import { Hono } from 'hono';
import { stream } from 'hono/streaming';
import { streamText } from 'ai';
import { createOpenAI } from '@ai-sdk/openai';
import { llmops } from '@llmops/sdk';

const llmopsClient = llmops();
const openai = createOpenAI(llmopsClient.provider());

const app = new Hono();

app.get('/', async (c) => {
  const result = streamText({
    model: openai.chat('@google/gemini-2.5-flash'),
    prompt: 'What model are you?',
  });

  return stream(c, async (stream) => {
    for await (const part of result.textStream) {
      await stream.write(part);
    }
  });
});

Architecture

LLMOps is organized as a monorepo with focused packages:

Next Steps

• Quickstart - Get up and running in minutes