LLMOps: A pluggable LLMOps toolkit for TypeScript applications.

LLMOps integrates with LangChain in two ways:

LLM Proxy — Route all LangChain LLM calls through LLMOps for cost tracking, guardrails, and provider routing.
Tracing — Send LangChain execution traces to LLMOps for full observability of chains, tools, and retrievers.

LLM Proxy

LLMOps exposes an OpenAI-compatible API. Point LangChain's ChatOpenAI at your LLMOps instance to route all LLM calls through it.

Installation

npm install @langchain/openai langchain

For Python:

pip install langchain-openai

Setup

There are two ways to connect LangChain to LLMOps depending on your architecture:

Using `provider()` (embedded)

If LLMOps is running inside your application (e.g., as Express middleware), use llmopsClient.provider(). This routes requests in-process without an extra HTTP hop:

chat.ts

import { ChatOpenAI } from '@langchain/openai';
import llmopsClient from './llmops';

const { baseURL, apiKey, fetch } = llmopsClient.provider();

const llm = new ChatOpenAI({
  configuration: { baseURL, fetch },
  apiKey,
  model: '@openai/gpt-4o-mini',
});

Since provider() bypasses the HTTP gateway, traces are not automatically created. Use langchainTracer() or an OTLP exporter to send traces to LLMOps.

Using explicit `baseURL` (proxy)

If LLMOps is running as a standalone proxy, or you want the gateway to automatically create traces, point LangChain at the gateway URL directly:

chat.ts

import { ChatOpenAI } from '@langchain/openai';

const llm = new ChatOpenAI({
  configuration: {
    baseURL: 'http://localhost:3000/llmops/api/genai/v1',
  },
  apiKey: process.env.LLMOPS_ENV_SECRET,
  model: '@openai/gpt-4o-mini',
});

This makes real HTTP requests to the gateway, which automatically creates traces for every call.

Environment variables

.env

LLMOPS_ENV_SECRET=your-environment-secret

The @slug/model format routes requests through a specific provider configured in your LLMOps dashboard. The @openai/ prefix routes through the provider with slug openai. You can use any provider slug configured in your environment (e.g., @anthropic/claude-sonnet-4-20250514, @openrouter/google/gemini-2.0-flash).

Config-based Routing

Instead of @slug/model, you can route via a config ID. Configs let you define model, provider, and guardrails in the LLMOps dashboard:

const llm = new ChatOpenAI({
  configuration: {
    baseURL: 'http://localhost:3000/llmops/api/genai/v1',
    defaultHeaders: {
      'x-llmops-config': 'your-config-id',
    },
  },
  apiKey: process.env.LLMOPS_ENV_SECRET,
  model: 'gpt-4o-mini',
});

Chat Completions

import { ChatOpenAI } from '@langchain/openai';

const llm = new ChatOpenAI({
  configuration: {
    baseURL: 'http://localhost:3000/llmops/api/genai/v1',
  },
  apiKey: process.env.LLMOPS_ENV_SECRET,
  model: '@openai/gpt-4o-mini',
});

const response = await llm.invoke('What is LLMOps?');
console.log(response.content);

Streaming

const stream = await llm.stream('Tell me a joke');

for await (const chunk of stream) {
  process.stdout.write(chunk.content as string);
}

Embeddings

import { OpenAIEmbeddings } from '@langchain/openai';

const embeddings = new OpenAIEmbeddings({
  configuration: {
    baseURL: 'http://localhost:3000/llmops/api/genai/v1',
  },
  apiKey: process.env.LLMOPS_ENV_SECRET,
  model: '@openai/text-embedding-3-small',
});

const vectors = await embeddings.embedQuery('Hello world');
console.log(vectors.length);

Tool Calling

LangChain tool calling works through LLMOps with no changes:

import { tool } from '@langchain/core/tools';
import { z } from 'zod';

const weatherTool = tool(
  async ({ city }) => `The weather in ${city} is sunny, 72°F.`,
  {
    name: 'get_weather',
    description: 'Get the weather for a city',
    schema: z.object({ city: z.string() }),
  }
);

const llmWithTools = llm.bindTools([weatherTool]);
const response = await llmWithTools.invoke('What is the weather in San Francisco?');

Tracing

LLMOps captures the full LangChain execution hierarchy — agents, chains, LLM calls, tool invocations, and retrievers — as nested spans in a single trace.

TypeScript (SDK)

The recommended approach for TypeScript. Uses LangChain's built-in LangChainTracer with an LLMOps-compatible client — no environment variables, no extra dependencies.

Install dependencies

npm install @langchain/core @langchain/openai @langchain/langgraph

Create the tracer

Use llmopsClient.langchainTracer() to create a client, then pass it to LangChain's LangChainTracer:

agent.ts

import { ChatOpenAI } from '@langchain/openai';
import { LangChainTracer } from '@langchain/core/tracers/tracer_langchain';
import { createReactAgent } from '@langchain/langgraph/prebuilt';
import llmopsClient from './llmops';

const tracer = new LangChainTracer({
  client: llmopsClient.langchainTracer(),
});

Use `provider()` for the LLM

Since the tracer captures all runs, use provider() for the LLM to avoid double-tracing:

agent.ts

const { baseURL, apiKey, fetch } = llmopsClient.provider();

const llm = new ChatOpenAI({
  configuration: { baseURL, fetch },
  apiKey,
  model: '@openai/gpt-4o-mini',
});

Pass the tracer to your agent or chain

Add the tracer to the callbacks array when invoking:

agent.ts

const agent = createReactAgent({ llm, tools });

const result = await agent.invoke(
  { messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }] },
  { callbacks: [tracer] }
);

Every step — the agent loop, each LLM call, each tool invocation — appears as a nested span under a single trace.

Python (Environment Variables)

LLMOps exposes a LangSmith-compatible endpoint. Set the standard LangChain tracing environment variables to point at your LLMOps instance:

.env

LANGCHAIN_TRACING_V2=true
LANGCHAIN_ENDPOINT=http://localhost:3000/llmops/api/langsmith
LANGCHAIN_API_KEY=your-environment-secret

Then use LangChain as usual — all runs are automatically traced:

app.py

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="http://localhost:3000/llmops/api/genai/v1",
    api_key=os.environ["LLMOPS_ENV_SECRET"],
    model="@openai/gpt-4o-mini",
)

# This call is traced end-to-end in LLMOps
response = llm.invoke("What is LLMOps?")

OpenTelemetry

LangChain also supports native OpenTelemetry export. Use the OTLP endpoint if you prefer OTel-based tracing:

tracing.py

from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

exporter = OTLPSpanExporter(
    endpoint="http://localhost:3000/llmops/api/otlp/v1/traces",
    headers={
        "Authorization": f"Bearer {os.environ['LLMOPS_ENV_SECRET']}",
    },
)

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(exporter))

from opentelemetry import trace
trace.set_tracer_provider(provider)

Python (Full Example)

A complete Python setup using LLMOps as a proxy for both LLM routing and tracing:

import os
from langchain_openai import ChatOpenAI, OpenAIEmbeddings

llm = ChatOpenAI(
    base_url="http://localhost:3000/llmops/api/genai/v1",
    api_key=os.environ["LLMOPS_ENV_SECRET"],
    model="@openai/gpt-4o-mini",
)

# Chat
response = llm.invoke("What is LLMOps?")
print(response.content)

# Streaming
for chunk in llm.stream("Tell me a joke"):
    print(chunk.content, end="")

# Embeddings
embeddings = OpenAIEmbeddings(
    base_url="http://localhost:3000/llmops/api/genai/v1",
    api_key=os.environ["LLMOPS_ENV_SECRET"],
    model="@openai/text-embedding-3-small",
)
vectors = embeddings.embed_query("Hello world")

LangChain

LLM Proxy

Installation

Setup

Using `provider()` (embedded)

Using explicit `baseURL` (proxy)

Environment variables

Provider Routing

Config-based Routing

Chat Completions

Streaming

Embeddings

Tool Calling

Tracing

TypeScript (SDK)

Install dependencies

Create the tracer

Use `provider()` for the LLM

Pass the tracer to your agent or chain

Python (Environment Variables)

OpenTelemetry

Python (Full Example)

On this page