LLMOps

Providers

LLMOps supports 70+ AI providers through its unified gateway. This page lists all supported providers and their configuration options.

Base Configuration

All providers share these base options:

PropertyTypeRequiredDescription
apiKeystringYesAPI key for the provider
customHoststringNoCustom base URL for the provider
requestTimeoutnumberNoRequest timeout in milliseconds
forwardHeadersstring[]NoHeaders to forward from incoming requests

Major Cloud Providers

OpenAI

providers: {
  openai: {
    apiKey: process.env.OPENAI_API_KEY,
    openaiOrganization: 'org-xxx', // Optional
    openaiProject: 'proj-xxx',     // Optional
  },
}
PropertyTypeDescription
openaiOrganizationstringOpenAI organization ID
openaiProjectstringOpenAI project ID

Anthropic

providers: {
  anthropic: {
    apiKey: process.env.ANTHROPIC_API_KEY,
  },
}

Azure OpenAI

providers: {
  'azure-openai': {
    apiKey: process.env.AZURE_OPENAI_API_KEY,
    resourceName: 'my-resource',
    deploymentId: 'gpt-4',
    apiVersion: '2024-02-15-preview',
  },
}
PropertyTypeDescription
resourceNamestringAzure resource name
deploymentIdstringAzure deployment ID
apiVersionstringAzure API version

Google Gemini

providers: {
  google: {
    apiKey: process.env.GOOGLE_API_KEY,
  },
}

Google Vertex AI

providers: {
  'vertex-ai': {
    apiKey: '', // Required but not used
    vertexProjectId: 'my-project',
    vertexRegion: 'us-central1',
    vertexServiceAccountJson: {
      // Your service account JSON credentials
    },
  },
}
PropertyTypeDescription
vertexProjectIdstringGoogle Cloud project ID
vertexRegionstringVertex AI region (e.g., us-central1)
vertexServiceAccountJsonobjectService account JSON credentials

AWS Bedrock

providers: {
  bedrock: {
    apiKey: '', // Required but not used
    awsAccessKeyId: process.env.AWS_ACCESS_KEY_ID,
    awsSecretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    awsRegion: 'us-east-1',
  },
}
PropertyTypeDescription
awsAccessKeyIdstringAWS access key ID
awsSecretAccessKeystringAWS secret access key
awsSessionTokenstringAWS session token (for temporary credentials)
awsRegionstringAWS region (e.g., us-east-1)

Inference Providers

Groq

providers: {
  groq: {
    apiKey: process.env.GROQ_API_KEY,
  },
}

Together AI

providers: {
  'together-ai': {
    apiKey: process.env.TOGETHER_API_KEY,
  },
}

Fireworks AI

providers: {
  'fireworks-ai': {
    apiKey: process.env.FIREWORKS_API_KEY,
  },
}

Cerebras

providers: {
  cerebras: {
    apiKey: process.env.CEREBRAS_API_KEY,
  },
}

SambaNova

providers: {
  sambanova: {
    apiKey: process.env.SAMBANOVA_API_KEY,
  },
}

Lambda

providers: {
  lambda: {
    apiKey: process.env.LAMBDA_API_KEY,
  },
}

DeepInfra

providers: {
  deepinfra: {
    apiKey: process.env.DEEPINFRA_API_KEY,
  },
}

Model Providers

DeepSeek

providers: {
  deepseek: {
    apiKey: process.env.DEEPSEEK_API_KEY,
  },
}

Mistral AI

providers: {
  'mistral-ai': {
    apiKey: process.env.MISTRAL_API_KEY,
  },
}

Cohere

providers: {
  cohere: {
    apiKey: process.env.COHERE_API_KEY,
  },
}

AI21

providers: {
  ai21: {
    apiKey: process.env.AI21_API_KEY,
  },
}

xAI (Grok)

providers: {
  'x-ai': {
    apiKey: process.env.XAI_API_KEY,
  },
}

Aggregators & Routers

OpenRouter

OpenRouter gives you access to hundreds of models through a single API key.

providers: {
  openrouter: {
    apiKey: process.env.OPENROUTER_API_KEY,
  },
}

Perplexity AI

providers: {
  'perplexity-ai': {
    apiKey: process.env.PERPLEXITY_API_KEY,
  },
}

Embedding Providers

Jina

providers: {
  jina: {
    apiKey: process.env.JINA_API_KEY,
  },
}

Voyage

providers: {
  voyage: {
    apiKey: process.env.VOYAGE_API_KEY,
  },
}

Nomic

providers: {
  nomic: {
    apiKey: process.env.NOMIC_API_KEY,
  },
}

Image Generation

Stability AI

providers: {
  'stability-ai': {
    apiKey: process.env.STABILITY_API_KEY,
  },
}

Replicate

providers: {
  replicate: {
    apiKey: process.env.REPLICATE_API_KEY,
  },
}

Segmind

providers: {
  segmind: {
    apiKey: process.env.SEGMIND_API_KEY,
  },
}

Self-Hosted / Local

Ollama

For local models running with Ollama:

providers: {
  ollama: {
    apiKey: '', // Required but not used
    customHost: 'http://localhost:11434',
  },
}

Hugging Face

providers: {
  huggingface: {
    apiKey: process.env.HF_API_KEY,
    huggingfaceBaseUrl: 'https://api-inference.huggingface.co', // Optional
  },
}

All Supported Providers

Here's the complete list of all 70+ supported providers:

ProviderKeyProviderKey
OpenAIopenaiAnthropicanthropic
Azure OpenAIazure-openaiGoogle Geminigoogle
Vertex AIvertex-aiAWS Bedrockbedrock
AWS SageMakersagemakerGroqgroq
Together AItogether-aiFireworks AIfireworks-ai
CerebrascerebrasSambaNovasambanova
LambdalambdaDeepInfradeepinfra
DeepSeekdeepseekMistral AImistral-ai
CoherecohereAI21ai21
xAIx-aiOpenRouteropenrouter
Perplexity AIperplexity-aiJinajina
VoyagevoyageNomicnomic
Stability AIstability-aiReplicatereplicate
SegmindsegmindOllamaollama
Hugging FacehuggingfaceCloudflare Workers AIworkers-ai
AnyscaleanyscaleReka AIreka-ai
MoonshotmoonshotZhipuzhipu
LingYilingyiNebiusnebius
OracleoracleAzure AIazure-ai

On this page