Variants

A Variant is a specific model configuration that can be attached to configs. Variants contain the actual LLM settings: provider, model, parameters, and system prompt.

Overview

Each variant:

• Has a name for easy identification
• Contains model configuration (provider, model, parameters)
• Automatically creates versions when edited
• Can be attached to multiple configs

Creating a Variant

When creating or editing a variant, you configure:

Model Parameters

Versioning

Every time you edit a variant, a new version is created. This provides:

• Audit Trail: See exactly what configuration was used at any point in time
• Safe Rollbacks: Revert to a previous version if issues arise
• Version Pinning: Production can use a specific version while staging tests newer versions

Version Pinning vs Latest

When targeting a variant to an environment:

Example Workflow

1. Create a variant with your initial configuration

   • Provider: OpenAI
   • Model: gpt-4-turbo
   • System Prompt: "You are a helpful assistant..."
   • Temperature: 0.7
   • Creates Version 1

2. Attach to a config and set up targeting rules

   • Production → Version 1 (pinned)
   • Staging → Latest

3. Iterate on the variant

   • Update the system prompt
   • Creates Version 2
   • Staging automatically uses Version 2
   • Production still uses Version 1

4. Promote to production

   • After testing in staging, update production targeting to Version 2

Multiple Variants per Config

A single config can have multiple variants attached. This enables:

• A/B Testing: Compare different model configurations
• Gradual Rollouts: Slowly shift traffic to new configurations
• Fallback Options: Use a backup variant if the primary fails

Example Variants