Feature Variants & A/B Testing

Feature variants allow you to create A/B/n tests and gradual rollouts with consistent user assignment.

Basic Variant Configuration

// feature-flags.config.ts
import { defineFeatureFlags } from '#feature-flags/handler'

export default defineFeatureFlags(() => {
  return {
    // Simple A/B test
    buttonColor: {
      enabled: true,
      value: 'blue', // default value
      variants: [
        { name: 'blue', weight: 50 },
        { name: 'red', weight: 50, value: 'red' }
      ]
    },
    
    // A/B/C/D test
    homepage: {
      enabled: true,
      variants: [
        { name: 'original', weight: 40, value: 'v1' },
        { name: 'redesign', weight: 30, value: 'v2' },
        { name: 'minimal', weight: 20, value: 'v3' },
        { name: 'experimental', weight: 10, value: 'v4' }
      ]
    },
    
    // Gradual rollout (20% get new feature)
    newFeature: {
      enabled: true,
      variants: [
        { name: 'disabled', weight: 80, value: false },
        { name: 'enabled', weight: 20, value: true }
      ]
    }
  }
})

How Variant Assignment Works

Persistent Assignment

Users receive the same variant consistently across sessions and devices. This persistence is crucial for accurate A/B testing because it ensures:

• Consistent Experience: Users don't see different variants on each visit
• Reliable Analytics: User behavior can be accurately tracked to a specific variant
• Fair Testing: Each user contributes to only one variant's metrics

Identifier Priority

To ensure persistent assignment, the module needs a stable identifier for each user. Identifiers are selected using this priority order:

1. User ID (context.user.id or context.userId) - Highest priority
2. Session ID (from cookies: session-id or nuxt-session)
3. IP Address (from request headers)
4. Fallback: 'anonymous' (if none of the above are available)

Best practice: Populate context.user.id in your middleware for the most consistent variant assignment:

// server/middleware/user-context.ts
export default defineEventHandler((event) => {
  const user = await getUserFromSession(event)
  
  if (user) {
    event.context.user = {
      id: user.id,  // This will be used for variant assignment
      role: user.role,
      // ... other user properties
    }
  }
})

Using Variants in Templates

<template>
  <!-- Different button colors based on variant -->
  <button 
    v-feature="'buttonColor:blue'"
    class="bg-blue-500 text-white px-4 py-2"
  >
    Blue Button (50% of users)
  </button>
  
  <button 
    v-feature="'buttonColor:red'" 
    class="bg-red-500 text-white px-4 py-2"
  >
    Red Button (50% of users)
  </button>
  
  <!-- Conditional content based on variant -->
  <div v-if="getVariant('homepage') === 'redesign'">
    <h1>Welcome to our new design!</h1>
  </div>
</template>

Programmatic Variant Checking

const { isEnabled, getVariant, getValue } = useFeatureFlags()

// Check if user is in specific variant
if (isEnabled('buttonColor:red')) {
  // User sees red button
}

// Get the assigned variant name
const variant = getVariant('buttonColor') // 'blue' | 'red'

// Get the variant value
const color = getValue('buttonColor') // 'blue' | 'red'

// Use in computed properties
const buttonClass = computed(() => {
  const color = getValue('buttonColor')
  return `bg-${color}-500 text-white px-4 py-2`
})

Common Patterns

Gradual Rollout

Gradually increase the percentage of users who see a new feature:

// Week 1: 10% of users
newCheckout: {
  enabled: true,
  variants: [
    { name: 'old', weight: 90, value: false },
    { name: 'new', weight: 10, value: true }
  ]
}

// Week 2: Increase to 25%
// { name: 'old', weight: 75, value: false },
// { name: 'new', weight: 25, value: true }

// Week 3: Increase to 50%
// { name: 'old', weight: 50, value: false },
// { name: 'new', weight: 50, value: true }

A/B Test

Compare two versions to determine which performs better:

ctaButton: {
  enabled: true,
  variants: [
    { name: 'control', weight: 50, value: { color: 'blue', text: 'Sign Up' } },
    { name: 'treatment', weight: 50, value: { color: 'red', text: 'Get Started' } }
  ]
}

Canary Release

Release new features to a small subset of users first:

newApiIntegration: {
  enabled: true,
  variants: [
    { name: 'old-api', weight: 95, value: { endpoint: '/api/v1', version: 1 } },
    { name: 'new-api', weight: 5, value: { endpoint: '/api/v2', version: 2 } }
  ]
}