Voice Configuration

Backend: services/api-gateway/app/core/voice_constants.py Frontend: apps/web-app/src/lib/voiceConstants.ts Status: Production Ready Last Updated: 2025-12-05

Overview

The VoiceAssist platform uses a centralized voice configuration system to ensure consistent voice selection across all components. This prevents issues where different parts of the system use different default voices, which would result in inconsistent user experience (e.g., dual voices playing simultaneously).

Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                     SINGLE SOURCE OF TRUTH                              │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│   Backend: voice_constants.py          Frontend: voiceConstants.ts      │
│   ┌───────────────────────────┐       ┌───────────────────────────┐    │
│   │ DEFAULT_VOICE_ID = BRIAN  │       │ DEFAULT_VOICE_ID = BRIAN  │    │
│   │ DEFAULT_TTS_MODEL         │       │ VoiceInfo metadata        │    │
│   │ ElevenLabsVoice enum      │       │ ElevenLabsVoices enum     │    │
│   └─────────────┬─────────────┘       └─────────────┬─────────────┘    │
│                 │                                   │                   │
│                 ▼                                   ▼                   │
│   ┌─────────────────────────┐         ┌─────────────────────────┐      │
│   │ config.py               │         │ voiceSettingsStore.ts   │      │
│   │ elevenlabs_service.py   │         │ ThinkerTalkerVoicePanel │      │
│   │ voice_pipeline_service  │         │ useBargeInPromptAudio   │      │
│   └─────────────────────────┘         └─────────────────────────┘      │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

Backend Configuration

voice_constants.py

The central configuration file for all voice-related constants:

from app.core.voice_constants import (
    DEFAULT_VOICE_ID,
    DEFAULT_TTS_MODEL,
    DEFAULT_TTS_OUTPUT_FORMAT,
    ElevenLabsVoice,
    get_openai_voice_for_elevenlabs,
)

# Current default voice
print(DEFAULT_VOICE_ID)  # "nPczCjzI2devNBz1zQrb" (Brian)

# Available voices
for voice in ElevenLabsVoice:
    print(f"{voice.name}: {voice.value}")

Available Voices

Services Using voice_constants

1. config.py - Environment configuration defaults
2. elevenlabs_service.py - ElevenLabs TTS service
3. voice_pipeline_service.py - Voice pipeline configuration

Frontend Configuration

voiceConstants.ts

The frontend equivalent for voice configuration:

import {
  DEFAULT_VOICE_ID,
  ElevenLabsVoices,
  VoiceInfo,
  getVoiceName,
  isValidVoiceId,
  getAvailableVoices,
} from "../lib/voiceConstants";

// Current default voice
console.log(DEFAULT_VOICE_ID); // "nPczCjzI2devNBz1zQrb" (Brian)

// Get voice name
console.log(getVoiceName(DEFAULT_VOICE_ID)); // "Brian"

// List all voices for a selector
const voices = getAvailableVoices();
voices.forEach((v) => console.log(`${v.name}: ${v.id}`));

Components Using voiceConstants

1. voiceSettingsStore.ts - User voice preferences
2. ThinkerTalkerVoicePanel.tsx - Voice mode UI
3. useBargeInPromptAudio.ts - Barge-in prompt audio

Changing the Default Voice

To change the default voice across the entire system, update only these two files:

Step 1: Update Backend

Edit services/api-gateway/app/core/voice_constants.py:

# Change from Brian to Josh
DEFAULT_VOICE_ID: str = ElevenLabsVoice.JOSH.value
DEFAULT_VOICE_NAME: str = "Josh"

Step 2: Update Frontend

Edit apps/web-app/src/lib/voiceConstants.ts:

// Change from Brian to Josh
export const DEFAULT_VOICE_ID = ElevenLabsVoices.JOSH;
export const DEFAULT_VOICE_NAME = "Josh";

Step 3: Rebuild and Deploy

# Rebuild Docker container
docker compose build voiceassist-server
docker compose up -d voiceassist-server

# Rebuild frontend (if needed)
cd apps/web-app && pnpm build

User Voice Selection

Users can override the default voice through the Voice Mode Settings:

1. Click the Settings button in Voice Mode
2. Select a voice from the Voice dropdown
3. The selected voice is stored in voiceSettingsStore (persisted in localStorage)
4. The frontend sends the selected voice_id to the backend with each request

The backend always uses the voice_id provided by the client. The default is only used when no voice_id is specified.

Preventing Dual Voice Issues

The centralized configuration prevents several common issues:

Problem: Multiple Default Voices

Before: Voice IDs were hardcoded in multiple files:

• config.py → Rachel
• voice_pipeline_service.py → Josh
• elevenlabs_service.py → Brian
• ThinkerTalkerVoicePanel.tsx → Josh

This caused dual voices when different components used different defaults.

After: All components import from the single source of truth:

• All backend services → voice_constants.py
• All frontend components → voiceConstants.ts

Problem: Browser TTS Fallback

Before: When ElevenLabs failed, the system fell back to browser TTS (SpeechSynthesis), which used a completely different voice.

After: Browser TTS fallback has been removed. If ElevenLabs fails, the prompt is silently skipped rather than played in a different voice.

Adding New Voices

Backend

1. Add the voice to ElevenLabsVoice enum in voice_constants.py:

class ElevenLabsVoice(str, Enum):
    # ... existing voices ...
    NEW_VOICE = "new-elevenlabs-voice-id"

2. Add voice info to get_voice_info():

cls.NEW_VOICE.value: {"name": "New Voice", "gender": "male", "style": "description"},

3. Add OpenAI fallback mapping:

ELEVENLABS_TO_OPENAI_VOICE_MAP: Dict[str, str] = {
    # ... existing mappings ...
    ElevenLabsVoice.NEW_VOICE.value: "onyx",
}

Frontend

1. Add the voice to ElevenLabsVoices in voiceConstants.ts:

export const ElevenLabsVoices = {
  // ... existing voices ...
  NEW_VOICE: "new-elevenlabs-voice-id",
} as const;

2. Add voice info to VoiceInfo:

[ElevenLabsVoices.NEW_VOICE]: { name: "New Voice", gender: "male", style: "description" },

API Reference

Backend: voice_constants.py

Frontend: voiceConstants.ts

Troubleshooting

Dual Voice Issue

If you hear two different voices:

1. Check that all backend services are using voice_constants.py
2. Check that all frontend components are using voiceConstants.ts
3. Verify the Docker container has been rebuilt with latest code
4. Clear browser localStorage to reset user preferences

Voice Not Changing

If the voice doesn't change after updating settings:

1. Ensure the voice_id is being sent with the WebSocket connection
2. Check the voiceSettingsStore has the new voice_id
3. Verify the backend logs show the correct voice_id

ElevenLabs API Errors

If ElevenLabs returns errors:

1. Check API key is configured: ELEVENLABS_API_KEY
2. Verify the voice_id is valid (use ElevenLabs dashboard)
3. Check rate limits and quota