Skip to main content

Voice SDK

The KoIA Voice SDK is a web voice client for integration with KoIA agents. This SDK allows you to create interactive voice experiences using the KoIA platform with full TypeScript support.

Features​

  • ✅ Audio only: Optimized for voice experiences without video
  • 🎙️ Noise cancellation: Built-in advanced audio processing
  • 📱 Mobile support: Works on mobile and desktop devices
  • 🔊 Device control: Management of microphones and speakers
  • 📊 Real-time analytics: Volume, speech, and network quality events
  • 🎯 TypeScript: Fully typed for better development experience
  • 🔄 Event-driven: Robust event system for complete control

Installation​

Via npm​

npm install @ko-ia/voice-sdk

Via yarn​

yarn add @ko-ia/voice-sdk

Via pnpm​

pnpm add @ko-ia/voice-sdk

Basic Usage​

Import​

import KoiaVoiceClient, { 
  KoiaVoiceConfig, 
  KoiaAgentOverrides,
  KOIA_VOICE_EVENTS 
} from '@ko-ia/voice-sdk';

// or alternative import
import { createKoiaVoiceClient } from '@ko-ia/voice-sdk';

Initial Configuration​

const config: KoiaVoiceConfig = {
  apiToken: 'your-koia-api-token',
  baseUrl: 'https://api.ko-ia.com', // optional, defaults to dev
  audioConfig: {
    enableNoiseCancellation: true,
    enableEchoCancellation: true,
    startAudioOff: false
  },
  advanced: {
    avoidEval: true,
    alwaysIncludeMicInPermissionPrompt: false
  }
};

const voiceClient = new KoiaVoiceClient(config);

Complete Example​

import KoiaVoiceClient, { 
  KoiaVoiceConfig, 
  KOIA_VOICE_EVENTS 
} from '@ko-ia/voice-sdk';

// Configuration
const config: KoiaVoiceConfig = {
  apiToken: 'your-token-here',
  audioConfig: {
    enableNoiseCancellation: true,
    startAudioOff: false
  }
};

// Create client
const client = new KoiaVoiceClient(config);

// Event listeners
client.on(KOIA_VOICE_EVENTS.CALL_START, () => {
  console.log('Voice session started!');
});

client.on(KOIA_VOICE_EVENTS.MESSAGE, (message) => {
  console.log('Message received:', message);
});

client.on(KOIA_VOICE_EVENTS.VOLUME_LEVEL, (level) => {
  console.log('Volume level:', level);
});

client.on(KOIA_VOICE_EVENTS.ERROR, (error) => {
  console.error('Error:', error);
});

client.on(KOIA_VOICE_EVENTS.CALL_END, () => {
  console.log('Session ended');
});

// Start session
async function startSession() {
  try {
    const session = await client.startVoiceSession('your-agent-id');
    console.log('Session created:', session);
  } catch (error) {
    console.error('Error starting:', error);
  }
}

// Controls during session
function controlSession() {
  // Make agent speak
  client.speakText('Hello! How can I help you today?');
  
  // Mute/unmute microphone
  client.setMicrophoneEnabled(false); // mute
  client.setMicrophoneEnabled(true);  // unmute
  
  // Send text message
  client.sendMessage({
    role: 'user',
    content: 'This is a text message'
  });
  
  // Check status
  console.log('Session active:', client.isSessionActive());
  console.log('Microphone active:', client.isMicrophoneEnabled());
}

// End session
function endSession() {
  client.endVoiceSession();
}

// Initialize
startSession();

API Reference​

KoiaVoiceClient​

Constructor​

new KoiaVoiceClient(config: KoiaVoiceConfig)

Main Methods​

startVoiceSession(agentId: string, overrides?: KoiaAgentOverrides): Promise<KoiaVoiceSession>​

Starts a new voice session with the specified agent.

const session = await client.startVoiceSession('agent-123', {
  customInstructions: 'Be more formal in responses',
  voiceSettings: {
    speed: 1.1,
    volume: 0.8
  }
});
endVoiceSession(): void​

Ends the current voice session.

client.endVoiceSession();
speakText(text: string, options?: SpeakOptions): void​

Makes the agent speak a specific text.

client.speakText('Hello, world!', {
  endSessionAfter: false,
  interruptionsEnabled: true
});
sendMessage(message: KoiaMessagePayload): void​

Sends a text message to the agent.

client.sendMessage({
  role: 'user',
  content: 'How is the weather today?',
  timestamp: new Date().toISOString()
});
setMicrophoneEnabled(enabled: boolean): void​

Controls the microphone state.

client.setMicrophoneEnabled(false); // mute
client.setMicrophoneEnabled(true);  // unmute
isMicrophoneEnabled(): boolean​

Checks if the microphone is active.

const isActive = client.isMicrophoneEnabled();
isSessionActive(): boolean​

Checks if there is an active session.

const isActive = client.isSessionActive();

Audio Device Methods​

getAudioDevices(): Promise<{microphones: MediaDeviceInfo[], speakers: MediaDeviceInfo[]}>​

Gets list of available audio devices.

const devices = await client.getAudioDevices();
console.log('Microphones:', devices.microphones);
console.log('Speakers:', devices.speakers);
setAudioInputDevice(deviceId: string): Promise<void>​

Sets the audio input device.

await client.setAudioInputDevice('device-id-123');
setAudioOutputDevice(deviceId: string): Promise<void>​

Sets the audio output device.

await client.setAudioOutputDevice('device-id-456');

Diagnostic Methods​

getDiagnosticInfo(): any​

Gets diagnostic information from the session.

const diagnostics = client.getDiagnosticInfo();
console.log('Diagnostics:', diagnostics);
getNetworkStats(): any​

Gets network statistics from the current session.

const stats = client.getNetworkStats();
console.log('Network statistics:', stats);

Events​

The SDK emits various events that you can listen to:

// Session events
client.on('call-start', () => {});
client.on('call-end', () => {});

// Audio events
client.on('speech-start', () => {});
client.on('speech-end', () => {});
client.on('volume-level', (level: number) => {});

// Message events
client.on('message', (message: any) => {});

// Error events
client.on('error', (error: any) => {});

// Network events
client.on('network-quality-change', (event: any) => {});
client.on('network-connection', (event: any) => {});

// Progress events
client.on('call-start-progress', (event: CallStartProgressEvent) => {});
client.on('call-start-success', (event: CallStartSuccessEvent) => {});
client.on('call-start-failed', (event: CallStartFailedEvent) => {});

TypeScript Types​

KoiaVoiceConfig​

interface KoiaVoiceConfig {
  apiToken: string;
  baseUrl?: string;
  audioConfig?: {
    enableNoiseCancellation?: boolean;
    enableEchoCancellation?: boolean;
    audioSource?: boolean;
    startAudioOff?: boolean;
  };
  advanced?: {
    avoidEval?: boolean;
    alwaysIncludeMicInPermissionPrompt?: boolean;
  };
}

KoiaAgentOverrides​

interface KoiaAgentOverrides {
  customInstructions?: string;
  voiceSettings?: {
    speed?: number;
    pitch?: number;
    volume?: number;
    voiceId?: string;
  };
  variables?: Record<string, any>;
  responseConfig?: {
    maxTokens?: number;
    temperature?: number;
    enableInterruptions?: boolean;
  };
}

KoiaMessagePayload​

interface KoiaMessagePayload {
  role: 'user' | 'assistant' | 'system';
  content: string;
  timestamp?: string;
}

Practical Examples​

Example 1: Simple Voice Chat​

import KoiaVoiceClient from '@ko-ia/voice-sdk';

const client = new KoiaVoiceClient({
  apiToken: 'your-koia-token'
});

async function startVoiceChat() {
  // Configure events
  client.on('call-start', () => {
    console.log('Voice chat started!');
    updateUI('Connected');
  });

  client.on('message', (message) => {
    displayMessage(message.content, 'assistant');
  });

  client.on('speech-start', () => {
    updateUI('Listening...');
  });

  client.on('speech-end', () => {
    updateUI('Processing...');
  });

  // Start session
  await client.startVoiceSession('support-agent');
}

Example 2: Device Control​

async function configureDevices() {
  // List available devices
  const devices = await client.getAudioDevices();
  
  // Create device selectors
  const micSelect = document.getElementById('mic-select');
  const speakerSelect = document.getElementById('speaker-select');
  
  devices.microphones.forEach(mic => {
    const option = document.createElement('option');
    option.value = mic.deviceId;
    option.text = mic.label;
    micSelect.appendChild(option);
  });
  
  devices.speakers.forEach(speaker => {
    const option = document.createElement('option');
    option.value = speaker.deviceId;
    option.text = speaker.label;
    speakerSelect.appendChild(option);
  });
  
  // Configure change events
  micSelect.onchange = (e) => {
    client.setAudioInputDevice(e.target.value);
  };
  
  speakerSelect.onchange = (e) => {
    client.setAudioOutputDevice(e.target.value);
  };
}

Example 3: Interface with Controls​

<!DOCTYPE html>
<html>
<head>
    <title>KoIA Voice Interface</title>
</head>
<body>
    <div id="voice-interface">
        <h1>KoIA Voice Assistant</h1>
        
        <div id="status">Disconnected</div>
        
        <div id="controls">
            <button id="start-btn">Start Conversation</button>
            <button id="end-btn" disabled>End</button>
            <button id="mute-btn" disabled>Mute</button>
        </div>
        
        <div id="devices">
            <label>Microphone:</label>
            <select id="mic-select"></select>
            
            <label>Speaker:</label>
            <select id="speaker-select"></select>
        </div>
        
        <div id="volume-indicator">
            <div id="volume-bar"></div>
        </div>
        
        <div id="messages"></div>
    </div>

    <script type="module">
        import KoiaVoiceClient from '@ko-ia/voice-sdk';

        const client = new KoiaVoiceClient({
            apiToken: 'your-token-here'
        });

        // DOM references
        const statusDiv = document.getElementById('status');
        const startBtn = document.getElementById('start-btn');
        const endBtn = document.getElementById('end-btn');
        const muteBtn = document.getElementById('mute-btn');
        const volumeBar = document.getElementById('volume-bar');
        const messagesDiv = document.getElementById('messages');

        // Client event listeners
        client.on('call-start', () => {
            statusDiv.textContent = 'Connected';
            startBtn.disabled = true;
            endBtn.disabled = false;
            muteBtn.disabled = false;
        });

        client.on('call-end', () => {
            statusDiv.textContent = 'Disconnected';
            startBtn.disabled = false;
            endBtn.disabled = true;
            muteBtn.disabled = true;
        });

        client.on('volume-level', (level) => {
            volumeBar.style.width = `${level * 100}%`;
        });

        client.on('message', (message) => {
            const msgDiv = document.createElement('div');
            msgDiv.className = `message ${message.role}`;
            msgDiv.textContent = message.content;
            messagesDiv.appendChild(msgDiv);
        });

        // Button event listeners
        startBtn.onclick = () => {
            client.startVoiceSession('your-agent-id');
        };

        endBtn.onclick = () => {
            client.endVoiceSession();
        };

        muteBtn.onclick = () => {
            const isMuted = !client.isMicrophoneEnabled();
            client.setMicrophoneEnabled(isMuted);
            muteBtn.textContent = isMuted ? 'Unmute' : 'Mute';
        };

        // Configure devices on initialization
        async function init() {
            const devices = await client.getAudioDevices();
            // ... device configuration code
        }

        init();
    </script>
</body>
</html>

Troubleshooting​

Microphone Permissions​

Make sure your application has permission to use the microphone:

// Check permissions
navigator.permissions.query({ name: 'microphone' })
  .then((result) => {
    if (result.state === 'denied') {
      console.warn('Microphone permission denied');
    }
  });

Connection Issues​

client.on('error', (error) => {
  switch (error.type) {
    case 'network':
      console.error('Network error:', error.message);
      // Try to reconnect
      break;
    case 'auth':
      console.error('Authentication error:', error.message);
      // Check token
      break;
    case 'device':
      console.error('Device error:', error.message);
      // Check audio devices
      break;
  }
});

Debug and Diagnostics​

// Get diagnostic information
const diagnostics = client.getDiagnosticInfo();
console.log('Diagnostics:', diagnostics);

// Network statistics
const networkStats = client.getNetworkStats();
console.log('Network:', networkStats);

Support​

License​

MIT © KoIA