279 lines
6.9 KiB
Markdown
279 lines
6.9 KiB
Markdown
# 🎧 TTS Conversation Mode
|
|
|
|
## Overview
|
|
|
|
EVE now supports **Audio Conversation Mode** - an immersive listening experience where assistant responses automatically play as audio with text hidden by default, similar to talking with a voice assistant like Alexa or Siri.
|
|
|
|
## Features
|
|
|
|
### Auto-Play Audio
|
|
- ✅ Assistant responses automatically start playing when received
|
|
- ✅ No manual clicking on speaker icon required
|
|
- ✅ Seamless conversation flow
|
|
|
|
### Hidden Text by Default
|
|
- ✅ Text is hidden by default to focus on audio
|
|
- ✅ "Show Text" / "Hide Text" toggle button on each message
|
|
- ✅ Text can be revealed anytime with one click
|
|
- ✅ Visual indicator shows Audio Mode is active
|
|
|
|
### Visual Feedback
|
|
- ✅ Animated pulsing audio controls during playback
|
|
- ✅ Purple "Audio Mode" badge on messages
|
|
- ✅ Pulsing volume icon on mode toggle button
|
|
- ✅ Loading spinner during audio generation
|
|
|
|
### Mode Toggle Options
|
|
|
|
**Option 1: Quick Toggle Button**
|
|
- Located above chat input (when TTS enabled)
|
|
- Click to instantly switch modes
|
|
- Visual states:
|
|
- 🔊 Purple gradient + pulsing = Audio Mode ON
|
|
- 📄 Gray = Text Mode (normal)
|
|
|
|
**Option 2: Settings Panel**
|
|
- Settings → Voice Settings
|
|
- Checkbox: "🎧 Audio Conversation Mode"
|
|
- Description explains functionality
|
|
|
|
## How to Use
|
|
|
|
### Enabling Audio Mode
|
|
|
|
**Quick Method:**
|
|
1. Enable TTS in Settings
|
|
2. Click the mode toggle button above input
|
|
3. Purple gradient + pulsing icon = Audio Mode active
|
|
|
|
**Settings Method:**
|
|
1. Open Settings (⚙️)
|
|
2. Go to Voice Settings
|
|
3. Enable "Text-to-speech"
|
|
4. Check "🎧 Audio Conversation Mode"
|
|
|
|
### Using Audio Mode
|
|
|
|
1. **Send a message** as normal
|
|
2. **Assistant responds** - Audio auto-plays automatically
|
|
3. **Text is hidden** - Focus on listening
|
|
4. **Want to see text?** Click "Show Text" button
|
|
5. **Hide text again?** Click "Hide Text" button
|
|
6. **Control playback** using pause/stop buttons
|
|
|
|
### Switching Modes
|
|
|
|
**To Audio Mode:**
|
|
- Click toggle button (shows 🔊)
|
|
- Or check box in Settings
|
|
|
|
**To Text Mode:**
|
|
- Click toggle button (shows 📄)
|
|
- Or uncheck box in Settings
|
|
|
|
## Visual Indicators
|
|
|
|
### Audio Mode Active
|
|
- **Toggle button**: Purple gradient with pulsing 🔊 icon
|
|
- **Messages**: Purple "Audio Mode" badge
|
|
- **Controls**: Pulsing animation during playback
|
|
- **Text**: Hidden with "Show Text" button visible
|
|
|
|
### Text Mode Active
|
|
- **Toggle button**: Gray with 📄 icon
|
|
- **Messages**: Normal display with text visible
|
|
- **Controls**: Standard speaker icon (click to play)
|
|
|
|
## Use Cases
|
|
|
|
### Hands-Free Interaction
|
|
- Listen while working on other tasks
|
|
- Accessibility for visually impaired users
|
|
- Multitasking while getting information
|
|
|
|
### Language Learning
|
|
- Hear pronunciation naturally
|
|
- Focus on listening comprehension
|
|
- Reduce reading, increase hearing
|
|
|
|
### Driving/Commuting
|
|
- Safe voice-only interaction
|
|
- No need to read screen
|
|
- Conversational experience
|
|
|
|
### Preference
|
|
- Some users prefer audio over reading
|
|
- More natural conversation feel
|
|
- Less eye strain
|
|
|
|
## Technical Details
|
|
|
|
### Auto-Play Implementation
|
|
```typescript
|
|
// In ChatMessage component
|
|
{ttsConversationMode && (
|
|
<TTSControls
|
|
autoPlay={true} // Triggers auto-play on mount
|
|
text={message.content}
|
|
/>
|
|
)}
|
|
```
|
|
|
|
### Text Visibility Toggle
|
|
```typescript
|
|
const [isTextVisible, setIsTextVisible] = useState(!ttsConversationMode)
|
|
|
|
// Button to toggle
|
|
<button onClick={() => setIsTextVisible(!isTextVisible)}>
|
|
{isTextVisible ? 'Hide Text' : 'Show Text'}
|
|
</button>
|
|
|
|
// Conditional rendering
|
|
{isTextVisible && <MessageContent content={message.content} />}
|
|
```
|
|
|
|
### Auto-Play Trigger
|
|
```typescript
|
|
// In TTSControls component
|
|
useEffect(() => {
|
|
if (autoPlay && voiceEnabled && !isPlaying && !isLoading) {
|
|
const timer = setTimeout(() => {
|
|
handlePlay() // Start playing after 500ms delay
|
|
}, 500)
|
|
return () => clearTimeout(timer)
|
|
}
|
|
}, []) // Only run on mount
|
|
```
|
|
|
|
## Settings Persistence
|
|
|
|
Audio Mode preference is:
|
|
- ✅ Saved to localStorage
|
|
- ✅ Persists across sessions
|
|
- ✅ Synced between toggle button and settings
|
|
- ✅ Independent of TTS enabled state
|
|
|
|
```typescript
|
|
// Settings store
|
|
ttsConversationMode: boolean // Defaults to false
|
|
```
|
|
|
|
## Behavior Details
|
|
|
|
### When Audio Mode is ON
|
|
1. **User sends message**
|
|
2. **Assistant responds**
|
|
3. **TTS auto-plays** (500ms delay for initialization)
|
|
4. **Text is hidden**
|
|
5. **Controls show** pause/stop buttons
|
|
6. **User can**:
|
|
- Show/hide text anytime
|
|
- Control playback (pause/resume/stop)
|
|
- Continue normal conversation
|
|
|
|
### When Audio Mode is OFF
|
|
1. **User sends message**
|
|
2. **Assistant responds**
|
|
3. **Text is shown** immediately
|
|
4. **No auto-play**
|
|
5. **User can** click speaker icon to manually play
|
|
|
|
## Limitations
|
|
|
|
### Current Limitations
|
|
- Auto-play only works if TTS is enabled
|
|
- Requires valid TTS voice selection
|
|
- Browser may block auto-play (user gesture required first time)
|
|
- One message plays at a time
|
|
|
|
### Future Enhancements
|
|
- **Queue system** - Auto-play multiple messages
|
|
- **Speed controls** during playback
|
|
- **Skip forward/back** buttons
|
|
- **Transcript following** - Highlight words as spoken
|
|
- **Voice response** - Auto-enable STT after assistant speaks
|
|
- **Conversation history** - Audio mode for old messages
|
|
|
|
## Troubleshooting
|
|
|
|
### Audio Not Auto-Playing
|
|
**Check:**
|
|
- ✅ TTS is enabled in settings
|
|
- ✅ Voice is selected (not "default")
|
|
- ✅ Audio Mode is ON (purple button)
|
|
- ✅ ElevenLabs API key configured (for ElevenLabs voices)
|
|
|
|
**Try:**
|
|
- Click speaker icon manually once (browser permission)
|
|
- Reload page
|
|
- Check browser console for errors
|
|
|
|
### Text Not Hiding
|
|
**Check:**
|
|
- ✅ Audio Mode is actually enabled
|
|
- ✅ Purple badge shows "Audio Mode"
|
|
- ✅ New messages (existing messages don't update)
|
|
|
|
**Try:**
|
|
- Send new message
|
|
- Toggle mode off and on
|
|
- Refresh page
|
|
|
|
### Controls Not Showing
|
|
**Check:**
|
|
- ✅ TTS is enabled
|
|
- ✅ Message is from assistant (not user)
|
|
|
|
### Mode Toggle Button Not Visible
|
|
**Check:**
|
|
- ✅ TTS is enabled in settings
|
|
- Only shows when `voiceEnabled === true`
|
|
|
|
## Tips
|
|
|
|
### Best Experience
|
|
1. **Use ElevenLabs voices** for highest quality
|
|
2. **Adjust speed** in settings for comfortable listening
|
|
3. **Enable continuous STT** for hands-free input
|
|
4. **Use quality headphones** for best audio
|
|
5. **Start with short messages** to test
|
|
|
|
### Recommended Settings
|
|
```
|
|
Audio Mode: ON
|
|
TTS Voice: ElevenLabs (any)
|
|
TTS Model: eleven_turbo_v2_5
|
|
Speed: 1.0x - 1.25x
|
|
Stability: 50%
|
|
Clarity: 75%
|
|
```
|
|
|
|
## Examples
|
|
|
|
### Audio-First Workflow
|
|
1. Enable Audio Mode
|
|
2. Ask: "Explain quantum computing"
|
|
3. Listen to audio explanation
|
|
4. If needed, click "Show Text" for reference
|
|
5. Ask follow-up question
|
|
6. Repeat
|
|
|
|
### Reading + Listening
|
|
1. Keep Audio Mode ON
|
|
2. Click "Show Text" on messages
|
|
3. Read while listening
|
|
4. Best for learning/comprehension
|
|
|
|
### Pure Audio Experience
|
|
1. Enable Audio Mode
|
|
2. Enable Continuous STT
|
|
3. Never touch keyboard
|
|
4. Speak questions, listen to answers
|
|
5. True voice assistant experience
|
|
|
|
---
|
|
|
|
**Status**: ✅ Complete
|
|
**Version**: v0.2.0-rc
|
|
**Date**: October 5, 2025
|