Initial commit
This commit is contained in:
Aodhan Collins
278
docs/tts/TTS_CONVERSATION_MODE.md
Normal file
278
docs/tts/TTS_CONVERSATION_MODE.md
Normal file
@@ -0,0 +1,278 @@
|
||||
# 🎧 TTS Conversation Mode
|
||||
|
||||
## Overview
|
||||
|
||||
EVE now supports **Audio Conversation Mode** - an immersive listening experience where assistant responses automatically play as audio with text hidden by default, similar to talking with a voice assistant like Alexa or Siri.
|
||||
|
||||
## Features
|
||||
|
||||
### Auto-Play Audio
|
||||
- ✅ Assistant responses automatically start playing when received
|
||||
- ✅ No manual clicking on speaker icon required
|
||||
- ✅ Seamless conversation flow
|
||||
|
||||
### Hidden Text by Default
|
||||
- ✅ Text is hidden by default to focus on audio
|
||||
- ✅ "Show Text" / "Hide Text" toggle button on each message
|
||||
- ✅ Text can be revealed anytime with one click
|
||||
- ✅ Visual indicator shows Audio Mode is active
|
||||
|
||||
### Visual Feedback
|
||||
- ✅ Animated pulsing audio controls during playback
|
||||
- ✅ Purple "Audio Mode" badge on messages
|
||||
- ✅ Pulsing volume icon on mode toggle button
|
||||
- ✅ Loading spinner during audio generation
|
||||
|
||||
### Mode Toggle Options
|
||||
|
||||
**Option 1: Quick Toggle Button**
|
||||
- Located above chat input (when TTS enabled)
|
||||
- Click to instantly switch modes
|
||||
- Visual states:
|
||||
- 🔊 Purple gradient + pulsing = Audio Mode ON
|
||||
- 📄 Gray = Text Mode (normal)
|
||||
|
||||
**Option 2: Settings Panel**
|
||||
- Settings → Voice Settings
|
||||
- Checkbox: "🎧 Audio Conversation Mode"
|
||||
- Description explains functionality
|
||||
|
||||
## How to Use
|
||||
|
||||
### Enabling Audio Mode
|
||||
|
||||
**Quick Method:**
|
||||
1. Enable TTS in Settings
|
||||
2. Click the mode toggle button above input
|
||||
3. Purple gradient + pulsing icon = Audio Mode active
|
||||
|
||||
**Settings Method:**
|
||||
1. Open Settings (⚙️)
|
||||
2. Go to Voice Settings
|
||||
3. Enable "Text-to-speech"
|
||||
4. Check "🎧 Audio Conversation Mode"
|
||||
|
||||
### Using Audio Mode
|
||||
|
||||
1. **Send a message** as normal
|
||||
2. **Assistant responds** - Audio auto-plays automatically
|
||||
3. **Text is hidden** - Focus on listening
|
||||
4. **Want to see text?** Click "Show Text" button
|
||||
5. **Hide text again?** Click "Hide Text" button
|
||||
6. **Control playback** using pause/stop buttons
|
||||
|
||||
### Switching Modes
|
||||
|
||||
**To Audio Mode:**
|
||||
- Click toggle button (shows 🔊)
|
||||
- Or check box in Settings
|
||||
|
||||
**To Text Mode:**
|
||||
- Click toggle button (shows 📄)
|
||||
- Or uncheck box in Settings
|
||||
|
||||
## Visual Indicators
|
||||
|
||||
### Audio Mode Active
|
||||
- **Toggle button**: Purple gradient with pulsing 🔊 icon
|
||||
- **Messages**: Purple "Audio Mode" badge
|
||||
- **Controls**: Pulsing animation during playback
|
||||
- **Text**: Hidden with "Show Text" button visible
|
||||
|
||||
### Text Mode Active
|
||||
- **Toggle button**: Gray with 📄 icon
|
||||
- **Messages**: Normal display with text visible
|
||||
- **Controls**: Standard speaker icon (click to play)
|
||||
|
||||
## Use Cases
|
||||
|
||||
### Hands-Free Interaction
|
||||
- Listen while working on other tasks
|
||||
- Accessibility for visually impaired users
|
||||
- Multitasking while getting information
|
||||
|
||||
### Language Learning
|
||||
- Hear pronunciation naturally
|
||||
- Focus on listening comprehension
|
||||
- Reduce reading, increase hearing
|
||||
|
||||
### Driving/Commuting
|
||||
- Safe voice-only interaction
|
||||
- No need to read screen
|
||||
- Conversational experience
|
||||
|
||||
### Preference
|
||||
- Some users prefer audio over reading
|
||||
- More natural conversation feel
|
||||
- Less eye strain
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Auto-Play Implementation
|
||||
```typescript
|
||||
// In ChatMessage component
|
||||
{ttsConversationMode && (
|
||||
<TTSControls
|
||||
autoPlay={true} // Triggers auto-play on mount
|
||||
text={message.content}
|
||||
/>
|
||||
)}
|
||||
```
|
||||
|
||||
### Text Visibility Toggle
|
||||
```typescript
|
||||
const [isTextVisible, setIsTextVisible] = useState(!ttsConversationMode)
|
||||
|
||||
// Button to toggle
|
||||
<button onClick={() => setIsTextVisible(!isTextVisible)}>
|
||||
{isTextVisible ? 'Hide Text' : 'Show Text'}
|
||||
</button>
|
||||
|
||||
// Conditional rendering
|
||||
{isTextVisible && <MessageContent content={message.content} />}
|
||||
```
|
||||
|
||||
### Auto-Play Trigger
|
||||
```typescript
|
||||
// In TTSControls component
|
||||
useEffect(() => {
|
||||
if (autoPlay && voiceEnabled && !isPlaying && !isLoading) {
|
||||
const timer = setTimeout(() => {
|
||||
handlePlay() // Start playing after 500ms delay
|
||||
}, 500)
|
||||
return () => clearTimeout(timer)
|
||||
}
|
||||
}, []) // Only run on mount
|
||||
```
|
||||
|
||||
## Settings Persistence
|
||||
|
||||
Audio Mode preference is:
|
||||
- ✅ Saved to localStorage
|
||||
- ✅ Persists across sessions
|
||||
- ✅ Synced between toggle button and settings
|
||||
- ✅ Independent of TTS enabled state
|
||||
|
||||
```typescript
|
||||
// Settings store
|
||||
ttsConversationMode: boolean // Defaults to false
|
||||
```
|
||||
|
||||
## Behavior Details
|
||||
|
||||
### When Audio Mode is ON
|
||||
1. **User sends message**
|
||||
2. **Assistant responds**
|
||||
3. **TTS auto-plays** (500ms delay for initialization)
|
||||
4. **Text is hidden**
|
||||
5. **Controls show** pause/stop buttons
|
||||
6. **User can**:
|
||||
- Show/hide text anytime
|
||||
- Control playback (pause/resume/stop)
|
||||
- Continue normal conversation
|
||||
|
||||
### When Audio Mode is OFF
|
||||
1. **User sends message**
|
||||
2. **Assistant responds**
|
||||
3. **Text is shown** immediately
|
||||
4. **No auto-play**
|
||||
5. **User can** click speaker icon to manually play
|
||||
|
||||
## Limitations
|
||||
|
||||
### Current Limitations
|
||||
- Auto-play only works if TTS is enabled
|
||||
- Requires valid TTS voice selection
|
||||
- Browser may block auto-play (user gesture required first time)
|
||||
- One message plays at a time
|
||||
|
||||
### Future Enhancements
|
||||
- **Queue system** - Auto-play multiple messages
|
||||
- **Speed controls** during playback
|
||||
- **Skip forward/back** buttons
|
||||
- **Transcript following** - Highlight words as spoken
|
||||
- **Voice response** - Auto-enable STT after assistant speaks
|
||||
- **Conversation history** - Audio mode for old messages
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Audio Not Auto-Playing
|
||||
**Check:**
|
||||
- ✅ TTS is enabled in settings
|
||||
- ✅ Voice is selected (not "default")
|
||||
- ✅ Audio Mode is ON (purple button)
|
||||
- ✅ ElevenLabs API key configured (for ElevenLabs voices)
|
||||
|
||||
**Try:**
|
||||
- Click speaker icon manually once (browser permission)
|
||||
- Reload page
|
||||
- Check browser console for errors
|
||||
|
||||
### Text Not Hiding
|
||||
**Check:**
|
||||
- ✅ Audio Mode is actually enabled
|
||||
- ✅ Purple badge shows "Audio Mode"
|
||||
- ✅ New messages (existing messages don't update)
|
||||
|
||||
**Try:**
|
||||
- Send new message
|
||||
- Toggle mode off and on
|
||||
- Refresh page
|
||||
|
||||
### Controls Not Showing
|
||||
**Check:**
|
||||
- ✅ TTS is enabled
|
||||
- ✅ Message is from assistant (not user)
|
||||
|
||||
### Mode Toggle Button Not Visible
|
||||
**Check:**
|
||||
- ✅ TTS is enabled in settings
|
||||
- Only shows when `voiceEnabled === true`
|
||||
|
||||
## Tips
|
||||
|
||||
### Best Experience
|
||||
1. **Use ElevenLabs voices** for highest quality
|
||||
2. **Adjust speed** in settings for comfortable listening
|
||||
3. **Enable continuous STT** for hands-free input
|
||||
4. **Use quality headphones** for best audio
|
||||
5. **Start with short messages** to test
|
||||
|
||||
### Recommended Settings
|
||||
```
|
||||
Audio Mode: ON
|
||||
TTS Voice: ElevenLabs (any)
|
||||
TTS Model: eleven_turbo_v2_5
|
||||
Speed: 1.0x - 1.25x
|
||||
Stability: 50%
|
||||
Clarity: 75%
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Audio-First Workflow
|
||||
1. Enable Audio Mode
|
||||
2. Ask: "Explain quantum computing"
|
||||
3. Listen to audio explanation
|
||||
4. If needed, click "Show Text" for reference
|
||||
5. Ask follow-up question
|
||||
6. Repeat
|
||||
|
||||
### Reading + Listening
|
||||
1. Keep Audio Mode ON
|
||||
2. Click "Show Text" on messages
|
||||
3. Read while listening
|
||||
4. Best for learning/comprehension
|
||||
|
||||
### Pure Audio Experience
|
||||
1. Enable Audio Mode
|
||||
2. Enable Continuous STT
|
||||
3. Never touch keyboard
|
||||
4. Speak questions, listen to answers
|
||||
5. True voice assistant experience
|
||||
|
||||
---
|
||||
|
||||
**Status**: ✅ Complete
|
||||
**Version**: v0.2.0-rc
|
||||
**Date**: October 5, 2025
|
||||
Reference in New Issue
Block a user