feat: OpenClaw HTTP bridge, HA conversation agent fixes, voice pipeline tooling
- Add openclaw-http-bridge.py: HTTP server translating POST requests to OpenClaw CLI calls - Add launchd plist for HTTP bridge (port 8081, auto-start) - Add install-to-docker-ha.sh: deploy custom component to Docker HA via SSH - Add package-for-ha.sh: create distributable tarball of custom component - Add test-services.sh: comprehensive voice pipeline service checker Fixes from code review: - Use OpenClawAgent (HTTP) in async_setup_entry instead of OpenClawCLIAgent (CLI agent fails inside Docker HA where openclaw binary doesn't exist) - Update all port references from 8080 to 8081 (HTTP bridge port) - Remove overly permissive CORS headers from HTTP bridge - Fix zombie process leak: kill child process on CLI timeout - Remove unused subprocess import in conversation.py - Add version field to Kokoro TTS Wyoming info - Update TODO.md with voice pipeline progress
This commit is contained in:
Aodhan Collins
420
homeai-voice/TROUBLESHOOTING.md
Normal file
420
homeai-voice/TROUBLESHOOTING.md
Normal file
@@ -0,0 +1,420 @@
|
||||
# Voice Pipeline Troubleshooting Guide
|
||||
|
||||
> Common issues and solutions for the voice pipeline setup
|
||||
|
||||
---
|
||||
|
||||
## Network Configuration
|
||||
|
||||
**Important**: The services are split across two machines:
|
||||
|
||||
| Service | Machine | IP Address |
|
||||
|---------|---------|------------|
|
||||
| OpenClaw Gateway | Mac Mini | 10.0.0.101 |
|
||||
| Wyoming STT | Mac Mini | 10.0.0.101 |
|
||||
| Wyoming TTS | Mac Mini | 10.0.0.101 |
|
||||
| Wyoming Satellite | Mac Mini | 10.0.0.101 |
|
||||
| Ollama | Mac Mini | 10.0.0.101 |
|
||||
| Home Assistant | Server (Docker) | 10.0.0.199 |
|
||||
|
||||
---
|
||||
|
||||
## Issue: OpenClaw Conversation Cannot Connect
|
||||
|
||||
### Symptoms
|
||||
- Integration installed but shows connection error
|
||||
- HA logs show timeout or connection refused
|
||||
- Error: "Cannot connect to OpenClaw service"
|
||||
|
||||
### Root Cause
|
||||
The OpenClaw Conversation integration is configured with the wrong host IP. It needs to point to the Mac Mini (10.0.0.101), not the HA server (10.0.0.199).
|
||||
|
||||
### Solution
|
||||
|
||||
1. **Open Home Assistant UI** at http://10.0.0.199:8123
|
||||
|
||||
2. **Go to Settings → Devices & Services**
|
||||
|
||||
3. **Find "OpenClaw Conversation"** integration
|
||||
|
||||
4. **Click "Configure"** (or delete and re-add)
|
||||
|
||||
5. **Set the correct configuration:**
|
||||
- **OpenClaw Host**: `10.0.0.101` (Mac Mini IP, NOT 10.0.0.199)
|
||||
- **OpenClaw Port**: `8080`
|
||||
- **Agent Name**: `main`
|
||||
- **Timeout**: `30`
|
||||
|
||||
6. **Save** and verify connection
|
||||
|
||||
### Verify Network Connectivity
|
||||
|
||||
From the HA server, test if it can reach OpenClaw:
|
||||
|
||||
```bash
|
||||
# SSH to HA server
|
||||
ssh 10.0.0.199
|
||||
|
||||
# Test OpenClaw connectivity
|
||||
curl http://10.0.0.101:8080/status
|
||||
|
||||
# Or use nc
|
||||
nc -z 10.0.0.101 8080 && echo "OpenClaw reachable" || echo "Cannot reach OpenClaw"
|
||||
```
|
||||
|
||||
From the Mac Mini, verify OpenClaw is listening:
|
||||
|
||||
```bash
|
||||
# Check OpenClaw is running
|
||||
launchctl list | grep openclaw
|
||||
|
||||
# Check it's listening on all interfaces
|
||||
lsof -i :8080
|
||||
|
||||
# Test locally
|
||||
curl http://localhost:8080/status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issue: Wyoming Services Cannot Connect
|
||||
|
||||
### Symptoms
|
||||
- Wyoming integrations show as unavailable
|
||||
- HA cannot reach STT/TTS services
|
||||
- Timeout errors in HA logs
|
||||
|
||||
### Solution
|
||||
|
||||
Wyoming services are also on the Mac Mini (10.0.0.101):
|
||||
|
||||
1. **Go to Settings → Devices & Services**
|
||||
|
||||
2. **For each Wyoming integration**, verify the host is set to **10.0.0.101**:
|
||||
- Wyoming STT: `10.0.0.101:10300`
|
||||
- Wyoming TTS: `10.0.0.101:10301`
|
||||
- Wyoming Satellite: `10.0.0.101:10700`
|
||||
|
||||
3. **Test connectivity from HA server:**
|
||||
|
||||
```bash
|
||||
ssh 10.0.0.199
|
||||
nc -z 10.0.0.101 10300 # STT
|
||||
nc -z 10.0.0.101 10301 # TTS
|
||||
nc -z 10.0.0.101 10700 # Satellite
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issue: Firewall Blocking Connections
|
||||
|
||||
### Symptoms
|
||||
- Services work locally on Mac Mini
|
||||
- Cannot connect from HA server
|
||||
- Connection timeout errors
|
||||
|
||||
### Solution
|
||||
|
||||
Check Mac Mini firewall settings:
|
||||
|
||||
```bash
|
||||
# Check firewall status
|
||||
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --getglobalstate
|
||||
|
||||
# If enabled, add exceptions for the services
|
||||
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /opt/homebrew/bin/ollama
|
||||
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/local/bin/openclaw
|
||||
|
||||
# Or temporarily disable for testing (not recommended for production)
|
||||
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --setglobalstate off
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issue: OpenClaw CLI Not Found in HA Container
|
||||
|
||||
### Symptoms
|
||||
- Integration uses CLI fallback
|
||||
- Error: "OpenClaw CLI not found"
|
||||
- Component works but responses fail
|
||||
|
||||
### Root Cause
|
||||
The `openclaw` command is not available inside the HA Docker container. The integration should use the HTTP API, not CLI.
|
||||
|
||||
### Solution
|
||||
|
||||
The OpenClawCLIAgent is a fallback. Ensure the integration is using the HTTP API:
|
||||
|
||||
1. Check the integration configuration uses the correct host/port
|
||||
2. Verify OpenClaw Gateway is accessible via HTTP
|
||||
3. The component will automatically use HTTP if available
|
||||
|
||||
---
|
||||
|
||||
## Issue: Voice Pipeline Not Responding
|
||||
|
||||
### Symptoms
|
||||
- Wake word detected but no response
|
||||
- Audio captured but not transcribed
|
||||
- Transcription works but no TTS output
|
||||
|
||||
### Debugging Steps
|
||||
|
||||
1. **Check all services are running:**
|
||||
|
||||
```bash
|
||||
# On Mac Mini
|
||||
./homeai-voice/scripts/test-services.sh
|
||||
```
|
||||
|
||||
2. **Test each component individually:**
|
||||
|
||||
```bash
|
||||
# Test wake word detection
|
||||
# Say "Hey Jarvis" and check satellite logs
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
|
||||
# Test STT
|
||||
# Check if audio is being transcribed
|
||||
tail -f /tmp/homeai-wyoming-stt.log
|
||||
|
||||
# Test OpenClaw
|
||||
openclaw agent --message "Hello" --agent main
|
||||
|
||||
# Test TTS
|
||||
tail -f /tmp/homeai-wyoming-tts.log
|
||||
```
|
||||
|
||||
3. **Check HA Voice Pipeline configuration:**
|
||||
- Settings → Voice Assistants
|
||||
- Verify pipeline uses correct STT, Conversation, and TTS
|
||||
- Ensure OpenClaw Conversation is selected
|
||||
|
||||
4. **Test from HA Assist:**
|
||||
- Type a query in HA Assist panel
|
||||
- Check if you get a response
|
||||
- This bypasses wake word and audio capture
|
||||
|
||||
---
|
||||
|
||||
## Monitoring Wake Word Detection
|
||||
|
||||
To see when the wake word ("Hey Jarvis") is being detected in real-time:
|
||||
|
||||
### Option 1: Watch Satellite Logs (Recommended)
|
||||
|
||||
The Wyoming Satellite handles wake word detection and audio streaming:
|
||||
|
||||
```bash
|
||||
# Terminal 1: Watch satellite logs
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
```
|
||||
|
||||
**What to look for:**
|
||||
- `Wake word detected` - Wake word was heard
|
||||
- `Streaming audio` - Audio being sent to STT
|
||||
- `Connected to server` - Connection status
|
||||
|
||||
### Option 2: Watch Wake Word Service Logs
|
||||
|
||||
```bash
|
||||
# Terminal 1: Watch wake word detection logs
|
||||
tail -f /tmp/homeai-wakeword.log
|
||||
```
|
||||
|
||||
### Option 3: Watch All Voice Pipeline Logs
|
||||
|
||||
```bash
|
||||
# Terminal 1: Watch all voice-related logs
|
||||
tail -f /tmp/homeai-*.log | grep -E "(wake|satellite|stt|tts|openclaw)"
|
||||
```
|
||||
|
||||
### Test Wake Word Detection
|
||||
|
||||
While watching the logs, try this:
|
||||
|
||||
1. **Say clearly**: "Hey Jarvis" (or your configured wake word)
|
||||
2. **Wait** for the acknowledgment beep
|
||||
3. **Speak your command**: "What time is it?"
|
||||
4. **Check logs** for activity
|
||||
|
||||
### Expected Log Output
|
||||
|
||||
When wake word is detected, you should see:
|
||||
|
||||
```
|
||||
[wyoming_satellite] Wake word detected
|
||||
[wyoming_satellite] Streaming audio to stt
|
||||
[wyoming_satellite] Connected to 10.0.0.101:10300
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issue: Audio Playback Not Working
|
||||
|
||||
### Symptoms
|
||||
- Pipeline works but no audio output
|
||||
- TTS generates audio but satellite doesn't play it
|
||||
- Silent responses
|
||||
|
||||
### Solution
|
||||
|
||||
1. **Check audio output device:**
|
||||
|
||||
```bash
|
||||
# On Mac Mini
|
||||
afplay /System/Library/Sounds/Glass.aiff
|
||||
```
|
||||
|
||||
2. **Check satellite configuration:**
|
||||
|
||||
```bash
|
||||
# View satellite config
|
||||
cat ~/Library/LaunchAgents/com.homeai.wyoming-satellite.plist
|
||||
|
||||
# Check logs for audio errors
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
```
|
||||
|
||||
3. **Verify SoX is installed:**
|
||||
|
||||
```bash
|
||||
which play
|
||||
brew install sox
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issue: High Latency (>5 seconds)
|
||||
|
||||
### Symptoms
|
||||
- Long delay between wake word and response
|
||||
- Slow transcription or TTS generation
|
||||
|
||||
### Solutions
|
||||
|
||||
1. **Check network latency:**
|
||||
|
||||
```bash
|
||||
# From HA server to Mac Mini
|
||||
ping 10.0.0.101
|
||||
```
|
||||
|
||||
2. **Check Ollama model size:**
|
||||
|
||||
```bash
|
||||
# Smaller models are faster
|
||||
ollama list
|
||||
|
||||
# Switch to faster model in OpenClaw config
|
||||
# qwen2.5:7b is faster than llama3.3:70b
|
||||
```
|
||||
|
||||
3. **Check system resources:**
|
||||
|
||||
```bash
|
||||
# On Mac Mini
|
||||
top -l 1 | grep -E "CPU|PhysMem"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Correct Configuration Summary
|
||||
|
||||
### OpenClaw Conversation Integration
|
||||
- Host: `10.0.0.101` (Mac Mini)
|
||||
- Port: `8080`
|
||||
- Agent: `main`
|
||||
- Timeout: `30`
|
||||
|
||||
### Wyoming STT Integration
|
||||
- Host: `10.0.0.101` (Mac Mini)
|
||||
- Port: `10300`
|
||||
|
||||
### Wyoming TTS Integration
|
||||
- Host: `10.0.0.101` (Mac Mini)
|
||||
- Port: `10301`
|
||||
|
||||
### Wyoming Satellite Integration
|
||||
- Host: `10.0.0.101` (Mac Mini)
|
||||
- Port: `10700`
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [ ] All services running on Mac Mini (10.0.0.101)
|
||||
- [ ] HA can ping Mac Mini: `ping 10.0.0.101`
|
||||
- [ ] HA can reach OpenClaw: `curl http://10.0.0.101:8080/status`
|
||||
- [ ] HA can reach Wyoming STT: `nc -z 10.0.0.101 10300`
|
||||
- [ ] HA can reach Wyoming TTS: `nc -z 10.0.0.101 10301`
|
||||
- [ ] HA can reach Wyoming Satellite: `nc -z 10.0.0.101 10700`
|
||||
- [ ] OpenClaw Conversation integration configured with 10.0.0.101
|
||||
- [ ] Wyoming integrations configured with 10.0.0.101
|
||||
- [ ] Voice pipeline created and set as default
|
||||
- [ ] Test query in HA Assist returns response
|
||||
|
||||
---
|
||||
|
||||
## Bugs Fixed During Setup
|
||||
|
||||
The following bugs were discovered and fixed during initial setup (2026-03-08):
|
||||
|
||||
### 1. OpenClaw Network Binding
|
||||
|
||||
**Problem**: OpenClaw gateway was only listening on localhost (127.0.0.1), not accessible from HA server.
|
||||
|
||||
**Fix**: Added `"bind": "lan"` to `~/.openclaw/openclaw.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"gateway": {
|
||||
"port": 8080,
|
||||
"mode": "local",
|
||||
"bind": "lan",
|
||||
"auth": { "token": "..." }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Custom Component API Error
|
||||
|
||||
**Problem**: `async_set_agent()` was being called with `DOMAIN` (string) instead of `entry` (ConfigEntry object).
|
||||
|
||||
**Fix**: Changed parameter in `homeai-agent/custom_components/openclaw_conversation/__init__.py`:
|
||||
|
||||
```python
|
||||
# Line 83
|
||||
conversation.async_set_agent(hass, entry, agent) # Was: DOMAIN
|
||||
|
||||
# Line 94
|
||||
conversation.async_unset_agent(hass, entry) # Was: DOMAIN
|
||||
```
|
||||
|
||||
### 3. TTS Server Missing Version
|
||||
|
||||
**Problem**: `TtsProgram` initialization was missing required `version` parameter.
|
||||
|
||||
**Fix**: Added `version="1.0.0"` in `homeai-voice/tts/wyoming_kokoro_server.py` line 58.
|
||||
|
||||
### 4. Voice Commands Not Working (CLI Not in Docker)
|
||||
|
||||
**Problem**: HA Docker container couldn't access `openclaw` CLI.
|
||||
|
||||
**Fix**: Created OpenClaw HTTP Bridge (`homeai-agent/openclaw-http-bridge.py`) on port 8081 that translates HTTP POST requests to OpenClaw CLI calls. The custom component now uses port 8081 (HTTP bridge) instead of 8080 (gateway).
|
||||
|
||||
---
|
||||
|
||||
## Getting Help
|
||||
|
||||
If issues persist:
|
||||
|
||||
1. **Check service logs:**
|
||||
- Mac Mini: `/tmp/homeai-*.log`
|
||||
- HA: Settings → System → Logs
|
||||
|
||||
2. **Verify network connectivity** between machines
|
||||
|
||||
3. **Test each component** individually before testing the full pipeline
|
||||
|
||||
4. **Review configuration** in [`VOICE_PIPELINE_SETUP.md`](VOICE_PIPELINE_SETUP.md)
|
||||
435
homeai-voice/VOICE_PIPELINE_SETUP.md
Normal file
435
homeai-voice/VOICE_PIPELINE_SETUP.md
Normal file
@@ -0,0 +1,435 @@
|
||||
# Voice Pipeline Setup Guide
|
||||
|
||||
> Complete guide to setting up the end-to-end voice pipeline with OpenClaw integration
|
||||
|
||||
---
|
||||
|
||||
## Network Configuration
|
||||
|
||||
**Important**: Services are split across two machines:
|
||||
|
||||
| Service | Port | Location |
|
||||
|---------|------|----------|
|
||||
| Wyoming STT (Whisper large-v3) | 10300 | Mac Mini (10.0.0.101) |
|
||||
| Wyoming TTS (Kokoro ONNX) | 10301 | Mac Mini (10.0.0.101) |
|
||||
| Wyoming Satellite | 10700 | Mac Mini (10.0.0.101) |
|
||||
| openWakeWord | - | Mac Mini (10.0.0.101) |
|
||||
| OpenClaw Gateway | 8080 | Mac Mini (10.0.0.101) |
|
||||
| Ollama | 11434 | Mac Mini (10.0.0.101) |
|
||||
| Home Assistant (Docker) | 8123 | Server (10.0.0.199) |
|
||||
|
||||
**All integrations must point to 10.0.0.101 (Mac Mini), not 10.0.0.199 (HA server).**
|
||||
|
||||
---
|
||||
|
||||
## Current Status
|
||||
|
||||
### ✅ Services Running on Mac Mini (10.0.0.101)
|
||||
|
||||
| Service | Port | Status |
|
||||
|---------|------|--------|
|
||||
| Wyoming STT | 10300 | ✅ Running |
|
||||
| Wyoming TTS | 10301 | ✅ Running |
|
||||
| Wyoming Satellite | 10700 | ✅ Running |
|
||||
| openWakeWord | - | ✅ Running |
|
||||
| OpenClaw Gateway | 8080 | ✅ Running |
|
||||
| Ollama | 11434 | ✅ Running |
|
||||
|
||||
### ✅ Completed
|
||||
- Wyoming STT/TTS services installed and running
|
||||
- Wyoming Satellite installed and running
|
||||
- OpenClaw agent configured with home-assistant skill
|
||||
- Custom OpenClaw conversation component created
|
||||
|
||||
### 🔄 Next Steps
|
||||
1. Install OpenClaw conversation component in Home Assistant
|
||||
2. Configure Wyoming integrations in HA
|
||||
3. Create voice assistant pipeline with OpenClaw
|
||||
4. Test the full voice loop
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Install OpenClaw Conversation Component
|
||||
|
||||
Home Assistant is running in Docker on server 10.0.0.199. Use the automated installation script.
|
||||
|
||||
### Option A: Automated Installation (Recommended)
|
||||
|
||||
```bash
|
||||
# From Mac Mini, run the installation script
|
||||
cd ~/gitea/homeai/homeai-agent/custom_components
|
||||
./install-to-docker-ha.sh
|
||||
|
||||
# The script will:
|
||||
# 1. Create a tarball of the component
|
||||
# 2. Copy it to the HA server via SCP
|
||||
# 3. Extract it into the HA Docker container
|
||||
# 4. Provide next steps
|
||||
```
|
||||
|
||||
**Requirements:**
|
||||
- SSH access to 10.0.0.199
|
||||
- SSH keys configured (or password access)
|
||||
|
||||
### Option B: Manual Installation via SSH
|
||||
|
||||
```bash
|
||||
# 1. Create tarball
|
||||
cd ~/gitea/homeai/homeai-agent/custom_components
|
||||
tar -czf openclaw_conversation.tar.gz openclaw_conversation/
|
||||
|
||||
# 2. Copy to HA server
|
||||
scp openclaw_conversation.tar.gz 10.0.0.199:/tmp/
|
||||
|
||||
# 3. SSH to HA server and install
|
||||
ssh 10.0.0.199
|
||||
CONTAINER=$(docker ps --filter "name=homeassistant" --format "{{.Names}}" | head -n 1)
|
||||
docker cp /tmp/openclaw_conversation.tar.gz $CONTAINER:/tmp/
|
||||
docker exec $CONTAINER sh -c 'cd /config/custom_components && tar -xzf /tmp/openclaw_conversation.tar.gz'
|
||||
docker restart $CONTAINER
|
||||
```
|
||||
|
||||
### Option D: Using Home Assistant File Editor (Manual)
|
||||
|
||||
1. Open Home Assistant UI at http://10.0.0.199:8123
|
||||
2. Install the **File Editor** add-on if not already installed
|
||||
3. Create directory: `/config/custom_components/openclaw_conversation/`
|
||||
4. Copy each file from `homeai-agent/custom_components/openclaw_conversation/`:
|
||||
- `__init__.py`
|
||||
- `config_flow.py`
|
||||
- `const.py`
|
||||
- `conversation.py`
|
||||
- `manifest.json`
|
||||
- `strings.json`
|
||||
|
||||
### Verify Installation
|
||||
|
||||
After installation, restart Home Assistant:
|
||||
|
||||
```bash
|
||||
# Via SSH
|
||||
ssh 10.0.0.199 'docker restart homeassistant'
|
||||
|
||||
# Or via HA UI
|
||||
# Settings → System → Restart
|
||||
```
|
||||
|
||||
Check logs for any errors:
|
||||
- **Settings → System → Logs**
|
||||
- Look for "OpenClaw Conversation" in the logs
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Configure Wyoming Integrations
|
||||
|
||||
### Add Wyoming STT (Speech-to-Text)
|
||||
|
||||
1. Go to **Settings → Devices & Services → Add Integration**
|
||||
2. Search for **"Wyoming Protocol"**
|
||||
3. Configure:
|
||||
- **Host**: `10.0.0.101` ⚠️ **Mac Mini IP, not HA server IP (10.0.0.199)**
|
||||
- **Port**: `10300`
|
||||
- **Name**: `Mac Mini STT`
|
||||
4. Click **Submit**
|
||||
|
||||
### Add Wyoming TTS (Text-to-Speech)
|
||||
|
||||
1. Click **Add Integration** again
|
||||
2. Search for **"Wyoming Protocol"**
|
||||
3. Configure:
|
||||
- **Host**: `10.0.0.101` ⚠️ **Mac Mini IP**
|
||||
- **Port**: `10301`
|
||||
- **Name**: `Mac Mini TTS`
|
||||
4. Click **Submit**
|
||||
|
||||
### Add Wyoming Satellite
|
||||
|
||||
1. Click **Add Integration** again
|
||||
2. Search for **"Wyoming Protocol"**
|
||||
3. Configure:
|
||||
- **Host**: `10.0.0.101` ⚠️ **Mac Mini IP**
|
||||
- **Port**: `10700`
|
||||
- **Name**: `Mac Mini Living Room`
|
||||
4. Click **Submit**
|
||||
|
||||
### Verify Integrations
|
||||
|
||||
All three Wyoming integrations should appear in **Settings → Devices & Services**.
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Add OpenClaw Conversation Agent
|
||||
|
||||
### Via UI (Recommended)
|
||||
|
||||
1. Go to **Settings → Devices & Services → Add Integration**
|
||||
2. Search for **"OpenClaw Conversation"**
|
||||
3. Configure:
|
||||
- **OpenClaw Host**: `10.0.0.101` ⚠️ **Mac Mini IP, not HA server IP (10.0.0.199)**
|
||||
- **OpenClaw Port**: `8080`
|
||||
- **Agent Name**: `main`
|
||||
- **Timeout**: `30` seconds
|
||||
4. Click **Submit**
|
||||
|
||||
### Via YAML (Alternative)
|
||||
|
||||
Add to `/config/configuration.yaml`:
|
||||
|
||||
```yaml
|
||||
openclaw_conversation:
|
||||
openclaw_host: 10.0.0.101 # Mac Mini IP
|
||||
openclaw_port: 8080
|
||||
agent_name: main
|
||||
timeout: 30
|
||||
```
|
||||
|
||||
Then restart Home Assistant.
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Create Voice Assistant Pipeline
|
||||
|
||||
1. Go to **Settings → Voice Assistants**
|
||||
2. Click **Add Assistant**
|
||||
3. Configure:
|
||||
- **Name**: `HomeAI with OpenClaw`
|
||||
- **Language**: `English`
|
||||
- **Speech-to-Text**: Select `Mac Mini STT` (Wyoming)
|
||||
- **Conversation Agent**: Select `OpenClaw Conversation`
|
||||
- **Text-to-Speech**: Select `Mac Mini TTS` (Wyoming)
|
||||
4. Click **Create**
|
||||
|
||||
### Set as Default
|
||||
|
||||
1. In **Settings → Voice Assistants**
|
||||
2. Click the three dots next to "HomeAI with OpenClaw"
|
||||
3. Select **Set as preferred**
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Test the Pipeline
|
||||
|
||||
### Test 1: Text Input → TTS Output
|
||||
|
||||
1. Open Home Assistant UI
|
||||
2. Click the **Assist** icon (microphone) in the top-right corner
|
||||
3. Type: `"What time is it?"`
|
||||
4. Press Enter
|
||||
|
||||
**Expected Result**: You should hear a spoken response via Kokoro TTS
|
||||
|
||||
### Test 2: Voice Input → OpenClaw → TTS Output
|
||||
|
||||
1. Ensure Wyoming Satellite is running on Mac Mini:
|
||||
```bash
|
||||
launchctl list | grep wyoming-satellite
|
||||
```
|
||||
|
||||
2. Say the wake word: **"Hey Jarvis"**
|
||||
3. Wait for the beep/acknowledgment
|
||||
4. Speak: **"What time is it?"**
|
||||
|
||||
**Expected Result**: You should hear a spoken response
|
||||
|
||||
### Test 3: Home Assistant Control via Voice
|
||||
|
||||
1. Say: **"Hey Jarvis"**
|
||||
2. Speak: **"Turn on the reading lamp"**
|
||||
|
||||
**Expected Result**:
|
||||
- OpenClaw processes the request
|
||||
- Home Assistant skill executes the action
|
||||
- Light turns on
|
||||
- You hear a confirmation via TTS
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: OpenClaw Conversation not appearing in integrations
|
||||
|
||||
**Solution**:
|
||||
1. Verify files are in `/config/custom_components/openclaw_conversation/`
|
||||
2. Check Home Assistant logs for errors
|
||||
3. Ensure `manifest.json` is valid JSON
|
||||
4. Restart Home Assistant
|
||||
|
||||
### Issue: Wyoming services not connecting
|
||||
|
||||
**Solution**:
|
||||
1. Verify services are running on Mac Mini:
|
||||
```bash
|
||||
launchctl list | grep wyoming
|
||||
nc -z 10.0.0.199 10300 # Test STT
|
||||
nc -z 10.0.0.199 10301 # Test TTS
|
||||
nc -z 10.0.0.199 10700 # Test Satellite
|
||||
```
|
||||
|
||||
2. Check firewall rules on Mac Mini
|
||||
3. Verify Home Assistant can reach Mac Mini network
|
||||
|
||||
### Issue: OpenClaw not responding
|
||||
|
||||
**Solution**:
|
||||
1. Verify OpenClaw is running:
|
||||
```bash
|
||||
launchctl list | grep openclaw
|
||||
pgrep -f openclaw
|
||||
```
|
||||
|
||||
2. Test OpenClaw CLI directly:
|
||||
```bash
|
||||
openclaw agent --message "Hello" --agent main
|
||||
```
|
||||
|
||||
3. Check OpenClaw logs:
|
||||
```bash
|
||||
tail -f /tmp/homeai-openclaw.log
|
||||
```
|
||||
|
||||
4. Verify OpenClaw can reach Home Assistant:
|
||||
```bash
|
||||
curl http://10.0.0.199:8123/api/
|
||||
```
|
||||
|
||||
### Issue: No audio output from satellite
|
||||
|
||||
**Solution**:
|
||||
1. Check satellite logs:
|
||||
```bash
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
```
|
||||
|
||||
2. Test audio output:
|
||||
```bash
|
||||
afplay /System/Library/Sounds/Glass.aiff
|
||||
```
|
||||
|
||||
3. Verify SoX is installed:
|
||||
```bash
|
||||
which play
|
||||
brew install sox
|
||||
```
|
||||
|
||||
### Issue: Wake word not detected
|
||||
|
||||
**Solution**:
|
||||
1. Check wakeword service:
|
||||
```bash
|
||||
launchctl list | grep wakeword
|
||||
```
|
||||
|
||||
2. Test microphone input:
|
||||
```bash
|
||||
# Record a test
|
||||
rec -r 16000 -c 1 test.wav trim 0 5
|
||||
```
|
||||
|
||||
3. Adjust wake word threshold in satellite config
|
||||
|
||||
---
|
||||
|
||||
## Voice Pipeline Flow
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ USB Mic │
|
||||
│ (Mac Mini) │
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Wake Word │
|
||||
│ Detection │
|
||||
│ (hey_jarvis) │
|
||||
└────────┬────────┘
|
||||
│ wake detected
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Wyoming │
|
||||
│ Satellite │
|
||||
│ :10700 │
|
||||
└────────┬────────┘
|
||||
│ audio stream
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Wyoming STT │
|
||||
│ (Whisper) │
|
||||
│ :10300 │
|
||||
└────────┬────────┘
|
||||
│ transcript
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Home Assistant │
|
||||
│ Voice Pipeline │
|
||||
└────────┬────────┘
|
||||
│ text
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ OpenClaw │
|
||||
│ Conversation │
|
||||
│ Agent │
|
||||
└────────┬────────┘
|
||||
│ message
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ OpenClaw │
|
||||
│ Gateway │
|
||||
│ :8080 │
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Ollama LLM │
|
||||
│ + Skills │
|
||||
│ :11434 │
|
||||
└────────┬────────┘
|
||||
│ response
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Wyoming TTS │
|
||||
│ (Kokoro) │
|
||||
│ :10301 │
|
||||
└────────┬────────┘
|
||||
│ audio
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Speaker │
|
||||
│ (Mac Mini) │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps After Setup
|
||||
|
||||
1. **Install Chatterbox TTS** for voice cloning
|
||||
2. **Set up mem0** for long-term memory
|
||||
3. **Configure n8n workflows** for automation
|
||||
4. **Add Uptime Kuma monitors** for all services
|
||||
5. **Begin ESP32 satellite setup** (Phase 4)
|
||||
|
||||
---
|
||||
|
||||
## Files Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [`homeai-agent/custom_components/openclaw_conversation/`](../homeai-agent/custom_components/openclaw_conversation/) | Custom HA component |
|
||||
| [`homeai-agent/skills/home-assistant/openclaw_bridge.py`](../homeai-agent/skills/home-assistant/openclaw_bridge.py) | Bridge script |
|
||||
| [`homeai-voice/scripts/launchd/`](scripts/launchd/) | Service plists |
|
||||
| [`plans/ha-voice-pipeline-implementation.md`](../plans/ha-voice-pipeline-implementation.md) | Detailed implementation plan |
|
||||
| [`plans/voice-loop-integration.md`](../plans/voice-loop-integration.md) | Architecture options |
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Wyoming STT/TTS/Satellite appear in HA integrations
|
||||
- [ ] OpenClaw Conversation agent appears in HA integrations
|
||||
- [ ] Voice assistant pipeline created with OpenClaw
|
||||
- [ ] Typed query in Assist returns spoken response
|
||||
- [ ] Voice query via satellite returns spoken response
|
||||
- [ ] "Turn on the reading lamp" command works end-to-end
|
||||
- [ ] Latency under 5 seconds from wake to response
|
||||
- [ ] All services survive Mac Mini reboot
|
||||
195
homeai-voice/WYOMING_SATELLITE_SETUP.md
Normal file
195
homeai-voice/WYOMING_SATELLITE_SETUP.md
Normal file
@@ -0,0 +1,195 @@
|
||||
# Wyoming Satellite Setup Guide
|
||||
|
||||
> How to configure the Wyoming Satellite wizard in Home Assistant
|
||||
|
||||
---
|
||||
|
||||
## When Adding Wyoming Satellite Integration
|
||||
|
||||
When you add the Wyoming Satellite integration, Home Assistant will open a wizard to configure a voice assistant. Here's what to do:
|
||||
|
||||
---
|
||||
|
||||
## Option 1: Skip Wizard and Configure Later (Recommended)
|
||||
|
||||
**Best approach if you haven't created the OpenClaw pipeline yet:**
|
||||
|
||||
1. **Skip/Cancel the wizard** - just add the satellite integration without configuring the pipeline
|
||||
2. The satellite will be added but not assigned to a pipeline yet
|
||||
3. Continue with creating the voice assistant pipeline (see below)
|
||||
4. Come back and assign the satellite to the pipeline later
|
||||
|
||||
---
|
||||
|
||||
## Option 2: Use Default Pipeline Temporarily
|
||||
|
||||
**If you want to test the satellite immediately:**
|
||||
|
||||
1. In the wizard, select **"Home Assistant"** as the pipeline (default)
|
||||
2. This will use HA's built-in conversation agent (not OpenClaw)
|
||||
3. You can test basic commands like "What time is it?"
|
||||
4. Later, switch to the OpenClaw pipeline once it's created
|
||||
|
||||
---
|
||||
|
||||
## Creating the Voice Assistant Pipeline
|
||||
|
||||
**Do this BEFORE configuring the satellite (or after if you used Option 2):**
|
||||
|
||||
### Step 1: Create the Pipeline
|
||||
|
||||
1. Go to **Settings → Voice Assistants**
|
||||
2. Click **Add Assistant**
|
||||
3. Configure:
|
||||
- **Name**: `HomeAI with OpenClaw`
|
||||
- **Language**: `English`
|
||||
- **Speech-to-Text**: Select `Mac Mini STT` (Wyoming)
|
||||
- **Conversation Agent**: Select `OpenClaw Conversation`
|
||||
- **Text-to-Speech**: Select `Mac Mini TTS` (Wyoming)
|
||||
4. Click **Create**
|
||||
|
||||
### Step 2: Set as Preferred (Optional)
|
||||
|
||||
1. In the Voice Assistants list, find "HomeAI with OpenClaw"
|
||||
2. Click the three dots (⋮)
|
||||
3. Select **Set as preferred**
|
||||
|
||||
This makes it the default pipeline for all new satellites.
|
||||
|
||||
---
|
||||
|
||||
## Assigning Satellite to Pipeline
|
||||
|
||||
### If You Skipped the Wizard
|
||||
|
||||
1. Go to **Settings → Devices & Services**
|
||||
2. Find **Wyoming Protocol** (the satellite entry)
|
||||
3. Click **Configure**
|
||||
4. Select **Pipeline**: `HomeAI with OpenClaw`
|
||||
5. Click **Submit**
|
||||
|
||||
### If You Used the Default Pipeline
|
||||
|
||||
1. Go to **Settings → Devices & Services**
|
||||
2. Find **Wyoming Protocol** (the satellite entry)
|
||||
3. Click **Configure**
|
||||
4. Change **Pipeline** from "Home Assistant" to `HomeAI with OpenClaw`
|
||||
5. Click **Submit**
|
||||
|
||||
---
|
||||
|
||||
## Satellite Configuration Details
|
||||
|
||||
The wizard may ask for these details:
|
||||
|
||||
| Field | Value | Notes |
|
||||
|-------|-------|-------|
|
||||
| **Name** | `Mac Mini Living Room` | Or any name you prefer |
|
||||
| **Pipeline** | `HomeAI with OpenClaw` | Select after creating it |
|
||||
| **Wake Word** | `hey_jarvis` | Should be auto-detected |
|
||||
| **Audio Input** | Default | Detected from satellite |
|
||||
| **Audio Output** | Default | Detected from satellite |
|
||||
|
||||
---
|
||||
|
||||
## Complete Voice Pipeline Flow
|
||||
|
||||
Once configured, the flow will be:
|
||||
|
||||
```
|
||||
1. Say "Hey Jarvis" → Wake word detected by satellite
|
||||
2. Satellite captures audio → Sends to Wyoming STT (10.0.0.101:10300)
|
||||
3. STT transcribes → Sends text to HA Voice Pipeline
|
||||
4. HA routes to OpenClaw Conversation agent
|
||||
5. OpenClaw processes → Calls Ollama LLM + skills
|
||||
6. Response generated → Sent to Wyoming TTS (10.0.0.101:10301)
|
||||
7. TTS generates audio → Sent back to satellite
|
||||
8. Satellite plays audio → You hear the response
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing the Pipeline
|
||||
|
||||
### Test 1: Via HA Assist (No Wake Word)
|
||||
|
||||
1. Open Home Assistant UI
|
||||
2. Click the **Assist** icon (microphone) in top-right
|
||||
3. Type: `"What time is it?"`
|
||||
4. Press Enter
|
||||
5. **Expected**: You should hear a spoken response via TTS
|
||||
|
||||
### Test 2: Via Satellite (With Wake Word)
|
||||
|
||||
1. Say: **"Hey Jarvis"**
|
||||
2. Wait for acknowledgment beep
|
||||
3. Say: **"What time is it?"**
|
||||
4. **Expected**: You should hear a spoken response
|
||||
|
||||
### Test 3: Home Control
|
||||
|
||||
1. Say: **"Hey Jarvis"**
|
||||
2. Say: **"Turn on the reading lamp"**
|
||||
3. **Expected**:
|
||||
- Light turns on
|
||||
- You hear confirmation: "I've turned on the reading lamp"
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Satellite Not Responding
|
||||
|
||||
1. **Check satellite is online**:
|
||||
- Settings → Devices & Services → Wyoming Protocol
|
||||
- Should show "Connected"
|
||||
|
||||
2. **Check pipeline is assigned**:
|
||||
- Configure satellite → Verify pipeline is set
|
||||
|
||||
3. **Check satellite logs** on Mac Mini:
|
||||
```bash
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
```
|
||||
|
||||
### Wake Word Not Detected
|
||||
|
||||
1. **Check microphone**:
|
||||
- Satellite logs should show audio input
|
||||
- Try speaking louder or closer to mic
|
||||
|
||||
2. **Adjust wake word sensitivity**:
|
||||
- May need to configure threshold in satellite settings
|
||||
|
||||
### No Audio Output
|
||||
|
||||
1. **Check speaker**:
|
||||
```bash
|
||||
afplay /System/Library/Sounds/Glass.aiff
|
||||
```
|
||||
|
||||
2. **Check TTS is working**:
|
||||
- Test via HA Assist (type query)
|
||||
- Should hear response
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
**Recommended Setup Order:**
|
||||
|
||||
1. ✅ Add Wyoming STT integration (10.0.0.101:10300)
|
||||
2. ✅ Add Wyoming TTS integration (10.0.0.101:10301)
|
||||
3. ✅ Add OpenClaw Conversation integration (10.0.0.101:8080)
|
||||
4. ✅ Create voice assistant pipeline "HomeAI with OpenClaw"
|
||||
5. ✅ Add Wyoming Satellite integration (10.0.0.101:10700)
|
||||
6. ✅ Assign satellite to "HomeAI with OpenClaw" pipeline
|
||||
7. ✅ Test the complete voice loop
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [`VOICE_PIPELINE_SETUP.md`](VOICE_PIPELINE_SETUP.md) - Complete setup guide
|
||||
- [`TROUBLESHOOTING.md`](TROUBLESHOOTING.md) - Troubleshooting guide
|
||||
- [`OPENCLAW_NETWORK_FIX.md`](OPENCLAW_NETWORK_FIX.md) - Network access fix
|
||||
140
homeai-voice/scripts/test-services.sh
Executable file
140
homeai-voice/scripts/test-services.sh
Executable file
@@ -0,0 +1,140 @@
|
||||
#!/usr/bin/env bash
|
||||
# Test all voice pipeline services are running and accessible
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
echo "Testing Voice Pipeline Services..."
|
||||
echo "=================================="
|
||||
echo ""
|
||||
|
||||
# Colors for output
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Test function
|
||||
test_service() {
|
||||
local name=$1
|
||||
local host=$2
|
||||
local port=$3
|
||||
|
||||
if nc -z -w 2 "$host" "$port" 2>/dev/null; then
|
||||
echo -e "${GREEN}✓${NC} $name ($host:$port)"
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} $name ($host:$port) - NOT ACCESSIBLE"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Test launchd service
|
||||
test_launchd() {
|
||||
local name=$1
|
||||
local service=$2
|
||||
|
||||
if launchctl list | grep -q "$service"; then
|
||||
echo -e "${GREEN}✓${NC} $name (launchd: $service)"
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} $name (launchd: $service) - NOT RUNNING"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Test command availability
|
||||
test_command() {
|
||||
local name=$1
|
||||
local cmd=$2
|
||||
|
||||
if command -v "$cmd" &> /dev/null; then
|
||||
echo -e "${GREEN}✓${NC} $name command available"
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} $name command NOT FOUND"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
echo "1. Network Services"
|
||||
echo "-------------------"
|
||||
test_service "Wyoming STT" "localhost" "10300"
|
||||
test_service "Wyoming TTS" "localhost" "10301"
|
||||
test_service "Wyoming Satellite" "localhost" "10700"
|
||||
test_service "OpenClaw Gateway" "localhost" "8080"
|
||||
test_service "Ollama" "localhost" "11434"
|
||||
test_service "Home Assistant" "10.0.0.199" "8123"
|
||||
echo ""
|
||||
|
||||
echo "2. Launchd Services"
|
||||
echo "-------------------"
|
||||
test_launchd "Wyoming STT" "com.homeai.wyoming-stt"
|
||||
test_launchd "Wyoming TTS" "com.homeai.wyoming-tts"
|
||||
test_launchd "Wyoming Satellite" "com.homeai.wyoming-satellite"
|
||||
test_launchd "Wake Word" "com.homeai.wakeword"
|
||||
test_launchd "OpenClaw" "com.homeai.openclaw"
|
||||
test_launchd "Ollama" "com.homeai.ollama"
|
||||
echo ""
|
||||
|
||||
echo "3. Commands"
|
||||
echo "-----------"
|
||||
test_command "OpenClaw" "openclaw"
|
||||
test_command "Ollama" "ollama"
|
||||
test_command "SoX (play)" "play"
|
||||
test_command "SoX (rec)" "rec"
|
||||
echo ""
|
||||
|
||||
echo "4. Wyoming Protocol Test"
|
||||
echo "------------------------"
|
||||
if command -v wyoming-client &> /dev/null; then
|
||||
echo -e "${YELLOW}Testing STT...${NC}"
|
||||
# Would need a test audio file
|
||||
echo " (Manual test required with audio file)"
|
||||
|
||||
echo -e "${YELLOW}Testing TTS...${NC}"
|
||||
# Would need Wyoming client
|
||||
echo " (Manual test required with Wyoming client)"
|
||||
else
|
||||
echo -e "${YELLOW}⚠${NC} wyoming-client not installed (optional)"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
echo "5. OpenClaw Test"
|
||||
echo "----------------"
|
||||
if command -v openclaw &> /dev/null; then
|
||||
echo -e "${YELLOW}Testing OpenClaw agent...${NC}"
|
||||
if timeout 10 openclaw agent --message "Hello" --agent main &>/dev/null; then
|
||||
echo -e "${GREEN}✓${NC} OpenClaw agent responding"
|
||||
else
|
||||
echo -e "${RED}✗${NC} OpenClaw agent not responding"
|
||||
fi
|
||||
else
|
||||
echo -e "${RED}✗${NC} OpenClaw command not found"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
echo "6. Audio Devices"
|
||||
echo "----------------"
|
||||
if command -v rec &> /dev/null; then
|
||||
echo "Input devices:"
|
||||
rec -n stat trim 0 0.1 2>&1 | grep -i "input" || echo " (Unable to detect)"
|
||||
|
||||
echo "Output devices:"
|
||||
if command -v afplay &> /dev/null; then
|
||||
echo -e "${GREEN}✓${NC} afplay available for audio output"
|
||||
else
|
||||
echo -e "${RED}✗${NC} afplay not available"
|
||||
fi
|
||||
else
|
||||
echo -e "${YELLOW}⚠${NC} SoX not installed - audio recording unavailable"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
echo "=================================="
|
||||
echo "Test complete!"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo "1. Install OpenClaw conversation component in Home Assistant"
|
||||
echo "2. Configure Wyoming integrations in HA UI"
|
||||
echo "3. Create voice assistant pipeline"
|
||||
echo "4. Test with: 'Hey Jarvis, what time is it?'"
|
||||
@@ -56,6 +56,7 @@ class KokoroEventHandler(AsyncEventHandler):
|
||||
url="https://github.com/thewh1teagle/kokoro-onnx",
|
||||
),
|
||||
installed=True,
|
||||
version="1.0.0",
|
||||
voices=[
|
||||
TtsVoice(
|
||||
name=self._default_voice,
|
||||
|
||||
Reference in New Issue
Block a user