aodhan/homeai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: OpenClaw HTTP bridge, HA conversation agent fixes, voice pipeline tooling

Browse Source 
- Add openclaw-http-bridge.py: HTTP server translating POST requests to OpenClaw CLI calls
- Add launchd plist for HTTP bridge (port 8081, auto-start)
- Add install-to-docker-ha.sh: deploy custom component to Docker HA via SSH
- Add package-for-ha.sh: create distributable tarball of custom component
- Add test-services.sh: comprehensive voice pipeline service checker

Fixes from code review:
- Use OpenClawAgent (HTTP) in async_setup_entry instead of OpenClawCLIAgent
  (CLI agent fails inside Docker HA where openclaw binary doesn't exist)
- Update all port references from 8080 to 8081 (HTTP bridge port)
- Remove overly permissive CORS headers from HTTP bridge
- Fix zombie process leak: kill child process on CLI timeout
- Remove unused subprocess import in conversation.py
- Add version field to Kokoro TTS Wyoming info
- Update TODO.md with voice pipeline progress
This commit is contained in:
Aodhan Collins
2026-03-08 22:46:04 +00:00
parent 6a0bae2a0b
commit 664bb6d275
16 changed files with 1901 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

435
homeai-voice/VOICE_PIPELINE_SETUP.md Normal file
View File

@@ -0,0 +1,435 @@
# Voice Pipeline Setup Guide
> Complete guide to setting up the end-to-end voice pipeline with OpenClaw integration
---
## Network Configuration
**Important**: Services are split across two machines:
| Service | Port | Location |
|---------|------|----------|
| Wyoming STT (Whisper large-v3) | 10300 | Mac Mini (10.0.0.101) |
| Wyoming TTS (Kokoro ONNX) | 10301 | Mac Mini (10.0.0.101) |
| Wyoming Satellite | 10700 | Mac Mini (10.0.0.101) |
| openWakeWord | - | Mac Mini (10.0.0.101) |
| OpenClaw Gateway | 8080 | Mac Mini (10.0.0.101) |
| Ollama | 11434 | Mac Mini (10.0.0.101) |
| Home Assistant (Docker) | 8123 | Server (10.0.0.199) |
**All integrations must point to 10.0.0.101 (Mac Mini), not 10.0.0.199 (HA server).**
---
## Current Status
### ✅ Services Running on Mac Mini (10.0.0.101)
| Service | Port | Status |
|---------|------|--------|
| Wyoming STT | 10300 | ✅ Running |
| Wyoming TTS | 10301 | ✅ Running |
| Wyoming Satellite | 10700 | ✅ Running |
| openWakeWord | - | ✅ Running |
| OpenClaw Gateway | 8080 | ✅ Running |
| Ollama | 11434 | ✅ Running |
### ✅ Completed
- Wyoming STT/TTS services installed and running
- Wyoming Satellite installed and running
- OpenClaw agent configured with home-assistant skill
- Custom OpenClaw conversation component created
### 🔄 Next Steps
1. Install OpenClaw conversation component in Home Assistant
2. Configure Wyoming integrations in HA
3. Create voice assistant pipeline with OpenClaw
4. Test the full voice loop
---
## Step 1: Install OpenClaw Conversation Component
Home Assistant is running in Docker on server 10.0.0.199. Use the automated installation script.
### Option A: Automated Installation (Recommended)
```bash
# From Mac Mini, run the installation script
cd ~/gitea/homeai/homeai-agent/custom_components
./install-to-docker-ha.sh
# The script will:
# 1. Create a tarball of the component
# 2. Copy it to the HA server via SCP
# 3. Extract it into the HA Docker container
# 4. Provide next steps
```
**Requirements:**
- SSH access to 10.0.0.199
- SSH keys configured (or password access)
### Option B: Manual Installation via SSH
```bash
# 1. Create tarball
cd ~/gitea/homeai/homeai-agent/custom_components
tar -czf openclaw_conversation.tar.gz openclaw_conversation/
# 2. Copy to HA server
scp openclaw_conversation.tar.gz 10.0.0.199:/tmp/
# 3. SSH to HA server and install
ssh 10.0.0.199
CONTAINER=$(docker ps --filter "name=homeassistant" --format "{{.Names}}" | head -n 1)
docker cp /tmp/openclaw_conversation.tar.gz $CONTAINER:/tmp/
docker exec $CONTAINER sh -c 'cd /config/custom_components && tar -xzf /tmp/openclaw_conversation.tar.gz'
docker restart $CONTAINER
```
### Option D: Using Home Assistant File Editor (Manual)
1. Open Home Assistant UI at http://10.0.0.199:8123
2. Install the **File Editor** add-on if not already installed
3. Create directory: `/config/custom_components/openclaw_conversation/`
4. Copy each file from `homeai-agent/custom_components/openclaw_conversation/`:
- `__init__.py`
- `config_flow.py`
- `const.py`
- `conversation.py`
- `manifest.json`
- `strings.json`
### Verify Installation
After installation, restart Home Assistant:
```bash
# Via SSH
ssh 10.0.0.199 'docker restart homeassistant'
# Or via HA UI
# Settings → System → Restart
```
Check logs for any errors:
- **Settings → System → Logs**
- Look for "OpenClaw Conversation" in the logs
---
## Step 2: Configure Wyoming Integrations
### Add Wyoming STT (Speech-to-Text)
1. Go to **Settings → Devices & Services → Add Integration**
2. Search for **"Wyoming Protocol"**
3. Configure:
- **Host**: `10.0.0.101` ⚠️ **Mac Mini IP, not HA server IP (10.0.0.199)**
- **Port**: `10300`
- **Name**: `Mac Mini STT`
4. Click **Submit**
### Add Wyoming TTS (Text-to-Speech)
1. Click **Add Integration** again
2. Search for **"Wyoming Protocol"**
3. Configure:
- **Host**: `10.0.0.101` ⚠️ **Mac Mini IP**
- **Port**: `10301`
- **Name**: `Mac Mini TTS`
4. Click **Submit**
### Add Wyoming Satellite
1. Click **Add Integration** again
2. Search for **"Wyoming Protocol"**
3. Configure:
- **Host**: `10.0.0.101` ⚠️ **Mac Mini IP**
- **Port**: `10700`
- **Name**: `Mac Mini Living Room`
4. Click **Submit**
### Verify Integrations
All three Wyoming integrations should appear in **Settings → Devices & Services**.
---
## Step 3: Add OpenClaw Conversation Agent
### Via UI (Recommended)
1. Go to **Settings → Devices & Services → Add Integration**
2. Search for **"OpenClaw Conversation"**
3. Configure:
- **OpenClaw Host**: `10.0.0.101` ⚠️ **Mac Mini IP, not HA server IP (10.0.0.199)**
- **OpenClaw Port**: `8080`
- **Agent Name**: `main`
- **Timeout**: `30` seconds
4. Click **Submit**
### Via YAML (Alternative)
Add to `/config/configuration.yaml`:
```yaml
openclaw_conversation:
openclaw_host: 10.0.0.101 # Mac Mini IP
openclaw_port: 8080
agent_name: main
timeout: 30
```
Then restart Home Assistant.
---
## Step 4: Create Voice Assistant Pipeline
1. Go to **Settings → Voice Assistants**
2. Click **Add Assistant**
3. Configure:
- **Name**: `HomeAI with OpenClaw`
- **Language**: `English`
- **Speech-to-Text**: Select `Mac Mini STT` (Wyoming)
- **Conversation Agent**: Select `OpenClaw Conversation`
- **Text-to-Speech**: Select `Mac Mini TTS` (Wyoming)
4. Click **Create**
### Set as Default
1. In **Settings → Voice Assistants**
2. Click the three dots next to "HomeAI with OpenClaw"
3. Select **Set as preferred**
---
## Step 5: Test the Pipeline
### Test 1: Text Input → TTS Output
1. Open Home Assistant UI
2. Click the **Assist** icon (microphone) in the top-right corner
3. Type: `"What time is it?"`
4. Press Enter
**Expected Result**: You should hear a spoken response via Kokoro TTS
### Test 2: Voice Input → OpenClaw → TTS Output
1. Ensure Wyoming Satellite is running on Mac Mini:
```bash
launchctl list | grep wyoming-satellite
```
2. Say the wake word: **"Hey Jarvis"**
3. Wait for the beep/acknowledgment
4. Speak: **"What time is it?"**
**Expected Result**: You should hear a spoken response
### Test 3: Home Assistant Control via Voice
1. Say: **"Hey Jarvis"**
2. Speak: **"Turn on the reading lamp"**
**Expected Result**:
- OpenClaw processes the request
- Home Assistant skill executes the action
- Light turns on
- You hear a confirmation via TTS
---
## Troubleshooting
### Issue: OpenClaw Conversation not appearing in integrations
**Solution**:
1. Verify files are in `/config/custom_components/openclaw_conversation/`
2. Check Home Assistant logs for errors
3. Ensure `manifest.json` is valid JSON
4. Restart Home Assistant
### Issue: Wyoming services not connecting
**Solution**:
1. Verify services are running on Mac Mini:
```bash
launchctl list | grep wyoming
nc -z 10.0.0.199 10300 # Test STT
nc -z 10.0.0.199 10301 # Test TTS
nc -z 10.0.0.199 10700 # Test Satellite
```
2. Check firewall rules on Mac Mini
3. Verify Home Assistant can reach Mac Mini network
### Issue: OpenClaw not responding
**Solution**:
1. Verify OpenClaw is running:
```bash
launchctl list | grep openclaw
pgrep -f openclaw
```
2. Test OpenClaw CLI directly:
```bash
openclaw agent --message "Hello" --agent main
```
3. Check OpenClaw logs:
```bash
tail -f /tmp/homeai-openclaw.log
```
4. Verify OpenClaw can reach Home Assistant:
```bash
curl http://10.0.0.199:8123/api/
```
### Issue: No audio output from satellite
**Solution**:
1. Check satellite logs:
```bash
tail -f /tmp/homeai-wyoming-satellite.log
```
2. Test audio output:
```bash
afplay /System/Library/Sounds/Glass.aiff
```
3. Verify SoX is installed:
```bash
which play
brew install sox
```
### Issue: Wake word not detected
**Solution**:
1. Check wakeword service:
```bash
launchctl list | grep wakeword
```
2. Test microphone input:
```bash
# Record a test
rec -r 16000 -c 1 test.wav trim 0 5
```
3. Adjust wake word threshold in satellite config
---
## Voice Pipeline Flow
```
┌─────────────────┐
│ USB Mic │
│ (Mac Mini) │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Wake Word │
│ Detection │
│ (hey_jarvis) │
└────────┬────────┘
│ wake detected
▼
┌─────────────────┐
│ Wyoming │
│ Satellite │
│ :10700 │
└────────┬────────┘
│ audio stream
▼
┌─────────────────┐
│ Wyoming STT │
│ (Whisper) │
│ :10300 │
└────────┬────────┘
│ transcript
▼
┌─────────────────┐
│ Home Assistant │
│ Voice Pipeline │
└────────┬────────┘
│ text
▼
┌─────────────────┐
│ OpenClaw │
│ Conversation │
│ Agent │
└────────┬────────┘
│ message
▼
┌─────────────────┐
│ OpenClaw │
│ Gateway │
│ :8080 │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Ollama LLM │
│ + Skills │
│ :11434 │
└────────┬────────┘
│ response
▼
┌─────────────────┐
│ Wyoming TTS │
│ (Kokoro) │
│ :10301 │
└────────┬────────┘
│ audio
▼
┌─────────────────┐
│ Speaker │
│ (Mac Mini) │
└─────────────────┘
```
---
## Next Steps After Setup
1. **Install Chatterbox TTS** for voice cloning
2. **Set up mem0** for long-term memory
3. **Configure n8n workflows** for automation
4. **Add Uptime Kuma monitors** for all services
5. **Begin ESP32 satellite setup** (Phase 4)
---
## Files Reference
| File | Purpose |
|------|---------|
| [`homeai-agent/custom_components/openclaw_conversation/`](../homeai-agent/custom_components/openclaw_conversation/) | Custom HA component |
| [`homeai-agent/skills/home-assistant/openclaw_bridge.py`](../homeai-agent/skills/home-assistant/openclaw_bridge.py) | Bridge script |
| [`homeai-voice/scripts/launchd/`](scripts/launchd/) | Service plists |
| [`plans/ha-voice-pipeline-implementation.md`](../plans/ha-voice-pipeline-implementation.md) | Detailed implementation plan |
| [`plans/voice-loop-integration.md`](../plans/voice-loop-integration.md) | Architecture options |
---
## Success Criteria
- [ ] Wyoming STT/TTS/Satellite appear in HA integrations
- [ ] OpenClaw Conversation agent appears in HA integrations
- [ ] Voice assistant pipeline created with OpenClaw
- [ ] Typed query in Assist returns spoken response
- [ ] Voice query via satellite returns spoken response
- [ ] "Turn on the reading lamp" command works end-to-end
- [ ] Latency under 5 seconds from wake to response
- [ ] All services survive Mac Mini reboot