homeai/homeai-voice/VOICE_PIPELINE_SETUP.md

Voice Pipeline Setup Guide

Complete guide to setting up the end-to-end voice pipeline with OpenClaw integration

---

Network Configuration

Important: Services are split across two machines:

Service	Port	Location
Wyoming STT (Whisper large-v3)	10300	Mac Mini (10.0.0.101)
Wyoming TTS (Kokoro ONNX)	10301	Mac Mini (10.0.0.101)
Wyoming Satellite	10700	Mac Mini (10.0.0.101)
openWakeWord	-	Mac Mini (10.0.0.101)
OpenClaw Gateway	8080	Mac Mini (10.0.0.101)
Ollama	11434	Mac Mini (10.0.0.101)
Home Assistant (Docker)	8123	Server (10.0.0.199)

All integrations must point to 10.0.0.101 (Mac Mini), not 10.0.0.199 (HA server).

---

Current Status

✅ Services Running on Mac Mini (10.0.0.101)

Service	Port	Status
Wyoming STT	10300	✅ Running
Wyoming TTS	10301	✅ Running
Wyoming Satellite	10700	✅ Running
openWakeWord	-	✅ Running
OpenClaw Gateway	8080	✅ Running
Ollama	11434	✅ Running

✅ Completed

• Wyoming STT/TTS services installed and running
• Wyoming Satellite installed and running
• OpenClaw agent configured with home-assistant skill
• Custom OpenClaw conversation component created

🔄 Next Steps

1. Install OpenClaw conversation component in Home Assistant
2. Configure Wyoming integrations in HA
3. Create voice assistant pipeline with OpenClaw
4. Test the full voice loop

---

Step 1: Install OpenClaw Conversation Component

Home Assistant is running in Docker on server 10.0.0.199. Use the automated installation script.

Option A: Automated Installation (Recommended)

# From Mac Mini, run the installation script
cd ~/gitea/homeai/homeai-agent/custom_components
./install-to-docker-ha.sh

# The script will:
# 1. Create a tarball of the component
# 2. Copy it to the HA server via SCP
# 3. Extract it into the HA Docker container
# 4. Provide next steps

Requirements:

• SSH access to 10.0.0.199
• SSH keys configured (or password access)

Option B: Manual Installation via SSH

# 1. Create tarball
cd ~/gitea/homeai/homeai-agent/custom_components
tar -czf openclaw_conversation.tar.gz openclaw_conversation/

# 2. Copy to HA server
scp openclaw_conversation.tar.gz 10.0.0.199:/tmp/

# 3. SSH to HA server and install
ssh 10.0.0.199
CONTAINER=$(docker ps --filter "name=homeassistant" --format "{{.Names}}" | head -n 1)
docker cp /tmp/openclaw_conversation.tar.gz $CONTAINER:/tmp/
docker exec $CONTAINER sh -c 'cd /config/custom_components && tar -xzf /tmp/openclaw_conversation.tar.gz'
docker restart $CONTAINER

Option D: Using Home Assistant File Editor (Manual)

1. Open Home Assistant UI at http://10.0.0.199:8123
2. Install the File Editor add-on if not already installed
3. Create directory: /config/custom_components/openclaw_conversation/
4. Copy each file from homeai-agent/custom_components/openclaw_conversation/:
   • __init__.py
   • config_flow.py
   • const.py
   • conversation.py
   • manifest.json
   • strings.json

Verify Installation

After installation, restart Home Assistant:

# Via SSH
ssh 10.0.0.199 'docker restart homeassistant'

# Or via HA UI
# Settings → System → Restart

Check logs for any errors:

• Settings → System → Logs
• Look for "OpenClaw Conversation" in the logs

---

Step 2: Configure Wyoming Integrations

Add Wyoming STT (Speech-to-Text)

1. Go to Settings → Devices & Services → Add Integration
2. Search for "Wyoming Protocol"
3. Configure:
   • Host: 10.0.0.101 ⚠️ Mac Mini IP, not HA server IP (10.0.0.199)
   • Port: 10300
   • Name: Mac Mini STT
4. Click Submit

Add Wyoming TTS (Text-to-Speech)

1. Click Add Integration again
2. Search for "Wyoming Protocol"
3. Configure:
   • Host: 10.0.0.101 ⚠️ Mac Mini IP
   • Port: 10301
   • Name: Mac Mini TTS
4. Click Submit

Add Wyoming Satellite

1. Click Add Integration again
2. Search for "Wyoming Protocol"
3. Configure:
   • Host: 10.0.0.101 ⚠️ Mac Mini IP
   • Port: 10700
   • Name: Mac Mini Living Room
4. Click Submit

Verify Integrations

All three Wyoming integrations should appear in Settings → Devices & Services.

---

Step 3: Add OpenClaw Conversation Agent

Via UI (Recommended)

1. Go to Settings → Devices & Services → Add Integration
2. Search for "OpenClaw Conversation"
3. Configure:
   • OpenClaw Host: 10.0.0.101 ⚠️ Mac Mini IP, not HA server IP (10.0.0.199)
   • OpenClaw Port: 8080
   • Agent Name: main
   • Timeout: 30 seconds
4. Click Submit

Via YAML (Alternative)

Add to /config/configuration.yaml:

openclaw_conversation:
  openclaw_host: 10.0.0.101  # Mac Mini IP
  openclaw_port: 8080
  agent_name: main
  timeout: 30

Then restart Home Assistant.

---

Step 4: Create Voice Assistant Pipeline

1. Go to Settings → Voice Assistants
2. Click Add Assistant
3. Configure:
   • Name: HomeAI with OpenClaw
   • Language: English
   • Speech-to-Text: Select Mac Mini STT (Wyoming)
   • Conversation Agent: Select OpenClaw Conversation
   • Text-to-Speech: Select Mac Mini TTS (Wyoming)
4. Click Create

Set as Default

1. In Settings → Voice Assistants
2. Click the three dots next to "HomeAI with OpenClaw"
3. Select Set as preferred

---

Step 5: Test the Pipeline

Test 1: Text Input → TTS Output

1. Open Home Assistant UI
2. Click the Assist icon (microphone) in the top-right corner
3. Type: "What time is it?"
4. Press Enter

Expected Result: You should hear a spoken response via Kokoro TTS

Test 2: Voice Input → OpenClaw → TTS Output

1. Ensure Wyoming Satellite is running on Mac Mini:

   launchctl list | grep wyoming-satellite
   

2. Say the wake word: "Hey Jarvis"

3. Wait for the beep/acknowledgment

4. Speak: "What time is it?"

Expected Result: You should hear a spoken response

Test 3: Home Assistant Control via Voice

1. Say: "Hey Jarvis"
2. Speak: "Turn on the reading lamp"

Expected Result:

• OpenClaw processes the request
• Home Assistant skill executes the action
• Light turns on
• You hear a confirmation via TTS

---

Troubleshooting

Issue: OpenClaw Conversation not appearing in integrations

Solution:

1. Verify files are in /config/custom_components/openclaw_conversation/
2. Check Home Assistant logs for errors
3. Ensure manifest.json is valid JSON
4. Restart Home Assistant

Issue: Wyoming services not connecting

Solution:

1. Verify services are running on Mac Mini:

   launchctl list | grep wyoming
   nc -z 10.0.0.199 10300  # Test STT
   nc -z 10.0.0.199 10301  # Test TTS
   nc -z 10.0.0.199 10700  # Test Satellite
   

2. Check firewall rules on Mac Mini

3. Verify Home Assistant can reach Mac Mini network

Issue: OpenClaw not responding

Solution:

1. Verify OpenClaw is running:

   launchctl list | grep openclaw
   pgrep -f openclaw
   

2. Test OpenClaw CLI directly:

   openclaw agent --message "Hello" --agent main
   

3. Check OpenClaw logs:

   tail -f /tmp/homeai-openclaw.log
   

4. Verify OpenClaw can reach Home Assistant:

   curl http://10.0.0.199:8123/api/
   

Issue: No audio output from satellite

Solution:

1. Check satellite logs:

   tail -f /tmp/homeai-wyoming-satellite.log
   

2. Test audio output:

   afplay /System/Library/Sounds/Glass.aiff
   

3. Verify SoX is installed:

   which play
   brew install sox
   

Issue: Wake word not detected

Solution:

1. Check wakeword service:

   launchctl list | grep wakeword
   

2. Test microphone input:

   # Record a test
   rec -r 16000 -c 1 test.wav trim 0 5
   

3. Adjust wake word threshold in satellite config

---

Voice Pipeline Flow

┌─────────────────┐
│   USB Mic       │
│  (Mac Mini)     │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Wake Word      │
│  Detection      │
│  (hey_jarvis)   │
└────────┬────────┘
         │ wake detected
         ▼
┌─────────────────┐
│ Wyoming         │
│ Satellite       │
│ :10700          │
└────────┬────────┘
         │ audio stream
         ▼
┌─────────────────┐
│ Wyoming STT     │
│ (Whisper)       │
│ :10300          │
└────────┬────────┘
         │ transcript
         ▼
┌─────────────────┐
│ Home Assistant  │
│ Voice Pipeline  │
└────────┬────────┘
         │ text
         ▼
┌─────────────────┐
│ OpenClaw        │
│ Conversation    │
│ Agent           │
└────────┬────────┘
         │ message
         ▼
┌─────────────────┐
│ OpenClaw        │
│ Gateway         │
│ :8080           │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Ollama LLM      │
│ + Skills        │
│ :11434          │
└────────┬────────┘
         │ response
         ▼
┌─────────────────┐
│ Wyoming TTS     │
│ (Kokoro)        │
│ :10301          │
└────────┬────────┘
         │ audio
         ▼
┌─────────────────┐
│   Speaker       │
│  (Mac Mini)     │
└─────────────────┘

---

Next Steps After Setup

1. Install Chatterbox TTS for voice cloning
2. Set up mem0 for long-term memory
3. Configure n8n workflows for automation
4. Add Uptime Kuma monitors for all services
5. Begin ESP32 satellite setup (Phase 4)

---

Files Reference

File	Purpose
homeai-agent/custom_components/openclaw_conversation/	Custom HA component
homeai-agent/skills/home-assistant/openclaw_bridge.py	Bridge script
homeai-voice/scripts/launchd/	Service plists
plans/ha-voice-pipeline-implementation.md	Detailed implementation plan
plans/voice-loop-integration.md	Architecture options

---

Success Criteria

• Wyoming STT/TTS/Satellite appear in HA integrations
• OpenClaw Conversation agent appears in HA integrations
• Voice assistant pipeline created with OpenClaw
• Typed query in Assist returns spoken response
• Voice query via satellite returns spoken response
• "Turn on the reading lamp" command works end-to-end
• Latency under 5 seconds from wake to response
• All services survive Mac Mini reboot