# OpenClaw Integration for Home Assistant Voice Pipeline

> This document describes how to integrate OpenClaw with Home Assistant's voice pipeline using the Wyoming protocol.

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────────────────┐
│                         Voice Pipeline Flow                             │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│   [Wyoming Satellite]        [Home Assistant]        [OpenClaw]        │
│          │                          │                     │             │
│          │  1. Wake word            │                     │             │
│          │  2. Stream audio ───────>│                     │             │
│          │                          │  3. Send to STT     │             │
│          │                          │  ────────────────>  │             │
│          │                          │                     │             │
│          │                          │  4. Transcript      │             │
│          │                          │  <────────────────  │             │
│          │                          │                     │             │
│          │                          │  5. Conversation    │             │
│          │                          │  ────────────────>  │             │
│          │                          │  (via bridge)       │             │
│          │                          │                     │             │
│          │                          │  6. Response        │             │
│          │                          │  <────────────────  │             │
│          │                          │                     │             │
│          │  7. TTS audio <─────────│                     │             │
│          │                          │                     │             │
│   [Speaker]                                                          │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘
```

## Components

### 1. Wyoming Satellite (`com.homeai.wyoming-satellite.plist`)

- **Port**: 10700 (exposes satellite for HA to connect)
- **Function**: Handles audio I/O, wake word detection, streaming to HA
- **Audio**: Uses SoX (`rec`/`play`) for macOS audio capture/playback
- **Note**: Replaces the old `wakeword_daemon.py` - wake word is now handled by HA's voice pipeline

### 2. Wyoming STT (`com.homeai.wyoming-stt.plist`)

- **Port**: 10300 (Whisper large-v3)
- **Function**: Speech-to-text transcription

### 3. Wyoming TTS (`com.homeai.wyoming-tts.plist`)

- **Port**: 10301 (Kokoro ONNX)
- **Function**: Text-to-speech synthesis

### 4. OpenClaw Bridge (`openclaw_bridge.py`)

- **Function**: Connects HA conversation agent to OpenClaw CLI
- **Usage**: Called via HA `shell_command` or `command_line` integration

### Deprecated: Wake Word Daemon

The old `com.homeai.wakeword.plist` service has been **disabled**. It was trying to notify `http://localhost:8080/wake` which doesn't exist in OpenClaw. Wake word detection is now handled by the Wyoming satellite through Home Assistant's voice pipeline.

## Home Assistant Configuration

### Step 1: Add Wyoming Protocol Integration

1. Go to **Settings → Integrations → Add Integration**
2. Search for **Wyoming Protocol**
3. Add the following services:

| Service | Host | Port |
|---------|------|------|
| Speech-to-Text | `10.0.0.199` | `10300` |
| Text-to-Speech | `10.0.0.199` | `10301` |
| Satellite | `10.0.0.199` | `10700` |

### Step 2: Configure Voice Assistant Pipeline

1. Go to **Settings → Voice Assistants**
2. Create a new pipeline:
   - **Name**: "HomeAI with OpenClaw"
   - **Speech-to-Text**: Wyoming (localhost:10300)
   - **Conversation Agent**: Home Assistant (or custom below)
   - **Text-to-Speech**: Wyoming (localhost:10301)

### Step 3: Add OpenClaw Bridge to HA

Add to your `configuration.yaml`:

```yaml
shell_command:
  openclaw_chat: 'python3 /Users/aodhan/gitea/homeai/homeai-agent/skills/home-assistant/openclaw_bridge.py "{{ message }}" --raw'
```

### Step 4: Create Automation for OpenClaw

Create an automation that routes voice commands to OpenClaw:

```yaml
automation:
  - alias: "Voice Command via OpenClaw"
    trigger:
      - platform: conversation
        command:
          - "ask jarvis *"
    action:
      - service: shell_command.openclaw_chat
        data:
          message: "{{ trigger.slots.command }}"
        response_variable: openclaw_response
      
      - service: tts.speak
        data:
          media_player_entity_id: media_player.living_room_speaker
          message: "{{ openclaw_response }}"
```

## Manual Testing

### Test STT
```bash
# Check if STT is running
nc -z localhost 10300 && echo "STT OK"
```

### Test TTS
```bash
# Check if TTS is running
nc -z localhost 10301 && echo "TTS OK"
```

### Test Satellite
```bash
# Check if satellite is running
nc -z localhost 10700 && echo "Satellite OK"
```

### Test OpenClaw Bridge
```bash
# Test the bridge directly
python3 homeai-agent/skills/home-assistant/openclaw_bridge.py "Turn on the living room lights"
```

### Test Full Pipeline
1. Load all services: `./homeai-voice/scripts/load-all-launchd.sh`
2. Open HA Assist panel (Settings → Voice Assistants → Assist)
3. Type or speak: "Turn on the study shelves light"
4. You should hear the TTS response

## Troubleshooting

### Satellite not connecting to HA
- Check that the satellite is running: `launchctl list com.homeai.wyoming-satellite`
- Check logs: `tail -f /tmp/homeai-wyoming-satellite.log`
- Verify HA can reach the satellite: Test from HA container/host

### No audio output
- Check SoX installation: `which play`
- Test SoX directly: `echo "test" | say` or `play /System/Library/Sounds/Glass.aiff`
- Check audio device permissions

### OpenClaw not responding
- Verify OpenClaw is running: `pgrep -f openclaw`
- Test CLI directly: `openclaw agent --message "Hello" --agent main`
- Check OpenClaw config: `cat ~/.openclaw/openclaw.json`

### Wyoming version conflicts
- The satellite requires wyoming 1.4.1 but faster-whisper requires 1.8+
- We've patched this - both should work with wyoming 1.8.0
- If issues occur, reinstall: `pip install 'wyoming>=1.8' wyoming-satellite`

## File Locations

| File | Purpose |
|------|---------|
| `~/.openclaw/openclaw.json` | OpenClaw configuration |
| `~/homeai-voice-env/` | Python virtual environment |
| `~/Library/LaunchAgents/com.homeai.*.plist` | Launchd services |
| `/tmp/homeai-*.log` | Service logs |

## Next Steps

1. [ ] Test voice pipeline end-to-end
2. [ ] Fine-tune wake word sensitivity
3. [ ] Add custom intents for OpenClaw
4. [ ] Implement conversation history/memory
5. [ ] Add ESP32 satellite support (P6)