feat(phase-04): Wyoming Satellite integration + OpenClaw HA components

## Voice Pipeline (P3) - Replace openWakeWord daemon with Wyoming Satellite approach - Add Wyoming Satellite service on port 10700 for HA voice pipeline - Update setup.sh with cross-platform sed compatibility (macOS/Linux) - Add version field to Kokoro TTS voice info - Update launchd service loader to use Wyoming Satellite ## Home Assistant Integration (P4) - Add custom conversation agent component (openclaw_conversation) - Fix: Use IntentResponse instead of plain strings (HA API requirement) - Support both HTTP API and CLI fallback modes - Config flow for easy HA UI setup - Add OpenClaw bridge scripts (Python + Bash) - Add ha-ctl utility for HA entity control - Fix: Use context manager for token file reading - Add HA configuration examples and documentation ## Infrastructure - Add mem0 backup automation (launchd + script) - Add n8n workflow templates (morning briefing, notification router) - Add VS Code workspace configuration - Reorganize model files into categorized folders: - lmstudio-community/ - mlx-community/ - bartowski/ - mradermacher/ ## Documentation - Update PROJECT_PLAN.md with Wyoming Satellite architecture - Update TODO.md with completed Wyoming integration tasks - Add OPENCLAW_INTEGRATION.md for HA setup guide ## Testing - Verified Wyoming services running (STT:10300, TTS:10301, Satellite:10700) - Verified OpenClaw CLI accessibility - Confirmed cross-platform compatibility fixes
2026-03-08 02:06:37 +00:00
parent 9eb5633115
commit 6a0bae2a0b
119 changed files with 780808 additions and 64 deletions
--- a/homeai-agent/skills/home-assistant/OPENCLAW_INTEGRATION.md
+++ b/homeai-agent/skills/home-assistant/OPENCLAW_INTEGRATION.md
@@ -0,0 +1,188 @@
+# OpenClaw Integration for Home Assistant Voice Pipeline
+
+> This document describes how to integrate OpenClaw with Home Assistant's voice pipeline using the Wyoming protocol.
+
+## Architecture Overview
+
+```
+┌─────────────────────────────────────────────────────────────────────────┐
+│                         Voice Pipeline Flow                             │
+├─────────────────────────────────────────────────────────────────────────┤
+│                                                                         │
+│   [Wyoming Satellite]        [Home Assistant]        [OpenClaw]        │
+│          │                          │                     │             │
+│          │  1. Wake word            │                     │             │
+│          │  2. Stream audio ───────>│                     │             │
+│          │                          │  3. Send to STT     │             │
+│          │                          │  ────────────────>  │             │
+│          │                          │                     │             │
+│          │                          │  4. Transcript      │             │
+│          │                          │  <────────────────  │             │
+│          │                          │                     │             │
+│          │                          │  5. Conversation    │             │
+│          │                          │  ────────────────>  │             │
+│          │                          │  (via bridge)       │             │
+│          │                          │                     │             │
+│          │                          │  6. Response        │             │
+│          │                          │  <────────────────  │             │
+│          │                          │                     │             │
+│          │  7. TTS audio <─────────│                     │             │
+│          │                          │                     │             │
+│   [Speaker]                                                          │
+│                                                                         │
+└─────────────────────────────────────────────────────────────────────────┘
+```
+
+## Components
+
+### 1. Wyoming Satellite (`com.homeai.wyoming-satellite.plist`)
+
+- **Port**: 10700 (exposes satellite for HA to connect)
+- **Function**: Handles audio I/O, wake word detection, streaming to HA
+- **Audio**: Uses SoX (`rec`/`play`) for macOS audio capture/playback
+- **Note**: Replaces the old `wakeword_daemon.py` - wake word is now handled by HA's voice pipeline
+
+### 2. Wyoming STT (`com.homeai.wyoming-stt.plist`)
+
+- **Port**: 10300 (Whisper large-v3)
+- **Function**: Speech-to-text transcription
+
+### 3. Wyoming TTS (`com.homeai.wyoming-tts.plist`)
+
+- **Port**: 10301 (Kokoro ONNX)
+- **Function**: Text-to-speech synthesis
+
+### 4. OpenClaw Bridge (`openclaw_bridge.py`)
+
+- **Function**: Connects HA conversation agent to OpenClaw CLI
+- **Usage**: Called via HA `shell_command` or `command_line` integration
+
+### Deprecated: Wake Word Daemon
+
+The old `com.homeai.wakeword.plist` service has been **disabled**. It was trying to notify `http://localhost:8080/wake` which doesn't exist in OpenClaw. Wake word detection is now handled by the Wyoming satellite through Home Assistant's voice pipeline.
+
+## Home Assistant Configuration
+
+### Step 1: Add Wyoming Protocol Integration
+
+1. Go to **Settings → Integrations → Add Integration**
+2. Search for **Wyoming Protocol**
+3. Add the following services:
+
+| Service | Host | Port |
+|---------|------|------|
+| Speech-to-Text | `10.0.0.199` | `10300` |
+| Text-to-Speech | `10.0.0.199` | `10301` |
+| Satellite | `10.0.0.199` | `10700` |
+
+### Step 2: Configure Voice Assistant Pipeline
+
+1. Go to **Settings → Voice Assistants**
+2. Create a new pipeline:
+   - **Name**: "HomeAI with OpenClaw"
+   - **Speech-to-Text**: Wyoming (localhost:10300)
+   - **Conversation Agent**: Home Assistant (or custom below)
+   - **Text-to-Speech**: Wyoming (localhost:10301)
+
+### Step 3: Add OpenClaw Bridge to HA
+
+Add to your `configuration.yaml`:
+
+```yaml
+shell_command:
+  openclaw_chat: 'python3 /Users/aodhan/gitea/homeai/homeai-agent/skills/home-assistant/openclaw_bridge.py "{{ message }}" --raw'
+```
+
+### Step 4: Create Automation for OpenClaw
+
+Create an automation that routes voice commands to OpenClaw:
+
+```yaml
+automation:
+  - alias: "Voice Command via OpenClaw"
+    trigger:
+      - platform: conversation
+        command:
+          - "ask jarvis *"
+    action:
+      - service: shell_command.openclaw_chat
+        data:
+          message: "{{ trigger.slots.command }}"
+        response_variable: openclaw_response
+      
+      - service: tts.speak
+        data:
+          media_player_entity_id: media_player.living_room_speaker
+          message: "{{ openclaw_response }}"
+```
+
+## Manual Testing
+
+### Test STT
+```bash
+# Check if STT is running
+nc -z localhost 10300 && echo "STT OK"
+```
+
+### Test TTS
+```bash
+# Check if TTS is running
+nc -z localhost 10301 && echo "TTS OK"
+```
+
+### Test Satellite
+```bash
+# Check if satellite is running
+nc -z localhost 10700 && echo "Satellite OK"
+```
+
+### Test OpenClaw Bridge
+```bash
+# Test the bridge directly
+python3 homeai-agent/skills/home-assistant/openclaw_bridge.py "Turn on the living room lights"
+```
+
+### Test Full Pipeline
+1. Load all services: `./homeai-voice/scripts/load-all-launchd.sh`
+2. Open HA Assist panel (Settings → Voice Assistants → Assist)
+3. Type or speak: "Turn on the study shelves light"
+4. You should hear the TTS response
+
+## Troubleshooting
+
+### Satellite not connecting to HA
+- Check that the satellite is running: `launchctl list com.homeai.wyoming-satellite`
+- Check logs: `tail -f /tmp/homeai-wyoming-satellite.log`
+- Verify HA can reach the satellite: Test from HA container/host
+
+### No audio output
+- Check SoX installation: `which play`
+- Test SoX directly: `echo "test" | say` or `play /System/Library/Sounds/Glass.aiff`
+- Check audio device permissions
+
+### OpenClaw not responding
+- Verify OpenClaw is running: `pgrep -f openclaw`
+- Test CLI directly: `openclaw agent --message "Hello" --agent main`
+- Check OpenClaw config: `cat ~/.openclaw/openclaw.json`
+
+### Wyoming version conflicts
+- The satellite requires wyoming 1.4.1 but faster-whisper requires 1.8+
+- We've patched this - both should work with wyoming 1.8.0
+- If issues occur, reinstall: `pip install 'wyoming>=1.8' wyoming-satellite`
+
+## File Locations
+
+| File | Purpose |
+|------|---------|
+| `~/.openclaw/openclaw.json` | OpenClaw configuration |
+| `~/homeai-voice-env/` | Python virtual environment |
+| `~/Library/LaunchAgents/com.homeai.*.plist` | Launchd services |
+| `/tmp/homeai-*.log` | Service logs |
+
+## Next Steps
+
+1. [ ] Test voice pipeline end-to-end
+2. [ ] Fine-tune wake word sensitivity
+3. [ ] Add custom intents for OpenClaw
+4. [ ] Implement conversation history/memory
+5. [ ] Add ESP32 satellite support (P6)
--- a/homeai-agent/skills/home-assistant/ha-configuration.yaml
+++ b/homeai-agent/skills/home-assistant/ha-configuration.yaml
@@ -0,0 +1,91 @@
+# Home Assistant Configuration for OpenClaw Integration
+# Add these sections to your configuration.yaml
+
+# ─── Shell Command Integration ────────────────────────────────────────────────
+# This allows HA to call OpenClaw via shell commands
+shell_command:
+  # Send a message to OpenClaw and get response
+  openclaw_chat: '/Users/aodhan/gitea/homeai/homeai-agent/skills/home-assistant/openclaw-bridge.sh "{{ message }}"'
+
+# ─── REST Command (Alternative) ───────────────────────────────────────────────
+# If OpenClaw exposes an HTTP API in the future
+rest_command:
+  openclaw_chat:
+    url: "http://localhost:8080/api/agent/message"
+    method: POST
+    headers:
+      Authorization: "Bearer {{ token }}"
+    content_type: "application/json"
+    payload: '{"message": "{{ message }}", "agent": "main"}'
+
+# ─── Command Line Sensor ──────────────────────────────────────────────────────
+# Execute OpenClaw and return the response as a sensor
+command_line:
+  - sensor:
+      name: "OpenClaw Response"
+      unique_id: openclaw_response
+      command: "/Users/aodhan/gitea/homeai/homeai-agent/skills/home-assistant/openclaw-bridge.sh '{{ states(\"input_text.openclaw_query\") }}'"
+      value_template: "{{ value_json.response }}"
+      scan_interval: 86400  # Only update when triggered
+
+# ─── Input Text for Query ─────────────────────────────────────────────────────
+input_text:
+  openclaw_query:
+    name: OpenClaw Query
+    initial: ""
+    max: 255
+
+# ─── Conversation Agent Integration ────────────────────────────────────────────
+# Custom conversation agent using OpenClaw
+# This requires the custom conversation agent below
+
+# ─── Intent Script ─────────────────────────────────────────────────────────────
+intent_script:
+  # Handle conversation intents
+  OpenClawConversation:
+    speech:
+      text: "{{ response }}"
+    action:
+      - service: shell_command.openclaw_chat
+        data:
+          message: "{{ text }}"
+        response_variable: openclaw_result
+      - set:
+          response: "{{ openclaw_result }}"
+
+# ─── Automation: Voice Pipeline with OpenClaw ─────────────────────────────────
+automation:
+  - alias: "Voice Command via OpenClaw"
+    trigger:
+      - platform: conversation
+        command:
+          - "ask jarvis *"
+    action:
+      - service: shell_command.openclaw_chat
+        data:
+          message: "{{ trigger.slots.command }}"
+        response_variable: openclaw_response
+      
+      - service: tts.speak
+        data:
+          media_player_entity_id: media_player.living_room_speaker
+          message: "{{ openclaw_response }}"
+
+# ─── Wyoming Protocol Configuration ───────────────────────────────────────────
+# Configure in HA UI:
+# 1. Settings → Integrations → Add Integration → Wyoming Protocol
+# 2. Add STT: host=10.0.0.199, port=10300
+# 3. Add TTS: host=10.0.0.199, port=10301
+# 4. Add Satellite: host=10.0.0.199, port=10700
+
+# ─── Voice Assistant Pipeline ─────────────────────────────────────────────────
+# Configure in HA UI:
+# 1. Settings → Voice Assistants → Add Pipeline
+# 2. Name: "HomeAI with OpenClaw"
+# 3. Speech-to-Text: Wyoming (localhost:10300)
+# 4. Conversation Agent: Use the automation above OR Home Assistant
+# 5. Text-to-Speech: Wyoming (localhost:10301)
+
+# ─── Custom Conversation Agent (Advanced) ─────────────────────────────────────
+# Create a custom component in custom_components/openclaw_conversation/
+# See: custom_components/openclaw_conversation/__init__.py
--- a/homeai-agent/skills/home-assistant/ha-ctl
+++ b/homeai-agent/skills/home-assistant/ha-ctl
@@ -0,0 +1,188 @@
+#!/usr/bin/env python3
+"""
+ha-ctl — Home Assistant control CLI for OpenClaw agent.
+
+Usage:
+  ha-ctl list [domain]                  List entities (optionally filtered by domain)
+  ha-ctl state <entity_id_or_name>      Get current state of an entity
+  ha-ctl on <entity_id_or_name>         Turn entity on
+  ha-ctl off <entity_id_or_name>        Turn entity off
+  ha-ctl toggle <entity_id_or_name>     Toggle entity
+  ha-ctl set <entity_id> <attr> <val>   Set attribute (e.g. brightness 128)
+  ha-ctl scene <scene_name>             Activate a scene
+
+Environment:
+  HASS_TOKEN   Long-lived access token
+  HA_URL       Base URL (default: https://10.0.0.199:8123)
+"""
+
+import sys
+import os
+import json
+import urllib.request
+import urllib.error
+import ssl
+
+HA_URL = os.environ.get("HA_URL", "https://10.0.0.199:8123").rstrip("/")
+TOKEN = os.environ.get("HASS_TOKEN") or os.environ.get("HA_TOKEN")
+
+if not TOKEN:
+    token_file = os.path.expanduser("~/.homeai/hass_token")
+    if os.path.exists(token_file):
+        with open(token_file) as f:
+            TOKEN = f.read().strip()
+
+if not TOKEN:
+    print("ERROR: No HASS_TOKEN set. Export HASS_TOKEN or write to ~/.homeai/hass_token", file=sys.stderr)
+    sys.exit(1)
+
+# Skip SSL verification for self-signed certs on local HA
+ctx = ssl.create_default_context()
+ctx.check_hostname = False
+ctx.verify_mode = ssl.CERT_NONE
+
+
+def api(method, path, data=None):
+    url = f"{HA_URL}/api{path}"
+    headers = {"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"}
+    body = json.dumps(data).encode() if data is not None else None
+    req = urllib.request.Request(url, data=body, headers=headers, method=method)
+    try:
+        with urllib.request.urlopen(req, context=ctx, timeout=10) as resp:
+            return json.loads(resp.read())
+    except urllib.error.HTTPError as e:
+        print(f"ERROR: HTTP {e.code} — {e.read().decode()}", file=sys.stderr)
+        sys.exit(1)
+
+
+def get_states():
+    return api("GET", "/states")
+
+
+def resolve_entity(query, states=None):
+    """Resolve a friendly name or partial entity_id to a full entity_id."""
+    if states is None:
+        states = get_states()
+    query_lower = query.lower().replace("_", " ")
+    # Exact entity_id match
+    for s in states:
+        if s["entity_id"] == query:
+            return s["entity_id"]
+    # Friendly name exact match
+    for s in states:
+        name = s.get("attributes", {}).get("friendly_name", "").lower()
+        if name == query_lower:
+            return s["entity_id"]
+    # Partial entity_id match
+    for s in states:
+        if query_lower in s["entity_id"].lower():
+            return s["entity_id"]
+    # Partial friendly name match
+    matches = []
+    for s in states:
+        name = s.get("attributes", {}).get("friendly_name", "").lower()
+        if query_lower in name:
+            matches.append(s)
+    if len(matches) == 1:
+        return matches[0]["entity_id"]
+    if len(matches) > 1:
+        names = [f"{m['entity_id']} ({m['attributes'].get('friendly_name','')})" for m in matches]
+        print(f"Ambiguous: {', '.join(names)}", file=sys.stderr)
+        sys.exit(1)
+    print(f"ERROR: No entity found matching '{query}'", file=sys.stderr)
+    sys.exit(1)
+
+
+def call_service(domain, service, entity_id, extra=None):
+    data = {"entity_id": entity_id}
+    if extra:
+        data.update(extra)
+    result = api("POST", f"/services/{domain}/{service}", data)
+    return result
+
+
+def cmd_list(args):
+    domain_filter = args[0] if args else None
+    states = get_states()
+    if domain_filter:
+        states = [s for s in states if s["entity_id"].startswith(domain_filter + ".")]
+    for s in sorted(states, key=lambda x: x["entity_id"]):
+        name = s.get("attributes", {}).get("friendly_name", "")
+        print(f"{s['entity_id']}\t{s['state']}\t{name}")
+
+
+def cmd_state(args):
+    if not args:
+        print("Usage: ha-ctl state <entity>", file=sys.stderr); sys.exit(1)
+    states = get_states()
+    eid = resolve_entity(args[0], states)
+    s = next(x for x in states if x["entity_id"] == eid)
+    print(f"Entity: {eid}")
+    print(f"State:  {s['state']}")
+    attrs = s.get("attributes", {})
+    for k, v in attrs.items():
+        print(f"  {k}: {v}")
+
+
+def cmd_control(action, args):
+    if not args:
+        print(f"Usage: ha-ctl {action} <entity>", file=sys.stderr); sys.exit(1)
+    states = get_states()
+    eid = resolve_entity(args[0], states)
+    domain = eid.split(".")[0]
+    service_map = {"on": "turn_on", "off": "turn_off", "toggle": "toggle"}
+    service = service_map[action]
+    call_service(domain, service, eid)
+    name = next((x.get("attributes", {}).get("friendly_name", eid)
+                 for x in states if x["entity_id"] == eid), eid)
+    print(f"OK: {service} → {name} ({eid})")
+
+
+def cmd_scene(args):
+    if not args:
+        print("Usage: ha-ctl scene <name>", file=sys.stderr); sys.exit(1)
+    states = get_states()
+    eid = resolve_entity(args[0], states)
+    call_service("scene", "turn_on", eid)
+    print(f"OK: scene activated → {eid}")
+
+
+def cmd_set(args):
+    if len(args) < 3:
+        print("Usage: ha-ctl set <entity> <attribute> <value>", file=sys.stderr); sys.exit(1)
+    states = get_states()
+    eid = resolve_entity(args[0], states)
+    domain = eid.split(".")[0]
+    attr, val = args[1], args[2]
+    try:
+        val = int(val)
+    except ValueError:
+        try:
+            val = float(val)
+        except ValueError:
+            pass
+    call_service(domain, "turn_on", eid, {attr: val})
+    print(f"OK: set {attr}={val} → {eid}")
+
+
+if __name__ == "__main__":
+    args = sys.argv[1:]
+    if not args:
+        print(__doc__)
+        sys.exit(0)
+    cmd = args[0]
+    rest = args[1:]
+    if cmd == "list":
+        cmd_list(rest)
+    elif cmd == "state":
+        cmd_state(rest)
+    elif cmd in ("on", "off", "toggle"):
+        cmd_control(cmd, rest)
+    elif cmd == "scene":
+        cmd_scene(rest)
+    elif cmd == "set":
+        cmd_set(rest)
+    else:
+        print(f"Unknown command: {cmd}", file=sys.stderr)
+        print(__doc__)
+        sys.exit(1)
--- a/homeai-agent/skills/home-assistant/openclaw-bridge.sh
+++ b/homeai-agent/skills/home-assistant/openclaw-bridge.sh
@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+# OpenClaw Bridge Script for Home Assistant
+# 
+# Usage: ./openclaw-bridge.sh "message to send to OpenClaw"
+# Returns: JSON response suitable for HA TTS
+
+set -euo pipefail
+
+MESSAGE="${1:-}"
+AGENT="${2:-main}"
+TIMEOUT="${3:-30}"
+
+if [[ -z "$MESSAGE" ]]; then
+  echo '{"error": "No message provided"}' >&2
+  exit 1
+fi
+
+# Run OpenClaw agent and capture response
+# The CLI outputs the response to stdout
+RESPONSE=$(openclaw agent --message "$MESSAGE" --agent "$AGENT" 2>/dev/null || echo "Error: OpenClaw command failed")
+
+# Output JSON for HA using jq for proper escaping
+if command -v jq &>/dev/null; then
+  echo "$RESPONSE" | jq -Rs '{response: .}'
+else
+  # Fallback: use Python for JSON encoding if jq is not available
+  python3 -c "import json,sys; print(json.dumps({'response': sys.stdin.read()}))" <<< "$RESPONSE"
+fi
--- a/homeai-agent/skills/home-assistant/openclaw_bridge.py
+++ b/homeai-agent/skills/home-assistant/openclaw_bridge.py
@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+"""OpenClaw Bridge for Home Assistant
+
+This script acts as a bridge between Home Assistant and OpenClaw.
+It can be called from HA via shell_command or command_line integration.
+
+Usage:
+    python openclaw_bridge.py "Your message here"
+    
+Output:
+    {"response": "OpenClaw's response text"}
+"""
+
+import argparse
+import json
+import subprocess
+import sys
+from pathlib import Path
+
+
+def call_openclaw(message: str, agent: str = "main", timeout: int = 30) -> str:
+    """Call OpenClaw CLI and return the response."""
+    try:
+        result = subprocess.run(
+            ["openclaw", "agent", "--message", message, "--agent", agent],
+            capture_output=True,
+            text=True,
+            timeout=timeout,
+        )
+        
+        if result.returncode != 0:
+            return f"Error: OpenClaw failed with code {result.returncode}"
+        
+        # Return stdout (the response)
+        return result.stdout.strip()
+    
+    except subprocess.TimeoutExpired:
+        return "Error: OpenClaw command timed out"
+    except FileNotFoundError:
+        return "Error: openclaw command not found. Is OpenClaw installed?"
+    except Exception as e:
+        return f"Error: {str(e)}"
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Bridge between Home Assistant and OpenClaw"
+    )
+    parser.add_argument("message", help="Message to send to OpenClaw")
+    parser.add_argument(
+        "--agent", default="main", help="Agent to use (default: main)"
+    )
+    parser.add_argument(
+        "--timeout", type=int, default=30, help="Timeout in seconds (default: 30)"
+    )
+    parser.add_argument(
+        "--raw", action="store_true", help="Output raw text instead of JSON"
+    )
+    
+    args = parser.parse_args()
+    
+    # Call OpenClaw
+    response = call_openclaw(args.message, args.agent, args.timeout)
+    
+    if args.raw:
+        print(response)
+    else:
+        # Output as JSON for HA
+        output = {"response": response}
+        print(json.dumps(output))
+
+
+if __name__ == "__main__":
+    main()