feat(phase-04): Wyoming Satellite integration + OpenClaw HA components

## Voice Pipeline (P3)
- Replace openWakeWord daemon with Wyoming Satellite approach
- Add Wyoming Satellite service on port 10700 for HA voice pipeline
- Update setup.sh with cross-platform sed compatibility (macOS/Linux)
- Add version field to Kokoro TTS voice info
- Update launchd service loader to use Wyoming Satellite

## Home Assistant Integration (P4)
- Add custom conversation agent component (openclaw_conversation)
  - Fix: Use IntentResponse instead of plain strings (HA API requirement)
  - Support both HTTP API and CLI fallback modes
  - Config flow for easy HA UI setup
- Add OpenClaw bridge scripts (Python + Bash)
- Add ha-ctl utility for HA entity control
  - Fix: Use context manager for token file reading
- Add HA configuration examples and documentation

## Infrastructure
- Add mem0 backup automation (launchd + script)
- Add n8n workflow templates (morning briefing, notification router)
- Add VS Code workspace configuration
- Reorganize model files into categorized folders:
  - lmstudio-community/
  - mlx-community/
  - bartowski/
  - mradermacher/

## Documentation
- Update PROJECT_PLAN.md with Wyoming Satellite architecture
- Update TODO.md with completed Wyoming integration tasks
- Add OPENCLAW_INTEGRATION.md for HA setup guide

## Testing
- Verified Wyoming services running (STT:10300, TTS:10301, Satellite:10700)
- Verified OpenClaw CLI accessibility
- Confirmed cross-platform compatibility fixes
This commit is contained in:
Aodhan Collins
2026-03-08 02:06:37 +00:00
parent 9eb5633115
commit 6a0bae2a0b
119 changed files with 780808 additions and 64 deletions

View File

@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Install OpenClaw Conversation custom component to Home Assistant
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
COMPONENT_NAME="openclaw_conversation"
# Detect Home Assistant config directory
if [[ -d "/config" ]]; then
HA_CONFIG="/config"
elif [[ -d "$HOME/.homeassistant" ]]; then
HA_CONFIG="$HOME/.homeassistant"
elif [[ -d "$HOME/homeassistant" ]]; then
HA_CONFIG="$HOME/homeassistant"
else
echo "Error: Could not find Home Assistant config directory"
echo "Please specify manually: ./install.sh /path/to/config"
exit 1
fi
# Allow override via command line
if [[ $# -ge 1 ]]; then
HA_CONFIG="$1"
fi
CUSTOM_COMPONENTS_DIR="$HA_CONFIG/custom_components"
TARGET_DIR="$CUSTOM_COMPONENTS_DIR/$COMPONENT_NAME"
echo "Installing OpenClaw Conversation custom component..."
echo " Source: $SCRIPT_DIR/$COMPONENT_NAME"
echo " Target: $TARGET_DIR"
echo ""
# Create custom_components directory if it doesn't exist
mkdir -p "$CUSTOM_COMPONENTS_DIR"
# Remove old installation if exists
if [[ -d "$TARGET_DIR" ]]; then
echo "Removing old installation..."
rm -rf "$TARGET_DIR"
fi
# Copy component files
cp -r "$SCRIPT_DIR/$COMPONENT_NAME" "$TARGET_DIR"
# Verify installation
if [[ -d "$TARGET_DIR" && -f "$TARGET_DIR/manifest.json" ]]; then
echo "✓ Installation successful!"
echo ""
echo "Next steps:"
echo " 1. Restart Home Assistant"
echo " 2. Go to Settings → Devices & Services → Add Integration"
echo " 3. Search for 'OpenClaw Conversation'"
echo " 4. Configure the settings (host: localhost, port: 8080)"
echo ""
echo " Or add to configuration.yaml:"
echo " openclaw_conversation:"
echo " openclaw_host: localhost"
echo " openclaw_port: 8080"
echo " agent_name: main"
echo " timeout: 30"
else
echo "✗ Installation failed"
exit 1
fi

View File

@@ -0,0 +1,114 @@
# OpenClaw Conversation - Home Assistant Custom Component
A custom conversation agent for Home Assistant that routes all voice/text queries to OpenClaw for processing.
## Features
- **Direct OpenClaw Integration**: Routes all conversation requests to OpenClaw
- **CLI-based Communication**: Uses the `openclaw` CLI command (fallback if HTTP API unavailable)
- **Configurable**: Set host, port, agent name, and timeout via UI
- **Voice Pipeline Compatible**: Works with Home Assistant's voice assistant pipeline
## Installation
### Method 1: Manual Copy
1. Copy the entire `openclaw_conversation` folder to your Home Assistant `custom_components` directory:
```bash
# On the HA host (if using HA OS or Container, use the File Editor add-on)
cp -r homeai-agent/custom_components/openclaw_conversation \
/config/custom_components/
```
2. Restart Home Assistant
3. Go to **Settings → Devices & Services → Add Integration**
4. Search for "OpenClaw Conversation"
5. Configure the settings:
- **OpenClaw Host**: `localhost` (or IP of Mac Mini)
- **OpenClaw Port**: `8080`
- **Agent Name**: `main` (or your configured agent)
- **Timeout**: `30` seconds
### Method 2: Using HACS (if available)
1. Add this repository to HACS as a custom repository
2. Install "OpenClaw Conversation"
3. Restart Home Assistant
## Configuration
### Via UI (Recommended)
After installation, configure via **Settings → Devices & Services → OpenClaw Conversation → Configure**.
### Via YAML (Alternative)
Add to your `configuration.yaml`:
```yaml
openclaw_conversation:
openclaw_host: localhost
openclaw_port: 8080
agent_name: main
timeout: 30
```
## Usage
Once configured, the OpenClaw agent will be available as a conversation agent in Home Assistant.
### Setting as Default Agent
1. Go to **Settings → Voice Assistants**
2. Edit your voice assistant pipeline
3. Set **Conversation Agent** to "OpenClaw Conversation"
4. Save
### Testing
1. Open the **Assist** panel in Home Assistant
2. Type a query like: "Turn on the reading lamp"
3. OpenClaw will process the request and return a response
## Architecture
```
[Voice Input] → [HA Voice Pipeline] → [OpenClaw Conversation Agent]
[OpenClaw CLI/API]
[Ollama LLM + Skills]
[HA Actions + TTS Response]
```
## Troubleshooting
### Agent Not Responding
1. Check OpenClaw is running: `pgrep -f openclaw`
2. Test CLI directly: `openclaw agent --message "Hello" --agent main`
3. Check HA logs: **Settings → System → Logs**
### Connection Errors
1. Verify OpenClaw host/port settings
2. Ensure OpenClaw is accessible from HA container/host
3. Check network connectivity: `curl http://localhost:8080/status`
## Files
| File | Purpose |
|------|---------|
| `manifest.json` | Component metadata |
| `__init__.py` | Component setup and registration |
| `config_flow.py` | Configuration UI flow |
| `const.py` | Constants and defaults |
| `conversation.py` | Conversation agent implementation |
| `strings.json` | UI translations |
## See Also
- [OpenClaw Integration Guide](../../skills/home-assistant/OPENCLAW_INTEGRATION.md)
- [Voice Pipeline Implementation](../../../plans/ha-voice-pipeline-implementation.md)

View File

@@ -0,0 +1,98 @@
"""OpenClaw Conversation integration for Home Assistant."""
from __future__ import annotations
import logging
from typing import Any
import voluptuous as vol
from homeassistant.config_entries import ConfigEntry
from homeassistant.const import Platform
from homeassistant.core import HomeAssistant
from homeassistant.helpers import config_validation as cv
from .const import (
CONF_AGENT_NAME,
CONF_OPENCLAW_HOST,
CONF_OPENCLAW_PORT,
CONF_TIMEOUT,
DEFAULT_AGENT,
DEFAULT_HOST,
DEFAULT_PORT,
DEFAULT_TIMEOUT,
DOMAIN,
)
from .conversation import OpenClawCLIAgent
_LOGGER = logging.getLogger(__name__)
PLATFORMS = [Platform.CONVERSATION]
CONFIG_SCHEMA = vol.Schema(
{
DOMAIN: vol.Schema(
{
vol.Optional(CONF_OPENCLAW_HOST, default=DEFAULT_HOST): cv.string,
vol.Optional(CONF_OPENCLAW_PORT, default=DEFAULT_PORT): cv.port,
vol.Optional(CONF_AGENT_NAME, default=DEFAULT_AGENT): cv.string,
vol.Optional(CONF_TIMEOUT, default=DEFAULT_TIMEOUT): cv.positive_int,
}
)
},
extra=vol.ALLOW_EXTRA,
)
async def async_setup(hass: HomeAssistant, config: dict[str, Any]) -> bool:
"""Set up the OpenClaw Conversation component."""
hass.data.setdefault(DOMAIN, {})
if DOMAIN not in config:
return True
conf = config[DOMAIN]
# Store config
hass.data[DOMAIN] = {
"config": conf,
}
# Register the conversation agent
agent = OpenClawCLIAgent(hass, conf)
# Add to conversation agent registry
from homeassistant.components import conversation
conversation.async_set_agent(hass, DOMAIN, agent)
_LOGGER.info("OpenClaw Conversation agent registered")
return True
async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
"""Set up OpenClaw Conversation from a config entry."""
hass.data.setdefault(DOMAIN, {})
# Store entry data
hass.data[DOMAIN][entry.entry_id] = entry.data
# Register the conversation agent
agent = OpenClawCLIAgent(hass, entry.data)
from homeassistant.components import conversation
conversation.async_set_agent(hass, DOMAIN, agent)
_LOGGER.info("OpenClaw Conversation agent registered from config entry")
return True
async def async_unload_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
"""Unload a config entry."""
# Unregister the conversation agent
from homeassistant.components import conversation
conversation.async_unset_agent(hass, DOMAIN)
hass.data[DOMAIN].pop(entry.entry_id, None)
return True

View File

@@ -0,0 +1,134 @@
"""Config flow for OpenClaw Conversation integration."""
from __future__ import annotations
import logging
from typing import Any
import voluptuous as vol
from homeassistant import config_entries
from homeassistant.const import CONF_HOST, CONF_PORT
from homeassistant.core import callback
from homeassistant.data_entry_flow import FlowResult
import homeassistant.helpers.config_validation as cv
from .const import (
CONF_AGENT_NAME,
CONF_OPENCLAW_HOST,
CONF_OPENCLAW_PORT,
CONF_TIMEOUT,
DEFAULT_AGENT,
DEFAULT_HOST,
DEFAULT_PORT,
DEFAULT_TIMEOUT,
DOMAIN,
)
_LOGGER = logging.getLogger(__name__)
STEP_USER_DATA_SCHEMA = vol.Schema(
{
vol.Optional(CONF_OPENCLAW_HOST, default=DEFAULT_HOST): cv.string,
vol.Optional(CONF_OPENCLAW_PORT, default=DEFAULT_PORT): cv.port,
vol.Optional(CONF_AGENT_NAME, default=DEFAULT_AGENT): cv.string,
vol.Optional(CONF_TIMEOUT, default=DEFAULT_TIMEOUT): cv.positive_int,
}
)
class ConfigFlow(config_entries.ConfigFlow, domain=DOMAIN):
"""Handle a config flow for OpenClaw Conversation."""
VERSION = 1
async def async_step_user(
self, user_input: dict[str, Any] | None = None
) -> FlowResult:
"""Handle the initial step."""
errors: dict[str, str] = {}
if user_input is not None:
# Validate the input
try:
# Test connection to OpenClaw
await self._test_openclaw_connection(
user_input[CONF_OPENCLAW_HOST],
user_input[CONF_OPENCLAW_PORT],
)
return self.async_create_entry(
title="OpenClaw Conversation",
data=user_input,
)
except ConnectionError:
errors["base"] = "cannot_connect"
except Exception:
_LOGGER.exception("Unexpected exception")
errors["base"] = "unknown"
return self.async_show_form(
step_id="user",
data_schema=STEP_USER_DATA_SCHEMA,
errors=errors,
)
async def _test_openclaw_connection(self, host: str, port: int) -> None:
"""Test if OpenClaw is reachable."""
import asyncio
try:
# Try to connect to OpenClaw
reader, writer = await asyncio.wait_for(
asyncio.open_connection(host, port),
timeout=5,
)
writer.close()
await writer.wait_closed()
except Exception as err:
raise ConnectionError(f"Cannot connect to OpenClaw: {err}")
@staticmethod
@callback
def async_get_options_flow(
config_entry: config_entries.ConfigEntry,
) -> OptionsFlow:
"""Create the options flow."""
return OptionsFlow(config_entry)
class OptionsFlow(config_entries.OptionsFlow):
"""Handle options flow for OpenClaw Conversation."""
def __init__(self, config_entry: config_entries.ConfigEntry) -> None:
"""Initialize options flow."""
self.config_entry = config_entry
async def async_step_init(
self, user_input: dict[str, Any] | None = None
) -> FlowResult:
"""Manage the options."""
errors: dict[str, str] = {}
if user_input is not None:
return self.async_create_entry(title="", data=user_input)
options = {
vol.Optional(
CONF_AGENT_NAME,
default=self.config_entry.options.get(
CONF_AGENT_NAME, DEFAULT_AGENT
),
): cv.string,
vol.Optional(
CONF_TIMEOUT,
default=self.config_entry.options.get(
CONF_TIMEOUT, DEFAULT_TIMEOUT
),
): cv.positive_int,
}
return self.async_show_form(
step_id="init",
data_schema=vol.Schema(options),
errors=errors,
)

View File

@@ -0,0 +1,26 @@
"""Constants for OpenClaw Conversation integration."""
DOMAIN = "openclaw_conversation"
# Configuration keys
CONF_OPENCLAW_HOST = "openclaw_host"
CONF_OPENCLAW_PORT = "openclaw_port"
CONF_AGENT_NAME = "agent_name"
CONF_TIMEOUT = "timeout"
# Defaults
DEFAULT_HOST = "localhost"
DEFAULT_PORT = 8080
DEFAULT_AGENT = "main"
DEFAULT_TIMEOUT = 30
# API endpoints
OPENCLAW_API_PATH = "/api/agent/message"
# Service names
SERVICE_PROCESS = "process"
# Attributes
ATTR_MESSAGE = "message"
ATTR_RESPONSE = "response"
ATTR_AGENT = "agent"

View File

@@ -0,0 +1,222 @@
"""Conversation agent for OpenClaw integration."""
from __future__ import annotations
import asyncio
import json
import logging
from typing import Any
import aiohttp
import voluptuous as vol
from homeassistant.components.conversation import (
AbstractConversationAgent,
ConversationInput,
ConversationResult,
)
from homeassistant.core import HomeAssistant
from homeassistant.helpers.aiohttp_client import async_get_clientsession
from homeassistant.helpers.intent import IntentResponse
from .const import (
CONF_AGENT_NAME,
CONF_OPENCLAW_HOST,
CONF_OPENCLAW_PORT,
CONF_TIMEOUT,
DEFAULT_AGENT,
DEFAULT_HOST,
DEFAULT_PORT,
DEFAULT_TIMEOUT,
DOMAIN,
OPENCLAW_API_PATH,
)
_LOGGER = logging.getLogger(__name__)
# Schema for configuration
CONFIG_SCHEMA = vol.Schema({
vol.Optional(CONF_OPENCLAW_HOST, default=DEFAULT_HOST): str,
vol.Optional(CONF_OPENCLAW_PORT, default=DEFAULT_PORT): int,
vol.Optional(CONF_AGENT_NAME, default=DEFAULT_AGENT): str,
vol.Optional(CONF_TIMEOUT, default=DEFAULT_TIMEOUT): int,
})
class OpenClawAgent(AbstractConversationAgent):
"""OpenClaw conversation agent."""
def __init__(self, hass: HomeAssistant, config: dict[str, Any]) -> None:
"""Initialize the agent."""
self.hass = hass
self.config = config
self.host = config.get(CONF_OPENCLAW_HOST, DEFAULT_HOST)
self.port = config.get(CONF_OPENCLAW_PORT, DEFAULT_PORT)
self.agent_name = config.get(CONF_AGENT_NAME, DEFAULT_AGENT)
self.timeout = config.get(CONF_TIMEOUT, DEFAULT_TIMEOUT)
@property
def supported_languages(self) -> list[str]:
"""Return a list of supported languages."""
return ["en"] # OpenClaw primarily supports English
@property
def attribution(self) -> dict[str, str] | None:
"""Return attribution information."""
return {
"name": "OpenClaw",
"url": "https://github.com/homeai/openclaw",
}
async def async_process(
self, user_input: ConversationInput
) -> ConversationResult:
"""Process a sentence and return a response."""
text = user_input.text
conversation_id = user_input.conversation_id
_LOGGER.debug("Processing message: %s", text)
try:
response_text = await self._call_openclaw(text)
# Create proper IntentResponse for Home Assistant
intent_response = IntentResponse(language=user_input.language or "en")
intent_response.async_set_speech(response_text)
return ConversationResult(
response=intent_response,
conversation_id=conversation_id,
)
except Exception as err:
_LOGGER.error("Error calling OpenClaw: %s", err)
intent_response = IntentResponse(language=user_input.language or "en")
intent_response.async_set_speech("I'm sorry, I encountered an error processing your request.")
return ConversationResult(
response=intent_response,
conversation_id=conversation_id,
)
async def _call_openclaw(self, message: str) -> str:
"""Call OpenClaw API and return the response."""
url = f"http://{self.host}:{self.port}{OPENCLAW_API_PATH}"
payload = {
"message": message,
"agent": self.agent_name,
}
session = async_get_clientsession(self.hass)
try:
async with asyncio.timeout(self.timeout):
async with session.post(
url,
json=payload,
headers={"Content-Type": "application/json"},
) as response:
if response.status != 200:
_LOGGER.error(
"OpenClaw returned status %s: %s",
response.status,
await response.text(),
)
return "I'm sorry, I couldn't process that request."
data = await response.json()
return data.get("response", "I didn't get a response.")
except asyncio.TimeoutError:
_LOGGER.error("Timeout calling OpenClaw")
return "I'm sorry, the request timed out."
except aiohttp.ClientError as err:
_LOGGER.error("Error connecting to OpenClaw: %s", err)
return "I'm sorry, I couldn't connect to the OpenClaw service."
class OpenClawCLIAgent(AbstractConversationAgent):
"""OpenClaw conversation agent using CLI (fallback if HTTP API unavailable)."""
def __init__(self, hass: HomeAssistant, config: dict[str, Any]) -> None:
"""Initialize the agent."""
self.hass = hass
self.config = config
self.agent_name = config.get(CONF_AGENT_NAME, DEFAULT_AGENT)
self.timeout = config.get(CONF_TIMEOUT, DEFAULT_TIMEOUT)
@property
def supported_languages(self) -> list[str]:
"""Return a list of supported languages."""
return ["en"]
@property
def attribution(self) -> dict[str, str] | None:
"""Return attribution information."""
return {
"name": "OpenClaw",
"url": "https://github.com/homeai/openclaw",
}
async def async_process(
self, user_input: ConversationInput
) -> ConversationResult:
"""Process a sentence using OpenClaw CLI."""
text = user_input.text
conversation_id = user_input.conversation_id
_LOGGER.debug("Processing message via CLI: %s", text)
try:
response_text = await self._call_openclaw_cli(text)
# Create proper IntentResponse for Home Assistant
intent_response = IntentResponse(language=user_input.language or "en")
intent_response.async_set_speech(response_text)
return ConversationResult(
response=intent_response,
conversation_id=conversation_id,
)
except Exception as err:
_LOGGER.error("Error calling OpenClaw CLI: %s", err)
intent_response = IntentResponse(language=user_input.language or "en")
intent_response.async_set_speech("I'm sorry, I encountered an error processing your request.")
return ConversationResult(
response=intent_response,
conversation_id=conversation_id,
)
async def _call_openclaw_cli(self, message: str) -> str:
"""Call OpenClaw CLI and return the response."""
import subprocess
cmd = [
"openclaw",
"agent",
"--message", message,
"--agent", self.agent_name,
]
try:
proc = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await asyncio.wait_for(
proc.communicate(),
timeout=self.timeout,
)
if proc.returncode != 0:
_LOGGER.error("OpenClaw CLI failed: %s", stderr.decode().strip())
return "I'm sorry, I couldn't process that request."
return stdout.decode().strip()
except asyncio.TimeoutError:
_LOGGER.error("Timeout calling OpenClaw CLI")
return "I'm sorry, the request timed out."
except FileNotFoundError:
_LOGGER.error("OpenClaw CLI not found")
return "I'm sorry, OpenClaw is not available."

View File

@@ -0,0 +1,11 @@
{
"domain": "openclaw_conversation",
"name": "OpenClaw Conversation",
"codeowners": ["@homeai"],
"config_flow": true,
"dependencies": ["conversation"],
"documentation": "https://github.com/homeai/homeai-agent",
"iot_class": "local_push",
"requirements": [],
"version": "1.0.0"
}

View File

@@ -0,0 +1,35 @@
{
"title": "OpenClaw Conversation",
"config": {
"step": {
"user": {
"title": "OpenClaw Conversation Setup",
"description": "Configure the OpenClaw conversation agent.",
"data": {
"openclaw_host": "OpenClaw Host",
"openclaw_port": "OpenClaw Port",
"agent_name": "Agent Name",
"timeout": "Timeout (seconds)"
}
}
},
"error": {
"cannot_connect": "Failed to connect to OpenClaw. Please check the host and port.",
"unknown": "Unexpected error occurred."
},
"abort": {
"already_configured": "OpenClaw Conversation is already configured."
}
},
"options": {
"step": {
"init": {
"title": "OpenClaw Conversation Options",
"data": {
"agent_name": "Agent Name",
"timeout": "Timeout (seconds)"
}
}
}
}
}

View File

@@ -0,0 +1,24 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.homeai.mem0-backup</string>
<key>ProgramArguments</key>
<array>
<string>/bin/bash</string>
<string>/Users/aodhan/gitea/homeai/homeai-agent/scripts/backup-memory.sh</string>
</array>
<key>StartCalendarInterval</key>
<dict>
<key>Hour</key>
<integer>3</integer>
<key>Minute</key>
<integer>0</integer>
</dict>
<key>StandardOutPath</key>
<string>/Users/aodhan/.openclaw/logs/mem0-backup.log</string>
<key>StandardErrorPath</key>
<string>/Users/aodhan/.openclaw/logs/mem0-backup.err</string>
</dict>
</plist>

View File

@@ -0,0 +1,29 @@
#!/bin/bash
# mem0 backup script
set -euo pipefail
MEMORY_DIR="/Users/aodhan/.openclaw/memory/"
# Check if directory exists
if [[ ! -d "$MEMORY_DIR" ]]; then
echo "Error: Memory directory $MEMORY_DIR does not exist" >&2
exit 1
fi
cd "$MEMORY_DIR"
# Check if git is initialized
if [[ ! -d ".git" ]]; then
echo "Error: Git not initialized in $MEMORY_DIR" >&2
exit 1
fi
# Check if there are changes to commit
if git diff --cached --quiet && git diff --quiet; then
echo "No changes to commit"
exit 0
fi
git add .
git commit -m "mem0 backup $(date)" || echo "Nothing to commit"
# git push # Uncomment when remote is configured

View File

@@ -0,0 +1,188 @@
# OpenClaw Integration for Home Assistant Voice Pipeline
> This document describes how to integrate OpenClaw with Home Assistant's voice pipeline using the Wyoming protocol.
## Architecture Overview
```
┌─────────────────────────────────────────────────────────────────────────┐
│ Voice Pipeline Flow │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ [Wyoming Satellite] [Home Assistant] [OpenClaw] │
│ │ │ │ │
│ │ 1. Wake word │ │ │
│ │ 2. Stream audio ───────>│ │ │
│ │ │ 3. Send to STT │ │
│ │ │ ────────────────> │ │
│ │ │ │ │
│ │ │ 4. Transcript │ │
│ │ │ <──────────────── │ │
│ │ │ │ │
│ │ │ 5. Conversation │ │
│ │ │ ────────────────> │ │
│ │ │ (via bridge) │ │
│ │ │ │ │
│ │ │ 6. Response │ │
│ │ │ <──────────────── │ │
│ │ │ │ │
│ │ 7. TTS audio <─────────│ │ │
│ │ │ │ │
│ [Speaker] │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
## Components
### 1. Wyoming Satellite (`com.homeai.wyoming-satellite.plist`)
- **Port**: 10700 (exposes satellite for HA to connect)
- **Function**: Handles audio I/O, wake word detection, streaming to HA
- **Audio**: Uses SoX (`rec`/`play`) for macOS audio capture/playback
- **Note**: Replaces the old `wakeword_daemon.py` - wake word is now handled by HA's voice pipeline
### 2. Wyoming STT (`com.homeai.wyoming-stt.plist`)
- **Port**: 10300 (Whisper large-v3)
- **Function**: Speech-to-text transcription
### 3. Wyoming TTS (`com.homeai.wyoming-tts.plist`)
- **Port**: 10301 (Kokoro ONNX)
- **Function**: Text-to-speech synthesis
### 4. OpenClaw Bridge (`openclaw_bridge.py`)
- **Function**: Connects HA conversation agent to OpenClaw CLI
- **Usage**: Called via HA `shell_command` or `command_line` integration
### Deprecated: Wake Word Daemon
The old `com.homeai.wakeword.plist` service has been **disabled**. It was trying to notify `http://localhost:8080/wake` which doesn't exist in OpenClaw. Wake word detection is now handled by the Wyoming satellite through Home Assistant's voice pipeline.
## Home Assistant Configuration
### Step 1: Add Wyoming Protocol Integration
1. Go to **Settings → Integrations → Add Integration**
2. Search for **Wyoming Protocol**
3. Add the following services:
| Service | Host | Port |
|---------|------|------|
| Speech-to-Text | `10.0.0.199` | `10300` |
| Text-to-Speech | `10.0.0.199` | `10301` |
| Satellite | `10.0.0.199` | `10700` |
### Step 2: Configure Voice Assistant Pipeline
1. Go to **Settings → Voice Assistants**
2. Create a new pipeline:
- **Name**: "HomeAI with OpenClaw"
- **Speech-to-Text**: Wyoming (localhost:10300)
- **Conversation Agent**: Home Assistant (or custom below)
- **Text-to-Speech**: Wyoming (localhost:10301)
### Step 3: Add OpenClaw Bridge to HA
Add to your `configuration.yaml`:
```yaml
shell_command:
openclaw_chat: 'python3 /Users/aodhan/gitea/homeai/homeai-agent/skills/home-assistant/openclaw_bridge.py "{{ message }}" --raw'
```
### Step 4: Create Automation for OpenClaw
Create an automation that routes voice commands to OpenClaw:
```yaml
automation:
- alias: "Voice Command via OpenClaw"
trigger:
- platform: conversation
command:
- "ask jarvis *"
action:
- service: shell_command.openclaw_chat
data:
message: "{{ trigger.slots.command }}"
response_variable: openclaw_response
- service: tts.speak
data:
media_player_entity_id: media_player.living_room_speaker
message: "{{ openclaw_response }}"
```
## Manual Testing
### Test STT
```bash
# Check if STT is running
nc -z localhost 10300 && echo "STT OK"
```
### Test TTS
```bash
# Check if TTS is running
nc -z localhost 10301 && echo "TTS OK"
```
### Test Satellite
```bash
# Check if satellite is running
nc -z localhost 10700 && echo "Satellite OK"
```
### Test OpenClaw Bridge
```bash
# Test the bridge directly
python3 homeai-agent/skills/home-assistant/openclaw_bridge.py "Turn on the living room lights"
```
### Test Full Pipeline
1. Load all services: `./homeai-voice/scripts/load-all-launchd.sh`
2. Open HA Assist panel (Settings → Voice Assistants → Assist)
3. Type or speak: "Turn on the study shelves light"
4. You should hear the TTS response
## Troubleshooting
### Satellite not connecting to HA
- Check that the satellite is running: `launchctl list com.homeai.wyoming-satellite`
- Check logs: `tail -f /tmp/homeai-wyoming-satellite.log`
- Verify HA can reach the satellite: Test from HA container/host
### No audio output
- Check SoX installation: `which play`
- Test SoX directly: `echo "test" | say` or `play /System/Library/Sounds/Glass.aiff`
- Check audio device permissions
### OpenClaw not responding
- Verify OpenClaw is running: `pgrep -f openclaw`
- Test CLI directly: `openclaw agent --message "Hello" --agent main`
- Check OpenClaw config: `cat ~/.openclaw/openclaw.json`
### Wyoming version conflicts
- The satellite requires wyoming 1.4.1 but faster-whisper requires 1.8+
- We've patched this - both should work with wyoming 1.8.0
- If issues occur, reinstall: `pip install 'wyoming>=1.8' wyoming-satellite`
## File Locations
| File | Purpose |
|------|---------|
| `~/.openclaw/openclaw.json` | OpenClaw configuration |
| `~/homeai-voice-env/` | Python virtual environment |
| `~/Library/LaunchAgents/com.homeai.*.plist` | Launchd services |
| `/tmp/homeai-*.log` | Service logs |
## Next Steps
1. [ ] Test voice pipeline end-to-end
2. [ ] Fine-tune wake word sensitivity
3. [ ] Add custom intents for OpenClaw
4. [ ] Implement conversation history/memory
5. [ ] Add ESP32 satellite support (P6)

View File

@@ -0,0 +1,91 @@
# Home Assistant Configuration for OpenClaw Integration
# Add these sections to your configuration.yaml
# ─── Shell Command Integration ────────────────────────────────────────────────
# This allows HA to call OpenClaw via shell commands
shell_command:
# Send a message to OpenClaw and get response
openclaw_chat: '/Users/aodhan/gitea/homeai/homeai-agent/skills/home-assistant/openclaw-bridge.sh "{{ message }}"'
# ─── REST Command (Alternative) ───────────────────────────────────────────────
# If OpenClaw exposes an HTTP API in the future
rest_command:
openclaw_chat:
url: "http://localhost:8080/api/agent/message"
method: POST
headers:
Authorization: "Bearer {{ token }}"
content_type: "application/json"
payload: '{"message": "{{ message }}", "agent": "main"}'
# ─── Command Line Sensor ──────────────────────────────────────────────────────
# Execute OpenClaw and return the response as a sensor
command_line:
- sensor:
name: "OpenClaw Response"
unique_id: openclaw_response
command: "/Users/aodhan/gitea/homeai/homeai-agent/skills/home-assistant/openclaw-bridge.sh '{{ states(\"input_text.openclaw_query\") }}'"
value_template: "{{ value_json.response }}"
scan_interval: 86400 # Only update when triggered
# ─── Input Text for Query ─────────────────────────────────────────────────────
input_text:
openclaw_query:
name: OpenClaw Query
initial: ""
max: 255
# ─── Conversation Agent Integration ────────────────────────────────────────────
# Custom conversation agent using OpenClaw
# This requires the custom conversation agent below
# ─── Intent Script ─────────────────────────────────────────────────────────────
intent_script:
# Handle conversation intents
OpenClawConversation:
speech:
text: "{{ response }}"
action:
- service: shell_command.openclaw_chat
data:
message: "{{ text }}"
response_variable: openclaw_result
- set:
response: "{{ openclaw_result }}"
# ─── Automation: Voice Pipeline with OpenClaw ─────────────────────────────────
automation:
- alias: "Voice Command via OpenClaw"
trigger:
- platform: conversation
command:
- "ask jarvis *"
action:
- service: shell_command.openclaw_chat
data:
message: "{{ trigger.slots.command }}"
response_variable: openclaw_response
- service: tts.speak
data:
media_player_entity_id: media_player.living_room_speaker
message: "{{ openclaw_response }}"
# ─── Wyoming Protocol Configuration ───────────────────────────────────────────
# Configure in HA UI:
# 1. Settings → Integrations → Add Integration → Wyoming Protocol
# 2. Add STT: host=10.0.0.199, port=10300
# 3. Add TTS: host=10.0.0.199, port=10301
# 4. Add Satellite: host=10.0.0.199, port=10700
# ─── Voice Assistant Pipeline ─────────────────────────────────────────────────
# Configure in HA UI:
# 1. Settings → Voice Assistants → Add Pipeline
# 2. Name: "HomeAI with OpenClaw"
# 3. Speech-to-Text: Wyoming (localhost:10300)
# 4. Conversation Agent: Use the automation above OR Home Assistant
# 5. Text-to-Speech: Wyoming (localhost:10301)
# ─── Custom Conversation Agent (Advanced) ─────────────────────────────────────
# Create a custom component in custom_components/openclaw_conversation/
# See: custom_components/openclaw_conversation/__init__.py

View File

@@ -0,0 +1,188 @@
#!/usr/bin/env python3
"""
ha-ctl — Home Assistant control CLI for OpenClaw agent.
Usage:
ha-ctl list [domain] List entities (optionally filtered by domain)
ha-ctl state <entity_id_or_name> Get current state of an entity
ha-ctl on <entity_id_or_name> Turn entity on
ha-ctl off <entity_id_or_name> Turn entity off
ha-ctl toggle <entity_id_or_name> Toggle entity
ha-ctl set <entity_id> <attr> <val> Set attribute (e.g. brightness 128)
ha-ctl scene <scene_name> Activate a scene
Environment:
HASS_TOKEN Long-lived access token
HA_URL Base URL (default: https://10.0.0.199:8123)
"""
import sys
import os
import json
import urllib.request
import urllib.error
import ssl
HA_URL = os.environ.get("HA_URL", "https://10.0.0.199:8123").rstrip("/")
TOKEN = os.environ.get("HASS_TOKEN") or os.environ.get("HA_TOKEN")
if not TOKEN:
token_file = os.path.expanduser("~/.homeai/hass_token")
if os.path.exists(token_file):
with open(token_file) as f:
TOKEN = f.read().strip()
if not TOKEN:
print("ERROR: No HASS_TOKEN set. Export HASS_TOKEN or write to ~/.homeai/hass_token", file=sys.stderr)
sys.exit(1)
# Skip SSL verification for self-signed certs on local HA
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
def api(method, path, data=None):
url = f"{HA_URL}/api{path}"
headers = {"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"}
body = json.dumps(data).encode() if data is not None else None
req = urllib.request.Request(url, data=body, headers=headers, method=method)
try:
with urllib.request.urlopen(req, context=ctx, timeout=10) as resp:
return json.loads(resp.read())
except urllib.error.HTTPError as e:
print(f"ERROR: HTTP {e.code} — {e.read().decode()}", file=sys.stderr)
sys.exit(1)
def get_states():
return api("GET", "/states")
def resolve_entity(query, states=None):
"""Resolve a friendly name or partial entity_id to a full entity_id."""
if states is None:
states = get_states()
query_lower = query.lower().replace("_", " ")
# Exact entity_id match
for s in states:
if s["entity_id"] == query:
return s["entity_id"]
# Friendly name exact match
for s in states:
name = s.get("attributes", {}).get("friendly_name", "").lower()
if name == query_lower:
return s["entity_id"]
# Partial entity_id match
for s in states:
if query_lower in s["entity_id"].lower():
return s["entity_id"]
# Partial friendly name match
matches = []
for s in states:
name = s.get("attributes", {}).get("friendly_name", "").lower()
if query_lower in name:
matches.append(s)
if len(matches) == 1:
return matches[0]["entity_id"]
if len(matches) > 1:
names = [f"{m['entity_id']} ({m['attributes'].get('friendly_name','')})" for m in matches]
print(f"Ambiguous: {', '.join(names)}", file=sys.stderr)
sys.exit(1)
print(f"ERROR: No entity found matching '{query}'", file=sys.stderr)
sys.exit(1)
def call_service(domain, service, entity_id, extra=None):
data = {"entity_id": entity_id}
if extra:
data.update(extra)
result = api("POST", f"/services/{domain}/{service}", data)
return result
def cmd_list(args):
domain_filter = args[0] if args else None
states = get_states()
if domain_filter:
states = [s for s in states if s["entity_id"].startswith(domain_filter + ".")]
for s in sorted(states, key=lambda x: x["entity_id"]):
name = s.get("attributes", {}).get("friendly_name", "")
print(f"{s['entity_id']}\t{s['state']}\t{name}")
def cmd_state(args):
if not args:
print("Usage: ha-ctl state <entity>", file=sys.stderr); sys.exit(1)
states = get_states()
eid = resolve_entity(args[0], states)
s = next(x for x in states if x["entity_id"] == eid)
print(f"Entity: {eid}")
print(f"State: {s['state']}")
attrs = s.get("attributes", {})
for k, v in attrs.items():
print(f" {k}: {v}")
def cmd_control(action, args):
if not args:
print(f"Usage: ha-ctl {action} <entity>", file=sys.stderr); sys.exit(1)
states = get_states()
eid = resolve_entity(args[0], states)
domain = eid.split(".")[0]
service_map = {"on": "turn_on", "off": "turn_off", "toggle": "toggle"}
service = service_map[action]
call_service(domain, service, eid)
name = next((x.get("attributes", {}).get("friendly_name", eid)
for x in states if x["entity_id"] == eid), eid)
print(f"OK: {service} → {name} ({eid})")
def cmd_scene(args):
if not args:
print("Usage: ha-ctl scene <name>", file=sys.stderr); sys.exit(1)
states = get_states()
eid = resolve_entity(args[0], states)
call_service("scene", "turn_on", eid)
print(f"OK: scene activated → {eid}")
def cmd_set(args):
if len(args) < 3:
print("Usage: ha-ctl set <entity> <attribute> <value>", file=sys.stderr); sys.exit(1)
states = get_states()
eid = resolve_entity(args[0], states)
domain = eid.split(".")[0]
attr, val = args[1], args[2]
try:
val = int(val)
except ValueError:
try:
val = float(val)
except ValueError:
pass
call_service(domain, "turn_on", eid, {attr: val})
print(f"OK: set {attr}={val} → {eid}")
if __name__ == "__main__":
args = sys.argv[1:]
if not args:
print(__doc__)
sys.exit(0)
cmd = args[0]
rest = args[1:]
if cmd == "list":
cmd_list(rest)
elif cmd == "state":
cmd_state(rest)
elif cmd in ("on", "off", "toggle"):
cmd_control(cmd, rest)
elif cmd == "scene":
cmd_scene(rest)
elif cmd == "set":
cmd_set(rest)
else:
print(f"Unknown command: {cmd}", file=sys.stderr)
print(__doc__)
sys.exit(1)

View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
# OpenClaw Bridge Script for Home Assistant
#
# Usage: ./openclaw-bridge.sh "message to send to OpenClaw"
# Returns: JSON response suitable for HA TTS
set -euo pipefail
MESSAGE="${1:-}"
AGENT="${2:-main}"
TIMEOUT="${3:-30}"
if [[ -z "$MESSAGE" ]]; then
echo '{"error": "No message provided"}' >&2
exit 1
fi
# Run OpenClaw agent and capture response
# The CLI outputs the response to stdout
RESPONSE=$(openclaw agent --message "$MESSAGE" --agent "$AGENT" 2>/dev/null || echo "Error: OpenClaw command failed")
# Output JSON for HA using jq for proper escaping
if command -v jq &>/dev/null; then
echo "$RESPONSE" | jq -Rs '{response: .}'
else
# Fallback: use Python for JSON encoding if jq is not available
python3 -c "import json,sys; print(json.dumps({'response': sys.stdin.read()}))" <<< "$RESPONSE"
fi

View File

@@ -0,0 +1,74 @@
#!/usr/bin/env python3
"""OpenClaw Bridge for Home Assistant
This script acts as a bridge between Home Assistant and OpenClaw.
It can be called from HA via shell_command or command_line integration.
Usage:
python openclaw_bridge.py "Your message here"
Output:
{"response": "OpenClaw's response text"}
"""
import argparse
import json
import subprocess
import sys
from pathlib import Path
def call_openclaw(message: str, agent: str = "main", timeout: int = 30) -> str:
"""Call OpenClaw CLI and return the response."""
try:
result = subprocess.run(
["openclaw", "agent", "--message", message, "--agent", agent],
capture_output=True,
text=True,
timeout=timeout,
)
if result.returncode != 0:
return f"Error: OpenClaw failed with code {result.returncode}"
# Return stdout (the response)
return result.stdout.strip()
except subprocess.TimeoutExpired:
return "Error: OpenClaw command timed out"
except FileNotFoundError:
return "Error: openclaw command not found. Is OpenClaw installed?"
except Exception as e:
return f"Error: {str(e)}"
def main():
parser = argparse.ArgumentParser(
description="Bridge between Home Assistant and OpenClaw"
)
parser.add_argument("message", help="Message to send to OpenClaw")
parser.add_argument(
"--agent", default="main", help="Agent to use (default: main)"
)
parser.add_argument(
"--timeout", type=int, default=30, help="Timeout in seconds (default: 30)"
)
parser.add_argument(
"--raw", action="store_true", help="Output raw text instead of JSON"
)
args = parser.parse_args()
# Call OpenClaw
response = call_openclaw(args.message, args.agent, args.timeout)
if args.raw:
print(response)
else:
# Output as JSON for HA
output = {"response": response}
print(json.dumps(output))
if __name__ == "__main__":
main()

36
homeai-agent/test_mem0.py Normal file
View File

@@ -0,0 +1,36 @@
import os
from mem0 import Memory
config = {
"vector_store": {
"provider": "chroma",
"config": {
"collection_name": "homeai_memory",
"path": os.path.expanduser("~/.openclaw/memory/chroma/"),
}
},
"llm": {
"provider": "ollama",
"config": {
"model": "qwen2.5:7b",
"ollama_base_url": "http://localhost:11434",
}
},
"embedder": {
"provider": "ollama",
"config": {
"model": "nomic-embed-text",
"ollama_base_url": "http://localhost:11434",
}
}
}
m = Memory.from_config(config)
# Test storing a memory
result = m.add("The user's favorite color is blue.", user_id="aodhan")
print(f"Store result: {result}")
# Test searching for the memory
search_results = m.search("What is the user's favorite color?", user_id="aodhan")
print(f"Search results: {search_results}")

View File

@@ -0,0 +1,114 @@
{
"nodes": [
{
"parameters": {
"rule": {
"interval": [
{
"field": "hours",
"minutes": 30,
"hours": 7
}
]
}
},
"id": "6e8b8c8a-8c8a-4c8a-8c8a-8c8a8c8a8c8a",
"name": "Schedule Trigger",
"type": "n8n-nodes-base.scheduleTrigger",
"typeVersion": 1,
"position": [
250,
300
]
},
{
"parameters": {
"method": "GET",
"url": "http://10.0.0.199:8123/api/states/weather.home",
"authentication": "predefinedCredentialType",
"nodeCredentialType": "homeAssistantApi",
"options": {}
},
"id": "7f9c9d9b-9d9b-5d9b-9d9b-9d9b9d9b9d9b",
"name": "Fetch Weather",
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.1,
"position": [
450,
300
]
},
{
"parameters": {
"jsCode": "const weather = $node[\"Fetch Weather\"].json;\nconst temp = weather.attributes.temperature;\nconst condition = weather.state;\nconst text = `Good morning! The current weather is ${condition} with a temperature of ${temp} degrees. Have a great day!`;\nreturn { briefing: text };"
},
"id": "8a0d0e0c-0e0c-6e0c-0e0c-0e0c0e0c0e0c",
"name": "Compose Briefing",
"type": "n8n-nodes-base.code",
"typeVersion": 2,
"position": [
650,
300
]
},
{
"parameters": {
"method": "POST",
"url": "http://localhost:8080/speak",
"sendBody": true,
"bodyParameters": {
"parameters": [
{
"name": "text",
"value": "={{ $json.briefing }}"
}
]
},
"options": {}
},
"id": "9b1e1f1d-1f1d-7f1d-1f1d-1f1d1f1d1f1d",
"name": "POST to OpenClaw",
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.1,
"position": [
850,
300
]
}
],
"connections": {
"Schedule Trigger": {
"main": [
[
{
"node": "Fetch Weather",
"type": "main",
"index": 0
}
]
]
},
"Fetch Weather": {
"main": [
[
{
"node": "Compose Briefing",
"type": "main",
"index": 0
}
]
]
},
"Compose Briefing": {
"main": [
[
{
"node": "POST to OpenClaw",
"type": "main",
"index": 0
}
]
]
}
}
}

View File

@@ -0,0 +1,107 @@
{
"nodes": [
{
"parameters": {
"httpMethod": "POST",
"path": "ha-notification",
"options": {}
},
"id": "a1b2c3d4-e5f6-4a5b-8c9d-0e1f2a3b4c5d",
"name": "HA Webhook",
"type": "n8n-nodes-base.webhook",
"typeVersion": 1,
"position": [
250,
300
]
},
{
"parameters": {
"conditions": {
"string": [
{
"value1": "={{ $json.body.urgency }}",
"value2": "high"
}
]
}
},
"id": "b2c3d4e5-f6a7-5b6c-9d0e-1f2a3b4c5d6e",
"name": "Classify Urgency",
"type": "n8n-nodes-base.if",
"typeVersion": 1,
"position": [
450,
300
]
},
{
"parameters": {
"method": "POST",
"url": "http://localhost:8080/speak",
"sendBody": true,
"bodyParameters": {
"parameters": [
{
"name": "text",
"value": "={{ $json.body.message }}"
}
]
},
"options": {}
},
"id": "c3d4e5f6-a7b8-6c7d-0e1f-2a3b4c5d6e7f",
"name": "TTS Immediately",
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.1,
"position": [
700,
200
]
},
{
"parameters": {
"jsCode": "return { status: 'queued', message: $json.body.message };"
},
"id": "d4e5f6a7-b8c9-7d8e-1f2a-3b4c5d6e7f8a",
"name": "Queue Notification",
"type": "n8n-nodes-base.code",
"typeVersion": 2,
"position": [
700,
400
]
}
],
"connections": {
"HA Webhook": {
"main": [
[
{
"node": "Classify Urgency",
"type": "main",
"index": 0
}
]
]
},
"Classify Urgency": {
"main": [
[
{
"node": "TTS Immediately",
"type": "main",
"index": 0
}
],
[
{
"node": "Queue Notification",
"type": "main",
"index": 1
}
]
]
}
}
}