feat: Music Assistant, Claude primary LLM, model tag in chat, setup.sh rewrite
- Deploy Music Assistant on Pi (10.0.0.199:8095) with host networking for Chromecast mDNS discovery, Spotify + SMB library support - Switch primary LLM from Ollama to Claude Sonnet 4 (Anthropic API), local models remain as fallback - Add model info tag under each assistant message in dashboard chat, persisted in conversation JSON - Rewrite homeai-agent/setup.sh: loads .env, injects API keys into plists, symlinks plists to ~/Library/LaunchAgents/, smoke tests services - Update install_service() in common.sh to use symlinks instead of copies - Open UFW ports on Pi for Music Assistant (8095, 8097, 8927) - Add ANTHROPIC_API_KEY to openclaw + bridge launchd plists Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Aodhan Collins
39
CLAUDE.md
39
CLAUDE.md
@@ -18,14 +18,15 @@ A self-hosted, always-on personal AI assistant running on a **Mac Mini M4 Pro (6
|
||||
| Storage | 1TB SSD |
|
||||
| Network | Gigabit Ethernet |
|
||||
|
||||
All AI inference runs locally on this machine. No cloud dependency required (cloud APIs optional).
|
||||
Primary LLM is Claude Sonnet 4 via Anthropic API. Local Ollama models available as fallback. All other inference (STT, TTS, image gen) runs locally.
|
||||
|
||||
---
|
||||
|
||||
## Core Stack
|
||||
|
||||
### AI & LLM
|
||||
- **Ollama** — local LLM runtime (target models: Llama 3.3 70B, Qwen 2.5 72B)
|
||||
- **Claude Sonnet 4** — primary LLM via Anthropic API (`anthropic/claude-sonnet-4-20250514`), used for all agent interactions
|
||||
- **Ollama** — local LLM runtime (fallback models: Llama 3.3 70B, Qwen 3.5 35B-A3B, Qwen 2.5 7B)
|
||||
- **Model keep-warm daemon** — `preload-models.sh` runs as a loop, checks every 5 min, re-pins evicted models with `keep_alive=-1`. Keeps `qwen2.5:7b` (small/fast) and `$HOMEAI_MEDIUM_MODEL` (default: `qwen3.5:35b-a3b`) always loaded in VRAM. Medium model is configurable via env var for per-persona model assignment.
|
||||
- **Open WebUI** — browser-based chat interface, runs as Docker container
|
||||
|
||||
@@ -45,13 +46,15 @@ All AI inference runs locally on this machine. No cloud dependency required (clo
|
||||
### Smart Home
|
||||
- **Home Assistant** — smart home control platform (Docker)
|
||||
- **Wyoming Protocol** — bridges Whisper STT + Kokoro/Piper TTS into Home Assistant
|
||||
- **Music Assistant** — self-hosted music control, integrates with Home Assistant
|
||||
- **Music Assistant** — self-hosted music control (Docker on Pi at 10.0.0.199:8095), Spotify + SMB library + Chromecast players
|
||||
- **Snapcast** — multi-room synchronised audio output
|
||||
|
||||
### AI Agent / Orchestration
|
||||
- **OpenClaw** — primary AI agent layer; receives voice commands, calls tools, manages personality
|
||||
- **OpenClaw Skills** — 13 skills total: home-assistant, image-generation, voice-assistant, vtube-studio, memory, service-monitor, character, routine, music, workflow, gitea, calendar, mode
|
||||
- **n8n** — visual workflow automation (Docker), chains AI actions
|
||||
- **Character Memory System** — two-tier JSON-based memories (personal per-character + general shared), injected into LLM system prompt with budget truncation
|
||||
- **Public/Private Mode** — routes requests to local Ollama (private) or cloud LLMs (public) with per-category overrides via `active-mode.json`. Default primary model is Claude Sonnet 4.
|
||||
|
||||
### Character & Personality
|
||||
- **Character Schema v2** — JSON spec with background, dialogue_style, appearance, skills, gaze_presets (v1 auto-migrated)
|
||||
@@ -93,7 +96,8 @@ ESP32-S3-BOX-3 (room)
|
||||
→ HA conversation agent → OpenClaw HTTP Bridge
|
||||
→ Bridge resolves character (satellite_id → character mapping)
|
||||
→ Bridge builds system prompt (profile + memories) and writes TTS config to state file
|
||||
→ OpenClaw CLI → Ollama LLM generates response
|
||||
→ Bridge checks active-mode.json for model routing (private=local, public=cloud)
|
||||
→ OpenClaw CLI → LLM generates response (Claude Sonnet 4 default, Ollama fallback)
|
||||
→ Response dispatched:
|
||||
→ Wyoming TTS reads state file → routes to Kokoro (local) or ElevenLabs (cloud)
|
||||
→ Audio sent back to ESP32-S3-BOX-3 (spoken response)
|
||||
@@ -154,11 +158,13 @@ This works for both ESP32/HA pipeline and dashboard chat.
|
||||
4. **OpenClaw** — installed, onboarded, connected to Ollama and Home Assistant ✅
|
||||
5. **ESP32-S3-BOX-3** — ESPHome flash, Wyoming Satellite, display faces ✅
|
||||
6. **Character system** — schema v2, dashboard editor, memory system, per-character TTS routing ✅
|
||||
7. **Animated visual** — PNG/GIF character visual for the web assistant (initial visual layer)
|
||||
8. **Android app** — companion app for mobile access to the assistant
|
||||
9. **ComfyUI** — image generation online, character-consistent model workflows
|
||||
10. **Extended integrations** — n8n workflows, Music Assistant, Snapcast, Gitea, code-server
|
||||
11. **Polish** — Authelia, Tailscale hardening, iOS widgets
|
||||
7. **OpenClaw skills expansion** — 9 new skills (memory, monitor, character, routine, music, workflow, gitea, calendar, mode) + public/private mode routing ✅
|
||||
8. **Music Assistant** — deployed on Pi (10.0.0.199:8095), Spotify + SMB + Chromecast players ✅
|
||||
9. **Animated visual** — PNG/GIF character visual for the web assistant (initial visual layer)
|
||||
10. **Android app** — companion app for mobile access to the assistant
|
||||
11. **ComfyUI** — image generation online, character-consistent model workflows
|
||||
12. **Extended integrations** — Snapcast, code-server
|
||||
13. **Polish** — Authelia, Tailscale hardening, iOS widgets
|
||||
|
||||
### Stretch Goals
|
||||
- **Live2D / VTube Studio** — full Live2D model with WebSocket API bridge (requires learning Live2D tooling)
|
||||
@@ -167,18 +173,27 @@ This works for both ESP32/HA pipeline and dashboard chat.
|
||||
|
||||
## Key Paths & Conventions
|
||||
|
||||
- All Docker compose files: `~/server/docker/`
|
||||
- Launchd plists (source): `homeai-*/launchd/` (symlinked to `~/Library/LaunchAgents/`)
|
||||
- Docker compose (Mac Mini): `homeai-infra/docker/docker-compose.yml`
|
||||
- Docker compose (Pi/SELBINA): `~/docker/selbina/` on 10.0.0.199
|
||||
- OpenClaw skills: `~/.openclaw/skills/`
|
||||
- OpenClaw workspace tools: `~/.openclaw/workspace/TOOLS.md`
|
||||
- OpenClaw config: `~/.openclaw/openclaw.json`
|
||||
- Character configs: `~/homeai-data/characters/`
|
||||
- Character memories: `~/homeai-data/memories/`
|
||||
- Conversation history: `~/homeai-data/conversations/`
|
||||
- Active TTS state: `~/homeai-data/active-tts-voice.json`
|
||||
- Active mode state: `~/homeai-data/active-mode.json`
|
||||
- Satellite → character map: `~/homeai-data/satellite-map.json`
|
||||
- Local routines: `~/homeai-data/routines/`
|
||||
- Voice reminders: `~/homeai-data/reminders.json`
|
||||
- Whisper models: `~/models/whisper/`
|
||||
- Ollama models: managed by Ollama at `~/.ollama/models/`
|
||||
- ComfyUI models: `~/ComfyUI/models/`
|
||||
- Voice reference audio: `~/voices/`
|
||||
- Gitea repos root: `~/gitea/`
|
||||
- Music Assistant (Pi): `~/docker/selbina/music-assistant/` on 10.0.0.199
|
||||
- Skills user guide: `homeai-agent/SKILLS_GUIDE.md`
|
||||
|
||||
---
|
||||
|
||||
@@ -188,6 +203,8 @@ This works for both ESP32/HA pipeline and dashboard chat.
|
||||
- ESP32-S3-BOX-3 units are dumb satellites — all intelligence stays on Mac Mini
|
||||
- The character JSON schema (from Character Manager) should be treated as a versioned spec; pipeline components read from it, never hardcode personality values
|
||||
- OpenClaw skills are the primary extension mechanism — new capabilities = new skills
|
||||
- Prefer local models; cloud API keys (Anthropic, OpenAI) are fallback only
|
||||
- Primary LLM is Claude Sonnet 4 (Anthropic API); local Ollama models are available as fallback
|
||||
- Launchd plists are symlinked from repo source to ~/Library/LaunchAgents/ — edit source, then bootout/bootstrap to reload
|
||||
- Music Assistant runs on Pi (10.0.0.199), not Mac Mini — needs host networking for Chromecast mDNS discovery
|
||||
- VTube Studio API bridge should be a standalone OpenClaw skill with clear event interface
|
||||
- mem0 memory store should be backed up as part of regular Gitea commits
|
||||
|
||||
10
TODO.md
10
TODO.md
@@ -16,7 +16,7 @@
|
||||
- [x] `docker compose up -d` — bring all services up
|
||||
- [x] Home Assistant onboarding — long-lived access token generated, stored in `.env`
|
||||
- [ ] Install Tailscale, verify all services reachable on Tailnet
|
||||
- [ ] Uptime Kuma: add monitors for all services, configure mobile alerts
|
||||
- [x] Uptime Kuma: add monitors for all services, configure mobile alerts
|
||||
- [ ] Verify all containers survive a cold reboot
|
||||
|
||||
### P2 · homeai-llm
|
||||
@@ -30,7 +30,7 @@
|
||||
- [x] Deploy Open WebUI via Docker compose (port 3030)
|
||||
- [x] Verify Open WebUI connected to Ollama, all models available
|
||||
- [x] Run pipeline benchmark (homeai-voice/scripts/benchmark_pipeline.py) — STT/LLM/TTS latency profiled
|
||||
- [ ] Add Ollama + Open WebUI to Uptime Kuma monitors
|
||||
- [x] Add Ollama + Open WebUI to Uptime Kuma monitors
|
||||
|
||||
---
|
||||
|
||||
@@ -56,7 +56,7 @@
|
||||
- [ ] Install Chatterbox TTS (MPS build), test with sample `.wav`
|
||||
- [ ] Install Qwen3-TTS via MLX (fallback)
|
||||
- [ ] Train custom wake word using character name
|
||||
- [ ] Add Wyoming STT/TTS to Uptime Kuma monitors
|
||||
- [x] Add Wyoming STT/TTS to Uptime Kuma monitors
|
||||
|
||||
---
|
||||
|
||||
@@ -197,8 +197,8 @@
|
||||
|
||||
### P10 · Integrations & Polish
|
||||
|
||||
- [ ] Deploy Music Assistant (Docker), integrate with Home Assistant
|
||||
- [ ] Write `skills/music` SKILL.md for OpenClaw
|
||||
- [x] Deploy Music Assistant (Docker on Pi 10.0.0.199:8095), Spotify + SMB + Chromecast
|
||||
- [x] Write `skills/music` SKILL.md for OpenClaw
|
||||
- [ ] Deploy Snapcast server on Mac Mini
|
||||
- [ ] Configure Snapcast clients on ESP32 units for multi-room audio
|
||||
- [ ] Configure Authelia as 2FA layer in front of web UIs
|
||||
|
||||
@@ -1,37 +1,38 @@
|
||||
# P4: homeai-agent — AI Agent, Skills & Automation
|
||||
|
||||
> Phase 3 | Depends on: P1 (HA), P2 (Ollama), P3 (Wyoming/TTS), P5 (character JSON)
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
OpenClaw running as the primary AI agent: receives voice/text input, loads character persona, calls tools (skills), manages memory (mem0), dispatches responses (TTS, HA actions, VTube expressions). n8n handles scheduled/automated workflows.
|
||||
> Phase 4 | Depends on: P1 (HA), P2 (Ollama), P3 (Wyoming/TTS), P5 (character JSON)
|
||||
> Status: **COMPLETE** (all skills implemented)
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Voice input (text from P3 Wyoming STT)
|
||||
Voice input (text from Wyoming STT via HA pipeline)
|
||||
↓
|
||||
OpenClaw API (port 8080)
|
||||
↓ loads character JSON from P5
|
||||
System prompt construction
|
||||
↓
|
||||
Ollama LLM (P2) — llama3.3:70b
|
||||
↓ response + tool calls
|
||||
Skill dispatcher
|
||||
├── home_assistant.py → HA REST API (P1)
|
||||
├── memory.py → mem0 (local)
|
||||
├── vtube_studio.py → VTube WS (P7)
|
||||
├── comfyui.py → ComfyUI API (P8)
|
||||
├── music.py → Music Assistant (Phase 7)
|
||||
└── weather.py → HA sensor data
|
||||
OpenClaw HTTP Bridge (port 8081)
|
||||
↓ resolves character, loads memories, checks mode
|
||||
System prompt construction (profile + memories)
|
||||
↓ checks active-mode.json for model routing
|
||||
OpenClaw CLI → LLM (Ollama local or cloud API)
|
||||
↓ response + tool calls via exec
|
||||
Skill dispatcher (CLIs on PATH)
|
||||
├── ha-ctl → Home Assistant REST API
|
||||
├── memory-ctl → JSON memory files
|
||||
├── monitor-ctl → service health checks
|
||||
├── character-ctl → character switching
|
||||
├── routine-ctl → scenes, scripts, multi-step routines
|
||||
├── music-ctl → media player control
|
||||
├── workflow-ctl → n8n workflow triggering
|
||||
├── gitea-ctl → Gitea repo/issue queries
|
||||
├── calendar-ctl → HA calendar + voice reminders
|
||||
├── mode-ctl → public/private LLM routing
|
||||
├── gaze-ctl → image generation
|
||||
└── vtube-ctl → VTube Studio expressions
|
||||
↓ final response text
|
||||
TTS dispatch:
|
||||
├── Chatterbox (voice clone, if active)
|
||||
└── Kokoro (via Wyoming, fallback)
|
||||
TTS dispatch (via active-tts-voice.json):
|
||||
├── Kokoro (local, Wyoming)
|
||||
└── ElevenLabs (cloud API)
|
||||
↓
|
||||
Audio playback to appropriate room
|
||||
```
|
||||
@@ -40,296 +41,148 @@ OpenClaw API (port 8080)
|
||||
|
||||
## OpenClaw Setup
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
# Confirm OpenClaw supports Ollama — check repo for latest install method
|
||||
pip install openclaw
|
||||
# or
|
||||
git clone https://github.com/<openclaw-repo>/openclaw
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
**Key question:** Verify OpenClaw's Ollama/OpenAI-compatible backend support before installation. If OpenClaw doesn't support local Ollama natively, use a thin adapter layer pointing its OpenAI endpoint at `http://localhost:11434/v1`.
|
||||
|
||||
### Config — `~/.openclaw/config.yaml`
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
|
||||
llm:
|
||||
provider: ollama # or openai-compatible
|
||||
base_url: http://localhost:11434/v1
|
||||
model: llama3.3:70b
|
||||
fast_model: qwen2.5:7b # used for quick intent classification
|
||||
|
||||
character:
|
||||
active: aria
|
||||
config_dir: ~/.openclaw/characters/
|
||||
|
||||
memory:
|
||||
provider: mem0
|
||||
store_path: ~/.openclaw/memory/
|
||||
embedding_model: nomic-embed-text
|
||||
embedding_url: http://localhost:11434/v1
|
||||
|
||||
api:
|
||||
host: 0.0.0.0
|
||||
port: 8080
|
||||
|
||||
tts:
|
||||
primary: chatterbox # when voice clone active
|
||||
fallback: kokoro-wyoming # Wyoming TTS endpoint
|
||||
wyoming_tts_url: tcp://localhost:10301
|
||||
|
||||
wake:
|
||||
endpoint: /wake # openWakeWord POSTs here to trigger listening
|
||||
```
|
||||
- **Runtime:** Node.js global install at `/opt/homebrew/bin/openclaw` (v2026.3.2)
|
||||
- **Config:** `~/.openclaw/openclaw.json`
|
||||
- **Gateway:** port 8080, mode local, launchd: `com.homeai.openclaw`
|
||||
- **Default model:** `ollama/qwen3.5:35b-a3b` (MoE, 35B total, 3B active, 26.7 tok/s)
|
||||
- **Cloud models (public mode):** `anthropic/claude-sonnet-4-20250514`, `openai/gpt-4o`
|
||||
- **Critical:** `commands.native: true` in config (enables exec tool for CLI skills)
|
||||
- **Critical:** `contextWindow: 32768` for large models (prevents GPU OOM)
|
||||
|
||||
---
|
||||
|
||||
## Skills
|
||||
## Skills (13 total)
|
||||
|
||||
All skills live in `~/.openclaw/skills/` (symlinked from `homeai-agent/skills/`).
|
||||
All skills follow the same pattern:
|
||||
- `~/.openclaw/skills/<name>/SKILL.md` — metadata + agent instructions
|
||||
- `~/.openclaw/skills/<name>/<tool>` — executable Python CLI (stdlib only)
|
||||
- Symlinked to `/opt/homebrew/bin/` for PATH access
|
||||
- Agent invokes via `exec` tool
|
||||
- Documented in `~/.openclaw/workspace/TOOLS.md`
|
||||
|
||||
### `home_assistant.py`
|
||||
### Existing Skills (4)
|
||||
|
||||
Wraps the HA REST API for common smart home actions.
|
||||
| Skill | CLI | Description |
|
||||
|-------|-----|-------------|
|
||||
| home-assistant | `ha-ctl` | Smart home device control |
|
||||
| image-generation | `gaze-ctl` | Image generation via ComfyUI/GAZE |
|
||||
| voice-assistant | (none) | Voice pipeline handling |
|
||||
| vtube-studio | `vtube-ctl` | VTube Studio expression control |
|
||||
|
||||
**Functions:**
|
||||
- `turn_on(entity_id, **kwargs)` — lights, switches, media players
|
||||
- `turn_off(entity_id)`
|
||||
- `toggle(entity_id)`
|
||||
- `set_light(entity_id, brightness=None, color_temp=None, rgb_color=None)`
|
||||
- `run_scene(scene_id)`
|
||||
- `get_state(entity_id)` → returns state + attributes
|
||||
- `list_entities(domain=None)` → returns entity list
|
||||
### New Skills (9) — Added 2026-03-17
|
||||
|
||||
Uses `HA_URL` and `HA_TOKEN` from `.env.services`.
|
||||
| Skill | CLI | Description |
|
||||
|-------|-----|-------------|
|
||||
| memory | `memory-ctl` | Store/search/recall memories |
|
||||
| service-monitor | `monitor-ctl` | Service health checks |
|
||||
| character | `character-ctl` | Character switching |
|
||||
| routine | `routine-ctl` | Scenes and multi-step routines |
|
||||
| music | `music-ctl` | Media player control |
|
||||
| workflow | `workflow-ctl` | n8n workflow management |
|
||||
| gitea | `gitea-ctl` | Gitea repo/issue/PR queries |
|
||||
| calendar | `calendar-ctl` | Calendar events and voice reminders |
|
||||
| mode | `mode-ctl` | Public/private LLM routing |
|
||||
|
||||
### `memory.py`
|
||||
|
||||
Wraps mem0 for persistent long-term memory.
|
||||
|
||||
**Functions:**
|
||||
- `remember(text, category=None)` — store a memory
|
||||
- `recall(query, limit=5)` — semantic search over memories
|
||||
- `forget(memory_id)` — delete a specific memory
|
||||
- `list_recent(n=10)` — list most recent memories
|
||||
|
||||
mem0 uses `nomic-embed-text` via Ollama for embeddings.
|
||||
|
||||
### `weather.py`
|
||||
|
||||
Pulls weather data from Home Assistant sensors (local weather station or HA weather integration).
|
||||
|
||||
**Functions:**
|
||||
- `get_current()` → temp, humidity, conditions
|
||||
- `get_forecast(days=3)` → forecast array
|
||||
|
||||
### `timer.py`
|
||||
|
||||
Simple timer/reminder management.
|
||||
|
||||
**Functions:**
|
||||
- `set_timer(duration_seconds, label=None)` → fires HA notification/TTS on expiry
|
||||
- `set_reminder(datetime_str, message)` → schedules future TTS playback
|
||||
- `list_timers()`
|
||||
- `cancel_timer(timer_id)`
|
||||
|
||||
### `music.py` (stub — completed in Phase 7)
|
||||
|
||||
```python
|
||||
def play(query: str): ... # "play jazz" → Music Assistant
|
||||
def pause(): ...
|
||||
def skip(): ...
|
||||
def set_volume(level: int): ... # 0-100
|
||||
```
|
||||
|
||||
### `vtube_studio.py` (implemented in P7)
|
||||
|
||||
Stub in P4, full implementation in P7:
|
||||
```python
|
||||
def trigger_expression(event: str): ... # "thinking", "happy", etc.
|
||||
def set_parameter(name: str, value: float): ...
|
||||
```
|
||||
|
||||
### `comfyui.py` (implemented in P8)
|
||||
|
||||
Stub in P4, full implementation in P8:
|
||||
```python
|
||||
def generate(workflow: str, params: dict) -> str: ... # returns image path
|
||||
```
|
||||
See `SKILLS_GUIDE.md` for full user documentation.
|
||||
|
||||
---
|
||||
|
||||
## mem0 — Long-Term Memory
|
||||
## HTTP Bridge
|
||||
|
||||
### Setup
|
||||
**File:** `openclaw-http-bridge.py` (runs in homeai-voice-env)
|
||||
**Port:** 8081, launchd: `com.homeai.openclaw-bridge`
|
||||
|
||||
```bash
|
||||
pip install mem0ai
|
||||
```
|
||||
|
||||
### Config
|
||||
|
||||
```python
|
||||
from mem0 import Memory
|
||||
|
||||
config = {
|
||||
"llm": {
|
||||
"provider": "ollama",
|
||||
"config": {
|
||||
"model": "llama3.3:70b",
|
||||
"ollama_base_url": "http://localhost:11434",
|
||||
}
|
||||
},
|
||||
"embedder": {
|
||||
"provider": "ollama",
|
||||
"config": {
|
||||
"model": "nomic-embed-text",
|
||||
"ollama_base_url": "http://localhost:11434",
|
||||
}
|
||||
},
|
||||
"vector_store": {
|
||||
"provider": "chroma",
|
||||
"config": {
|
||||
"collection_name": "homeai_memory",
|
||||
"path": "~/.openclaw/memory/chroma",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
memory = Memory.from_config(config)
|
||||
```
|
||||
|
||||
> **Decision point:** Start with Chroma (local file-based). If semantic recall quality is poor, migrate to Qdrant (Docker container).
|
||||
|
||||
### Backup
|
||||
|
||||
Daily cron (via launchd) commits mem0 data to Gitea:
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
cd ~/.openclaw/memory
|
||||
git add .
|
||||
git commit -m "mem0 backup $(date +%Y-%m-%d)"
|
||||
git push origin main
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## n8n Workflows
|
||||
|
||||
n8n runs in Docker (deployed in P1). Workflows exported as JSON and stored in `homeai-agent/workflows/`.
|
||||
|
||||
### Starter Workflows
|
||||
|
||||
**`morning-briefing.json`**
|
||||
- Trigger: time-based (e.g., 7:30 AM on weekdays)
|
||||
- Steps: fetch weather → fetch calendar events → compose briefing → POST to OpenClaw TTS → speak aloud
|
||||
|
||||
**`notification-router.json`**
|
||||
- Trigger: HA webhook (new notification)
|
||||
- Steps: classify urgency → if high: TTS immediately; if low: queue for next interaction
|
||||
|
||||
**`memory-backup.json`**
|
||||
- Trigger: daily schedule
|
||||
- Steps: commit mem0 data to Gitea
|
||||
|
||||
### n8n ↔ OpenClaw Integration
|
||||
|
||||
OpenClaw exposes a webhook endpoint that n8n can call to trigger TTS or run a skill:
|
||||
|
||||
```
|
||||
POST http://localhost:8080/speak
|
||||
{
|
||||
"text": "Good morning. It is 7:30 and the weather is...",
|
||||
"room": "all"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Surface (OpenClaw)
|
||||
|
||||
Key endpoints consumed by other projects:
|
||||
### Endpoints
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|---|---|---|
|
||||
| `/chat` | POST | Send text, get response (+ fires skills) |
|
||||
| `/wake` | POST | Wake word trigger from openWakeWord |
|
||||
| `/speak` | POST | TTS only — no LLM, just speak text |
|
||||
| `/skill/<name>` | POST | Call a specific skill directly |
|
||||
| `/memory` | GET/POST | Read/write memories |
|
||||
|----------|--------|-------------|
|
||||
| `/api/agent/message` | POST | Send message → LLM → response |
|
||||
| `/api/tts` | POST | Text-to-speech (Kokoro or ElevenLabs) |
|
||||
| `/api/stt` | POST | Speech-to-text (Wyoming/Whisper) |
|
||||
| `/wake` | POST | Wake word notification |
|
||||
| `/status` | GET | Health check |
|
||||
|
||||
---
|
||||
### Request Flow
|
||||
|
||||
## Directory Layout
|
||||
1. Resolve character: explicit `character_id` > `satellite_id` mapping > default
|
||||
2. Build system prompt: profile fields + metadata + personal/general memories
|
||||
3. Write TTS config to `active-tts-voice.json`
|
||||
4. Load mode from `active-mode.json`, resolve model (private → local, public → cloud)
|
||||
5. Call OpenClaw CLI with `--model` flag if public mode
|
||||
6. Detect/re-prompt if model promises action but doesn't call exec tool
|
||||
7. Return response
|
||||
|
||||
```
|
||||
homeai-agent/
|
||||
├── skills/
|
||||
│ ├── home_assistant.py
|
||||
│ ├── memory.py
|
||||
│ ├── weather.py
|
||||
│ ├── timer.py
|
||||
│ ├── music.py # stub
|
||||
│ ├── vtube_studio.py # stub
|
||||
│ └── comfyui.py # stub
|
||||
├── workflows/
|
||||
│ ├── morning-briefing.json
|
||||
│ ├── notification-router.json
|
||||
│ └── memory-backup.json
|
||||
└── config/
|
||||
├── config.yaml.example
|
||||
└── mem0-config.py
|
||||
```
|
||||
### Timeout Strategy
|
||||
|
||||
| State | Timeout |
|
||||
|-------|---------|
|
||||
| Model warm (loaded in VRAM) | 120s |
|
||||
| Model cold (loading) | 180s |
|
||||
|
||||
---
|
||||
|
||||
## Interface Contracts
|
||||
## Daemons
|
||||
|
||||
**Consumes:**
|
||||
- Ollama API: `http://localhost:11434/v1`
|
||||
- HA API: `$HA_URL` with `$HA_TOKEN`
|
||||
- Wyoming TTS: `tcp://localhost:10301`
|
||||
- Character JSON: `~/.openclaw/characters/<active>.json` (from P5)
|
||||
|
||||
**Exposes:**
|
||||
- OpenClaw HTTP API: `http://localhost:8080` — consumed by P3 (voice), P7 (visual triggers), P8 (image skill)
|
||||
|
||||
**Add to `.env.services`:**
|
||||
```dotenv
|
||||
OPENCLAW_URL=http://localhost:8080
|
||||
```
|
||||
| Daemon | Plist | Purpose |
|
||||
|--------|-------|---------|
|
||||
| `com.homeai.openclaw` | `launchd/com.homeai.openclaw.plist` | OpenClaw gateway (port 8080) |
|
||||
| `com.homeai.openclaw-bridge` | `launchd/com.homeai.openclaw-bridge.plist` | HTTP bridge (port 8081) |
|
||||
| `com.homeai.reminder-daemon` | `launchd/com.homeai.reminder-daemon.plist` | Voice reminder checker (60s interval) |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Steps
|
||||
## Data Files
|
||||
|
||||
- [ ] Confirm OpenClaw installation method and Ollama compatibility
|
||||
- [ ] Install OpenClaw, write `config.yaml` pointing at Ollama and HA
|
||||
- [ ] Verify OpenClaw responds to a basic text query via `/chat`
|
||||
- [ ] Write `home_assistant.py` skill — test lights on/off via voice
|
||||
- [ ] Write `memory.py` skill — test store and recall
|
||||
- [ ] Write `weather.py` skill — verify HA weather sensor data
|
||||
- [ ] Write `timer.py` skill — test set/fire a timer
|
||||
- [ ] Write skill stubs: `music.py`, `vtube_studio.py`, `comfyui.py`
|
||||
- [ ] Set up mem0 with Chroma backend, test semantic recall
|
||||
- [ ] Write and test memory backup launchd job
|
||||
- [ ] Deploy n8n via Docker (P1 task if not done)
|
||||
- [ ] Build morning briefing n8n workflow
|
||||
- [ ] Symlink `homeai-agent/skills/` → `~/.openclaw/skills/`
|
||||
- [ ] Verify full voice → agent → HA action flow (with P3 pipeline)
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `~/homeai-data/memories/personal/*.json` | Per-character memories |
|
||||
| `~/homeai-data/memories/general.json` | Shared general memories |
|
||||
| `~/homeai-data/characters/*.json` | Character profiles (schema v2) |
|
||||
| `~/homeai-data/satellite-map.json` | Satellite → character mapping |
|
||||
| `~/homeai-data/active-tts-voice.json` | Current TTS engine/voice |
|
||||
| `~/homeai-data/active-mode.json` | Public/private mode state |
|
||||
| `~/homeai-data/routines/*.json` | Local routine definitions |
|
||||
| `~/homeai-data/reminders.json` | Pending voice reminders |
|
||||
| `~/homeai-data/conversations/*.json` | Chat conversation history |
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
## Environment Variables (OpenClaw Plist)
|
||||
|
||||
- [ ] "Turn on the living room lights" → lights turn on via HA
|
||||
- [ ] "Remember that I prefer jazz in the mornings" → mem0 stores it; "What do I like in the mornings?" → recalls it
|
||||
- [ ] Morning briefing n8n workflow fires on schedule and speaks via TTS
|
||||
- [ ] OpenClaw `/status` returns healthy
|
||||
- [ ] OpenClaw survives Mac Mini reboot (launchd or Docker — TBD based on OpenClaw's preferred run method)
|
||||
| Variable | Purpose |
|
||||
|----------|---------|
|
||||
| `HASS_TOKEN` / `HA_TOKEN` | Home Assistant API token |
|
||||
| `HA_URL` | Home Assistant URL |
|
||||
| `GAZE_API_KEY` | Image generation API key |
|
||||
| `N8N_API_KEY` | n8n automation API key |
|
||||
| `GITEA_TOKEN` | Gitea API token |
|
||||
| `ANTHROPIC_API_KEY` | Claude API key (public mode) |
|
||||
| `OPENAI_API_KEY` | OpenAI API key (public mode) |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Status
|
||||
|
||||
- [x] OpenClaw installed and configured
|
||||
- [x] HTTP bridge with character resolution and memory injection
|
||||
- [x] ha-ctl — smart home control
|
||||
- [x] gaze-ctl — image generation
|
||||
- [x] vtube-ctl — VTube Studio expressions
|
||||
- [x] memory-ctl — memory store/search/recall
|
||||
- [x] monitor-ctl — service health checks
|
||||
- [x] character-ctl — character switching
|
||||
- [x] routine-ctl — scenes and multi-step routines
|
||||
- [x] music-ctl — media player control
|
||||
- [x] workflow-ctl — n8n workflow triggering
|
||||
- [x] gitea-ctl — Gitea integration
|
||||
- [x] calendar-ctl — calendar + voice reminders
|
||||
- [x] mode-ctl — public/private LLM routing
|
||||
- [x] Bridge mode routing (active-mode.json → --model flag)
|
||||
- [x] Cloud providers in openclaw.json (Anthropic, OpenAI)
|
||||
- [x] Dashboard /api/mode endpoint
|
||||
- [x] Reminder daemon (com.homeai.reminder-daemon)
|
||||
- [x] TOOLS.md updated with all skills
|
||||
- [ ] Set N8N_API_KEY (requires generating in n8n UI)
|
||||
- [ ] Set GITEA_TOKEN (requires generating in Gitea UI)
|
||||
- [ ] Set ANTHROPIC_API_KEY / OPENAI_API_KEY for public mode
|
||||
- [ ] End-to-end voice test of each skill
|
||||
|
||||
386
homeai-agent/SKILLS_GUIDE.md
Normal file
386
homeai-agent/SKILLS_GUIDE.md
Normal file
@@ -0,0 +1,386 @@
|
||||
# OpenClaw Skills — User Guide
|
||||
|
||||
> All skills are invoked by voice or chat. Say a natural command and the AI agent will route it to the right tool automatically.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Skill | CLI | What it does |
|
||||
|-------|-----|-------------|
|
||||
| Home Assistant | `ha-ctl` | Control lights, switches, sensors, climate |
|
||||
| Image Generation | `gaze-ctl` | Generate images via ComfyUI/GAZE |
|
||||
| Memory | `memory-ctl` | Store and recall things about you |
|
||||
| Service Monitor | `monitor-ctl` | Check if services are running |
|
||||
| Character Switcher | `character-ctl` | Switch AI personalities |
|
||||
| Routines & Scenes | `routine-ctl` | Create and trigger multi-step automations |
|
||||
| Music | `music-ctl` | Play, pause, skip, volume control |
|
||||
| n8n Workflows | `workflow-ctl` | Trigger automation workflows |
|
||||
| Gitea | `gitea-ctl` | Query repos, commits, issues |
|
||||
| Calendar & Reminders | `calendar-ctl` | View calendar, set voice reminders |
|
||||
| Public/Private Mode | `mode-ctl` | Route to local or cloud LLMs |
|
||||
|
||||
---
|
||||
|
||||
## Phase A — Core Skills
|
||||
|
||||
### Memory (`memory-ctl`)
|
||||
|
||||
The agent can remember things about you and recall them later. Memories persist across conversations and are visible in the dashboard.
|
||||
|
||||
**Voice examples:**
|
||||
- "Remember that my favorite color is blue"
|
||||
- "I take my coffee black"
|
||||
- "What do you know about me?"
|
||||
- "Forget that I said I like jazz"
|
||||
|
||||
**CLI usage:**
|
||||
```bash
|
||||
memory-ctl add personal "User's favorite color is blue" --category preference
|
||||
memory-ctl add general "Living room speaker is a Sonos" --category fact
|
||||
memory-ctl search "coffee"
|
||||
memory-ctl list --type personal
|
||||
memory-ctl delete <memory_id>
|
||||
```
|
||||
|
||||
**Categories:** `preference`, `fact`, `routine`
|
||||
|
||||
**How it works:** Memories are stored as JSON in `~/homeai-data/memories/`. Personal memories are per-character (each character has their own relationship with you). General memories are shared across all characters.
|
||||
|
||||
---
|
||||
|
||||
### Service Monitor (`monitor-ctl`)
|
||||
|
||||
Ask the assistant if everything is healthy, check specific services, or see what models are loaded.
|
||||
|
||||
**Voice examples:**
|
||||
- "Is everything running?"
|
||||
- "What models are loaded?"
|
||||
- "Is Home Assistant up?"
|
||||
- "Show me the Docker containers"
|
||||
|
||||
**CLI usage:**
|
||||
```bash
|
||||
monitor-ctl status # Full health check (all services)
|
||||
monitor-ctl check ollama # Single service
|
||||
monitor-ctl ollama # Models loaded, VRAM usage
|
||||
monitor-ctl docker # Docker container status
|
||||
```
|
||||
|
||||
**Services checked:** Ollama, OpenClaw Bridge, OpenClaw Gateway, Wyoming STT, Wyoming TTS, Dashboard, n8n, Uptime Kuma, Home Assistant, Gitea
|
||||
|
||||
---
|
||||
|
||||
### Character Switcher (`character-ctl`)
|
||||
|
||||
Switch between AI personalities on the fly. Each character has their own voice, personality, and memories.
|
||||
|
||||
**Voice examples:**
|
||||
- "Talk to Aria"
|
||||
- "Switch to Sucy"
|
||||
- "Who can I talk to?"
|
||||
- "Who am I talking to?"
|
||||
- "Tell me about Aria"
|
||||
|
||||
**CLI usage:**
|
||||
```bash
|
||||
character-ctl list # See all characters
|
||||
character-ctl active # Who is the current default
|
||||
character-ctl switch "Aria" # Switch (fuzzy name matching)
|
||||
character-ctl info "Sucy" # Character profile
|
||||
character-ctl map homeai-kitchen.local aria_123 # Map a satellite to a character
|
||||
```
|
||||
|
||||
**How it works:** Switching updates the default character in `satellite-map.json` and writes the TTS voice config. The new character takes effect on the next request.
|
||||
|
||||
---
|
||||
|
||||
## Phase B — Home Assistant Extensions
|
||||
|
||||
### Routines & Scenes (`routine-ctl`)
|
||||
|
||||
Create and trigger Home Assistant scenes and multi-step routines by voice.
|
||||
|
||||
**Voice examples:**
|
||||
- "Activate movie mode"
|
||||
- "Run the bedtime routine"
|
||||
- "What scenes do I have?"
|
||||
- "Create a morning routine"
|
||||
|
||||
**CLI usage:**
|
||||
```bash
|
||||
routine-ctl list-scenes # HA scenes
|
||||
routine-ctl list-scripts # HA scripts
|
||||
routine-ctl trigger "movie_mode" # Activate scene/script
|
||||
routine-ctl create-scene "cozy" --entities '[{"entity_id":"light.lamp","state":"on","brightness":80}]'
|
||||
routine-ctl create-routine "bedtime" --steps '[
|
||||
{"type":"ha","cmd":"off \"All Lights\""},
|
||||
{"type":"delay","seconds":2},
|
||||
{"type":"tts","text":"Good night!"}
|
||||
]'
|
||||
routine-ctl run "bedtime" # Execute routine
|
||||
routine-ctl list-routines # List local routines
|
||||
routine-ctl delete-routine "bedtime" # Remove routine
|
||||
```
|
||||
|
||||
**Step types:**
|
||||
| Type | Description | Fields |
|
||||
|------|-------------|--------|
|
||||
| `scene` | Trigger an HA scene | `target` (scene name) |
|
||||
| `ha` | Run an ha-ctl command | `cmd` (e.g. `off "Lamp"`) |
|
||||
| `delay` | Wait between steps | `seconds` |
|
||||
| `tts` | Speak text aloud | `text` |
|
||||
|
||||
**Storage:** Routines are saved as JSON in `~/homeai-data/routines/`.
|
||||
|
||||
---
|
||||
|
||||
### Music Control (`music-ctl`)
|
||||
|
||||
Control music playback through Home Assistant media players — works with Spotify, Music Assistant, Chromecast, and any HA media player.
|
||||
|
||||
**Voice examples:**
|
||||
- "Play some jazz"
|
||||
- "Pause the music"
|
||||
- "Next song"
|
||||
- "What's playing?"
|
||||
- "Turn the volume to 50"
|
||||
- "Play Bohemian Rhapsody on the kitchen speaker"
|
||||
- "Shuffle on"
|
||||
|
||||
**CLI usage:**
|
||||
```bash
|
||||
music-ctl players # List available players
|
||||
music-ctl play "jazz" # Search and play
|
||||
music-ctl play # Resume paused playback
|
||||
music-ctl pause # Pause
|
||||
music-ctl next # Skip to next
|
||||
music-ctl prev # Go to previous
|
||||
music-ctl volume 50 # Set volume (0-100)
|
||||
music-ctl now-playing # Current track info
|
||||
music-ctl shuffle on # Enable shuffle
|
||||
music-ctl play "rock" --player media_player.kitchen # Target specific player
|
||||
```
|
||||
|
||||
**How it works:** All commands go through HA's `media_player` services. The `--player` flag defaults to the first active (playing/paused) player. Multi-room audio works through Snapcast zones, which appear as separate `media_player` entities.
|
||||
|
||||
**Prerequisites:** At least one media player configured in Home Assistant (Spotify integration, Music Assistant, or Chromecast).
|
||||
|
||||
---
|
||||
|
||||
## Phase C — External Service Skills
|
||||
|
||||
### n8n Workflows (`workflow-ctl`)
|
||||
|
||||
List and trigger n8n automation workflows by voice.
|
||||
|
||||
**Voice examples:**
|
||||
- "Run the backup workflow"
|
||||
- "What workflows do I have?"
|
||||
- "Did the last workflow succeed?"
|
||||
|
||||
**CLI usage:**
|
||||
```bash
|
||||
workflow-ctl list # All workflows
|
||||
workflow-ctl trigger "backup" # Trigger by name (fuzzy match)
|
||||
workflow-ctl trigger "abc123" --data '{"key":"val"}' # Trigger with data
|
||||
workflow-ctl status <execution_id> # Check execution result
|
||||
workflow-ctl history --limit 5 # Recent executions
|
||||
```
|
||||
|
||||
**Setup required:**
|
||||
1. Generate an API key in n8n: Settings → API → Create API Key
|
||||
2. Set `N8N_API_KEY` in the OpenClaw launchd plist
|
||||
3. Restart OpenClaw: `launchctl kickstart -k gui/501/com.homeai.openclaw`
|
||||
|
||||
---
|
||||
|
||||
### Gitea (`gitea-ctl`)
|
||||
|
||||
Query your self-hosted Gitea repositories, commits, issues, and pull requests.
|
||||
|
||||
**Voice examples:**
|
||||
- "What repos do I have?"
|
||||
- "Show recent commits for homeai"
|
||||
- "Any open issues?"
|
||||
- "Create an issue for the TTS bug"
|
||||
|
||||
**CLI usage:**
|
||||
```bash
|
||||
gitea-ctl repos # List all repos
|
||||
gitea-ctl commits aodhan/homeai --limit 5 # Recent commits
|
||||
gitea-ctl issues aodhan/homeai --state open # Open issues
|
||||
gitea-ctl prs aodhan/homeai # Pull requests
|
||||
gitea-ctl create-issue aodhan/homeai "Bug title" --body "Description here"
|
||||
```
|
||||
|
||||
**Setup required:**
|
||||
1. Generate a token in Gitea: Settings → Applications → Generate Token
|
||||
2. Set `GITEA_TOKEN` in the OpenClaw launchd plist
|
||||
3. Restart OpenClaw
|
||||
|
||||
---
|
||||
|
||||
### Calendar & Reminders (`calendar-ctl`)
|
||||
|
||||
Read calendar events from Home Assistant and set voice reminders that speak via TTS when due.
|
||||
|
||||
**Voice examples:**
|
||||
- "What's on my calendar today?"
|
||||
- "What's coming up this week?"
|
||||
- "Remind me in 30 minutes to check the oven"
|
||||
- "Remind me at 5pm to call mum"
|
||||
- "What reminders do I have?"
|
||||
- "Cancel that reminder"
|
||||
|
||||
**CLI usage:**
|
||||
```bash
|
||||
calendar-ctl today # Today's events
|
||||
calendar-ctl upcoming --days 3 # Next 3 days
|
||||
calendar-ctl add "Dentist" --start 2026-03-18T14:00:00 --end 2026-03-18T15:00:00
|
||||
calendar-ctl remind "Check the oven" --at "in 30 minutes"
|
||||
calendar-ctl remind "Call mum" --at "at 5pm"
|
||||
calendar-ctl remind "Team standup" --at "tomorrow 9am"
|
||||
calendar-ctl reminders # List pending
|
||||
calendar-ctl cancel-reminder <id> # Cancel
|
||||
```
|
||||
|
||||
**Supported time formats:**
|
||||
| Format | Example |
|
||||
|--------|---------|
|
||||
| Relative | `in 30 minutes`, `in 2 hours` |
|
||||
| Absolute | `at 5pm`, `at 17:00`, `at 5:30pm` |
|
||||
| Tomorrow | `tomorrow 9am`, `tomorrow at 14:00` |
|
||||
| Combined | `in 1 hour 30 minutes` |
|
||||
|
||||
**How reminders work:** A background daemon (`com.homeai.reminder-daemon`) checks `~/homeai-data/reminders.json` every 60 seconds. When a reminder is due, it POSTs to the TTS bridge and speaks the reminder aloud. Fired reminders are automatically cleaned up after 24 hours.
|
||||
|
||||
**Prerequisites:** Calendar entity configured in Home Assistant (Google Calendar, CalDAV, or local calendar integration).
|
||||
|
||||
---
|
||||
|
||||
## Phase D — Public/Private Mode
|
||||
|
||||
### Mode Controller (`mode-ctl`)
|
||||
|
||||
Route AI requests to local LLMs (private, no data leaves the machine) or cloud LLMs (public, faster/more capable) with per-category overrides.
|
||||
|
||||
**Voice examples:**
|
||||
- "Switch to public mode"
|
||||
- "Go private"
|
||||
- "What mode am I in?"
|
||||
- "Use Claude for coding"
|
||||
- "Keep health queries private"
|
||||
|
||||
**CLI usage:**
|
||||
```bash
|
||||
mode-ctl status # Current mode and overrides
|
||||
mode-ctl private # All requests → local Ollama
|
||||
mode-ctl public # All requests → cloud LLM
|
||||
mode-ctl set-provider anthropic # Use Claude (default)
|
||||
mode-ctl set-provider openai # Use GPT-4o
|
||||
mode-ctl override coding public # Always use cloud for coding
|
||||
mode-ctl override health private # Always keep health local
|
||||
mode-ctl list-overrides # Show all category rules
|
||||
```
|
||||
|
||||
**Default category rules:**
|
||||
|
||||
| Always Private | Always Public | Follows Global Mode |
|
||||
|---------------|--------------|-------------------|
|
||||
| Personal finance | Web search | General chat |
|
||||
| Health | Coding help | Smart home |
|
||||
| Passwords | Complex reasoning | Music |
|
||||
| Private conversations | Translation | Calendar |
|
||||
|
||||
**How it works:** The HTTP bridge reads `~/homeai-data/active-mode.json` before each request. Based on the mode and any category overrides, it passes `--model` to the OpenClaw CLI to route to either `ollama/qwen3.5:35b-a3b` (private) or `anthropic/claude-sonnet-4-20250514` / `openai/gpt-4o` (public).
|
||||
|
||||
**Setup required for public mode:**
|
||||
1. Set `ANTHROPIC_API_KEY` and/or `OPENAI_API_KEY` in the OpenClaw launchd plist
|
||||
2. Restart OpenClaw: `launchctl kickstart -k gui/501/com.homeai.openclaw`
|
||||
|
||||
**Dashboard:** The mode can also be toggled via the dashboard API at `GET/POST /api/mode`.
|
||||
|
||||
---
|
||||
|
||||
## Administration
|
||||
|
||||
### Adding API Keys
|
||||
|
||||
All API keys are stored in the OpenClaw launchd plist at:
|
||||
```
|
||||
~/gitea/homeai/homeai-agent/launchd/com.homeai.openclaw.plist
|
||||
```
|
||||
|
||||
After editing, deploy and restart:
|
||||
```bash
|
||||
cp ~/gitea/homeai/homeai-agent/launchd/com.homeai.openclaw.plist ~/Library/LaunchAgents/
|
||||
launchctl kickstart -k gui/501/com.homeai.openclaw
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Purpose | Required for |
|
||||
|----------|---------|-------------|
|
||||
| `HASS_TOKEN` | Home Assistant API token | ha-ctl, routine-ctl, music-ctl, calendar-ctl |
|
||||
| `HA_URL` | Home Assistant URL | Same as above |
|
||||
| `GAZE_API_KEY` | Image generation API key | gaze-ctl |
|
||||
| `N8N_API_KEY` | n8n automation API key | workflow-ctl |
|
||||
| `GITEA_TOKEN` | Gitea API token | gitea-ctl |
|
||||
| `ANTHROPIC_API_KEY` | Claude API key | mode-ctl (public mode) |
|
||||
| `OPENAI_API_KEY` | OpenAI API key | mode-ctl (public mode) |
|
||||
|
||||
### Skill File Locations
|
||||
|
||||
```
|
||||
~/.openclaw/skills/
|
||||
├── home-assistant/ ha-ctl → /opt/homebrew/bin/ha-ctl
|
||||
├── image-generation/ gaze-ctl → /opt/homebrew/bin/gaze-ctl
|
||||
├── memory/ memory-ctl → /opt/homebrew/bin/memory-ctl
|
||||
├── service-monitor/ monitor-ctl → /opt/homebrew/bin/monitor-ctl
|
||||
├── character/ character-ctl → /opt/homebrew/bin/character-ctl
|
||||
├── routine/ routine-ctl → /opt/homebrew/bin/routine-ctl
|
||||
├── music/ music-ctl → /opt/homebrew/bin/music-ctl
|
||||
├── workflow/ workflow-ctl → /opt/homebrew/bin/workflow-ctl
|
||||
├── gitea/ gitea-ctl → /opt/homebrew/bin/gitea-ctl
|
||||
├── calendar/ calendar-ctl → /opt/homebrew/bin/calendar-ctl
|
||||
├── mode/ mode-ctl → /opt/homebrew/bin/mode-ctl
|
||||
├── voice-assistant/ (no CLI)
|
||||
└── vtube-studio/ vtube-ctl → /opt/homebrew/bin/vtube-ctl
|
||||
```
|
||||
|
||||
### Data File Locations
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `~/homeai-data/memories/personal/*.json` | Per-character memories |
|
||||
| `~/homeai-data/memories/general.json` | Shared general memories |
|
||||
| `~/homeai-data/characters/*.json` | Character profiles |
|
||||
| `~/homeai-data/satellite-map.json` | Satellite → character mapping |
|
||||
| `~/homeai-data/active-tts-voice.json` | Current TTS voice config |
|
||||
| `~/homeai-data/active-mode.json` | Public/private mode state |
|
||||
| `~/homeai-data/routines/*.json` | Local routine definitions |
|
||||
| `~/homeai-data/reminders.json` | Pending voice reminders |
|
||||
| `~/homeai-data/conversations/*.json` | Chat conversation history |
|
||||
|
||||
### Creating a New Skill
|
||||
|
||||
Every skill follows the same pattern:
|
||||
|
||||
1. Create directory: `~/.openclaw/skills/<name>/`
|
||||
2. Write `SKILL.md` with YAML frontmatter (`name`, `description`) + usage docs
|
||||
3. Create Python CLI (stdlib only: `urllib.request`, `json`, `os`, `sys`, `re`, `datetime`)
|
||||
4. `chmod +x` the CLI and symlink to `/opt/homebrew/bin/`
|
||||
5. Add env vars to the OpenClaw launchd plist if needed
|
||||
6. Add a section to `~/.openclaw/workspace/TOOLS.md`
|
||||
7. Restart OpenClaw: `launchctl kickstart -k gui/501/com.homeai.openclaw`
|
||||
8. Test: `openclaw agent --message "test prompt" --agent main`
|
||||
|
||||
### Daemons
|
||||
|
||||
| Daemon | Plist | Purpose |
|
||||
|--------|-------|---------|
|
||||
| `com.homeai.reminder-daemon` | `homeai-agent/launchd/com.homeai.reminder-daemon.plist` | Fires TTS reminders when due |
|
||||
| `com.homeai.openclaw` | `homeai-agent/launchd/com.homeai.openclaw.plist` | OpenClaw gateway |
|
||||
| `com.homeai.openclaw-bridge` | `homeai-agent/launchd/com.homeai.openclaw-bridge.plist` | HTTP bridge (voice pipeline) |
|
||||
| `com.homeai.preload-models` | `homeai-llm/scripts/preload-models.sh` | Keeps models warm in VRAM |
|
||||
@@ -37,6 +37,8 @@
|
||||
<string>/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
|
||||
<key>ELEVENLABS_API_KEY</key>
|
||||
<string>sk_ec10e261c6190307a37aa161a9583504dcf25a0cabe5dbd5</string>
|
||||
<key>ANTHROPIC_API_KEY</key>
|
||||
<string>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</string>
|
||||
</dict>
|
||||
</dict>
|
||||
</plist>
|
||||
|
||||
@@ -30,6 +30,18 @@
|
||||
<string>eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJmZGQ1NzZlYWNkMTU0ZTY2ODY1OTkzYTlhNTIxM2FmNyIsImlhdCI6MTc3MjU4ODYyOCwiZXhwIjoyMDg3OTQ4NjI4fQ.CTAU1EZgpVLp_aRnk4vg6cQqwS5N-p8jQkAAXTxFmLY</string>
|
||||
<key>GAZE_API_KEY</key>
|
||||
<string>e63401f17e4845e1059f830267f839fe7fc7b6083b1cb1730863318754d799f4</string>
|
||||
<key>N8N_URL</key>
|
||||
<string>http://localhost:5678</string>
|
||||
<key>N8N_API_KEY</key>
|
||||
<string></string>
|
||||
<key>GITEA_URL</key>
|
||||
<string>http://10.0.0.199:3000</string>
|
||||
<key>GITEA_TOKEN</key>
|
||||
<string></string>
|
||||
<key>ANTHROPIC_API_KEY</key>
|
||||
<string>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</string>
|
||||
<key>OPENAI_API_KEY</key>
|
||||
<string></string>
|
||||
</dict>
|
||||
|
||||
<key>RunAtLoad</key>
|
||||
|
||||
30
homeai-agent/launchd/com.homeai.reminder-daemon.plist
Normal file
30
homeai-agent/launchd/com.homeai.reminder-daemon.plist
Normal file
@@ -0,0 +1,30 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
|
||||
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>com.homeai.reminder-daemon</string>
|
||||
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/Users/aodhan/homeai-voice-env/bin/python3</string>
|
||||
<string>/Users/aodhan/gitea/homeai/homeai-agent/reminder-daemon.py</string>
|
||||
</array>
|
||||
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
|
||||
<key>KeepAlive</key>
|
||||
<true/>
|
||||
|
||||
<key>StandardOutPath</key>
|
||||
<string>/tmp/homeai-reminder-daemon.log</string>
|
||||
|
||||
<key>StandardErrorPath</key>
|
||||
<string>/tmp/homeai-reminder-daemon-error.log</string>
|
||||
|
||||
<key>ThrottleInterval</key>
|
||||
<integer>10</integer>
|
||||
</dict>
|
||||
</plist>
|
||||
@@ -48,6 +48,7 @@ TIMEOUT_WARM = 120 # Model already loaded in VRAM
|
||||
TIMEOUT_COLD = 180 # Model needs loading first (~10-20s load + inference)
|
||||
OLLAMA_PS_URL = "http://localhost:11434/api/ps"
|
||||
VTUBE_BRIDGE_URL = "http://localhost:8002"
|
||||
DEFAULT_MODEL = "anthropic/claude-sonnet-4-20250514"
|
||||
|
||||
|
||||
def _vtube_fire_and_forget(path: str, data: dict):
|
||||
@@ -83,6 +84,31 @@ CHARACTERS_DIR = Path("/Users/aodhan/homeai-data/characters")
|
||||
SATELLITE_MAP_PATH = Path("/Users/aodhan/homeai-data/satellite-map.json")
|
||||
MEMORIES_DIR = Path("/Users/aodhan/homeai-data/memories")
|
||||
ACTIVE_TTS_VOICE_PATH = Path("/Users/aodhan/homeai-data/active-tts-voice.json")
|
||||
ACTIVE_MODE_PATH = Path("/Users/aodhan/homeai-data/active-mode.json")
|
||||
|
||||
# Cloud provider model mappings for mode routing
|
||||
CLOUD_MODELS = {
|
||||
"anthropic": "anthropic/claude-sonnet-4-20250514",
|
||||
"openai": "openai/gpt-4o",
|
||||
}
|
||||
|
||||
|
||||
def load_mode() -> dict:
|
||||
"""Load the public/private mode configuration."""
|
||||
try:
|
||||
with open(ACTIVE_MODE_PATH) as f:
|
||||
return json.load(f)
|
||||
except Exception:
|
||||
return {"mode": "private", "cloud_provider": "anthropic", "overrides": {}}
|
||||
|
||||
|
||||
def resolve_model(mode_data: dict) -> str | None:
|
||||
"""Resolve which model to use based on mode. Returns None for default (private/local)."""
|
||||
mode = mode_data.get("mode", "private")
|
||||
if mode == "private":
|
||||
return None # Use OpenClaw default (ollama/qwen3.5:35b-a3b)
|
||||
provider = mode_data.get("cloud_provider", "anthropic")
|
||||
return CLOUD_MODELS.get(provider, CLOUD_MODELS["anthropic"])
|
||||
|
||||
|
||||
def clean_text_for_tts(text: str) -> str:
|
||||
@@ -505,10 +531,13 @@ class OpenClawBridgeHandler(BaseHTTPRequestHandler):
|
||||
self._send_json_response(200, {"status": "ok", "message": "Wake word received"})
|
||||
|
||||
@staticmethod
|
||||
def _call_openclaw(message: str, agent: str, timeout: int) -> str:
|
||||
def _call_openclaw(message: str, agent: str, timeout: int, model: str = None) -> str:
|
||||
"""Call OpenClaw CLI and return stdout."""
|
||||
cmd = ["/opt/homebrew/bin/openclaw", "agent", "--message", message, "--agent", agent]
|
||||
if model:
|
||||
cmd.extend(["--model", model])
|
||||
result = subprocess.run(
|
||||
["/opt/homebrew/bin/openclaw", "agent", "--message", message, "--agent", agent],
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout,
|
||||
@@ -587,6 +616,15 @@ class OpenClawBridgeHandler(BaseHTTPRequestHandler):
|
||||
if system_prompt:
|
||||
message = f"System Context: {system_prompt}\n\nUser Request: {message}"
|
||||
|
||||
# Load mode and resolve model routing
|
||||
mode_data = load_mode()
|
||||
model_override = resolve_model(mode_data)
|
||||
active_model = model_override or DEFAULT_MODEL
|
||||
if model_override:
|
||||
print(f"[OpenClaw Bridge] Mode: PUBLIC → {model_override}")
|
||||
else:
|
||||
print(f"[OpenClaw Bridge] Mode: PRIVATE ({active_model})")
|
||||
|
||||
# Check if model is warm to set appropriate timeout
|
||||
warm = is_model_warm()
|
||||
timeout = TIMEOUT_WARM if warm else TIMEOUT_COLD
|
||||
@@ -597,7 +635,7 @@ class OpenClawBridgeHandler(BaseHTTPRequestHandler):
|
||||
|
||||
# Call OpenClaw CLI (use full path for launchd compatibility)
|
||||
try:
|
||||
response_text = self._call_openclaw(message, agent, timeout)
|
||||
response_text = self._call_openclaw(message, agent, timeout, model=model_override)
|
||||
|
||||
# Re-prompt if the model promised to act but didn't call a tool.
|
||||
# Detect "I'll do X" / "Let me X" responses that lack any result.
|
||||
@@ -607,11 +645,11 @@ class OpenClawBridgeHandler(BaseHTTPRequestHandler):
|
||||
"You just said you would do something but didn't actually call the exec tool. "
|
||||
"Do NOT explain what you will do — call the tool NOW using exec and return the result."
|
||||
)
|
||||
response_text = self._call_openclaw(followup, agent, timeout)
|
||||
response_text = self._call_openclaw(followup, agent, timeout, model=model_override)
|
||||
|
||||
# Signal avatar: idle (TTS handler will override to 'speaking' if voice is used)
|
||||
_vtube_fire_and_forget("/expression", {"event": "idle"})
|
||||
self._send_json_response(200, {"response": response_text})
|
||||
self._send_json_response(200, {"response": response_text, "model": active_model})
|
||||
except subprocess.TimeoutExpired:
|
||||
self._send_json_response(504, {"error": f"OpenClaw command timed out after {timeout}s (model was {'warm' if warm else 'cold'})"})
|
||||
except subprocess.CalledProcessError as e:
|
||||
|
||||
90
homeai-agent/reminder-daemon.py
Executable file
90
homeai-agent/reminder-daemon.py
Executable file
@@ -0,0 +1,90 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
HomeAI Reminder Daemon — checks ~/homeai-data/reminders.json every 60s
|
||||
and fires TTS via POST http://localhost:8081/api/tts when reminders are due.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
import urllib.request
|
||||
from datetime import datetime
|
||||
|
||||
REMINDERS_FILE = os.path.expanduser("~/homeai-data/reminders.json")
|
||||
TTS_URL = "http://localhost:8081/api/tts"
|
||||
CHECK_INTERVAL = 60 # seconds
|
||||
|
||||
|
||||
def load_reminders():
|
||||
try:
|
||||
with open(REMINDERS_FILE) as f:
|
||||
return json.load(f)
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
return {"reminders": []}
|
||||
|
||||
|
||||
def save_reminders(data):
|
||||
with open(REMINDERS_FILE, "w") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
|
||||
|
||||
def fire_tts(message):
|
||||
"""Speak reminder via the OpenClaw bridge TTS endpoint."""
|
||||
try:
|
||||
payload = json.dumps({"text": f"Reminder: {message}"}).encode()
|
||||
req = urllib.request.Request(
|
||||
TTS_URL,
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST"
|
||||
)
|
||||
urllib.request.urlopen(req, timeout=30)
|
||||
print(f"[{datetime.now().isoformat()}] TTS fired: {message}")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"[{datetime.now().isoformat()}] TTS error: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def check_reminders():
|
||||
data = load_reminders()
|
||||
now = datetime.now()
|
||||
changed = False
|
||||
|
||||
for r in data.get("reminders", []):
|
||||
if r.get("fired"):
|
||||
continue
|
||||
|
||||
try:
|
||||
due = datetime.fromisoformat(r["due_at"])
|
||||
except (KeyError, ValueError):
|
||||
continue
|
||||
|
||||
if now >= due:
|
||||
print(f"[{now.isoformat()}] Reminder due: {r.get('message', '?')}")
|
||||
fire_tts(r["message"])
|
||||
r["fired"] = True
|
||||
changed = True
|
||||
|
||||
if changed:
|
||||
# Clean up fired reminders older than 24h
|
||||
cutoff = (now.timestamp() - 86400) * 1000
|
||||
data["reminders"] = [
|
||||
r for r in data["reminders"]
|
||||
if not r.get("fired") or int(r.get("id", "0")) > cutoff
|
||||
]
|
||||
save_reminders(data)
|
||||
|
||||
|
||||
def main():
|
||||
print(f"[{datetime.now().isoformat()}] Reminder daemon started (check every {CHECK_INTERVAL}s)")
|
||||
while True:
|
||||
try:
|
||||
check_reminders()
|
||||
except Exception as e:
|
||||
print(f"[{datetime.now().isoformat()}] Error: {e}")
|
||||
time.sleep(CHECK_INTERVAL)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
230
homeai-agent/setup.sh
Normal file → Executable file
230
homeai-agent/setup.sh
Normal file → Executable file
@@ -1,17 +1,20 @@
|
||||
#!/usr/bin/env bash
|
||||
# homeai-agent/setup.sh — P4: OpenClaw agent + skills + mem0
|
||||
# homeai-agent/setup.sh — OpenClaw agent, HTTP bridge, skills, reminder daemon
|
||||
#
|
||||
# Components:
|
||||
# - OpenClaw — AI agent runtime (port 8080)
|
||||
# - skills/ — home_assistant, memory, weather, timer, music stubs
|
||||
# - mem0 — long-term memory (Chroma backend)
|
||||
# - n8n workflows — morning briefing, notification router, memory backup
|
||||
# - OpenClaw gateway — AI agent runtime (port 8080)
|
||||
# - OpenClaw HTTP bridge — HA ↔ OpenClaw translator (port 8081)
|
||||
# - 13 skills — home-assistant, image-generation, voice-assistant,
|
||||
# vtube-studio, memory, service-monitor, character,
|
||||
# routine, music, workflow, gitea, calendar, mode
|
||||
# - Reminder daemon — fires TTS when reminders are due
|
||||
#
|
||||
# Prerequisites:
|
||||
# - P1 (homeai-infra) — Home Assistant running, HA_TOKEN set
|
||||
# - P2 (homeai-llm) — Ollama running with llama3.3:70b + nomic-embed-text
|
||||
# - P3 (homeai-voice) — Wyoming TTS running (for voice output)
|
||||
# - P5 (homeai-character) — aria.json character config exists
|
||||
# - Ollama running (port 11434)
|
||||
# - Home Assistant reachable (HA_TOKEN set in .env)
|
||||
# - Wyoming TTS running (port 10301)
|
||||
# - homeai-voice-env venv exists (for bridge + reminder daemon)
|
||||
# - At least one character JSON in ~/homeai-data/characters/
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
@@ -19,47 +22,196 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
source "${REPO_DIR}/scripts/common.sh"
|
||||
|
||||
log_section "P4: Agent (OpenClaw + skills + mem0)"
|
||||
log_section "P4: Agent (OpenClaw + HTTP Bridge + Skills)"
|
||||
detect_platform
|
||||
|
||||
# ─── Prerequisite check ────────────────────────────────────────────────────────
|
||||
# ─── Load environment ────────────────────────────────────────────────────────
|
||||
ENV_FILE="${REPO_DIR}/.env"
|
||||
if [[ -f "$ENV_FILE" ]]; then
|
||||
log_info "Loading .env..."
|
||||
load_env "$ENV_FILE"
|
||||
else
|
||||
log_warn "No .env found at ${ENV_FILE} — API keys may be missing"
|
||||
fi
|
||||
|
||||
# ─── Prerequisite checks ────────────────────────────────────────────────────
|
||||
log_info "Checking prerequisites..."
|
||||
|
||||
for service in "http://localhost:11434:Ollama(P2)" "http://localhost:8123:HomeAssistant(P1)"; do
|
||||
url="${service%%:*}"; name="${service##*:}"
|
||||
if ! curl -sf "$url" -o /dev/null 2>/dev/null; then
|
||||
require_command node "brew install node"
|
||||
require_command openclaw "npm install -g openclaw"
|
||||
|
||||
VOICE_ENV="${HOME}/homeai-voice-env"
|
||||
if [[ ! -d "$VOICE_ENV" ]]; then
|
||||
die "homeai-voice-env not found at $VOICE_ENV — run homeai-voice/setup.sh first"
|
||||
fi
|
||||
|
||||
# Check key services (non-fatal)
|
||||
for check in "http://localhost:11434:Ollama" "http://localhost:10301:Wyoming-TTS"; do
|
||||
url="${check%%:*}"; name="${check##*:}"
|
||||
if curl -sf "$url" -o /dev/null 2>/dev/null; then
|
||||
log_success "$name reachable"
|
||||
else
|
||||
log_warn "$name not reachable at $url"
|
||||
fi
|
||||
done
|
||||
|
||||
load_env_services
|
||||
if [[ -z "${HA_TOKEN:-}" ]]; then
|
||||
log_warn "HA_TOKEN not set in ~/.env.services — needed for home_assistant skill"
|
||||
# Check required env vars
|
||||
MISSING_KEYS=()
|
||||
[[ -z "${HA_TOKEN:-}" ]] && MISSING_KEYS+=("HA_TOKEN")
|
||||
[[ -z "${ANTHROPIC_API_KEY:-}" ]] && MISSING_KEYS+=("ANTHROPIC_API_KEY")
|
||||
if [[ ${#MISSING_KEYS[@]} -gt 0 ]]; then
|
||||
log_warn "Missing env vars: ${MISSING_KEYS[*]} — set these in ${ENV_FILE}"
|
||||
fi
|
||||
|
||||
# ─── TODO: Implementation ──────────────────────────────────────────────────────
|
||||
# ─── Ensure data directories ─────────────────────────────────────────────────
|
||||
DATA_DIR="${HOME}/homeai-data"
|
||||
for dir in characters memories memories/personal conversations routines; do
|
||||
mkdir -p "${DATA_DIR}/${dir}"
|
||||
done
|
||||
log_success "Data directories verified"
|
||||
|
||||
# ─── OpenClaw config ─────────────────────────────────────────────────────────
|
||||
OPENCLAW_DIR="${HOME}/.openclaw"
|
||||
OPENCLAW_CONFIG="${OPENCLAW_DIR}/openclaw.json"
|
||||
|
||||
if [[ ! -f "$OPENCLAW_CONFIG" ]]; then
|
||||
die "OpenClaw config not found at $OPENCLAW_CONFIG — run: openclaw doctor --fix"
|
||||
fi
|
||||
log_success "OpenClaw config exists at $OPENCLAW_CONFIG"
|
||||
|
||||
# Verify Anthropic provider is configured
|
||||
if ! grep -q '"anthropic"' "$OPENCLAW_CONFIG" 2>/dev/null; then
|
||||
log_warn "Anthropic provider not found in openclaw.json — add it for Claude support"
|
||||
fi
|
||||
|
||||
# ─── Install skills ──────────────────────────────────────────────────────────
|
||||
SKILLS_SRC="${SCRIPT_DIR}/skills"
|
||||
SKILLS_DEST="${OPENCLAW_DIR}/skills"
|
||||
|
||||
if [[ -d "$SKILLS_SRC" ]]; then
|
||||
log_info "Syncing skills..."
|
||||
mkdir -p "$SKILLS_DEST"
|
||||
for skill_dir in "$SKILLS_SRC"/*/; do
|
||||
skill_name="$(basename "$skill_dir")"
|
||||
dest="${SKILLS_DEST}/${skill_name}"
|
||||
if [[ -L "$dest" ]]; then
|
||||
log_info " ${skill_name} (symlinked)"
|
||||
elif [[ -d "$dest" ]]; then
|
||||
# Replace copy with symlink
|
||||
rm -rf "$dest"
|
||||
ln -s "$skill_dir" "$dest"
|
||||
log_step "${skill_name} → symlinked"
|
||||
else
|
||||
ln -s "$skill_dir" "$dest"
|
||||
log_step "${skill_name} → installed"
|
||||
fi
|
||||
done
|
||||
log_success "Skills synced ($(ls -d "$SKILLS_DEST"/*/ 2>/dev/null | wc -l | tr -d ' ') total)"
|
||||
else
|
||||
log_warn "No skills directory at $SKILLS_SRC"
|
||||
fi
|
||||
|
||||
# ─── Install launchd services (macOS) ────────────────────────────────────────
|
||||
if [[ "$OS_TYPE" == "macos" ]]; then
|
||||
log_info "Installing launchd agents..."
|
||||
|
||||
LAUNCHD_DIR="${SCRIPT_DIR}/launchd"
|
||||
AGENTS_DIR="${HOME}/Library/LaunchAgents"
|
||||
mkdir -p "$AGENTS_DIR"
|
||||
|
||||
# Inject API keys into plists that need them
|
||||
_inject_plist_key() {
|
||||
local plist="$1" key="$2" value="$3"
|
||||
if [[ -n "$value" ]] && grep -q "<key>${key}</key>" "$plist" 2>/dev/null; then
|
||||
# Use python for reliable XML-safe replacement
|
||||
python3 -c "
|
||||
import sys, re
|
||||
with open('$plist') as f: content = f.read()
|
||||
pattern = r'(<key>${key}</key>\s*<string>)[^<]*(</string>)'
|
||||
content = re.sub(pattern, r'\g<1>${value}\g<2>', content)
|
||||
with open('$plist', 'w') as f: f.write(content)
|
||||
"
|
||||
fi
|
||||
}
|
||||
|
||||
# Update API keys in plist source files before linking
|
||||
OPENCLAW_PLIST="${LAUNCHD_DIR}/com.homeai.openclaw.plist"
|
||||
BRIDGE_PLIST="${LAUNCHD_DIR}/com.homeai.openclaw-bridge.plist"
|
||||
|
||||
if [[ -f "$OPENCLAW_PLIST" ]]; then
|
||||
_inject_plist_key "$OPENCLAW_PLIST" "ANTHROPIC_API_KEY" "${ANTHROPIC_API_KEY:-}"
|
||||
_inject_plist_key "$OPENCLAW_PLIST" "OPENAI_API_KEY" "${OPENAI_API_KEY:-}"
|
||||
_inject_plist_key "$OPENCLAW_PLIST" "HA_TOKEN" "${HA_TOKEN:-}"
|
||||
_inject_plist_key "$OPENCLAW_PLIST" "HASS_TOKEN" "${HA_TOKEN:-}"
|
||||
_inject_plist_key "$OPENCLAW_PLIST" "GITEA_TOKEN" "${GITEA_TOKEN:-}"
|
||||
_inject_plist_key "$OPENCLAW_PLIST" "N8N_API_KEY" "${N8N_API_KEY:-}"
|
||||
fi
|
||||
|
||||
if [[ -f "$BRIDGE_PLIST" ]]; then
|
||||
_inject_plist_key "$BRIDGE_PLIST" "ANTHROPIC_API_KEY" "${ANTHROPIC_API_KEY:-}"
|
||||
_inject_plist_key "$BRIDGE_PLIST" "ELEVENLABS_API_KEY" "${ELEVENLABS_API_KEY:-}"
|
||||
fi
|
||||
|
||||
# Symlink and load each plist
|
||||
for plist in "$LAUNCHD_DIR"/*.plist; do
|
||||
[[ ! -f "$plist" ]] && continue
|
||||
plist_name="$(basename "$plist")"
|
||||
plist_label="${plist_name%.plist}"
|
||||
dest="${AGENTS_DIR}/${plist_name}"
|
||||
|
||||
# Unload if already running
|
||||
launchctl bootout "gui/$(id -u)/${plist_label}" 2>/dev/null || true
|
||||
|
||||
# Symlink source → LaunchAgents
|
||||
ln -sf "$(cd "$(dirname "$plist")" && pwd)/${plist_name}" "$dest"
|
||||
|
||||
# Load
|
||||
launchctl bootstrap "gui/$(id -u)" "$dest" 2>/dev/null && \
|
||||
log_success " ${plist_label} → loaded" || \
|
||||
log_warn " ${plist_label} → failed to load (check: launchctl print gui/$(id -u)/${plist_label})"
|
||||
done
|
||||
fi
|
||||
|
||||
# ─── Smoke test ──────────────────────────────────────────────────────────────
|
||||
log_info "Running smoke tests..."
|
||||
|
||||
sleep 2 # Give services a moment to start
|
||||
|
||||
# Check gateway
|
||||
if curl -sf "http://localhost:8080" -o /dev/null 2>/dev/null; then
|
||||
log_success "OpenClaw gateway responding on :8080"
|
||||
else
|
||||
log_warn "OpenClaw gateway not responding on :8080 — check: tail /tmp/homeai-openclaw.log"
|
||||
fi
|
||||
|
||||
# Check bridge
|
||||
if curl -sf "http://localhost:8081/status" -o /dev/null 2>/dev/null; then
|
||||
log_success "HTTP bridge responding on :8081"
|
||||
else
|
||||
log_warn "HTTP bridge not responding on :8081 — check: tail /tmp/homeai-openclaw-bridge.log"
|
||||
fi
|
||||
|
||||
# ─── Summary ─────────────────────────────────────────────────────────────────
|
||||
print_summary "Agent Setup Complete" \
|
||||
"OpenClaw gateway" "http://localhost:8080" \
|
||||
"HTTP bridge" "http://localhost:8081" \
|
||||
"OpenClaw config" "$OPENCLAW_CONFIG" \
|
||||
"Skills directory" "$SKILLS_DEST" \
|
||||
"Character data" "${DATA_DIR}/characters/" \
|
||||
"Memory data" "${DATA_DIR}/memories/" \
|
||||
"Reminder data" "${DATA_DIR}/reminders.json" \
|
||||
"Gateway log" "/tmp/homeai-openclaw.log" \
|
||||
"Bridge log" "/tmp/homeai-openclaw-bridge.log"
|
||||
|
||||
cat <<'EOF'
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ P4: homeai-agent — NOT YET IMPLEMENTED │
|
||||
│ │
|
||||
│ OPEN QUESTION: Which OpenClaw version/fork to use? │
|
||||
│ Decide before implementing. See homeai-agent/PLAN.md. │
|
||||
│ │
|
||||
│ Implementation steps: │
|
||||
│ 1. Install OpenClaw (pip install or git clone) │
|
||||
│ 2. Create ~/.openclaw/config.yaml from config/config.yaml.example │
|
||||
│ 3. Create skills: home_assistant, memory, weather, timer, music│
|
||||
│ 4. Install mem0 + Chroma backend │
|
||||
│ 5. Create systemd/launchd service for OpenClaw (port 8080) │
|
||||
│ 6. Import n8n workflows from workflows/ │
|
||||
│ 7. Smoke test: POST /chat "turn on living room lights" │
|
||||
│ │
|
||||
│ Interface contracts: │
|
||||
│ OPENCLAW_URL=http://localhost:8080 │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
To reload a service after editing its plist:
|
||||
launchctl bootout gui/$(id -u)/com.homeai.<service>
|
||||
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.homeai.<service>.plist
|
||||
|
||||
To test the agent:
|
||||
curl -X POST http://localhost:8081/api/agent/message \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"message":"say hello","agent":"main"}'
|
||||
|
||||
EOF
|
||||
|
||||
log_info "P4 is not yet implemented. See homeai-agent/PLAN.md for details."
|
||||
exit 0
|
||||
|
||||
@@ -107,16 +107,25 @@ export default function MessageBubble({ message, onReplay, character }) {
|
||||
>
|
||||
{isUser ? message.content : <RichContent text={message.content} />}
|
||||
</div>
|
||||
{!isUser && !message.isError && onReplay && (
|
||||
<button
|
||||
onClick={() => onReplay(message.content)}
|
||||
className="mt-1 ml-1 text-gray-500 hover:text-indigo-400 transition-colors"
|
||||
title="Replay audio"
|
||||
>
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M15.536 8.464a5 5 0 010 7.072M17.95 6.05a8 8 0 010 11.9M6.5 9H4a1 1 0 00-1 1v4a1 1 0 001 1h2.5l4 4V5l-4 4z" />
|
||||
</svg>
|
||||
</button>
|
||||
{!isUser && (
|
||||
<div className="flex items-center gap-2 mt-1 ml-1">
|
||||
{message.model && (
|
||||
<span className="text-[10px] text-gray-500 font-mono">
|
||||
{message.model}
|
||||
</span>
|
||||
)}
|
||||
{!message.isError && onReplay && (
|
||||
<button
|
||||
onClick={() => onReplay(message.content)}
|
||||
className="text-gray-500 hover:text-indigo-400 transition-colors"
|
||||
title="Replay audio"
|
||||
>
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M15.536 8.464a5 5 0 010 7.072M17.95 6.05a8 8 0 010 11.9M6.5 9H4a1 1 0 00-1 1v4a1 1 0 001 1h2.5l4 4V5l-4 4z" />
|
||||
</svg>
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -80,12 +80,13 @@ export function useChat(conversationId, conversationMeta, onConversationUpdate)
|
||||
setIsLoading(true)
|
||||
|
||||
try {
|
||||
const response = await sendMessage(text.trim(), conversationMeta?.characterId || null)
|
||||
const { response, model } = await sendMessage(text.trim(), conversationMeta?.characterId || null)
|
||||
const assistantMsg = {
|
||||
id: Date.now() + 1,
|
||||
role: 'assistant',
|
||||
content: response,
|
||||
timestamp: new Date().toISOString(),
|
||||
...(model && { model }),
|
||||
}
|
||||
const allMessages = [...newMessages, assistantMsg]
|
||||
setMessages(allMessages)
|
||||
|
||||
@@ -31,7 +31,7 @@ export async function sendMessage(text, characterId = null) {
|
||||
throw new Error(err.error || `HTTP ${res.status}`)
|
||||
}
|
||||
const data = await res.json()
|
||||
return data.response
|
||||
return { response: data.response, model: data.model || null }
|
||||
}
|
||||
|
||||
export async function synthesize(text, voice, engine = 'kokoro', model = null) {
|
||||
|
||||
@@ -6,6 +6,7 @@ const CHARACTERS_DIR = '/Users/aodhan/homeai-data/characters'
|
||||
const SATELLITE_MAP_PATH = '/Users/aodhan/homeai-data/satellite-map.json'
|
||||
const CONVERSATIONS_DIR = '/Users/aodhan/homeai-data/conversations'
|
||||
const MEMORIES_DIR = '/Users/aodhan/homeai-data/memories'
|
||||
const MODE_PATH = '/Users/aodhan/homeai-data/active-mode.json'
|
||||
const GAZE_HOST = 'http://10.0.0.101:5782'
|
||||
const GAZE_API_KEY = process.env.GAZE_API_KEY || ''
|
||||
|
||||
@@ -649,6 +650,54 @@ print(json.dumps(c.model_dump(), default=str))
|
||||
}
|
||||
}
|
||||
|
||||
function modePlugin() {
|
||||
return {
|
||||
name: 'mode-api',
|
||||
configureServer(server) {
|
||||
server.middlewares.use('/api/mode', async (req, res, next) => {
|
||||
if (req.method === 'OPTIONS') {
|
||||
res.writeHead(204, { 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Methods': 'GET,POST', 'Access-Control-Allow-Headers': 'Content-Type' })
|
||||
res.end()
|
||||
return
|
||||
}
|
||||
|
||||
const { readFile, writeFile } = await import('fs/promises')
|
||||
const DEFAULT_MODE = { mode: 'private', cloud_provider: 'anthropic', cloud_model: 'claude-sonnet-4-20250514', local_model: 'ollama/qwen3.5:35b-a3b', overrides: {}, updated_at: '' }
|
||||
|
||||
if (req.method === 'GET') {
|
||||
try {
|
||||
const raw = await readFile(MODE_PATH, 'utf-8')
|
||||
res.writeHead(200, { 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*' })
|
||||
res.end(raw)
|
||||
} catch {
|
||||
res.writeHead(200, { 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*' })
|
||||
res.end(JSON.stringify(DEFAULT_MODE))
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if (req.method === 'POST') {
|
||||
try {
|
||||
const chunks = []
|
||||
for await (const chunk of req) chunks.push(chunk)
|
||||
const data = JSON.parse(Buffer.concat(chunks).toString())
|
||||
data.updated_at = new Date().toISOString()
|
||||
await writeFile(MODE_PATH, JSON.stringify(data, null, 2))
|
||||
res.writeHead(200, { 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*' })
|
||||
res.end(JSON.stringify({ ok: true }))
|
||||
} catch (err) {
|
||||
res.writeHead(500, { 'Content-Type': 'application/json' })
|
||||
res.end(JSON.stringify({ error: err.message }))
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
next()
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
function bridgeProxyPlugin() {
|
||||
return {
|
||||
name: 'bridge-proxy',
|
||||
@@ -725,6 +774,7 @@ export default defineConfig({
|
||||
gazeProxyPlugin(),
|
||||
characterLookupPlugin(),
|
||||
healthCheckPlugin(),
|
||||
modePlugin(),
|
||||
bridgeProxyPlugin(),
|
||||
tailwindcss(),
|
||||
react(),
|
||||
|
||||
@@ -85,6 +85,7 @@ services:
|
||||
- homeai.service=n8n
|
||||
- homeai.url=http://localhost:5678
|
||||
|
||||
|
||||
networks:
|
||||
homeai:
|
||||
external: true
|
||||
|
||||
389
plans/OPENCLAW_SKILLS.md
Normal file
389
plans/OPENCLAW_SKILLS.md
Normal file
@@ -0,0 +1,389 @@
|
||||
# OpenClaw Skills Expansion Plan
|
||||
|
||||
## Context
|
||||
|
||||
The HomeAI project has 4 custom OpenClaw skills (home-assistant, voice-assistant, image-generation, vtube-studio) and a working voice pipeline. The user wants to build 8 new skills plus a Public/Private mode system to dramatically expand what the assistant can do via voice and chat.
|
||||
|
||||
## Skill Format Convention
|
||||
|
||||
Every skill follows the established pattern from ha-ctl:
|
||||
|
||||
- Lives in `~/.openclaw/skills/<name>/`
|
||||
- `SKILL.md` with YAML frontmatter (name, description) + agent instructions
|
||||
- Optional Python CLI (stdlib only: `urllib.request`, `json`, `os`, `sys`, `re`, `datetime`)
|
||||
- CLI symlinked to `/opt/homebrew/bin/` for PATH access
|
||||
- Agent invokes via `exec` tool
|
||||
- Entry added to `~/.openclaw/workspace/TOOLS.md` for reinforcement
|
||||
- New env vars added to `homeai-agent/launchd/com.homeai.openclaw.plist`
|
||||
|
||||
---
|
||||
|
||||
## Phase A — Core Skills (no new services needed)
|
||||
|
||||
### 1. Memory Recall (`memory-ctl`)
|
||||
|
||||
**Purpose:** Let the agent actively store, search, and recall memories mid-conversation.
|
||||
|
||||
**Files:**
|
||||
- `~/.openclaw/skills/memory/SKILL.md`
|
||||
- `~/.openclaw/skills/memory/memory-ctl` → symlink `/opt/homebrew/bin/memory-ctl`
|
||||
|
||||
**Commands:**
|
||||
```
|
||||
memory-ctl add <personal|general> "<content>" [--category preference|fact|routine] [--character-id ID]
|
||||
memory-ctl search "<query>" [--type personal|general] [--character-id ID]
|
||||
memory-ctl list [--type personal|general] [--character-id ID] [--limit 10]
|
||||
memory-ctl delete <memory_id> [--type personal|general] [--character-id ID]
|
||||
```
|
||||
|
||||
**Details:**
|
||||
- Reads/writes existing files: `~/homeai-data/memories/personal/{char_id}.json` and `general.json`
|
||||
- Matches existing schema: `{"memories": [{"id": "m_<timestamp>", "content": "...", "category": "...", "createdAt": "..."}]}`
|
||||
- Search: keyword token matching (split query, score by substring hits in content)
|
||||
- `--character-id` defaults to `HOMEAI_CHARACTER_ID` env var or satellite-map default
|
||||
- Dashboard memory UI will immediately reflect agent-created memories (same files)
|
||||
- **Env vars:** `HOMEAI_CHARACTER_ID` (optional, set by bridge)
|
||||
|
||||
---
|
||||
|
||||
### 2. Service Monitor (`monitor-ctl`)
|
||||
|
||||
**Purpose:** "Is everything running?" → spoken health report.
|
||||
|
||||
**Files:**
|
||||
- `~/.openclaw/skills/service-monitor/SKILL.md`
|
||||
- `~/.openclaw/skills/service-monitor/monitor-ctl` → symlink `/opt/homebrew/bin/monitor-ctl`
|
||||
|
||||
**Commands:**
|
||||
```
|
||||
monitor-ctl status # All services summary
|
||||
monitor-ctl check <service> # Single service (ollama, bridge, ha, tts, stt, dashboard, n8n, gitea, kuma)
|
||||
monitor-ctl ollama # Ollama-specific: loaded models, VRAM
|
||||
monitor-ctl docker # Docker container status (runs: docker ps --format json)
|
||||
```
|
||||
|
||||
**Details:**
|
||||
- Checks hardcoded service endpoints with 3s timeout:
|
||||
- Ollama (`localhost:11434/api/ps`), Bridge (`localhost:8081/status`), Gateway (`localhost:8080/status`)
|
||||
- Wyoming STT (TCP `localhost:10300`), TTS (TCP `localhost:10301`)
|
||||
- Dashboard (`localhost:5173`), n8n (`localhost:5678`), Kuma (`localhost:3001`)
|
||||
- HA (`10.0.0.199:8123/api/`), Gitea (`10.0.0.199:3000`)
|
||||
- `ollama` subcommand parses `/api/ps` for model names, sizes, expiry
|
||||
- `docker` runs `docker ps --format '{{json .}}'` via subprocess
|
||||
- Pure stdlib (`urllib.request` + `socket.create_connection` for TCP)
|
||||
- **Env vars:** Uses existing `HASS_TOKEN`, `HA_URL`
|
||||
|
||||
---
|
||||
|
||||
### 3. Character Switcher (`character-ctl`)
|
||||
|
||||
**Purpose:** "Talk to Aria" → swap persona, TTS voice, system prompt.
|
||||
|
||||
**Files:**
|
||||
- `~/.openclaw/skills/character/SKILL.md`
|
||||
- `~/.openclaw/skills/character/character-ctl` → symlink `/opt/homebrew/bin/character-ctl`
|
||||
|
||||
**Commands:**
|
||||
```
|
||||
character-ctl list # All characters (name, id, tts engine)
|
||||
character-ctl active # Current default character
|
||||
character-ctl switch "<name_or_id>" # Set as default
|
||||
character-ctl info "<name_or_id>" # Profile summary
|
||||
character-ctl map <satellite_id> <character_id> # Map satellite → character
|
||||
```
|
||||
|
||||
**Details:**
|
||||
- Reads character JSONs from `~/homeai-data/characters/`
|
||||
- `switch` updates `satellite-map.json` default + writes `active-tts-voice.json`
|
||||
- Fuzzy name resolution: case-insensitive match on `display_name` → `name` → `id` → partial match
|
||||
- Switch takes effect on next bridge request (`SKILL.md` tells agent to inform user)
|
||||
- **Env vars:** None new
|
||||
|
||||
---
|
||||
|
||||
## Phase B — Home Assistant Extensions
|
||||
|
||||
### 4. Routine/Scene Builder (`routine-ctl`)
|
||||
|
||||
**Purpose:** Create and trigger multi-device scenes and routines from voice.
|
||||
|
||||
**Files:**
|
||||
- `~/.openclaw/skills/routine/SKILL.md`
|
||||
- `~/.openclaw/skills/routine/routine-ctl` → symlink `/opt/homebrew/bin/routine-ctl`
|
||||
|
||||
**Commands:**
|
||||
```
|
||||
routine-ctl list-scenes # HA scenes
|
||||
routine-ctl list-scripts # HA scripts
|
||||
routine-ctl trigger "<scene_or_script>" # Activate
|
||||
routine-ctl create-scene "<name>" --entities '[{"entity_id":"light.x","state":"on","brightness":128}]'
|
||||
routine-ctl list-routines # Local multi-step routines
|
||||
routine-ctl create-routine "<name>" --steps '[{"type":"scene","target":"movie_mode"},{"type":"delay","seconds":5},{"type":"ha","cmd":"off \"TV Backlight\""}]'
|
||||
routine-ctl run "<routine_name>" # Execute steps sequentially
|
||||
```
|
||||
|
||||
**Details:**
|
||||
- HA scenes via REST API: `POST /api/services/scene/turn_on`, `POST /api/services/scene/create`
|
||||
- Local routines stored in `~/homeai-data/routines/*.json`
|
||||
- Step types: `scene` (trigger HA scene), `ha` (subprocess call to ha-ctl), `delay` (sleep), `tts` (curl to bridge `/api/tts`)
|
||||
- `run` executes steps sequentially, reports progress
|
||||
- **New data path:** `~/homeai-data/routines/`
|
||||
- **Env vars:** Uses existing `HASS_TOKEN`, `HA_URL`
|
||||
|
||||
---
|
||||
|
||||
### 5. Music Control (`music-ctl`)
|
||||
|
||||
**Purpose:** Play/control music with multi-room and Spotify support.
|
||||
|
||||
**Files:**
|
||||
- `~/.openclaw/skills/music/SKILL.md`
|
||||
- `~/.openclaw/skills/music/music-ctl` → symlink `/opt/homebrew/bin/music-ctl`
|
||||
|
||||
**Commands:**
|
||||
```
|
||||
music-ctl players # List media_player entities
|
||||
music-ctl play ["query"] [--player ID] # Play/resume (search + play if query given)
|
||||
music-ctl pause [--player ID] # Pause
|
||||
music-ctl next / prev [--player ID] # Skip tracks
|
||||
music-ctl volume <0-100> [--player ID] # Set volume
|
||||
music-ctl now-playing [--player ID] # Current track info
|
||||
music-ctl queue [--player ID] # Queue contents
|
||||
music-ctl shuffle <on|off> [--player ID] # Toggle shuffle
|
||||
music-ctl search "<query>" # Search library
|
||||
```
|
||||
|
||||
**Details:**
|
||||
- All commands go through HA `media_player` services (same API pattern as ha-ctl)
|
||||
- `play` with query uses `media_player/play_media` with `media_content_type: music`
|
||||
- Spotify appears as a `media_player` entity via HA Spotify integration — no separate API needed
|
||||
- `players` lists all `media_player` entities (Music Assistant zones, Spotify Connect, Chromecast, etc.)
|
||||
- `--player` defaults to first active player or a configurable default
|
||||
- Multi-room: Snapcast zones appear as separate `media_player` entities
|
||||
- `now-playing` reads state attributes: `media_title`, `media_artist`, `media_album`, `media_position`
|
||||
- **Env vars:** Uses existing `HASS_TOKEN`, `HA_URL`
|
||||
- **Prerequisite:** Music Assistant Docker container configured + HA integration, OR Spotify HA integration
|
||||
|
||||
---
|
||||
|
||||
## Phase C — External Service Skills
|
||||
|
||||
### 6. n8n Workflow Trigger (`workflow-ctl`)
|
||||
|
||||
**Purpose:** List and trigger n8n workflows by voice.
|
||||
|
||||
**Files:**
|
||||
- `~/.openclaw/skills/workflow/SKILL.md`
|
||||
- `~/.openclaw/skills/workflow/workflow-ctl` → symlink `/opt/homebrew/bin/workflow-ctl`
|
||||
|
||||
**Commands:**
|
||||
```
|
||||
workflow-ctl list # All workflows (name, active, id)
|
||||
workflow-ctl trigger "<name_or_id>" [--data '{"key":"val"}'] # Fire webhook
|
||||
workflow-ctl status <execution_id> # Execution status
|
||||
workflow-ctl history [--limit 10] # Recent executions
|
||||
```
|
||||
|
||||
**Details:**
|
||||
- n8n REST API at `http://localhost:5678/api/v1/`
|
||||
- Auth via API key header: `X-N8N-API-KEY`
|
||||
- `trigger` prefers webhook trigger (`POST /webhook/<path>`), falls back to `POST /api/v1/workflows/<id>/execute`
|
||||
- Fuzzy name matching on workflow names
|
||||
- **Env vars (new):** `N8N_URL` (default `http://localhost:5678`), `N8N_API_KEY` (generate in n8n Settings → API)
|
||||
|
||||
---
|
||||
|
||||
### 7. Gitea Integration (`gitea-ctl`)
|
||||
|
||||
**Purpose:** Query self-hosted repos, commits, issues, PRs.
|
||||
|
||||
**Files:**
|
||||
- `~/.openclaw/skills/gitea/SKILL.md`
|
||||
- `~/.openclaw/skills/gitea/gitea-ctl` → symlink `/opt/homebrew/bin/gitea-ctl`
|
||||
|
||||
**Commands:**
|
||||
```
|
||||
gitea-ctl repos [--limit 20] # List repos
|
||||
gitea-ctl commits <owner/repo> [--limit 10] # Recent commits
|
||||
gitea-ctl issues <owner/repo> [--state open] # List issues
|
||||
gitea-ctl prs <owner/repo> [--state open] # List PRs
|
||||
gitea-ctl create-issue <owner/repo> "<title>" [--body TEXT]
|
||||
```
|
||||
|
||||
**Details:**
|
||||
- Gitea REST API v1 at `http://10.0.0.199:3000/api/v1/`
|
||||
- Auth: `Authorization: token <GITEA_TOKEN>`
|
||||
- Pure stdlib `urllib.request`
|
||||
- **Env vars (new):** `GITEA_URL` (default `http://10.0.0.199:3000`), `GITEA_TOKEN` (generate in Gitea → Settings → Applications)
|
||||
|
||||
---
|
||||
|
||||
### 8. Calendar/Reminders (`calendar-ctl`)
|
||||
|
||||
**Purpose:** Read calendar, create events, set voice reminders.
|
||||
|
||||
**Files:**
|
||||
- `~/.openclaw/skills/calendar/SKILL.md`
|
||||
- `~/.openclaw/skills/calendar/calendar-ctl` → symlink `/opt/homebrew/bin/calendar-ctl`
|
||||
|
||||
**Commands:**
|
||||
```
|
||||
calendar-ctl today [--calendar ID] # Today's events
|
||||
calendar-ctl upcoming [--days 7] # Next N days
|
||||
calendar-ctl add "<summary>" --start <ISO> --end <ISO> [--calendar ID]
|
||||
calendar-ctl remind "<message>" --at "<time>" # Set reminder (e.g. "in 30 minutes", "at 5pm", "tomorrow 9am")
|
||||
calendar-ctl reminders # List pending reminders
|
||||
calendar-ctl cancel-reminder <id> # Cancel reminder
|
||||
```
|
||||
|
||||
**Details:**
|
||||
- Calendar read: `GET /api/calendars/<entity_id>?start=<ISO>&end=<ISO>` via HA API
|
||||
- Calendar write: `POST /api/services/calendar/create_event`
|
||||
- Reminders stored locally in `~/homeai-data/reminders.json`
|
||||
- Relative time parsing with `datetime` + `re` (stdlib): "in 30 minutes", "at 5pm", "tomorrow 9am"
|
||||
- Reminder daemon (`com.homeai.reminder-daemon`): Python script checking `reminders.json` every 60s, fires TTS via `POST http://localhost:8081/api/tts` when due
|
||||
- **New data path:** `~/homeai-data/reminders.json`
|
||||
- **New daemon:** `homeai-agent/reminder-daemon.py` + `homeai-agent/launchd/com.homeai.reminder-daemon.plist`
|
||||
- **Env vars:** Uses existing `HASS_TOKEN`, `HA_URL`
|
||||
|
||||
---
|
||||
|
||||
## Phase D — Public/Private Mode System
|
||||
|
||||
### 9. Mode Controller (`mode-ctl`)
|
||||
|
||||
**Purpose:** Route requests to cloud LLMs (speed/power) or local LLMs (privacy) with per-task rules and manual toggle.
|
||||
|
||||
**Files:**
|
||||
- `~/.openclaw/skills/mode/SKILL.md`
|
||||
- `~/.openclaw/skills/mode/mode-ctl` → symlink `/opt/homebrew/bin/mode-ctl`
|
||||
|
||||
**Commands:**
|
||||
```
|
||||
mode-ctl status # Current mode + overrides
|
||||
mode-ctl private # Switch to local-only
|
||||
mode-ctl public # Switch to cloud LLMs
|
||||
mode-ctl set-provider <anthropic|openai> # Preferred cloud provider
|
||||
mode-ctl override <category> <private|public> # Per-category routing
|
||||
mode-ctl list-overrides # Show all overrides
|
||||
```
|
||||
|
||||
**State file:** `~/homeai-data/active-mode.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"mode": "private",
|
||||
"cloud_provider": "anthropic",
|
||||
"cloud_model": "claude-sonnet-4-20250514",
|
||||
"overrides": {
|
||||
"web_search": "public",
|
||||
"coding": "public",
|
||||
"personal_finance": "private",
|
||||
"health": "private"
|
||||
},
|
||||
"updated_at": "2026-03-17T..."
|
||||
}
|
||||
```
|
||||
|
||||
**How model routing works — Bridge modification:**
|
||||
|
||||
The HTTP bridge (`openclaw-http-bridge.py`) is modified to:
|
||||
|
||||
1. New function `load_mode()` reads `active-mode.json`
|
||||
2. New function `resolve_model(mode, category=None)` returns model string
|
||||
3. In `_handle_agent_request()`, after character resolution, check mode → pass `--model` flag to OpenClaw CLI
|
||||
- **Private:** `ollama/qwen3.5:35b-a3b` (current default, no change)
|
||||
- **Public:** `anthropic/claude-sonnet-4-20250514` or `openai/gpt-4o` (per provider setting)
|
||||
|
||||
**OpenClaw config changes (`openclaw.json`):** Add cloud providers to `models.providers`:
|
||||
|
||||
```json
|
||||
"anthropic": {
|
||||
"baseUrl": "https://api.anthropic.com/v1",
|
||||
"apiKey": "${ANTHROPIC_API_KEY}",
|
||||
"api": "anthropic",
|
||||
"models": [{"id": "claude-sonnet-4-20250514", "contextWindow": 200000, "maxTokens": 8192}]
|
||||
},
|
||||
"openai": {
|
||||
"baseUrl": "https://api.openai.com/v1",
|
||||
"apiKey": "${OPENAI_API_KEY}",
|
||||
"api": "openai",
|
||||
"models": [{"id": "gpt-4o", "contextWindow": 128000, "maxTokens": 4096}]
|
||||
}
|
||||
```
|
||||
|
||||
**Per-task classification:** The `SKILL.md` provides a category reference table. The agent self-classifies each request and checks overrides. Default categories:
|
||||
|
||||
- **Always private:** personal finance, health, passwords, private conversations
|
||||
- **Always public:** web search, coding help, complex reasoning, translation
|
||||
- **Follow global mode:** general chat, smart home, music, calendar
|
||||
|
||||
**Dashboard integration:** Add mode toggle to dashboard sidebar via new Vite middleware endpoint `GET/POST /api/mode` reading/writing `active-mode.json`.
|
||||
|
||||
- **Env vars (new):** `ANTHROPIC_API_KEY`, `OPENAI_API_KEY` (add to OpenClaw plist)
|
||||
- **Bridge file modified:** `homeai-agent/openclaw-http-bridge.py` — add ~40 lines for mode loading + model resolution
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
| # | Skill | Complexity | Dependencies |
|
||||
|---|-------|-----------|-------------|
|
||||
| 1 | `memory-ctl` | Simple | None |
|
||||
| 2 | `monitor-ctl` | Simple | None |
|
||||
| 3 | `character-ctl` | Simple | None |
|
||||
| 4 | `routine-ctl` | Medium | ha-ctl existing |
|
||||
| 5 | `music-ctl` | Medium | Music Assistant or Spotify in HA |
|
||||
| 6 | `workflow-ctl` | Simple | n8n API key |
|
||||
| 7 | `gitea-ctl` | Simple | Gitea API token |
|
||||
| 8 | `calendar-ctl` | Medium | HA calendar + new reminder daemon |
|
||||
| 9 | `mode-ctl` | High | Cloud API keys + bridge modification |
|
||||
|
||||
## Per-Skill Implementation Steps
|
||||
|
||||
For each skill:
|
||||
|
||||
1. Create `SKILL.md` with frontmatter + agent instructions + examples
|
||||
2. Create Python CLI (`chmod +x`), stdlib only
|
||||
3. Symlink to `/opt/homebrew/bin/`
|
||||
4. Test CLI standalone: `<tool> --help`, `<tool> <command>`
|
||||
5. Add env vars to `com.homeai.openclaw.plist` if needed
|
||||
6. Restart OpenClaw: `launchctl kickstart -k gui/501/com.homeai.openclaw`
|
||||
7. Add section to `~/.openclaw/workspace/TOOLS.md`
|
||||
8. Test via: `openclaw agent --message "test prompt" --agent main`
|
||||
9. Test via voice: wake word + spoken command
|
||||
|
||||
## Verification
|
||||
|
||||
- **Unit test each CLI:** Run each command manually and verify JSON output
|
||||
- **Agent test:** `openclaw agent --message "remember that my favorite color is blue"` (memory-ctl)
|
||||
- **Voice test:** Wake word → "Is everything running?" → spoken health report (monitor-ctl)
|
||||
- **Mode test:** `mode-ctl public` → send a complex query → verify it routes to cloud model in bridge logs
|
||||
- **Dashboard test:** Check memory UI shows agent-created memories, mode toggle works
|
||||
- **Cross-skill test:** "Switch to Sucy and play some jazz" → character-ctl + music-ctl in one turn
|
||||
|
||||
## Critical Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `~/.openclaw/workspace/TOOLS.md` | Add sections for all 9 new skills |
|
||||
| `homeai-agent/openclaw-http-bridge.py` | Mode routing (Phase D only) |
|
||||
| `homeai-agent/launchd/com.homeai.openclaw.plist` | New env vars |
|
||||
| `~/.openclaw/openclaw.json` | Add anthropic + openai providers (Phase D) |
|
||||
| `homeai-dashboard/vite.config.js` | `/api/mode` endpoint (Phase D) |
|
||||
|
||||
## New Files Created
|
||||
|
||||
- `~/.openclaw/skills/memory/` (SKILL.md + memory-ctl)
|
||||
- `~/.openclaw/skills/service-monitor/` (SKILL.md + monitor-ctl)
|
||||
- `~/.openclaw/skills/character/` (SKILL.md + character-ctl)
|
||||
- `~/.openclaw/skills/routine/` (SKILL.md + routine-ctl)
|
||||
- `~/.openclaw/skills/music/` (SKILL.md + music-ctl)
|
||||
- `~/.openclaw/skills/workflow/` (SKILL.md + workflow-ctl)
|
||||
- `~/.openclaw/skills/gitea/` (SKILL.md + gitea-ctl)
|
||||
- `~/.openclaw/skills/calendar/` (SKILL.md + calendar-ctl + reminder-daemon.py)
|
||||
- `~/.openclaw/skills/mode/` (SKILL.md + mode-ctl)
|
||||
- `~/homeai-data/routines/` (directory)
|
||||
- `~/homeai-data/reminders.json` (file)
|
||||
- `~/homeai-data/active-mode.json` (file)
|
||||
- `homeai-agent/reminder-daemon.py` + launchd plist
|
||||
@@ -228,11 +228,19 @@ install_service() {
|
||||
log_warn "No launchd plist at $launchd_file — skipping service install."
|
||||
return
|
||||
fi
|
||||
local plist_dest="${HOME}/Library/LaunchAgents/$(basename "$launchd_file")"
|
||||
local plist_name
|
||||
plist_name="$(basename "$launchd_file")"
|
||||
local plist_dest="${HOME}/Library/LaunchAgents/${plist_name}"
|
||||
local plist_label="${plist_name%.plist}"
|
||||
local abs_source
|
||||
abs_source="$(cd "$(dirname "$launchd_file")" && pwd)/$(basename "$launchd_file")"
|
||||
log_step "Installing launchd agent: $name"
|
||||
cp "$launchd_file" "$plist_dest"
|
||||
launchctl load -w "$plist_dest"
|
||||
log_success "LaunchAgent '$name' installed and loaded."
|
||||
# Unload existing service if running
|
||||
launchctl bootout "gui/$(id -u)/${plist_label}" 2>/dev/null || true
|
||||
# Symlink so edits to repo source take effect on reload
|
||||
ln -sf "$abs_source" "$plist_dest"
|
||||
launchctl bootstrap "gui/$(id -u)" "$plist_dest"
|
||||
log_success "LaunchAgent '$name' symlinked and loaded."
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user