feat: Music Assistant, Claude primary LLM, model tag in chat, setup.sh rewrite

- Deploy Music Assistant on Pi (10.0.0.199:8095) with host networking for Chromecast mDNS discovery, Spotify + SMB library support - Switch primary LLM from Ollama to Claude Sonnet 4 (Anthropic API), local models remain as fallback - Add model info tag under each assistant message in dashboard chat, persisted in conversation JSON - Rewrite homeai-agent/setup.sh: loads .env, injects API keys into plists, symlinks plists to ~/Library/LaunchAgents/, smoke tests services - Update install_service() in common.sh to use symlinks instead of copies - Open UFW ports on Pi for Music Assistant (8095, 8097, 8927) - Add ANTHROPIC_API_KEY to openclaw + bridge launchd plists Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 22:21:28 +00:00
parent 60eb89ea42
commit 117254d560
17 changed files with 1399 additions and 361 deletions
--- a/plans/OPENCLAW_SKILLS.md
+++ b/plans/OPENCLAW_SKILLS.md
@@ -0,0 +1,389 @@
+# OpenClaw Skills Expansion Plan
+
+## Context
+
+The HomeAI project has 4 custom OpenClaw skills (home-assistant, voice-assistant, image-generation, vtube-studio) and a working voice pipeline. The user wants to build 8 new skills plus a Public/Private mode system to dramatically expand what the assistant can do via voice and chat.
+
+## Skill Format Convention
+
+Every skill follows the established pattern from ha-ctl:
+
+- Lives in `~/.openclaw/skills/<name>/`
+- `SKILL.md` with YAML frontmatter (name, description) + agent instructions
+- Optional Python CLI (stdlib only: `urllib.request`, `json`, `os`, `sys`, `re`, `datetime`)
+- CLI symlinked to `/opt/homebrew/bin/` for PATH access
+- Agent invokes via `exec` tool
+- Entry added to `~/.openclaw/workspace/TOOLS.md` for reinforcement
+- New env vars added to `homeai-agent/launchd/com.homeai.openclaw.plist`
+
+---
+
+## Phase A — Core Skills (no new services needed)
+
+### 1. Memory Recall (`memory-ctl`)
+
+**Purpose:** Let the agent actively store, search, and recall memories mid-conversation.
+
+**Files:**
+- `~/.openclaw/skills/memory/SKILL.md`
+- `~/.openclaw/skills/memory/memory-ctl` → symlink `/opt/homebrew/bin/memory-ctl`
+
+**Commands:**
+```
+memory-ctl add <personal|general> "<content>" [--category preference|fact|routine] [--character-id ID]
+memory-ctl search "<query>" [--type personal|general] [--character-id ID]
+memory-ctl list [--type personal|general] [--character-id ID] [--limit 10]
+memory-ctl delete <memory_id> [--type personal|general] [--character-id ID]
+```
+
+**Details:**
+- Reads/writes existing files: `~/homeai-data/memories/personal/{char_id}.json` and `general.json`
+- Matches existing schema: `{"memories": [{"id": "m_<timestamp>", "content": "...", "category": "...", "createdAt": "..."}]}`
+- Search: keyword token matching (split query, score by substring hits in content)
+- `--character-id` defaults to `HOMEAI_CHARACTER_ID` env var or satellite-map default
+- Dashboard memory UI will immediately reflect agent-created memories (same files)
+- **Env vars:** `HOMEAI_CHARACTER_ID` (optional, set by bridge)
+
+---
+
+### 2. Service Monitor (`monitor-ctl`)
+
+**Purpose:** "Is everything running?" → spoken health report.
+
+**Files:**
+- `~/.openclaw/skills/service-monitor/SKILL.md`
+- `~/.openclaw/skills/service-monitor/monitor-ctl` → symlink `/opt/homebrew/bin/monitor-ctl`
+
+**Commands:**
+```
+monitor-ctl status              # All services summary
+monitor-ctl check <service>     # Single service (ollama, bridge, ha, tts, stt, dashboard, n8n, gitea, kuma)
+monitor-ctl ollama              # Ollama-specific: loaded models, VRAM
+monitor-ctl docker              # Docker container status (runs: docker ps --format json)
+```
+
+**Details:**
+- Checks hardcoded service endpoints with 3s timeout:
+  - Ollama (`localhost:11434/api/ps`), Bridge (`localhost:8081/status`), Gateway (`localhost:8080/status`)
+  - Wyoming STT (TCP `localhost:10300`), TTS (TCP `localhost:10301`)
+  - Dashboard (`localhost:5173`), n8n (`localhost:5678`), Kuma (`localhost:3001`)
+  - HA (`10.0.0.199:8123/api/`), Gitea (`10.0.0.199:3000`)
+- `ollama` subcommand parses `/api/ps` for model names, sizes, expiry
+- `docker` runs `docker ps --format '{{json .}}'` via subprocess
+- Pure stdlib (`urllib.request` + `socket.create_connection` for TCP)
+- **Env vars:** Uses existing `HASS_TOKEN`, `HA_URL`
+
+---
+
+### 3. Character Switcher (`character-ctl`)
+
+**Purpose:** "Talk to Aria" → swap persona, TTS voice, system prompt.
+
+**Files:**
+- `~/.openclaw/skills/character/SKILL.md`
+- `~/.openclaw/skills/character/character-ctl` → symlink `/opt/homebrew/bin/character-ctl`
+
+**Commands:**
+```
+character-ctl list                              # All characters (name, id, tts engine)
+character-ctl active                            # Current default character
+character-ctl switch "<name_or_id>"             # Set as default
+character-ctl info "<name_or_id>"               # Profile summary
+character-ctl map <satellite_id> <character_id> # Map satellite → character
+```
+
+**Details:**
+- Reads character JSONs from `~/homeai-data/characters/`
+- `switch` updates `satellite-map.json` default + writes `active-tts-voice.json`
+- Fuzzy name resolution: case-insensitive match on `display_name` → `name` → `id` → partial match
+- Switch takes effect on next bridge request (`SKILL.md` tells agent to inform user)
+- **Env vars:** None new
+
+---
+
+## Phase B — Home Assistant Extensions
+
+### 4. Routine/Scene Builder (`routine-ctl`)
+
+**Purpose:** Create and trigger multi-device scenes and routines from voice.
+
+**Files:**
+- `~/.openclaw/skills/routine/SKILL.md`
+- `~/.openclaw/skills/routine/routine-ctl` → symlink `/opt/homebrew/bin/routine-ctl`
+
+**Commands:**
+```
+routine-ctl list-scenes                         # HA scenes
+routine-ctl list-scripts                        # HA scripts
+routine-ctl trigger "<scene_or_script>"         # Activate
+routine-ctl create-scene "<name>" --entities '[{"entity_id":"light.x","state":"on","brightness":128}]'
+routine-ctl list-routines                       # Local multi-step routines
+routine-ctl create-routine "<name>" --steps '[{"type":"scene","target":"movie_mode"},{"type":"delay","seconds":5},{"type":"ha","cmd":"off \"TV Backlight\""}]'
+routine-ctl run "<routine_name>"                # Execute steps sequentially
+```
+
+**Details:**
+- HA scenes via REST API: `POST /api/services/scene/turn_on`, `POST /api/services/scene/create`
+- Local routines stored in `~/homeai-data/routines/*.json`
+- Step types: `scene` (trigger HA scene), `ha` (subprocess call to ha-ctl), `delay` (sleep), `tts` (curl to bridge `/api/tts`)
+- `run` executes steps sequentially, reports progress
+- **New data path:** `~/homeai-data/routines/`
+- **Env vars:** Uses existing `HASS_TOKEN`, `HA_URL`
+
+---
+
+### 5. Music Control (`music-ctl`)
+
+**Purpose:** Play/control music with multi-room and Spotify support.
+
+**Files:**
+- `~/.openclaw/skills/music/SKILL.md`
+- `~/.openclaw/skills/music/music-ctl` → symlink `/opt/homebrew/bin/music-ctl`
+
+**Commands:**
+```
+music-ctl players                               # List media_player entities
+music-ctl play ["query"] [--player ID]          # Play/resume (search + play if query given)
+music-ctl pause [--player ID]                   # Pause
+music-ctl next / prev [--player ID]             # Skip tracks
+music-ctl volume <0-100> [--player ID]          # Set volume
+music-ctl now-playing [--player ID]             # Current track info
+music-ctl queue [--player ID]                   # Queue contents
+music-ctl shuffle <on|off> [--player ID]        # Toggle shuffle
+music-ctl search "<query>"                      # Search library
+```
+
+**Details:**
+- All commands go through HA `media_player` services (same API pattern as ha-ctl)
+- `play` with query uses `media_player/play_media` with `media_content_type: music`
+- Spotify appears as a `media_player` entity via HA Spotify integration — no separate API needed
+- `players` lists all `media_player` entities (Music Assistant zones, Spotify Connect, Chromecast, etc.)
+- `--player` defaults to first active player or a configurable default
+- Multi-room: Snapcast zones appear as separate `media_player` entities
+- `now-playing` reads state attributes: `media_title`, `media_artist`, `media_album`, `media_position`
+- **Env vars:** Uses existing `HASS_TOKEN`, `HA_URL`
+- **Prerequisite:** Music Assistant Docker container configured + HA integration, OR Spotify HA integration
+
+---
+
+## Phase C — External Service Skills
+
+### 6. n8n Workflow Trigger (`workflow-ctl`)
+
+**Purpose:** List and trigger n8n workflows by voice.
+
+**Files:**
+- `~/.openclaw/skills/workflow/SKILL.md`
+- `~/.openclaw/skills/workflow/workflow-ctl` → symlink `/opt/homebrew/bin/workflow-ctl`
+
+**Commands:**
+```
+workflow-ctl list                                # All workflows (name, active, id)
+workflow-ctl trigger "<name_or_id>" [--data '{"key":"val"}']  # Fire webhook
+workflow-ctl status <execution_id>               # Execution status
+workflow-ctl history [--limit 10]                # Recent executions
+```
+
+**Details:**
+- n8n REST API at `http://localhost:5678/api/v1/`
+- Auth via API key header: `X-N8N-API-KEY`
+- `trigger` prefers webhook trigger (`POST /webhook/<path>`), falls back to `POST /api/v1/workflows/<id>/execute`
+- Fuzzy name matching on workflow names
+- **Env vars (new):** `N8N_URL` (default `http://localhost:5678`), `N8N_API_KEY` (generate in n8n Settings → API)
+
+---
+
+### 7. Gitea Integration (`gitea-ctl`)
+
+**Purpose:** Query self-hosted repos, commits, issues, PRs.
+
+**Files:**
+- `~/.openclaw/skills/gitea/SKILL.md`
+- `~/.openclaw/skills/gitea/gitea-ctl` → symlink `/opt/homebrew/bin/gitea-ctl`
+
+**Commands:**
+```
+gitea-ctl repos [--limit 20]                    # List repos
+gitea-ctl commits <owner/repo> [--limit 10]     # Recent commits
+gitea-ctl issues <owner/repo> [--state open]    # List issues
+gitea-ctl prs <owner/repo> [--state open]       # List PRs
+gitea-ctl create-issue <owner/repo> "<title>" [--body TEXT]
+```
+
+**Details:**
+- Gitea REST API v1 at `http://10.0.0.199:3000/api/v1/`
+- Auth: `Authorization: token <GITEA_TOKEN>`
+- Pure stdlib `urllib.request`
+- **Env vars (new):** `GITEA_URL` (default `http://10.0.0.199:3000`), `GITEA_TOKEN` (generate in Gitea → Settings → Applications)
+
+---
+
+### 8. Calendar/Reminders (`calendar-ctl`)
+
+**Purpose:** Read calendar, create events, set voice reminders.
+
+**Files:**
+- `~/.openclaw/skills/calendar/SKILL.md`
+- `~/.openclaw/skills/calendar/calendar-ctl` → symlink `/opt/homebrew/bin/calendar-ctl`
+
+**Commands:**
+```
+calendar-ctl today [--calendar ID]               # Today's events
+calendar-ctl upcoming [--days 7]                 # Next N days
+calendar-ctl add "<summary>" --start <ISO> --end <ISO> [--calendar ID]
+calendar-ctl remind "<message>" --at "<time>"    # Set reminder (e.g. "in 30 minutes", "at 5pm", "tomorrow 9am")
+calendar-ctl reminders                           # List pending reminders
+calendar-ctl cancel-reminder <id>                # Cancel reminder
+```
+
+**Details:**
+- Calendar read: `GET /api/calendars/<entity_id>?start=<ISO>&end=<ISO>` via HA API
+- Calendar write: `POST /api/services/calendar/create_event`
+- Reminders stored locally in `~/homeai-data/reminders.json`
+- Relative time parsing with `datetime` + `re` (stdlib): "in 30 minutes", "at 5pm", "tomorrow 9am"
+- Reminder daemon (`com.homeai.reminder-daemon`): Python script checking `reminders.json` every 60s, fires TTS via `POST http://localhost:8081/api/tts` when due
+- **New data path:** `~/homeai-data/reminders.json`
+- **New daemon:** `homeai-agent/reminder-daemon.py` + `homeai-agent/launchd/com.homeai.reminder-daemon.plist`
+- **Env vars:** Uses existing `HASS_TOKEN`, `HA_URL`
+
+---
+
+## Phase D — Public/Private Mode System
+
+### 9. Mode Controller (`mode-ctl`)
+
+**Purpose:** Route requests to cloud LLMs (speed/power) or local LLMs (privacy) with per-task rules and manual toggle.
+
+**Files:**
+- `~/.openclaw/skills/mode/SKILL.md`
+- `~/.openclaw/skills/mode/mode-ctl` → symlink `/opt/homebrew/bin/mode-ctl`
+
+**Commands:**
+```
+mode-ctl status                                  # Current mode + overrides
+mode-ctl private                                 # Switch to local-only
+mode-ctl public                                  # Switch to cloud LLMs
+mode-ctl set-provider <anthropic|openai>         # Preferred cloud provider
+mode-ctl override <category> <private|public>    # Per-category routing
+mode-ctl list-overrides                          # Show all overrides
+```
+
+**State file:** `~/homeai-data/active-mode.json`
+
+```json
+{
+  "mode": "private",
+  "cloud_provider": "anthropic",
+  "cloud_model": "claude-sonnet-4-20250514",
+  "overrides": {
+    "web_search": "public",
+    "coding": "public",
+    "personal_finance": "private",
+    "health": "private"
+  },
+  "updated_at": "2026-03-17T..."
+}
+```
+
+**How model routing works — Bridge modification:**
+
+The HTTP bridge (`openclaw-http-bridge.py`) is modified to:
+
+1. New function `load_mode()` reads `active-mode.json`
+2. New function `resolve_model(mode, category=None)` returns model string
+3. In `_handle_agent_request()`, after character resolution, check mode → pass `--model` flag to OpenClaw CLI
+   - **Private:** `ollama/qwen3.5:35b-a3b` (current default, no change)
+   - **Public:** `anthropic/claude-sonnet-4-20250514` or `openai/gpt-4o` (per provider setting)
+
+**OpenClaw config changes (`openclaw.json`):** Add cloud providers to `models.providers`:
+
+```json
+"anthropic": {
+  "baseUrl": "https://api.anthropic.com/v1",
+  "apiKey": "${ANTHROPIC_API_KEY}",
+  "api": "anthropic",
+  "models": [{"id": "claude-sonnet-4-20250514", "contextWindow": 200000, "maxTokens": 8192}]
+},
+"openai": {
+  "baseUrl": "https://api.openai.com/v1",
+  "apiKey": "${OPENAI_API_KEY}",
+  "api": "openai",
+  "models": [{"id": "gpt-4o", "contextWindow": 128000, "maxTokens": 4096}]
+}
+```
+
+**Per-task classification:** The `SKILL.md` provides a category reference table. The agent self-classifies each request and checks overrides. Default categories:
+
+- **Always private:** personal finance, health, passwords, private conversations
+- **Always public:** web search, coding help, complex reasoning, translation
+- **Follow global mode:** general chat, smart home, music, calendar
+
+**Dashboard integration:** Add mode toggle to dashboard sidebar via new Vite middleware endpoint `GET/POST /api/mode` reading/writing `active-mode.json`.
+
+- **Env vars (new):** `ANTHROPIC_API_KEY`, `OPENAI_API_KEY` (add to OpenClaw plist)
+- **Bridge file modified:** `homeai-agent/openclaw-http-bridge.py` — add ~40 lines for mode loading + model resolution
+
+---
+
+## Implementation Order
+
+| # | Skill | Complexity | Dependencies |
+|---|-------|-----------|-------------|
+| 1 | `memory-ctl` | Simple | None |
+| 2 | `monitor-ctl` | Simple | None |
+| 3 | `character-ctl` | Simple | None |
+| 4 | `routine-ctl` | Medium | ha-ctl existing |
+| 5 | `music-ctl` | Medium | Music Assistant or Spotify in HA |
+| 6 | `workflow-ctl` | Simple | n8n API key |
+| 7 | `gitea-ctl` | Simple | Gitea API token |
+| 8 | `calendar-ctl` | Medium | HA calendar + new reminder daemon |
+| 9 | `mode-ctl` | High | Cloud API keys + bridge modification |
+
+## Per-Skill Implementation Steps
+
+For each skill:
+
+1. Create `SKILL.md` with frontmatter + agent instructions + examples
+2. Create Python CLI (`chmod +x`), stdlib only
+3. Symlink to `/opt/homebrew/bin/`
+4. Test CLI standalone: `<tool> --help`, `<tool> <command>`
+5. Add env vars to `com.homeai.openclaw.plist` if needed
+6. Restart OpenClaw: `launchctl kickstart -k gui/501/com.homeai.openclaw`
+7. Add section to `~/.openclaw/workspace/TOOLS.md`
+8. Test via: `openclaw agent --message "test prompt" --agent main`
+9. Test via voice: wake word + spoken command
+
+## Verification
+
+- **Unit test each CLI:** Run each command manually and verify JSON output
+- **Agent test:** `openclaw agent --message "remember that my favorite color is blue"` (memory-ctl)
+- **Voice test:** Wake word → "Is everything running?" → spoken health report (monitor-ctl)
+- **Mode test:** `mode-ctl public` → send a complex query → verify it routes to cloud model in bridge logs
+- **Dashboard test:** Check memory UI shows agent-created memories, mode toggle works
+- **Cross-skill test:** "Switch to Sucy and play some jazz" → character-ctl + music-ctl in one turn
+
+## Critical Files to Modify
+
+| File | Changes |
+|------|---------|
+| `~/.openclaw/workspace/TOOLS.md` | Add sections for all 9 new skills |
+| `homeai-agent/openclaw-http-bridge.py` | Mode routing (Phase D only) |
+| `homeai-agent/launchd/com.homeai.openclaw.plist` | New env vars |
+| `~/.openclaw/openclaw.json` | Add anthropic + openai providers (Phase D) |
+| `homeai-dashboard/vite.config.js` | `/api/mode` endpoint (Phase D) |
+
+## New Files Created
+
+- `~/.openclaw/skills/memory/` (SKILL.md + memory-ctl)
+- `~/.openclaw/skills/service-monitor/` (SKILL.md + monitor-ctl)
+- `~/.openclaw/skills/character/` (SKILL.md + character-ctl)
+- `~/.openclaw/skills/routine/` (SKILL.md + routine-ctl)
+- `~/.openclaw/skills/music/` (SKILL.md + music-ctl)
+- `~/.openclaw/skills/workflow/` (SKILL.md + workflow-ctl)
+- `~/.openclaw/skills/gitea/` (SKILL.md + gitea-ctl)
+- `~/.openclaw/skills/calendar/` (SKILL.md + calendar-ctl + reminder-daemon.py)
+- `~/.openclaw/skills/mode/` (SKILL.md + mode-ctl)
+- `~/homeai-data/routines/` (directory)
+- `~/homeai-data/reminders.json` (file)
+- `~/homeai-data/active-mode.json` (file)
+- `homeai-agent/reminder-daemon.py` + launchd plist