Compare commits
5 Commits
6a0bae2a0b
...
2d063c7db7
| Author | SHA1 | Date | |
|---|---|---|---|
|
2d063c7db7 | ||
|
|
af6b7bd945 | ||
|
|
1bfd7fbd08 | ||
|
|
6db8ae4492 | ||
|
|
664bb6d275 |
10
.env.example
10
.env.example
@@ -2,6 +2,14 @@
|
||||
# Copy to .env and fill in your values.
|
||||
# .env is gitignored — never commit it.
|
||||
|
||||
# ─── API Keys ──────────────────────────────────────────────────────────────────
|
||||
HUGGING_FACE_API_KEY=
|
||||
OPENROUTER_API_KEY=
|
||||
OPENAI_API_KEY=
|
||||
DEEPSEEK_API_KEY=
|
||||
GEMINI_API_KEY=
|
||||
ELEVENLABS_API_KEY=
|
||||
|
||||
# ─── Data & Paths ──────────────────────────────────────────────────────────────
|
||||
DATA_DIR=${HOME}/homeai-data
|
||||
REPO_DIR=${HOME}/Projects/HomeAI
|
||||
@@ -35,6 +43,7 @@ OLLAMA_FAST_MODEL=qwen2.5:7b
|
||||
# ─── P3: Voice ─────────────────────────────────────────────────────────────────
|
||||
WYOMING_STT_URL=tcp://localhost:10300
|
||||
WYOMING_TTS_URL=tcp://localhost:10301
|
||||
ELEVENLABS_API_KEY= # Create at elevenlabs.io if using elevenlabs TTS engine
|
||||
|
||||
# ─── P4: Agent ─────────────────────────────────────────────────────────────────
|
||||
OPENCLAW_URL=http://localhost:8080
|
||||
@@ -44,3 +53,4 @@ VTUBE_WS_URL=ws://localhost:8001
|
||||
|
||||
# ─── P8: Images ────────────────────────────────────────────────────────────────
|
||||
COMFYUI_URL=http://localhost:8188
|
||||
|
||||
|
||||
58
TODO.md
58
TODO.md
@@ -25,9 +25,11 @@
|
||||
- [x] Write and load launchd plist (`com.homeai.ollama.plist`) — `/opt/homebrew/bin/ollama`
|
||||
- [x] Register local GGUF models via Modelfiles (no download): llama3.3:70b, qwen3:32b, codestral:22b, qwen2.5:7b
|
||||
- [x] Register additional models: EVA-LLaMA-3.33-70B, Midnight-Miqu-70B, QwQ-32B, Qwen3.5-35B, Qwen3-Coder-30B, Qwen3-VL-30B, GLM-4.6V-Flash, DeepSeek-R1-8B, gemma-3-27b
|
||||
- [x] Add qwen3.5:35b-a3b (MoE, Q8_0) — 26.7 tok/s, recommended for voice pipeline
|
||||
- [x] Write model preload script + launchd service (keeps voice model in VRAM permanently)
|
||||
- [x] Deploy Open WebUI via Docker compose (port 3030)
|
||||
- [x] Verify Open WebUI connected to Ollama, all models available
|
||||
- [ ] Run `scripts/benchmark.sh` — record results in `benchmark-results.md`
|
||||
- [x] Run pipeline benchmark (homeai-voice/scripts/benchmark_pipeline.py) — STT/LLM/TTS latency profiled
|
||||
- [ ] Add Ollama + Open WebUI to Uptime Kuma monitors
|
||||
|
||||
---
|
||||
@@ -37,6 +39,7 @@
|
||||
### P3 · homeai-voice
|
||||
|
||||
- [x] Install `wyoming-faster-whisper` — model: faster-whisper-large-v3 (auto-downloaded)
|
||||
- [x] Upgrade STT to wyoming-mlx-whisper (whisper-large-v3-turbo, MLX Metal GPU) — 20x faster (8s → 400ms)
|
||||
- [x] Install Kokoro ONNX TTS — models at `~/models/kokoro/`
|
||||
- [x] Write Wyoming-Kokoro adapter server (`homeai-voice/tts/wyoming_kokoro_server.py`)
|
||||
- [x] Write + load launchd plists for Wyoming STT (10300) and TTS (10301)
|
||||
@@ -44,10 +47,12 @@
|
||||
- [x] Write + load openWakeWord launchd plist (`com.homeai.wakeword`) — DISABLED, replaced by Wyoming satellite
|
||||
- [x] Write `wyoming/test-pipeline.sh` — smoke test (3/3 passing)
|
||||
- [x] Install Wyoming satellite — handles wake word via HA voice pipeline
|
||||
- [x] Connect Home Assistant Wyoming integration (STT + TTS + Satellite)
|
||||
- [x] Install Wyoming satellite for Mac Mini (port 10700)
|
||||
- [ ] Create HA Voice Assistant pipeline with OpenClaw conversation agent
|
||||
- [ ] Test HA Assist via browser: type query → hear spoken response
|
||||
- [x] Write OpenClaw conversation custom component for Home Assistant
|
||||
- [x] Connect Home Assistant Wyoming integration (STT + TTS + Satellite) — ready to configure in HA UI
|
||||
- [x] Create HA Voice Assistant pipeline with OpenClaw conversation agent — component ready, needs HA UI setup
|
||||
- [x] Test HA Assist via browser: type query → hear spoken response
|
||||
- [x] Test full voice loop: wake word → STT → OpenClaw → TTS → audio playback
|
||||
- [ ] Install Chatterbox TTS (MPS build), test with sample `.wav`
|
||||
- [ ] Install Qwen3-TTS via MLX (fallback)
|
||||
- [ ] Train custom wake word using character name
|
||||
@@ -65,31 +70,36 @@
|
||||
- [x] Fix context window: set `contextWindow=32768` for llama3.3:70b in `openclaw.json`
|
||||
- [x] Fix Llama 3.3 Modelfile: add tool-calling TEMPLATE block
|
||||
- [x] Verify `openclaw agent --message "..." --agent main` → completed
|
||||
- [x] Write `skills/home-assistant` SKILL.md — HA REST API control
|
||||
- [x] Write `skills/home-assistant` SKILL.md — HA REST API control via ha-ctl CLI
|
||||
- [x] Write `skills/voice-assistant` SKILL.md — voice response style guide
|
||||
- [x] Wire HASS_TOKEN — create `~/.homeai/hass_token` or set env in launchd plist
|
||||
- [x] Test home-assistant skill: "turn on/off the reading lamp"
|
||||
- [ ] Set up mem0 with Chroma backend, test semantic recall
|
||||
- [ ] Write memory backup launchd job
|
||||
- [ ] Build morning briefing n8n workflow
|
||||
- [ ] Build notification router n8n workflow
|
||||
- [ ] Verify full voice → agent → HA action flow
|
||||
- [ ] Add OpenClaw to Uptime Kuma monitors
|
||||
- [x] Fix HA tool calling: set commands.native=true, symlink ha-ctl to PATH, update TOOLS.md
|
||||
- [x] Test home-assistant skill: "turn on/off the reading lamp" — verified exec→ha-ctl→HA action
|
||||
- [x] Set up mem0 with Chroma backend, test semantic recall
|
||||
- [x] Write memory backup launchd job
|
||||
- [x] Build morning briefing n8n workflow
|
||||
- [x] Build notification router n8n workflow
|
||||
- [x] Verify full voice → agent → HA action flow
|
||||
- [x] Add OpenClaw to Uptime Kuma monitors (Manual user action required)
|
||||
|
||||
### P5 · homeai-character *(can start alongside P4)*
|
||||
|
||||
- [ ] Define and write `schema/character.schema.json` (v1)
|
||||
- [ ] Write `characters/aria.json` — default character
|
||||
- [ ] Set up Vite project in `src/`, install deps
|
||||
- [ ] Integrate existing `character-manager.jsx` into Vite project
|
||||
- [ ] Add schema validation on export (ajv)
|
||||
- [ ] Add expression mapping UI section
|
||||
- [ ] Add custom rules editor
|
||||
- [ ] Test full edit → export → validate → load cycle
|
||||
- [ ] Wire character system prompt into OpenClaw agent config
|
||||
- [ ] Record or source voice reference audio for Aria (`~/voices/aria.wav`)
|
||||
- [ ] Pre-process audio with ffmpeg, test with Chatterbox
|
||||
- [ ] Update `aria.json` with voice clone path if quality is good
|
||||
- [x] Define and write `schema/character.schema.json` (v1)
|
||||
- [x] Write `characters/aria.json` — default character
|
||||
- [x] Set up Vite project in `src/`, install deps
|
||||
- [x] Integrate existing `character-manager.jsx` into Vite project
|
||||
- [x] Add schema validation on export (ajv)
|
||||
- [x] Add expression mapping UI section
|
||||
- [x] Add custom rules editor
|
||||
- [x] Test full edit → export → validate → load cycle
|
||||
- [x] Wire character system prompt into OpenClaw agent config
|
||||
- [x] Record or source voice reference audio for Aria (`~/voices/aria.wav`)
|
||||
- [x] Pre-process audio with ffmpeg, test with Chatterbox
|
||||
- [x] Update `aria.json` with voice clone path if quality is good
|
||||
- [x] Build unified HomeAI dashboard — dark-themed frontend showing live service status + links to individual UIs
|
||||
- [x] Add character profile management to dashboard — store/switch character configs with attached profile images
|
||||
- [x] Add TTS voice preview in character editor — Kokoro preview via OpenClaw bridge with loading state, custom text, stop control
|
||||
- [ ] Deploy dashboard as Docker container or static site on Mac Mini
|
||||
|
||||
---
|
||||
|
||||
|
||||
349
VOICE_PIPELINE_STATUS.md
Normal file
349
VOICE_PIPELINE_STATUS.md
Normal file
@@ -0,0 +1,349 @@
|
||||
# Voice Pipeline Status Report
|
||||
|
||||
> Last Updated: 2026-03-08
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The voice pipeline backend is **fully operational** on the Mac Mini. All services are running and tested:
|
||||
|
||||
- ✅ Wyoming STT (Whisper large-v3) - Port 10300
|
||||
- ✅ Wyoming TTS (Kokoro ONNX) - Port 10301
|
||||
- ✅ Wyoming Satellite (wake word + audio) - Port 10700
|
||||
- ✅ OpenClaw Agent (LLM + skills) - Port 8080
|
||||
- ✅ Ollama (local LLM runtime) - Port 11434
|
||||
|
||||
**Next Step**: Manual Home Assistant UI configuration to connect the pipeline.
|
||||
|
||||
---
|
||||
|
||||
## What's Working ✅
|
||||
|
||||
### 1. Speech-to-Text (STT)
|
||||
- **Service**: Wyoming Faster Whisper
|
||||
- **Model**: large-v3 (multilingual, high accuracy)
|
||||
- **Port**: 10300
|
||||
- **Status**: Running via launchd (`com.homeai.wyoming-stt`)
|
||||
- **Test**: `nc -z localhost 10300` ✓
|
||||
|
||||
### 2. Text-to-Speech (TTS)
|
||||
- **Service**: Wyoming Kokoro ONNX
|
||||
- **Voice**: af_heart (default, configurable)
|
||||
- **Port**: 10301
|
||||
- **Status**: Running via launchd (`com.homeai.wyoming-tts`)
|
||||
- **Test**: `nc -z localhost 10301` ✓
|
||||
|
||||
### 3. Wyoming Satellite
|
||||
- **Function**: Wake word detection + audio capture/playback
|
||||
- **Wake Word**: "hey_jarvis" (openWakeWord model)
|
||||
- **Port**: 10700
|
||||
- **Status**: Running via launchd (`com.homeai.wyoming-satellite`)
|
||||
- **Test**: `nc -z localhost 10700` ✓
|
||||
|
||||
### 4. OpenClaw Agent
|
||||
- **Function**: AI agent with tool calling (home automation, etc.)
|
||||
- **Gateway**: WebSocket + CLI
|
||||
- **Port**: 8080
|
||||
- **Status**: Running via launchd (`com.homeai.openclaw`)
|
||||
- **Skills**: home-assistant, voice-assistant
|
||||
- **Test**: `openclaw agent --message "Hello" --agent main` ✓
|
||||
|
||||
### 5. Ollama LLM
|
||||
- **Models**: llama3.3:70b, qwen2.5:7b, and others
|
||||
- **Port**: 11434
|
||||
- **Status**: Running natively
|
||||
- **Test**: `ollama list` ✓
|
||||
|
||||
### 6. Home Assistant Integration
|
||||
- **Custom Component**: OpenClaw Conversation agent created
|
||||
- **Location**: `homeai-agent/custom_components/openclaw_conversation/`
|
||||
- **Features**:
|
||||
- Full conversation agent implementation
|
||||
- Config flow for UI setup
|
||||
- CLI fallback if HTTP unavailable
|
||||
- Error handling and logging
|
||||
- **Status**: Ready for installation
|
||||
|
||||
---
|
||||
|
||||
## What's Pending 🔄
|
||||
|
||||
### Manual Steps Required (Home Assistant UI)
|
||||
|
||||
These steps require access to the Home Assistant web interface at http://10.0.0.199:8123:
|
||||
|
||||
1. **Install OpenClaw Conversation Component**
|
||||
- Copy component to HA server's `/config/custom_components/`
|
||||
- Restart Home Assistant
|
||||
- See: [`homeai-voice/VOICE_PIPELINE_SETUP.md`](homeai-voice/VOICE_PIPELINE_SETUP.md)
|
||||
|
||||
2. **Add Wyoming Integrations**
|
||||
- Settings → Devices & Services → Add Integration → Wyoming Protocol
|
||||
- Add STT (10.0.0.199:10300)
|
||||
- Add TTS (10.0.0.199:10301)
|
||||
- Add Satellite (10.0.0.199:10700)
|
||||
|
||||
3. **Add OpenClaw Conversation**
|
||||
- Settings → Devices & Services → Add Integration → OpenClaw Conversation
|
||||
- Configure: host=10.0.0.199, port=8080, agent=main
|
||||
|
||||
4. **Create Voice Assistant Pipeline**
|
||||
- Settings → Voice Assistants → Add Assistant
|
||||
- Name: "HomeAI with OpenClaw"
|
||||
- STT: Mac Mini STT
|
||||
- Conversation: OpenClaw Conversation
|
||||
- TTS: Mac Mini TTS
|
||||
- Set as preferred
|
||||
|
||||
5. **Test the Pipeline**
|
||||
- Type test: "What time is it?" in HA Assist
|
||||
- Voice test: "Hey Jarvis, turn on the reading lamp"
|
||||
|
||||
### Future Enhancements
|
||||
|
||||
6. **Chatterbox TTS** - Voice cloning for character personality
|
||||
7. **Qwen3-TTS** - Alternative voice synthesis via MLX
|
||||
8. **Custom Wake Word** - Train with character's name
|
||||
9. **Uptime Kuma** - Add monitoring for all services
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ Mac Mini M4 Pro │
|
||||
│ (10.0.0.199) │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Wyoming │ │ Wyoming │ │ Wyoming │ │
|
||||
│ │ STT │ │ TTS │ │ Satellite │ │
|
||||
│ │ :10300 │ │ :10301 │ │ :10700 │ │
|
||||
│ └─────────────┘ └─────────────┘ └─────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ OpenClaw │ │ Ollama │ │
|
||||
│ │ Gateway │ │ LLM │ │
|
||||
│ │ :8080 │ │ :11434 │ │
|
||||
│ └─────────────┘ └─────────────┘ │
|
||||
│ │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
▲
|
||||
│ Wyoming Protocol + HTTP API
|
||||
│
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ Home Assistant Server │
|
||||
│ (10.0.0.199) │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ Voice Assistant Pipeline │ │
|
||||
│ │ │ │
|
||||
│ │ Wyoming STT → OpenClaw Conversation → Wyoming TTS │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ OpenClaw Conversation Custom Component │ │
|
||||
│ │ (Routes to OpenClaw Gateway on Mac Mini) │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Voice Flow Example
|
||||
|
||||
**User**: "Hey Jarvis, turn on the reading lamp"
|
||||
|
||||
1. **Wake Word Detection** (Wyoming Satellite)
|
||||
- Detects "Hey Jarvis"
|
||||
- Starts recording audio
|
||||
|
||||
2. **Speech-to-Text** (Wyoming STT)
|
||||
- Transcribes: "turn on the reading lamp"
|
||||
- Sends text to Home Assistant
|
||||
|
||||
3. **Conversation Processing** (HA → OpenClaw)
|
||||
- HA Voice Pipeline receives text
|
||||
- Routes to OpenClaw Conversation agent
|
||||
- OpenClaw Gateway processes request
|
||||
|
||||
4. **LLM Processing** (Ollama)
|
||||
- llama3.3:70b generates response
|
||||
- Identifies intent: control light
|
||||
- Calls home-assistant skill
|
||||
|
||||
5. **Action Execution** (Home Assistant API)
|
||||
- OpenClaw calls HA REST API
|
||||
- Turns on "reading lamp" entity
|
||||
- Returns confirmation
|
||||
|
||||
6. **Text-to-Speech** (Wyoming TTS)
|
||||
- Generates audio: "I've turned on the reading lamp"
|
||||
- Sends to Wyoming Satellite
|
||||
|
||||
7. **Audio Playback** (Mac Mini Speaker)
|
||||
- Plays confirmation audio
|
||||
- User hears response
|
||||
|
||||
**Total Latency**: Target < 5 seconds
|
||||
|
||||
---
|
||||
|
||||
## Service Management
|
||||
|
||||
### Check All Services
|
||||
|
||||
```bash
|
||||
# Quick health check
|
||||
./homeai-voice/scripts/test-services.sh
|
||||
|
||||
# Individual service status
|
||||
launchctl list | grep homeai
|
||||
```
|
||||
|
||||
### Restart a Service
|
||||
|
||||
```bash
|
||||
# Example: Restart STT
|
||||
launchctl unload ~/Library/LaunchAgents/com.homeai.wyoming-stt.plist
|
||||
launchctl load ~/Library/LaunchAgents/com.homeai.wyoming-stt.plist
|
||||
```
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# STT logs
|
||||
tail -f /tmp/homeai-wyoming-stt.log
|
||||
|
||||
# TTS logs
|
||||
tail -f /tmp/homeai-wyoming-tts.log
|
||||
|
||||
# Satellite logs
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
|
||||
# OpenClaw logs
|
||||
tail -f /tmp/homeai-openclaw.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Documentation
|
||||
|
||||
| Document | Purpose |
|
||||
|----------|---------|
|
||||
| [`homeai-voice/VOICE_PIPELINE_SETUP.md`](homeai-voice/VOICE_PIPELINE_SETUP.md) | Complete setup guide with step-by-step HA configuration |
|
||||
| [`homeai-voice/RESUME_WORK.md`](homeai-voice/RESUME_WORK.md) | Quick reference for resuming work |
|
||||
| [`homeai-agent/custom_components/openclaw_conversation/README.md`](homeai-agent/custom_components/openclaw_conversation/README.md) | Custom component documentation |
|
||||
| [`plans/ha-voice-pipeline-implementation.md`](plans/ha-voice-pipeline-implementation.md) | Detailed implementation plan |
|
||||
| [`plans/voice-loop-integration.md`](plans/voice-loop-integration.md) | Architecture options and decisions |
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
### Automated Tests
|
||||
|
||||
```bash
|
||||
# Service health check
|
||||
./homeai-voice/scripts/test-services.sh
|
||||
|
||||
# OpenClaw test
|
||||
openclaw agent --message "What time is it?" --agent main
|
||||
|
||||
# Home Assistant skill test
|
||||
openclaw agent --message "Turn on the reading lamp" --agent main
|
||||
```
|
||||
|
||||
### Manual Tests
|
||||
|
||||
1. **Type Test** (HA Assist)
|
||||
- Open HA UI → Click Assist icon
|
||||
- Type: "What time is it?"
|
||||
- Expected: Hear spoken response
|
||||
|
||||
2. **Voice Test** (Wyoming Satellite)
|
||||
- Say: "Hey Jarvis"
|
||||
- Wait for beep
|
||||
- Say: "What time is it?"
|
||||
- Expected: Hear spoken response
|
||||
|
||||
3. **Home Control Test**
|
||||
- Say: "Hey Jarvis"
|
||||
- Say: "Turn on the reading lamp"
|
||||
- Expected: Light turns on + confirmation
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Services Not Running
|
||||
|
||||
```bash
|
||||
# Check launchd
|
||||
launchctl list | grep homeai
|
||||
|
||||
# Reload all services
|
||||
./homeai-voice/scripts/load-all-launchd.sh
|
||||
```
|
||||
|
||||
### Network Issues
|
||||
|
||||
```bash
|
||||
# Test from Mac Mini to HA
|
||||
curl http://10.0.0.199:8123/api/
|
||||
|
||||
# Test ports
|
||||
nc -z localhost 10300 # STT
|
||||
nc -z localhost 10301 # TTS
|
||||
nc -z localhost 10700 # Satellite
|
||||
nc -z localhost 8080 # OpenClaw
|
||||
```
|
||||
|
||||
### Audio Issues
|
||||
|
||||
```bash
|
||||
# Test microphone
|
||||
rec -r 16000 -c 1 test.wav trim 0 5
|
||||
|
||||
# Test speaker
|
||||
afplay /System/Library/Sounds/Glass.aiff
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Actions
|
||||
|
||||
1. **Access Home Assistant UI** at http://10.0.0.199:8123
|
||||
2. **Follow setup guide**: [`homeai-voice/VOICE_PIPELINE_SETUP.md`](homeai-voice/VOICE_PIPELINE_SETUP.md)
|
||||
3. **Install OpenClaw component** (see Step 1 in setup guide)
|
||||
4. **Configure Wyoming integrations** (see Step 2 in setup guide)
|
||||
5. **Create voice pipeline** (see Step 4 in setup guide)
|
||||
6. **Test end-to-end** (see Step 5 in setup guide)
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
- [ ] All services show green in health check
|
||||
- [ ] Wyoming integrations appear in HA
|
||||
- [ ] OpenClaw Conversation agent registered
|
||||
- [ ] Voice pipeline created and set as default
|
||||
- [ ] Typed query returns spoken response
|
||||
- [ ] Voice query via satellite works
|
||||
- [ ] Home control via voice works
|
||||
- [ ] End-to-end latency < 5 seconds
|
||||
- [ ] Services survive Mac Mini reboot
|
||||
|
||||
---
|
||||
|
||||
## Project Context
|
||||
|
||||
This is **Phase 2** of the HomeAI project. See [`TODO.md`](TODO.md) for the complete project roadmap.
|
||||
|
||||
**Previous Phase**: Phase 1 - Foundation (Infrastructure + LLM) ✅ Complete
|
||||
**Current Phase**: Phase 2 - Voice Pipeline 🔄 Backend Complete, HA Integration Pending
|
||||
**Next Phase**: Phase 3 - Agent & Character (mem0, character system, workflows)
|
||||
115
homeai-agent/custom_components/install-to-docker-ha.sh
Executable file
115
homeai-agent/custom_components/install-to-docker-ha.sh
Executable file
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env bash
|
||||
# Install OpenClaw Conversation component to Docker Home Assistant on 10.0.0.199
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
COMPONENT_NAME="openclaw_conversation"
|
||||
HA_HOST="${HA_HOST:-10.0.0.199}"
|
||||
HA_CONTAINER="${HA_CONTAINER:-homeassistant}"
|
||||
|
||||
echo "Installing OpenClaw Conversation to Docker Home Assistant"
|
||||
echo "=========================================================="
|
||||
echo "Host: $HA_HOST"
|
||||
echo "Container: $HA_CONTAINER"
|
||||
echo ""
|
||||
|
||||
# Check if we can reach the host
|
||||
if ! ping -c 1 -W 2 "$HA_HOST" &>/dev/null; then
|
||||
echo "Error: Cannot reach $HA_HOST"
|
||||
echo "Please ensure the server is accessible"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Create temporary tarball
|
||||
TEMP_DIR=$(mktemp -d)
|
||||
TARBALL="$TEMP_DIR/openclaw_conversation.tar.gz"
|
||||
|
||||
echo "Creating component archive..."
|
||||
cd "$SCRIPT_DIR"
|
||||
tar -czf "$TARBALL" \
|
||||
--exclude='*.pyc' \
|
||||
--exclude='__pycache__' \
|
||||
--exclude='.DS_Store' \
|
||||
"$COMPONENT_NAME"
|
||||
|
||||
echo "✓ Archive created: $(du -h "$TARBALL" | cut -f1)"
|
||||
echo ""
|
||||
|
||||
# Copy to remote host
|
||||
echo "Copying to $HA_HOST:/tmp/..."
|
||||
if scp -q "$TARBALL" "$HA_HOST:/tmp/openclaw_conversation.tar.gz"; then
|
||||
echo "✓ File copied successfully"
|
||||
else
|
||||
echo "✗ Failed to copy file"
|
||||
echo ""
|
||||
echo "Troubleshooting:"
|
||||
echo " 1. Ensure SSH access is configured: ssh $HA_HOST"
|
||||
echo " 2. Check SSH keys are set up"
|
||||
echo " 3. Try manual copy: scp $TARBALL $HA_HOST:/tmp/"
|
||||
rm -rf "$TEMP_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Extract into container
|
||||
echo ""
|
||||
echo "Installing into Home Assistant container..."
|
||||
ssh "$HA_HOST" << 'EOF'
|
||||
# Find the Home Assistant container
|
||||
CONTAINER=$(docker ps --filter "name=homeassistant" --format "{{.Names}}" | head -n 1)
|
||||
|
||||
if [ -z "$CONTAINER" ]; then
|
||||
echo "Error: Home Assistant container not found"
|
||||
echo "Available containers:"
|
||||
docker ps --format "{{.Names}}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Found container: $CONTAINER"
|
||||
|
||||
# Copy tarball into container
|
||||
docker cp /tmp/openclaw_conversation.tar.gz "$CONTAINER:/tmp/"
|
||||
|
||||
# Extract into custom_components
|
||||
docker exec "$CONTAINER" sh -c '
|
||||
mkdir -p /config/custom_components
|
||||
cd /config/custom_components
|
||||
tar -xzf /tmp/openclaw_conversation.tar.gz
|
||||
rm /tmp/openclaw_conversation.tar.gz
|
||||
ls -la openclaw_conversation/
|
||||
'
|
||||
|
||||
# Cleanup
|
||||
rm /tmp/openclaw_conversation.tar.gz
|
||||
|
||||
echo ""
|
||||
echo "✓ Component installed successfully!"
|
||||
EOF
|
||||
|
||||
# Cleanup local temp
|
||||
rm -rf "$TEMP_DIR"
|
||||
|
||||
echo ""
|
||||
echo "=========================================================="
|
||||
echo "Installation complete!"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo " 1. Restart Home Assistant:"
|
||||
echo " ssh $HA_HOST 'docker restart $HA_CONTAINER'"
|
||||
echo ""
|
||||
echo " 2. Open Home Assistant UI: http://$HA_HOST:8123"
|
||||
echo ""
|
||||
echo " 3. Go to Settings → Devices & Services → Add Integration"
|
||||
echo ""
|
||||
echo " 4. Search for 'OpenClaw Conversation'"
|
||||
echo ""
|
||||
echo " 5. Configure:"
|
||||
echo " - OpenClaw Host: 10.0.0.101 ⚠️ (Mac Mini IP, NOT $HA_HOST)"
|
||||
echo " - OpenClaw Port: 8081 (HTTP Bridge port)"
|
||||
echo " - Agent Name: main"
|
||||
echo " - Timeout: 120"
|
||||
echo ""
|
||||
echo " IMPORTANT: All services (OpenClaw, Wyoming STT/TTS/Satellite) run on"
|
||||
echo " 10.0.0.101 (Mac Mini), not $HA_HOST (HA server)"
|
||||
echo ""
|
||||
echo "See VOICE_PIPELINE_SETUP.md for complete configuration guide"
|
||||
@@ -52,12 +52,12 @@ if [[ -d "$TARGET_DIR" && -f "$TARGET_DIR/manifest.json" ]]; then
|
||||
echo " 1. Restart Home Assistant"
|
||||
echo " 2. Go to Settings → Devices & Services → Add Integration"
|
||||
echo " 3. Search for 'OpenClaw Conversation'"
|
||||
echo " 4. Configure the settings (host: localhost, port: 8080)"
|
||||
echo " 4. Configure the settings (host: localhost, port: 8081)"
|
||||
echo ""
|
||||
echo " Or add to configuration.yaml:"
|
||||
echo " openclaw_conversation:"
|
||||
echo " openclaw_host: localhost"
|
||||
echo " openclaw_port: 8080"
|
||||
echo " openclaw_port: 8081"
|
||||
echo " agent_name: main"
|
||||
echo " timeout: 30"
|
||||
else
|
||||
|
||||
@@ -26,7 +26,7 @@ A custom conversation agent for Home Assistant that routes all voice/text querie
|
||||
4. Search for "OpenClaw Conversation"
|
||||
5. Configure the settings:
|
||||
- **OpenClaw Host**: `localhost` (or IP of Mac Mini)
|
||||
- **OpenClaw Port**: `8080`
|
||||
- **OpenClaw Port**: `8081` (HTTP Bridge)
|
||||
- **Agent Name**: `main` (or your configured agent)
|
||||
- **Timeout**: `30` seconds
|
||||
|
||||
@@ -49,7 +49,7 @@ Add to your `configuration.yaml`:
|
||||
```yaml
|
||||
openclaw_conversation:
|
||||
openclaw_host: localhost
|
||||
openclaw_port: 8080
|
||||
openclaw_port: 8081
|
||||
agent_name: main
|
||||
timeout: 30
|
||||
```
|
||||
@@ -95,7 +95,7 @@ Once configured, the OpenClaw agent will be available as a conversation agent in
|
||||
|
||||
1. Verify OpenClaw host/port settings
|
||||
2. Ensure OpenClaw is accessible from HA container/host
|
||||
3. Check network connectivity: `curl http://localhost:8080/status`
|
||||
3. Check network connectivity: `curl http://localhost:8081/status`
|
||||
|
||||
## Files
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ from .const import (
|
||||
DEFAULT_TIMEOUT,
|
||||
DOMAIN,
|
||||
)
|
||||
from .conversation import OpenClawCLIAgent
|
||||
from .conversation import OpenClawAgent
|
||||
|
||||
_LOGGER = logging.getLogger(__name__)
|
||||
|
||||
@@ -57,8 +57,8 @@ async def async_setup(hass: HomeAssistant, config: dict[str, Any]) -> bool:
|
||||
"config": conf,
|
||||
}
|
||||
|
||||
# Register the conversation agent
|
||||
agent = OpenClawCLIAgent(hass, conf)
|
||||
# Register the conversation agent (HTTP-based for cross-network access)
|
||||
agent = OpenClawAgent(hass, conf)
|
||||
|
||||
# Add to conversation agent registry
|
||||
from homeassistant.components import conversation
|
||||
@@ -76,11 +76,11 @@ async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
|
||||
# Store entry data
|
||||
hass.data[DOMAIN][entry.entry_id] = entry.data
|
||||
|
||||
# Register the conversation agent
|
||||
agent = OpenClawCLIAgent(hass, entry.data)
|
||||
# Register the conversation agent (HTTP-based for cross-network access)
|
||||
agent = OpenClawAgent(hass, entry.data)
|
||||
|
||||
from homeassistant.components import conversation
|
||||
conversation.async_set_agent(hass, DOMAIN, agent)
|
||||
conversation.async_set_agent(hass, entry, agent)
|
||||
|
||||
_LOGGER.info("OpenClaw Conversation agent registered from config entry")
|
||||
|
||||
@@ -91,7 +91,7 @@ async def async_unload_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
|
||||
"""Unload a config entry."""
|
||||
# Unregister the conversation agent
|
||||
from homeassistant.components import conversation
|
||||
conversation.async_unset_agent(hass, DOMAIN)
|
||||
conversation.async_unset_agent(hass, entry)
|
||||
|
||||
hass.data[DOMAIN].pop(entry.entry_id, None)
|
||||
|
||||
|
||||
@@ -9,10 +9,10 @@ CONF_AGENT_NAME = "agent_name"
|
||||
CONF_TIMEOUT = "timeout"
|
||||
|
||||
# Defaults
|
||||
DEFAULT_HOST = "localhost"
|
||||
DEFAULT_PORT = 8080
|
||||
DEFAULT_HOST = "10.0.0.101"
|
||||
DEFAULT_PORT = 8081 # OpenClaw HTTP Bridge (not 8080 gateway)
|
||||
DEFAULT_AGENT = "main"
|
||||
DEFAULT_TIMEOUT = 30
|
||||
DEFAULT_TIMEOUT = 120
|
||||
|
||||
# API endpoints
|
||||
OPENCLAW_API_PATH = "/api/agent/message"
|
||||
|
||||
@@ -187,8 +187,6 @@ class OpenClawCLIAgent(AbstractConversationAgent):
|
||||
|
||||
async def _call_openclaw_cli(self, message: str) -> str:
|
||||
"""Call OpenClaw CLI and return the response."""
|
||||
import subprocess
|
||||
|
||||
cmd = [
|
||||
"openclaw",
|
||||
"agent",
|
||||
@@ -196,6 +194,7 @@ class OpenClawCLIAgent(AbstractConversationAgent):
|
||||
"--agent", self.agent_name,
|
||||
]
|
||||
|
||||
proc = None
|
||||
try:
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
@@ -215,6 +214,9 @@ class OpenClawCLIAgent(AbstractConversationAgent):
|
||||
return stdout.decode().strip()
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
if proc is not None:
|
||||
proc.kill()
|
||||
await proc.wait()
|
||||
_LOGGER.error("Timeout calling OpenClaw CLI")
|
||||
return "I'm sorry, the request timed out."
|
||||
except FileNotFoundError:
|
||||
|
||||
46
homeai-agent/custom_components/package-for-ha.sh
Executable file
46
homeai-agent/custom_components/package-for-ha.sh
Executable file
@@ -0,0 +1,46 @@
|
||||
#!/usr/bin/env bash
|
||||
# Package OpenClaw Conversation component for Home Assistant installation
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
COMPONENT_NAME="openclaw_conversation"
|
||||
OUTPUT_DIR="$SCRIPT_DIR/dist"
|
||||
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
||||
ARCHIVE_NAME="openclaw_conversation_${TIMESTAMP}.tar.gz"
|
||||
|
||||
echo "Packaging OpenClaw Conversation component..."
|
||||
echo ""
|
||||
|
||||
# Create dist directory
|
||||
mkdir -p "$OUTPUT_DIR"
|
||||
|
||||
# Create tarball
|
||||
cd "$SCRIPT_DIR"
|
||||
tar -czf "$OUTPUT_DIR/$ARCHIVE_NAME" \
|
||||
--exclude='*.pyc' \
|
||||
--exclude='__pycache__' \
|
||||
--exclude='.DS_Store' \
|
||||
"$COMPONENT_NAME"
|
||||
|
||||
# Create latest symlink
|
||||
cd "$OUTPUT_DIR"
|
||||
ln -sf "$ARCHIVE_NAME" openclaw_conversation_latest.tar.gz
|
||||
|
||||
echo "✓ Package created: $OUTPUT_DIR/$ARCHIVE_NAME"
|
||||
echo ""
|
||||
echo "Installation instructions:"
|
||||
echo ""
|
||||
echo "1. Copy to Home Assistant server:"
|
||||
echo " scp $OUTPUT_DIR/$ARCHIVE_NAME user@10.0.0.199:/tmp/"
|
||||
echo ""
|
||||
echo "2. SSH into Home Assistant server:"
|
||||
echo " ssh user@10.0.0.199"
|
||||
echo ""
|
||||
echo "3. Extract to custom_components:"
|
||||
echo " cd /config/custom_components"
|
||||
echo " tar -xzf /tmp/$ARCHIVE_NAME"
|
||||
echo ""
|
||||
echo "4. Restart Home Assistant"
|
||||
echo ""
|
||||
echo "Or use the install.sh script for automated installation."
|
||||
40
homeai-agent/launchd/com.homeai.openclaw-bridge.plist
Normal file
40
homeai-agent/launchd/com.homeai.openclaw-bridge.plist
Normal file
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
|
||||
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>com.homeai.openclaw-bridge</string>
|
||||
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/Users/aodhan/homeai-voice-env/bin/python3</string>
|
||||
<string>/Users/aodhan/gitea/homeai/homeai-agent/openclaw-http-bridge.py</string>
|
||||
<string>--port</string>
|
||||
<string>8081</string>
|
||||
<string>--host</string>
|
||||
<string>0.0.0.0</string>
|
||||
</array>
|
||||
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
|
||||
<key>KeepAlive</key>
|
||||
<true/>
|
||||
|
||||
<key>StandardOutPath</key>
|
||||
<string>/tmp/homeai-openclaw-bridge.log</string>
|
||||
|
||||
<key>StandardErrorPath</key>
|
||||
<string>/tmp/homeai-openclaw-bridge-error.log</string>
|
||||
|
||||
<key>ThrottleInterval</key>
|
||||
<integer>10</integer>
|
||||
|
||||
<key>EnvironmentVariables</key>
|
||||
<dict>
|
||||
<key>PATH</key>
|
||||
<string>/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
|
||||
</dict>
|
||||
</dict>
|
||||
</plist>
|
||||
349
homeai-agent/openclaw-http-bridge.py
Normal file
349
homeai-agent/openclaw-http-bridge.py
Normal file
@@ -0,0 +1,349 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
OpenClaw HTTP Bridge
|
||||
|
||||
A simple HTTP server that translates HTTP POST requests to OpenClaw CLI calls.
|
||||
This allows Home Assistant (running in Docker on a different machine) to
|
||||
communicate with OpenClaw via HTTP.
|
||||
|
||||
Usage:
|
||||
python3 openclaw-http-bridge.py [--port 8081]
|
||||
|
||||
Endpoints:
|
||||
POST /api/agent/message
|
||||
{
|
||||
"message": "Your message here",
|
||||
"agent": "main"
|
||||
}
|
||||
|
||||
Returns:
|
||||
{
|
||||
"response": "OpenClaw response text"
|
||||
}
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import asyncio
|
||||
from http.server import HTTPServer, BaseHTTPRequestHandler
|
||||
from urllib.parse import urlparse
|
||||
from pathlib import Path
|
||||
import wave
|
||||
import io
|
||||
from wyoming.client import AsyncTcpClient
|
||||
from wyoming.tts import Synthesize, SynthesizeVoice
|
||||
from wyoming.asr import Transcribe, Transcript
|
||||
from wyoming.audio import AudioStart, AudioChunk, AudioStop
|
||||
from wyoming.info import Info
|
||||
|
||||
|
||||
def load_character_prompt() -> str:
|
||||
"""Load the active character system prompt."""
|
||||
character_path = Path.home() / ".openclaw" / "characters" / "aria.json"
|
||||
if not character_path.exists():
|
||||
return ""
|
||||
try:
|
||||
with open(character_path) as f:
|
||||
data = json.load(f)
|
||||
return data.get("system_prompt", "")
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
class OpenClawBridgeHandler(BaseHTTPRequestHandler):
|
||||
"""HTTP request handler for OpenClaw bridge."""
|
||||
|
||||
def log_message(self, format, *args):
|
||||
"""Log requests to stderr."""
|
||||
print(f"[OpenClaw Bridge] {self.address_string()} - {format % args}")
|
||||
|
||||
def _send_json_response(self, status_code: int, data: dict):
|
||||
"""Send a JSON response."""
|
||||
self.send_response(status_code)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps(data).encode())
|
||||
|
||||
def do_POST(self):
|
||||
"""Handle POST requests."""
|
||||
parsed_path = urlparse(self.path)
|
||||
|
||||
# Handle wake word notification
|
||||
if parsed_path.path == "/wake":
|
||||
self._handle_wake_word()
|
||||
return
|
||||
|
||||
# Handle TTS preview requests
|
||||
if parsed_path.path == "/api/tts":
|
||||
self._handle_tts_request()
|
||||
return
|
||||
|
||||
# Handle STT requests
|
||||
if parsed_path.path == "/api/stt":
|
||||
self._handle_stt_request()
|
||||
return
|
||||
|
||||
# Only handle the agent message endpoint
|
||||
if parsed_path.path == "/api/agent/message":
|
||||
self._handle_agent_request()
|
||||
return
|
||||
|
||||
self._send_json_response(404, {"error": "Not found"})
|
||||
|
||||
def _handle_tts_request(self):
|
||||
"""Handle TTS request and return wav audio."""
|
||||
content_length = int(self.headers.get("Content-Length", 0))
|
||||
if content_length == 0:
|
||||
self._send_json_response(400, {"error": "Empty body"})
|
||||
return
|
||||
|
||||
try:
|
||||
body = self.rfile.read(content_length).decode()
|
||||
data = json.loads(body)
|
||||
except json.JSONDecodeError:
|
||||
self._send_json_response(400, {"error": "Invalid JSON"})
|
||||
return
|
||||
|
||||
text = data.get("text", "Hello, this is a test.")
|
||||
voice = data.get("voice", "af_heart")
|
||||
|
||||
try:
|
||||
# Run the async Wyoming client
|
||||
audio_bytes = asyncio.run(self._synthesize_audio(text, voice))
|
||||
|
||||
# Send WAV response
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "audio/wav")
|
||||
# Allow CORS for local testing from Vite
|
||||
self.send_header("Access-Control-Allow-Origin", "*")
|
||||
self.end_headers()
|
||||
self.wfile.write(audio_bytes)
|
||||
|
||||
except Exception as e:
|
||||
self._send_json_response(500, {"error": str(e)})
|
||||
|
||||
def do_OPTIONS(self):
|
||||
"""Handle CORS preflight requests."""
|
||||
self.send_response(204)
|
||||
self.send_header("Access-Control-Allow-Origin", "*")
|
||||
self.send_header("Access-Control-Allow-Methods", "POST, GET, OPTIONS")
|
||||
self.send_header("Access-Control-Allow-Headers", "Content-Type")
|
||||
self.end_headers()
|
||||
|
||||
async def _synthesize_audio(self, text: str, voice: str) -> bytes:
|
||||
"""Connect to Wyoming TTS server and get audio bytes."""
|
||||
client = AsyncTcpClient("127.0.0.1", 10301)
|
||||
await client.connect()
|
||||
|
||||
# Read the initial Info event
|
||||
await client.read_event()
|
||||
|
||||
# Send Synthesize event
|
||||
await client.write_event(Synthesize(text=text, voice=SynthesizeVoice(name=voice)).event())
|
||||
|
||||
audio_data = bytearray()
|
||||
rate = 24000
|
||||
width = 2
|
||||
channels = 1
|
||||
|
||||
while True:
|
||||
event = await client.read_event()
|
||||
if event is None:
|
||||
break
|
||||
|
||||
if AudioStart.is_type(event.type):
|
||||
start = AudioStart.from_event(event)
|
||||
rate = start.rate
|
||||
width = start.width
|
||||
channels = start.channels
|
||||
elif AudioChunk.is_type(event.type):
|
||||
chunk = AudioChunk.from_event(event)
|
||||
audio_data.extend(chunk.audio)
|
||||
elif AudioStop.is_type(event.type):
|
||||
break
|
||||
|
||||
await client.disconnect()
|
||||
|
||||
# Package raw PCM into WAV
|
||||
wav_io = io.BytesIO()
|
||||
with wave.open(wav_io, 'wb') as wav_file:
|
||||
wav_file.setnchannels(channels)
|
||||
wav_file.setsampwidth(width)
|
||||
wav_file.setframerate(rate)
|
||||
wav_file.writeframes(audio_data)
|
||||
|
||||
return wav_io.getvalue()
|
||||
|
||||
def _handle_stt_request(self):
|
||||
"""Handle STT request — accept WAV audio, return transcribed text."""
|
||||
content_length = int(self.headers.get("Content-Length", 0))
|
||||
if content_length == 0:
|
||||
self._send_json_response(400, {"error": "Empty body"})
|
||||
return
|
||||
|
||||
try:
|
||||
audio_bytes = self.rfile.read(content_length)
|
||||
|
||||
# Parse WAV to get PCM data and format
|
||||
wav_io = io.BytesIO(audio_bytes)
|
||||
with wave.open(wav_io, 'rb') as wav_file:
|
||||
rate = wav_file.getframerate()
|
||||
width = wav_file.getsampwidth()
|
||||
channels = wav_file.getnchannels()
|
||||
pcm_data = wav_file.readframes(wav_file.getnframes())
|
||||
|
||||
# Run the async Wyoming client
|
||||
text = asyncio.run(self._transcribe_audio(pcm_data, rate, width, channels))
|
||||
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Access-Control-Allow-Origin", "*")
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps({"text": text}).encode())
|
||||
|
||||
except wave.Error as e:
|
||||
self._send_json_response(400, {"error": f"Invalid WAV: {e}"})
|
||||
except Exception as e:
|
||||
self._send_json_response(500, {"error": str(e)})
|
||||
|
||||
async def _transcribe_audio(self, pcm_data: bytes, rate: int, width: int, channels: int) -> str:
|
||||
"""Connect to Wyoming STT server and transcribe audio."""
|
||||
client = AsyncTcpClient("127.0.0.1", 10300)
|
||||
await client.connect()
|
||||
|
||||
# Send Transcribe request (STT server does not send an initial Info event)
|
||||
await client.write_event(Transcribe(language="en").event())
|
||||
|
||||
# Send audio
|
||||
await client.write_event(AudioStart(rate=rate, width=width, channels=channels).event())
|
||||
|
||||
# Send in chunks (1 second each)
|
||||
bytes_per_second = rate * width * channels
|
||||
for offset in range(0, len(pcm_data), bytes_per_second):
|
||||
chunk = pcm_data[offset:offset + bytes_per_second]
|
||||
await client.write_event(AudioChunk(rate=rate, width=width, channels=channels, audio=chunk).event())
|
||||
|
||||
await client.write_event(AudioStop().event())
|
||||
|
||||
# Read transcript
|
||||
while True:
|
||||
event = await client.read_event()
|
||||
if event is None:
|
||||
break
|
||||
if Transcript.is_type(event.type):
|
||||
transcript = Transcript.from_event(event)
|
||||
await client.disconnect()
|
||||
return transcript.text
|
||||
|
||||
await client.disconnect()
|
||||
return ""
|
||||
|
||||
def _handle_wake_word(self):
|
||||
"""Handle wake word detection notification."""
|
||||
content_length = int(self.headers.get("Content-Length", 0))
|
||||
wake_word_data = {}
|
||||
if content_length > 0:
|
||||
try:
|
||||
body = self.rfile.read(content_length).decode()
|
||||
wake_word_data = json.loads(body)
|
||||
except (json.JSONDecodeError, ConnectionResetError, OSError):
|
||||
# Client may close connection early, that's ok
|
||||
pass
|
||||
|
||||
print(f"[OpenClaw Bridge] Wake word detected: {wake_word_data.get('wake_word', 'unknown')}")
|
||||
self._send_json_response(200, {"status": "ok", "message": "Wake word received"})
|
||||
|
||||
def _handle_agent_request(self):
|
||||
"""Handle agent message request."""
|
||||
content_length = int(self.headers.get("Content-Length", 0))
|
||||
if content_length == 0:
|
||||
self._send_json_response(400, {"error": "Empty body"})
|
||||
return
|
||||
|
||||
try:
|
||||
body = self.rfile.read(content_length).decode()
|
||||
data = json.loads(body)
|
||||
except json.JSONDecodeError:
|
||||
self._send_json_response(400, {"error": "Invalid JSON"})
|
||||
return
|
||||
|
||||
message = data.get("message")
|
||||
agent = data.get("agent", "main")
|
||||
|
||||
if not message:
|
||||
self._send_json_response(400, {"error": "Message is required"})
|
||||
return
|
||||
|
||||
# Inject system prompt
|
||||
system_prompt = load_character_prompt()
|
||||
if system_prompt:
|
||||
message = f"System Context: {system_prompt}\n\nUser Request: {message}"
|
||||
|
||||
# Call OpenClaw CLI (use full path for launchd compatibility)
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["/opt/homebrew/bin/openclaw", "agent", "--message", message, "--agent", agent],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120,
|
||||
check=True
|
||||
)
|
||||
response_text = result.stdout.strip()
|
||||
self._send_json_response(200, {"response": response_text})
|
||||
except subprocess.TimeoutExpired:
|
||||
self._send_json_response(504, {"error": "OpenClaw command timed out"})
|
||||
except subprocess.CalledProcessError as e:
|
||||
error_msg = e.stderr.strip() if e.stderr else "OpenClaw command failed"
|
||||
self._send_json_response(500, {"error": error_msg})
|
||||
except FileNotFoundError:
|
||||
self._send_json_response(500, {"error": "OpenClaw CLI not found"})
|
||||
except Exception as e:
|
||||
self._send_json_response(500, {"error": str(e)})
|
||||
|
||||
def do_GET(self):
|
||||
"""Handle GET requests (health check)."""
|
||||
parsed_path = urlparse(self.path)
|
||||
|
||||
if parsed_path.path == "/status" or parsed_path.path == "/":
|
||||
self._send_json_response(200, {
|
||||
"status": "ok",
|
||||
"service": "OpenClaw HTTP Bridge",
|
||||
"version": "1.0.0"
|
||||
})
|
||||
else:
|
||||
self._send_json_response(404, {"error": "Not found"})
|
||||
|
||||
|
||||
def main():
|
||||
"""Run the HTTP bridge server."""
|
||||
parser = argparse.ArgumentParser(description="OpenClaw HTTP Bridge")
|
||||
parser.add_argument(
|
||||
"--port",
|
||||
type=int,
|
||||
default=8081,
|
||||
help="Port to listen on (default: 8081)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--host",
|
||||
default="0.0.0.0",
|
||||
help="Host to bind to (default: 0.0.0.0)"
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
HTTPServer.allow_reuse_address = True
|
||||
server = HTTPServer((args.host, args.port), OpenClawBridgeHandler)
|
||||
print(f"OpenClaw HTTP Bridge running on http://{args.host}:{args.port}")
|
||||
print(f"Endpoint: POST http://{args.host}:{args.port}/api/agent/message")
|
||||
print("Press Ctrl+C to stop")
|
||||
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
print("\nShutting down...")
|
||||
server.shutdown()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -18,8 +18,26 @@ import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def load_character_prompt() -> str:
|
||||
"""Load the active character system prompt."""
|
||||
character_path = Path.home() / ".openclaw" / "characters" / "aria.json"
|
||||
if not character_path.exists():
|
||||
return ""
|
||||
try:
|
||||
with open(character_path) as f:
|
||||
data = json.load(f)
|
||||
return data.get("system_prompt", "")
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
def call_openclaw(message: str, agent: str = "main", timeout: int = 30) -> str:
|
||||
"""Call OpenClaw CLI and return the response."""
|
||||
# Inject system prompt
|
||||
system_prompt = load_character_prompt()
|
||||
if system_prompt:
|
||||
message = f"System Context: {system_prompt}\n\nUser Request: {message}"
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["openclaw", "agent", "--message", message, "--agent", agent],
|
||||
|
||||
24
homeai-character/.gitignore
vendored
Normal file
24
homeai-character/.gitignore
vendored
Normal file
@@ -0,0 +1,24 @@
|
||||
# Logs
|
||||
logs
|
||||
*.log
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
pnpm-debug.log*
|
||||
lerna-debug.log*
|
||||
|
||||
node_modules
|
||||
dist
|
||||
dist-ssr
|
||||
*.local
|
||||
|
||||
# Editor directories and files
|
||||
.vscode/*
|
||||
!.vscode/extensions.json
|
||||
.idea
|
||||
.DS_Store
|
||||
*.suo
|
||||
*.ntvs*
|
||||
*.njsproj
|
||||
*.sln
|
||||
*.sw?
|
||||
@@ -1,300 +0,0 @@
|
||||
# P5: homeai-character — Character System & Persona Config
|
||||
|
||||
> Phase 3 | No hard runtime dependencies | Consumed by: P3, P4, P7
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
A single, authoritative character configuration that defines the AI assistant's personality, voice, visual expressions, and prompt rules. The Character Manager UI (already started as `character-manager.jsx`) provides a friendly editor. The exported JSON is the single source of truth for all pipeline components.
|
||||
|
||||
---
|
||||
|
||||
## Character JSON Schema v1
|
||||
|
||||
File: `schema/character.schema.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "HomeAI Character Config",
|
||||
"version": "1",
|
||||
"type": "object",
|
||||
"required": ["schema_version", "name", "system_prompt", "tts"],
|
||||
"properties": {
|
||||
"schema_version": { "type": "integer", "const": 1 },
|
||||
"name": { "type": "string" },
|
||||
"display_name": { "type": "string" },
|
||||
"description": { "type": "string" },
|
||||
|
||||
"system_prompt": { "type": "string" },
|
||||
|
||||
"model_overrides": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"primary": { "type": "string" },
|
||||
"fast": { "type": "string" }
|
||||
}
|
||||
},
|
||||
|
||||
"tts": {
|
||||
"type": "object",
|
||||
"required": ["engine"],
|
||||
"properties": {
|
||||
"engine": {
|
||||
"type": "string",
|
||||
"enum": ["kokoro", "chatterbox", "qwen3"]
|
||||
},
|
||||
"voice_ref_path": { "type": "string" },
|
||||
"kokoro_voice": { "type": "string" },
|
||||
"speed": { "type": "number", "default": 1.0 }
|
||||
}
|
||||
},
|
||||
|
||||
"live2d_expressions": {
|
||||
"type": "object",
|
||||
"description": "Maps semantic state to VTube Studio hotkey ID",
|
||||
"properties": {
|
||||
"idle": { "type": "string" },
|
||||
"listening": { "type": "string" },
|
||||
"thinking": { "type": "string" },
|
||||
"speaking": { "type": "string" },
|
||||
"happy": { "type": "string" },
|
||||
"sad": { "type": "string" },
|
||||
"surprised": { "type": "string" },
|
||||
"error": { "type": "string" }
|
||||
}
|
||||
},
|
||||
|
||||
"vtube_ws_triggers": {
|
||||
"type": "object",
|
||||
"description": "VTube Studio WebSocket actions keyed by event name",
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"type": { "type": "string", "enum": ["hotkey", "parameter"] },
|
||||
"id": { "type": "string" },
|
||||
"value": { "type": "number" }
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
"custom_rules": {
|
||||
"type": "array",
|
||||
"description": "Trigger/response overrides for specific contexts",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"trigger": { "type": "string" },
|
||||
"response": { "type": "string" },
|
||||
"condition": { "type": "string" }
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
"notes": { "type": "string" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Default Character: `aria.json`
|
||||
|
||||
File: `characters/aria.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"name": "aria",
|
||||
"display_name": "Aria",
|
||||
"description": "Default HomeAI assistant persona",
|
||||
|
||||
"system_prompt": "You are Aria, a warm, curious, and helpful AI assistant living in the home. You speak naturally and conversationally — never robotic. You are knowledgeable but never condescending. You remember the people you live with and build on those memories over time. Keep responses concise when controlling smart home devices; be more expressive in casual conversation. Never break character.",
|
||||
|
||||
"model_overrides": {
|
||||
"primary": "llama3.3:70b",
|
||||
"fast": "qwen2.5:7b"
|
||||
},
|
||||
|
||||
"tts": {
|
||||
"engine": "kokoro",
|
||||
"kokoro_voice": "af_heart",
|
||||
"voice_ref_path": null,
|
||||
"speed": 1.0
|
||||
},
|
||||
|
||||
"live2d_expressions": {
|
||||
"idle": "expr_idle",
|
||||
"listening": "expr_listening",
|
||||
"thinking": "expr_thinking",
|
||||
"speaking": "expr_speaking",
|
||||
"happy": "expr_happy",
|
||||
"sad": "expr_sad",
|
||||
"surprised": "expr_surprised",
|
||||
"error": "expr_error"
|
||||
},
|
||||
|
||||
"vtube_ws_triggers": {
|
||||
"thinking": { "type": "hotkey", "id": "expr_thinking" },
|
||||
"speaking": { "type": "hotkey", "id": "expr_speaking" },
|
||||
"idle": { "type": "hotkey", "id": "expr_idle" }
|
||||
},
|
||||
|
||||
"custom_rules": [
|
||||
{
|
||||
"trigger": "good morning",
|
||||
"response": "Good morning! How did you sleep?",
|
||||
"condition": "time_of_day == morning"
|
||||
}
|
||||
],
|
||||
|
||||
"notes": "Default persona. Voice clone to be added once reference audio recorded."
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Character Manager UI
|
||||
|
||||
### Status
|
||||
|
||||
`character-manager.jsx` already exists — needs:
|
||||
1. Schema validation before export (reject malformed JSONs)
|
||||
2. File system integration: save/load from `characters/` directory
|
||||
3. Live preview of system prompt
|
||||
4. Expression mapping UI for Live2D states
|
||||
|
||||
### Tech Stack
|
||||
|
||||
- React + Vite (local dev server, not deployed)
|
||||
- Tailwind CSS (or minimal CSS)
|
||||
- Runs at `http://localhost:5173` during editing
|
||||
|
||||
### File Structure
|
||||
|
||||
```
|
||||
homeai-character/
|
||||
├── src/
|
||||
│ ├── character-manager.jsx ← existing, extend here
|
||||
│ ├── SchemaValidator.js ← validate against character.schema.json
|
||||
│ ├── ExpressionMapper.jsx ← UI for Live2D expression mapping
|
||||
│ └── main.jsx
|
||||
├── schema/
|
||||
│ └── character.schema.json
|
||||
├── characters/
|
||||
│ ├── aria.json ← default character
|
||||
│ └── .gitkeep
|
||||
├── package.json
|
||||
└── vite.config.js
|
||||
```
|
||||
|
||||
### Character Manager Features
|
||||
|
||||
| Feature | Description |
|
||||
|---|---|
|
||||
| Basic info | name, display name, description |
|
||||
| System prompt | Multi-line editor with char count |
|
||||
| Model overrides | Dropdown: primary + fast model |
|
||||
| TTS config | Engine picker, voice selector, speed slider, voice ref path |
|
||||
| Expression mapping | Table: state → VTube hotkey ID |
|
||||
| VTube WS triggers | JSON editor for advanced triggers |
|
||||
| Custom rules | Add/edit/delete trigger-response pairs |
|
||||
| Notes | Free-text notes field |
|
||||
| Export | Validates schema, writes to `characters/<name>.json` |
|
||||
| Import | Load existing character JSON for editing |
|
||||
|
||||
### Schema Validation
|
||||
|
||||
```javascript
|
||||
import Ajv from 'ajv'
|
||||
import schema from '../schema/character.schema.json'
|
||||
|
||||
const ajv = new Ajv()
|
||||
const validate = ajv.compile(schema)
|
||||
|
||||
export function validateCharacter(config) {
|
||||
const valid = validate(config)
|
||||
if (!valid) throw new Error(ajv.errorsText(validate.errors))
|
||||
return true
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Voice Clone Workflow
|
||||
|
||||
1. Record 30–60 seconds of clean speech at `~/voices/<name>-raw.wav`
|
||||
- Quiet room, consistent mic distance, natural conversational tone
|
||||
2. Pre-process: `ffmpeg -i raw.wav -ar 22050 -ac 1 aria.wav`
|
||||
3. Place at `~/voices/aria.wav`
|
||||
4. Update character JSON: `"voice_ref_path": "~/voices/aria.wav"`, `"engine": "chatterbox"`
|
||||
5. Test: run Chatterbox with the reference, verify voice quality
|
||||
6. If unsatisfactory, try Qwen3-TTS as alternative
|
||||
|
||||
---
|
||||
|
||||
## Pipeline Integration
|
||||
|
||||
### How P4 (OpenClaw) loads the character
|
||||
|
||||
```python
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
def load_character(name: str) -> dict:
|
||||
path = Path.home() / ".openclaw" / "characters" / f"{name}.json"
|
||||
config = json.loads(path.read_text())
|
||||
assert config["schema_version"] == 1, "Unsupported schema version"
|
||||
return config
|
||||
|
||||
# System prompt injection
|
||||
character = load_character("aria")
|
||||
system_prompt = character["system_prompt"]
|
||||
# Pass to Ollama as system message
|
||||
```
|
||||
|
||||
OpenClaw hot-reloads the character JSON on file change — no restart required.
|
||||
|
||||
### How P3 selects TTS engine
|
||||
|
||||
```python
|
||||
character = load_character(active_name)
|
||||
tts_cfg = character["tts"]
|
||||
|
||||
if tts_cfg["engine"] == "chatterbox":
|
||||
tts = ChatterboxTTS(voice_ref=tts_cfg["voice_ref_path"])
|
||||
elif tts_cfg["engine"] == "qwen3":
|
||||
tts = Qwen3TTS()
|
||||
else: # kokoro (default)
|
||||
tts = KokoroWyomingClient(voice=tts_cfg.get("kokoro_voice", "af_heart"))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
- [ ] Define and write `schema/character.schema.json` (v1)
|
||||
- [ ] Write `characters/aria.json` — default character with placeholder expression IDs
|
||||
- [ ] Set up Vite project in `src/` (install deps: `npm install`)
|
||||
- [ ] Integrate existing `character-manager.jsx` into new Vite project
|
||||
- [ ] Add schema validation on export (`ajv`)
|
||||
- [ ] Add expression mapping UI section
|
||||
- [ ] Add custom rules editor
|
||||
- [ ] Test full edit → export → validate → load cycle
|
||||
- [ ] Record or source voice reference audio for Aria
|
||||
- [ ] Pre-process audio and test with Chatterbox
|
||||
- [ ] Update `aria.json` with voice clone path if quality is good
|
||||
- [ ] Write `SchemaValidator.js` as standalone utility (used by P4 at runtime too)
|
||||
- [ ] Document schema in `schema/README.md`
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] `aria.json` validates against `character.schema.json` without errors
|
||||
- [ ] Character Manager UI can load, edit, and export `aria.json`
|
||||
- [ ] OpenClaw loads `aria.json` system prompt and applies it to Ollama requests
|
||||
- [ ] P3 TTS engine selection correctly follows `tts.engine` field
|
||||
- [ ] Schema version check in P4 fails gracefully with a clear error message
|
||||
- [ ] Voice clone sounds natural (if Chatterbox path taken)
|
||||
16
homeai-character/README.md
Normal file
16
homeai-character/README.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# React + Vite
|
||||
|
||||
This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
|
||||
|
||||
Currently, two official plugins are available:
|
||||
|
||||
- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Babel](https://babeljs.io/) (or [oxc](https://oxc.rs) when used in [rolldown-vite](https://vite.dev/guide/rolldown)) for Fast Refresh
|
||||
- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/) for Fast Refresh
|
||||
|
||||
## React Compiler
|
||||
|
||||
The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
|
||||
|
||||
## Expanding the ESLint configuration
|
||||
|
||||
If you are developing a production application, we recommend using TypeScript with type-aware lint rules enabled. Check out the [TS template](https://github.com/vitejs/vite/tree/main/packages/create-vite/template-react-ts) for information on how to integrate TypeScript and [`typescript-eslint`](https://typescript-eslint.io) in your project.
|
||||
@@ -1,686 +0,0 @@
|
||||
import { useState, useEffect, useCallback } from "react";
|
||||
|
||||
const STORAGE_KEY = "ai-character-profiles";
|
||||
|
||||
const DEFAULT_MODELS = [
|
||||
"llama3.3:70b", "qwen2.5:72b", "mistral-large", "llama3.1:8b",
|
||||
"qwen2.5:14b", "gemma3:27b", "deepseek-r1:14b", "phi4:14b"
|
||||
];
|
||||
|
||||
const TTS_MODELS = ["Kokoro", "Chatterbox", "F5-TTS", "Qwen3-TTS", "Piper"];
|
||||
const STT_MODELS = ["Whisper Large-v3", "Whisper Medium", "Whisper Small", "Whisper Turbo"];
|
||||
const IMAGE_MODELS = ["SDXL", "Flux.1-dev", "Flux.1-schnell", "SD 1.5", "Pony Diffusion"];
|
||||
|
||||
const PERSONALITY_TRAITS = [
|
||||
"Warm", "Witty", "Calm", "Energetic", "Sarcastic", "Nurturing",
|
||||
"Curious", "Playful", "Formal", "Casual", "Empathetic", "Direct",
|
||||
"Creative", "Analytical", "Protective", "Mischievous"
|
||||
];
|
||||
|
||||
const SPEAKING_STYLES = [
|
||||
"Conversational", "Poetic", "Concise", "Verbose", "Academic",
|
||||
"Informal", "Dramatic", "Deadpan", "Enthusiastic", "Measured"
|
||||
];
|
||||
|
||||
const EMPTY_CHARACTER = {
|
||||
id: null,
|
||||
name: "",
|
||||
tagline: "",
|
||||
avatar: "",
|
||||
accentColor: "#7c6fff",
|
||||
personality: {
|
||||
traits: [],
|
||||
speakingStyle: "",
|
||||
coreValues: "",
|
||||
quirks: "",
|
||||
backstory: "",
|
||||
motivation: "",
|
||||
},
|
||||
prompts: {
|
||||
systemPrompt: "",
|
||||
wakeWordResponse: "",
|
||||
fallbackResponse: "",
|
||||
errorResponse: "",
|
||||
customPrompts: [],
|
||||
},
|
||||
models: {
|
||||
llm: "",
|
||||
tts: "",
|
||||
stt: "",
|
||||
imageGen: "",
|
||||
voiceCloneRef: "",
|
||||
ttsSpeed: 1.0,
|
||||
temperature: 0.7,
|
||||
},
|
||||
liveRepresentation: {
|
||||
live2dModel: "",
|
||||
idleExpression: "",
|
||||
speakingExpression: "",
|
||||
thinkingExpression: "",
|
||||
happyExpression: "",
|
||||
vtsTriggers: "",
|
||||
},
|
||||
userNotes: "",
|
||||
createdAt: null,
|
||||
updatedAt: null,
|
||||
};
|
||||
|
||||
const TABS = ["Identity", "Personality", "Prompts", "Models", "Live2D", "Notes"];
|
||||
|
||||
const TAB_ICONS = {
|
||||
Identity: "◈",
|
||||
Personality: "◉",
|
||||
Prompts: "◎",
|
||||
Models: "⬡",
|
||||
Live2D: "◇",
|
||||
Notes: "▣",
|
||||
};
|
||||
|
||||
function generateId() {
|
||||
return Date.now().toString(36) + Math.random().toString(36).slice(2);
|
||||
}
|
||||
|
||||
function ColorPicker({ value, onChange }) {
|
||||
const presets = [
|
||||
"#7c6fff","#ff6b9d","#00d4aa","#ff9f43","#48dbfb",
|
||||
"#ff6348","#a29bfe","#fd79a8","#55efc4","#fdcb6e"
|
||||
];
|
||||
return (
|
||||
<div style={{ display: "flex", gap: 8, alignItems: "center", flexWrap: "wrap" }}>
|
||||
{presets.map(c => (
|
||||
<button key={c} onClick={() => onChange(c)} style={{
|
||||
width: 28, height: 28, borderRadius: "50%", background: c, border: value === c ? "3px solid #fff" : "3px solid transparent",
|
||||
cursor: "pointer", outline: "none", boxShadow: value === c ? `0 0 0 2px ${c}` : "none", transition: "all 0.2s"
|
||||
}} />
|
||||
))}
|
||||
<input type="color" value={value} onChange={e => onChange(e.target.value)}
|
||||
style={{ width: 28, height: 28, borderRadius: "50%", border: "none", cursor: "pointer", background: "none", padding: 0 }} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function TagSelector({ options, selected, onChange, max = 6 }) {
|
||||
return (
|
||||
<div style={{ display: "flex", flexWrap: "wrap", gap: 8 }}>
|
||||
{options.map(opt => {
|
||||
const active = selected.includes(opt);
|
||||
return (
|
||||
<button key={opt} onClick={() => {
|
||||
if (active) onChange(selected.filter(s => s !== opt));
|
||||
else if (selected.length < max) onChange([...selected, opt]);
|
||||
}} style={{
|
||||
padding: "5px 14px", borderRadius: 20, fontSize: 13, fontFamily: "inherit",
|
||||
background: active ? "var(--accent)" : "rgba(255,255,255,0.06)",
|
||||
color: active ? "#fff" : "rgba(255,255,255,0.55)",
|
||||
border: active ? "1px solid var(--accent)" : "1px solid rgba(255,255,255,0.1)",
|
||||
cursor: "pointer", transition: "all 0.18s", fontWeight: active ? 600 : 400,
|
||||
}}>
|
||||
{opt}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function Field({ label, hint, children }) {
|
||||
return (
|
||||
<div style={{ marginBottom: 22 }}>
|
||||
<label style={{ display: "block", fontSize: 12, fontWeight: 700, letterSpacing: "0.08em", textTransform: "uppercase", color: "rgba(255,255,255,0.45)", marginBottom: 6 }}>
|
||||
{label}
|
||||
</label>
|
||||
{hint && <p style={{ fontSize: 12, color: "rgba(255,255,255,0.3)", marginBottom: 8, marginTop: -2 }}>{hint}</p>}
|
||||
{children}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function Input({ value, onChange, placeholder, type = "text" }) {
|
||||
return (
|
||||
<input type={type} value={value} onChange={e => onChange(e.target.value)} placeholder={placeholder}
|
||||
style={{
|
||||
width: "100%", background: "rgba(255,255,255,0.05)", border: "1px solid rgba(255,255,255,0.1)",
|
||||
borderRadius: 8, padding: "10px 14px", color: "#fff", fontSize: 14, fontFamily: "inherit",
|
||||
outline: "none", boxSizing: "border-box", transition: "border-color 0.2s",
|
||||
}}
|
||||
onFocus={e => e.target.style.borderColor = "var(--accent)"}
|
||||
onBlur={e => e.target.style.borderColor = "rgba(255,255,255,0.1)"}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function Textarea({ value, onChange, placeholder, rows = 4 }) {
|
||||
return (
|
||||
<textarea value={value} onChange={e => onChange(e.target.value)} placeholder={placeholder} rows={rows}
|
||||
style={{
|
||||
width: "100%", background: "rgba(255,255,255,0.05)", border: "1px solid rgba(255,255,255,0.1)",
|
||||
borderRadius: 8, padding: "10px 14px", color: "#fff", fontSize: 14, fontFamily: "inherit",
|
||||
outline: "none", boxSizing: "border-box", resize: "vertical", lineHeight: 1.6,
|
||||
transition: "border-color 0.2s",
|
||||
}}
|
||||
onFocus={e => e.target.style.borderColor = "var(--accent)"}
|
||||
onBlur={e => e.target.style.borderColor = "rgba(255,255,255,0.1)"}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function Select({ value, onChange, options, placeholder }) {
|
||||
return (
|
||||
<select value={value} onChange={e => onChange(e.target.value)}
|
||||
style={{
|
||||
width: "100%", background: "rgba(20,20,35,0.95)", border: "1px solid rgba(255,255,255,0.1)",
|
||||
borderRadius: 8, padding: "10px 14px", color: value ? "#fff" : "rgba(255,255,255,0.35)",
|
||||
fontSize: 14, fontFamily: "inherit", outline: "none", cursor: "pointer",
|
||||
appearance: "none", backgroundImage: `url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='12' height='8' viewBox='0 0 12 8'%3E%3Cpath d='M1 1l5 5 5-5' stroke='rgba(255,255,255,0.3)' stroke-width='2' fill='none'/%3E%3C/svg%3E")`,
|
||||
backgroundRepeat: "no-repeat", backgroundPosition: "right 14px center",
|
||||
}}>
|
||||
<option value="">{placeholder || "Select..."}</option>
|
||||
{options.map(o => <option key={o} value={o}>{o}</option>)}
|
||||
</select>
|
||||
);
|
||||
}
|
||||
|
||||
function Slider({ value, onChange, min, max, step, label }) {
|
||||
return (
|
||||
<div style={{ display: "flex", alignItems: "center", gap: 14 }}>
|
||||
<input type="range" min={min} max={max} step={step} value={value}
|
||||
onChange={e => onChange(parseFloat(e.target.value))}
|
||||
style={{ flex: 1, accentColor: "var(--accent)", cursor: "pointer" }} />
|
||||
<span style={{ fontSize: 14, color: "rgba(255,255,255,0.7)", minWidth: 38, textAlign: "right", fontVariantNumeric: "tabular-nums" }}>
|
||||
{value.toFixed(1)}
|
||||
</span>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function CustomPromptsEditor({ prompts, onChange }) {
|
||||
const add = () => onChange([...prompts, { trigger: "", response: "" }]);
|
||||
const remove = i => onChange(prompts.filter((_, idx) => idx !== i));
|
||||
const update = (i, field, val) => {
|
||||
const next = [...prompts];
|
||||
next[i] = { ...next[i], [field]: val };
|
||||
onChange(next);
|
||||
};
|
||||
return (
|
||||
<div>
|
||||
{prompts.map((p, i) => (
|
||||
<div key={i} style={{ background: "rgba(255,255,255,0.04)", borderRadius: 10, padding: 14, marginBottom: 10, position: "relative" }}>
|
||||
<button onClick={() => remove(i)} style={{
|
||||
position: "absolute", top: 10, right: 10, background: "rgba(255,80,80,0.15)",
|
||||
border: "none", color: "#ff6b6b", borderRadius: 6, cursor: "pointer", padding: "2px 8px", fontSize: 12
|
||||
}}>✕</button>
|
||||
<div style={{ marginBottom: 8 }}>
|
||||
<Input value={p.trigger} onChange={v => update(i, "trigger", v)} placeholder="Trigger keyword or context..." />
|
||||
</div>
|
||||
<Textarea value={p.response} onChange={v => update(i, "response", v)} placeholder="Custom response or behaviour..." rows={2} />
|
||||
</div>
|
||||
))}
|
||||
<button onClick={add} style={{
|
||||
width: "100%", padding: "10px", background: "rgba(255,255,255,0.04)",
|
||||
border: "1px dashed rgba(255,255,255,0.15)", borderRadius: 8, color: "rgba(255,255,255,0.45)",
|
||||
cursor: "pointer", fontSize: 13, fontFamily: "inherit", transition: "all 0.2s"
|
||||
}}
|
||||
onMouseEnter={e => e.target.style.borderColor = "var(--accent)"}
|
||||
onMouseLeave={e => e.target.style.borderColor = "rgba(255,255,255,0.15)"}
|
||||
>+ Add Custom Prompt</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function CharacterCard({ character, active, onSelect, onDelete }) {
|
||||
const initials = character.name ? character.name.slice(0, 2).toUpperCase() : "??";
|
||||
return (
|
||||
<div onClick={() => onSelect(character.id)} style={{
|
||||
padding: "14px 16px", borderRadius: 12, cursor: "pointer", marginBottom: 8,
|
||||
background: active ? `linear-gradient(135deg, ${character.accentColor}22, ${character.accentColor}11)` : "rgba(255,255,255,0.04)",
|
||||
border: active ? `1px solid ${character.accentColor}66` : "1px solid rgba(255,255,255,0.07)",
|
||||
transition: "all 0.2s", position: "relative",
|
||||
}}>
|
||||
<div style={{ display: "flex", alignItems: "center", gap: 12 }}>
|
||||
<div style={{
|
||||
width: 40, height: 40, borderRadius: "50%", background: `linear-gradient(135deg, ${character.accentColor}, ${character.accentColor}88)`,
|
||||
display: "flex", alignItems: "center", justifyContent: "center", fontSize: 14, fontWeight: 800,
|
||||
color: "#fff", flexShrink: 0, boxShadow: `0 4px 12px ${character.accentColor}44`
|
||||
}}>{initials}</div>
|
||||
<div style={{ flex: 1, minWidth: 0 }}>
|
||||
<div style={{ fontWeight: 700, fontSize: 15, color: "#fff", whiteSpace: "nowrap", overflow: "hidden", textOverflow: "ellipsis" }}>
|
||||
{character.name || "Unnamed"}
|
||||
</div>
|
||||
{character.tagline && (
|
||||
<div style={{ fontSize: 12, color: "rgba(255,255,255,0.4)", whiteSpace: "nowrap", overflow: "hidden", textOverflow: "ellipsis" }}>
|
||||
{character.tagline}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
<button onClick={e => { e.stopPropagation(); onDelete(character.id); }} style={{
|
||||
background: "none", border: "none", color: "rgba(255,255,255,0.2)", cursor: "pointer",
|
||||
fontSize: 16, padding: "2px 6px", borderRadius: 4, transition: "color 0.15s", flexShrink: 0
|
||||
}}
|
||||
onMouseEnter={e => e.target.style.color = "#ff6b6b"}
|
||||
onMouseLeave={e => e.target.style.color = "rgba(255,255,255,0.2)"}
|
||||
>×</button>
|
||||
</div>
|
||||
{character.personality.traits.length > 0 && (
|
||||
<div style={{ display: "flex", gap: 4, flexWrap: "wrap", marginTop: 10 }}>
|
||||
{character.personality.traits.slice(0, 3).map(t => (
|
||||
<span key={t} style={{
|
||||
fontSize: 10, padding: "2px 8px", borderRadius: 10, fontWeight: 600, letterSpacing: "0.04em",
|
||||
background: `${character.accentColor}22`, color: character.accentColor, border: `1px solid ${character.accentColor}44`
|
||||
}}>{t}</span>
|
||||
))}
|
||||
{character.personality.traits.length > 3 && (
|
||||
<span style={{ fontSize: 10, color: "rgba(255,255,255,0.3)", padding: "2px 4px" }}>+{character.personality.traits.length - 3}</span>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function ExportModal({ character, onClose }) {
|
||||
const json = JSON.stringify(character, null, 2);
|
||||
const [copied, setCopied] = useState(false);
|
||||
const copy = () => {
|
||||
navigator.clipboard.writeText(json);
|
||||
setCopied(true);
|
||||
setTimeout(() => setCopied(false), 2000);
|
||||
};
|
||||
return (
|
||||
<div style={{
|
||||
position: "fixed", inset: 0, background: "rgba(0,0,0,0.7)", zIndex: 100,
|
||||
display: "flex", alignItems: "center", justifyContent: "center", padding: 24
|
||||
}} onClick={onClose}>
|
||||
<div onClick={e => e.stopPropagation()} style={{
|
||||
background: "#13131f", border: "1px solid rgba(255,255,255,0.1)", borderRadius: 16,
|
||||
padding: 28, width: "100%", maxWidth: 640, maxHeight: "80vh", display: "flex", flexDirection: "column"
|
||||
}}>
|
||||
<div style={{ display: "flex", justifyContent: "space-between", alignItems: "center", marginBottom: 16 }}>
|
||||
<h3 style={{ margin: 0, fontSize: 18, color: "#fff" }}>Export Character</h3>
|
||||
<button onClick={onClose} style={{ background: "none", border: "none", color: "rgba(255,255,255,0.4)", fontSize: 22, cursor: "pointer" }}>×</button>
|
||||
</div>
|
||||
<pre style={{
|
||||
flex: 1, overflow: "auto", background: "rgba(0,0,0,0.3)", borderRadius: 10,
|
||||
padding: 16, fontSize: 12, color: "rgba(255,255,255,0.7)", lineHeight: 1.6, margin: 0
|
||||
}}>{json}</pre>
|
||||
<button onClick={copy} style={{
|
||||
marginTop: 16, padding: "12px", background: "var(--accent)", border: "none",
|
||||
borderRadius: 10, color: "#fff", fontWeight: 700, fontSize: 14, cursor: "pointer",
|
||||
fontFamily: "inherit", transition: "opacity 0.2s"
|
||||
}}>{copied ? "✓ Copied!" : "Copy to Clipboard"}</button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export default function CharacterManager() {
|
||||
const [characters, setCharacters] = useState([]);
|
||||
const [activeId, setActiveId] = useState(null);
|
||||
const [activeTab, setActiveTab] = useState("Identity");
|
||||
const [exportModal, setExportModal] = useState(false);
|
||||
const [saved, setSaved] = useState(false);
|
||||
|
||||
// Load from storage
|
||||
useEffect(() => {
|
||||
try {
|
||||
const stored = localStorage.getItem(STORAGE_KEY);
|
||||
if (stored) {
|
||||
const parsed = JSON.parse(stored);
|
||||
setCharacters(parsed);
|
||||
if (parsed.length > 0) setActiveId(parsed[0].id);
|
||||
}
|
||||
} catch (e) {}
|
||||
}, []);
|
||||
|
||||
// Save to storage
|
||||
const saveToStorage = useCallback((chars) => {
|
||||
try {
|
||||
localStorage.setItem(STORAGE_KEY, JSON.stringify(chars));
|
||||
} catch (e) {}
|
||||
}, []);
|
||||
|
||||
const activeCharacter = characters.find(c => c.id === activeId) || null;
|
||||
|
||||
const updateCharacter = (updater) => {
|
||||
setCharacters(prev => {
|
||||
const next = prev.map(c => c.id === activeId ? { ...updater(c), updatedAt: new Date().toISOString() } : c);
|
||||
saveToStorage(next);
|
||||
return next;
|
||||
});
|
||||
setSaved(true);
|
||||
setTimeout(() => setSaved(false), 1500);
|
||||
};
|
||||
|
||||
const createCharacter = () => {
|
||||
const newChar = {
|
||||
...JSON.parse(JSON.stringify(EMPTY_CHARACTER)),
|
||||
id: generateId(),
|
||||
accentColor: ["#7c6fff","#ff6b9d","#00d4aa","#ff9f43","#48dbfb"][Math.floor(Math.random() * 5)],
|
||||
createdAt: new Date().toISOString(),
|
||||
updatedAt: new Date().toISOString(),
|
||||
};
|
||||
const next = [newChar, ...characters];
|
||||
setCharacters(next);
|
||||
setActiveId(newChar.id);
|
||||
setActiveTab("Identity");
|
||||
saveToStorage(next);
|
||||
};
|
||||
|
||||
const deleteCharacter = (id) => {
|
||||
const next = characters.filter(c => c.id !== id);
|
||||
setCharacters(next);
|
||||
saveToStorage(next);
|
||||
if (activeId === id) setActiveId(next.length > 0 ? next[0].id : null);
|
||||
};
|
||||
|
||||
const accentColor = activeCharacter?.accentColor || "#7c6fff";
|
||||
|
||||
const set = (path, value) => {
|
||||
updateCharacter(c => {
|
||||
const parts = path.split(".");
|
||||
const next = JSON.parse(JSON.stringify(c));
|
||||
let obj = next;
|
||||
for (let i = 0; i < parts.length - 1; i++) obj = obj[parts[i]];
|
||||
obj[parts[parts.length - 1]] = value;
|
||||
return next;
|
||||
});
|
||||
};
|
||||
|
||||
const renderTab = () => {
|
||||
if (!activeCharacter) return null;
|
||||
const c = activeCharacter;
|
||||
|
||||
switch (activeTab) {
|
||||
case "Identity":
|
||||
return (
|
||||
<div>
|
||||
<Field label="Character Name">
|
||||
<Input value={c.name} onChange={v => set("name", v)} placeholder="e.g. Aria, Nova, Echo..." />
|
||||
</Field>
|
||||
<Field label="Tagline" hint="A short phrase that captures their essence">
|
||||
<Input value={c.tagline} onChange={v => set("tagline", v)} placeholder="e.g. Your curious, warm-hearted companion" />
|
||||
</Field>
|
||||
<Field label="Accent Color" hint="Used for UI theming and visual identity">
|
||||
<ColorPicker value={c.accentColor} onChange={v => set("accentColor", v)} />
|
||||
</Field>
|
||||
<Field label="Live2D / Avatar Reference" hint="Filename or URL of the character's visual model">
|
||||
<Input value={c.avatar} onChange={v => set("avatar", v)} placeholder="e.g. aria_v2.model3.json" />
|
||||
</Field>
|
||||
<Field label="Backstory" hint="Who are they? Where do they come from? Keep it rich.">
|
||||
<Textarea value={c.personality.backstory} onChange={v => set("personality.backstory", v)}
|
||||
placeholder="Write a detailed origin story, background, and personal history for this character..." rows={5} />
|
||||
</Field>
|
||||
<Field label="Core Motivation" hint="What drives them? What do they care about most?">
|
||||
<Textarea value={c.personality.motivation} onChange={v => set("personality.motivation", v)}
|
||||
placeholder="e.g. A deep desire to help and grow alongside their human companion..." rows={3} />
|
||||
</Field>
|
||||
</div>
|
||||
);
|
||||
|
||||
case "Personality":
|
||||
return (
|
||||
<div>
|
||||
<Field label="Personality Traits" hint={`Select up to 6 traits (${c.personality.traits.length}/6)`}>
|
||||
<TagSelector options={PERSONALITY_TRAITS} selected={c.personality.traits}
|
||||
onChange={v => set("personality.traits", v)} max={6} />
|
||||
</Field>
|
||||
<Field label="Speaking Style">
|
||||
<TagSelector options={SPEAKING_STYLES} selected={c.personality.speakingStyle ? [c.personality.speakingStyle] : []}
|
||||
onChange={v => set("personality.speakingStyle", v[v.length - 1] || "")} max={1} />
|
||||
</Field>
|
||||
<Field label="Core Values" hint="What principles guide their responses and behaviour?">
|
||||
<Textarea value={c.personality.coreValues} onChange={v => set("personality.coreValues", v)}
|
||||
placeholder="e.g. Honesty, kindness, intellectual curiosity, loyalty to their user..." rows={3} />
|
||||
</Field>
|
||||
<Field label="Quirks & Mannerisms" hint="Unique behavioural patterns, phrases, habits that make them feel real">
|
||||
<Textarea value={c.personality.quirks} onChange={v => set("personality.quirks", v)}
|
||||
placeholder="e.g. Tends to use nautical metaphors. Hums softly when thinking. Has strong opinions about tea..." rows={3} />
|
||||
</Field>
|
||||
</div>
|
||||
);
|
||||
|
||||
case "Prompts":
|
||||
return (
|
||||
<div>
|
||||
<Field label="System Prompt" hint="The core instruction set defining who this character is to the LLM">
|
||||
<Textarea value={c.prompts.systemPrompt} onChange={v => set("prompts.systemPrompt", v)}
|
||||
placeholder="You are [name], a [description]. Your personality is [traits]. You speak in a [style] manner. You care deeply about [values]..." rows={8} />
|
||||
</Field>
|
||||
<Field label="Wake Word Response" hint="First response when activated by wake word">
|
||||
<Textarea value={c.prompts.wakeWordResponse} onChange={v => set("prompts.wakeWordResponse", v)}
|
||||
placeholder="e.g. 'Yes? I'm here.' or 'Hmm? What do you need?'" rows={2} />
|
||||
</Field>
|
||||
<Field label="Fallback Response" hint="When the character doesn't understand or can't help">
|
||||
<Textarea value={c.prompts.fallbackResponse} onChange={v => set("prompts.fallbackResponse", v)}
|
||||
placeholder="e.g. 'I'm not sure I follow — could you say that differently?'" rows={2} />
|
||||
</Field>
|
||||
<Field label="Error Response" hint="When something goes wrong technically">
|
||||
<Textarea value={c.prompts.errorResponse} onChange={v => set("prompts.errorResponse", v)}
|
||||
placeholder="e.g. 'Something went wrong on my end. Give me a moment.'" rows={2} />
|
||||
</Field>
|
||||
<Field label="Custom Prompt Rules" hint="Context-specific overrides and triggers">
|
||||
<CustomPromptsEditor prompts={c.prompts.customPrompts}
|
||||
onChange={v => set("prompts.customPrompts", v)} />
|
||||
</Field>
|
||||
</div>
|
||||
);
|
||||
|
||||
case "Models":
|
||||
return (
|
||||
<div>
|
||||
<Field label="LLM (Language Model)" hint="Primary reasoning and conversation model via Ollama">
|
||||
<Select value={c.models.llm} onChange={v => set("models.llm", v)} options={DEFAULT_MODELS} placeholder="Select LLM..." />
|
||||
</Field>
|
||||
<Field label="LLM Temperature" hint="Higher = more creative, lower = more focused">
|
||||
<Slider value={c.models.temperature} onChange={v => set("models.temperature", v)} min={0} max={2} step={0.1} />
|
||||
</Field>
|
||||
<Field label="Text-to-Speech Engine">
|
||||
<Select value={c.models.tts} onChange={v => set("models.tts", v)} options={TTS_MODELS} placeholder="Select TTS..." />
|
||||
</Field>
|
||||
<Field label="TTS Speed">
|
||||
<Slider value={c.models.ttsSpeed} onChange={v => set("models.ttsSpeed", v)} min={0.5} max={2.0} step={0.1} />
|
||||
</Field>
|
||||
<Field label="Voice Clone Reference" hint="Path or filename of reference audio for voice cloning">
|
||||
<Input value={c.models.voiceCloneRef} onChange={v => set("models.voiceCloneRef", v)} placeholder="e.g. /voices/aria_reference.wav" />
|
||||
</Field>
|
||||
<Field label="Speech-to-Text Engine">
|
||||
<Select value={c.models.stt} onChange={v => set("models.stt", v)} options={STT_MODELS} placeholder="Select STT..." />
|
||||
</Field>
|
||||
<Field label="Image Generation Model" hint="Used when character generates images or self-portraits">
|
||||
<Select value={c.models.imageGen} onChange={v => set("models.imageGen", v)} options={IMAGE_MODELS} placeholder="Select image model..." />
|
||||
</Field>
|
||||
</div>
|
||||
);
|
||||
|
||||
case "Live2D":
|
||||
return (
|
||||
<div>
|
||||
<Field label="Live2D Model File" hint="Path to .model3.json file, relative to VTube Studio models folder">
|
||||
<Input value={c.liveRepresentation.live2dModel} onChange={v => set("liveRepresentation.live2dModel", v)} placeholder="e.g. Aria/aria.model3.json" />
|
||||
</Field>
|
||||
<Field label="Idle Expression" hint="VTube Studio expression name when listening/waiting">
|
||||
<Input value={c.liveRepresentation.idleExpression} onChange={v => set("liveRepresentation.idleExpression", v)} placeholder="e.g. idle_blink" />
|
||||
</Field>
|
||||
<Field label="Speaking Expression" hint="Expression triggered when TTS audio is playing">
|
||||
<Input value={c.liveRepresentation.speakingExpression} onChange={v => set("liveRepresentation.speakingExpression", v)} placeholder="e.g. talking_smile" />
|
||||
</Field>
|
||||
<Field label="Thinking Expression" hint="Triggered while LLM is processing a response">
|
||||
<Input value={c.liveRepresentation.thinkingExpression} onChange={v => set("liveRepresentation.thinkingExpression", v)} placeholder="e.g. thinking_tilt" />
|
||||
</Field>
|
||||
<Field label="Happy / Positive Expression" hint="Triggered on positive sentiment responses">
|
||||
<Input value={c.liveRepresentation.happyExpression} onChange={v => set("liveRepresentation.happyExpression", v)} placeholder="e.g. happy_bright" />
|
||||
</Field>
|
||||
<Field label="VTube Studio Custom Triggers" hint="Additional WebSocket API trigger mappings (JSON)">
|
||||
<Textarea value={c.liveRepresentation.vtsTriggers} onChange={v => set("liveRepresentation.vtsTriggers", v)}
|
||||
placeholder={'{\n "on_error": "expression_concerned",\n "on_wake": "expression_alert"\n}'} rows={5} />
|
||||
</Field>
|
||||
</div>
|
||||
);
|
||||
|
||||
case "Notes":
|
||||
return (
|
||||
<div>
|
||||
<Field label="Developer Notes" hint="Freeform notes, ideas, todos, and observations about this character">
|
||||
<Textarea value={c.userNotes} onChange={v => set("userNotes", v)}
|
||||
placeholder={"Ideas, observations, things to try...\n\n- Voice reference sounds slightly too formal, adjust Chatterbox guidance scale\n- Try adding more nautical metaphors to system prompt\n- Need to map 'confused' expression in VTS\n- Consider adding weather awareness skill"}
|
||||
rows={16} />
|
||||
</Field>
|
||||
<div style={{ background: "rgba(255,255,255,0.03)", borderRadius: 10, padding: 16, fontSize: 12, color: "rgba(255,255,255,0.35)", lineHeight: 1.7 }}>
|
||||
<div style={{ marginBottom: 4, fontWeight: 700, color: "rgba(255,255,255,0.45)", letterSpacing: "0.06em", textTransform: "uppercase", fontSize: 11 }}>Character Info</div>
|
||||
<div>ID: <span style={{ color: "rgba(255,255,255,0.5)", fontFamily: "monospace" }}>{c.id}</span></div>
|
||||
{c.createdAt && <div>Created: {new Date(c.createdAt).toLocaleString()}</div>}
|
||||
{c.updatedAt && <div>Updated: {new Date(c.updatedAt).toLocaleString()}</div>}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div style={{
|
||||
"--accent": accentColor,
|
||||
minHeight: "100vh",
|
||||
background: "#0d0d18",
|
||||
color: "#fff",
|
||||
fontFamily: "'DM Sans', 'Segoe UI', system-ui, sans-serif",
|
||||
display: "flex",
|
||||
flexDirection: "column",
|
||||
}}>
|
||||
<style>{`
|
||||
@import url('https://fonts.googleapis.com/css2?family=DM+Sans:wght@400;500;600;700;800&family=DM+Mono:wght@400;500&display=swap');
|
||||
* { box-sizing: border-box; }
|
||||
::-webkit-scrollbar { width: 6px; }
|
||||
::-webkit-scrollbar-track { background: transparent; }
|
||||
::-webkit-scrollbar-thumb { background: rgba(255,255,255,0.1); border-radius: 3px; }
|
||||
input::placeholder, textarea::placeholder { color: rgba(255,255,255,0.2); }
|
||||
select option { background: #13131f; }
|
||||
`}</style>
|
||||
|
||||
{/* Header */}
|
||||
<div style={{
|
||||
padding: "18px 28px", borderBottom: "1px solid rgba(255,255,255,0.06)",
|
||||
display: "flex", alignItems: "center", justifyContent: "space-between",
|
||||
background: "rgba(0,0,0,0.2)", backdropFilter: "blur(10px)",
|
||||
position: "sticky", top: 0, zIndex: 10,
|
||||
}}>
|
||||
<div style={{ display: "flex", alignItems: "center", gap: 14 }}>
|
||||
<div style={{
|
||||
width: 36, height: 36, borderRadius: 10,
|
||||
background: `linear-gradient(135deg, ${accentColor}, ${accentColor}88)`,
|
||||
display: "flex", alignItems: "center", justifyContent: "center", fontSize: 18,
|
||||
boxShadow: `0 4px 16px ${accentColor}44`
|
||||
}}>◈</div>
|
||||
<div>
|
||||
<div style={{ fontWeight: 800, fontSize: 17, letterSpacing: "-0.01em" }}>Character Manager</div>
|
||||
<div style={{ fontSize: 12, color: "rgba(255,255,255,0.35)" }}>AI Personality Configuration</div>
|
||||
</div>
|
||||
</div>
|
||||
<div style={{ display: "flex", gap: 10, alignItems: "center" }}>
|
||||
{saved && <span style={{ fontSize: 12, color: accentColor, fontWeight: 600 }}>✓ Saved</span>}
|
||||
{activeCharacter && (
|
||||
<button onClick={() => setExportModal(true)} style={{
|
||||
padding: "8px 16px", background: "rgba(255,255,255,0.07)", border: "1px solid rgba(255,255,255,0.12)",
|
||||
borderRadius: 8, color: "rgba(255,255,255,0.7)", fontSize: 13, cursor: "pointer",
|
||||
fontFamily: "inherit", fontWeight: 600, transition: "all 0.2s"
|
||||
}}>Export JSON</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div style={{ display: "flex", flex: 1, overflow: "hidden" }}>
|
||||
{/* Sidebar */}
|
||||
<div style={{
|
||||
width: 260, borderRight: "1px solid rgba(255,255,255,0.06)",
|
||||
display: "flex", flexDirection: "column", background: "rgba(0,0,0,0.15)",
|
||||
flexShrink: 0,
|
||||
}}>
|
||||
<div style={{ padding: "16px 16px 8px" }}>
|
||||
<button onClick={createCharacter} style={{
|
||||
width: "100%", padding: "11px", background: `linear-gradient(135deg, ${accentColor}cc, ${accentColor}88)`,
|
||||
border: "none", borderRadius: 10, color: "#fff", fontWeight: 700, fontSize: 14,
|
||||
cursor: "pointer", fontFamily: "inherit", transition: "opacity 0.2s",
|
||||
boxShadow: `0 4px 16px ${accentColor}33`
|
||||
}}>+ New Character</button>
|
||||
</div>
|
||||
<div style={{ flex: 1, overflowY: "auto", padding: "4px 16px 16px" }}>
|
||||
{characters.length === 0 ? (
|
||||
<div style={{ textAlign: "center", padding: "40px 16px", color: "rgba(255,255,255,0.2)", fontSize: 13, lineHeight: 1.6 }}>
|
||||
No characters yet.<br />Create your first one above.
|
||||
</div>
|
||||
) : (
|
||||
characters.map(c => (
|
||||
<CharacterCard key={c.id} character={c} active={c.id === activeId}
|
||||
onSelect={setActiveId} onDelete={deleteCharacter} />
|
||||
))
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Main editor */}
|
||||
{activeCharacter ? (
|
||||
<div style={{ flex: 1, display: "flex", flexDirection: "column", overflow: "hidden" }}>
|
||||
{/* Character header */}
|
||||
<div style={{
|
||||
padding: "20px 28px 0", borderBottom: "1px solid rgba(255,255,255,0.06)",
|
||||
background: `linear-gradient(180deg, ${accentColor}0a 0%, transparent 100%)`,
|
||||
}}>
|
||||
<div style={{ display: "flex", alignItems: "center", gap: 16, marginBottom: 18 }}>
|
||||
<div style={{
|
||||
width: 52, height: 52, borderRadius: 16, flexShrink: 0,
|
||||
background: `linear-gradient(135deg, ${accentColor}, ${accentColor}66)`,
|
||||
display: "flex", alignItems: "center", justifyContent: "center",
|
||||
fontSize: 20, fontWeight: 800, boxShadow: `0 6px 20px ${accentColor}44`
|
||||
}}>
|
||||
{activeCharacter.name ? activeCharacter.name.slice(0, 2).toUpperCase() : "??"}
|
||||
</div>
|
||||
<div>
|
||||
<div style={{ fontSize: 22, fontWeight: 800, letterSpacing: "-0.02em", lineHeight: 1.2 }}>
|
||||
{activeCharacter.name || <span style={{ color: "rgba(255,255,255,0.25)" }}>Unnamed Character</span>}
|
||||
</div>
|
||||
{activeCharacter.tagline && (
|
||||
<div style={{ fontSize: 14, color: "rgba(255,255,255,0.45)", marginTop: 2 }}>{activeCharacter.tagline}</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
{/* Tabs */}
|
||||
<div style={{ display: "flex", gap: 2 }}>
|
||||
{TABS.map(tab => (
|
||||
<button key={tab} onClick={() => setActiveTab(tab)} style={{
|
||||
padding: "9px 16px", background: "none", border: "none",
|
||||
borderBottom: activeTab === tab ? `2px solid ${accentColor}` : "2px solid transparent",
|
||||
color: activeTab === tab ? "#fff" : "rgba(255,255,255,0.4)",
|
||||
fontSize: 13, fontWeight: activeTab === tab ? 700 : 500,
|
||||
cursor: "pointer", fontFamily: "inherit", transition: "all 0.18s",
|
||||
display: "flex", alignItems: "center", gap: 6,
|
||||
}}>
|
||||
<span style={{ fontSize: 11 }}>{TAB_ICONS[tab]}</span>{tab}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Tab content */}
|
||||
<div style={{ flex: 1, overflowY: "auto", padding: "24px 28px" }}>
|
||||
{renderTab()}
|
||||
</div>
|
||||
</div>
|
||||
) : (
|
||||
<div style={{
|
||||
flex: 1, display: "flex", alignItems: "center", justifyContent: "center",
|
||||
flexDirection: "column", gap: 16, color: "rgba(255,255,255,0.2)"
|
||||
}}>
|
||||
<div style={{ fontSize: 64, opacity: 0.3 }}>◈</div>
|
||||
<div style={{ fontSize: 16, fontWeight: 600 }}>No character selected</div>
|
||||
<div style={{ fontSize: 13 }}>Create a new character to get started</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{exportModal && activeCharacter && (
|
||||
<ExportModal character={activeCharacter} onClose={() => setExportModal(false)} />
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
29
homeai-character/eslint.config.js
Normal file
29
homeai-character/eslint.config.js
Normal file
@@ -0,0 +1,29 @@
|
||||
import js from '@eslint/js'
|
||||
import globals from 'globals'
|
||||
import reactHooks from 'eslint-plugin-react-hooks'
|
||||
import reactRefresh from 'eslint-plugin-react-refresh'
|
||||
import { defineConfig, globalIgnores } from 'eslint/config'
|
||||
|
||||
export default defineConfig([
|
||||
globalIgnores(['dist']),
|
||||
{
|
||||
files: ['**/*.{js,jsx}'],
|
||||
extends: [
|
||||
js.configs.recommended,
|
||||
reactHooks.configs.flat.recommended,
|
||||
reactRefresh.configs.vite,
|
||||
],
|
||||
languageOptions: {
|
||||
ecmaVersion: 2020,
|
||||
globals: globals.browser,
|
||||
parserOptions: {
|
||||
ecmaVersion: 'latest',
|
||||
ecmaFeatures: { jsx: true },
|
||||
sourceType: 'module',
|
||||
},
|
||||
},
|
||||
rules: {
|
||||
'no-unused-vars': ['error', { varsIgnorePattern: '^[A-Z_]' }],
|
||||
},
|
||||
},
|
||||
])
|
||||
13
homeai-character/index.html
Normal file
13
homeai-character/index.html
Normal file
@@ -0,0 +1,13 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>HomeAI Dashboard</title>
|
||||
</head>
|
||||
<body class="bg-gray-950 text-gray-100">
|
||||
<div id="root"></div>
|
||||
<script type="module" src="/src/main.jsx"></script>
|
||||
</body>
|
||||
</html>
|
||||
3397
homeai-character/package-lock.json
generated
Normal file
3397
homeai-character/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
34
homeai-character/package.json
Normal file
34
homeai-character/package.json
Normal file
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"name": "homeai-character",
|
||||
"private": true,
|
||||
"version": "0.0.0",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
"build": "vite build",
|
||||
"lint": "eslint .",
|
||||
"preview": "vite preview"
|
||||
},
|
||||
"dependencies": {
|
||||
"@tailwindcss/vite": "^4.2.1",
|
||||
"ajv": "^8.18.0",
|
||||
"react": "^19.2.0",
|
||||
"react-dom": "^19.2.0",
|
||||
"react-router-dom": "^7.13.1",
|
||||
"tailwindcss": "^4.2.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@eslint/js": "^9.39.1",
|
||||
"@types/react": "^19.2.7",
|
||||
"@types/react-dom": "^19.2.3",
|
||||
"@vitejs/plugin-react": "^5.1.1",
|
||||
"eslint": "^9.39.1",
|
||||
"eslint-plugin-react-hooks": "^7.0.1",
|
||||
"eslint-plugin-react-refresh": "^0.4.24",
|
||||
"globals": "^16.5.0",
|
||||
"vite": "^8.0.0-beta.13"
|
||||
},
|
||||
"overrides": {
|
||||
"vite": "^8.0.0-beta.13"
|
||||
}
|
||||
}
|
||||
1
homeai-character/public/vite.svg
Normal file
1
homeai-character/public/vite.svg
Normal file
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="31.88" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 257"><defs><linearGradient id="IconifyId1813088fe1fbc01fb466" x1="-.828%" x2="57.636%" y1="7.652%" y2="78.411%"><stop offset="0%" stop-color="#41D1FF"></stop><stop offset="100%" stop-color="#BD34FE"></stop></linearGradient><linearGradient id="IconifyId1813088fe1fbc01fb467" x1="43.376%" x2="50.316%" y1="2.242%" y2="89.03%"><stop offset="0%" stop-color="#FFEA83"></stop><stop offset="8.333%" stop-color="#FFDD35"></stop><stop offset="100%" stop-color="#FFA800"></stop></linearGradient></defs><path fill="url(#IconifyId1813088fe1fbc01fb466)" d="M255.153 37.938L134.897 252.976c-2.483 4.44-8.862 4.466-11.382.048L.875 37.958c-2.746-4.814 1.371-10.646 6.827-9.67l120.385 21.517a6.537 6.537 0 0 0 2.322-.004l117.867-21.483c5.438-.991 9.574 4.796 6.877 9.62Z"></path><path fill="url(#IconifyId1813088fe1fbc01fb467)" d="M185.432.063L96.44 17.501a3.268 3.268 0 0 0-2.634 3.014l-5.474 92.456a3.268 3.268 0 0 0 3.997 3.378l24.777-5.718c2.318-.535 4.413 1.507 3.936 3.838l-7.361 36.047c-.495 2.426 1.782 4.5 4.151 3.78l15.304-4.649c2.372-.72 4.652 1.36 4.15 3.788l-11.698 56.621c-.732 3.542 3.979 5.473 5.943 2.437l1.313-2.028l72.516-144.72c1.215-2.423-.88-5.186-3.54-4.672l-25.505 4.922c-2.396.462-4.435-1.77-3.759-4.114l16.646-57.705c.677-2.35-1.37-4.583-3.769-4.113Z"></path></svg>
|
||||
|
After Width: | Height: | Size: 1.5 KiB |
82
homeai-character/schema/character.schema.json
Normal file
82
homeai-character/schema/character.schema.json
Normal file
@@ -0,0 +1,82 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "HomeAI Character Config",
|
||||
"version": "1",
|
||||
"type": "object",
|
||||
"required": ["schema_version", "name", "system_prompt", "tts"],
|
||||
"properties": {
|
||||
"schema_version": { "type": "integer", "const": 1 },
|
||||
"name": { "type": "string" },
|
||||
"display_name": { "type": "string" },
|
||||
"description": { "type": "string" },
|
||||
|
||||
"system_prompt": { "type": "string" },
|
||||
|
||||
"model_overrides": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"primary": { "type": "string" },
|
||||
"fast": { "type": "string" }
|
||||
}
|
||||
},
|
||||
|
||||
"tts": {
|
||||
"type": "object",
|
||||
"required": ["engine"],
|
||||
"properties": {
|
||||
"engine": {
|
||||
"type": "string",
|
||||
"enum": ["kokoro", "chatterbox", "qwen3", "elevenlabs"]
|
||||
},
|
||||
"voice_ref_path": { "type": "string" },
|
||||
"kokoro_voice": { "type": "string" },
|
||||
"elevenlabs_voice_id": { "type": "string" },
|
||||
"elevenlabs_model": { "type": "string", "default": "eleven_monolingual_v1" },
|
||||
"speed": { "type": "number", "default": 1.0 }
|
||||
}
|
||||
},
|
||||
|
||||
"live2d_expressions": {
|
||||
"type": "object",
|
||||
"description": "Maps semantic state to VTube Studio hotkey ID",
|
||||
"properties": {
|
||||
"idle": { "type": "string" },
|
||||
"listening": { "type": "string" },
|
||||
"thinking": { "type": "string" },
|
||||
"speaking": { "type": "string" },
|
||||
"happy": { "type": "string" },
|
||||
"sad": { "type": "string" },
|
||||
"surprised": { "type": "string" },
|
||||
"error": { "type": "string" }
|
||||
}
|
||||
},
|
||||
|
||||
"vtube_ws_triggers": {
|
||||
"type": "object",
|
||||
"description": "VTube Studio WebSocket actions keyed by event name",
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"type": { "type": "string", "enum": ["hotkey", "parameter"] },
|
||||
"id": { "type": "string" },
|
||||
"value": { "type": "number" }
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
"custom_rules": {
|
||||
"type": "array",
|
||||
"description": "Trigger/response overrides for specific contexts",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"trigger": { "type": "string" },
|
||||
"response": { "type": "string" },
|
||||
"condition": { "type": "string" }
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
"notes": { "type": "string" }
|
||||
}
|
||||
}
|
||||
@@ -1,55 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# homeai-character/setup.sh — P5: Character Manager + persona JSON
|
||||
#
|
||||
# Components:
|
||||
# - character.schema.json — v1 character config schema
|
||||
# - aria.json — default character config
|
||||
# - Character Manager UI — Vite/React app for editing (dev server :5173)
|
||||
#
|
||||
# No hard runtime dependencies (can be developed standalone).
|
||||
# Output (aria.json) is consumed by P3, P4, P7.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
source "${REPO_DIR}/scripts/common.sh"
|
||||
|
||||
log_section "P5: Character Manager"
|
||||
detect_platform
|
||||
|
||||
# ─── Prerequisite check ────────────────────────────────────────────────────────
|
||||
log_info "Checking prerequisites..."
|
||||
|
||||
if ! command_exists node; then
|
||||
log_warn "Node.js not found — required for Character Manager UI"
|
||||
log_warn "Install: https://nodejs.org (v18+ recommended)"
|
||||
fi
|
||||
|
||||
# ─── TODO: Implementation ──────────────────────────────────────────────────────
|
||||
cat <<'EOF'
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ P5: homeai-character — NOT YET IMPLEMENTED │
|
||||
│ │
|
||||
│ Implementation steps: │
|
||||
│ 1. Create schema/character.schema.json (v1) │
|
||||
│ 2. Create characters/aria.json (default persona) │
|
||||
│ 3. Set up Vite/React project in src/ │
|
||||
│ 4. Extend character-manager.jsx with full UI │
|
||||
│ 5. Add schema validation (ajv) │
|
||||
│ 6. Add expression mapper UI for Live2D │
|
||||
│ 7. Wire export to ~/.openclaw/characters/ │
|
||||
│ │
|
||||
│ Dev server: │
|
||||
│ cd homeai-character && npm run dev → http://localhost:5173 │
|
||||
│ │
|
||||
│ Interface contracts: │
|
||||
│ Output: ~/.openclaw/characters/<name>.json │
|
||||
│ Schema: homeai-character/schema/character.schema.json │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
|
||||
EOF
|
||||
|
||||
log_info "P5 is not yet implemented. See homeai-character/PLAN.md for details."
|
||||
exit 0
|
||||
22
homeai-character/src/App.css
Normal file
22
homeai-character/src/App.css
Normal file
@@ -0,0 +1,22 @@
|
||||
/* Scrollbar styling for dark theme */
|
||||
::-webkit-scrollbar {
|
||||
width: 8px;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-track {
|
||||
background: #0a0a0f;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-thumb {
|
||||
background: #374151;
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-thumb:hover {
|
||||
background: #4b5563;
|
||||
}
|
||||
|
||||
/* Selection color */
|
||||
::selection {
|
||||
background: rgba(99, 102, 241, 0.3);
|
||||
}
|
||||
112
homeai-character/src/App.jsx
Normal file
112
homeai-character/src/App.jsx
Normal file
@@ -0,0 +1,112 @@
|
||||
import { BrowserRouter, Routes, Route, NavLink } from 'react-router-dom';
|
||||
import ServiceStatus from './ServiceStatus';
|
||||
import CharacterProfiles from './CharacterProfiles';
|
||||
import CharacterManager from './CharacterManager';
|
||||
|
||||
function NavItem({ to, children, icon }) {
|
||||
return (
|
||||
<NavLink
|
||||
to={to}
|
||||
className={({ isActive }) =>
|
||||
`flex items-center gap-3 px-4 py-2.5 rounded-lg text-sm font-medium transition-colors ${
|
||||
isActive
|
||||
? 'bg-gray-800 text-white'
|
||||
: 'text-gray-400 hover:text-gray-200 hover:bg-gray-800/50'
|
||||
}`
|
||||
}
|
||||
>
|
||||
{icon}
|
||||
<span>{children}</span>
|
||||
</NavLink>
|
||||
);
|
||||
}
|
||||
|
||||
function Layout({ children }) {
|
||||
return (
|
||||
<div className="min-h-screen bg-gray-950 flex">
|
||||
{/* Sidebar */}
|
||||
<aside className="w-64 bg-gray-900 border-r border-gray-800 flex flex-col fixed h-full">
|
||||
{/* Logo */}
|
||||
<div className="px-6 py-5 border-b border-gray-800">
|
||||
<div className="flex items-center gap-3">
|
||||
<div className="w-9 h-9 rounded-lg bg-gradient-to-br from-indigo-500 to-purple-600 flex items-center justify-center">
|
||||
<svg className="w-5 h-5 text-white" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M2.25 12l8.954-8.955c.44-.439 1.152-.439 1.591 0L21.75 12M4.5 9.75v10.125c0 .621.504 1.125 1.125 1.125H9.75v-4.875c0-.621.504-1.125 1.125-1.125h2.25c.621 0 1.125.504 1.125 1.125V21h4.125c.621 0 1.125-.504 1.125-1.125V9.75M8.25 21h8.25" />
|
||||
</svg>
|
||||
</div>
|
||||
<div>
|
||||
<h1 className="text-lg font-bold text-white tracking-tight">HomeAI</h1>
|
||||
<p className="text-xs text-gray-500">LINDBLUM</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Nav */}
|
||||
<nav className="flex-1 px-3 py-4 space-y-1">
|
||||
<NavItem
|
||||
to="/"
|
||||
icon={
|
||||
<svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M3.75 6A2.25 2.25 0 016 3.75h2.25A2.25 2.25 0 0110.5 6v2.25a2.25 2.25 0 01-2.25 2.25H6a2.25 2.25 0 01-2.25-2.25V6zM3.75 15.75A2.25 2.25 0 016 13.5h2.25a2.25 2.25 0 012.25 2.25V18a2.25 2.25 0 01-2.25 2.25H6A2.25 2.25 0 013.75 18v-2.25zM13.5 6a2.25 2.25 0 012.25-2.25H18A2.25 2.25 0 0120.25 6v2.25A2.25 2.25 0 0118 10.5h-2.25a2.25 2.25 0 01-2.25-2.25V6zM13.5 15.75a2.25 2.25 0 012.25-2.25H18a2.25 2.25 0 012.25 2.25V18A2.25 2.25 0 0118 20.25h-2.25A2.25 2.25 0 0113.5 18v-2.25z" />
|
||||
</svg>
|
||||
}
|
||||
>
|
||||
Dashboard
|
||||
</NavItem>
|
||||
|
||||
<NavItem
|
||||
to="/characters"
|
||||
icon={
|
||||
<svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M15.75 6a3.75 3.75 0 11-7.5 0 3.75 3.75 0 017.5 0zM4.501 20.118a7.5 7.5 0 0114.998 0A17.933 17.933 0 0112 21.75c-2.676 0-5.216-.584-7.499-1.632z" />
|
||||
</svg>
|
||||
}
|
||||
>
|
||||
Characters
|
||||
</NavItem>
|
||||
|
||||
<NavItem
|
||||
to="/editor"
|
||||
icon={
|
||||
<svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M9.594 3.94c.09-.542.56-.94 1.11-.94h2.593c.55 0 1.02.398 1.11.94l.213 1.281c.063.374.313.686.645.87.074.04.147.083.22.127.324.196.72.257 1.075.124l1.217-.456a1.125 1.125 0 011.37.49l1.296 2.247a1.125 1.125 0 01-.26 1.431l-1.003.827c-.293.24-.438.613-.431.992a6.759 6.759 0 010 .255c-.007.378.138.75.43.99l1.005.828c.424.35.534.954.26 1.43l-1.298 2.247a1.125 1.125 0 01-1.369.491l-1.217-.456c-.355-.133-.75-.072-1.076.124a6.57 6.57 0 01-.22.128c-.331.183-.581.495-.644.869l-.213 1.28c-.09.543-.56.941-1.11.941h-2.594c-.55 0-1.02-.398-1.11-.94l-.213-1.281c-.062-.374-.312-.686-.644-.87a6.52 6.52 0 01-.22-.127c-.325-.196-.72-.257-1.076-.124l-1.217.456a1.125 1.125 0 01-1.369-.49l-1.297-2.247a1.125 1.125 0 01.26-1.431l1.004-.827c.292-.24.437-.613.43-.992a6.932 6.932 0 010-.255c.007-.378-.138-.75-.43-.99l-1.004-.828a1.125 1.125 0 01-.26-1.43l1.297-2.247a1.125 1.125 0 011.37-.491l1.216.456c.356.133.751.072 1.076-.124.072-.044.146-.087.22-.128.332-.183.582-.495.644-.869l.214-1.281z" />
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M15 12a3 3 0 11-6 0 3 3 0 016 0z" />
|
||||
</svg>
|
||||
}
|
||||
>
|
||||
Editor
|
||||
</NavItem>
|
||||
</nav>
|
||||
|
||||
{/* Footer */}
|
||||
<div className="px-6 py-4 border-t border-gray-800">
|
||||
<p className="text-xs text-gray-600">HomeAI v0.1.0</p>
|
||||
<p className="text-xs text-gray-700">Mac Mini M4 Pro</p>
|
||||
</div>
|
||||
</aside>
|
||||
|
||||
{/* Main content */}
|
||||
<main className="flex-1 ml-64 p-8">
|
||||
<div className="max-w-6xl mx-auto">
|
||||
{children}
|
||||
</div>
|
||||
</main>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function App() {
|
||||
return (
|
||||
<BrowserRouter>
|
||||
<Layout>
|
||||
<Routes>
|
||||
<Route path="/" element={<ServiceStatus />} />
|
||||
<Route path="/characters" element={<CharacterProfiles />} />
|
||||
<Route path="/editor" element={<CharacterManager />} />
|
||||
</Routes>
|
||||
</Layout>
|
||||
</BrowserRouter>
|
||||
);
|
||||
}
|
||||
|
||||
export default App;
|
||||
585
homeai-character/src/CharacterManager.jsx
Normal file
585
homeai-character/src/CharacterManager.jsx
Normal file
@@ -0,0 +1,585 @@
|
||||
import React, { useState, useEffect, useRef } from 'react';
|
||||
import { validateCharacter } from './SchemaValidator';
|
||||
|
||||
const DEFAULT_CHARACTER = {
|
||||
schema_version: 1,
|
||||
name: "aria",
|
||||
display_name: "Aria",
|
||||
description: "Default HomeAI assistant persona",
|
||||
system_prompt: "You are Aria, a warm, curious, and helpful AI assistant living in the home. You speak naturally and conversationally — never robotic. You are knowledgeable but never condescending. You remember the people you live with and build on those memories over time. Keep responses concise when controlling smart home devices; be more expressive in casual conversation. Never break character.",
|
||||
model_overrides: {
|
||||
primary: "llama3.3:70b",
|
||||
fast: "qwen2.5:7b"
|
||||
},
|
||||
tts: {
|
||||
engine: "kokoro",
|
||||
kokoro_voice: "af_heart",
|
||||
speed: 1.0
|
||||
},
|
||||
live2d_expressions: {
|
||||
idle: "expr_idle",
|
||||
listening: "expr_listening",
|
||||
thinking: "expr_thinking",
|
||||
speaking: "expr_speaking",
|
||||
happy: "expr_happy",
|
||||
sad: "expr_sad",
|
||||
surprised: "expr_surprised",
|
||||
error: "expr_error"
|
||||
},
|
||||
vtube_ws_triggers: {
|
||||
thinking: { type: "hotkey", id: "expr_thinking" },
|
||||
speaking: { type: "hotkey", id: "expr_speaking" },
|
||||
idle: { type: "hotkey", id: "expr_idle" }
|
||||
},
|
||||
custom_rules: [
|
||||
{ trigger: "good morning", response: "Good morning! How did you sleep?", condition: "time_of_day == morning" }
|
||||
],
|
||||
notes: ""
|
||||
};
|
||||
|
||||
export default function CharacterManager() {
|
||||
const [character, setCharacter] = useState(() => {
|
||||
// Check if we're editing from profiles page
|
||||
const editData = sessionStorage.getItem('edit_character');
|
||||
if (editData) {
|
||||
sessionStorage.removeItem('edit_character');
|
||||
try {
|
||||
return JSON.parse(editData);
|
||||
} catch {
|
||||
return DEFAULT_CHARACTER;
|
||||
}
|
||||
}
|
||||
return DEFAULT_CHARACTER;
|
||||
});
|
||||
const [error, setError] = useState(null);
|
||||
const [saved, setSaved] = useState(false);
|
||||
|
||||
// TTS preview state
|
||||
const [ttsState, setTtsState] = useState('idle'); // idle | loading | playing
|
||||
const [previewText, setPreviewText] = useState('');
|
||||
const audioRef = useRef(null);
|
||||
const objectUrlRef = useRef(null);
|
||||
|
||||
// ElevenLabs state
|
||||
const [elevenLabsApiKey, setElevenLabsApiKey] = useState(localStorage.getItem('elevenlabs_api_key') || '');
|
||||
const [elevenLabsVoices, setElevenLabsVoices] = useState([]);
|
||||
const [elevenLabsModels, setElevenLabsModels] = useState([]);
|
||||
const [isLoadingElevenLabs, setIsLoadingElevenLabs] = useState(false);
|
||||
|
||||
const fetchElevenLabsData = async (key) => {
|
||||
if (!key) return;
|
||||
setIsLoadingElevenLabs(true);
|
||||
try {
|
||||
const headers = { 'xi-api-key': key };
|
||||
const [voicesRes, modelsRes] = await Promise.all([
|
||||
fetch('https://api.elevenlabs.io/v1/voices', { headers }),
|
||||
fetch('https://api.elevenlabs.io/v1/models', { headers })
|
||||
]);
|
||||
if (!voicesRes.ok || !modelsRes.ok) {
|
||||
throw new Error('Failed to fetch from ElevenLabs API (check API key)');
|
||||
}
|
||||
const voicesData = await voicesRes.json();
|
||||
const modelsData = await modelsRes.json();
|
||||
setElevenLabsVoices(voicesData.voices || []);
|
||||
setElevenLabsModels(modelsData.filter(m => m.can_do_text_to_speech) || []);
|
||||
localStorage.setItem('elevenlabs_api_key', key);
|
||||
} catch (err) {
|
||||
setError(err.message);
|
||||
} finally {
|
||||
setIsLoadingElevenLabs(false);
|
||||
}
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
if (elevenLabsApiKey && character.tts.engine === 'elevenlabs') {
|
||||
fetchElevenLabsData(elevenLabsApiKey);
|
||||
}
|
||||
}, [character.tts.engine]);
|
||||
|
||||
// Cleanup audio on unmount
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
if (audioRef.current) { audioRef.current.pause(); audioRef.current = null; }
|
||||
if (objectUrlRef.current) { URL.revokeObjectURL(objectUrlRef.current); }
|
||||
window.speechSynthesis.cancel();
|
||||
};
|
||||
}, []);
|
||||
|
||||
const handleExport = () => {
|
||||
try {
|
||||
validateCharacter(character);
|
||||
setError(null);
|
||||
const dataStr = "data:text/json;charset=utf-8," + encodeURIComponent(JSON.stringify(character, null, 2));
|
||||
const a = document.createElement('a');
|
||||
a.href = dataStr;
|
||||
a.download = `${character.name || 'character'}.json`;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
a.remove();
|
||||
} catch (err) {
|
||||
setError(err.message);
|
||||
}
|
||||
};
|
||||
|
||||
const handleSaveToProfiles = () => {
|
||||
try {
|
||||
validateCharacter(character);
|
||||
setError(null);
|
||||
|
||||
const profileId = sessionStorage.getItem('edit_character_profile_id');
|
||||
const storageKey = 'homeai_characters';
|
||||
const raw = localStorage.getItem(storageKey);
|
||||
let profiles = raw ? JSON.parse(raw) : [];
|
||||
|
||||
if (profileId) {
|
||||
profiles = profiles.map(p =>
|
||||
p.id === profileId ? { ...p, data: character } : p
|
||||
);
|
||||
sessionStorage.removeItem('edit_character_profile_id');
|
||||
} else {
|
||||
const id = character.name + '_' + Date.now();
|
||||
profiles.push({ id, data: character, image: null, addedAt: new Date().toISOString() });
|
||||
}
|
||||
|
||||
localStorage.setItem(storageKey, JSON.stringify(profiles));
|
||||
setSaved(true);
|
||||
setTimeout(() => setSaved(false), 2000);
|
||||
} catch (err) {
|
||||
setError(err.message);
|
||||
}
|
||||
};
|
||||
|
||||
const handleImport = (e) => {
|
||||
const file = e.target.files[0];
|
||||
if (!file) return;
|
||||
const reader = new FileReader();
|
||||
reader.onload = (e) => {
|
||||
try {
|
||||
const importedChar = JSON.parse(e.target.result);
|
||||
validateCharacter(importedChar);
|
||||
setCharacter(importedChar);
|
||||
setError(null);
|
||||
} catch (err) {
|
||||
setError(`Import failed: ${err.message}`);
|
||||
}
|
||||
};
|
||||
reader.readAsText(file);
|
||||
};
|
||||
|
||||
const handleChange = (field, value) => {
|
||||
setCharacter(prev => ({ ...prev, [field]: value }));
|
||||
};
|
||||
|
||||
const handleNestedChange = (parent, field, value) => {
|
||||
setCharacter(prev => ({
|
||||
...prev,
|
||||
[parent]: { ...prev[parent], [field]: value }
|
||||
}));
|
||||
};
|
||||
|
||||
const handleRuleChange = (index, field, value) => {
|
||||
setCharacter(prev => {
|
||||
const newRules = [...(prev.custom_rules || [])];
|
||||
newRules[index] = { ...newRules[index], [field]: value };
|
||||
return { ...prev, custom_rules: newRules };
|
||||
});
|
||||
};
|
||||
|
||||
const addRule = () => {
|
||||
setCharacter(prev => ({
|
||||
...prev,
|
||||
custom_rules: [...(prev.custom_rules || []), { trigger: "", response: "", condition: "" }]
|
||||
}));
|
||||
};
|
||||
|
||||
const removeRule = (index) => {
|
||||
setCharacter(prev => {
|
||||
const newRules = [...(prev.custom_rules || [])];
|
||||
newRules.splice(index, 1);
|
||||
return { ...prev, custom_rules: newRules };
|
||||
});
|
||||
};
|
||||
|
||||
const stopPreview = () => {
|
||||
if (audioRef.current) {
|
||||
audioRef.current.pause();
|
||||
audioRef.current = null;
|
||||
}
|
||||
if (objectUrlRef.current) {
|
||||
URL.revokeObjectURL(objectUrlRef.current);
|
||||
objectUrlRef.current = null;
|
||||
}
|
||||
window.speechSynthesis.cancel();
|
||||
setTtsState('idle');
|
||||
};
|
||||
|
||||
const previewTTS = async () => {
|
||||
stopPreview();
|
||||
const text = previewText || `Hi, I am ${character.display_name}. This is a preview of my voice.`;
|
||||
|
||||
if (character.tts.engine === 'kokoro') {
|
||||
setTtsState('loading');
|
||||
let blob;
|
||||
try {
|
||||
const response = await fetch('/api/tts', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ text, voice: character.tts.kokoro_voice })
|
||||
});
|
||||
if (!response.ok) throw new Error('TTS bridge returned ' + response.status);
|
||||
blob = await response.blob();
|
||||
} catch (err) {
|
||||
setTtsState('idle');
|
||||
setError(`Kokoro preview failed: ${err.message}. Falling back to browser TTS.`);
|
||||
runBrowserTTS(text);
|
||||
return;
|
||||
}
|
||||
const url = URL.createObjectURL(blob);
|
||||
objectUrlRef.current = url;
|
||||
const audio = new Audio(url);
|
||||
audio.playbackRate = character.tts.speed;
|
||||
audio.onended = () => { stopPreview(); };
|
||||
audio.onerror = () => { stopPreview(); };
|
||||
audioRef.current = audio;
|
||||
setTtsState('playing');
|
||||
audio.play().catch(() => { /* interrupted — stopPreview already handles cleanup */ });
|
||||
} else {
|
||||
runBrowserTTS(text);
|
||||
}
|
||||
};
|
||||
|
||||
const runBrowserTTS = (text) => {
|
||||
const utterance = new SpeechSynthesisUtterance(text);
|
||||
utterance.rate = character.tts.speed;
|
||||
const voices = window.speechSynthesis.getVoices();
|
||||
const preferredVoice = voices.find(v => v.lang.startsWith('en') && v.name.includes('Female')) || voices.find(v => v.lang.startsWith('en'));
|
||||
if (preferredVoice) utterance.voice = preferredVoice;
|
||||
setTtsState('playing');
|
||||
utterance.onend = () => setTtsState('idle');
|
||||
window.speechSynthesis.cancel();
|
||||
window.speechSynthesis.speak(utterance);
|
||||
};
|
||||
|
||||
const inputClass = "w-full bg-gray-800 border border-gray-700 text-gray-200 p-2 rounded-lg focus:border-indigo-500 focus:ring-1 focus:ring-indigo-500 outline-none transition-colors";
|
||||
const selectClass = "w-full bg-gray-800 border border-gray-700 text-gray-200 p-2 rounded-lg focus:border-indigo-500 focus:ring-1 focus:ring-indigo-500 outline-none transition-colors";
|
||||
const labelClass = "block text-sm font-medium text-gray-400 mb-1";
|
||||
const cardClass = "bg-gray-900 border border-gray-800 p-5 rounded-xl space-y-4";
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
<div className="flex justify-between items-center">
|
||||
<div>
|
||||
<h1 className="text-3xl font-bold text-gray-100">Character Editor</h1>
|
||||
<p className="text-sm text-gray-500 mt-1">
|
||||
Editing: {character.display_name || character.name}
|
||||
</p>
|
||||
</div>
|
||||
<div className="flex gap-3">
|
||||
<label className="cursor-pointer flex items-center gap-2 px-4 py-2 bg-gray-800 hover:bg-gray-700 text-gray-300 rounded-lg border border-gray-700 transition-colors">
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M3 16.5v2.25A2.25 2.25 0 005.25 21h13.5A2.25 2.25 0 0021 18.75V16.5m-13.5-9L12 3m0 0l4.5 4.5M12 3v13.5" />
|
||||
</svg>
|
||||
Import
|
||||
<input type="file" accept=".json" className="hidden" onChange={handleImport} />
|
||||
</label>
|
||||
<button
|
||||
onClick={handleSaveToProfiles}
|
||||
className={`flex items-center gap-2 px-4 py-2 rounded-lg transition-colors ${
|
||||
saved
|
||||
? 'bg-emerald-600 text-white'
|
||||
: 'bg-indigo-600 hover:bg-indigo-500 text-white'
|
||||
}`}
|
||||
>
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
{saved
|
||||
? <path strokeLinecap="round" strokeLinejoin="round" d="M4.5 12.75l6 6 9-13.5" />
|
||||
: <path strokeLinecap="round" strokeLinejoin="round" d="M17.593 3.322c1.1.128 1.907 1.077 1.907 2.185V21L12 17.25 4.5 21V5.507c0-1.108.806-2.057 1.907-2.185a48.507 48.507 0 0111.186 0z" />
|
||||
}
|
||||
</svg>
|
||||
{saved ? 'Saved' : 'Save to Profiles'}
|
||||
</button>
|
||||
<button
|
||||
onClick={handleExport}
|
||||
className="flex items-center gap-2 px-4 py-2 bg-gray-800 hover:bg-gray-700 text-gray-300 rounded-lg border border-gray-700 transition-colors"
|
||||
>
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M3 16.5v2.25A2.25 2.25 0 005.25 21h13.5A2.25 2.25 0 0021 18.75V16.5M16.5 12L12 16.5m0 0L7.5 12m4.5 4.5V3" />
|
||||
</svg>
|
||||
Export JSON
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{error && (
|
||||
<div className="bg-red-900/30 border border-red-500/50 text-red-300 px-4 py-3 rounded-lg text-sm">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="grid grid-cols-1 md:grid-cols-2 gap-6">
|
||||
{/* Basic Info */}
|
||||
<div className={cardClass}>
|
||||
<h2 className="text-lg font-semibold text-gray-200">Basic Info</h2>
|
||||
<div>
|
||||
<label className={labelClass}>Name (ID)</label>
|
||||
<input type="text" className={inputClass} value={character.name} onChange={(e) => handleChange('name', e.target.value)} />
|
||||
</div>
|
||||
<div>
|
||||
<label className={labelClass}>Display Name</label>
|
||||
<input type="text" className={inputClass} value={character.display_name} onChange={(e) => handleChange('display_name', e.target.value)} />
|
||||
</div>
|
||||
<div>
|
||||
<label className={labelClass}>Description</label>
|
||||
<input type="text" className={inputClass} value={character.description} onChange={(e) => handleChange('description', e.target.value)} />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* TTS Configuration */}
|
||||
<div className={cardClass}>
|
||||
<h2 className="text-lg font-semibold text-gray-200">TTS Configuration</h2>
|
||||
<div>
|
||||
<label className={labelClass}>Engine</label>
|
||||
<select className={selectClass} value={character.tts.engine} onChange={(e) => handleNestedChange('tts', 'engine', e.target.value)}>
|
||||
<option value="kokoro">Kokoro</option>
|
||||
<option value="chatterbox">Chatterbox</option>
|
||||
<option value="qwen3">Qwen3</option>
|
||||
<option value="elevenlabs">ElevenLabs</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
{character.tts.engine === 'elevenlabs' && (
|
||||
<div className="space-y-4 border border-gray-700 p-4 rounded-lg bg-gray-800/50">
|
||||
<div>
|
||||
<label className="block text-xs font-medium mb-1 text-gray-500">ElevenLabs API Key (Local Use Only)</label>
|
||||
<div className="flex gap-2">
|
||||
<input type="password" placeholder="sk_..." className={inputClass + " text-sm"} value={elevenLabsApiKey} onChange={(e) => setElevenLabsApiKey(e.target.value)} />
|
||||
<button onClick={() => fetchElevenLabsData(elevenLabsApiKey)} disabled={isLoadingElevenLabs} className="bg-indigo-600 text-white px-3 py-1 rounded-lg text-sm whitespace-nowrap hover:bg-indigo-500 disabled:opacity-50 transition-colors">
|
||||
{isLoadingElevenLabs ? 'Loading...' : 'Fetch'}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<label className={labelClass}>Voice ID</label>
|
||||
{elevenLabsVoices.length > 0 ? (
|
||||
<select className={selectClass} value={character.tts.elevenlabs_voice_id || ''} onChange={(e) => handleNestedChange('tts', 'elevenlabs_voice_id', e.target.value)}>
|
||||
<option value="">-- Select Voice --</option>
|
||||
{elevenLabsVoices.map(v => (
|
||||
<option key={v.voice_id} value={v.voice_id}>{v.name} ({v.category})</option>
|
||||
))}
|
||||
</select>
|
||||
) : (
|
||||
<input type="text" className={inputClass} value={character.tts.elevenlabs_voice_id || ''} onChange={(e) => handleNestedChange('tts', 'elevenlabs_voice_id', e.target.value)} placeholder="e.g. 21m00Tcm4TlvDq8ikWAM" />
|
||||
)}
|
||||
</div>
|
||||
<div>
|
||||
<label className={labelClass}>Model</label>
|
||||
{elevenLabsModels.length > 0 ? (
|
||||
<select className={selectClass} value={character.tts.elevenlabs_model || 'eleven_monolingual_v1'} onChange={(e) => handleNestedChange('tts', 'elevenlabs_model', e.target.value)}>
|
||||
<option value="">-- Select Model --</option>
|
||||
{elevenLabsModels.map(m => (
|
||||
<option key={m.model_id} value={m.model_id}>{m.name} ({m.model_id})</option>
|
||||
))}
|
||||
</select>
|
||||
) : (
|
||||
<input type="text" className={inputClass} value={character.tts.elevenlabs_model || 'eleven_monolingual_v1'} onChange={(e) => handleNestedChange('tts', 'elevenlabs_model', e.target.value)} placeholder="e.g. eleven_monolingual_v1" />
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{character.tts.engine === 'kokoro' && (
|
||||
<div>
|
||||
<label className={labelClass}>Kokoro Voice</label>
|
||||
<select className={selectClass} value={character.tts.kokoro_voice || 'af_heart'} onChange={(e) => handleNestedChange('tts', 'kokoro_voice', e.target.value)}>
|
||||
<option value="af_heart">af_heart (American Female)</option>
|
||||
<option value="af_alloy">af_alloy (American Female)</option>
|
||||
<option value="af_aoede">af_aoede (American Female)</option>
|
||||
<option value="af_bella">af_bella (American Female)</option>
|
||||
<option value="af_jessica">af_jessica (American Female)</option>
|
||||
<option value="af_kore">af_kore (American Female)</option>
|
||||
<option value="af_nicole">af_nicole (American Female)</option>
|
||||
<option value="af_nova">af_nova (American Female)</option>
|
||||
<option value="af_river">af_river (American Female)</option>
|
||||
<option value="af_sarah">af_sarah (American Female)</option>
|
||||
<option value="af_sky">af_sky (American Female)</option>
|
||||
<option value="am_adam">am_adam (American Male)</option>
|
||||
<option value="am_echo">am_echo (American Male)</option>
|
||||
<option value="am_eric">am_eric (American Male)</option>
|
||||
<option value="am_fenrir">am_fenrir (American Male)</option>
|
||||
<option value="am_liam">am_liam (American Male)</option>
|
||||
<option value="am_michael">am_michael (American Male)</option>
|
||||
<option value="am_onyx">am_onyx (American Male)</option>
|
||||
<option value="am_puck">am_puck (American Male)</option>
|
||||
<option value="am_santa">am_santa (American Male)</option>
|
||||
<option value="bf_alice">bf_alice (British Female)</option>
|
||||
<option value="bf_emma">bf_emma (British Female)</option>
|
||||
<option value="bf_isabella">bf_isabella (British Female)</option>
|
||||
<option value="bf_lily">bf_lily (British Female)</option>
|
||||
<option value="bm_daniel">bm_daniel (British Male)</option>
|
||||
<option value="bm_fable">bm_fable (British Male)</option>
|
||||
<option value="bm_george">bm_george (British Male)</option>
|
||||
<option value="bm_lewis">bm_lewis (British Male)</option>
|
||||
</select>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{character.tts.engine === 'chatterbox' && (
|
||||
<div>
|
||||
<label className={labelClass}>Voice Reference Path</label>
|
||||
<input type="text" className={inputClass} value={character.tts.voice_ref_path || ''} onChange={(e) => handleNestedChange('tts', 'voice_ref_path', e.target.value)} />
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div>
|
||||
<label className={labelClass}>Speed: {character.tts.speed}</label>
|
||||
<input type="range" min="0.5" max="2.0" step="0.1" className="w-full accent-indigo-500" value={character.tts.speed} onChange={(e) => handleNestedChange('tts', 'speed', parseFloat(e.target.value))} />
|
||||
</div>
|
||||
<div>
|
||||
<label className={labelClass}>Preview Text</label>
|
||||
<input
|
||||
type="text"
|
||||
className={inputClass}
|
||||
value={previewText}
|
||||
onChange={(e) => setPreviewText(e.target.value)}
|
||||
placeholder={`Hi, I am ${character.display_name}. This is a preview of my voice.`}
|
||||
/>
|
||||
</div>
|
||||
<div className="flex gap-2">
|
||||
<button
|
||||
onClick={previewTTS}
|
||||
disabled={ttsState === 'loading'}
|
||||
className={`flex-1 flex items-center justify-center gap-2 px-4 py-2 rounded-lg transition-colors ${
|
||||
ttsState === 'loading'
|
||||
? 'bg-indigo-800 text-indigo-300 cursor-wait'
|
||||
: ttsState === 'playing'
|
||||
? 'bg-emerald-600 hover:bg-emerald-500 text-white'
|
||||
: 'bg-indigo-600 hover:bg-indigo-500 text-white'
|
||||
}`}
|
||||
>
|
||||
{ttsState === 'loading' && (
|
||||
<svg className="w-4 h-4 animate-spin" viewBox="0 0 24 24" fill="none">
|
||||
<circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4" />
|
||||
<path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4z" />
|
||||
</svg>
|
||||
)}
|
||||
{ttsState === 'loading' ? 'Synthesizing...' : ttsState === 'playing' ? 'Playing...' : 'Preview Voice'}
|
||||
</button>
|
||||
{ttsState !== 'idle' && (
|
||||
<button
|
||||
onClick={stopPreview}
|
||||
className="px-4 py-2 bg-red-600 hover:bg-red-500 text-white rounded-lg transition-colors"
|
||||
>
|
||||
Stop
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
<p className="text-xs text-gray-600">
|
||||
{character.tts.engine === 'kokoro'
|
||||
? 'Previews via local Kokoro TTS bridge (port 8081 → Wyoming 10301).'
|
||||
: 'Uses browser TTS for preview. Local TTS available with Kokoro engine.'}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* System Prompt */}
|
||||
<div className={cardClass}>
|
||||
<div className="flex justify-between items-center">
|
||||
<h2 className="text-lg font-semibold text-gray-200">System Prompt</h2>
|
||||
<span className="text-xs text-gray-600">{character.system_prompt.length} chars</span>
|
||||
</div>
|
||||
<textarea
|
||||
className={inputClass + " h-32 resize-y"}
|
||||
value={character.system_prompt}
|
||||
onChange={(e) => handleChange('system_prompt', e.target.value)}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 md:grid-cols-2 gap-6">
|
||||
{/* Live2D Expressions */}
|
||||
<div className={cardClass}>
|
||||
<h2 className="text-lg font-semibold text-gray-200">Live2D Expressions</h2>
|
||||
{Object.entries(character.live2d_expressions).map(([key, val]) => (
|
||||
<div key={key} className="flex justify-between items-center gap-4">
|
||||
<label className="text-sm font-medium text-gray-400 w-1/3 capitalize">{key}</label>
|
||||
<input type="text" className={inputClass + " w-2/3"} value={val} onChange={(e) => handleNestedChange('live2d_expressions', key, e.target.value)} />
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
|
||||
{/* Model Overrides */}
|
||||
<div className={cardClass}>
|
||||
<h2 className="text-lg font-semibold text-gray-200">Model Overrides</h2>
|
||||
<div>
|
||||
<label className={labelClass}>Primary Model</label>
|
||||
<select className={selectClass} value={character.model_overrides?.primary || 'llama3.3:70b'} onChange={(e) => handleNestedChange('model_overrides', 'primary', e.target.value)}>
|
||||
<option value="llama3.3:70b">llama3.3:70b</option>
|
||||
<option value="qwen2.5:7b">qwen2.5:7b</option>
|
||||
<option value="qwen3:32b">qwen3:32b</option>
|
||||
<option value="codestral:22b">codestral:22b</option>
|
||||
<option value="gemma-3-27b">gemma-3-27b</option>
|
||||
<option value="DeepSeek-R1-8B">DeepSeek-R1-8B</option>
|
||||
</select>
|
||||
</div>
|
||||
<div>
|
||||
<label className={labelClass}>Fast Model</label>
|
||||
<select className={selectClass} value={character.model_overrides?.fast || 'qwen2.5:7b'} onChange={(e) => handleNestedChange('model_overrides', 'fast', e.target.value)}>
|
||||
<option value="qwen2.5:7b">qwen2.5:7b</option>
|
||||
<option value="llama3.3:70b">llama3.3:70b</option>
|
||||
<option value="qwen3:32b">qwen3:32b</option>
|
||||
<option value="codestral:22b">codestral:22b</option>
|
||||
<option value="gemma-3-27b">gemma-3-27b</option>
|
||||
<option value="DeepSeek-R1-8B">DeepSeek-R1-8B</option>
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Custom Rules */}
|
||||
<div className={cardClass}>
|
||||
<div className="flex justify-between items-center">
|
||||
<h2 className="text-lg font-semibold text-gray-200">Custom Rules</h2>
|
||||
<button onClick={addRule} className="flex items-center gap-1 bg-indigo-600 hover:bg-indigo-500 text-white px-3 py-1.5 rounded-lg text-sm transition-colors">
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M12 4.5v15m7.5-7.5h-15" />
|
||||
</svg>
|
||||
Add Rule
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{(!character.custom_rules || character.custom_rules.length === 0) ? (
|
||||
<p className="text-sm text-gray-600 italic">No custom rules defined.</p>
|
||||
) : (
|
||||
<div className="space-y-4">
|
||||
{character.custom_rules.map((rule, idx) => (
|
||||
<div key={idx} className="border border-gray-700 p-4 rounded-lg relative bg-gray-800/50">
|
||||
<button
|
||||
onClick={() => removeRule(idx)}
|
||||
className="absolute top-3 right-3 text-gray-500 hover:text-red-400 transition-colors"
|
||||
title="Remove Rule"
|
||||
>
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M6 18L18 6M6 6l12 12" />
|
||||
</svg>
|
||||
</button>
|
||||
<div className="grid grid-cols-1 md:grid-cols-2 gap-4 mt-1">
|
||||
<div>
|
||||
<label className="block text-xs font-medium mb-1 text-gray-500">Trigger</label>
|
||||
<input type="text" className={inputClass + " text-sm"} value={rule.trigger || ''} onChange={(e) => handleRuleChange(idx, 'trigger', e.target.value)} />
|
||||
</div>
|
||||
<div>
|
||||
<label className="block text-xs font-medium mb-1 text-gray-500">Condition (Optional)</label>
|
||||
<input type="text" className={inputClass + " text-sm"} value={rule.condition || ''} onChange={(e) => handleRuleChange(idx, 'condition', e.target.value)} placeholder="e.g. time_of_day == morning" />
|
||||
</div>
|
||||
<div className="md:col-span-2">
|
||||
<label className="block text-xs font-medium mb-1 text-gray-500">Response</label>
|
||||
<textarea className={inputClass + " text-sm h-16 resize-y"} value={rule.response || ''} onChange={(e) => handleRuleChange(idx, 'response', e.target.value)} />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
297
homeai-character/src/CharacterProfiles.jsx
Normal file
297
homeai-character/src/CharacterProfiles.jsx
Normal file
@@ -0,0 +1,297 @@
|
||||
import { useState, useEffect } from 'react';
|
||||
import { useNavigate } from 'react-router-dom';
|
||||
import { validateCharacter } from './SchemaValidator';
|
||||
|
||||
const STORAGE_KEY = 'homeai_characters';
|
||||
const ACTIVE_KEY = 'homeai_active_character';
|
||||
|
||||
function loadProfiles() {
|
||||
try {
|
||||
const raw = localStorage.getItem(STORAGE_KEY);
|
||||
return raw ? JSON.parse(raw) : [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function saveProfiles(profiles) {
|
||||
localStorage.setItem(STORAGE_KEY, JSON.stringify(profiles));
|
||||
}
|
||||
|
||||
function getActiveId() {
|
||||
return localStorage.getItem(ACTIVE_KEY) || null;
|
||||
}
|
||||
|
||||
function setActiveId(id) {
|
||||
localStorage.setItem(ACTIVE_KEY, id);
|
||||
}
|
||||
|
||||
export default function CharacterProfiles() {
|
||||
const [profiles, setProfiles] = useState(loadProfiles);
|
||||
const [activeId, setActive] = useState(getActiveId);
|
||||
const [error, setError] = useState(null);
|
||||
const [dragOver, setDragOver] = useState(false);
|
||||
const navigate = useNavigate();
|
||||
|
||||
useEffect(() => {
|
||||
saveProfiles(profiles);
|
||||
}, [profiles]);
|
||||
|
||||
const handleImport = (e) => {
|
||||
const files = Array.from(e.target?.files || []);
|
||||
importFiles(files);
|
||||
if (e.target) e.target.value = '';
|
||||
};
|
||||
|
||||
const importFiles = (files) => {
|
||||
files.forEach(file => {
|
||||
if (!file.name.endsWith('.json')) return;
|
||||
const reader = new FileReader();
|
||||
reader.onload = (ev) => {
|
||||
try {
|
||||
const data = JSON.parse(ev.target.result);
|
||||
validateCharacter(data);
|
||||
const id = data.name + '_' + Date.now();
|
||||
setProfiles(prev => [...prev, { id, data, image: null, addedAt: new Date().toISOString() }]);
|
||||
setError(null);
|
||||
} catch (err) {
|
||||
setError(`Import failed for ${file.name}: ${err.message}`);
|
||||
}
|
||||
};
|
||||
reader.readAsText(file);
|
||||
});
|
||||
};
|
||||
|
||||
const handleDrop = (e) => {
|
||||
e.preventDefault();
|
||||
setDragOver(false);
|
||||
const files = Array.from(e.dataTransfer.files);
|
||||
importFiles(files);
|
||||
};
|
||||
|
||||
const handleImageUpload = (profileId, e) => {
|
||||
const file = e.target.files[0];
|
||||
if (!file) return;
|
||||
const reader = new FileReader();
|
||||
reader.onload = (ev) => {
|
||||
setProfiles(prev =>
|
||||
prev.map(p => p.id === profileId ? { ...p, image: ev.target.result } : p)
|
||||
);
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
};
|
||||
|
||||
const removeProfile = (id) => {
|
||||
setProfiles(prev => prev.filter(p => p.id !== id));
|
||||
if (activeId === id) {
|
||||
setActive(null);
|
||||
localStorage.removeItem(ACTIVE_KEY);
|
||||
}
|
||||
};
|
||||
|
||||
const activateProfile = (id) => {
|
||||
setActive(id);
|
||||
setActiveId(id);
|
||||
};
|
||||
|
||||
const exportProfile = (profile) => {
|
||||
const dataStr = "data:text/json;charset=utf-8," + encodeURIComponent(JSON.stringify(profile.data, null, 2));
|
||||
const a = document.createElement('a');
|
||||
a.href = dataStr;
|
||||
a.download = `${profile.data.name || 'character'}.json`;
|
||||
a.click();
|
||||
};
|
||||
|
||||
const editProfile = (profile) => {
|
||||
// Store the profile data for the editor to pick up
|
||||
sessionStorage.setItem('edit_character', JSON.stringify(profile.data));
|
||||
sessionStorage.setItem('edit_character_profile_id', profile.id);
|
||||
navigate('/editor');
|
||||
};
|
||||
|
||||
const activeProfile = profiles.find(p => p.id === activeId);
|
||||
|
||||
return (
|
||||
<div className="space-y-8">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h1 className="text-3xl font-bold text-gray-100">Characters</h1>
|
||||
<p className="text-sm text-gray-500 mt-1">
|
||||
{profiles.length} profile{profiles.length !== 1 ? 's' : ''} stored
|
||||
{activeProfile && (
|
||||
<span className="ml-2 text-emerald-400">
|
||||
Active: {activeProfile.data.display_name || activeProfile.data.name}
|
||||
</span>
|
||||
)}
|
||||
</p>
|
||||
</div>
|
||||
<label className="flex items-center gap-2 px-4 py-2 bg-indigo-600 hover:bg-indigo-500 text-white rounded-lg cursor-pointer transition-colors">
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M12 4.5v15m7.5-7.5h-15" />
|
||||
</svg>
|
||||
Import JSON
|
||||
<input type="file" accept=".json" multiple className="hidden" onChange={handleImport} />
|
||||
</label>
|
||||
</div>
|
||||
|
||||
{error && (
|
||||
<div className="bg-red-900/30 border border-red-500/50 text-red-300 px-4 py-3 rounded-lg text-sm">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Drop zone */}
|
||||
<div
|
||||
onDragOver={(e) => { e.preventDefault(); setDragOver(true); }}
|
||||
onDragLeave={() => setDragOver(false)}
|
||||
onDrop={handleDrop}
|
||||
className={`border-2 border-dashed rounded-xl p-8 text-center transition-colors ${
|
||||
dragOver
|
||||
? 'border-indigo-500 bg-indigo-500/10'
|
||||
: 'border-gray-700 hover:border-gray-600'
|
||||
}`}
|
||||
>
|
||||
<svg className="w-10 h-10 mx-auto text-gray-600 mb-3" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M3 16.5v2.25A2.25 2.25 0 005.25 21h13.5A2.25 2.25 0 0021 18.75V16.5m-13.5-9L12 3m0 0l4.5 4.5M12 3v13.5" />
|
||||
</svg>
|
||||
<p className="text-gray-500 text-sm">Drop character JSON files here to import</p>
|
||||
</div>
|
||||
|
||||
{/* Profile grid */}
|
||||
{profiles.length === 0 ? (
|
||||
<div className="text-center py-16">
|
||||
<svg className="w-16 h-16 mx-auto text-gray-700 mb-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M15.75 6a3.75 3.75 0 11-7.5 0 3.75 3.75 0 017.5 0zM4.501 20.118a7.5 7.5 0 0114.998 0A17.933 17.933 0 0112 21.75c-2.676 0-5.216-.584-7.499-1.632z" />
|
||||
</svg>
|
||||
<p className="text-gray-500">No character profiles yet. Import a JSON file to get started.</p>
|
||||
</div>
|
||||
) : (
|
||||
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-6">
|
||||
{profiles.map(profile => {
|
||||
const isActive = profile.id === activeId;
|
||||
const char = profile.data;
|
||||
return (
|
||||
<div
|
||||
key={profile.id}
|
||||
className={`relative rounded-xl border overflow-hidden transition-all duration-200 ${
|
||||
isActive
|
||||
? 'border-emerald-500/60 bg-emerald-500/5 ring-1 ring-emerald-500/30'
|
||||
: 'border-gray-700 bg-gray-800/50 hover:border-gray-600'
|
||||
}`}
|
||||
>
|
||||
{/* Image area */}
|
||||
<div className="relative h-48 bg-gray-900 flex items-center justify-center overflow-hidden group">
|
||||
{profile.image ? (
|
||||
<img
|
||||
src={profile.image}
|
||||
alt={char.display_name || char.name}
|
||||
className="w-full h-full object-cover"
|
||||
/>
|
||||
) : (
|
||||
<div className="text-6xl font-bold text-gray-700 select-none">
|
||||
{(char.display_name || char.name || '?')[0].toUpperCase()}
|
||||
</div>
|
||||
)}
|
||||
{/* Image upload overlay */}
|
||||
<label className="absolute inset-0 flex items-center justify-center bg-black/50 opacity-0 group-hover:opacity-100 transition-opacity cursor-pointer">
|
||||
<div className="text-center">
|
||||
<svg className="w-8 h-8 mx-auto text-white/80 mb-1" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M6.827 6.175A2.31 2.31 0 015.186 7.23c-.38.054-.757.112-1.134.175C2.999 7.58 2.25 8.507 2.25 9.574V18a2.25 2.25 0 002.25 2.25h15A2.25 2.25 0 0021.75 18V9.574c0-1.067-.75-1.994-1.802-2.169a47.865 47.865 0 00-1.134-.175 2.31 2.31 0 01-1.64-1.055l-.822-1.316a2.192 2.192 0 00-1.736-1.039 48.774 48.774 0 00-5.232 0 2.192 2.192 0 00-1.736 1.039l-.821 1.316z" />
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M16.5 12.75a4.5 4.5 0 11-9 0 4.5 4.5 0 019 0z" />
|
||||
</svg>
|
||||
<span className="text-xs text-white/70">Change image</span>
|
||||
</div>
|
||||
<input
|
||||
type="file"
|
||||
accept="image/*"
|
||||
className="hidden"
|
||||
onChange={(e) => handleImageUpload(profile.id, e)}
|
||||
/>
|
||||
</label>
|
||||
{/* Active badge */}
|
||||
{isActive && (
|
||||
<span className="absolute top-2 right-2 px-2 py-0.5 bg-emerald-500 text-white text-xs font-medium rounded-full">
|
||||
Active
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Info */}
|
||||
<div className="p-4 space-y-3">
|
||||
<div>
|
||||
<h3 className="text-lg font-semibold text-gray-200">
|
||||
{char.display_name || char.name}
|
||||
</h3>
|
||||
<p className="text-xs text-gray-500 mt-0.5">{char.description}</p>
|
||||
</div>
|
||||
|
||||
{/* Meta chips */}
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
<span className="px-2 py-0.5 bg-gray-700/70 text-gray-400 text-xs rounded-full">
|
||||
{char.tts?.engine || 'kokoro'}
|
||||
</span>
|
||||
<span className="px-2 py-0.5 bg-gray-700/70 text-gray-400 text-xs rounded-full">
|
||||
{char.model_overrides?.primary || 'default'}
|
||||
</span>
|
||||
{char.tts?.kokoro_voice && (
|
||||
<span className="px-2 py-0.5 bg-gray-700/70 text-gray-400 text-xs rounded-full">
|
||||
{char.tts.kokoro_voice}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Actions */}
|
||||
<div className="flex gap-2 pt-1">
|
||||
{!isActive ? (
|
||||
<button
|
||||
onClick={() => activateProfile(profile.id)}
|
||||
className="flex-1 px-3 py-1.5 bg-emerald-600 hover:bg-emerald-500 text-white text-sm rounded-lg transition-colors"
|
||||
>
|
||||
Activate
|
||||
</button>
|
||||
) : (
|
||||
<button
|
||||
disabled
|
||||
className="flex-1 px-3 py-1.5 bg-gray-700 text-gray-500 text-sm rounded-lg cursor-not-allowed"
|
||||
>
|
||||
Active
|
||||
</button>
|
||||
)}
|
||||
<button
|
||||
onClick={() => editProfile(profile)}
|
||||
className="px-3 py-1.5 bg-gray-700 hover:bg-gray-600 text-gray-300 text-sm rounded-lg transition-colors"
|
||||
title="Edit"
|
||||
>
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M16.862 4.487l1.687-1.688a1.875 1.875 0 112.652 2.652L10.582 16.07a4.5 4.5 0 01-1.897 1.13L6 18l.8-2.685a4.5 4.5 0 011.13-1.897l8.932-8.931zm0 0L19.5 7.125M18 14v4.75A2.25 2.25 0 0115.75 21H5.25A2.25 2.25 0 013 18.75V8.25A2.25 2.25 0 015.25 6H10" />
|
||||
</svg>
|
||||
</button>
|
||||
<button
|
||||
onClick={() => exportProfile(profile)}
|
||||
className="px-3 py-1.5 bg-gray-700 hover:bg-gray-600 text-gray-300 text-sm rounded-lg transition-colors"
|
||||
title="Export"
|
||||
>
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M3 16.5v2.25A2.25 2.25 0 005.25 21h13.5A2.25 2.25 0 0021 18.75V16.5M16.5 12L12 16.5m0 0L7.5 12m4.5 4.5V3" />
|
||||
</svg>
|
||||
</button>
|
||||
<button
|
||||
onClick={() => removeProfile(profile.id)}
|
||||
className="px-3 py-1.5 bg-gray-700 hover:bg-red-600 text-gray-300 hover:text-white text-sm rounded-lg transition-colors"
|
||||
title="Delete"
|
||||
>
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M14.74 9l-.346 9m-4.788 0L9.26 9m9.968-3.21c.342.052.682.107 1.022.166m-1.022-.165L18.16 19.673a2.25 2.25 0 01-2.244 2.077H8.084a2.25 2.25 0 01-2.244-2.077L4.772 5.79m14.456 0a48.108 48.108 0 00-3.478-.397m-12 .562c.34-.059.68-.114 1.022-.165m0 0a48.11 48.11 0 013.478-.397m7.5 0v-.916c0-1.18-.91-2.164-2.09-2.201a51.964 51.964 0 00-3.32 0c-1.18.037-2.09 1.022-2.09 2.201v.916m7.5 0a48.667 48.667 0 00-7.5 0" />
|
||||
</svg>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
13
homeai-character/src/SchemaValidator.js
Normal file
13
homeai-character/src/SchemaValidator.js
Normal file
@@ -0,0 +1,13 @@
|
||||
import Ajv from 'ajv'
|
||||
import schema from '../schema/character.schema.json'
|
||||
|
||||
const ajv = new Ajv({ allErrors: true, strict: false })
|
||||
const validate = ajv.compile(schema)
|
||||
|
||||
export function validateCharacter(config) {
|
||||
const valid = validate(config)
|
||||
if (!valid) {
|
||||
throw new Error(ajv.errorsText(validate.errors))
|
||||
}
|
||||
return true
|
||||
}
|
||||
389
homeai-character/src/ServiceStatus.jsx
Normal file
389
homeai-character/src/ServiceStatus.jsx
Normal file
@@ -0,0 +1,389 @@
|
||||
import { useState, useEffect, useCallback } from 'react';
|
||||
|
||||
const SERVICES = [
|
||||
{
|
||||
name: 'Ollama',
|
||||
url: 'http://localhost:11434',
|
||||
healthPath: '/api/tags',
|
||||
uiUrl: null,
|
||||
description: 'Local LLM runtime',
|
||||
category: 'AI & LLM',
|
||||
restart: { type: 'launchd', id: 'gui/501/com.homeai.ollama' },
|
||||
},
|
||||
{
|
||||
name: 'Open WebUI',
|
||||
url: 'http://localhost:3030',
|
||||
healthPath: '/',
|
||||
uiUrl: 'http://localhost:3030',
|
||||
description: 'Chat interface',
|
||||
category: 'AI & LLM',
|
||||
restart: { type: 'docker', id: 'homeai-open-webui' },
|
||||
},
|
||||
{
|
||||
name: 'OpenClaw Gateway',
|
||||
url: 'http://localhost:8080',
|
||||
healthPath: '/',
|
||||
uiUrl: null,
|
||||
description: 'Agent gateway',
|
||||
category: 'Agent',
|
||||
restart: { type: 'launchd', id: 'gui/501/com.homeai.openclaw' },
|
||||
},
|
||||
{
|
||||
name: 'OpenClaw Bridge',
|
||||
url: 'http://localhost:8081',
|
||||
healthPath: '/',
|
||||
uiUrl: null,
|
||||
description: 'HTTP-to-CLI bridge',
|
||||
category: 'Agent',
|
||||
restart: { type: 'launchd', id: 'gui/501/com.homeai.openclaw-bridge' },
|
||||
},
|
||||
{
|
||||
name: 'Wyoming STT',
|
||||
url: 'http://localhost:10300',
|
||||
healthPath: '/',
|
||||
uiUrl: null,
|
||||
description: 'Whisper speech-to-text',
|
||||
category: 'Voice',
|
||||
tcp: true,
|
||||
restart: { type: 'launchd', id: 'gui/501/com.homeai.wyoming-stt' },
|
||||
},
|
||||
{
|
||||
name: 'Wyoming TTS',
|
||||
url: 'http://localhost:10301',
|
||||
healthPath: '/',
|
||||
uiUrl: null,
|
||||
description: 'Kokoro text-to-speech',
|
||||
category: 'Voice',
|
||||
tcp: true,
|
||||
restart: { type: 'launchd', id: 'gui/501/com.homeai.wyoming-tts' },
|
||||
},
|
||||
{
|
||||
name: 'Wyoming Satellite',
|
||||
url: 'http://localhost:10700',
|
||||
healthPath: '/',
|
||||
uiUrl: null,
|
||||
description: 'Mac Mini mic/speaker satellite',
|
||||
category: 'Voice',
|
||||
tcp: true,
|
||||
restart: { type: 'launchd', id: 'gui/501/com.homeai.wyoming-satellite' },
|
||||
},
|
||||
{
|
||||
name: 'Character Dashboard',
|
||||
url: 'http://localhost:5173',
|
||||
healthPath: '/',
|
||||
uiUrl: 'http://localhost:5173',
|
||||
description: 'Character manager & service status',
|
||||
category: 'Agent',
|
||||
restart: { type: 'launchd', id: 'gui/501/com.homeai.character-dashboard' },
|
||||
},
|
||||
{
|
||||
name: 'Home Assistant',
|
||||
url: 'https://10.0.0.199:8123',
|
||||
healthPath: '/api/',
|
||||
uiUrl: 'https://10.0.0.199:8123',
|
||||
description: 'Smart home platform',
|
||||
category: 'Smart Home',
|
||||
},
|
||||
{
|
||||
name: 'Uptime Kuma',
|
||||
url: 'http://localhost:3001',
|
||||
healthPath: '/',
|
||||
uiUrl: 'http://localhost:3001',
|
||||
description: 'Service health monitoring',
|
||||
category: 'Infrastructure',
|
||||
restart: { type: 'docker', id: 'homeai-uptime-kuma' },
|
||||
},
|
||||
{
|
||||
name: 'n8n',
|
||||
url: 'http://localhost:5678',
|
||||
healthPath: '/',
|
||||
uiUrl: 'http://localhost:5678',
|
||||
description: 'Workflow automation',
|
||||
category: 'Infrastructure',
|
||||
restart: { type: 'docker', id: 'homeai-n8n' },
|
||||
},
|
||||
{
|
||||
name: 'code-server',
|
||||
url: 'http://localhost:8090',
|
||||
healthPath: '/',
|
||||
uiUrl: 'http://localhost:8090',
|
||||
description: 'Browser-based VS Code',
|
||||
category: 'Infrastructure',
|
||||
restart: { type: 'docker', id: 'homeai-code-server' },
|
||||
},
|
||||
{
|
||||
name: 'Portainer',
|
||||
url: 'https://10.0.0.199:9443',
|
||||
healthPath: '/',
|
||||
uiUrl: 'https://10.0.0.199:9443',
|
||||
description: 'Docker management',
|
||||
category: 'Infrastructure',
|
||||
},
|
||||
{
|
||||
name: 'Gitea',
|
||||
url: 'http://10.0.0.199:3000',
|
||||
healthPath: '/',
|
||||
uiUrl: 'http://10.0.0.199:3000',
|
||||
description: 'Self-hosted Git',
|
||||
category: 'Infrastructure',
|
||||
},
|
||||
];
|
||||
|
||||
const CATEGORY_ICONS = {
|
||||
'AI & LLM': (
|
||||
<svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M9.813 15.904L9 18.75l-.813-2.846a4.5 4.5 0 00-3.09-3.09L2.25 12l2.846-.813a4.5 4.5 0 003.09-3.09L9 5.25l.813 2.846a4.5 4.5 0 003.09 3.09L15.75 12l-2.846.813a4.5 4.5 0 00-3.09 3.09zM18.259 8.715L18 9.75l-.259-1.035a3.375 3.375 0 00-2.455-2.456L14.25 6l1.036-.259a3.375 3.375 0 002.455-2.456L18 2.25l.259 1.035a3.375 3.375 0 002.455 2.456L21.75 6l-1.036.259a3.375 3.375 0 00-2.455 2.456zM16.894 20.567L16.5 21.75l-.394-1.183a2.25 2.25 0 00-1.423-1.423L13.5 18.75l1.183-.394a2.25 2.25 0 001.423-1.423l.394-1.183.394 1.183a2.25 2.25 0 001.423 1.423l1.183.394-1.183.394a2.25 2.25 0 00-1.423 1.423z" />
|
||||
</svg>
|
||||
),
|
||||
'Agent': (
|
||||
<svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M8.25 3v1.5M4.5 8.25H3m18 0h-1.5M4.5 12H3m18 0h-1.5m-15 3.75H3m18 0h-1.5M8.25 19.5V21M12 3v1.5m0 15V21m3.75-18v1.5m0 15V21m-9-1.5h10.5a2.25 2.25 0 002.25-2.25V6.75a2.25 2.25 0 00-2.25-2.25H6.75A2.25 2.25 0 004.5 6.75v10.5a2.25 2.25 0 002.25 2.25zm.75-12h9v9h-9v-9z" />
|
||||
</svg>
|
||||
),
|
||||
'Voice': (
|
||||
<svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M12 18.75a6 6 0 006-6v-1.5m-6 7.5a6 6 0 01-6-6v-1.5m6 7.5v3.75m-3.75 0h7.5M12 15.75a3 3 0 01-3-3V4.5a3 3 0 116 0v8.25a3 3 0 01-3 3z" />
|
||||
</svg>
|
||||
),
|
||||
'Smart Home': (
|
||||
<svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M2.25 12l8.954-8.955c.44-.439 1.152-.439 1.591 0L21.75 12M4.5 9.75v10.125c0 .621.504 1.125 1.125 1.125H9.75v-4.875c0-.621.504-1.125 1.125-1.125h2.25c.621 0 1.125.504 1.125 1.125V21h4.125c.621 0 1.125-.504 1.125-1.125V9.75M8.25 21h8.25" />
|
||||
</svg>
|
||||
),
|
||||
'Infrastructure': (
|
||||
<svg className="w-5 h-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M5.25 14.25h13.5m-13.5 0a3 3 0 01-3-3m3 3a3 3 0 100 6h13.5a3 3 0 100-6m-16.5-3a3 3 0 013-3h13.5a3 3 0 013 3m-19.5 0a4.5 4.5 0 01.9-2.7L5.737 5.1a3.375 3.375 0 012.7-1.35h7.126c1.062 0 2.062.5 2.7 1.35l2.587 3.45a4.5 4.5 0 01.9 2.7m0 0a3 3 0 01-3 3m0 3h.008v.008h-.008v-.008zm0-6h.008v.008h-.008v-.008zm-3 6h.008v.008h-.008v-.008zm0-6h.008v.008h-.008v-.008z" />
|
||||
</svg>
|
||||
),
|
||||
};
|
||||
|
||||
function StatusDot({ status }) {
|
||||
const colors = {
|
||||
online: 'bg-emerald-400 shadow-emerald-400/50',
|
||||
offline: 'bg-red-400 shadow-red-400/50',
|
||||
checking: 'bg-amber-400 shadow-amber-400/50 animate-pulse',
|
||||
unknown: 'bg-gray-500',
|
||||
};
|
||||
return (
|
||||
<span className={`inline-block w-2.5 h-2.5 rounded-full shadow-lg ${colors[status] || colors.unknown}`} />
|
||||
);
|
||||
}
|
||||
|
||||
export default function ServiceStatus() {
|
||||
const [statuses, setStatuses] = useState(() =>
|
||||
Object.fromEntries(SERVICES.map(s => [s.name, { status: 'checking', lastCheck: null, responseTime: null }]))
|
||||
);
|
||||
const [lastRefresh, setLastRefresh] = useState(null);
|
||||
const [restarting, setRestarting] = useState({});
|
||||
|
||||
const checkService = useCallback(async (service) => {
|
||||
try {
|
||||
// Route all checks through the server-side proxy to avoid CORS and
|
||||
// self-signed SSL cert issues in the browser.
|
||||
const target = encodeURIComponent(service.url + service.healthPath);
|
||||
const modeParam = service.tcp ? '&mode=tcp' : '';
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), 8000);
|
||||
|
||||
const res = await fetch(`/api/health?url=${target}${modeParam}`, { signal: controller.signal });
|
||||
clearTimeout(timeout);
|
||||
|
||||
const data = await res.json();
|
||||
return { status: data.status, lastCheck: new Date(), responseTime: data.responseTime };
|
||||
} catch {
|
||||
return { status: 'offline', lastCheck: new Date(), responseTime: null };
|
||||
}
|
||||
}, []);
|
||||
|
||||
const refreshAll = useCallback(async () => {
|
||||
// Mark all as checking
|
||||
setStatuses(prev =>
|
||||
Object.fromEntries(Object.entries(prev).map(([k, v]) => [k, { ...v, status: 'checking' }]))
|
||||
);
|
||||
|
||||
const results = await Promise.allSettled(
|
||||
SERVICES.map(async (service) => {
|
||||
const result = await checkService(service);
|
||||
return { name: service.name, ...result };
|
||||
})
|
||||
);
|
||||
|
||||
const newStatuses = {};
|
||||
for (const r of results) {
|
||||
if (r.status === 'fulfilled') {
|
||||
newStatuses[r.value.name] = {
|
||||
status: r.value.status,
|
||||
lastCheck: r.value.lastCheck,
|
||||
responseTime: r.value.responseTime,
|
||||
};
|
||||
}
|
||||
}
|
||||
setStatuses(prev => ({ ...prev, ...newStatuses }));
|
||||
setLastRefresh(new Date());
|
||||
}, [checkService]);
|
||||
|
||||
useEffect(() => {
|
||||
refreshAll();
|
||||
const interval = setInterval(refreshAll, 30000);
|
||||
return () => clearInterval(interval);
|
||||
}, [refreshAll]);
|
||||
|
||||
const restartService = useCallback(async (service) => {
|
||||
if (!service.restart) return;
|
||||
setRestarting(prev => ({ ...prev, [service.name]: true }));
|
||||
try {
|
||||
const res = await fetch('/api/service/restart', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(service.restart),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (!data.ok) {
|
||||
console.error(`Restart failed for ${service.name}:`, data.error);
|
||||
}
|
||||
// Wait a moment for the service to come back, then re-check
|
||||
setTimeout(async () => {
|
||||
const result = await checkService(service);
|
||||
setStatuses(prev => ({ ...prev, [service.name]: result }));
|
||||
setRestarting(prev => ({ ...prev, [service.name]: false }));
|
||||
}, 3000);
|
||||
} catch (err) {
|
||||
console.error(`Restart failed for ${service.name}:`, err);
|
||||
setRestarting(prev => ({ ...prev, [service.name]: false }));
|
||||
}
|
||||
}, [checkService]);
|
||||
|
||||
const categories = [...new Set(SERVICES.map(s => s.category))];
|
||||
const onlineCount = Object.values(statuses).filter(s => s.status === 'online').length;
|
||||
const offlineCount = Object.values(statuses).filter(s => s.status === 'offline').length;
|
||||
const totalCount = SERVICES.length;
|
||||
const allOnline = onlineCount === totalCount;
|
||||
|
||||
return (
|
||||
<div className="space-y-8">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h1 className="text-3xl font-bold text-gray-100">Service Status</h1>
|
||||
<p className="text-sm text-gray-500 mt-1">
|
||||
{onlineCount}/{totalCount} services online
|
||||
{lastRefresh && (
|
||||
<span className="ml-3">
|
||||
Last check: {lastRefresh.toLocaleTimeString()}
|
||||
</span>
|
||||
)}
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
onClick={refreshAll}
|
||||
className="flex items-center gap-2 px-4 py-2 bg-gray-800 hover:bg-gray-700 text-gray-300 rounded-lg border border-gray-700 transition-colors"
|
||||
>
|
||||
<svg className="w-4 h-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M16.023 9.348h4.992v-.001M2.985 19.644v-4.992m0 0h4.992m-4.993 0l3.181 3.183a8.25 8.25 0 0013.803-3.7M4.031 9.865a8.25 8.25 0 0113.803-3.7l3.181 3.182" />
|
||||
</svg>
|
||||
Refresh
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Summary bar */}
|
||||
<div className="h-2 rounded-full bg-gray-800 overflow-hidden flex">
|
||||
{allOnline ? (
|
||||
<div
|
||||
className="h-full bg-gradient-to-r from-purple-500 to-indigo-500 transition-all duration-500"
|
||||
style={{ width: '100%' }}
|
||||
/>
|
||||
) : (
|
||||
<>
|
||||
<div
|
||||
className="h-full bg-gradient-to-r from-emerald-500 to-emerald-400 transition-all duration-500"
|
||||
style={{ width: `${(onlineCount / totalCount) * 100}%` }}
|
||||
/>
|
||||
<div
|
||||
className="h-full bg-gradient-to-r from-red-500 to-red-400 transition-all duration-500"
|
||||
style={{ width: `${(offlineCount / totalCount) * 100}%` }}
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Service grid by category */}
|
||||
{categories.map(category => (
|
||||
<div key={category}>
|
||||
<div className="flex items-center gap-2 mb-4">
|
||||
<span className="text-gray-400">{CATEGORY_ICONS[category]}</span>
|
||||
<h2 className="text-lg font-semibold text-gray-300">{category}</h2>
|
||||
</div>
|
||||
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4">
|
||||
{SERVICES.filter(s => s.category === category).map(service => {
|
||||
const st = statuses[service.name] || { status: 'unknown' };
|
||||
return (
|
||||
<div
|
||||
key={service.name}
|
||||
className={`relative rounded-xl border p-4 transition-all duration-200 ${
|
||||
st.status === 'online'
|
||||
? 'bg-gray-800/50 border-gray-700 hover:border-emerald-500/50'
|
||||
: st.status === 'offline'
|
||||
? 'bg-gray-800/50 border-red-500/30 hover:border-red-500/50'
|
||||
: 'bg-gray-800/50 border-gray-700'
|
||||
}`}
|
||||
>
|
||||
<div className="flex items-start justify-between">
|
||||
<div className="flex-1">
|
||||
<div className="flex items-center gap-2">
|
||||
<StatusDot status={st.status} />
|
||||
<h3 className="font-medium text-gray-200">{service.name}</h3>
|
||||
</div>
|
||||
<p className="text-xs text-gray-500 mt-1">{service.description}</p>
|
||||
{st.responseTime !== null && (
|
||||
<p className="text-xs text-gray-600 mt-0.5">{st.responseTime}ms</p>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
{service.restart && st.status === 'offline' && (
|
||||
<button
|
||||
onClick={() => restartService(service)}
|
||||
disabled={restarting[service.name]}
|
||||
className="text-xs px-2.5 py-1 rounded-md bg-amber-600/80 hover:bg-amber-500 disabled:bg-gray-700 disabled:text-gray-500 text-white transition-colors flex items-center gap-1"
|
||||
>
|
||||
{restarting[service.name] ? (
|
||||
<>
|
||||
<svg className="w-3 h-3 animate-spin" fill="none" viewBox="0 0 24 24">
|
||||
<circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4" />
|
||||
<path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4z" />
|
||||
</svg>
|
||||
Restarting
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<svg className="w-3 h-3" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M5.636 18.364a9 9 0 010-12.728m12.728 0a9 9 0 010 12.728M12 9v3m0 0v3m0-3h3m-3 0H9" />
|
||||
</svg>
|
||||
Restart
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
)}
|
||||
{service.uiUrl && (
|
||||
<a
|
||||
href={service.uiUrl}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="text-xs px-2.5 py-1 rounded-md bg-gray-700 hover:bg-gray-600 text-gray-300 transition-colors flex items-center gap-1"
|
||||
>
|
||||
Open
|
||||
<svg className="w-3 h-3" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M13.5 6H5.25A2.25 2.25 0 003 8.25v10.5A2.25 2.25 0 005.25 21h10.5A2.25 2.25 0 0018 18.75V10.5m-10.5 6L21 3m0 0h-5.25M21 3v5.25" />
|
||||
</svg>
|
||||
</a>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
1
homeai-character/src/assets/react.svg
Normal file
1
homeai-character/src/assets/react.svg
Normal file
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="35.93" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 228"><path fill="#00D8FF" d="M210.483 73.824a171.49 171.49 0 0 0-8.24-2.597c.465-1.9.893-3.777 1.273-5.621c6.238-30.281 2.16-54.676-11.769-62.708c-13.355-7.7-35.196.329-57.254 19.526a171.23 171.23 0 0 0-6.375 5.848a155.866 155.866 0 0 0-4.241-3.917C100.759 3.829 77.587-4.822 63.673 3.233C50.33 10.957 46.379 33.89 51.995 62.588a170.974 170.974 0 0 0 1.892 8.48c-3.28.932-6.445 1.924-9.474 2.98C17.309 83.498 0 98.307 0 113.668c0 15.865 18.582 31.778 46.812 41.427a145.52 145.52 0 0 0 6.921 2.165a167.467 167.467 0 0 0-2.01 9.138c-5.354 28.2-1.173 50.591 12.134 58.266c13.744 7.926 36.812-.22 59.273-19.855a145.567 145.567 0 0 0 5.342-4.923a168.064 168.064 0 0 0 6.92 6.314c21.758 18.722 43.246 26.282 56.54 18.586c13.731-7.949 18.194-32.003 12.4-61.268a145.016 145.016 0 0 0-1.535-6.842c1.62-.48 3.21-.974 4.76-1.488c29.348-9.723 48.443-25.443 48.443-41.52c0-15.417-17.868-30.326-45.517-39.844Zm-6.365 70.984c-1.4.463-2.836.91-4.3 1.345c-3.24-10.257-7.612-21.163-12.963-32.432c5.106-11 9.31-21.767 12.459-31.957c2.619.758 5.16 1.557 7.61 2.4c23.69 8.156 38.14 20.213 38.14 29.504c0 9.896-15.606 22.743-40.946 31.14Zm-10.514 20.834c2.562 12.94 2.927 24.64 1.23 33.787c-1.524 8.219-4.59 13.698-8.382 15.893c-8.067 4.67-25.32-1.4-43.927-17.412a156.726 156.726 0 0 1-6.437-5.87c7.214-7.889 14.423-17.06 21.459-27.246c12.376-1.098 24.068-2.894 34.671-5.345a134.17 134.17 0 0 1 1.386 6.193ZM87.276 214.515c-7.882 2.783-14.16 2.863-17.955.675c-8.075-4.657-11.432-22.636-6.853-46.752a156.923 156.923 0 0 1 1.869-8.499c10.486 2.32 22.093 3.988 34.498 4.994c7.084 9.967 14.501 19.128 21.976 27.15a134.668 134.668 0 0 1-4.877 4.492c-9.933 8.682-19.886 14.842-28.658 17.94ZM50.35 144.747c-12.483-4.267-22.792-9.812-29.858-15.863c-6.35-5.437-9.555-10.836-9.555-15.216c0-9.322 13.897-21.212 37.076-29.293c2.813-.98 5.757-1.905 8.812-2.773c3.204 10.42 7.406 21.315 12.477 32.332c-5.137 11.18-9.399 22.249-12.634 32.792a134.718 134.718 0 0 1-6.318-1.979Zm12.378-84.26c-4.811-24.587-1.616-43.134 6.425-47.789c8.564-4.958 27.502 2.111 47.463 19.835a144.318 144.318 0 0 1 3.841 3.545c-7.438 7.987-14.787 17.08-21.808 26.988c-12.04 1.116-23.565 2.908-34.161 5.309a160.342 160.342 0 0 1-1.76-7.887Zm110.427 27.268a347.8 347.8 0 0 0-7.785-12.803c8.168 1.033 15.994 2.404 23.343 4.08c-2.206 7.072-4.956 14.465-8.193 22.045a381.151 381.151 0 0 0-7.365-13.322Zm-45.032-43.861c5.044 5.465 10.096 11.566 15.065 18.186a322.04 322.04 0 0 0-30.257-.006c4.974-6.559 10.069-12.652 15.192-18.18ZM82.802 87.83a323.167 323.167 0 0 0-7.227 13.238c-3.184-7.553-5.909-14.98-8.134-22.152c7.304-1.634 15.093-2.97 23.209-3.984a321.524 321.524 0 0 0-7.848 12.897Zm8.081 65.352c-8.385-.936-16.291-2.203-23.593-3.793c2.26-7.3 5.045-14.885 8.298-22.6a321.187 321.187 0 0 0 7.257 13.246c2.594 4.48 5.28 8.868 8.038 13.147Zm37.542 31.03c-5.184-5.592-10.354-11.779-15.403-18.433c4.902.192 9.899.29 14.978.29c5.218 0 10.376-.117 15.453-.343c-4.985 6.774-10.018 12.97-15.028 18.486Zm52.198-57.817c3.422 7.8 6.306 15.345 8.596 22.52c-7.422 1.694-15.436 3.058-23.88 4.071a382.417 382.417 0 0 0 7.859-13.026a347.403 347.403 0 0 0 7.425-13.565Zm-16.898 8.101a358.557 358.557 0 0 1-12.281 19.815a329.4 329.4 0 0 1-23.444.823c-7.967 0-15.716-.248-23.178-.732a310.202 310.202 0 0 1-12.513-19.846h.001a307.41 307.41 0 0 1-10.923-20.627a310.278 310.278 0 0 1 10.89-20.637l-.001.001a307.318 307.318 0 0 1 12.413-19.761c7.613-.576 15.42-.876 23.31-.876H128c7.926 0 15.743.303 23.354.883a329.357 329.357 0 0 1 12.335 19.695a358.489 358.489 0 0 1 11.036 20.54a329.472 329.472 0 0 1-11 20.722Zm22.56-122.124c8.572 4.944 11.906 24.881 6.52 51.026c-.344 1.668-.73 3.367-1.15 5.09c-10.622-2.452-22.155-4.275-34.23-5.408c-7.034-10.017-14.323-19.124-21.64-27.008a160.789 160.789 0 0 1 5.888-5.4c18.9-16.447 36.564-22.941 44.612-18.3ZM128 90.808c12.625 0 22.86 10.235 22.86 22.86s-10.235 22.86-22.86 22.86s-22.86-10.235-22.86-22.86s10.235-22.86 22.86-22.86Z"></path></svg>
|
||||
|
After Width: | Height: | Size: 4.0 KiB |
13
homeai-character/src/index.css
Normal file
13
homeai-character/src/index.css
Normal file
@@ -0,0 +1,13 @@
|
||||
@import "tailwindcss";
|
||||
|
||||
body {
|
||||
margin: 0;
|
||||
background-color: #030712;
|
||||
color: #f3f4f6;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
-moz-osx-font-smoothing: grayscale;
|
||||
}
|
||||
|
||||
#root {
|
||||
min-height: 100vh;
|
||||
}
|
||||
10
homeai-character/src/main.jsx
Normal file
10
homeai-character/src/main.jsx
Normal file
@@ -0,0 +1,10 @@
|
||||
import { StrictMode } from 'react'
|
||||
import { createRoot } from 'react-dom/client'
|
||||
import './index.css'
|
||||
import App from './App.jsx'
|
||||
|
||||
createRoot(document.getElementById('root')).render(
|
||||
<StrictMode>
|
||||
<App />
|
||||
</StrictMode>,
|
||||
)
|
||||
169
homeai-character/vite.config.js
Normal file
169
homeai-character/vite.config.js
Normal file
@@ -0,0 +1,169 @@
|
||||
import { defineConfig } from 'vite'
|
||||
import react from '@vitejs/plugin-react'
|
||||
import tailwindcss from '@tailwindcss/vite'
|
||||
|
||||
function healthCheckPlugin() {
|
||||
return {
|
||||
name: 'health-check-proxy',
|
||||
configureServer(server) {
|
||||
server.middlewares.use('/api/health', async (req, res) => {
|
||||
const params = new URL(req.url, 'http://localhost').searchParams;
|
||||
const url = params.get('url');
|
||||
const mode = params.get('mode'); // 'tcp' for raw TCP port check
|
||||
if (!url) {
|
||||
res.writeHead(400, { 'Content-Type': 'application/json' });
|
||||
res.end(JSON.stringify({ error: 'Missing url param' }));
|
||||
return;
|
||||
}
|
||||
const start = Date.now();
|
||||
const parsedUrl = new URL(url);
|
||||
|
||||
try {
|
||||
if (mode === 'tcp') {
|
||||
// TCP socket connect check for non-HTTP services (e.g. Wyoming)
|
||||
const { default: net } = await import('net');
|
||||
await new Promise((resolve, reject) => {
|
||||
const socket = net.createConnection(
|
||||
{ host: parsedUrl.hostname, port: parseInt(parsedUrl.port), timeout: 5000 },
|
||||
() => { socket.destroy(); resolve(); }
|
||||
);
|
||||
socket.on('error', reject);
|
||||
socket.on('timeout', () => { socket.destroy(); reject(new Error('timeout')); });
|
||||
});
|
||||
} else {
|
||||
// HTTP/HTTPS health check
|
||||
const { default: https } = await import('https');
|
||||
const { default: http } = await import('http');
|
||||
const client = parsedUrl.protocol === 'https:' ? https : http;
|
||||
|
||||
await new Promise((resolve, reject) => {
|
||||
const reqObj = client.get(url, { rejectUnauthorized: false, timeout: 5000 }, (resp) => {
|
||||
resp.resume();
|
||||
resolve();
|
||||
});
|
||||
reqObj.on('error', reject);
|
||||
reqObj.on('timeout', () => { reqObj.destroy(); reject(new Error('timeout')); });
|
||||
});
|
||||
}
|
||||
|
||||
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||
res.end(JSON.stringify({ status: 'online', responseTime: Date.now() - start }));
|
||||
} catch {
|
||||
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||
res.end(JSON.stringify({ status: 'offline', responseTime: null }));
|
||||
}
|
||||
});
|
||||
// Service restart — runs launchctl or docker restart
|
||||
server.middlewares.use('/api/service/restart', async (req, res) => {
|
||||
if (req.method === 'OPTIONS') {
|
||||
res.writeHead(204, { 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Methods': 'POST', 'Access-Control-Allow-Headers': 'Content-Type' });
|
||||
res.end();
|
||||
return;
|
||||
}
|
||||
if (req.method !== 'POST') {
|
||||
res.writeHead(405);
|
||||
res.end();
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const chunks = [];
|
||||
for await (const chunk of req) chunks.push(chunk);
|
||||
const { type, id } = JSON.parse(Buffer.concat(chunks).toString());
|
||||
|
||||
if (!type || !id) {
|
||||
res.writeHead(400, { 'Content-Type': 'application/json' });
|
||||
res.end(JSON.stringify({ ok: false, error: 'Missing type or id' }));
|
||||
return;
|
||||
}
|
||||
|
||||
// Whitelist valid service IDs to prevent command injection
|
||||
const ALLOWED_LAUNCHD = [
|
||||
'gui/501/com.homeai.ollama',
|
||||
'gui/501/com.homeai.openclaw',
|
||||
'gui/501/com.homeai.openclaw-bridge',
|
||||
'gui/501/com.homeai.wyoming-stt',
|
||||
'gui/501/com.homeai.wyoming-tts',
|
||||
'gui/501/com.homeai.wyoming-satellite',
|
||||
'gui/501/com.homeai.character-dashboard',
|
||||
];
|
||||
const ALLOWED_DOCKER = [
|
||||
'homeai-open-webui',
|
||||
'homeai-uptime-kuma',
|
||||
'homeai-n8n',
|
||||
'homeai-code-server',
|
||||
];
|
||||
|
||||
let cmd;
|
||||
if (type === 'launchd' && ALLOWED_LAUNCHD.includes(id)) {
|
||||
cmd = ['launchctl', 'kickstart', '-k', id];
|
||||
} else if (type === 'docker' && ALLOWED_DOCKER.includes(id)) {
|
||||
cmd = ['docker', 'restart', id];
|
||||
} else {
|
||||
res.writeHead(403, { 'Content-Type': 'application/json' });
|
||||
res.end(JSON.stringify({ ok: false, error: 'Service not in allowed list' }));
|
||||
return;
|
||||
}
|
||||
|
||||
const { execFile } = await import('child_process');
|
||||
const { promisify } = await import('util');
|
||||
const execFileAsync = promisify(execFile);
|
||||
const { stdout, stderr } = await execFileAsync(cmd[0], cmd.slice(1), { timeout: 30000 });
|
||||
|
||||
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||
res.end(JSON.stringify({ ok: true, stdout: stdout.trim(), stderr: stderr.trim() }));
|
||||
} catch (err) {
|
||||
res.writeHead(500, { 'Content-Type': 'application/json' });
|
||||
res.end(JSON.stringify({ ok: false, error: err.message }));
|
||||
}
|
||||
});
|
||||
|
||||
// TTS preview proxy — forwards POST to OpenClaw bridge, returns audio
|
||||
server.middlewares.use('/api/tts', async (req, res) => {
|
||||
if (req.method !== 'POST') {
|
||||
res.writeHead(405);
|
||||
res.end();
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const { default: http } = await import('http');
|
||||
const chunks = [];
|
||||
for await (const chunk of req) chunks.push(chunk);
|
||||
const body = Buffer.concat(chunks);
|
||||
|
||||
await new Promise((resolve, reject) => {
|
||||
const proxyReq = http.request(
|
||||
'http://localhost:8081/api/tts',
|
||||
{ method: 'POST', headers: { 'Content-Type': 'application/json', 'Content-Length': body.length }, timeout: 30000 },
|
||||
(proxyRes) => {
|
||||
res.writeHead(proxyRes.statusCode, {
|
||||
'Content-Type': proxyRes.headers['content-type'] || 'audio/wav',
|
||||
});
|
||||
proxyRes.pipe(res);
|
||||
proxyRes.on('end', resolve);
|
||||
}
|
||||
);
|
||||
proxyReq.on('error', reject);
|
||||
proxyReq.on('timeout', () => { proxyReq.destroy(); reject(new Error('timeout')); });
|
||||
proxyReq.write(body);
|
||||
proxyReq.end();
|
||||
});
|
||||
} catch {
|
||||
res.writeHead(502, { 'Content-Type': 'application/json' });
|
||||
res.end(JSON.stringify({ error: 'TTS bridge unreachable' }));
|
||||
}
|
||||
});
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// https://vite.dev/config/
|
||||
export default defineConfig({
|
||||
plugins: [
|
||||
healthCheckPlugin(),
|
||||
tailwindcss(),
|
||||
react(),
|
||||
],
|
||||
server: {
|
||||
host: '0.0.0.0',
|
||||
},
|
||||
})
|
||||
28
homeai-llm/launchd/com.homeai.preload-models.plist
Normal file
28
homeai-llm/launchd/com.homeai.preload-models.plist
Normal file
@@ -0,0 +1,28 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
|
||||
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>com.homeai.preload-models</string>
|
||||
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/bin/bash</string>
|
||||
<string>/Users/aodhan/gitea/homeai/homeai-llm/scripts/preload-models.sh</string>
|
||||
</array>
|
||||
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
|
||||
<key>StandardOutPath</key>
|
||||
<string>/tmp/homeai-preload-models.log</string>
|
||||
|
||||
<key>StandardErrorPath</key>
|
||||
<string>/tmp/homeai-preload-models-error.log</string>
|
||||
|
||||
<!-- Delay 15s to let Ollama start first -->
|
||||
<key>ThrottleInterval</key>
|
||||
<integer>15</integer>
|
||||
</dict>
|
||||
</plist>
|
||||
55
homeai-llm/modelfiles/Qwen3.5-35B-A3B.Modelfile
Normal file
55
homeai-llm/modelfiles/Qwen3.5-35B-A3B.Modelfile
Normal file
@@ -0,0 +1,55 @@
|
||||
FROM /Users/aodhan/gitea/homeai/homeai-llm/modelfiles/lmstudio-community/Qwen3.5-35B-A3B-GGUF/Qwen3.5-35B-A3B-Q8_0.gguf
|
||||
|
||||
TEMPLATE """{{- if or .System .Tools }}<|im_start|>system
|
||||
{{- if .System }}
|
||||
{{ .System }}
|
||||
{{- end }}
|
||||
{{- if .Tools }}
|
||||
|
||||
# Tools
|
||||
|
||||
You may call one or more functions to assist with the user query.
|
||||
|
||||
You are provided with function signatures within <tools></tools> XML tags:
|
||||
<tools>
|
||||
{{- range .Tools }}
|
||||
{"type": "function", "function": {"name": "{{ .Function.Name }}", "description": "{{ .Function.Description }}", "parameters": {{ .Function.Parameters }}}}
|
||||
{{- end }}
|
||||
</tools>
|
||||
|
||||
For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
|
||||
<tool_call>
|
||||
{"name": <function-name>, "arguments": <args-json-object>}
|
||||
</tool_call>
|
||||
{{- end }}<|im_end|>
|
||||
{{- end }}
|
||||
{{- range $i, $_ := .Messages }}
|
||||
{{- $last := eq (len (slice $.Messages $i)) 1 }}
|
||||
{{- if eq .Role "user" }}<|im_start|>user
|
||||
{{ .Content }}<|im_end|>
|
||||
{{ else if eq .Role "assistant" }}<|im_start|>assistant
|
||||
{{- if .ToolCalls }}
|
||||
{{- range .ToolCalls }}
|
||||
<tool_call>
|
||||
{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
|
||||
</tool_call>
|
||||
{{- end }}
|
||||
{{- else }}{{ .Content }}
|
||||
{{- end }}{{ if not $last }}<|im_end|>
|
||||
{{ end }}
|
||||
{{- else if eq .Role "tool" }}<|im_start|>user
|
||||
<tool_response>
|
||||
{{ .Content }}
|
||||
</tool_response><|im_end|>
|
||||
{{ end }}
|
||||
{{- end }}<|im_start|>assistant
|
||||
"""
|
||||
|
||||
SYSTEM You are a helpful AI assistant.
|
||||
PARAMETER num_ctx 32768
|
||||
PARAMETER stop <|im_end|>
|
||||
PARAMETER stop <|endoftext|>
|
||||
PARAMETER temperature 0.6
|
||||
PARAMETER top_p 0.95
|
||||
PARAMETER presence_penalty 1.5
|
||||
PARAMETER top_k 20
|
||||
19
homeai-llm/scripts/preload-models.sh
Executable file
19
homeai-llm/scripts/preload-models.sh
Executable file
@@ -0,0 +1,19 @@
|
||||
#!/bin/bash
|
||||
# Pre-load voice pipeline models into Ollama with infinite keep_alive.
|
||||
# Run after Ollama starts (called by launchd or manually).
|
||||
# Only pins lightweight/MoE models — large dense models (70B) use default expiry.
|
||||
|
||||
OLLAMA_URL="http://localhost:11434"
|
||||
|
||||
# Wait for Ollama to be ready
|
||||
for i in $(seq 1 30); do
|
||||
curl -sf "$OLLAMA_URL/api/tags" > /dev/null 2>&1 && break
|
||||
sleep 2
|
||||
done
|
||||
|
||||
# Pin qwen3.5:35b-a3b (MoE, 38.7GB VRAM, voice pipeline default)
|
||||
echo "[preload] Loading qwen3.5:35b-a3b with keep_alive=-1..."
|
||||
curl -sf "$OLLAMA_URL/api/generate" \
|
||||
-d '{"model":"qwen3.5:35b-a3b","prompt":"ready","stream":false,"keep_alive":-1,"options":{"num_ctx":512}}' \
|
||||
> /dev/null 2>&1
|
||||
echo "[preload] qwen3.5:35b-a3b pinned in memory"
|
||||
420
homeai-voice/TROUBLESHOOTING.md
Normal file
420
homeai-voice/TROUBLESHOOTING.md
Normal file
@@ -0,0 +1,420 @@
|
||||
# Voice Pipeline Troubleshooting Guide
|
||||
|
||||
> Common issues and solutions for the voice pipeline setup
|
||||
|
||||
---
|
||||
|
||||
## Network Configuration
|
||||
|
||||
**Important**: The services are split across two machines:
|
||||
|
||||
| Service | Machine | IP Address |
|
||||
|---------|---------|------------|
|
||||
| OpenClaw Gateway | Mac Mini | 10.0.0.101 |
|
||||
| Wyoming STT | Mac Mini | 10.0.0.101 |
|
||||
| Wyoming TTS | Mac Mini | 10.0.0.101 |
|
||||
| Wyoming Satellite | Mac Mini | 10.0.0.101 |
|
||||
| Ollama | Mac Mini | 10.0.0.101 |
|
||||
| Home Assistant | Server (Docker) | 10.0.0.199 |
|
||||
|
||||
---
|
||||
|
||||
## Issue: OpenClaw Conversation Cannot Connect
|
||||
|
||||
### Symptoms
|
||||
- Integration installed but shows connection error
|
||||
- HA logs show timeout or connection refused
|
||||
- Error: "Cannot connect to OpenClaw service"
|
||||
|
||||
### Root Cause
|
||||
The OpenClaw Conversation integration is configured with the wrong host IP. It needs to point to the Mac Mini (10.0.0.101), not the HA server (10.0.0.199).
|
||||
|
||||
### Solution
|
||||
|
||||
1. **Open Home Assistant UI** at http://10.0.0.199:8123
|
||||
|
||||
2. **Go to Settings → Devices & Services**
|
||||
|
||||
3. **Find "OpenClaw Conversation"** integration
|
||||
|
||||
4. **Click "Configure"** (or delete and re-add)
|
||||
|
||||
5. **Set the correct configuration:**
|
||||
- **OpenClaw Host**: `10.0.0.101` (Mac Mini IP, NOT 10.0.0.199)
|
||||
- **OpenClaw Port**: `8080`
|
||||
- **Agent Name**: `main`
|
||||
- **Timeout**: `30`
|
||||
|
||||
6. **Save** and verify connection
|
||||
|
||||
### Verify Network Connectivity
|
||||
|
||||
From the HA server, test if it can reach OpenClaw:
|
||||
|
||||
```bash
|
||||
# SSH to HA server
|
||||
ssh 10.0.0.199
|
||||
|
||||
# Test OpenClaw connectivity
|
||||
curl http://10.0.0.101:8080/status
|
||||
|
||||
# Or use nc
|
||||
nc -z 10.0.0.101 8080 && echo "OpenClaw reachable" || echo "Cannot reach OpenClaw"
|
||||
```
|
||||
|
||||
From the Mac Mini, verify OpenClaw is listening:
|
||||
|
||||
```bash
|
||||
# Check OpenClaw is running
|
||||
launchctl list | grep openclaw
|
||||
|
||||
# Check it's listening on all interfaces
|
||||
lsof -i :8080
|
||||
|
||||
# Test locally
|
||||
curl http://localhost:8080/status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issue: Wyoming Services Cannot Connect
|
||||
|
||||
### Symptoms
|
||||
- Wyoming integrations show as unavailable
|
||||
- HA cannot reach STT/TTS services
|
||||
- Timeout errors in HA logs
|
||||
|
||||
### Solution
|
||||
|
||||
Wyoming services are also on the Mac Mini (10.0.0.101):
|
||||
|
||||
1. **Go to Settings → Devices & Services**
|
||||
|
||||
2. **For each Wyoming integration**, verify the host is set to **10.0.0.101**:
|
||||
- Wyoming STT: `10.0.0.101:10300`
|
||||
- Wyoming TTS: `10.0.0.101:10301`
|
||||
- Wyoming Satellite: `10.0.0.101:10700`
|
||||
|
||||
3. **Test connectivity from HA server:**
|
||||
|
||||
```bash
|
||||
ssh 10.0.0.199
|
||||
nc -z 10.0.0.101 10300 # STT
|
||||
nc -z 10.0.0.101 10301 # TTS
|
||||
nc -z 10.0.0.101 10700 # Satellite
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issue: Firewall Blocking Connections
|
||||
|
||||
### Symptoms
|
||||
- Services work locally on Mac Mini
|
||||
- Cannot connect from HA server
|
||||
- Connection timeout errors
|
||||
|
||||
### Solution
|
||||
|
||||
Check Mac Mini firewall settings:
|
||||
|
||||
```bash
|
||||
# Check firewall status
|
||||
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --getglobalstate
|
||||
|
||||
# If enabled, add exceptions for the services
|
||||
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /opt/homebrew/bin/ollama
|
||||
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/local/bin/openclaw
|
||||
|
||||
# Or temporarily disable for testing (not recommended for production)
|
||||
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --setglobalstate off
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issue: OpenClaw CLI Not Found in HA Container
|
||||
|
||||
### Symptoms
|
||||
- Integration uses CLI fallback
|
||||
- Error: "OpenClaw CLI not found"
|
||||
- Component works but responses fail
|
||||
|
||||
### Root Cause
|
||||
The `openclaw` command is not available inside the HA Docker container. The integration should use the HTTP API, not CLI.
|
||||
|
||||
### Solution
|
||||
|
||||
The OpenClawCLIAgent is a fallback. Ensure the integration is using the HTTP API:
|
||||
|
||||
1. Check the integration configuration uses the correct host/port
|
||||
2. Verify OpenClaw Gateway is accessible via HTTP
|
||||
3. The component will automatically use HTTP if available
|
||||
|
||||
---
|
||||
|
||||
## Issue: Voice Pipeline Not Responding
|
||||
|
||||
### Symptoms
|
||||
- Wake word detected but no response
|
||||
- Audio captured but not transcribed
|
||||
- Transcription works but no TTS output
|
||||
|
||||
### Debugging Steps
|
||||
|
||||
1. **Check all services are running:**
|
||||
|
||||
```bash
|
||||
# On Mac Mini
|
||||
./homeai-voice/scripts/test-services.sh
|
||||
```
|
||||
|
||||
2. **Test each component individually:**
|
||||
|
||||
```bash
|
||||
# Test wake word detection
|
||||
# Say "Hey Jarvis" and check satellite logs
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
|
||||
# Test STT
|
||||
# Check if audio is being transcribed
|
||||
tail -f /tmp/homeai-wyoming-stt.log
|
||||
|
||||
# Test OpenClaw
|
||||
openclaw agent --message "Hello" --agent main
|
||||
|
||||
# Test TTS
|
||||
tail -f /tmp/homeai-wyoming-tts.log
|
||||
```
|
||||
|
||||
3. **Check HA Voice Pipeline configuration:**
|
||||
- Settings → Voice Assistants
|
||||
- Verify pipeline uses correct STT, Conversation, and TTS
|
||||
- Ensure OpenClaw Conversation is selected
|
||||
|
||||
4. **Test from HA Assist:**
|
||||
- Type a query in HA Assist panel
|
||||
- Check if you get a response
|
||||
- This bypasses wake word and audio capture
|
||||
|
||||
---
|
||||
|
||||
## Monitoring Wake Word Detection
|
||||
|
||||
To see when the wake word ("Hey Jarvis") is being detected in real-time:
|
||||
|
||||
### Option 1: Watch Satellite Logs (Recommended)
|
||||
|
||||
The Wyoming Satellite handles wake word detection and audio streaming:
|
||||
|
||||
```bash
|
||||
# Terminal 1: Watch satellite logs
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
```
|
||||
|
||||
**What to look for:**
|
||||
- `Wake word detected` - Wake word was heard
|
||||
- `Streaming audio` - Audio being sent to STT
|
||||
- `Connected to server` - Connection status
|
||||
|
||||
### Option 2: Watch Wake Word Service Logs
|
||||
|
||||
```bash
|
||||
# Terminal 1: Watch wake word detection logs
|
||||
tail -f /tmp/homeai-wakeword.log
|
||||
```
|
||||
|
||||
### Option 3: Watch All Voice Pipeline Logs
|
||||
|
||||
```bash
|
||||
# Terminal 1: Watch all voice-related logs
|
||||
tail -f /tmp/homeai-*.log | grep -E "(wake|satellite|stt|tts|openclaw)"
|
||||
```
|
||||
|
||||
### Test Wake Word Detection
|
||||
|
||||
While watching the logs, try this:
|
||||
|
||||
1. **Say clearly**: "Hey Jarvis" (or your configured wake word)
|
||||
2. **Wait** for the acknowledgment beep
|
||||
3. **Speak your command**: "What time is it?"
|
||||
4. **Check logs** for activity
|
||||
|
||||
### Expected Log Output
|
||||
|
||||
When wake word is detected, you should see:
|
||||
|
||||
```
|
||||
[wyoming_satellite] Wake word detected
|
||||
[wyoming_satellite] Streaming audio to stt
|
||||
[wyoming_satellite] Connected to 10.0.0.101:10300
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issue: Audio Playback Not Working
|
||||
|
||||
### Symptoms
|
||||
- Pipeline works but no audio output
|
||||
- TTS generates audio but satellite doesn't play it
|
||||
- Silent responses
|
||||
|
||||
### Solution
|
||||
|
||||
1. **Check audio output device:**
|
||||
|
||||
```bash
|
||||
# On Mac Mini
|
||||
afplay /System/Library/Sounds/Glass.aiff
|
||||
```
|
||||
|
||||
2. **Check satellite configuration:**
|
||||
|
||||
```bash
|
||||
# View satellite config
|
||||
cat ~/Library/LaunchAgents/com.homeai.wyoming-satellite.plist
|
||||
|
||||
# Check logs for audio errors
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
```
|
||||
|
||||
3. **Verify SoX is installed:**
|
||||
|
||||
```bash
|
||||
which play
|
||||
brew install sox
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Issue: High Latency (>5 seconds)
|
||||
|
||||
### Symptoms
|
||||
- Long delay between wake word and response
|
||||
- Slow transcription or TTS generation
|
||||
|
||||
### Solutions
|
||||
|
||||
1. **Check network latency:**
|
||||
|
||||
```bash
|
||||
# From HA server to Mac Mini
|
||||
ping 10.0.0.101
|
||||
```
|
||||
|
||||
2. **Check Ollama model size:**
|
||||
|
||||
```bash
|
||||
# Smaller models are faster
|
||||
ollama list
|
||||
|
||||
# Switch to faster model in OpenClaw config
|
||||
# qwen2.5:7b is faster than llama3.3:70b
|
||||
```
|
||||
|
||||
3. **Check system resources:**
|
||||
|
||||
```bash
|
||||
# On Mac Mini
|
||||
top -l 1 | grep -E "CPU|PhysMem"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Correct Configuration Summary
|
||||
|
||||
### OpenClaw Conversation Integration
|
||||
- Host: `10.0.0.101` (Mac Mini)
|
||||
- Port: `8080`
|
||||
- Agent: `main`
|
||||
- Timeout: `30`
|
||||
|
||||
### Wyoming STT Integration
|
||||
- Host: `10.0.0.101` (Mac Mini)
|
||||
- Port: `10300`
|
||||
|
||||
### Wyoming TTS Integration
|
||||
- Host: `10.0.0.101` (Mac Mini)
|
||||
- Port: `10301`
|
||||
|
||||
### Wyoming Satellite Integration
|
||||
- Host: `10.0.0.101` (Mac Mini)
|
||||
- Port: `10700`
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [ ] All services running on Mac Mini (10.0.0.101)
|
||||
- [ ] HA can ping Mac Mini: `ping 10.0.0.101`
|
||||
- [ ] HA can reach OpenClaw: `curl http://10.0.0.101:8080/status`
|
||||
- [ ] HA can reach Wyoming STT: `nc -z 10.0.0.101 10300`
|
||||
- [ ] HA can reach Wyoming TTS: `nc -z 10.0.0.101 10301`
|
||||
- [ ] HA can reach Wyoming Satellite: `nc -z 10.0.0.101 10700`
|
||||
- [ ] OpenClaw Conversation integration configured with 10.0.0.101
|
||||
- [ ] Wyoming integrations configured with 10.0.0.101
|
||||
- [ ] Voice pipeline created and set as default
|
||||
- [ ] Test query in HA Assist returns response
|
||||
|
||||
---
|
||||
|
||||
## Bugs Fixed During Setup
|
||||
|
||||
The following bugs were discovered and fixed during initial setup (2026-03-08):
|
||||
|
||||
### 1. OpenClaw Network Binding
|
||||
|
||||
**Problem**: OpenClaw gateway was only listening on localhost (127.0.0.1), not accessible from HA server.
|
||||
|
||||
**Fix**: Added `"bind": "lan"` to `~/.openclaw/openclaw.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"gateway": {
|
||||
"port": 8080,
|
||||
"mode": "local",
|
||||
"bind": "lan",
|
||||
"auth": { "token": "..." }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Custom Component API Error
|
||||
|
||||
**Problem**: `async_set_agent()` was being called with `DOMAIN` (string) instead of `entry` (ConfigEntry object).
|
||||
|
||||
**Fix**: Changed parameter in `homeai-agent/custom_components/openclaw_conversation/__init__.py`:
|
||||
|
||||
```python
|
||||
# Line 83
|
||||
conversation.async_set_agent(hass, entry, agent) # Was: DOMAIN
|
||||
|
||||
# Line 94
|
||||
conversation.async_unset_agent(hass, entry) # Was: DOMAIN
|
||||
```
|
||||
|
||||
### 3. TTS Server Missing Version
|
||||
|
||||
**Problem**: `TtsProgram` initialization was missing required `version` parameter.
|
||||
|
||||
**Fix**: Added `version="1.0.0"` in `homeai-voice/tts/wyoming_kokoro_server.py` line 58.
|
||||
|
||||
### 4. Voice Commands Not Working (CLI Not in Docker)
|
||||
|
||||
**Problem**: HA Docker container couldn't access `openclaw` CLI.
|
||||
|
||||
**Fix**: Created OpenClaw HTTP Bridge (`homeai-agent/openclaw-http-bridge.py`) on port 8081 that translates HTTP POST requests to OpenClaw CLI calls. The custom component now uses port 8081 (HTTP bridge) instead of 8080 (gateway).
|
||||
|
||||
---
|
||||
|
||||
## Getting Help
|
||||
|
||||
If issues persist:
|
||||
|
||||
1. **Check service logs:**
|
||||
- Mac Mini: `/tmp/homeai-*.log`
|
||||
- HA: Settings → System → Logs
|
||||
|
||||
2. **Verify network connectivity** between machines
|
||||
|
||||
3. **Test each component** individually before testing the full pipeline
|
||||
|
||||
4. **Review configuration** in [`VOICE_PIPELINE_SETUP.md`](VOICE_PIPELINE_SETUP.md)
|
||||
435
homeai-voice/VOICE_PIPELINE_SETUP.md
Normal file
435
homeai-voice/VOICE_PIPELINE_SETUP.md
Normal file
@@ -0,0 +1,435 @@
|
||||
# Voice Pipeline Setup Guide
|
||||
|
||||
> Complete guide to setting up the end-to-end voice pipeline with OpenClaw integration
|
||||
|
||||
---
|
||||
|
||||
## Network Configuration
|
||||
|
||||
**Important**: Services are split across two machines:
|
||||
|
||||
| Service | Port | Location |
|
||||
|---------|------|----------|
|
||||
| Wyoming STT (Whisper large-v3) | 10300 | Mac Mini (10.0.0.101) |
|
||||
| Wyoming TTS (Kokoro ONNX) | 10301 | Mac Mini (10.0.0.101) |
|
||||
| Wyoming Satellite | 10700 | Mac Mini (10.0.0.101) |
|
||||
| openWakeWord | - | Mac Mini (10.0.0.101) |
|
||||
| OpenClaw Gateway | 8080 | Mac Mini (10.0.0.101) |
|
||||
| Ollama | 11434 | Mac Mini (10.0.0.101) |
|
||||
| Home Assistant (Docker) | 8123 | Server (10.0.0.199) |
|
||||
|
||||
**All integrations must point to 10.0.0.101 (Mac Mini), not 10.0.0.199 (HA server).**
|
||||
|
||||
---
|
||||
|
||||
## Current Status
|
||||
|
||||
### ✅ Services Running on Mac Mini (10.0.0.101)
|
||||
|
||||
| Service | Port | Status |
|
||||
|---------|------|--------|
|
||||
| Wyoming STT | 10300 | ✅ Running |
|
||||
| Wyoming TTS | 10301 | ✅ Running |
|
||||
| Wyoming Satellite | 10700 | ✅ Running |
|
||||
| openWakeWord | - | ✅ Running |
|
||||
| OpenClaw Gateway | 8080 | ✅ Running |
|
||||
| Ollama | 11434 | ✅ Running |
|
||||
|
||||
### ✅ Completed
|
||||
- Wyoming STT/TTS services installed and running
|
||||
- Wyoming Satellite installed and running
|
||||
- OpenClaw agent configured with home-assistant skill
|
||||
- Custom OpenClaw conversation component created
|
||||
|
||||
### 🔄 Next Steps
|
||||
1. Install OpenClaw conversation component in Home Assistant
|
||||
2. Configure Wyoming integrations in HA
|
||||
3. Create voice assistant pipeline with OpenClaw
|
||||
4. Test the full voice loop
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Install OpenClaw Conversation Component
|
||||
|
||||
Home Assistant is running in Docker on server 10.0.0.199. Use the automated installation script.
|
||||
|
||||
### Option A: Automated Installation (Recommended)
|
||||
|
||||
```bash
|
||||
# From Mac Mini, run the installation script
|
||||
cd ~/gitea/homeai/homeai-agent/custom_components
|
||||
./install-to-docker-ha.sh
|
||||
|
||||
# The script will:
|
||||
# 1. Create a tarball of the component
|
||||
# 2. Copy it to the HA server via SCP
|
||||
# 3. Extract it into the HA Docker container
|
||||
# 4. Provide next steps
|
||||
```
|
||||
|
||||
**Requirements:**
|
||||
- SSH access to 10.0.0.199
|
||||
- SSH keys configured (or password access)
|
||||
|
||||
### Option B: Manual Installation via SSH
|
||||
|
||||
```bash
|
||||
# 1. Create tarball
|
||||
cd ~/gitea/homeai/homeai-agent/custom_components
|
||||
tar -czf openclaw_conversation.tar.gz openclaw_conversation/
|
||||
|
||||
# 2. Copy to HA server
|
||||
scp openclaw_conversation.tar.gz 10.0.0.199:/tmp/
|
||||
|
||||
# 3. SSH to HA server and install
|
||||
ssh 10.0.0.199
|
||||
CONTAINER=$(docker ps --filter "name=homeassistant" --format "{{.Names}}" | head -n 1)
|
||||
docker cp /tmp/openclaw_conversation.tar.gz $CONTAINER:/tmp/
|
||||
docker exec $CONTAINER sh -c 'cd /config/custom_components && tar -xzf /tmp/openclaw_conversation.tar.gz'
|
||||
docker restart $CONTAINER
|
||||
```
|
||||
|
||||
### Option D: Using Home Assistant File Editor (Manual)
|
||||
|
||||
1. Open Home Assistant UI at http://10.0.0.199:8123
|
||||
2. Install the **File Editor** add-on if not already installed
|
||||
3. Create directory: `/config/custom_components/openclaw_conversation/`
|
||||
4. Copy each file from `homeai-agent/custom_components/openclaw_conversation/`:
|
||||
- `__init__.py`
|
||||
- `config_flow.py`
|
||||
- `const.py`
|
||||
- `conversation.py`
|
||||
- `manifest.json`
|
||||
- `strings.json`
|
||||
|
||||
### Verify Installation
|
||||
|
||||
After installation, restart Home Assistant:
|
||||
|
||||
```bash
|
||||
# Via SSH
|
||||
ssh 10.0.0.199 'docker restart homeassistant'
|
||||
|
||||
# Or via HA UI
|
||||
# Settings → System → Restart
|
||||
```
|
||||
|
||||
Check logs for any errors:
|
||||
- **Settings → System → Logs**
|
||||
- Look for "OpenClaw Conversation" in the logs
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Configure Wyoming Integrations
|
||||
|
||||
### Add Wyoming STT (Speech-to-Text)
|
||||
|
||||
1. Go to **Settings → Devices & Services → Add Integration**
|
||||
2. Search for **"Wyoming Protocol"**
|
||||
3. Configure:
|
||||
- **Host**: `10.0.0.101` ⚠️ **Mac Mini IP, not HA server IP (10.0.0.199)**
|
||||
- **Port**: `10300`
|
||||
- **Name**: `Mac Mini STT`
|
||||
4. Click **Submit**
|
||||
|
||||
### Add Wyoming TTS (Text-to-Speech)
|
||||
|
||||
1. Click **Add Integration** again
|
||||
2. Search for **"Wyoming Protocol"**
|
||||
3. Configure:
|
||||
- **Host**: `10.0.0.101` ⚠️ **Mac Mini IP**
|
||||
- **Port**: `10301`
|
||||
- **Name**: `Mac Mini TTS`
|
||||
4. Click **Submit**
|
||||
|
||||
### Add Wyoming Satellite
|
||||
|
||||
1. Click **Add Integration** again
|
||||
2. Search for **"Wyoming Protocol"**
|
||||
3. Configure:
|
||||
- **Host**: `10.0.0.101` ⚠️ **Mac Mini IP**
|
||||
- **Port**: `10700`
|
||||
- **Name**: `Mac Mini Living Room`
|
||||
4. Click **Submit**
|
||||
|
||||
### Verify Integrations
|
||||
|
||||
All three Wyoming integrations should appear in **Settings → Devices & Services**.
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Add OpenClaw Conversation Agent
|
||||
|
||||
### Via UI (Recommended)
|
||||
|
||||
1. Go to **Settings → Devices & Services → Add Integration**
|
||||
2. Search for **"OpenClaw Conversation"**
|
||||
3. Configure:
|
||||
- **OpenClaw Host**: `10.0.0.101` ⚠️ **Mac Mini IP, not HA server IP (10.0.0.199)**
|
||||
- **OpenClaw Port**: `8080`
|
||||
- **Agent Name**: `main`
|
||||
- **Timeout**: `30` seconds
|
||||
4. Click **Submit**
|
||||
|
||||
### Via YAML (Alternative)
|
||||
|
||||
Add to `/config/configuration.yaml`:
|
||||
|
||||
```yaml
|
||||
openclaw_conversation:
|
||||
openclaw_host: 10.0.0.101 # Mac Mini IP
|
||||
openclaw_port: 8080
|
||||
agent_name: main
|
||||
timeout: 30
|
||||
```
|
||||
|
||||
Then restart Home Assistant.
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Create Voice Assistant Pipeline
|
||||
|
||||
1. Go to **Settings → Voice Assistants**
|
||||
2. Click **Add Assistant**
|
||||
3. Configure:
|
||||
- **Name**: `HomeAI with OpenClaw`
|
||||
- **Language**: `English`
|
||||
- **Speech-to-Text**: Select `Mac Mini STT` (Wyoming)
|
||||
- **Conversation Agent**: Select `OpenClaw Conversation`
|
||||
- **Text-to-Speech**: Select `Mac Mini TTS` (Wyoming)
|
||||
4. Click **Create**
|
||||
|
||||
### Set as Default
|
||||
|
||||
1. In **Settings → Voice Assistants**
|
||||
2. Click the three dots next to "HomeAI with OpenClaw"
|
||||
3. Select **Set as preferred**
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Test the Pipeline
|
||||
|
||||
### Test 1: Text Input → TTS Output
|
||||
|
||||
1. Open Home Assistant UI
|
||||
2. Click the **Assist** icon (microphone) in the top-right corner
|
||||
3. Type: `"What time is it?"`
|
||||
4. Press Enter
|
||||
|
||||
**Expected Result**: You should hear a spoken response via Kokoro TTS
|
||||
|
||||
### Test 2: Voice Input → OpenClaw → TTS Output
|
||||
|
||||
1. Ensure Wyoming Satellite is running on Mac Mini:
|
||||
```bash
|
||||
launchctl list | grep wyoming-satellite
|
||||
```
|
||||
|
||||
2. Say the wake word: **"Hey Jarvis"**
|
||||
3. Wait for the beep/acknowledgment
|
||||
4. Speak: **"What time is it?"**
|
||||
|
||||
**Expected Result**: You should hear a spoken response
|
||||
|
||||
### Test 3: Home Assistant Control via Voice
|
||||
|
||||
1. Say: **"Hey Jarvis"**
|
||||
2. Speak: **"Turn on the reading lamp"**
|
||||
|
||||
**Expected Result**:
|
||||
- OpenClaw processes the request
|
||||
- Home Assistant skill executes the action
|
||||
- Light turns on
|
||||
- You hear a confirmation via TTS
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: OpenClaw Conversation not appearing in integrations
|
||||
|
||||
**Solution**:
|
||||
1. Verify files are in `/config/custom_components/openclaw_conversation/`
|
||||
2. Check Home Assistant logs for errors
|
||||
3. Ensure `manifest.json` is valid JSON
|
||||
4. Restart Home Assistant
|
||||
|
||||
### Issue: Wyoming services not connecting
|
||||
|
||||
**Solution**:
|
||||
1. Verify services are running on Mac Mini:
|
||||
```bash
|
||||
launchctl list | grep wyoming
|
||||
nc -z 10.0.0.199 10300 # Test STT
|
||||
nc -z 10.0.0.199 10301 # Test TTS
|
||||
nc -z 10.0.0.199 10700 # Test Satellite
|
||||
```
|
||||
|
||||
2. Check firewall rules on Mac Mini
|
||||
3. Verify Home Assistant can reach Mac Mini network
|
||||
|
||||
### Issue: OpenClaw not responding
|
||||
|
||||
**Solution**:
|
||||
1. Verify OpenClaw is running:
|
||||
```bash
|
||||
launchctl list | grep openclaw
|
||||
pgrep -f openclaw
|
||||
```
|
||||
|
||||
2. Test OpenClaw CLI directly:
|
||||
```bash
|
||||
openclaw agent --message "Hello" --agent main
|
||||
```
|
||||
|
||||
3. Check OpenClaw logs:
|
||||
```bash
|
||||
tail -f /tmp/homeai-openclaw.log
|
||||
```
|
||||
|
||||
4. Verify OpenClaw can reach Home Assistant:
|
||||
```bash
|
||||
curl http://10.0.0.199:8123/api/
|
||||
```
|
||||
|
||||
### Issue: No audio output from satellite
|
||||
|
||||
**Solution**:
|
||||
1. Check satellite logs:
|
||||
```bash
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
```
|
||||
|
||||
2. Test audio output:
|
||||
```bash
|
||||
afplay /System/Library/Sounds/Glass.aiff
|
||||
```
|
||||
|
||||
3. Verify SoX is installed:
|
||||
```bash
|
||||
which play
|
||||
brew install sox
|
||||
```
|
||||
|
||||
### Issue: Wake word not detected
|
||||
|
||||
**Solution**:
|
||||
1. Check wakeword service:
|
||||
```bash
|
||||
launchctl list | grep wakeword
|
||||
```
|
||||
|
||||
2. Test microphone input:
|
||||
```bash
|
||||
# Record a test
|
||||
rec -r 16000 -c 1 test.wav trim 0 5
|
||||
```
|
||||
|
||||
3. Adjust wake word threshold in satellite config
|
||||
|
||||
---
|
||||
|
||||
## Voice Pipeline Flow
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ USB Mic │
|
||||
│ (Mac Mini) │
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Wake Word │
|
||||
│ Detection │
|
||||
│ (hey_jarvis) │
|
||||
└────────┬────────┘
|
||||
│ wake detected
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Wyoming │
|
||||
│ Satellite │
|
||||
│ :10700 │
|
||||
└────────┬────────┘
|
||||
│ audio stream
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Wyoming STT │
|
||||
│ (Whisper) │
|
||||
│ :10300 │
|
||||
└────────┬────────┘
|
||||
│ transcript
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Home Assistant │
|
||||
│ Voice Pipeline │
|
||||
└────────┬────────┘
|
||||
│ text
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ OpenClaw │
|
||||
│ Conversation │
|
||||
│ Agent │
|
||||
└────────┬────────┘
|
||||
│ message
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ OpenClaw │
|
||||
│ Gateway │
|
||||
│ :8080 │
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Ollama LLM │
|
||||
│ + Skills │
|
||||
│ :11434 │
|
||||
└────────┬────────┘
|
||||
│ response
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Wyoming TTS │
|
||||
│ (Kokoro) │
|
||||
│ :10301 │
|
||||
└────────┬────────┘
|
||||
│ audio
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Speaker │
|
||||
│ (Mac Mini) │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps After Setup
|
||||
|
||||
1. **Install Chatterbox TTS** for voice cloning
|
||||
2. **Set up mem0** for long-term memory
|
||||
3. **Configure n8n workflows** for automation
|
||||
4. **Add Uptime Kuma monitors** for all services
|
||||
5. **Begin ESP32 satellite setup** (Phase 4)
|
||||
|
||||
---
|
||||
|
||||
## Files Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [`homeai-agent/custom_components/openclaw_conversation/`](../homeai-agent/custom_components/openclaw_conversation/) | Custom HA component |
|
||||
| [`homeai-agent/skills/home-assistant/openclaw_bridge.py`](../homeai-agent/skills/home-assistant/openclaw_bridge.py) | Bridge script |
|
||||
| [`homeai-voice/scripts/launchd/`](scripts/launchd/) | Service plists |
|
||||
| [`plans/ha-voice-pipeline-implementation.md`](../plans/ha-voice-pipeline-implementation.md) | Detailed implementation plan |
|
||||
| [`plans/voice-loop-integration.md`](../plans/voice-loop-integration.md) | Architecture options |
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Wyoming STT/TTS/Satellite appear in HA integrations
|
||||
- [ ] OpenClaw Conversation agent appears in HA integrations
|
||||
- [ ] Voice assistant pipeline created with OpenClaw
|
||||
- [ ] Typed query in Assist returns spoken response
|
||||
- [ ] Voice query via satellite returns spoken response
|
||||
- [ ] "Turn on the reading lamp" command works end-to-end
|
||||
- [ ] Latency under 5 seconds from wake to response
|
||||
- [ ] All services survive Mac Mini reboot
|
||||
195
homeai-voice/WYOMING_SATELLITE_SETUP.md
Normal file
195
homeai-voice/WYOMING_SATELLITE_SETUP.md
Normal file
@@ -0,0 +1,195 @@
|
||||
# Wyoming Satellite Setup Guide
|
||||
|
||||
> How to configure the Wyoming Satellite wizard in Home Assistant
|
||||
|
||||
---
|
||||
|
||||
## When Adding Wyoming Satellite Integration
|
||||
|
||||
When you add the Wyoming Satellite integration, Home Assistant will open a wizard to configure a voice assistant. Here's what to do:
|
||||
|
||||
---
|
||||
|
||||
## Option 1: Skip Wizard and Configure Later (Recommended)
|
||||
|
||||
**Best approach if you haven't created the OpenClaw pipeline yet:**
|
||||
|
||||
1. **Skip/Cancel the wizard** - just add the satellite integration without configuring the pipeline
|
||||
2. The satellite will be added but not assigned to a pipeline yet
|
||||
3. Continue with creating the voice assistant pipeline (see below)
|
||||
4. Come back and assign the satellite to the pipeline later
|
||||
|
||||
---
|
||||
|
||||
## Option 2: Use Default Pipeline Temporarily
|
||||
|
||||
**If you want to test the satellite immediately:**
|
||||
|
||||
1. In the wizard, select **"Home Assistant"** as the pipeline (default)
|
||||
2. This will use HA's built-in conversation agent (not OpenClaw)
|
||||
3. You can test basic commands like "What time is it?"
|
||||
4. Later, switch to the OpenClaw pipeline once it's created
|
||||
|
||||
---
|
||||
|
||||
## Creating the Voice Assistant Pipeline
|
||||
|
||||
**Do this BEFORE configuring the satellite (or after if you used Option 2):**
|
||||
|
||||
### Step 1: Create the Pipeline
|
||||
|
||||
1. Go to **Settings → Voice Assistants**
|
||||
2. Click **Add Assistant**
|
||||
3. Configure:
|
||||
- **Name**: `HomeAI with OpenClaw`
|
||||
- **Language**: `English`
|
||||
- **Speech-to-Text**: Select `Mac Mini STT` (Wyoming)
|
||||
- **Conversation Agent**: Select `OpenClaw Conversation`
|
||||
- **Text-to-Speech**: Select `Mac Mini TTS` (Wyoming)
|
||||
4. Click **Create**
|
||||
|
||||
### Step 2: Set as Preferred (Optional)
|
||||
|
||||
1. In the Voice Assistants list, find "HomeAI with OpenClaw"
|
||||
2. Click the three dots (⋮)
|
||||
3. Select **Set as preferred**
|
||||
|
||||
This makes it the default pipeline for all new satellites.
|
||||
|
||||
---
|
||||
|
||||
## Assigning Satellite to Pipeline
|
||||
|
||||
### If You Skipped the Wizard
|
||||
|
||||
1. Go to **Settings → Devices & Services**
|
||||
2. Find **Wyoming Protocol** (the satellite entry)
|
||||
3. Click **Configure**
|
||||
4. Select **Pipeline**: `HomeAI with OpenClaw`
|
||||
5. Click **Submit**
|
||||
|
||||
### If You Used the Default Pipeline
|
||||
|
||||
1. Go to **Settings → Devices & Services**
|
||||
2. Find **Wyoming Protocol** (the satellite entry)
|
||||
3. Click **Configure**
|
||||
4. Change **Pipeline** from "Home Assistant" to `HomeAI with OpenClaw`
|
||||
5. Click **Submit**
|
||||
|
||||
---
|
||||
|
||||
## Satellite Configuration Details
|
||||
|
||||
The wizard may ask for these details:
|
||||
|
||||
| Field | Value | Notes |
|
||||
|-------|-------|-------|
|
||||
| **Name** | `Mac Mini Living Room` | Or any name you prefer |
|
||||
| **Pipeline** | `HomeAI with OpenClaw` | Select after creating it |
|
||||
| **Wake Word** | `hey_jarvis` | Should be auto-detected |
|
||||
| **Audio Input** | Default | Detected from satellite |
|
||||
| **Audio Output** | Default | Detected from satellite |
|
||||
|
||||
---
|
||||
|
||||
## Complete Voice Pipeline Flow
|
||||
|
||||
Once configured, the flow will be:
|
||||
|
||||
```
|
||||
1. Say "Hey Jarvis" → Wake word detected by satellite
|
||||
2. Satellite captures audio → Sends to Wyoming STT (10.0.0.101:10300)
|
||||
3. STT transcribes → Sends text to HA Voice Pipeline
|
||||
4. HA routes to OpenClaw Conversation agent
|
||||
5. OpenClaw processes → Calls Ollama LLM + skills
|
||||
6. Response generated → Sent to Wyoming TTS (10.0.0.101:10301)
|
||||
7. TTS generates audio → Sent back to satellite
|
||||
8. Satellite plays audio → You hear the response
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing the Pipeline
|
||||
|
||||
### Test 1: Via HA Assist (No Wake Word)
|
||||
|
||||
1. Open Home Assistant UI
|
||||
2. Click the **Assist** icon (microphone) in top-right
|
||||
3. Type: `"What time is it?"`
|
||||
4. Press Enter
|
||||
5. **Expected**: You should hear a spoken response via TTS
|
||||
|
||||
### Test 2: Via Satellite (With Wake Word)
|
||||
|
||||
1. Say: **"Hey Jarvis"**
|
||||
2. Wait for acknowledgment beep
|
||||
3. Say: **"What time is it?"**
|
||||
4. **Expected**: You should hear a spoken response
|
||||
|
||||
### Test 3: Home Control
|
||||
|
||||
1. Say: **"Hey Jarvis"**
|
||||
2. Say: **"Turn on the reading lamp"**
|
||||
3. **Expected**:
|
||||
- Light turns on
|
||||
- You hear confirmation: "I've turned on the reading lamp"
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Satellite Not Responding
|
||||
|
||||
1. **Check satellite is online**:
|
||||
- Settings → Devices & Services → Wyoming Protocol
|
||||
- Should show "Connected"
|
||||
|
||||
2. **Check pipeline is assigned**:
|
||||
- Configure satellite → Verify pipeline is set
|
||||
|
||||
3. **Check satellite logs** on Mac Mini:
|
||||
```bash
|
||||
tail -f /tmp/homeai-wyoming-satellite.log
|
||||
```
|
||||
|
||||
### Wake Word Not Detected
|
||||
|
||||
1. **Check microphone**:
|
||||
- Satellite logs should show audio input
|
||||
- Try speaking louder or closer to mic
|
||||
|
||||
2. **Adjust wake word sensitivity**:
|
||||
- May need to configure threshold in satellite settings
|
||||
|
||||
### No Audio Output
|
||||
|
||||
1. **Check speaker**:
|
||||
```bash
|
||||
afplay /System/Library/Sounds/Glass.aiff
|
||||
```
|
||||
|
||||
2. **Check TTS is working**:
|
||||
- Test via HA Assist (type query)
|
||||
- Should hear response
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
**Recommended Setup Order:**
|
||||
|
||||
1. ✅ Add Wyoming STT integration (10.0.0.101:10300)
|
||||
2. ✅ Add Wyoming TTS integration (10.0.0.101:10301)
|
||||
3. ✅ Add OpenClaw Conversation integration (10.0.0.101:8080)
|
||||
4. ✅ Create voice assistant pipeline "HomeAI with OpenClaw"
|
||||
5. ✅ Add Wyoming Satellite integration (10.0.0.101:10700)
|
||||
6. ✅ Assign satellite to "HomeAI with OpenClaw" pipeline
|
||||
7. ✅ Test the complete voice loop
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [`VOICE_PIPELINE_SETUP.md`](VOICE_PIPELINE_SETUP.md) - Complete setup guide
|
||||
- [`TROUBLESHOOTING.md`](TROUBLESHOOTING.md) - Troubleshooting guide
|
||||
- [`OPENCLAW_NETWORK_FIX.md`](OPENCLAW_NETWORK_FIX.md) - Network access fix
|
||||
381
homeai-voice/scripts/benchmark_pipeline.py
Normal file
381
homeai-voice/scripts/benchmark_pipeline.py
Normal file
@@ -0,0 +1,381 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Voice Pipeline Benchmark
|
||||
|
||||
Measures latency of each stage independently:
|
||||
1. STT (Wyoming Whisper, port 10300)
|
||||
2. LLM (Ollama API, port 11434) — multiple models
|
||||
3. TTS (Wyoming Kokoro, port 10301)
|
||||
4. End-to-end via OpenClaw HTTP Bridge (port 8081)
|
||||
|
||||
Usage:
|
||||
python benchmark_pipeline.py [--rounds 3] [--models llama3.3:70b,qwen3:32b]
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import io
|
||||
import json
|
||||
import statistics
|
||||
import sys
|
||||
import time
|
||||
import wave
|
||||
from urllib.request import Request, urlopen
|
||||
from urllib.error import URLError
|
||||
|
||||
from wyoming.audio import AudioChunk, AudioStart, AudioStop
|
||||
from wyoming.asr import Transcribe, Transcript
|
||||
from wyoming.client import AsyncTcpClient
|
||||
from wyoming.tts import Synthesize, SynthesizeVoice
|
||||
|
||||
# --- Config ---
|
||||
STT_HOST, STT_PORT = "127.0.0.1", 10300
|
||||
TTS_HOST, TTS_PORT = "127.0.0.1", 10301
|
||||
OLLAMA_URL = "http://localhost:11434"
|
||||
BRIDGE_URL = "http://localhost:8081"
|
||||
|
||||
TEST_PROMPTS = [
|
||||
"What is the capital of France?",
|
||||
"Turn on the living room lights.",
|
||||
"What's the weather like today?",
|
||||
]
|
||||
|
||||
LONG_PROMPT = "Explain in two sentences how a heat pump works."
|
||||
|
||||
|
||||
# --- Helpers ---
|
||||
|
||||
def http_post_json(url: str, data: dict, timeout: int = 180) -> tuple[dict, float]:
|
||||
"""POST JSON, return (response_dict, elapsed_seconds)."""
|
||||
body = json.dumps(data).encode()
|
||||
req = Request(url, data=body, headers={"Content-Type": "application/json"})
|
||||
t0 = time.perf_counter()
|
||||
resp = urlopen(req, timeout=timeout)
|
||||
raw = resp.read()
|
||||
elapsed = time.perf_counter() - t0
|
||||
return json.loads(raw), elapsed
|
||||
|
||||
|
||||
def http_post_raw(url: str, data: bytes, content_type: str, timeout: int = 180) -> tuple[bytes, float]:
|
||||
"""POST raw bytes, return (response_bytes, elapsed_seconds)."""
|
||||
req = Request(url, data=data, headers={"Content-Type": content_type})
|
||||
t0 = time.perf_counter()
|
||||
resp = urlopen(req, timeout=timeout)
|
||||
raw = resp.read()
|
||||
elapsed = time.perf_counter() - t0
|
||||
return raw, elapsed
|
||||
|
||||
|
||||
# --- Stage 1: TTS ---
|
||||
|
||||
async def benchmark_tts(text: str) -> tuple[bytes, float]:
|
||||
"""Synthesize text via Wyoming TTS, return (wav_bytes, elapsed)."""
|
||||
t0 = time.perf_counter()
|
||||
client = AsyncTcpClient(TTS_HOST, TTS_PORT)
|
||||
await client.connect()
|
||||
await client.read_event() # Info
|
||||
|
||||
await client.write_event(
|
||||
Synthesize(text=text, voice=SynthesizeVoice(name="af_heart")).event()
|
||||
)
|
||||
|
||||
audio_data = bytearray()
|
||||
rate, width, channels = 24000, 2, 1
|
||||
|
||||
while True:
|
||||
event = await client.read_event()
|
||||
if event is None:
|
||||
break
|
||||
if AudioStart.is_type(event.type):
|
||||
start = AudioStart.from_event(event)
|
||||
rate, width, channels = start.rate, start.width, start.channels
|
||||
elif AudioChunk.is_type(event.type):
|
||||
audio_data.extend(AudioChunk.from_event(event).audio)
|
||||
elif AudioStop.is_type(event.type):
|
||||
break
|
||||
|
||||
await client.disconnect()
|
||||
elapsed = time.perf_counter() - t0
|
||||
|
||||
# Package as WAV
|
||||
wav_io = io.BytesIO()
|
||||
with wave.open(wav_io, "wb") as wf:
|
||||
wf.setnchannels(channels)
|
||||
wf.setsampwidth(width)
|
||||
wf.setframerate(rate)
|
||||
wf.writeframes(audio_data)
|
||||
|
||||
duration_s = len(audio_data) / (rate * width * channels)
|
||||
return wav_io.getvalue(), elapsed, duration_s
|
||||
|
||||
|
||||
# --- Stage 2: STT ---
|
||||
|
||||
async def benchmark_stt(wav_bytes: bytes) -> tuple[str, float]:
|
||||
"""Transcribe WAV via Wyoming STT, return (text, elapsed)."""
|
||||
wav_io = io.BytesIO(wav_bytes)
|
||||
with wave.open(wav_io, "rb") as wf:
|
||||
rate = wf.getframerate()
|
||||
width = wf.getsampwidth()
|
||||
channels = wf.getnchannels()
|
||||
pcm = wf.readframes(wf.getnframes())
|
||||
|
||||
t0 = time.perf_counter()
|
||||
client = AsyncTcpClient(STT_HOST, STT_PORT)
|
||||
await client.connect()
|
||||
|
||||
await client.write_event(Transcribe(language="en").event())
|
||||
await client.write_event(AudioStart(rate=rate, width=width, channels=channels).event())
|
||||
|
||||
chunk_size = rate * width * channels # 1 second
|
||||
for off in range(0, len(pcm), chunk_size):
|
||||
await client.write_event(
|
||||
AudioChunk(rate=rate, width=width, channels=channels, audio=pcm[off:off + chunk_size]).event()
|
||||
)
|
||||
await client.write_event(AudioStop().event())
|
||||
|
||||
text = ""
|
||||
while True:
|
||||
event = await client.read_event()
|
||||
if event is None:
|
||||
break
|
||||
if Transcript.is_type(event.type):
|
||||
text = Transcript.from_event(event).text
|
||||
break
|
||||
|
||||
await client.disconnect()
|
||||
elapsed = time.perf_counter() - t0
|
||||
return text, elapsed
|
||||
|
||||
|
||||
# --- Stage 3: LLM ---
|
||||
|
||||
def benchmark_llm(model: str, prompt: str, warm: bool = False) -> dict:
|
||||
"""
|
||||
Call Ollama /api/generate, return timing breakdown.
|
||||
If warm=True, we assume the model is already loaded.
|
||||
"""
|
||||
data = {
|
||||
"model": model,
|
||||
"prompt": prompt,
|
||||
"stream": False,
|
||||
"options": {"num_ctx": 2048}, # small ctx for benchmark speed
|
||||
}
|
||||
body = json.dumps(data).encode()
|
||||
req = Request(
|
||||
f"{OLLAMA_URL}/api/generate",
|
||||
data=body,
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
|
||||
t0 = time.perf_counter()
|
||||
resp = urlopen(req, timeout=300)
|
||||
raw = resp.read()
|
||||
wall_time = time.perf_counter() - t0
|
||||
|
||||
result = json.loads(raw)
|
||||
|
||||
# Ollama returns timing in nanoseconds
|
||||
load_ns = result.get("load_duration", 0)
|
||||
prompt_ns = result.get("prompt_eval_duration", 0)
|
||||
eval_ns = result.get("eval_duration", 0)
|
||||
total_ns = result.get("total_duration", 0)
|
||||
prompt_tokens = result.get("prompt_eval_count", 0)
|
||||
eval_tokens = result.get("eval_count", 0)
|
||||
|
||||
return {
|
||||
"model": model,
|
||||
"wall_time_s": wall_time,
|
||||
"load_s": load_ns / 1e9,
|
||||
"prompt_eval_s": prompt_ns / 1e9,
|
||||
"eval_s": eval_ns / 1e9,
|
||||
"total_s": total_ns / 1e9,
|
||||
"prompt_tokens": prompt_tokens,
|
||||
"eval_tokens": eval_tokens,
|
||||
"tokens_per_sec": eval_tokens / (eval_ns / 1e9) if eval_ns > 0 else 0,
|
||||
"prompt_tokens_per_sec": prompt_tokens / (prompt_ns / 1e9) if prompt_ns > 0 else 0,
|
||||
"response": result.get("response", "")[:200],
|
||||
}
|
||||
|
||||
|
||||
def warm_model(model: str):
|
||||
"""Send a tiny request to load the model into GPU memory."""
|
||||
print(f" Warming up {model}...", end=" ", flush=True)
|
||||
try:
|
||||
data = json.dumps({"model": model, "prompt": "hi", "stream": False, "options": {"num_ctx": 512}}).encode()
|
||||
req = Request(f"{OLLAMA_URL}/api/generate", data=data, headers={"Content-Type": "application/json"})
|
||||
urlopen(req, timeout=300).read()
|
||||
print("ready")
|
||||
except Exception as e:
|
||||
print(f"warning: {e}")
|
||||
|
||||
|
||||
# --- Stage 4: End-to-end via bridge ---
|
||||
|
||||
def benchmark_e2e(message: str) -> tuple[str, float]:
|
||||
"""Call the OpenClaw HTTP bridge end-to-end."""
|
||||
data = {"message": message, "agent": "main"}
|
||||
resp, elapsed = http_post_json(f"{BRIDGE_URL}/api/agent/message", data, timeout=300)
|
||||
return resp.get("response", ""), elapsed
|
||||
|
||||
|
||||
# --- Formatting ---
|
||||
|
||||
def fmt_time(seconds: float) -> str:
|
||||
if seconds < 1:
|
||||
return f"{seconds*1000:.0f}ms"
|
||||
return f"{seconds:.1f}s"
|
||||
|
||||
|
||||
def print_table(rows: list[dict], columns: list[tuple[str, str, int]]):
|
||||
"""Print a formatted table. columns = [(header, key, width), ...]"""
|
||||
header = " | ".join(h.ljust(w) for h, _, w in columns)
|
||||
print(header)
|
||||
print("-" * len(header))
|
||||
for row in rows:
|
||||
line = " | ".join(str(row.get(k, "")).ljust(w) for _, k, w in columns)
|
||||
print(line)
|
||||
|
||||
|
||||
# --- Main ---
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Voice Pipeline Benchmark")
|
||||
parser.add_argument("--rounds", type=int, default=2, help="Rounds per test (default: 2)")
|
||||
parser.add_argument(
|
||||
"--models",
|
||||
default="qwen2.5:7b,qwen3:32b,llama3.3:70b",
|
||||
help="Comma-separated Ollama models to test",
|
||||
)
|
||||
parser.add_argument("--skip-stt", action="store_true", help="Skip STT benchmark")
|
||||
parser.add_argument("--skip-tts", action="store_true", help="Skip TTS benchmark")
|
||||
parser.add_argument("--skip-llm", action="store_true", help="Skip LLM benchmark")
|
||||
parser.add_argument("--skip-e2e", action="store_true", help="Skip end-to-end benchmark")
|
||||
parser.add_argument("--prompt", default=None, help="Custom prompt for LLM benchmark")
|
||||
args = parser.parse_args()
|
||||
|
||||
models = [m.strip() for m in args.models.split(",")]
|
||||
llm_prompt = args.prompt or LONG_PROMPT
|
||||
|
||||
print("=" * 70)
|
||||
print(" VOICE PIPELINE BENCHMARK")
|
||||
print("=" * 70)
|
||||
print(f" Rounds: {args.rounds}")
|
||||
print(f" Models: {', '.join(models)}")
|
||||
print(f" LLM prompt: {llm_prompt!r}")
|
||||
print()
|
||||
|
||||
# ── TTS Benchmark ──
|
||||
test_wav = None
|
||||
if not args.skip_tts:
|
||||
print("── TTS (Kokoro, Wyoming port 10301) ──")
|
||||
tts_times = []
|
||||
tts_durations = []
|
||||
for i in range(args.rounds):
|
||||
text = TEST_PROMPTS[i % len(TEST_PROMPTS)]
|
||||
wav, elapsed, audio_dur = asyncio.run(benchmark_tts(text))
|
||||
tts_times.append(elapsed)
|
||||
tts_durations.append(audio_dur)
|
||||
test_wav = wav
|
||||
print(f" Round {i+1}: {fmt_time(elapsed)} → {audio_dur:.1f}s audio (RTF: {elapsed/audio_dur:.2f}x) text={text!r}")
|
||||
|
||||
avg_tts = statistics.mean(tts_times)
|
||||
avg_dur = statistics.mean(tts_durations)
|
||||
print(f" Average: {fmt_time(avg_tts)} for {avg_dur:.1f}s audio (RTF: {avg_tts/avg_dur:.2f}x)")
|
||||
print()
|
||||
|
||||
# ── STT Benchmark ──
|
||||
if not args.skip_stt:
|
||||
print("── STT (Whisper large-v3, Wyoming port 10300) ──")
|
||||
if test_wav is None:
|
||||
# Generate a test WAV first
|
||||
print(" Generating test audio via TTS...")
|
||||
test_wav, _, _ = asyncio.run(benchmark_tts("The quick brown fox jumps over the lazy dog."))
|
||||
|
||||
stt_times = []
|
||||
for i in range(args.rounds):
|
||||
text, elapsed = asyncio.run(benchmark_stt(test_wav))
|
||||
stt_times.append(elapsed)
|
||||
print(f" Round {i+1}: {fmt_time(elapsed)} → {text!r}")
|
||||
|
||||
print(f" Average: {fmt_time(statistics.mean(stt_times))}")
|
||||
print()
|
||||
|
||||
# ── LLM Benchmark ──
|
||||
if not args.skip_llm:
|
||||
print("── LLM (Ollama) ──")
|
||||
print(f" Prompt: {llm_prompt!r}")
|
||||
print()
|
||||
|
||||
all_results = []
|
||||
for model in models:
|
||||
print(f" Model: {model}")
|
||||
warm_model(model)
|
||||
|
||||
model_runs = []
|
||||
for i in range(args.rounds):
|
||||
result = benchmark_llm(model, llm_prompt, warm=True)
|
||||
model_runs.append(result)
|
||||
print(
|
||||
f" Round {i+1}: wall={fmt_time(result['wall_time_s'])} "
|
||||
f"load={fmt_time(result['load_s'])} "
|
||||
f"prompt_eval={fmt_time(result['prompt_eval_s'])} ({result['prompt_tokens']}tok, {result['prompt_tokens_per_sec']:.0f}t/s) "
|
||||
f"gen={fmt_time(result['eval_s'])} ({result['eval_tokens']}tok, {result['tokens_per_sec']:.1f}t/s)"
|
||||
)
|
||||
# Truncate response for display
|
||||
resp_preview = result["response"][:100].replace("\n", " ")
|
||||
print(f" → {resp_preview}")
|
||||
|
||||
# Summarize
|
||||
avg_wall = statistics.mean(r["wall_time_s"] for r in model_runs)
|
||||
avg_tps = statistics.mean(r["tokens_per_sec"] for r in model_runs)
|
||||
avg_prompt_tps = statistics.mean(r["prompt_tokens_per_sec"] for r in model_runs)
|
||||
avg_tokens = statistics.mean(r["eval_tokens"] for r in model_runs)
|
||||
all_results.append({
|
||||
"model": model,
|
||||
"avg_wall": fmt_time(avg_wall),
|
||||
"avg_gen_tps": f"{avg_tps:.1f}",
|
||||
"avg_prompt_tps": f"{avg_prompt_tps:.0f}",
|
||||
"avg_tokens": f"{avg_tokens:.0f}",
|
||||
})
|
||||
print()
|
||||
|
||||
# Summary table
|
||||
print(" ┌─ LLM Summary ─────────────────────────────────────────────┐")
|
||||
print(f" {'Model':<25s} {'Wall time':>10s} {'Gen t/s':>10s} {'Prompt t/s':>11s} {'Avg tokens':>11s}")
|
||||
print(f" {'─'*25} {'─'*10} {'─'*10} {'─'*11} {'─'*11}")
|
||||
for r in all_results:
|
||||
print(f" {r['model']:<25s} {r['avg_wall']:>10s} {r['avg_gen_tps']:>10s} {r['avg_prompt_tps']:>11s} {r['avg_tokens']:>11s}")
|
||||
print()
|
||||
|
||||
# ── End-to-end ──
|
||||
if not args.skip_e2e:
|
||||
print("── End-to-End (Bridge → OpenClaw → Ollama → response) ──")
|
||||
print(" (Does not include STT/TTS — just text in → text out via bridge)")
|
||||
e2e_prompt = "What time is it?"
|
||||
for i in range(args.rounds):
|
||||
try:
|
||||
resp, elapsed = benchmark_e2e(e2e_prompt)
|
||||
preview = resp[:100].replace("\n", " ")
|
||||
print(f" Round {i+1}: {fmt_time(elapsed)} → {preview}")
|
||||
except Exception as e:
|
||||
print(f" Round {i+1}: ERROR - {e}")
|
||||
print()
|
||||
|
||||
# ── Pipeline estimate ──
|
||||
print("=" * 70)
|
||||
print(" ESTIMATED PIPELINE LATENCY (per voice interaction)")
|
||||
print("=" * 70)
|
||||
print(" wake word detection ~instant (runs locally)")
|
||||
print(" + STT (Whisper) see above")
|
||||
print(" + LLM (inference) see above (dominant cost)")
|
||||
print(" + TTS (Kokoro) see above")
|
||||
print(" ─────────────────────────────────────")
|
||||
print(" Tip: smaller models (7B, 32B) dramatically reduce LLM latency.")
|
||||
print(" The 70B model at ~12 tok/s needs ~5-8s for a typical reply.")
|
||||
print(" A 7B model at ~80 tok/s would need <1s for the same reply.")
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -13,7 +13,7 @@
|
||||
<string>--wake-word</string>
|
||||
<string>hey_jarvis</string>
|
||||
<string>--notify-url</string>
|
||||
<string>http://localhost:8080/wake</string>
|
||||
<string>http://localhost:8081/wake</string>
|
||||
</array>
|
||||
|
||||
<key>RunAtLoad</key>
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>com.homeai.wyoming-elevenlabs</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/Users/aodhan/homeai-voice-env/bin/python3</string>
|
||||
<string>/Users/aodhan/gitea/homeai/homeai-voice/tts/wyoming_elevenlabs_server.py</string>
|
||||
<string>--uri</string>
|
||||
<string>tcp://0.0.0.0:10302</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
<key>KeepAlive</key>
|
||||
<true/>
|
||||
<key>StandardOutPath</key>
|
||||
<string>/tmp/homeai-wyoming-elevenlabs.log</string>
|
||||
<key>StandardErrorPath</key>
|
||||
<string>/tmp/homeai-wyoming-elevenlabs.log</string>
|
||||
<key>EnvironmentVariables</key>
|
||||
<dict>
|
||||
<key>PATH</key>
|
||||
<string>/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
|
||||
</dict>
|
||||
</dict>
|
||||
</plist>
|
||||
@@ -18,9 +18,9 @@
|
||||
<string>--area</string>
|
||||
<string>Living Room</string>
|
||||
<string>--mic-command</string>
|
||||
<string>rec -q -r 16000 -c 1 -b 16 -t raw -</string>
|
||||
<string>/opt/homebrew/bin/rec -q -r 16000 -c 1 -b 16 -t raw -</string>
|
||||
<string>--snd-command</string>
|
||||
<string>play -q -r 24000 -c 1 -b 16 -t raw -</string>
|
||||
<string>/opt/homebrew/bin/play -q -t raw -r 24000 -c 1 -b 16 -e signed-integer -</string>
|
||||
<string>--mic-command-rate</string>
|
||||
<string>16000</string>
|
||||
<string>--mic-command-width</string>
|
||||
@@ -33,10 +33,18 @@
|
||||
<string>2</string>
|
||||
<string>--snd-command-channels</string>
|
||||
<string>1</string>
|
||||
<string>--wake-command</string>
|
||||
<string>/Users/aodhan/homeai-voice-env/bin/python3 /Users/aodhan/gitea/homeai/homeai-voice/wyoming/wakeword_command.py --wake-word hey_jarvis --threshold 0.5</string>
|
||||
<string>--wake-command-rate</string>
|
||||
<string>16000</string>
|
||||
<string>--wake-command-width</string>
|
||||
<string>2</string>
|
||||
<string>--wake-command-channels</string>
|
||||
<string>1</string>
|
||||
<string>--awake-wav</string>
|
||||
<string>/System/Library/Sounds/Glass.aiff</string>
|
||||
<string>/Users/aodhan/homeai-data/sounds/awake.wav</string>
|
||||
<string>--done-wav</string>
|
||||
<string>/System/Library/Sounds/Blow.aiff</string>
|
||||
<string>/Users/aodhan/homeai-data/sounds/done.wav</string>
|
||||
<string>--no-zeroconf</string>
|
||||
</array>
|
||||
|
||||
|
||||
@@ -8,21 +8,11 @@
|
||||
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/Users/aodhan/homeai-voice-env/bin/wyoming-faster-whisper</string>
|
||||
<string>/Users/aodhan/homeai-whisper-mlx-env/bin/wyoming-mlx-whisper</string>
|
||||
<string>--uri</string>
|
||||
<string>tcp://0.0.0.0:10300</string>
|
||||
<string>--model</string>
|
||||
<string>large-v3</string>
|
||||
<string>--language</string>
|
||||
<string>en</string>
|
||||
<string>--device</string>
|
||||
<string>cpu</string>
|
||||
<string>--compute-type</string>
|
||||
<string>int8</string>
|
||||
<string>--data-dir</string>
|
||||
<string>/Users/aodhan/models/whisper</string>
|
||||
<string>--download-dir</string>
|
||||
<string>/Users/aodhan/models/whisper</string>
|
||||
</array>
|
||||
|
||||
<key>RunAtLoad</key>
|
||||
|
||||
10
homeai-voice/scripts/monitor-wakeword.sh
Normal file
10
homeai-voice/scripts/monitor-wakeword.sh
Normal file
@@ -0,0 +1,10 @@
|
||||
#!/bin/bash
|
||||
# Monitor wake word detection in real-time
|
||||
|
||||
echo "Monitoring wake word detection..."
|
||||
echo "Say 'Hey Jarvis' to test"
|
||||
echo "Press Ctrl+C to stop"
|
||||
echo ""
|
||||
|
||||
# Watch both the wake word log and bridge log
|
||||
tail -f /tmp/homeai-wakeword-error.log /tmp/homeai-openclaw-bridge.log 2>/dev/null | grep -E "(Wake word detected|Listening|Failed to notify)"
|
||||
140
homeai-voice/scripts/test-services.sh
Executable file
140
homeai-voice/scripts/test-services.sh
Executable file
@@ -0,0 +1,140 @@
|
||||
#!/usr/bin/env bash
|
||||
# Test all voice pipeline services are running and accessible
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
echo "Testing Voice Pipeline Services..."
|
||||
echo "=================================="
|
||||
echo ""
|
||||
|
||||
# Colors for output
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Test function
|
||||
test_service() {
|
||||
local name=$1
|
||||
local host=$2
|
||||
local port=$3
|
||||
|
||||
if nc -z -w 2 "$host" "$port" 2>/dev/null; then
|
||||
echo -e "${GREEN}✓${NC} $name ($host:$port)"
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} $name ($host:$port) - NOT ACCESSIBLE"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Test launchd service
|
||||
test_launchd() {
|
||||
local name=$1
|
||||
local service=$2
|
||||
|
||||
if launchctl list | grep -q "$service"; then
|
||||
echo -e "${GREEN}✓${NC} $name (launchd: $service)"
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} $name (launchd: $service) - NOT RUNNING"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Test command availability
|
||||
test_command() {
|
||||
local name=$1
|
||||
local cmd=$2
|
||||
|
||||
if command -v "$cmd" &> /dev/null; then
|
||||
echo -e "${GREEN}✓${NC} $name command available"
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} $name command NOT FOUND"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
echo "1. Network Services"
|
||||
echo "-------------------"
|
||||
test_service "Wyoming STT" "localhost" "10300"
|
||||
test_service "Wyoming TTS" "localhost" "10301"
|
||||
test_service "Wyoming Satellite" "localhost" "10700"
|
||||
test_service "OpenClaw Gateway" "localhost" "8080"
|
||||
test_service "Ollama" "localhost" "11434"
|
||||
test_service "Home Assistant" "10.0.0.199" "8123"
|
||||
echo ""
|
||||
|
||||
echo "2. Launchd Services"
|
||||
echo "-------------------"
|
||||
test_launchd "Wyoming STT" "com.homeai.wyoming-stt"
|
||||
test_launchd "Wyoming TTS" "com.homeai.wyoming-tts"
|
||||
test_launchd "Wyoming Satellite" "com.homeai.wyoming-satellite"
|
||||
test_launchd "Wake Word" "com.homeai.wakeword"
|
||||
test_launchd "OpenClaw" "com.homeai.openclaw"
|
||||
test_launchd "Ollama" "com.homeai.ollama"
|
||||
echo ""
|
||||
|
||||
echo "3. Commands"
|
||||
echo "-----------"
|
||||
test_command "OpenClaw" "openclaw"
|
||||
test_command "Ollama" "ollama"
|
||||
test_command "SoX (play)" "play"
|
||||
test_command "SoX (rec)" "rec"
|
||||
echo ""
|
||||
|
||||
echo "4. Wyoming Protocol Test"
|
||||
echo "------------------------"
|
||||
if command -v wyoming-client &> /dev/null; then
|
||||
echo -e "${YELLOW}Testing STT...${NC}"
|
||||
# Would need a test audio file
|
||||
echo " (Manual test required with audio file)"
|
||||
|
||||
echo -e "${YELLOW}Testing TTS...${NC}"
|
||||
# Would need Wyoming client
|
||||
echo " (Manual test required with Wyoming client)"
|
||||
else
|
||||
echo -e "${YELLOW}⚠${NC} wyoming-client not installed (optional)"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
echo "5. OpenClaw Test"
|
||||
echo "----------------"
|
||||
if command -v openclaw &> /dev/null; then
|
||||
echo -e "${YELLOW}Testing OpenClaw agent...${NC}"
|
||||
if timeout 10 openclaw agent --message "Hello" --agent main &>/dev/null; then
|
||||
echo -e "${GREEN}✓${NC} OpenClaw agent responding"
|
||||
else
|
||||
echo -e "${RED}✗${NC} OpenClaw agent not responding"
|
||||
fi
|
||||
else
|
||||
echo -e "${RED}✗${NC} OpenClaw command not found"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
echo "6. Audio Devices"
|
||||
echo "----------------"
|
||||
if command -v rec &> /dev/null; then
|
||||
echo "Input devices:"
|
||||
rec -n stat trim 0 0.1 2>&1 | grep -i "input" || echo " (Unable to detect)"
|
||||
|
||||
echo "Output devices:"
|
||||
if command -v afplay &> /dev/null; then
|
||||
echo -e "${GREEN}✓${NC} afplay available for audio output"
|
||||
else
|
||||
echo -e "${RED}✗${NC} afplay not available"
|
||||
fi
|
||||
else
|
||||
echo -e "${YELLOW}⚠${NC} SoX not installed - audio recording unavailable"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
echo "=================================="
|
||||
echo "Test complete!"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo "1. Install OpenClaw conversation component in Home Assistant"
|
||||
echo "2. Configure Wyoming integrations in HA UI"
|
||||
echo "3. Create voice assistant pipeline"
|
||||
echo "4. Test with: 'Hey Jarvis, what time is it?'"
|
||||
186
homeai-voice/tts/wyoming_elevenlabs_server.py
Normal file
186
homeai-voice/tts/wyoming_elevenlabs_server.py
Normal file
@@ -0,0 +1,186 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Wyoming TTS server backed by ElevenLabs.
|
||||
|
||||
Usage:
|
||||
python wyoming_elevenlabs_server.py --uri tcp://0.0.0.0:10302 --voice-id 21m00Tcm4TlvDq8ikWAM
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import wave
|
||||
import io
|
||||
from urllib import request, error
|
||||
|
||||
from wyoming.audio import AudioChunk, AudioStart, AudioStop
|
||||
from wyoming.event import Event
|
||||
from wyoming.info import Attribution, Info, TtsProgram, TtsVoice, TtsVoiceSpeaker
|
||||
from wyoming.server import AsyncEventHandler, AsyncServer
|
||||
from wyoming.tts import Synthesize
|
||||
|
||||
_LOGGER = logging.getLogger(__name__)
|
||||
|
||||
SAMPLE_RATE = 24000
|
||||
SAMPLE_WIDTH = 2 # int16
|
||||
CHANNELS = 1
|
||||
CHUNK_SECONDS = 1 # stream in 1-second chunks
|
||||
|
||||
|
||||
class ElevenLabsEventHandler(AsyncEventHandler):
|
||||
def __init__(self, default_voice_id: str, default_model: str, api_key: str, speed: float, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
self._default_voice_id = default_voice_id
|
||||
self._default_model = default_model
|
||||
self._api_key = api_key
|
||||
self._speed = speed
|
||||
|
||||
# Send info immediately on connect
|
||||
asyncio.ensure_future(self._send_info())
|
||||
|
||||
async def _send_info(self):
|
||||
info = Info(
|
||||
tts=[
|
||||
TtsProgram(
|
||||
name="elevenlabs",
|
||||
description="ElevenLabs API TTS",
|
||||
attribution=Attribution(
|
||||
name="ElevenLabs",
|
||||
url="https://elevenlabs.io/",
|
||||
),
|
||||
installed=True,
|
||||
version="1.0.0",
|
||||
voices=[
|
||||
TtsVoice(
|
||||
name=self._default_voice_id,
|
||||
description="ElevenLabs Voice",
|
||||
attribution=Attribution(name="elevenlabs", url=""),
|
||||
installed=True,
|
||||
languages=["en-us"],
|
||||
version="1.0",
|
||||
speakers=[TtsVoiceSpeaker(name=self._default_voice_id)],
|
||||
)
|
||||
],
|
||||
)
|
||||
]
|
||||
)
|
||||
await self.write_event(info.event())
|
||||
|
||||
async def handle_event(self, event: Event) -> bool:
|
||||
if Synthesize.is_type(event.type):
|
||||
synthesize = Synthesize.from_event(event)
|
||||
text = synthesize.text
|
||||
voice_id = self._default_voice_id
|
||||
|
||||
if synthesize.voice and synthesize.voice.name:
|
||||
voice_id = synthesize.voice.name
|
||||
|
||||
_LOGGER.debug("Synthesizing %r with voice_id=%s model=%s", text, voice_id, self._default_model)
|
||||
|
||||
try:
|
||||
loop = asyncio.get_event_loop()
|
||||
audio_bytes = await loop.run_in_executor(
|
||||
None, lambda: self._call_elevenlabs_api(text, voice_id)
|
||||
)
|
||||
|
||||
if audio_bytes is None:
|
||||
raise Exception("Failed to generate audio from ElevenLabs")
|
||||
|
||||
await self.write_event(
|
||||
AudioStart(rate=SAMPLE_RATE, width=SAMPLE_WIDTH, channels=CHANNELS).event()
|
||||
)
|
||||
|
||||
chunk_size = SAMPLE_RATE * SAMPLE_WIDTH * CHANNELS * CHUNK_SECONDS
|
||||
for i in range(0, len(audio_bytes), chunk_size):
|
||||
await self.write_event(
|
||||
AudioChunk(
|
||||
rate=SAMPLE_RATE,
|
||||
width=SAMPLE_WIDTH,
|
||||
channels=CHANNELS,
|
||||
audio=audio_bytes[i : i + chunk_size],
|
||||
).event()
|
||||
)
|
||||
|
||||
await self.write_event(AudioStop().event())
|
||||
_LOGGER.info("Synthesized audio completed")
|
||||
|
||||
except Exception:
|
||||
_LOGGER.exception("Synthesis error")
|
||||
await self.write_event(AudioStop().event())
|
||||
|
||||
return True # keep connection open
|
||||
|
||||
def _call_elevenlabs_api(self, text: str, voice_id: str) -> bytes:
|
||||
import json
|
||||
url = f"https://api.elevenlabs.io/v1/text-to-speech/{voice_id}?output_format=pcm_24000"
|
||||
|
||||
headers = {
|
||||
"Accept": "audio/pcm",
|
||||
"Content-Type": "application/json",
|
||||
"xi-api-key": self._api_key
|
||||
}
|
||||
|
||||
data = {
|
||||
"text": text,
|
||||
"model_id": self._default_model,
|
||||
}
|
||||
|
||||
req = request.Request(url, data=json.dumps(data).encode('utf-8'), headers=headers, method='POST')
|
||||
try:
|
||||
with request.urlopen(req) as response:
|
||||
if response.status == 200:
|
||||
return response.read()
|
||||
else:
|
||||
_LOGGER.error(f"ElevenLabs API Error: {response.status}")
|
||||
return None
|
||||
except error.HTTPError as e:
|
||||
_LOGGER.error(f"ElevenLabs HTTP Error: {e.code} - {e.read().decode('utf-8')}")
|
||||
return None
|
||||
except Exception as e:
|
||||
_LOGGER.error(f"ElevenLabs Request Error: {str(e)}")
|
||||
return None
|
||||
|
||||
|
||||
async def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--uri", default="tcp://0.0.0.0:10302")
|
||||
parser.add_argument("--voice-id", default="21m00Tcm4TlvDq8ikWAM", help="Default ElevenLabs Voice ID")
|
||||
parser.add_argument("--model", default="eleven_monolingual_v1", help="ElevenLabs Model ID")
|
||||
parser.add_argument("--speed", type=float, default=1.0)
|
||||
parser.add_argument("--debug", action="store_true")
|
||||
args = parser.parse_args()
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.DEBUG if args.debug else logging.INFO,
|
||||
format="%(asctime)s %(levelname)s %(name)s %(message)s",
|
||||
)
|
||||
|
||||
api_key = os.environ.get("ELEVENLABS_API_KEY")
|
||||
if not api_key:
|
||||
# Try to read from .env file directly if not exported in shell
|
||||
try:
|
||||
env_path = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), '.env')
|
||||
if os.path.exists(env_path):
|
||||
with open(env_path, 'r') as f:
|
||||
for line in f:
|
||||
if line.startswith('ELEVENLABS_API_KEY='):
|
||||
api_key = line.split('=', 1)[1].strip()
|
||||
break
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if not api_key:
|
||||
_LOGGER.warning("ELEVENLABS_API_KEY environment variable not set. API calls will fail.")
|
||||
|
||||
_LOGGER.info("Starting ElevenLabs Wyoming TTS on %s (voice-id=%s, model=%s)", args.uri, args.voice_id, args.model)
|
||||
|
||||
server = AsyncServer.from_uri(args.uri)
|
||||
|
||||
def handler_factory(reader, writer):
|
||||
return ElevenLabsEventHandler(args.voice_id, args.model, api_key, args.speed, reader, writer)
|
||||
|
||||
await server.run(handler_factory)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -63,7 +63,6 @@ class KokoroEventHandler(AsyncEventHandler):
|
||||
attribution=Attribution(name="kokoro", url=""),
|
||||
installed=True,
|
||||
languages=["en-us"],
|
||||
version="1.0",
|
||||
speakers=[TtsVoiceSpeaker(name=self._default_voice)],
|
||||
)
|
||||
],
|
||||
|
||||
77
homeai-voice/wyoming/wakeword_command.py
Normal file
77
homeai-voice/wyoming/wakeword_command.py
Normal file
@@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Wake word detection command for Wyoming Satellite.
|
||||
|
||||
The satellite feeds raw 16kHz 16-bit mono audio via stdin.
|
||||
This script reads that audio, runs openWakeWord, and prints
|
||||
the wake word name to stdout when detected.
|
||||
|
||||
Usage (called by wyoming-satellite --wake-command):
|
||||
python wakeword_command.py [--wake-word hey_jarvis] [--threshold 0.5]
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
import numpy as np
|
||||
import logging
|
||||
|
||||
_LOGGER = logging.getLogger(__name__)
|
||||
|
||||
SAMPLE_RATE = 16000
|
||||
CHUNK_SIZE = 1280 # ~80ms at 16kHz — recommended by openWakeWord
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--wake-word", default="hey_jarvis")
|
||||
parser.add_argument("--threshold", type=float, default=0.5)
|
||||
parser.add_argument("--cooldown", type=float, default=3.0)
|
||||
parser.add_argument("--debug", action="store_true")
|
||||
args = parser.parse_args()
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.DEBUG if args.debug else logging.WARNING,
|
||||
format="%(asctime)s %(levelname)s %(message)s",
|
||||
stream=sys.stderr,
|
||||
)
|
||||
|
||||
import openwakeword
|
||||
from openwakeword.model import Model
|
||||
|
||||
oww = Model(
|
||||
wakeword_models=[args.wake_word],
|
||||
inference_framework="onnx",
|
||||
)
|
||||
|
||||
import time
|
||||
last_trigger = 0.0
|
||||
bytes_per_chunk = CHUNK_SIZE * 2 # 16-bit = 2 bytes per sample
|
||||
|
||||
_LOGGER.debug("Wake word command ready, reading audio from stdin")
|
||||
|
||||
try:
|
||||
while True:
|
||||
raw = sys.stdin.buffer.read(bytes_per_chunk)
|
||||
if not raw:
|
||||
break
|
||||
if len(raw) < bytes_per_chunk:
|
||||
# Pad with zeros if short read
|
||||
raw = raw + b'\x00' * (bytes_per_chunk - len(raw))
|
||||
|
||||
chunk = np.frombuffer(raw, dtype=np.int16)
|
||||
oww.predict(chunk)
|
||||
|
||||
for ww, scores in oww.prediction_buffer.items():
|
||||
score = scores[-1] if scores else 0.0
|
||||
if score >= args.threshold:
|
||||
now = time.time()
|
||||
if now - last_trigger >= args.cooldown:
|
||||
last_trigger = now
|
||||
# Print wake word name to stdout — satellite reads this
|
||||
print(ww, flush=True)
|
||||
_LOGGER.debug("Wake word detected: %s (score=%.3f)", ww, score)
|
||||
except (KeyboardInterrupt, BrokenPipeError):
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
92
plans/p5_development_plan.md
Normal file
92
plans/p5_development_plan.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# P5: HomeAI Character System Development Plan
|
||||
|
||||
> Created: 2026-03-07 | Phase: 3 - Agent & Character
|
||||
|
||||
## Overview
|
||||
Phase 5 (P5) focuses on creating a unified, JSON-based character configuration system that serves as the single source of truth for the AI assistant's personality, voice, visual expressions, and behavioral rules. This configuration will be consumed by OpenClaw (P4), the Voice Pipeline (P3), and the Visual Layer (P7).
|
||||
|
||||
A key component of this phase is building the **Character Manager UI**—a local React application that provides a user-friendly interface for editing character definitions, validating them against a strict JSON schema, and exporting them for use by the agent.
|
||||
|
||||
---
|
||||
|
||||
## 1. Schema & Foundation
|
||||
|
||||
The first step is establishing the strict data contract that all other services will rely on.
|
||||
|
||||
### 1.1 Define Character Schema
|
||||
- Create `homeai-character/schema/character.schema.json` (v1).
|
||||
- Define required fields: `schema_version`, `name`, `system_prompt`, `tts`.
|
||||
- Define optional/advanced fields: `model_overrides`, `live2d_expressions`, `vtube_ws_triggers`, `custom_rules`, `notes`.
|
||||
- Document the schema in `homeai-character/schema/README.md`.
|
||||
|
||||
### 1.2 Create Default Character Profile
|
||||
- Create `homeai-character/characters/aria.json` conforming to the schema.
|
||||
- Define the default system prompt for "Aria" (warm, helpful, concise for smart home tasks).
|
||||
- Configure default TTS settings (`engine: "kokoro"`, `kokoro_voice: "af_heart"`).
|
||||
- Add placeholder mappings for `live2d_expressions` and `vtube_ws_triggers`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Character Manager UI Development
|
||||
|
||||
Transform the existing prototype (`character-manager.jsx`) into a fully functional local web tool.
|
||||
|
||||
### 2.1 Project Initialization
|
||||
- Scaffold a new Vite + React project in `homeai-character/src/`.
|
||||
- Install necessary dependencies: `react`, `react-dom`, `ajv` (for schema validation), and styling utilities (e.g., Tailwind CSS).
|
||||
- Migrate the existing `character-manager.jsx` into the new project structure.
|
||||
|
||||
### 2.2 Schema Validation Integration
|
||||
- Implement `SchemaValidator.js` using `ajv` to validate character configurations against `character.schema.json`.
|
||||
- Enforce validation checks before allowing the user to export or save a character profile.
|
||||
- Display clear error messages in the UI if validation fails.
|
||||
|
||||
### 2.3 UI Feature Implementation
|
||||
- **Basic Info & Prompt Editor:** Fields for name, description, and a multi-line editor for the system prompt (with character count).
|
||||
- **TTS Configuration:** Dropdowns for engine selection (Kokoro, Chatterbox, Qwen3) and inputs for voice reference paths/speed.
|
||||
- **Expression Mapping Table:** UI to map semantic states (idle, listening, thinking, speaking, etc.) to VTube Studio hotkey IDs.
|
||||
- **Custom Rules Editor:** Interface to add, edit, and delete trigger/response/condition pairs.
|
||||
- **Import/Export Pipeline:** Functionality to load an existing JSON file, edit it, and download/save the validated output.
|
||||
|
||||
---
|
||||
|
||||
## 3. Pipeline Integration (Wiring it up)
|
||||
|
||||
Ensure that the generated character configurations are actually used by the rest of the HomeAI ecosystem.
|
||||
|
||||
### 3.1 OpenClaw Integration (P4 Link)
|
||||
- Configure OpenClaw to load the active character from `~/.openclaw/characters/aria.json`.
|
||||
- Modify OpenClaw's initialization to inject the `system_prompt` from the JSON into Ollama requests.
|
||||
- Implement schema version checking in OpenClaw (fail gracefully if `schema_version` is unsupported).
|
||||
- Ensure OpenClaw supports hot-reloading if the character JSON is updated.
|
||||
|
||||
### 3.2 Voice Pipeline Integration (P3 Link)
|
||||
- Update the TTS dispatch logic to read the `tts` configuration block from the character JSON.
|
||||
- Dynamically route TTS requests based on the `engine` field (e.g., routing to Kokoro vs. Chatterbox).
|
||||
|
||||
---
|
||||
|
||||
## 4. Custom Voice Cloning (Optional/Advanced)
|
||||
|
||||
If moving beyond the default Kokoro voice, set up a custom voice clone.
|
||||
|
||||
### 4.1 Audio Processing
|
||||
- Record 30-60 seconds of clean reference audio for the character (`~/voices/aria-raw.wav`).
|
||||
- Pre-process the audio using FFmpeg: `ffmpeg -i aria-raw.wav -ar 22050 -ac 1 aria.wav`.
|
||||
- Move the processed file to the designated directory (`~/voices/aria.wav`).
|
||||
|
||||
### 4.2 Configuration & Testing
|
||||
- Update `aria.json` to use `"engine": "chatterbox"` and set `"voice_ref_path"` to the new audio file.
|
||||
- Test the voice output. If the quality is insufficient, evaluate Qwen3-TTS as a fallback alternative.
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria Checklist
|
||||
|
||||
- [ ] `character.schema.json` is fully defined and documented.
|
||||
- [ ] `aria.json` is created and passes strict validation against the schema.
|
||||
- [ ] Vite-based Character Manager UI runs locally without errors.
|
||||
- [ ] Character Manager successfully imports, edits, validates, and exports character JSONs.
|
||||
- [ ] OpenClaw successfully reads `aria.json` and applies the system prompt to LLM generation.
|
||||
- [ ] TTS engine selection dynamically respects the configuration in the character JSON.
|
||||
- [ ] (Optional) Custom voice reference audio is processed and tested.
|
||||
Reference in New Issue
Block a user