feat: character system v2 — schema upgrade, memory system, per-character TTS routing
Character schema v2: background, dialogue_style, appearance, skills, gaze_presets with automatic v1→v2 migration. LLM-assisted character creation via Character MCP server. Two-tier memory system (personal per-character + general shared) with budget-based injection into LLM system prompt. Per-character TTS voice routing via state file — Wyoming TTS server reads active config to route between Kokoro (local) and ElevenLabs (cloud PCM 24kHz). Dashboard: memories page, conversation history, character profile on cards, auto-TTS engine selection from character config. Also includes VTube Studio expression bridge and ComfyUI API guide. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Aodhan Collins
@@ -1,17 +1,16 @@
|
||||
#!/usr/bin/env bash
|
||||
# homeai-visual/setup.sh — P7: VTube Studio bridge + Live2D expressions
|
||||
# homeai-visual/setup.sh — P7: VTube Studio Expression Bridge
|
||||
#
|
||||
# Components:
|
||||
# - vtube_studio.py — WebSocket client skill for OpenClaw
|
||||
# - lipsync.py — amplitude-based lip sync
|
||||
# - auth.py — VTube Studio token management
|
||||
# Sets up:
|
||||
# - Python venv with websockets
|
||||
# - vtube-bridge daemon (HTTP ↔ WebSocket bridge)
|
||||
# - vtube-ctl CLI (symlinked to PATH)
|
||||
# - launchd service
|
||||
#
|
||||
# Prerequisites:
|
||||
# - P4 (homeai-agent) — OpenClaw running
|
||||
# - P5 (homeai-character) — aria.json with live2d_expressions set
|
||||
# - macOS: VTube Studio installed (Mac App Store)
|
||||
# - Linux: N/A — VTube Studio is macOS/Windows/iOS only
|
||||
# Linux dev can test the skill code but not the VTube Studio side
|
||||
# - VTube Studio installed (Mac App Store) with WebSocket API enabled
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
@@ -19,42 +18,61 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
source "${REPO_DIR}/scripts/common.sh"
|
||||
|
||||
log_section "P7: VTube Studio Bridge"
|
||||
detect_platform
|
||||
VENV_DIR="$HOME/homeai-visual-env"
|
||||
PLIST_SRC="${SCRIPT_DIR}/launchd/com.homeai.vtube-bridge.plist"
|
||||
PLIST_DST="$HOME/Library/LaunchAgents/com.homeai.vtube-bridge.plist"
|
||||
VTUBE_CTL_SRC="$HOME/.openclaw/skills/vtube-studio/scripts/vtube-ctl"
|
||||
|
||||
if [[ "$OS_TYPE" == "linux" ]]; then
|
||||
log_warn "VTube Studio is not available on Linux."
|
||||
log_warn "This sub-project requires macOS (Mac Mini)."
|
||||
log_section "P7: VTube Studio Expression Bridge"
|
||||
|
||||
# ─── Python venv ──────────────────────────────────────────────────────────────
|
||||
|
||||
if [[ ! -d "$VENV_DIR" ]]; then
|
||||
log_info "Creating Python venv at $VENV_DIR..."
|
||||
python3 -m venv "$VENV_DIR"
|
||||
fi
|
||||
|
||||
# ─── TODO: Implementation ──────────────────────────────────────────────────────
|
||||
cat <<'EOF'
|
||||
log_info "Installing dependencies..."
|
||||
"$VENV_DIR/bin/pip" install --upgrade pip -q
|
||||
"$VENV_DIR/bin/pip" install websockets -q
|
||||
log_ok "Python venv ready ($(${VENV_DIR}/bin/python3 --version))"
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ P7: homeai-visual — NOT YET IMPLEMENTED │
|
||||
│ │
|
||||
│ macOS only (VTube Studio is macOS/iOS/Windows) │
|
||||
│ │
|
||||
│ Implementation steps: │
|
||||
│ 1. Install VTube Studio from Mac App Store │
|
||||
│ 2. Enable WebSocket API in VTube Studio (Settings → port 8001) │
|
||||
│ 3. Source/purchase Live2D model │
|
||||
│ 4. Create expression hotkeys for 8 states │
|
||||
│ 5. Implement skills/vtube_studio.py (WebSocket client) │
|
||||
│ 6. Implement skills/lipsync.py (amplitude → MouthOpen param) │
|
||||
│ 7. Implement skills/auth.py (token request + persistence) │
|
||||
│ 8. Register vtube_studio skill with OpenClaw │
|
||||
│ 9. Update aria.json live2d_expressions with hotkey IDs │
|
||||
│ 10. Test all 8 expression states │
|
||||
│ │
|
||||
│ On Linux: implement Python skills, test WebSocket protocol │
|
||||
│ with a mock server before connecting to real VTube Studio. │
|
||||
│ │
|
||||
│ Interface contracts: │
|
||||
│ VTUBE_WS_URL=ws://localhost:8001 │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
# ─── vtube-ctl symlink ───────────────────────────────────────────────────────
|
||||
|
||||
EOF
|
||||
if [[ -f "$VTUBE_CTL_SRC" ]]; then
|
||||
chmod +x "$VTUBE_CTL_SRC"
|
||||
ln -sf "$VTUBE_CTL_SRC" /opt/homebrew/bin/vtube-ctl
|
||||
log_ok "vtube-ctl symlinked to /opt/homebrew/bin/vtube-ctl"
|
||||
else
|
||||
log_warn "vtube-ctl not found at $VTUBE_CTL_SRC — skipping symlink"
|
||||
fi
|
||||
|
||||
log_info "P7 is not yet implemented. See homeai-visual/PLAN.md for details."
|
||||
exit 0
|
||||
# ─── launchd service ─────────────────────────────────────────────────────────
|
||||
|
||||
if [[ -f "$PLIST_SRC" ]]; then
|
||||
# Unload if already loaded
|
||||
launchctl bootout "gui/$(id -u)/com.homeai.vtube-bridge" 2>/dev/null || true
|
||||
|
||||
cp "$PLIST_SRC" "$PLIST_DST"
|
||||
launchctl bootstrap "gui/$(id -u)" "$PLIST_DST"
|
||||
log_ok "launchd service loaded: com.homeai.vtube-bridge"
|
||||
else
|
||||
log_warn "Plist not found at $PLIST_SRC — skipping launchd setup"
|
||||
fi
|
||||
|
||||
# ─── Status ──────────────────────────────────────────────────────────────────
|
||||
|
||||
echo ""
|
||||
log_info "VTube Bridge setup complete."
|
||||
log_info ""
|
||||
log_info "Next steps:"
|
||||
log_info " 1. Install VTube Studio from Mac App Store"
|
||||
log_info " 2. Enable WebSocket API: Settings > WebSocket API > port 8001"
|
||||
log_info " 3. Load a Live2D model"
|
||||
log_info " 4. Create expression hotkeys (idle, listening, thinking, speaking, happy, sad, surprised, error)"
|
||||
log_info " 5. Run: vtube-ctl auth (click Allow in VTube Studio)"
|
||||
log_info " 6. Run: python3 ${SCRIPT_DIR}/scripts/test-expressions.py --all"
|
||||
log_info " 7. Update aria.json with real hotkey UUIDs"
|
||||
log_info ""
|
||||
log_info "Logs: /tmp/homeai-vtube-bridge.log"
|
||||
log_info "Bridge: http://localhost:8002/status"
|
||||
|
||||
Reference in New Issue
Block a user