Visual Assistant

This project will give an LLM a visual persona for the purpose of chatting with the user. This will start with simple static png images that change to reflect emotion and chat content and work up to using Live2D models.

MVP Features

Chat interface
Persona selection
Static PNG images for expressions
Changing expressions based on chat content
Use of memories to track user information and allow for consistent behavior

Future Features

Live2D models
TTS

Project Structure

visual-assistant/ ├── core/ # Core application logic │ ├── memory/ # Memory storage and retrieval system │ ├── persona/ # Persona configuration handlers │ └── emotion_engine/ # Emotion detection algorithms ├── chat_interface/ # Web interface components │ ├── static/ # CSS/JS assets │ └── templates/ # HTML templates ├── persona/ # Persona definitions │ ├── default/ # Default persona assets │ └── custom/ # User-created personas ├── utils/ # Utility functions │ ├── image_processing.py │ └── config_loader.py ├── assets/ # Static assets │ ├── expressions/ # PNG expression images │ └── models/ # Live2D model storage ├── tests/ # Test suite └── documentation/ # Project docs

Technical Stack

Backend: Python 3.10+ (FastAPI/Flask)
Frontend: JavaScript/HTML5/CSS3
Mobile: Android (Kotlin/Java)
Memory Storage: SQLite/Redis
Image Processing: Pillow
Dependency Management: Poetry

Deployment

Containerization: Docker
CI/CD: GitHub Actions
Cloud Providers: AWS/Heroku/Firebase
Monitoring: Sentry

1.8 KiB Raw Blame History