aodhan/character-browser
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add extra prompts, endless generation, random character default, and small fixes

Browse Source 
- Add extra positive/negative prompt textareas to all 9 detail pages with session persistence
- Add Endless generation button to all detail pages (continuous preview generation until stopped)
- Default character selector to "Random Character" on all secondary detail pages
- Fix queue clear endpoint (remove spurious auth check)
- Refactor app.py into routes/ and services/ modules
- Update CLAUDE.md with new architecture documentation
- Various data file updates and cleanup

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Aodhan Collins
2026-03-13 02:07:16 +00:00
parent 1b8a798c31
commit 5e4348ebc1
170 changed files with 17367 additions and 9781 deletions
Download Patch File Download Diff File Expand all files Collapse all files

183
CLAUDE.md
View File

@@ -10,10 +10,67 @@ The app is deployed locally, connects to a local ComfyUI instance at `http://127
## Architecture
### Entry Point
- `app.py` — Single-file Flask app (~4500+ lines). All routes, helpers, ComfyUI integration, LLM calls, and sync functions live here.
- `models.py` — SQLAlchemy models only.
- `comfy_workflow.json` — ComfyUI workflow template with placeholder strings.
### File Structure
```
app.py # ~186 lines: Flask init, config, logging, route registration, startup/migrations
models.py # SQLAlchemy models only
comfy_workflow.json # ComfyUI workflow template with placeholder strings
utils.py # Pure constants + helpers (no Flask/DB deps)
services/
__init__.py
comfyui.py # ComfyUI HTTP client (queue_prompt, get_history, get_image)
workflow.py # Workflow building (_prepare_workflow, _apply_checkpoint_settings)
prompts.py # Prompt building + dedup (build_prompt, build_extras_prompt)
llm.py # LLM integration + MCP tool calls (call_llm, load_prompt)
mcp.py # MCP/Docker server lifecycle (ensure_mcp_server_running)
sync.py # All sync_*() functions + preset resolution helpers
job_queue.py # Background job queue (_enqueue_job, _make_finalize, worker thread)
file_io.py # LoRA/checkpoint scanning, file helpers
routes/
__init__.py # register_routes(app) — imports and calls all route modules
characters.py # Character CRUD + generation + outfit management
outfits.py # Outfit routes
actions.py # Action routes
styles.py # Style routes
scenes.py # Scene routes
detailers.py # Detailer routes
checkpoints.py # Checkpoint routes
looks.py # Look routes
presets.py # Preset routes
generator.py # Generator mix-and-match page
gallery.py # Gallery browsing + image/resource deletion
settings.py # Settings page + status APIs + context processors
strengths.py # Strengths gallery system
transfer.py # Resource transfer system
queue_api.py # /api/queue/* endpoints
```
### Dependency Graph
```
app.py
├── models.py (unchanged)
├── utils.py (no deps except stdlib)
├── services/
│ ├── comfyui.py ← utils (for config)
│ ├── prompts.py ← utils, models
│ ├── workflow.py ← prompts, utils, models
│ ├── llm.py ← mcp (for tool calls)
│ ├── mcp.py ← (stdlib only: subprocess, os)
│ ├── sync.py ← models, utils
│ ├── job_queue.py ← comfyui, models
│ └── file_io.py ← models, utils
└── routes/
├── All route modules ← services/*, utils, models
└── (routes never import from other routes)
```
**No circular imports**: routes → services → utils/models. Services never import routes. Utils never imports services.
### Route Registration Pattern
Routes use a `register_routes(app)` closure pattern — each route module defines a function that receives the Flask `app` object and registers routes via `@app.route()` closures. This preserves all existing `url_for()` endpoint names without requiring Blueprint prefixes. Helper functions used only by routes in that module are defined inside `register_routes()` before the routes that reference them.
### Database
SQLite at `instance/database.db`, managed by Flask-SQLAlchemy. The DB is a cache of the JSON files on disk — the JSON files are the source of truth.
@@ -65,76 +122,79 @@ LoRA nodes chain: `4 → 16 → 17 → 18 → 19`. Unused LoRA nodes are bypasse
---
## Key Helpers
## Key Functions by Module
### `build_prompt(data, selected_fields, default_fields, active_outfit)`
Converts a character (or combined) data dict into `{"main", "face", "hand"}` prompt strings.
### `utils.py` — Constants and Pure Helpers
Field selection priority:
1. `selected_fields` (from form submission) — if non-empty, use it exclusively
2. `default_fields` (saved in DB per character) — used if no form selection
3. Select all (fallback)
- **`_IDENTITY_KEYS` / `_WARDROBE_KEYS`** — Lists of canonical field names for the `identity` and `wardrobe` sections. Used by `_ensure_character_fields()`.
- **`ALLOWED_EXTENSIONS`** — Permitted upload file extensions.
- **`_LORA_DEFAULTS`** — Default LoRA directory paths per category.
- **`parse_orientation(orientation_str)`** — Converts orientation codes (`1F`, `2F`, `1M1F`, etc.) into Danbooru tags.
- **`_resolve_lora_weight(lora_data)`** — Extracts and validates LoRA weight from a lora data dict.
- **`allowed_file(filename)`** — Checks file extension against `ALLOWED_EXTENSIONS`.
Fields are addressed as `"section::key"` strings (e.g. `"identity::hair"`, `"wardrobe::top"`, `"special::name"`, `"lora::lora_triggers"`).
### `services/prompts.py` — Prompt Building
Wardrobe format: Characters support a **nested** format where `wardrobe` is a dict of outfit names → outfit dicts (e.g. `{"default": {...}, "swimsuit": {...}}`). Legacy characters have a flat `wardrobe` dict. `Character.get_active_wardrobe()` handles both.
- **`build_prompt(data, selected_fields, default_fields, active_outfit)`** — Converts a character (or combined) data dict into `{"main", "face", "hand"}` prompt strings. Field selection priority: `selected_fields` → `default_fields` → select all (fallback). Fields are addressed as `"section::key"` strings (e.g. `"identity::hair"`, `"wardrobe::top"`). Characters support a **nested** wardrobe format where `wardrobe` is a dict of outfit names → outfit dicts.
- **`build_extras_prompt(actions, outfits, scenes, styles, detailers)`** — Used by the Generator page. Combines prompt text from all checked items across categories into a single string.
- **`_cross_dedup_prompts(positive, negative)`** — Cross-deduplicates tags between positive and negative prompt strings. Equal counts cancel completely; excess on one side is retained.
- **`_resolve_character(character_slug)`** — Returns a `Character` ORM object for a given slug string. Handles `"__random__"` sentinel.
- **`_ensure_character_fields(character, selected_fields, ...)`** — Mutates `selected_fields` in place, appending populated identity/wardrobe keys. Called in every secondary-category generate route after `_resolve_character()`.
- **`_append_background(prompts, character=None)`** — Appends `"<primary_color> simple background"` tag to `prompts['main']`.
### `build_extras_prompt(actions, outfits, scenes, styles, detailers)`
Used by the Generator page. Combines prompt text from all checked items across categories into a single string. LoRA triggers, wardrobe fields, scene fields, style fields, detailer prompts — all concatenated.
### `services/workflow.py` — Workflow Wiring
### `_prepare_workflow(workflow, character, prompts, checkpoint, custom_negative, outfit, action, style, detailer, scene, width, height, checkpoint_data, look)`
The core workflow wiring function. Mutates the loaded workflow dict in place and returns it. Key behaviours:
- Replaces `{{POSITIVE_PROMPT}}`, `{{FACE_PROMPT}}`, `{{HAND_PROMPT}}` in node inputs.
- If `look` is provided, uses the Look's LoRA in node 16 instead of the character's LoRA, and prepends the Look's negative to node 7.
- Chains LoRA nodes dynamically — only active LoRAs participate in the chain.
- Randomises seeds for nodes 3, 11, 13.
- Applies checkpoint-specific settings (steps, cfg, sampler, VAE, base prompts) via `_apply_checkpoint_settings`.
- Runs `_cross_dedup_prompts` on nodes 6 and 7 as the final step before returning.
- **`_prepare_workflow(workflow, character, prompts, ...)`** — Core workflow wiring function. Replaces prompt placeholders, chains LoRA nodes dynamically, randomises seeds, applies checkpoint settings, runs cross-dedup as the final step.
- **`_apply_checkpoint_settings(workflow, ckpt_data)`** — Applies checkpoint-specific sampler/prompt/VAE settings.
- **`_get_default_checkpoint()`** — Returns `(checkpoint_path, checkpoint_data)` from session, database Settings, or workflow file fallback.
- **`_log_workflow_prompts(label, workflow)`** — Logs the fully assembled workflow prompts in a readable block.
### `_cross_dedup_prompts(positive, negative)`
Cross-deduplicates tags between the positive and negative prompt strings. For each tag present in both, repeatedly removes the first occurrence from each side until the tag exists on only one side (or neither). Equal counts cancel completely; any excess on one side is retained. This allows deliberate overrides: adding a tag twice in positive while it appears once in negative leaves one copy in positive.
### `services/job_queue.py` — Background Job Queue
Called as the last step of `_prepare_workflow`, after `_apply_checkpoint_settings` has added `base_positive`/`base_negative`, so it operates on fully-assembled prompts.
- **`_enqueue_job(label, workflow, finalize_fn)`** — Adds a generation job to the queue.
- **`_make_finalize(category, slug, db_model_class=None, action=None)`** — Factory returning a callback that retrieves the generated image from ComfyUI, saves it, and optionally updates the DB cover image.
- **`_prune_job_history(max_age_seconds=3600)`** — Removes old terminal-state jobs from memory.
- **`init_queue_worker(flask_app)`** — Stores the app reference and starts the worker thread.
### `_IDENTITY_KEYS` / `_WARDROBE_KEYS` (module-level constants)
Lists of canonical field names for the `identity` and `wardrobe` sections. Used by `_ensure_character_fields()` to avoid hard-coding key lists in every route.
### `services/comfyui.py` — ComfyUI HTTP Client
### `_resolve_character(character_slug)`
Returns a `Character` ORM object for a given slug string. Handles the `"__random__"` sentinel by selecting a random character. Returns `None` if `character_slug` is falsy or no match is found. Every route that accepts an optional character dropdown (outfit, action, style, scene, detailer, checkpoint, look generate routes) uses this instead of an inline if/elif block.
- **`queue_prompt(prompt_workflow, client_id)`** — POSTs workflow to ComfyUI's `/prompt` endpoint.
- **`get_history(prompt_id)`** — Polls ComfyUI for job completion.
- **`get_image(filename, subfolder, folder_type)`** — Retrieves generated image bytes.
- **`_ensure_checkpoint_loaded(checkpoint_path)`** — Forces ComfyUI to load a specific checkpoint.
### `_ensure_character_fields(character, selected_fields, include_wardrobe=True, include_defaults=False)`
Mutates `selected_fields` in place, appending any populated identity, wardrobe, and optional defaults keys that are not already present. Ensures `"special::name"` is always included. Called in every secondary-category generate route immediately after `_resolve_character()` to guarantee the character's essential fields are sent to `build_prompt`.
### `services/llm.py` — LLM Integration
### `_append_background(prompts, character=None)`
Appends a `"<primary_color> simple background"` tag (or plain `"simple background"` if no primary color) to `prompts['main']`. Called in outfit, action, style, detailer, and checkpoint generate routes instead of repeating the same inline string construction.
- **`call_llm(prompt, system_prompt)`** — OpenAI-compatible chat completion supporting OpenRouter (cloud) and Ollama/LMStudio (local). Implements a tool-calling loop (up to 10 turns) using `DANBOORU_TOOLS` via MCP Docker container.
- **`load_prompt(filename)`** — Loads system prompt text from `data/prompts/`.
- **`call_mcp_tool()`** — Synchronous wrapper for MCP tool calls.
### `_make_finalize(category, slug, db_model_class=None, action=None)`
Factory function that returns a `_finalize(comfy_prompt_id, job)` callback closure. The closure:
1. Calls `get_history()` and `get_image()` to retrieve the generated image from ComfyUI.
2. Saves the image to `static/uploads/<category>/<slug>/gen_<timestamp>.png`.
3. Sets `job['result']` with `image_url` and `relative_path`.
4. If `db_model_class` is provided **and** (`action` is `None` or `action == 'replace'`), updates the ORM object's `image_path` and commits.
### `services/sync.py` — Data Synchronization
All `generate` routes pass a `_make_finalize(...)` call as the finalize argument to `_enqueue_job()` instead of defining an inline closure.
- **`sync_characters()`, `sync_outfits()`, `sync_actions()`, etc.** — Load JSON files from `data/` directories into SQLite. One function per category.
- **`_resolve_preset_entity(type, id)`** / **`_resolve_preset_fields(preset_data)`** — Preset resolution helpers.
### `_prune_job_history(max_age_seconds=3600)`
Removes entries from `_job_history` that are in a terminal state (`done`, `failed`, `removed`) and older than `max_age_seconds`. Called at the end of every worker loop iteration to prevent unbounded memory growth.
### `services/file_io.py` — File & DB Helpers
### `_queue_generation(character, action, selected_fields, client_id)`
Convenience wrapper for character detail page generation. Loads workflow, calls `build_prompt`, calls `_prepare_workflow`, calls `queue_prompt`.
- **`get_available_loras(category)`** — Scans filesystem for available LoRA files in a category.
- **`get_available_checkpoints()`** — Scans checkpoint directories.
- **`_count_look_assignments()`** / **`_count_outfit_lora_assignments()`** — DB aggregate queries.
### `_queue_detailer_generation(detailer_obj, character, selected_fields, client_id, action, extra_positive, extra_negative)`
Generation helper for the detailer detail page. Merges the detailer's `prompt` list (flattened — `prompt` may be a list or string) and LoRA triggers into the character data before calling `build_prompt`. Appends `extra_positive` to `prompts["main"]` and passes `extra_negative` as `custom_negative` to `_prepare_workflow`. Accepts an optional `action` (Action model object) passed through to `_prepare_workflow` for node 18 LoRA wiring.
### `services/mcp.py` — MCP/Docker Lifecycle
### `_get_default_checkpoint()`
Returns `(checkpoint_path, checkpoint_data)` from the Flask session's `default_checkpoint` key. The global default checkpoint is set via the navbar dropdown and stored server-side in the filesystem session.
- **`ensure_mcp_server_running()`** — Ensures the danbooru-mcp Docker container is running.
- **`ensure_character_mcp_server_running()`** — Ensures the character-mcp Docker container is running.
### `call_llm(prompt, system_prompt)`
OpenAI-compatible chat completion call supporting:
- **OpenRouter** (cloud, configured API key + model)
- **Ollama** / **LMStudio** (local, configured base URL + model)
### Route-local Helpers
Implements a tool-calling loop (up to 10 turns) using `DANBOORU_TOOLS` (search_tags, validate_tags, suggest_tags) via an MCP Docker container (`danbooru-mcp:latest`). If the provider rejects tools (HTTP 400), retries without tools.
Some helpers are defined inside a route module's `register_routes()` since they're only used by routes in that file:
- `routes/scenes.py`: `_queue_scene_generation()` — scene-specific workflow builder
- `routes/detailers.py`: `_queue_detailer_generation()` — detailer-specific generation helper
- `routes/styles.py`: `_build_style_workflow()` — style-specific workflow builder
- `routes/checkpoints.py`: `_build_checkpoint_workflow()` — checkpoint-specific workflow builder
- `routes/strengths.py`: `_build_strengths_prompts()`, `_prepare_strengths_workflow()` — strengths gallery helpers
- `routes/transfer.py`: `_create_minimal_template()` — transfer template builder
- `routes/gallery.py`: `_scan_gallery_images()`, `_enrich_with_names()`, `_parse_comfy_png_metadata()`
---
@@ -377,13 +437,14 @@ Absolute paths on disk:
To add a new content category (e.g. "Poses" as a separate concept from Actions), the pattern is:
1. **Model** (`models.py`): Add a new SQLAlchemy model with the standard fields.
2. **Sync function** (`app.py`): Add `sync_newcategory()` following the pattern of `sync_outfits()`.
3. **Data directory**: Add `app.config['NEWCATEGORY_DIR'] = 'data/newcategory'`.
4. **Routes**: Implement index, detail, edit, generate, finalize_generation, replace_cover_from_preview, upload, save_defaults, clone, rescan routes. Follow the outfit/scene pattern exactly.
5. **Templates**: Create `templates/newcategory/{index,detail,edit,create}.html` extending `layout.html`.
6. **Nav**: Add link to navbar in `templates/layout.html`.
7. **`with_appcontext`**: Call `sync_newcategory()` in the `with app.app_context()` block at the bottom of `app.py`.
8. **Generator page**: Add to `generator()` route, `build_extras_prompt()`, and `templates/generator.html` accordion.
2. **Sync function** (`services/sync.py`): Add `sync_newcategory()` following the pattern of `sync_outfits()`.
3. **Data directory** (`app.py`): Add `app.config['NEWCATEGORY_DIR'] = 'data/newcategory'`.
4. **Routes** (`routes/newcategory.py`): Create a new route module with a `register_routes(app)` function. Implement index, detail, edit, generate, replace_cover_from_preview, upload, save_defaults, clone, rescan routes. Follow `routes/outfits.py` or `routes/scenes.py` exactly.
5. **Route registration** (`routes/__init__.py`): Import and call `newcategory.register_routes(app)`.
6. **Templates**: Create `templates/newcategory/{index,detail,edit,create}.html` extending `layout.html`.
7. **Nav**: Add link to navbar in `templates/layout.html`.
8. **Startup** (`app.py`): Import and call `sync_newcategory()` in the `with app.app_context()` block.
9. **Generator page**: Add to `routes/generator.py`, `services/prompts.py` `build_extras_prompt()`, and `templates/generator.html` accordion.
---