character-browser/CLAUDE.md

# GAZE — Character Browser: LLM Development Guide

## What This Project Is

GAZE is a Flask web app for managing AI image generation assets and generating images via ComfyUI. It is a **personal creative tool** for organizing characters, outfits, actions, styles, scenes, and detailers — all of which map to Stable Diffusion LoRAs and prompt fragments — and generating images by wiring those assets into a ComfyUI workflow at runtime.

The app is deployed locally, connects to a local ComfyUI instance at `http://127.0.0.1:8188`, and uses SQLite for persistence. LoRA and model files live on `/mnt/alexander/AITools/Image Models/`.

---

## Architecture

### File Structure

```
app.py                          # ~186 lines: Flask init, config, logging, route registration, startup/migrations
models.py                       # SQLAlchemy models only
comfy_workflow.json              # ComfyUI workflow template with placeholder strings
utils.py                        # Pure constants + helpers (no Flask/DB deps)
services/
  __init__.py
  comfyui.py                    # ComfyUI HTTP client (queue_prompt, get_history, get_image)
  workflow.py                   # Workflow building (_prepare_workflow, _apply_checkpoint_settings)
  prompts.py                    # Prompt building + dedup (build_prompt, build_extras_prompt)
  llm.py                        # LLM integration + MCP tool calls (call_llm, load_prompt)
  mcp.py                        # MCP/Docker server lifecycle (ensure_mcp_server_running)
  sync.py                       # All sync_*() functions + preset resolution helpers
  job_queue.py                  # Background job queue (_enqueue_job, _make_finalize, worker thread)
  file_io.py                    # LoRA/checkpoint scanning, file helpers
  generation.py                 # Shared generation logic (generate_from_preset)
routes/
  __init__.py                   # register_routes(app) — imports and calls all route modules
  characters.py                 # Character CRUD + generation + outfit management
  outfits.py                    # Outfit routes
  actions.py                    # Action routes
  styles.py                     # Style routes
  scenes.py                     # Scene routes
  detailers.py                  # Detailer routes
  checkpoints.py                # Checkpoint routes
  looks.py                      # Look routes
  presets.py                    # Preset routes
  generator.py                  # Generator mix-and-match page
  gallery.py                    # Gallery browsing + image/resource deletion
  settings.py                   # Settings page + status APIs + context processors
  strengths.py                  # Strengths gallery system
  transfer.py                   # Resource transfer system
  queue_api.py                  # /api/queue/* endpoints
  api.py                        # REST API v1 (preset generation, auth)
  regenerate.py                 # Tag regeneration (single + bulk, via LLM queue)
  search.py                     # Global search across resources and gallery images
```

### Dependency Graph

```
app.py
  ├── models.py (unchanged)
  ├── utils.py (no deps except stdlib)
  ├── services/
  │   ├── comfyui.py        ← utils (for config)
  │   ├── prompts.py        ← utils, models
  │   ├── workflow.py       ← prompts, utils, models
  │   ├── llm.py            ← mcp (for tool calls)
  │   ├── mcp.py            ← (stdlib only: subprocess, os)
  │   ├── sync.py           ← models, utils
  │   ├── job_queue.py      ← comfyui, models
  │   ├── file_io.py        ← models, utils
  │   └── generation.py     ← prompts, workflow, job_queue, sync, models
  └── routes/
      ├── All route modules  ← services/*, utils, models
      └── (routes never import from other routes)
```

**No circular imports**: routes → services → utils/models. Services never import routes. Utils never imports services.

### Route Registration Pattern

Routes use a `register_routes(app)` closure pattern — each route module defines a function that receives the Flask `app` object and registers routes via `@app.route()` closures. This preserves all existing `url_for()` endpoint names without requiring Blueprint prefixes. Helper functions used only by routes in that module are defined inside `register_routes()` before the routes that reference them.

### Database
SQLite at `instance/database.db`, managed by Flask-SQLAlchemy. The DB is a cache of the JSON files on disk — the JSON files are the source of truth.

**Models**: `Character`, `Look`, `Outfit`, `Action`, `Style`, `Scene`, `Detailer`, `Checkpoint`, `Settings`

The `Settings` model stores LLM provider config, LoRA/checkpoint directory paths, default checkpoint, and `api_key` for REST API authentication.

All category models (except Settings and Checkpoint) share this pattern:
- `{entity}_id` — canonical ID (from JSON, often matches filename without extension)
- `slug` — URL-safe version of the ID (alphanumeric + underscores only, via `re.sub(r'[^a-zA-Z0-9_]', '', id)`)
- `name` — display name
- `filename` — original JSON filename
- `data` — full JSON blob (SQLAlchemy JSON column)
- `default_fields` — list of `section::key` strings saved as the user's preferred prompt fields
- `image_path` — relative path under `static/uploads/`
- `is_favourite` — boolean (DB-only, not in JSON; toggled from detail pages)
- `is_nsfw` — boolean (mirrored in both DB column and JSON `tags.nsfw`; synced on rescan)

### Data Flow: JSON → DB → Prompt → ComfyUI

1. **JSON files** in `data/{characters,clothing,actions,styles,scenes,detailers,looks}/` are loaded by `sync_*()` functions into SQLite.
2. At generation time, `build_prompt(data, selected_fields, default_fields, active_outfit)` converts the character JSON blob into `{"main": ..., "face": ..., "hand": ...}` prompt strings.
3. `_prepare_workflow(workflow, character, prompts, ...)` wires prompts and LoRAs into the loaded `comfy_workflow.json`.
4. `queue_prompt(workflow, client_id)` POSTs the workflow to ComfyUI's `/prompt` endpoint.
5. The app polls `get_history(prompt_id)` and retrieves the image via `get_image(filename, subfolder, type)`.

---

## ComfyUI Workflow Node Map

The workflow (`comfy_workflow.json`) uses string node IDs. These are the critical nodes:

| Node | Role |
|------|------|
| `3` | Main KSampler |
| `4` | Checkpoint loader |
| `5` | Empty latent (width/height) |
| `6` | Positive prompt — contains `{{POSITIVE_PROMPT}}` placeholder |
| `7` | Negative prompt |
| `8` | VAE decode |
| `9` | Save image |
| `11` | Face ADetailer |
| `13` | Hand ADetailer |
| `14` | Face detailer prompt — contains `{{FACE_PROMPT}}` placeholder |
| `15` | Hand detailer prompt — contains `{{HAND_PROMPT}}` placeholder |
| `16` | Character LoRA (or Look LoRA when a Look is active) |
| `17` | Outfit LoRA |
| `18` | Action LoRA |
| `19` | Style / Detailer / Scene LoRA (priority: style > detailer > scene) |

LoRA nodes chain: `4 → 16 → 17 → 18 → 19`. Unused LoRA nodes are bypassed by pointing `model_source`/`clip_source` directly to the prior node. All model/clip consumers (nodes 3, 6, 7, 11, 13, 14, 15) are wired to the final `model_source`/`clip_source` at the end of `_prepare_workflow`.

---

## Key Functions by Module

### `utils.py` — Constants and Pure Helpers

- **`_IDENTITY_KEYS` / `_WARDROBE_KEYS`** — Lists of canonical field names for the `identity` and `wardrobe` sections. Used by `_ensure_character_fields()`.
- **`ALLOWED_EXTENSIONS`** — Permitted upload file extensions.
- **`_LORA_DEFAULTS`** — Default LoRA directory paths per category.
- **`parse_orientation(orientation_str)`** — Converts orientation codes (`1F`, `2F`, `1M1F`, etc.) into Danbooru tags.
- **`_resolve_lora_weight(lora_data)`** — Extracts and validates LoRA weight from a lora data dict.
- **`allowed_file(filename)`** — Checks file extension against `ALLOWED_EXTENSIONS`.

### `services/prompts.py` — Prompt Building

- **`build_prompt(data, selected_fields, default_fields, active_outfit)`** — Converts a character (or combined) data dict into `{"main", "face", "hand"}` prompt strings. Field selection priority: `selected_fields` → `default_fields` → select all (fallback). Fields are addressed as `"section::key"` strings (e.g. `"identity::hair"`, `"wardrobe::top"`). Characters support a **nested** wardrobe format where `wardrobe` is a dict of outfit names → outfit dicts.
- **`build_extras_prompt(actions, outfits, scenes, styles, detailers)`** — Used by the Generator page. Combines prompt text from all checked items across categories into a single string.
- **`_cross_dedup_prompts(positive, negative)`** — Cross-deduplicates tags between positive and negative prompt strings. Equal counts cancel completely; excess on one side is retained.
- **`_resolve_character(character_slug)`** — Returns a `Character` ORM object for a given slug string. Handles `"__random__"` sentinel.
- **`_ensure_character_fields(character, selected_fields, ...)`** — Mutates `selected_fields` in place, appending populated identity/wardrobe keys. Called in every secondary-category generate route after `_resolve_character()`.
- **`_append_background(prompts, character=None)`** — Appends `"<primary_color> simple background"` tag to `prompts['main']`.

### `services/workflow.py` — Workflow Wiring

- **`_prepare_workflow(workflow, character, prompts, ...)`** — Core workflow wiring function. Replaces prompt placeholders, chains LoRA nodes dynamically, randomises seeds, applies checkpoint settings, runs cross-dedup as the final step.
- **`_apply_checkpoint_settings(workflow, ckpt_data)`** — Applies checkpoint-specific sampler/prompt/VAE settings.
- **`_get_default_checkpoint()`** — Returns `(checkpoint_path, checkpoint_data)` from session, database Settings, or workflow file fallback.
- **`_log_workflow_prompts(label, workflow)`** — Logs the fully assembled workflow prompts in a readable block.

### `services/job_queue.py` — Background Job Queue

Two independent queues with separate worker threads:

- **ComfyUI queue** (`_job_queue` + `_queue_worker`): Image generation jobs.
  - **`_enqueue_job(label, workflow, finalize_fn)`** — Adds a generation job to the queue.
  - **`_make_finalize(category, slug, db_model_class=None, action=None)`** — Factory returning a callback that retrieves the generated image from ComfyUI, saves it, and optionally updates the DB cover image.
- **LLM queue** (`_llm_queue` + `_llm_queue_worker`): LLM task jobs (tag regeneration, bulk create with overwrite).
  - **`_enqueue_task(label, task_fn)`** — Adds an LLM task job. `task_fn` receives the job dict and runs inside `app.app_context()`.
- **Shared**: Both queues share `_job_history` (for status lookup by job ID) and `_job_queue_lock`.
- **`_prune_job_history(max_age_seconds=3600)`** — Removes old terminal-state jobs from memory.
- **`init_queue_worker(flask_app)`** — Stores the app reference and starts both worker threads.

### `services/comfyui.py` — ComfyUI HTTP Client

- **`queue_prompt(prompt_workflow, client_id)`** — POSTs workflow to ComfyUI's `/prompt` endpoint.
- **`get_history(prompt_id)`** — Polls ComfyUI for job completion.
- **`get_image(filename, subfolder, folder_type)`** — Retrieves generated image bytes.
- **`_ensure_checkpoint_loaded(checkpoint_path)`** — Forces ComfyUI to load a specific checkpoint.

### `services/llm.py` — LLM Integration

- **`call_llm(prompt, system_prompt)`** — OpenAI-compatible chat completion supporting OpenRouter (cloud) and Ollama/LMStudio (local). Implements a tool-calling loop (up to 10 turns) using `DANBOORU_TOOLS` via MCP Docker container. Safe to call from background threads (uses `has_request_context()` fallback for OpenRouter HTTP-Referer header).
- **`load_prompt(filename)`** — Loads system prompt text from `data/prompts/`.
- **`call_mcp_tool()`** — Synchronous wrapper for MCP tool calls.

### `services/sync.py` — Data Synchronization

- **`sync_characters()`, `sync_outfits()`, `sync_actions()`, etc.** — Load JSON files from `data/` directories into SQLite. One function per category.
- **`_sync_nsfw_from_tags(entity, data)`** — Reads `data['tags']['nsfw']` and sets `entity.is_nsfw`. Called in every sync function on both create and update paths.
- **`_resolve_preset_entity(type, id)`** / **`_resolve_preset_fields(preset_data)`** — Preset resolution helpers.

### `services/file_io.py` — File & DB Helpers

- **`get_available_loras(category)`** — Scans filesystem for available LoRA files in a category.
- **`get_available_checkpoints()`** — Scans checkpoint directories.
- **`_count_look_assignments()`** / **`_count_outfit_lora_assignments()`** — DB aggregate queries.

### `services/generation.py` — Shared Generation Logic

- **`generate_from_preset(preset, overrides=None)`** — Core preset generation function used by both the web route and the REST API. Resolves entities, builds prompts, wires the workflow, and enqueues the job. The `overrides` dict accepts: `action`, `checkpoint`, `extra_positive`, `extra_negative`, `seed`, `width`, `height`. Has no `request` or `session` dependencies.

### `services/mcp.py` — MCP/Docker Lifecycle

- **`ensure_mcp_server_running()`** — Ensures the danbooru-mcp Docker container is running.
- **`ensure_character_mcp_server_running()`** — Ensures the character-mcp Docker container is running.

### Route-local Helpers

Some helpers are defined inside a route module's `register_routes()` since they're only used by routes in that file:
- `routes/scenes.py`: `_queue_scene_generation()` — scene-specific workflow builder
- `routes/detailers.py`: `_queue_detailer_generation()` — detailer-specific generation helper
- `routes/styles.py`: `_build_style_workflow()` — style-specific workflow builder
- `routes/checkpoints.py`: `_build_checkpoint_workflow()` — checkpoint-specific workflow builder
- `routes/strengths.py`: `_build_strengths_prompts()`, `_prepare_strengths_workflow()` — strengths gallery helpers
- `routes/transfer.py`: `_create_minimal_template()` — transfer template builder
- `routes/gallery.py`: `_scan_gallery_images()`, `_enrich_with_names()`, `_parse_comfy_png_metadata()`, `_write_sidecar()` — gallery image sidecar JSON I/O
- `routes/regenerate.py`: Tag regeneration routes (single + category bulk + all), tag migration
- `routes/search.py`: `_search_resources()`, `_search_images()` — global search across resources and gallery

---

## JSON Data Schemas

### Character (`data/characters/*.json`)
```json
{
  "character_id": "tifa_lockhart",
  "character_name": "Tifa Lockhart",
  "identity": { "base_specs": "", "hair": "", "eyes": "", "hands": "", "arms": "", "torso": "", "pelvis": "", "legs": "", "feet": "", "extra": "" },
  "defaults": { "expression": "", "pose": "", "scene": "" },
  "wardrobe": {
    "default": { "full_body": "", "headwear": "", "top": "", "bottom": "", "legwear": "", "footwear": "", "hands": "", "gloves": "", "accessories": "" }
  },
  "styles": { "aesthetic": "", "primary_color": "", "secondary_color": "", "tertiary_color": "" },
  "lora": { "lora_name": "Illustrious/Looks/tifa.safetensors", "lora_weight": 0.8, "lora_triggers": "" },
  "tags": { "origin_series": "Final Fantasy VII", "origin_type": "game", "nsfw": false },
  "participants": { "orientation": "1F", "solo_focus": "true" }
}
```
`participants` is optional; when absent, `(solo:1.2)` is injected. `orientation` is parsed by `parse_orientation()` into Danbooru tags (`1girl`, `hetero`, etc.).

### Outfit (`data/clothing/*.json`)
```json
{
  "outfit_id": "french_maid_01",
  "outfit_name": "French Maid",
  "wardrobe": { "full_body": "", "headwear": "", "top": "", "bottom": "", "legwear": "", "footwear": "", "hands": "", "accessories": "" },
  "lora": { "lora_name": "Illustrious/Clothing/maid.safetensors", "lora_weight": 0.8, "lora_triggers": "" },
  "tags": { "outfit_type": "Uniform", "nsfw": false }
}
```

### Action (`data/actions/*.json`)
```json
{
  "action_id": "sitting",
  "action_name": "Sitting",
  "action": { "full_body": "", "additional": "", "head": "", "eyes": "", "arms": "", "hands": "" },
  "lora": { "lora_name": "", "lora_weight": 1.0, "lora_triggers": "" },
  "tags": { "participants": "1girl", "nsfw": false }
}
```

### Scene (`data/scenes/*.json`)
```json
{
  "scene_id": "beach",
  "scene_name": "Beach",
  "scene": { "background": "", "foreground": "", "furniture": "", "colors": "", "lighting": "", "theme": "" },
  "lora": { "lora_name": "", "lora_weight": 1.0, "lora_triggers": "" },
  "tags": { "scene_type": "Outdoor", "nsfw": false }
}
```

### Style (`data/styles/*.json`)
```json
{
  "style_id": "watercolor",
  "style_name": "Watercolor",
  "style": { "artist_name": "", "artistic_style": "" },
  "lora": { "lora_name": "", "lora_weight": 1.0, "lora_triggers": "" },
  "tags": { "style_type": "Watercolor", "nsfw": false }
}
```

### Detailer (`data/detailers/*.json`)
```json
{
  "detailer_id": "detailed_skin",
  "detailer_name": "Detailed Skin",
  "prompt": ["detailed skin", "pores"],
  "focus": { "face": true, "hands": true },
  "lora": { "lora_name": "", "lora_weight": 1.0, "lora_triggers": "" },
  "tags": { "associated_resource": "skin", "adetailer_targets": ["face", "hands"], "nsfw": false }
}
```

### Look (`data/looks/*.json`)
```json
{
  "look_id": "tifa_casual",
  "look_name": "Tifa Casual",
  "character_id": "tifa_lockhart",
  "positive": "casual clothes, jeans",
  "negative": "revealing",
  "lora": { "lora_name": "Illustrious/Looks/tifa_casual.safetensors", "lora_weight": 0.85, "lora_triggers": "" },
  "tags": { "origin_series": "Final Fantasy VII", "origin_type": "game", "nsfw": false }
}
```
Looks occupy LoRA node 16, overriding the character's own LoRA. The Look's `negative` is prepended to the workflow's negative prompt.

### Checkpoint (`data/checkpoints/*.json`)
```json
{
  "checkpoint_path": "Illustrious/model.safetensors",
  "checkpoint_name": "Model Display Name",
  "base_positive": "anime",
  "base_negative": "text, logo",
  "steps": 25,
  "cfg": 5,
  "sampler_name": "euler_ancestral",
  "scheduler": "normal",
  "vae": "integrated"
}
```
Checkpoint JSONs are keyed by `checkpoint_path`. If no JSON exists for a discovered model file, `_default_checkpoint_data()` provides defaults.

---

## URL Routes

### Characters
- `GET /` — character gallery (index)
- `GET /character/<slug>` — character detail with generation UI
- `POST /character/<slug>/generate` — queue generation (AJAX or form); returns `{"job_id": ...}`
- `POST /character/<slug>/replace_cover_from_preview` — promote preview to cover
- `GET/POST /character/<slug>/edit` — edit character data
- `POST /character/<slug>/upload` — upload cover image
- `POST /character/<slug>/save_defaults` — save default field selection
- `POST /character/<slug>/outfit/switch|add|delete|rename` — manage per-character wardrobe outfits
- `GET/POST /create` — create new character (blank or LLM-generated)
- `POST /rescan` — sync DB from JSON files

### Category Pattern (Outfits, Actions, Styles, Scenes, Detailers)
Each category follows the same URL pattern:
- `GET /<category>/` — library with favourite/NSFW filter controls
- `GET /<category>/<slug>` — detail + generation UI
- `POST /<category>/<slug>/generate` — queue generation; returns `{"job_id": ...}`
- `POST /<category>/<slug>/replace_cover_from_preview`
- `GET/POST /<category>/<slug>/edit`
- `POST /<category>/<slug>/upload`
- `POST /<category>/<slug>/save_defaults`
- `POST /<category>/<slug>/favourite` — toggle `is_favourite` (AJAX)
- `POST /<category>/<slug>/clone` — duplicate entry
- `POST /<category>/<slug>/save_json` — save raw JSON (from modal editor)
- `POST /<category>/rescan`
- `POST /<category>/bulk_create` — LLM-generate entries from LoRA files on disk

### Looks
- `GET /looks` — gallery
- `GET /look/<slug>` — detail
- `GET/POST /look/<slug>/edit`
- `POST /look/<slug>/generate` — queue generation; returns `{"job_id": ...}`
- `POST /look/<slug>/replace_cover_from_preview`
- `GET/POST /look/create`
- `POST /looks/rescan`

### Generator (Mix & Match)
- `GET/POST /generator` — freeform generator with multi-select accordion UI
- `POST /generator/preview_prompt` — AJAX: preview composed prompt without generating

### Checkpoints
- `GET /checkpoints` — gallery
- `GET /checkpoint/<slug>` — detail + generation settings editor
- `POST /checkpoint/<slug>/save_json`
- `POST /checkpoints/rescan`

### REST API (`/api/v1/`)
Authenticated via `X-API-Key` header (or `api_key` query param). Key is stored in `Settings.api_key` and managed from the Settings page.
- `GET /api/v1/presets` — list all presets (id, slug, name, has_cover)
- `POST /api/v1/generate/<preset_slug>` — queue generation from a preset; accepts JSON body with optional `checkpoint`, `extra_positive`, `extra_negative`, `seed`, `width`, `height`, `count` (1–20); returns `{"jobs": [{"job_id": ..., "status": "queued"}]}`
- `GET /api/v1/job/<job_id>` — poll job status; returns `{"id", "label", "status", "error", "result"}`
- `POST /api/key/regenerate` — generate a new API key (Settings page)

See `API_GUIDE.md` for full usage examples.

### Job Queue API
All generation routes use the background job queue. Frontend polls:
- `GET /api/queue/<job_id>/status` — returns `{"status": "pending"|"running"|"done"|"failed", "result": {...}}`

Image retrieval is handled server-side by the `_make_finalize()` callback; there are no separate client-facing finalize routes.

### Search
- `GET /search` — global search page; query params: `q` (search term), `category` (all/characters/outfits/etc.), `nsfw` (all/sfw/nsfw), `type` (all/resources/images)

### Tag Regeneration
- `POST /api/<category>/<slug>/regenerate_tags` — single entity tag regeneration via LLM queue
- `POST /admin/bulk_regenerate_tags/<category>` — queue LLM tag regeneration for all entities in a category
- `POST /admin/bulk_regenerate_tags` — queue LLM tag regeneration for all resources across all categories
- `POST /admin/migrate_tags` — convert old list-format tags to new dict format

### Gallery Image Metadata
- `POST /gallery/image/favourite` — toggle favourite on a gallery image (writes sidecar JSON)
- `POST /gallery/image/nsfw` — toggle NSFW on a gallery image (writes sidecar JSON)

### Utilities
- `POST /set_default_checkpoint` — save default checkpoint to session and persist to `comfy_workflow.json`
- `GET /get_missing_{characters,outfits,actions,scenes,styles,detailers,looks,checkpoints}` — AJAX: list items without cover images (sorted by display name)
- `POST /generate_missing` — batch generate covers for all characters missing one (uses job queue)
- `POST /clear_all_covers` / `clear_all_{outfit,action,scene,style,detailer,look,checkpoint}_covers`
- `GET /gallery` — global image gallery browsing `static/uploads/`
- `GET/POST /settings` — LLM provider configuration
- `POST /resource/<category>/<slug>/delete` — soft (JSON only) or hard (JSON + safetensors) delete

---

## Frontend

- Bootstrap 5.3 (CDN). Custom styles in `static/style.css`.
- All templates extend `templates/layout.html`. The base layout provides:
  - `{% block content %}` — main page content
  - `{% block scripts %}` — additional JS at end of body
  - Navbar with links to all sections
  - Global default checkpoint selector (saves to session via AJAX)
  - Resource delete modal (soft/hard) shared across gallery pages
  - `initJsonEditor(saveUrl)` — shared JSON editor modal (simple form + raw textarea tabs)
- Context processors inject `all_checkpoints`, `default_checkpoint_path`, and `COMFYUI_WS_URL` into every template. The `random_gen_image(category, slug)` template global returns a random image path from `static/uploads/<category>/<slug>/` for use as a fallback cover when `image_path` is not set.
- **No `{% block head %}` exists** in layout.html — do not try to use it.
- Generation is async: JS submits the form via AJAX (`X-Requested-With: XMLHttpRequest`), receives a `{"job_id": ...}` response, then polls `/api/queue/<job_id>/status` every ~1.5 seconds until `status == "done"`. The server-side worker handles all ComfyUI polling and image saving via the `_make_finalize()` callback. There are no client-facing finalize HTTP routes.
- **Batch generation** (library pages): Uses a two-phase pattern:
  1. **Queue phase**: All jobs are submitted upfront via sequential fetch calls, collecting job IDs
  2. **Poll phase**: All jobs are polled concurrently via `Promise.all()`, updating UI as each completes
  3. **Progress tracking**: Displays currently processing items in real-time using a `Set` to track active jobs
  4. **Sorting**: All batch operations sort items by display `name` (not `filename`) for better UX
- **Fallback covers** (library pages): When a resource has no assigned `image_path` but has generated images in its upload folder, a random image is shown at 50% opacity (CSS class `fallback-cover`). The image changes on each page load. Resources with no generations show "No Image".

---

## LLM Integration

### System Prompts
Text files in `data/prompts/` define JSON output schemas for LLM-generated entries:
- `character_system.txt` — character JSON schema
- `outfit_system.txt` — outfit JSON schema
- `action_system.txt`, `scene_system.txt`, `style_system.txt`, `detailer_system.txt`, `look_system.txt`, `checkpoint_system.txt`
- `preset_system.txt` — preset JSON schema
- `regenerate_tags_system.txt` — tag regeneration schema (all per-category tag structures)

Used by: character/outfit/action/scene/style create forms, bulk_create routes, and tag regeneration. All system prompts include NSFW awareness preamble.

### Danbooru MCP Tools
The LLM loop in `call_llm()` provides three tools via a Docker-based MCP server (`danbooru-mcp:latest`):
- `search_tags(query, limit, category)` — prefix search
- `validate_tags(tags)` — exact-match validation
- `suggest_tags(partial, limit, category)` — autocomplete

The LLM uses these to verify and discover correct Danbooru-compatible tags for prompts.

All system prompts (`character_system.txt`, `outfit_system.txt`, `action_system.txt`, `scene_system.txt`, `style_system.txt`, `detailer_system.txt`, `look_system.txt`, `checkpoint_system.txt`) instruct the LLM to use these tools before finalising any tag values. `checkpoint_system.txt` applies them specifically to the `base_positive` and `base_negative` fields.

---

## Tagging System

Tags are **semantic metadata** for organizing and filtering resources. They are **not injected into generation prompts** — tags are purely for the UI (search, filtering, categorization).

### Tag Schema Per Category

| Category | Tag fields | Example |
|----------|-----------|---------|
| Character | `origin_series`, `origin_type`, `nsfw` | `{"origin_series": "Final Fantasy VII", "origin_type": "game", "nsfw": false}` |
| Look | `origin_series`, `origin_type`, `nsfw` | same as Character |
| Outfit | `outfit_type`, `nsfw` | `{"outfit_type": "Uniform", "nsfw": false}` |
| Action | `participants`, `nsfw` | `{"participants": "1girl, 1boy", "nsfw": true}` |
| Style | `style_type`, `nsfw` | `{"style_type": "Anime", "nsfw": false}` |
| Scene | `scene_type`, `nsfw` | `{"scene_type": "Indoor", "nsfw": false}` |
| Detailer | `associated_resource`, `adetailer_targets`, `nsfw` | `{"associated_resource": "skin", "adetailer_targets": ["face", "hands"], "nsfw": false}` |
| Checkpoint | `art_style`, `base_model`, `nsfw` | `{"art_style": "anime", "base_model": "Illustrious", "nsfw": false}` |

### Favourite / NSFW Columns

- `is_favourite` — DB-only boolean. Toggled via `POST /<category>/<slug>/favourite`. Not stored in JSON (user preference, not asset metadata).
- `is_nsfw` — DB column **and** `tags.nsfw` in JSON. Synced from JSON on rescan via `_sync_nsfw_from_tags()`. Editable from edit pages.

### Library Filtering

All library index pages support query params:
- `?favourite=on` — show only favourites
- `?nsfw=sfw|nsfw|all` — filter by NSFW status
- Results are ordered by `is_favourite DESC, name ASC` (favourites sort first).

### Gallery Image Sidecar Files

Gallery images can have per-image favourite/NSFW metadata stored in sidecar JSON files at `{image_path}.json` (e.g. `static/uploads/characters/tifa/gen_123.png.json`). Sidecar schema: `{"is_favourite": bool, "is_nsfw": bool}`.

---

## LoRA File Paths

LoRA filenames in JSON are stored as paths relative to ComfyUI's `models/lora/` root:

| Category | Path prefix | Example |
|----------|-------------|---------|
| Character / Look | `Illustrious/Looks/` | `Illustrious/Looks/tifa_v2.safetensors` |
| Outfit | `Illustrious/Clothing/` | `Illustrious/Clothing/maid.safetensors` |
| Action | `Illustrious/Poses/` | `Illustrious/Poses/sitting.safetensors` |
| Style | `Illustrious/Styles/` | `Illustrious/Styles/watercolor.safetensors` |
| Detailer | `Illustrious/Detailers/` | `Illustrious/Detailers/skin.safetensors` |
| Scene | `Illustrious/Backgrounds/` | `Illustrious/Backgrounds/beach.safetensors` |

Checkpoint paths: `Illustrious/<filename>.safetensors` or `Noob/<filename>.safetensors`.

Absolute paths on disk:
- Checkpoints: `/mnt/alexander/AITools/Image Models/Stable-diffusion/{Illustrious,Noob}/`
- LoRAs: `/mnt/alexander/AITools/Image Models/lora/Illustrious/{Looks,Clothing,Poses,Styles,Detailers,Backgrounds}/`

---

## Adding a New Category

To add a new content category (e.g. "Poses" as a separate concept from Actions), the pattern is:

1. **Model** (`models.py`): Add a new SQLAlchemy model with the standard fields.
2. **Sync function** (`services/sync.py`): Add `sync_newcategory()` following the pattern of `sync_outfits()`.
3. **Data directory** (`app.py`): Add `app.config['NEWCATEGORY_DIR'] = 'data/newcategory'`.
4. **Routes** (`routes/newcategory.py`): Create a new route module with a `register_routes(app)` function. Implement index, detail, edit, generate, replace_cover_from_preview, upload, save_defaults, clone, rescan routes. Follow `routes/outfits.py` or `routes/scenes.py` exactly.
5. **Route registration** (`routes/__init__.py`): Import and call `newcategory.register_routes(app)`.
6. **Templates**: Create `templates/newcategory/{index,detail,edit,create}.html` extending `layout.html`.
7. **Nav**: Add link to navbar in `templates/layout.html`.
8. **Startup** (`app.py`): Import and call `sync_newcategory()` in the `with app.app_context()` block.
9. **Generator page**: Add to `routes/generator.py`, `services/prompts.py` `build_extras_prompt()`, and `templates/generator.html` accordion.

---

## Session Keys

The Flask filesystem session stores:
- `default_checkpoint` — checkpoint_path string for the global default
- `prefs_{slug}` — selected_fields list for character detail page
- `preview_{slug}` — relative image path of last character preview
- `prefs_outfit_{slug}`, `preview_outfit_{slug}`, `char_outfit_{slug}` — outfit detail state
- `prefs_action_{slug}`, `preview_action_{slug}`, `char_action_{slug}` — action detail state
- `prefs_scene_{slug}`, `preview_scene_{slug}`, `char_scene_{slug}` — scene detail state
- `prefs_detailer_{slug}`, `preview_detailer_{slug}`, `char_detailer_{slug}`, `action_detailer_{slug}`, `extra_pos_detailer_{slug}`, `extra_neg_detailer_{slug}` — detailer detail state (selected fields, preview image, character, action LoRA, extra positive prompt, extra negative prompt)
- `prefs_style_{slug}`, `preview_style_{slug}`, `char_style_{slug}` — style detail state
- `prefs_look_{slug}`, `preview_look_{slug}` — look detail state

---

## Running the App

### Directly (development)

```bash
cd /mnt/alexander/Projects/character-browser
source venv/bin/activate
python app.py
```

The app runs in debug mode on port 5000 by default. ComfyUI must be running at `http://127.0.0.1:8188`.

The DB is initialised and all sync functions are called inside `with app.app_context():` at the bottom of `app.py` before `app.run()`.

### Docker

```bash
docker compose up -d
```

The compose file (`docker-compose.yml`) runs two services:
- **`danbooru-mcp`** — built from `https://git.liveaodh.com/aodhan/danbooru-mcp.git`; the MCP tag-search container used by `call_llm()`.
- **`app`** — the Flask app, exposed on host port **5782** → container port 5000.

Key environment variables set by compose:
- `COMFYUI_URL=http://10.0.0.200:8188` — points at ComfyUI on the Docker host network.
- `SKIP_MCP_AUTOSTART=true` — disables the app's built-in danbooru-mcp launch logic (compose manages it).

Volumes mounted into the app container:
- `./data`, `./static/uploads`, `./instance`, `./flask_session` — persistent app data.
- `/Volumes/ImageModels:/ImageModels:ro` — model files for checkpoint/LoRA scanning (**requires Docker Desktop file sharing enabled for `/Volumes/ImageModels`**).
- `/var/run/docker.sock` — Docker socket so the app can exec danbooru-mcp tool containers.

---

## Common Pitfalls

- **SQLAlchemy JSON mutation**: After modifying a JSON column dict in place, always call `flag_modified(obj, "data")` or the change won't be detected.
- **Dual write**: Every edit route writes back to both the DB (`db.session.commit()`) and the JSON file on disk. Both must be kept in sync.
- **Slug generation**: `re.sub(r'[^a-zA-Z0-9_]', '', id)` — note this removes hyphens and dots, not just replaces them. Character IDs like `yuna_(ff10)` become slug `yunaffx10`. This is intentional.
- **Checkpoint slugs use underscore replacement**: `re.sub(r'[^a-zA-Z0-9_]', '_', ...)` (replaces with `_`, not removes) to preserve readability in paths.
- **LoRA chaining**: If a LoRA node has no LoRA (name is empty/None), the node is skipped and `model_source`/`clip_source` pass through unchanged. Do not set the node inputs for skipped nodes.
- **AJAX detection**: `request.headers.get('X-Requested-With') == 'XMLHttpRequest'` determines whether to return JSON or redirect.
- **Session must be marked modified for JSON responses**: After setting session values in AJAX-responding routes, set `session.modified = True`.
- **Detailer `prompt` is a list**: The `prompt` field in detailer JSON is stored as a list of strings (e.g. `["detailed skin", "pores"]`), not a plain string. In generate routes, the detailer prompt is injected directly into `prompts['main']` after `build_prompt()` returns (not via tags or `build_prompt` itself).
- **`_make_finalize` action semantics**: Pass `action=None` when the route should always update the DB cover (e.g. batch generate, checkpoint generate). Pass `action=request.form.get('action')` for routes that support both "preview" (no DB update) and "replace" (update DB). The factory skips the DB write when `action` is truthy and not `"replace"`.
- **LLM queue runs without request context**: `_enqueue_task()` callbacks execute in a background thread with only `app.app_context()`. Do not access `flask.request`, `flask.session`, or other request-scoped objects inside `task_fn`. Use `has_request_context()` guard if code is shared between HTTP handlers and background tasks.
- **Tags are metadata only**: Tags (`data['tags']`) are never injected into generation prompts. They are purely for UI filtering and search. The old pattern of `parts.extend(data.get('tags', []))` in prompt building has been removed.