Add Docker support and refactor prompt/queue internals

- Add Dockerfile, docker-compose.yml, .dockerignore for containerised deployment
- Extract _resolve_character(), _ensure_character_fields(), _append_background() helpers to eliminate repeated inline character-field injection and background-tag patterns across all secondary-category generate routes
- Add _IDENTITY_KEYS / _WARDROBE_KEYS constants
- Fix build_extras_prompt() bug: detailer prompt (a list) was being appended as a single item instead of extended
- Replace all per-route _finalize closures with _make_finalize() factory, reducing duplication across 10 generate routes
- Add _prune_job_history() called each worker loop iteration to prevent unbounded memory growth
- Remove 10 orphaned legacy finalize_generation HTTP routes and check_status route (superseded by job queue API since job-queue branch)
- Remove one-time migration scripts (migrate_actions, migrate_detailers, migrate_lora_weight_range, migrate_wardrobe)
- Update CLAUDE.md and README.md to document new helpers, queue architecture, and Docker deployment

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

CLAUDE.md

@@ -96,6 +96,30 @@ Cross-deduplicates tags between the positive and negative prompt strings. For ea
 
 Called as the last step of `_prepare_workflow`, after `_apply_checkpoint_settings` has added `base_positive`/`base_negative`, so it operates on fully-assembled prompts.
 
+### `_IDENTITY_KEYS` / `_WARDROBE_KEYS` (module-level constants)
+Lists of canonical field names for the `identity` and `wardrobe` sections. Used by `_ensure_character_fields()` to avoid hard-coding key lists in every route.
+
+### `_resolve_character(character_slug)`
+Returns a `Character` ORM object for a given slug string. Handles the `"__random__"` sentinel by selecting a random character. Returns `None` if `character_slug` is falsy or no match is found. Every route that accepts an optional character dropdown (outfit, action, style, scene, detailer, checkpoint, look generate routes) uses this instead of an inline if/elif block.
+
+### `_ensure_character_fields(character, selected_fields, include_wardrobe=True, include_defaults=False)`
+Mutates `selected_fields` in place, appending any populated identity, wardrobe, and optional defaults keys that are not already present. Ensures `"special::name"` is always included. Called in every secondary-category generate route immediately after `_resolve_character()` to guarantee the character's essential fields are sent to `build_prompt`.
+
+### `_append_background(prompts, character=None)`
+Appends a `"<primary_color> simple background"` tag (or plain `"simple background"` if no primary color) to `prompts['main']`. Called in outfit, action, style, detailer, and checkpoint generate routes instead of repeating the same inline string construction.
+
+### `_make_finalize(category, slug, db_model_class=None, action=None)`
+Factory function that returns a `_finalize(comfy_prompt_id, job)` callback closure. The closure:
+1. Calls `get_history()` and `get_image()` to retrieve the generated image from ComfyUI.
+2. Saves the image to `static/uploads/<category>/<slug>/gen_<timestamp>.png`.
+3. Sets `job['result']` with `image_url` and `relative_path`.
+4. If `db_model_class` is provided **and** (`action` is `None` or `action == 'replace'`), updates the ORM object's `image_path` and commits.
+
+All `generate` routes pass a `_make_finalize(...)` call as the finalize argument to `_enqueue_job()` instead of defining an inline closure.
+
+### `_prune_job_history(max_age_seconds=3600)`
+Removes entries from `_job_history` that are in a terminal state (`done`, `failed`, `removed`) and older than `max_age_seconds`. Called at the end of every worker loop iteration to prevent unbounded memory growth.
+
 ### `_queue_generation(character, action, selected_fields, client_id)`
 Convenience wrapper for character detail page generation. Loads workflow, calls `build_prompt`, calls `_prepare_workflow`, calls `queue_prompt`.
 
@@ -225,8 +249,7 @@ Checkpoint JSONs are keyed by `checkpoint_path`. If no JSON exists for a discove
 ### Characters
 - `GET /` — character gallery (index)
 - `GET /character/<slug>` — character detail with generation UI
-- `POST /character/<slug>/generate` — queue generation (AJAX or form)
-- `POST /character/<slug>/finalize_generation/<prompt_id>` — retrieve image from ComfyUI
+- `POST /character/<slug>/generate` — queue generation (AJAX or form); returns `{"job_id": ...}`
 - `POST /character/<slug>/replace_cover_from_preview` — promote preview to cover
 - `GET/POST /character/<slug>/edit` — edit character data
 - `POST /character/<slug>/upload` — upload cover image
@@ -239,8 +262,7 @@ Checkpoint JSONs are keyed by `checkpoint_path`. If no JSON exists for a discove
 Each category follows the same URL pattern:
 - `GET /<category>/` — gallery
 - `GET /<category>/<slug>` — detail + generation UI
-- `POST /<category>/<slug>/generate` — queue generation
-- `POST /<category>/<slug>/finalize_generation/<prompt_id>` — retrieve image
+- `POST /<category>/<slug>/generate` — queue generation; returns `{"job_id": ...}`
 - `POST /<category>/<slug>/replace_cover_from_preview`
 - `GET/POST /<category>/<slug>/edit`
 - `POST /<category>/<slug>/upload`
@@ -254,15 +276,13 @@ Each category follows the same URL pattern:
 - `GET /looks` — gallery
 - `GET /look/<slug>` — detail
 - `GET/POST /look/<slug>/edit`
-- `POST /look/<slug>/generate`
-- `POST /look/<slug>/finalize_generation/<prompt_id>`
+- `POST /look/<slug>/generate` — queue generation; returns `{"job_id": ...}`
 - `POST /look/<slug>/replace_cover_from_preview`
 - `GET/POST /look/create`
 - `POST /looks/rescan`
 
 ### Generator (Mix & Match)
 - `GET/POST /generator` — freeform generator with multi-select accordion UI
-- `POST /generator/finalize/<slug>/<prompt_id>` — retrieve image
 - `POST /generator/preview_prompt` — AJAX: preview composed prompt without generating
 
 ### Checkpoints
@@ -271,11 +291,16 @@ Each category follows the same URL pattern:
 - `POST /checkpoint/<slug>/save_json`
 - `POST /checkpoints/rescan`
 
+### Job Queue API
+All generation routes use the background job queue. Frontend polls:
+- `GET /api/queue/<job_id>/status` — returns `{"status": "pending"|"running"|"done"|"failed", "result": {...}}`
+
+Image retrieval is handled server-side by the `_make_finalize()` callback; there are no separate client-facing finalize routes.
+
 ### Utilities
 - `POST /set_default_checkpoint` — save default checkpoint to session
-- `GET /check_status/<prompt_id>` — poll ComfyUI for completion
 - `GET /get_missing_{characters,outfits,actions,scenes}` — AJAX: list items without cover images
-- `POST /generate_missing` — batch generate covers for characters
+- `POST /generate_missing` — batch generate covers for all characters missing one (uses job queue)
 - `POST /clear_all_covers` / `clear_all_{outfit,action,scene}_covers`
 - `GET /gallery` — global image gallery browsing `static/uploads/`
 - `GET/POST /settings` — LLM provider configuration
@@ -295,7 +320,7 @@ Each category follows the same URL pattern:
   - `initJsonEditor(saveUrl)` — shared JSON editor modal (simple form + raw textarea tabs)
 - Context processors inject `all_checkpoints`, `default_checkpoint_path`, and `COMFYUI_WS_URL` into every template.
 - **No `{% block head %}` exists** in layout.html — do not try to use it.
-- Generation is async: JS submits the form via AJAX (`X-Requested-With: XMLHttpRequest`), receives a `prompt_id`, opens a WebSocket to ComfyUI to show progress, then calls the finalize endpoint to save the image.
+- Generation is async: JS submits the form via AJAX (`X-Requested-With: XMLHttpRequest`), receives a `{"job_id": ...}` response, then polls `/api/queue/<job_id>/status` every ~1.5 seconds until `status == "done"`. The server-side worker handles all ComfyUI polling and image saving via the `_make_finalize()` callback. There are no client-facing finalize HTTP routes.
 
 ---
 
@@ -374,6 +399,8 @@ The Flask filesystem session stores:
 
 ## Running the App
 
+### Directly (development)
+
 ```bash
 cd /mnt/alexander/Projects/character-browser
 source venv/bin/activate
@@ -384,6 +411,25 @@ The app runs in debug mode on port 5000 by default. ComfyUI must be running at `
 
 The DB is initialised and all sync functions are called inside `with app.app_context():` at the bottom of `app.py` before `app.run()`.
 
+### Docker
+
+```bash
+docker compose up -d
+```
+
+The compose file (`docker-compose.yml`) runs two services:
+- **`danbooru-mcp`** — built from `https://git.liveaodh.com/aodhan/danbooru-mcp.git`; the MCP tag-search container used by `call_llm()`.
+- **`app`** — the Flask app, exposed on host port **5782** → container port 5000.
+
+Key environment variables set by compose:
+- `COMFYUI_URL=http://10.0.0.200:8188` — points at ComfyUI on the Docker host network.
+- `SKIP_MCP_AUTOSTART=true` — disables the app's built-in danbooru-mcp launch logic (compose manages it).
+
+Volumes mounted into the app container:
+- `./data`, `./static/uploads`, `./instance`, `./flask_session` — persistent app data.
+- `/Volumes/ImageModels:/ImageModels:ro` — model files for checkpoint/LoRA scanning (**requires Docker Desktop file sharing enabled for `/Volumes/ImageModels`**).
+- `/var/run/docker.sock` — Docker socket so the app can exec danbooru-mcp tool containers.
+
 ---
 
 ## Common Pitfalls
@@ -396,3 +442,4 @@ The DB is initialised and all sync functions are called inside `with app.app_con
 - **AJAX detection**: `request.headers.get('X-Requested-With') == 'XMLHttpRequest'` determines whether to return JSON or redirect.
 - **Session must be marked modified for JSON responses**: After setting session values in AJAX-responding routes, set `session.modified = True`.
 - **Detailer `prompt` is a list**: The `prompt` field in detailer JSON is stored as a list of strings (e.g. `["detailed skin", "pores"]`), not a plain string. When merging into `tags` for `build_prompt`, use `extend` for lists and `append` for strings — never append the list object itself or `", ".join()` will fail on the nested list item.
+- **`_make_finalize` action semantics**: Pass `action=None` when the route should always update the DB cover (e.g. batch generate, checkpoint generate). Pass `action=request.form.get('action')` for routes that support both "preview" (no DB update) and "replace" (update DB). The factory skips the DB write when `action` is truthy and not `"replace"`.