character-browser/plans/APP_REFACTOR.md

APP_REFACTOR.md — Split app.py into Modules

Goal

Split the 8,478-line app.py into a clean module structure using Flask Blueprints for routes and plain Python modules for services/utilities. The JSON files and DB remain the source of truth — no data migration needed.

---

Target Structure

app.py                          # ~80 lines: Flask init, config, register blueprints, startup
models.py                       # Unchanged
utils.py                        # Pure helpers (no Flask/DB deps beyond current_app)
services/
  __init__.py
  comfyui.py                    # ComfyUI HTTP client
  workflow.py                   # Workflow building + checkpoint settings
  prompts.py                    # Prompt building + dedup
  llm.py                        # LLM integration + MCP tool calls
  mcp.py                        # MCP/Docker server lifecycle
  sync.py                       # All sync_*() functions
  job_queue.py                  # Background job queue + worker thread
  file_io.py                    # LoRA/checkpoint scanning, file uploads, image saving
routes/
  __init__.py                   # register_blueprints() helper
  characters.py                 # Character CRUD + generate + outfit management
  outfits.py                    # Outfit routes
  actions.py                    # Action routes
  styles.py                     # Style routes
  scenes.py                     # Scene routes
  detailers.py                  # Detailer routes
  checkpoints.py                # Checkpoint routes
  looks.py                      # Look routes
  presets.py                    # Preset routes
  generator.py                  # Generator mix-and-match page
  gallery.py                    # Gallery browsing + image deletion
  settings.py                   # Settings page + status API + context processors
  strengths.py                  # Strengths gallery system
  transfer.py                   # Resource transfer system
  queue_api.py                  # /api/queue/* endpoints

---

Module Contents — Exact Function Mapping

app.py (entry point, ~80 lines)

Keep only:

• Flask app creation, config, extensions (SQLAlchemy, Session)
• Logging setup
• from routes import register_blueprints; register_blueprints(app)
• with app.app_context(): block (DB init, migrations, sync calls, worker start)

utils.py

Function	Current Line	Notes
parse_orientation()	768	Pure logic
_resolve_lora_weight()	833	Pure logic
allowed_file()	765	Pure logic
_IDENTITY_KEYS	850	Constant
_WARDROBE_KEYS	851	Constant
ALLOWED_EXTENSIONS	728	Constant
_LORA_DEFAULTS	730	Constant

services/job_queue.py

Function/Global	Current Line	Notes
_job_queue_lock	75	Global
_job_queue	76	Global
_job_history	77	Global
_queue_worker_event	78	Global
_enqueue_job()	81	Needs comfyui.queue_prompt, comfyui.get_history
_queue_worker()	102	The background thread loop
_make_finalize()	227	Needs comfyui.get_history, comfyui.get_image, DB models
_prune_job_history()	292	Pure logic on globals

services/comfyui.py

Function	Current Line	Notes
queue_prompt()	1083	HTTP POST to ComfyUI
get_history()	1116	HTTP GET + polling
get_image()	1148	HTTP GET for image bytes
_ensure_checkpoint_loaded()	1056	HTTP POST to load checkpoint

All use current_app.config['COMFYUI_URL'] — pass URL as param or read from config.

services/workflow.py

Function	Current Line	Notes
_prepare_workflow()	3119	Core workflow wiring
_apply_checkpoint_settings()	6445	Checkpoint-specific settings
_log_workflow_prompts()	3030	Logging helper
_build_style_workflow()	5089	Style-specific workflow builder
_build_checkpoint_workflow()	6501	Checkpoint-specific workflow builder
_queue_scene_generation()	5603	Scene generation helper
_queue_detailer_generation()	6091	Detailer generation helper
_get_default_checkpoint()	3273	Reads session + DB

services/prompts.py

Function	Current Line	Notes
build_prompt()	933	Core prompt builder
build_extras_prompt()	2104	Generator page prompt builder
_dedup_tags()	797	Tag deduplication
_cross_dedup_prompts()	808	Cross-dedup positive/negative
_build_strengths_prompts()	7577	Strengths-specific prompt builder
_get_character_data_without_lora()	7563	Helper for strengths
_resolve_character()	853	Resolve slug → Character
_ensure_character_fields()	861	Mutate selected_fields
_append_background()	889	Add background tag

services/llm.py

Function/Constant	Current Line	Notes
DANBOORU_TOOLS	1844	Tool definitions constant
call_mcp_tool()	1904	Sync MCP tool call
load_prompt()	1911	Load system prompt from file
call_llm()	1918	LLM chat completion + tool loop

services/mcp.py

Function	Current Line	Notes
_ensure_mcp_repo()	423	Clone danbooru-mcp repo
ensure_mcp_server_running()	457	Start danbooru-mcp container
_ensure_character_mcp_repo()	496	Clone character-mcp repo
ensure_character_mcp_server_running()	530	Start character-mcp container

services/sync.py

Function	Current Line	Notes
sync_characters()	1158
sync_outfits()	1218
ensure_default_outfit()	1276
sync_looks()	1360
sync_presets()	1415
sync_actions()	1534
sync_styles()	1591
sync_detailers()	1648
sync_scenes()	1705
sync_checkpoints()	1776
_default_checkpoint_data()	1762
_resolve_preset_entity()	1484	Used by preset routes too
_resolve_preset_fields()	1494	Used by preset routes too
_PRESET_ENTITY_MAP	1472	Constant

services/file_io.py

Function	Current Line	Notes
get_available_loras()	739	Reads Settings + filesystem
get_available_checkpoints()	750	Reads Settings + filesystem
_count_look_assignments()	895	DB query
_count_outfit_lora_assignments()	907	DB query
_scan_gallery_images()	7233	Filesystem scan
_enrich_with_names()	7276	DB lookups for gallery
_parse_comfy_png_metadata()	7340	PNG metadata parsing

routes/__init__.py

def register_blueprints(app):
    from routes.characters import bp as characters_bp
    from routes.outfits import bp as outfits_bp
    # ... etc for all route modules
    app.register_blueprint(characters_bp)
    app.register_blueprint(outfits_bp)
    # ...

routes/characters.py

Route	Current Line
GET / (index)	2093
POST /rescan	2098
GET /character/<slug>	2289
GET/POST /character/<slug>/transfer	2305
GET/POST /create	2465
GET/POST /character/<slug>/edit	2718
POST /character/<slug>/outfit/* (6 routes)	2800–2988
POST /character/<slug>/upload	2989
POST /character/<slug>/replace_cover_from_preview	3018
POST /character/<slug>/generate	3344
POST /character/<slug>/save_defaults	3379
GET /get_missing_characters	3303
POST /clear_all_covers	3308
POST /generate_missing	3316

routes/outfits.py

Route	Current Line
GET /outfits	3844
POST /outfits/rescan	3850
POST /outfits/bulk_create	3856
GET /outfit/<slug>	3966
GET/POST /outfit/<slug>/edit	3991
POST /outfit/<slug>/upload	4062
POST /outfit/<slug>/generate	4091
POST /outfit/<slug>/replace_cover_from_preview	4187
GET/POST /outfit/create	4199
POST /outfit/<slug>/save_defaults	4321
POST /outfit/<slug>/clone	4330
POST /outfit/<slug>/save_json	4380
Helper: _get_linked_characters_for_outfit()	3956

routes/actions.py

Route	Current Line
GET /actions	4398
POST /actions/rescan	4403
GET /action/<slug>	4409
GET/POST /action/<slug>/edit	4430
POST /action/<slug>/upload	4501
POST /action/<slug>/generate	4530
POST /action/<slug>/replace_cover_from_preview	4706
POST /action/<slug>/save_defaults	4718
POST /actions/bulk_create	4727
GET/POST /action/create	4831
POST /action/<slug>/clone	4905
POST /action/<slug>/save_json	4947
GET /get_missing_actions	3401
POST /clear_all_action_covers	3406

routes/styles.py

Route	Current Line
GET /styles	4965
POST /styles/rescan	4970
GET /style/<slug>	4976
GET/POST /style/<slug>/edit	4997
POST /style/<slug>/upload	5060
POST /style/<slug>/generate	5142
POST /style/<slug>/save_defaults	5185
POST /style/<slug>/replace_cover_from_preview	5194
GET /get_missing_styles	5206
POST /clear_all_style_covers	5224
POST /styles/generate_missing	5232
POST /styles/bulk_create	5262
GET/POST /style/create	5358
POST /style/<slug>/clone	5412
POST /style/<slug>/save_json	5453

routes/scenes.py

Route	Current Line
GET /scenes	5471
POST /scenes/rescan	5476
GET /scene/<slug>	5482
GET/POST /scene/<slug>/edit	5503
POST /scene/<slug>/upload	5574
POST /scene/<slug>/generate	5680
POST /scene/<slug>/save_defaults	5721
POST /scene/<slug>/replace_cover_from_preview	5730
POST /scenes/bulk_create	5742
GET/POST /scene/create	5844
POST /scene/<slug>/clone	5902
POST /scene/<slug>/save_json	5943
GET /get_missing_scenes	3414
POST /clear_all_scene_covers	3419

routes/detailers.py

Route	Current Line
GET /detailers	5961
POST /detailers/rescan	5966
GET /detailer/<slug>	5972
GET/POST /detailer/<slug>/edit	5999
POST /detailer/<slug>/upload	6062
POST /detailer/<slug>/generate	6154
POST /detailer/<slug>/save_defaults	6206
POST /detailer/<slug>/replace_cover_from_preview	6215
POST /detailer/<slug>/save_json	6227
POST /detailers/bulk_create	6243
GET/POST /detailer/create	6340
GET /get_missing_detailers	5211
POST /clear_all_detailer_covers	5216

routes/checkpoints.py

Route	Current Line
GET /checkpoints	6396
POST /checkpoints/rescan	6401
GET /checkpoint/<slug>	6407
POST /checkpoint/<slug>/upload	6425
POST /checkpoint/<slug>/generate	6531
POST /checkpoint/<slug>/replace_cover_from_preview	6558
POST /checkpoint/<slug>/save_json	6570
GET /get_missing_checkpoints	6586
POST /clear_all_checkpoint_covers	6591
POST /checkpoints/bulk_create	6598

routes/looks.py

Route	Current Line
GET /looks	6702
POST /looks/rescan	6708
GET /look/<slug>	6714
GET/POST /look/<slug>/edit	6748
POST /look/<slug>/upload	6803
POST /look/<slug>/generate	6820
POST /look/<slug>/replace_cover_from_preview	6898
POST /look/<slug>/save_defaults	6910
POST /look/<slug>/generate_character	6919
POST /look/<slug>/save_json	7043
GET/POST /look/create	7060
GET /get_missing_looks	7103
POST /clear_all_look_covers	7108
POST /looks/bulk_create	7116

routes/presets.py

Route	Current Line
GET /presets	3429
GET /preset/<slug>	3435
POST /preset/<slug>/generate	3442
POST /preset/<slug>/replace_cover_from_preview	3595
POST /preset/<slug>/upload	3608
GET/POST /preset/<slug>/edit	3628
POST /preset/<slug>/save_json	3710
POST /preset/<slug>/clone	3728
POST /presets/rescan	3757
GET/POST /preset/create	3764
GET /get_missing_presets	3836

routes/generator.py

Route	Current Line
GET/POST /generator	2168
POST /generator/preview_prompt	2256

routes/gallery.py

Route	Current Line
GET /gallery	7296
GET /gallery/prompt-data	7414
POST /gallery/delete	7434
POST /resource/<category>/<slug>/delete	7457

Constants: GALLERY_CATEGORIES, _MODEL_MAP

routes/settings.py

Route/Function	Current Line
GET/POST /settings	2066
POST /set_default_checkpoint	588
GET /api/status/comfyui	625
GET /api/comfyui/loaded_checkpoint	638
GET /api/status/mcp	659
GET /api/status/llm	674
GET /api/status/character-mcp	712
POST /get_openrouter_models	2035
POST /get_local_models	2051
Context processor: inject_comfyui_ws_url()	569
Context processor: inject_default_checkpoint()	582

routes/strengths.py

Route	Current Line
POST /strengths/<category>/<slug>/generate	7796
GET /strengths/<category>/<slug>/list	7892
POST /strengths/<category>/<slug>/clear	7917
POST /strengths/<category>/<slug>/save_range	7939

Constants: _STRENGTHS_MODEL_MAP, _CATEGORY_LORA_NODES, _STRENGTHS_DATA_DIRS Helper: _prepare_strengths_workflow() (7690)

routes/transfer.py

Route	Current Line
GET/POST /resource/<category>/<slug>/transfer	8118

Constants: _RESOURCE_TRANSFER_MAP, _TRANSFER_TARGET_CATEGORIES Helper: _create_minimal_template() (8045)

routes/queue_api.py

Route	Current Line
GET /api/queue	306
GET /api/queue/count	323
POST /api/queue/<job_id>/remove	331
POST /api/queue/<job_id>/pause	348
POST /api/queue/clear	365
GET /api/queue/<job_id>/status	396

---

Dependency Graph

app.py
  ├── models.py (unchanged)
  ├── utils.py (no deps except stdlib)
  ├── services/
  │   ├── comfyui.py        ← utils (for config)
  │   ├── prompts.py        ← utils, models
  │   ├── workflow.py       ← comfyui, prompts, utils, models
  │   ├── llm.py            ← mcp (for tool calls)
  │   ├── mcp.py            ← (stdlib only: subprocess, docker)
  │   ├── sync.py           ← models, utils
  │   ├── job_queue.py      ← comfyui, models
  │   └── file_io.py        ← models, utils
  └── routes/
      ├── All route modules  ← services/*, utils, models
      └── (routes never import from other routes)

No circular imports: routes → services → utils/models. Services never import routes. Utils never imports services.

---

Migration Phases

Phase 1 — Extract services (no route changes)

Order matters — extract leaf dependencies first:

1. utils.py — Pure constants and helpers. Zero risk. Cut parse_orientation, _resolve_lora_weight, allowed_file, ALLOWED_EXTENSIONS, _LORA_DEFAULTS, _IDENTITY_KEYS, _WARDROBE_KEYS out of app.py. Add imports in app.py to keep everything working.

2. services/comfyui.py — queue_prompt, get_history, get_image, _ensure_checkpoint_loaded. These only use requests + config URL. Accept comfyui_url as parameter or read current_app.config.

3. services/mcp.py — _ensure_mcp_repo, ensure_mcp_server_running, _ensure_character_mcp_repo, ensure_character_mcp_server_running. Only uses subprocess/os/logging.

4. services/llm.py — DANBOORU_TOOLS, call_mcp_tool, load_prompt, call_llm. Depends on services/mcp for tool calls and models.Settings for config.

5. services/prompts.py — build_prompt, build_extras_prompt, _dedup_tags, _cross_dedup_prompts, _resolve_character, _ensure_character_fields, _append_background, _build_strengths_prompts, _get_character_data_without_lora. Depends on utils and models.

6. services/workflow.py — _prepare_workflow, _apply_checkpoint_settings, _log_workflow_prompts, _build_style_workflow, _build_checkpoint_workflow, _queue_scene_generation, _queue_detailer_generation, _get_default_checkpoint. Depends on comfyui, prompts, utils, models.

7. services/sync.py — All sync_*() + _default_checkpoint_data, _resolve_preset_entity, _resolve_preset_fields, _PRESET_ENTITY_MAP. Depends on models, utils.

8. services/file_io.py — get_available_loras, get_available_checkpoints, _count_look_assignments, _count_outfit_lora_assignments, _scan_gallery_images, _enrich_with_names, _parse_comfy_png_metadata. Depends on models, utils.

9. services/job_queue.py — Queue globals, _enqueue_job, _queue_worker, _make_finalize, _prune_job_history. Depends on comfyui, models. Extract last because the worker thread references many things.

After Phase 1: app.py still has all routes, but imports helpers from services/* and utils. Each service module is independently testable. The app should work identically.

Phase 2 — Extract routes into Blueprints

Order: Start with the smallest/most isolated, work toward characters (largest).

1. routes/queue_api.py — 6 routes, only depends on job_queue globals. Blueprint prefix: none (keeps /api/queue/*).

2. routes/settings.py — Settings page + status APIs + context processors. Register context processors on the app via app.context_processor in register_blueprints().

3. routes/gallery.py — Gallery + resource delete. Depends on file_io, models.

4. routes/transfer.py — Transfer system. Self-contained with its own constants.

5. routes/strengths.py — Strengths system. Self-contained with its own constants + workflow helper.

6. routes/generator.py — Generator page. Depends on prompts.build_extras_prompt.

7. routes/checkpoints.py — Smallest category. Good test case for the category pattern.

8. routes/presets.py — Preset CRUD. Depends on sync._resolve_preset_*.

9. routes/looks.py — Look CRUD + generate_character.

10. routes/detailers.py — Detailer CRUD.

11. routes/scenes.py — Scene CRUD.

12. routes/styles.py — Style CRUD.

13. routes/actions.py — Action CRUD.

14. routes/outfits.py — Outfit CRUD + linked characters helper.

15. routes/characters.py — Character CRUD + outfit management + transfer. Largest blueprint (~1300 lines). Do last since it has the most cross-cutting concerns.

After Phase 2: app.py is ~80 lines. All routes live in blueprints. Full functionality preserved.

Phase 3 — Verification & cleanup

1. Run the app, test every page manually (index, detail, generate, edit, clone, delete for each category).
2. Test batch generation, generator page, gallery, settings, strengths, transfer.
3. Remove any dead imports from app.py.
4. Update CLAUDE.md to reflect new file structure.

---

Blueprint Pattern Template

Each route blueprint follows this pattern:

from flask import Blueprint, render_template, request, redirect, url_for, flash, session, jsonify
from models import db, Outfit  # only what's needed
from services.workflow import _prepare_workflow
from services.prompts import build_prompt, _resolve_character, _ensure_character_fields, _append_background
from services.job_queue import _enqueue_job, _make_finalize
from services.file_io import get_available_loras
from services.llm import call_llm, load_prompt
from utils import allowed_file

import logging
logger = logging.getLogger('gaze')

bp = Blueprint('outfits', __name__)

@bp.route('/outfits')
def outfits_index():
    ...

---

Risk Mitigation

• Circular imports: Enforced by the dependency graph — routes → services → utils/models. If a service needs something from another service, import it at function level if needed.
• current_app vs app: Routes already use request, session, etc. which are context-local. Services that need app config use current_app.config[...] inside function bodies (not at module level).
• Thread safety: job_queue.py keeps the same threading globals. The worker thread is started in app.py's startup block, same as before.
• Session access: Only route functions access session. Services that currently read session (like _get_default_checkpoint) will stay in services/workflow.py and import session from flask — this is fine since they're only called from within request context.
• Testing: After each phase-1 extraction, verify the app starts and the affected functionality works before proceeding to the next module.

---

Lines of Code Estimate (per module)

Module	Approx Lines
app.py (final)	~100
utils.py	~120
services/comfyui.py	~120
services/mcp.py	~150
services/llm.py	~200
services/prompts.py	~350
services/workflow.py	~400
services/sync.py	~800
services/job_queue.py	~250
services/file_io.py	~250
routes/characters.py	~1300
routes/outfits.py	~550
routes/actions.py	~570
routes/styles.py	~500
routes/scenes.py	~500
routes/detailers.py	~450
routes/checkpoints.py	~350
routes/looks.py	~550
routes/presets.py	~450
routes/generator.py	~150
routes/gallery.py	~300
routes/settings.py	~250
routes/strengths.py	~400
routes/transfer.py	~200
routes/queue_api.py	~120
Total	~8,430