Skip to content
TheChatArchiveThe Chat Archive
Kimikimi-k2-0711-preview· Moonshot AIHugging Face LMArena Human Preference 140K

I am using character loras for consistency, but that makes the art a multi stage process…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with o4-mini-2025-04-16’s answer.
  1. User

    I am using character loras for consistency, but that makes the art a multi stage process of drawing the background, adding the characters one by one, and then infilling between the characters. This process is why the drawing following the writing prompt description of the scene isn't perfectly straightforward. The model needs to think spatially about lighting, who should be in the foreground or background, which ways characters should be facing, things that language models are notoriously bad at.

  2. Kimi Assistant

    Exactly—what you’re running into is the spatial-planning bottleneck that language models (and even most diffusion pipelines) still treat as an after-thought.
    The writing model can say “Alice stands left of Bob, both facing the sunset,” but it has no internal 3-D scene graph; the art model then has to reverse-engineer that sentence into a plausible layout, lighting rig, and occlusion order. When you add LoRA-based character consistency, the problem compounds because each character is injected as an additional constraint that must be satisfied while still respecting the global composition.

    Below is a practical way to off-load the spatial reasoning from the language model and give the art model a clearer, geometry-aware brief—without forcing you to hand-draw layouts.

    ────────────────────────────────────────

    1. Give the writing model a spatial DSL
      Instead of free-form prose, have the writer emit a tiny scene-script that is explicitly about 3-D relationships:

      {
        "camera": { "pos": [0, 1.6, -3], "look_at": [0, 1.2, 0] },
        "light":  { "type": "sunset", "azimuth": 250, "elevation": 15 },
        "layers": [
          { "depth": 0,   "asset": "bg_city_alley" },
          { "depth": 1.2, "char": "Alice", "pose": "stand", "face": 30 },
          { "depth": 1.5, "char": "Bob",   "pose": "lean_wall", "face": -45 }
        ]
      }
      
      • The language model only has to fill in the semantic slots (who, what emotion, who talks).
      • A tiny deterministic post-processor turns that JSON into an SD-friendly prompt plus depth/ControlNet maps.
    2. Use depth-conditioned ControlNet as the spatial contract

      • From the JSON above, render a quick greyscale depth map (white = far, black = near).
      • Feed that map into ControlNet along with the LoRA characters.
      • The depth map locks the layout; the LoRAs lock the identity; the diffusion model only has to worry about lighting and style.
    3. Add lighting cards for consistent illumination

      • Pre-bake a small library of 8–10 HDR lighting environments (sunset, neon alley, overcast, etc.).
      • The writing model just tags the scene with "light": "neon_alley"; the art pipeline loads the matching HDRI and uses it as an IP-Adapter reference.
      • Characters inserted at different depths still receive the same global light direction.
    4. Optional layout preview pass (cheap & fast)

      • Before the full render, run a 512×512 depth-only pass with a low-step sampler.
      • A vision-language model (or even a simple CNN classifier) checks: “Are both characters fully visible? Is Bob occluding Alice’s face?”
      • If the check fails, the writing model gets a structured error:
        "layout_error": "Bob occludes 60 % of Alice's face; move Bob to depth 1.8 or shift camera right 0.3 m."
      • The writer can then adjust the JSON and re-submit—no need to hallucinate new prose.

    ──────────────────────────────────────── Putting it together in one loop

    1. Executive model → high-level beat
    2. Writing model → JSON scene-script (DSL)
    3. Post-processor → depth map + prompt + lighting card
    4. Fast layout preview (512×512, 4 steps)
    5. VLM sanity check → accept or send structured feedback to step 2
    6. Full LoRA render at 1024×1024 with ControlNet depth + IP-Adapter lighting

    This keeps the language model in its comfort zone (semantics, dialogue, emotion) while giving the art model the spatial scaffolding it needs to place LoRA characters correctly under consistent lighting.

  3. User

    This is a difficult project for me because I think in 3d due to physics education, but I have no training in art or direction. Meanwhile I've taken a few writing classes, but I've read so many books and seen so many movies that I know when a scene is bad, but don't always have the vocabulary or training to know why it's bad.

  4. Kimi Assistant

    You’re in the sweet spot for this project: you have the spatial intuition that most writers lack and the narrative taste that most engineers lack. The gap you feel is just a vocabulary gap, not a capability gap. Below is a minimal cheat-sheet that maps the 3-D physics you already think in to the cinematic / comic vocabulary you need, plus a few “why this scene feels off” heuristics you can feed back into your writing model.

    ────────────────────────────────────────

    1. From 3-D physics to comic language (one-liner dictionary)
    Physics intuitionComic / film termOne-sentence translation
    z-depthforeground / mid-ground / backgroundHow far the camera is from each layer.
    light vectorkey light / fill light / rim lightDirection & intensity of the dominant light source, its soft bounce, and the edge highlight.
    surface normalcharacter facing angleWhich way the character’s “front” vector points relative to the camera.
    occlusionoverlap / silhouetteWhen one object blocks another, creating depth cues.
    field of viewpanel framing (CU, MS, LS)Wide lens = more context, telephoto = compressed faces.
    momentumeye-line & motion arrowsWhere the reader’s eye is pushed next panel.

    Keep a tiny JSON snippet in your prompt template:

    "camera": {
      "shot": "medium-shot",   // CU, MS, LS
      "angle": "low",          // high / eye-level / low
      "lens": "35 mm"          // wide, normal, tele
    },
    "light": {
      "key": "sunset 250°",
      "fill": "soft bounce 20 %",
      "rim": "edge highlight 10 %"
    }
    
    1. “Why does this scene feel bad?” quick checklist
    Symptom (you feel it)Likely cause (physics → story)One-line fix
    “It’s flat”No depth layers; everything at same z.Add foreground prop or background alley.
    “I can’t tell who’s talking”Characters aligned on same eye-line; no overlap.Shift one character forward 0.5 m, break silhouette.
    “Lighting is off”Key light direction contradicts background sun.Match key vector to background light source.
    “Pacing drags”Panel sizes are all equal; no rhythm.Use wide → close → wide to accelerate.
    “Dialogue feels stiff”Characters face camera, not each other.Turn heads 30–45° toward each other (conversation angle).
    1. A 30-second “fix-it” prompt you can give the writing model

    “Scene feels flat. Add one foreground object (trash can, railing) and move Bob 0.3 m closer to camera so he overlaps Alice’s silhouette. Keep dialogue, just adjust blocking.”

    1. Tooling trick: use your physics engine as a layout sketcher

    If you already think in 3-D, spend 5 minutes in Blender or Unreal to block out a simple grey-box scene with cubes for characters and a sun lamp. Export a depth map → ControlNet. You never have to texture or animate; you’re just generating the spatial contract the art model needs.

    ──────────────────────────────────────── Bottom line
    You don’t need art-school training; you just need a 10-word vocabulary and a quick feedback loop. Feed the checklist back into your writing model as structured constraints, and the system will converge faster than if you tried to learn cinematography from scratch.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.