The hardest problem in turning a script into video isn’t making one good shot. It’s making the thousandth shot, and having the same character still look like the same character.
Ask a model for ten shots of “Mira walking down the hallway” and you’ll often get ten different people who happen to share a name. We call this identity drift, and beating it is the problem we build the whole product around.
This is a working summary of how we approach it. Treat it as a notebook in public, not a paper — it changes often.
The problem, framed precisely
Given a script that names a character and describes a setting, generate N shots such that:
- The character’s face, proportions, and wardrobe are recognizably the same across shots.
- The set and the lighting belong to one coherent scene.
- The composition still varies meaningfully — this is a sequence, not a stack of duplicates.
The third constraint is what makes it hard. You can’t just lock a seed and call it consistent; you’d get visual stutter, not storytelling.
We don’t train models — we condition them
A theme worth stating up front: we don’t fine-tune a model per character, and we don’t run our own inference. We treat the cast, sets, and styles as first-class entities with versioned reference images, and we lean on hosted generation models to do the rendering. The work is in what we feed them and how we check the result.
1. Reference bundles, not just prompts
Every character, set, and visual style is an entity that owns a reference image — a “model sheet.” When we generate a shot, we don’t hand the model a paragraph of text and hope. We assemble a reference bundle: the character’s current look, the set, the style, sometimes a prior frame — and pass them as actual image inputs.
This is where multi-reference image-editing models earn their place. A model that accepts several reference images and a text instruction will hold an identity far better than one conditioned on text alone.
2. A per-character look timeline
A character isn’t one look. They change wardrobe between scenes, age across the story, get rained on. So a character carries a timeline of looks, and when we generate a shot we resolve the right look for that point in the story — not just “Mira,” but “Mira, act two, in the soaked coat.”
3. A consistency check that has to pass
Generation isn’t the last step — evaluation is. After a shot is generated, we score it against its references before it’s allowed to stand:
- Identity, set, and style via embedding distance — is this the same face, the same room, the same grade?
- Prompt and action adherence via a vision model acting as judge, but only on borderline cases, because that’s the expensive check.
A shot moves through generated → evaluating → passed. If it fails, we generate alternates (best-of-N) and keep the best, with a bounded retry so a stubborn shot can’t loop forever. Change a character’s reference later and every shot that depended on it is marked stale and re-checked.
That eval pass-rate, measured over a fixed set of scenes, is also how we compare prompts and models against each other. It’s the closest thing we have to a unit test for “does this look right.”
What still doesn’t work
- Multiple named characters in one shot is the next frontier — bundles compose, identities still bleed.
- Large pose departures from the reference are fragile.
- Long-range drift, especially lighting, creeps back over a long scene.
For now, this is a notebook in public. If it’s interesting to you, say hi.