1. Introduction

Every frame in an AI-generated video is an act of creation from scratch.

There is no actor standing under studio lights, no physical face that persists between takes, and no memory of the previous shot. For every scene, every camera angle, every expression, and every change in lighting, the model must reconstruct an entire human identity anew—armed with little more than a reference image and a text prompt.

This gives rise to a challenge that has no equivalent in traditional filmmaking.

A character who looked perfect in one shot may emerge subtly—or dramatically—different in the next, not because the story demands it, but because the model has drifted. A different jawline, altered eyes, inconsistent hair, or a face that simply no longer feels like the same person. The illusion breaks.

This is the character consistency problem—the defining unsolved challenge of generative video production.

In multi-scene AI productions, where fictional characters must remain recognisable across dozens of independently generated keyframes, even minor deviations accumulate into visible, trust-breaking inconsistencies. Viewers notice immediately. Creators are trapped in expensive regeneration loops, repeatedly asking models to recreate characters they have no shared history with. The extraordinary promise of AI filmmaking is constrained by a fundamental limitation: no matter how accurate the reference image, the model will reconstruct the character's identity slightly differently every time — and across dozens of shots, those differences compound into a character that no longer feels like the same person. Each keyframe is produced independently, from the same reference image, yet the model's stochastic nature means the reconstructed identity shifts slightly every time. The same reference can produce subtly different jawlines, altered eyes, or a face that simply no longer feels like the same person across shots.

This paper focuses on character consistency at the keyframe level—the still images that serve as the visual foundation of AI-generated videos. Every video clip begins with a keyframe that the video model animates into motion. If the character's identity is not preserved in that first frame, consistency is compromised before the video generation process even begins.

In the sections that follow, we dissect the failure modes underlying identity drift, present four interventions developed to address them, and introduce the evaluation framework used to measure progress. This is the first paper in a broader series on character consistency in generative filmmaking; companion papers explore face-swap post-processing and character LoRA fine-tuning as additional layers of the solution.

1.1 Shot Types and Face Visibility

The consistency challenge varies significantly by shot type — how close the virtual camera is to the character determines how much of the face is visible, and therefore how tightly the model must match the intended identity. Three shot types are central to this work:

• Close-Up: Frame cuts at shoulder level. The face fills most of the image; any identity mismatch is immediately and harshly visible.

• Medium Shot: Frame cuts at waist level. The face is clearly visible but shares space with the body and environment.

• Wide Shot: Full character in frame, head to toe. The face occupies a smaller portion of the image — making wide shots the hardest category to get right, and historically the most likely to fail.

Because each keyframe is generated independently — with no memory of prior frames — errors do not cancel out. They compound across shot types and accumulate over the course of a production.

Figure 1. Illustration of the three shot types used in this evaluation. Left: Close up shot, Centre: Medium shot, Right: wide shot

1.2 Failure Modes Observed

Three distinct failure modes were identified in a production AI video generation pipeline:

• Training data bias: Image generation models carry strong associations between names, contexts, and visual appearances absorbed during pre-training. When these prior associations conflict with the reference images provided at inference time, the model may render a face consistent with its learned prior rather than the intended character. Reference image conditioning does not always win.

• General hallucination: The model draws on scene-context priors from training rather than the provided references, substituting contextually plausible but entirely incorrect faces or costumes. A scene set in a period garden, for instance, prompted the model to dress characters in traditional wedding attire that was never specified in any reference or prompt.

• Attribute entanglement in appearance prompts: When a character's appearance description includes both clothing details and facial or grooming attributes — such as a slight stubble or messier hair — the model treats these as independent styling instructions rather than modifications to an already-established identity. Because the model has no prior knowledge of that specific character with those features, it cannot make a targeted change. Instead, it averages across similar appearances seen in training data, often producing a face that looks like a plausible but entirely different person. The intended output was the same character with slightly different hair or stubble; the actual output is a new face that happens to match the described grooming — with the original identity lost in the process.

2. Platform Context

The work described in this paper was conducted on an AI-powered video generation platform. A creator provides a story concept; the platform generates complete video episodes end-to-end. Two pipeline steps are directly relevant to the character consistency problem:

• Character generation: A reference image is produced for each character — initially a single front-facing portrait. This image defines the character's intended visual identity and serves as the anchor for all downstream generation.

• Keyframe generation: A still image — the keyframe — is generated independently for each scene, using the character reference image and a text description of that scene. Each keyframe serves as the start frame that the video generation model uses to animate the corresponding video clip. Because keyframes are generated independently with no memory of prior frames, every scene is a fresh reconstruction of the character's identity from reference alone.

All interventions described in this paper target keyframe generation. Improving the fidelity of keyframes directly improves the visual foundation that video generation builds upon.

3. Approach

Four interventions were developed and validated, each targeting a specific failure mode. The table below summarises the incremental lift each contributed to overall face similarity scores, and the cumulative effect of all four combined.

3.1 Isolating Facial Attributes in Generation Prompts (~20% lift)

A key observation was that generation models do not treat facial identity as separate from other visual attributes. When prompts describing a character's appearance include hair, skin tone, or grooming details alongside non-facial attributes such as clothing, the model may modify the face as a side effect of the outfit update. This produces a reference image carrying a subtly different face, which then propagates into all downstream keyframes generated from it. Ensuring that appearance prompts contain only non-facial attributes — clothing and accessories only — prevents this unintended facial alteration at the source.

3.2 Character Sheet as Reference (~15% lift)

The baseline pipeline provided only a single front-facing image as the character reference. For any non-frontal shot, the model must extrapolate facial geometry from this single view — and typically defaults to a statistically average face for unseen angles rather than the specific character's features.

A character sheet is a composite image containing multiple views of the same character arranged in a grid, generated in a single inference call by an image generation model such as Nano Banana. Generating all views in one call is important: producing multiple profile images sequentially introduces facial drift between images, since each generation is independent. A single-call composite ensures all views are internally consistent representations of the same identity.

The sheet design used here places full-body views across different poses and angles in the top rows, with face close-up crops at the corresponding angles in the bottom row. This layout gives the generation model both compositional context — how the character is proportioned and posed — and the high-resolution facial detail it needs to anchor identity. Face detail density proved to matter more than pose variety: sheets built only from full-body views shrink the face to a small fraction of the composite, which can cause the model to fall back on generic priors and hallucinate an entirely different identity.

Figure 5. A character sheet generated in a single inference call. Top row: full-body views across angles. Bottom row: corresponding face close-up crops.

3.3 Angle-Aligned References (~8% additional lift)

A further refinement for wide shots: rather than providing the full character sheet, the model receives two images — a full front-facing portrait and a face close-up cropped directly from the character sheet at the angle most closely matching the shot's camera direction. Because the close-up is derived from the same single-inference-call character sheet, it is guaranteed to be identity-consistent with the front-facing portrait. This gives the model the most relevant high-resolution facial signal for the specific angle being rendered, without the attention dilution of a dense multi-panel composite.

3.4 Higher-Fidelity Generation Model for Wide and Medium Shots (~27% lift)

The highest-impact change was switching from Nano Banana to GPT Image 2 as the generation model for wide and medium shots. GPT Image 2 conditions more faithfully on reference images — using the provided visual references as strong anchors rather than allowing pre-training priors to dominate — and delivered the largest single gain in face similarity across all interventions.

GPT Image 2 is not used for close-up shots. Nano Banana excels at rendering fine-grained photorealistic detail — skin texture, pores, subtle surface quality — that is critical when the face fills most of the frame. For close-up shots, Nano Banana paired with a character sheet as the reference remains the strongest combination. GPT Image 2 is therefore applied selectively to wide and medium shots, where stronger reference conditioning delivers the greatest consistency gains without sacrificing the photorealism that close-up shots demand.

4. Validation

4.1 Metrics

Three categories of metrics capture different dimensions of character consistency at the keyframe level:

ArcFace similarity score: A value between 0 and 1 measuring identity similarity between the face in a generated keyframe and the character's reference image. For each keyframe, the character's face is detected, the reference image whose angle best matches the detected pose is selected, and cosine similarity is computed between the two face embeddings — compact mathematical representations of facial features produced by a model trained specifically for face recognition. Scores are computed per keyframe.

Pass rate: The percentage of generated keyframes whose ArcFace score meets or exceeds a defined acceptance threshold. This is the primary binary quality gate — a keyframe either passes (usable as a start frame for video generation) or fails (requires regeneration).

VLM attribute overlap: A vision-language model evaluates softer attributes — skin tone, facial features, general appearance — that ArcFace does not capture. The overlap percentage is the share of keyframes where the VLM judge confirms a match with the character reference.

Prompt iterations per accepted output: The average number of generation attempts required before producing one acceptable keyframe. A direct measure of creator effort and pipeline efficiency.

4.1 Results

The operational efficiency gains mirror the consistency improvements. For close-up shots, the average number of generation attempts required before an acceptable keyframe was produced dropped by 50%. Medium shots saw a 40% reduction. Wide shots, which required the most iterations in the baseline, also improved by 40% — dropping from the highest iteration count of any shot type to a level comparable with the post-improvement close-up baseline. Creators spend significantly less time in regeneration loops and more time in actual production.

Across all three shot types, the combined pipeline delivered consistent improvements in every metric category. Wide shots showed the largest relative gains in both face similarity and pass rate, reflecting how much headroom existed from a low baseline. Close-up shots, the strongest category in the baseline, showed more modest but consistent gains. The pattern held for attribute consistency and operational efficiency: widest improvements in wide shots, meaningful but smaller gains in close-ups.

The results confirm that no single intervention is sufficient. The cumulative effect of prompt-level attribute isolation, richer reference images, and a more reference-faithful generation model for wider shots together account for the observed gains — each targeting a different failure mode, and each degrading the outcome if removed.

5. Future Scope and Open Challenges

5.1 Companion Papers in This Series

The four interventions described in this paper improve character consistency by giving the generation model better inputs at keyframe generation time. Two complementary approaches — which operate after keyframe generation rather than during it — are covered in companion papers:

• Face-swap post-processing: Even after applying character sheet references and model selection, some keyframes may still exhibit residual identity drift. An agentic face-swap framework is available as a corrective step: it detects the character's face in the generated keyframe and replaces it with the correct identity from the reference image — preserving all other elements of the scene (composition, lighting, background, costume) unchanged. The content creator can invoke this selectively for keyframes that do not meet the identity threshold, without triggering a full regeneration.

• Character LoRA fine-tuning: Trains lightweight model adapters on character-specific images, embedding identity directly into the generation model's weights. This reduces dependence on reference image conditioning at inference time and enables more robust identity reproduction across diverse angles, scenes, and costumes.

5.2 Open Challenges

• Profile shots remain the hardest category. Full left and right profile keyframes score significantly lower than frontal shots across all methods tested. 

• Multi-character keyframes score lower than single-character scenes because the generation model must simultaneously reconstruct two correct identities with attention shared between them. Independent generation of each character followed by compositing is a promising structural direction.

5.3 Conclusion

Character consistency is not a single problem with a single fix — it is a layered challenge that sits at the intersection of model conditioning, reference image design, prompt engineering, and model selection. The work described in this paper demonstrates that meaningful, measurable progress is achievable through a structured, incremental approach: identify the specific failure modes, design targeted interventions for each, and validate their effect independently before combining them.

The four interventions presented here — isolating facial attributes in prompts, using a generated multi-angle character sheet as reference, aligning references to the shot angle, and selecting the right generation model per shot type — together deliver a substantial cumulative lift in keyframe-level character consistency. Wide shots, historically the weakest category, saw the most dramatic improvement. Operationally, the pipeline requires significantly fewer regeneration attempts per accepted keyframe, translating directly into faster turnaround and lower creator effort.

Critically, these gains come without sacrificing image quality. The model choices are deliberate: GPT Image 2 where reference fidelity matters most, Nano Banana where photorealistic surface detail is paramount. The reference design is deliberate too: a single-inference character sheet that is internally consistent by construction, with angle-specific close-ups available for targeted use.

What this paper establishes is the baseline layer of a broader solution. Face-swap post-processing and character LoRA fine-tuning — covered in companion papers — address the cases where reference-level interventions alone are insufficient. Together, these three layers form a progressively stronger consistency guarantee: better inputs at generation time, corrective identity transfer in post-processing, and identity embedded directly in model weights. The goal is a pipeline where a fictional character looks exactly like themselves, every time, across every shot — and this paper is the first step toward that.

Part 1 of a multi-paper series on character consistency in AI-generated video. Companion papers cover face-swap post-processing and character LoRA fine-tuning. All results were obtained in a production AI video generation pipeline; no client names or proprietary identifiers are disclosed.