Falcon: The Video Breakthrough Behind Griffin

Abstract

Conversation is more than a talking-head: people nod while listening, interrupt, and react to each other’s expressions within a fraction of a second. For AI to converse face to face, its visual embodiment must do the same. Recent video diffusion avatar-generation models have moved far beyond the limited expressiveness of earlier 3D-prior methods, yet they are built to animate speech, not to hold a conversation. Two limitations prevent generation for human-like interaction. First, although they generate video faster than its playback, they do not necessarily provide low true audio-to-video latency, causing delays in live deployments. Second, they fix text prompts across the full video, so behaviour cannot adapt to context.

In this work, we present Falcon, a video diffusion model built for human-like conversation that generates autoregressively, one latent at a time. In a conversational setting, this small chunk size gives fine control granularity and a mean audio-to-video latency of 430ms on a B200, 50% lower than the next-lowest baseline, allowing Falcon to interrupt, backchannel and react quickly. Falcon is driven by streaming audio and text controls: for example, by a streaming TTS and a streaming VLM that issues controls that allow changing the non-verbal behaviour of the avatar in real-time. Together with a three-stage training process, this lets Falcon generate 720p video in real time, including complex hand gestures, while achieving state-of-the-art FID, DOVER, and THEval on a conversational test set despite lower latency and greater control.

Date: October 15, 2026
Correspondence: george@tavus.io, jack@tavus.io

1. Introduction

Real-time virtual characters (aka avatars) have been a longstanding goal in the artificial intelligence community. These systems require many components, including multimodal large language models (MLLMs), speech generation, and conversational models. One of the most elusive components, however, is real-time, photorealistic, high-quality video generation. Such a model must not only produce high-resolution video faster than it is played back, but also minimise latency to provide sub-second responses to the user’s input. Furthermore, these models need strong control, responding to arbitrary control signals for emotions and gestures with minimal delay.

Earlier works tended to rely on 3D priors, for example, the 3D Morphable Model (3DMM) [2, 10, 21], which was fit to video and animated using an audio-to-expression model. The coarse renders of this model would then be converted to video using neural networks [27, 36], NeRFs [13, 19] and recently Gaussian splatting [6, 20]. Other works also built on explicit landmarks [22, 37] or implicit keypoints [4, 12] for proxy 3D geometry. These 3D priors acted as strong regularisers, constraining the generation process in a way that reduced artefacts and provided a semantic, low-dimensional target space that can be easily controlled. This constraint, however, also limits the expressiveness of the generated video to that of the underlying 3D model. This also meant that hand motions, hair dynamics and backgrounds were hard to model. They also generally offer little control beyond audio and, occasionally, explicit emotion. Gestures with these models are not possible.

For these reasons, many recent works have switched to video diffusion models. Progress in this area has been immense, with the latest models nearly indistinguishable from real video. Models such as Seedance 2.5 [3], Kling 3.0 [18], and Veo 3.1 [11] lead the way in closed-source video and audiovisual generation. Open-source models are now not far behind, with models such as Wan2.2 [28], LTX2.3 [14], and MiniMax H3 [23] easy to access and build upon. However, these models are typically slow and expensive to run and can produce only fixed-length videos, deteriorating rapidly when extrapolated beyond their training horizon. Over a relatively short period, a series of methods [7, 16, 34] have been developed to distil and convert these powerful foundation models into real-time autoregressive generators.

Researchers have tried to apply these methods to the avatar domain but have hit limitations. While other methods claim real-time performance, they suffer from significant latency. Generating a video chunk faster than it is consumed is not enough; the delay between the audio becoming available and its effect appearing in the generated video must also be minimised to enable deployment in a real conversational setup. We call this audio-to-video latency. Several factors contribute to high latency, including lookahead audio [8], long target chunks [5, 26], or spreading denoising steps across GPUs and starting the denoising process for latents several chunks in the future [17]. Furthermore, existing real-time avatar methods typically use fixed or restricted textual conditioning and cannot be updated online at every generation step. These limitations prevent human-like interactions by introducing longer-than-necessary pauses or producing unrealistic reactions that do not emotionally match the conversation.

In this work, we introduce Falcon. Falcon is a real-time streaming generator designed for natural, human-like face-to-face conversation. Falcon has a mean measured audio-to-video latency below 500 ms on an H100 and a per-latent text conditioning mechanism. Our model can generate 720p video in real time on a single GPU with fine-grained text and audio controls. It achieves the lowest mean audio-to-video latency among the evaluated video-diffusion avatar models: 430 ms, 50% lower than AvatarForcing, the fastest evaluated baseline, under our audio-to-video latency protocol. To the best of our knowledge, we are also the first video-diffusion-based avatar model to follow both streaming text prompt changes and streaming audio. Despite the added low-latency constraints and the greater control, we still achieve state-of-the-art metrics, outperforming several recently published baselines.

We achieve this through a few key architectural designs and algorithmic advances in training. First, our VAE uses 8× temporal compression. At 25 fps, one latent represents 320 ms of video, allowing generation in small streaming units. We introduce a spatio-temporal history compression mechanism that enables fast generation without a KV cache. Second, we introduce a three-stage training pipeline: we train the model to be few-step using DMD, autoregressive using Teacher Forcing, and stable over long horizons using a variant of self-forcing. Third, by training in a conversational, prompt-aware manner, including novel methods for prompt switching, our model can quickly adapt to streaming text prompts in a conversational setting.

2. Falcon: Real-time Streaming Human Video Generation

We design our model with the express goal of maximising conversational quality, including natural motion, visual quality, and prompt adherence, while minimising true audio-to-video latency. To achieve this goal, we made several important architectural and methodological decisions, which we detail in this section.

We start from a pretrained video diffusion transformer and adapt it to generate video conditioned on audio and text. Its variational autoencoder (VAE) uses 8× temporal compression: at 25 fps, each latent represents eight video frames, or 320 ms. Generating one latent at a time gives Falcon a small streaming unit while retaining the visual capabilities of the pretrained model.

We then apply a three-stage training process section 2.3 to convert it into a few-step autoregressive and streaming video generator. Each stage achieves a specific objective. First making the model few step, then autoregressive, then finally to handle long generation by exposing the model to its own imperfect output. Finally, we post-train the model to improve visual quality in specific regions section 2.4.

2.1. Training Data

We curate a training dataset of approximately 3000 hours of video footage. We collect this internal dataset from a variety of open-source and commercially licensed datasets, as well as datasets purchased through third-party vendors. We apply several rounds of filtering to remove poor-quality videos. In particular, we apply lip-sync filters based on the AVSR metric [31] and LSE-C [24], as well as quality filters including bitrate and IQA. We also remove samples with either no detected face or multiple faces. We standardise the video at 25 fps and keep the audio synchronised during preprocessing.

For captioning, we need a dataset with streaming captions. We use a multi-stage annotation pipeline that combines facial landmarks and action units, gaze estimates, head and body motion, hand pose, and speech activity with visual-state descriptions generated by Qwen3-VL [1]. We then use the Qwen3 text model to integrate these observations into temporally consistent descriptions of expressions and movements. We combine these annotations through a deterministic composition stage to produce captions every 400 ms, organised into five channels: visual expression, head motion, body motion, hand motion, and emotion.

2.2. Base Model

2.3. Training Process

Our main training process consists of three stages. In stage 1, we convert the model into a few-step generator using DMD2. This enables much faster training in later stages, but the model remains bidirectional and non-autoregressive. In stage 2, we introduce a novel method to produce an autoregressive, causal model that predicts one latent at a time, in contrast to existing state-of-the-art works, which require generating multiple latents at once [5, 8, 17, 26]. While capable of real-time streaming, this model degrades quickly over long horizons due to error accumulation. In stage 3, we address error accumulation with a modified variant of Self-Forcing++ that exposes the model to its own rollouts. Figure 2 summarises the three-stage training process.

2.3.1. Stage 1: DMD

The first stage of our training process involves converting the model from a many-step flow matching model to one that uses only a few steps. For this, we use Distribution Matching Distillation [33], specifically DMD2 [32]. In this stage, we use the pretrained base model as a bidirectional teacher and distil it into a few-step student model with the same architecture. Let v_θ(x_t, c, t) denote the velocity predicted by the student with parameters θ, given noised latent x_t, conditions c and denoising timestep t. We write x̂ = G_θ(z, c) for the generated latent obtained using the student sampler, starting from noise z. This generator model, called the student, induces a conditional distribution p_fake(x | c) over generated latents. Ideally, we want to minimise the KL divergence between p_fake and p_real, the teacher’s target distribution. Both distributions are conditioned on c; we suppress this conditioning in the KL notation and sampling subscript below:

However, the score of the student’s evolving output distribution is not directly available, so we must train a second model, a critic, to track the fake distribution p_fake. Rather than evaluating the KL divergence directly, we need only its gradient, which we can use for a gradient-descent optimisation step. In practice, distribution matching is performed on noise-corrupted latents x_t = α_tx̂ + σ_tϵ, where ϵ ∼ 𝒩(0, I) and α_t, σ_t specify the noise schedule. Let s_real(x_t, c, t) ≈ ∇_xt log p_real,t(x_t | c) be the score estimated by the teacher, and similarly s_fake for the score estimated by the critic tracking the student. For the noise-averaged KL objective, with noise-level weighting w(t), the gradient of the DMD loss is estimated as:

To prevent a training-inference mismatch, the DMD procedure uses backward simulation. Here, instead of constructing the student’s input by linearly blending noise and ground-truth data, we start from pure noise and follow the student’s inference sampling schedule until we reach the selected step. This ensures that the training step uses imperfect few-step predictions as input, rather than noise-corrupted ground-truth samples. The selected student sampling step is distinct from the noise level used to evaluate the DMD score difference.

To stabilise training, the critic must accurately estimate the generator’s score. We therefore update the critic more frequently than the generator. We also include an adversarial term in the loss, with the discriminator seeing clean video latents from real data and clean video latents predicted at the generator’s endpoint. Unlike DMD2, which uses features from the score critic, we use a standalone discriminator for our model. We also distil several forms of guidance, including text CFG, audio guidance and Spatio-Temporal Guidance (STG).

This stage outputs a model that generates expressive video in only a few forward calls. However, it remains a bidirectional model, limited to its training horizon and not capable of streaming.

2.3.2. Stage 2: Teacher Forcing

To create a streaming model, we train an autoregressive few-step generator using Robust Teacher Forcing. The model predicts one new latent at a time, conditioned on a reference image and past video. The reference preserves identity, while the history supports temporal consistency. We keep the reference uncompressed and use high, medium, and low compression for older, intermediate, and recent history, respectively. Figure 4 shows these groups with symbolic lengths L, M, and S. During this stage, the history and target come from ground-truth video.

We train the model on our dataset using low-rank adaptors [15] to avoid loosing the base model’s general video generation capabilities. At this stage, we train using teacher forcing, that is, the provided history frames are all from the ground-truth video. The target frame also comes from the ground truth as the next frame in the sequence. We add noise at one of the denoising levels from the DMD procedure, and the model predicts the velocity. We supervise the predicted velocity against the ground truth using a flow-matching objective. At this stage, there is no DMD, but the model remains a few-step model and is now autoregressive.

2.3.3. Stage 3: Modified Self-Forcing

We now have an autoregressive few-step model that is capable of real-time streaming, however is not robust in long conversations. In Stage 2, we train the model using ground-truth inputs for the history slots. This means it is never trained to handle its own imperfect outputs, as it must at inference, and therefore suffers from exposure bias. This shows up as drift in long videos, identity changes, structural changes and background visual artefacts. Figure 5 illustrates this behaviour in an example sequence.

To address this, we train our model on its own outputs, simulating the inference process during training.

In order to retain the benefits coming from the teacher model in the previous DMD training we continue with the DMD framework of Stage 1. Inspired by Self-Forcing++, we use a modified self-forcing framework in which the model is exposed to long rollouts of its own outputs. Here, we roll out inference steps, generating a prefix of frames that the model is then asked to continue. We detach this rollout, then apply the teacher score model and the critic to a window of frames. We gradually increase rollout duration to improve stability over longer sequences.

In this stage, the model uses the same setup as in Stage 2. The model remains conditioned on a clean reference frame and its video history.

2.4. Post Training

The preceding stages optimise distribution matching and long-horizon stability, but do not directly target small, perceptually salient regions such as the eyes and teeth, which are sometimes prone to display artefacts. To address this, we apply a final reward post-training stage based on DanceGRPO [30]. We merge the Teacher Forcing and Self-Forcing++ adapters into a frozen generator, and optimise only a newly injected low-rank adapter. This isolates quality alignment from the streaming behaviour learned in earlier stages.

Avatar-aware reward and baseline anchoring. We decode only generated frames and combine face coverage, face/eye/teeth quality, reference-eye perceptual similarity [35], face-embedding identity [9], temporal flicker, and local eye or mouth instability. A fixed-scale reference-colour term captures drift that would disappear under purely relative scoring; prompt, hand-motion, and lip-sync rewards can be added through the same interface. Relative component scores are normalised within each candidate group before aggregation.

Because group-relative ranking cannot detect when every candidate is worse than the starting model, each policy trajectory is paired with a frozen baseline rollout using identical initial and transition noise. With group-normalised component scores r̄_i^(m), the advantage is

The paired margin provides an absolute ‘‘do no worse than the starting policy’’ signal. To limit reward exploitation, face rewards require minimum generated-frame detection coverage, final-frame identity requires a detected face, and teeth quality is scored only when the mouth is sufficiently open.

Sequence-level group-relative optimisation. For each audio, reference-image, and streaming-caption condition c, we sample K stochastic autoregressive trajectories. Each trajectory generates multiple consecutive blocks and commits every predicted clean latent to the history for the next block. We evaluate rewards under streaming inference conditions, so they reflect error accumulation over time.

We treat each stochastic denoising transition as an action. For adjacent states z_i,j and z_i,j+1 with autoregressive history h_i,j, let ρ_i,j be the likelihood ratio between the current and rollout policies. We optimise the clipped objective

The sequence reward supervises all eligible transitions without a learned value model. Rollouts are generated without gradients and replayed one transition at a time to keep memory tractable during training.

3. Evaluation

In this section, we evaluate our model against the state of the art.

3.1. Evaluation Data

We evaluate our model using actual conversational data. To maintain fairness in the comparisons, we select a dataset that neither the baselines nor our model are trained on. For this reason, we use a selection of internally collected conversational data. This consists of thousands of hours of high-quality video footage from conversations between two people. This dataset is disjoint from our training set.

From this, we select a random subset of 58 video pairs for evaluation. We apply the same captioning process as described in Section 2.1 to these videos and preprocess them by cropping and resizing to each model’s resolution, with the face centred. We call this processed subset the Dyadic Bench.

In addition, we provide qualitative results from an internally collected dataset representative of typical studio-like avatar reference images and audio. We refer to this dataset as the Tavus PALs.

3.2. Baselines

We select four state-of-the-art open-source streaming avatar models as our baselines. Each model is optimised for real-time human video generation from input audio and, optionally, a prompt. They are also open-source and include inference code. These baselines are as follows:

LiveTalk: This is a model based on OmniAvatar-1.3B, a multimodal variant of Wan2.1 [5]. This work uses self-forcing, with comments on its limited effectiveness over long training regimes. They therefore curate the input data (reference images and text) and train an ODE stage to convergence before starting DMD. For better stability, they also allocate a larger proportion of the KV cache to early-generated frames rather than rolling history. They generate one block of three latent frames at a time, with four denoising steps.

SoulX-FlashTalk: This 14B Wan2.1 model [26] retains bidirectional sequence attention within video chunks and is autoregressive across chunks. First the model is adapted to lower resolution and shorter clips, then is DMD run with rolled-out histories. The real-time performance uses multi-GPU inference and system optimisations.

AvatarForcing: A 1.3B Wan2.1 model [8] that uses a sliding window for generation. It adjusts the reference anchor’s RoPE at each forward pass to keep it at a fixed relative position. The model performs one forward pass per step, but uses a window of four blocks, including the block being emitted, with each getting progressively cleaner as it moves towards the current timestep. This improves throughput, at the cost of a fixed latency overhead. ODE initialisation is followed by DMD on sliding-window student rollouts.

LiveAvatar: Based on WanS2V [17], also a 14B Wan2.1 variant, this model distils the teacher into a 4-step student with first diffusion forcing, followed by self-forcing, with a persistent appearance anchor and dynamically adjusted positional encoding. The KV cache is noised to the same level as the target frames. Four denoising stages are assigned to separate GPUs, with different blocks moving through the pipeline.

3.3. Metrics

We aim to show that our model is fast and low-latency in Section 3.4.1 and high quality in terms of visual quality, lip sync, prompt following, and overall suitability as a talking head model. For latency, we measure the delay between audio availability and emission of the corresponding synchronised audio and video, as detailed in Section 3.4.1. For visual quality, we use the reference-free video quality metric DOVER [29] and the distribution-level FID. For lip-sync, we use Lip Sync Error Confidence (LSE-C) [24]. We also use the Talking Head Evaluation framework THEval [25], reporting its aggregate score alongside eight components covering appearance quality, motion dynamics, silent-lip stability, and lip synchronisation. For the component scores, each generated-video metric value m is normalised by its closeness to the corresponding ground-truth value g as max(0, 1 − |m − g|/(|g| + 10⁻⁹)), then averaged over videos with valid measurements. Higher normalised scores indicate closer agreement with ground truth. For prompt following on streaming prompts, we do not know of any existing metrics, so we show results qualitatively.

3.4. Results

Figure 7 shows the results of the four selected metrics. Falcon achieves the best FID, DOVER and aggregate THEval scores among the evaluated baselines. For LSE-C, Falcon scores higher than three of the four baselines, but lower than AvatarForcing (7.27 versus 7.73). AvatarForcing also has the highest FID, indicating a larger discrepancy from the reference distribution under this metric. Separately, in qualitative inspection, we observe smooth, synchronised but limited lip motion in some AvatarForcing outputs; this observation is not established by FID alone.

Figure 8 gives the component-level breakdown of THEval. Falcon has the highest normalised score in six of the eight components; AvatarForcing leads on head motion dynamics and LiveAvatar on lip-sync MAD similarity. These results describe complementary aspects of quality rather than a uniform advantage on every measure.

3.4.1. Latency

Real-time throughput alone does not establish low conversational latency. In particular, generating Xms of video in under Xms does not account for playback. If generating one second at a time, the conversation will take at worst the time to generate the one second chunk containing that audio, plus one second to play back the audio from the rest of the chunk. This is why shorter generation chunk sizes are preferable, and why our model targets a single latent generation at a time.

We measure audio-to-video latency on the Dyadic Bench. For this experiment, we supply the complete audio upfront, rather than having it arrive incrementally from a real-time TTS system or speech-to-speech model. Each model breaks up the audio into chunks aligned with its own video latent frame rate. We record these chunks before video generation and match them to the audio emitted with the corresponding video, measuring the elapsed time until output emission. This measures generation-pipeline latency with audio already available; it excludes upstream speech generation, the time needed to accumulate audio or lookahead in a live stream, and downstream playback delay.

Figure 9 shows the result of this experiment. We report the minimum and maximum recorded latency, as well as the mean over video generations from the Dyadic Bench. Falcon has 50% lower mean latency than AvatarForcing, the fastest evaluated baseline (0.43 s versus 0.86 s), and an approximately 46% lower observed maximum latency (0.59 s versus 1.10 s). SoulX-FlashTalk uses 8×H800s, and LiveAvatar uses 5×H100s with its Timestep-forcing Pipeline Parallelism. All other methods are tested using a B200.

3.4.2. Prompt Responsiveness

We evaluate prompt responsiveness qualitatively using fixed prompts and prompts that change during generation. For fixed prompts, we show a wave (Figure 11), a surprised expression (Figure 12), and a happy expression (Figure 13) from the Tavus PALs dataset. Models receive the same reference image, audio and prompt, with seeds held fixed within each model’s comparisons. Frames are sampled at 1.5, 4, 6.5, and 9 seconds. These examples show Falcon following action and emotion prompts; they are qualitative illustrations rather than a quantitative measure of prompt adherence.

For changing prompts, we update the prompt every five seconds during continuous generation (Figure 10). Falcon responds to the successive actions, whereas the baselines do not consistently follow the changes. Baseline frames are sampled four seconds after each requested switch; Falcon frames are selected within each prompt interval to show clear responses.

4. Conclusion and Next Steps

We have presented Falcon, a significant step forward in conversational video generation. Our model runs in real time on a single GPU and achieves 50% lower latency than the closest baselines while outperforming them across several metrics. We are also, to the best of our knowledge, the first model to support streaming video captions that can change multiple times per second. We achieve this with our additional Teacher Forcing stage, a temporally compressed video representation, and explicit training with varying captions using a custom dataset.

Temporal compression supports fast generation, but sets the granularity of each generated chunk. With 8× VAE temporal compression at 25 fps, one latent represents eight video frames, or 320 ms. A direction for future work is to reduce this temporal unit while maintaining real-time throughput. Furthermore, while we train with varying captions, we do not modify the model to accommodate them, which can lead to occasional missed or delayed actions. We are actively investigating models designed for streaming captions.

References

[1] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-VL technical report. arXiv preprint arXiv:2511.21631, 2025.

[2] Volker Blanz and Thomas Vetter. A morphable model for the synthesis of 3D faces. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’99, pages 187–194, 1999. doi: 10.1145/311535.311556. URL https://doi.org/10.1145/311535.311556.

[3] ByteDance Seed. Seedance 2.5. https://seed.bytedance.com/en/seedance2_5, n.d. Accessed 16 September 2026.

[4] Xuyang Cao, Guoxin Wang, Sheng Shi, Jun Zhao, Yang Yao, Jintao Fei, and Minyu Gao. Joyvasa: Portrait and animal image animation with diffusion-based audio-driven facial dynamics and head motion generation, 2024. URL https://arxiv.org/abs/2411.09209.

[5] Ethan Chern, Zhulin Hu, Bohao Tang, Jiadi Su, Steffi Chern, Zhijie Deng, and Pengfei Liu. LiveTalk: Real-time multimodal interactive video diffusion via improved on-policy distillation. arXiv preprint arXiv:2512.23576, 2025.

[6] Kyusun Cho, Joungbin Lee, Heeji Yoon, Yeobin Hong, Jaehoon Ko, Sangjun Ahn, and Seungryong Kim. GaussianTalker: Real-time talking head synthesis with 3D gaussian splatting. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 10985–10994, 2024.

[7] Justin Cui, Jie Wu, Ming Li, Tao Yang, Xiaojie Li, Rui Wang, Andrew Bai, Yuanhao Ban, and Cho-Jui Hsieh. Self-Forcing++: Towards minute-scale high-quality video generation. arXiv preprint arXiv:2510.02283, 2025.

[8] Liyuan Cui, Wentao Hu, Wenyuan Zhang, Zesong Yang, Fan Shi, and Xiaoqiang Liu. AvatarForcing: One-step streaming talking avatars via local-future sliding-window denoising, 2026. URL https://arxiv.org/abs/2603.14331.

[9] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. ArcFace: Additive angular margin loss for deep face recognition. In CVPR, 2019.

[10] Bernhard Egger, William A. P. Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhöfer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, Christian Theobalt, Volker Blanz, and Thomas Vetter. 3D morphable face models—past, present, and future. ACM Transactions on Graphics, 39(5), 2020. doi: 10.1145/3395208. URL https://doi.org/10.1145/3395208.

[11] Google. Veo: Google AI Studio. https://aistudio.google.com/models/veo, n.d. Accessed 16 September 2026.

[12] Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang. Liveportrait: Efficient portrait animation with stitching and retargeting control. arXiv preprint arXiv:2407.03168, 2024.

[13] Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, and Juyong Zhang. AD-NeRF: Audio driven neural radiance fields for talking head synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.

[14] Yoav HaCohen, Benny Brazowski, Nisan Chiprut, Yaki Bitterman, Andrew Kvochko, Avishai Berkowitz, Daniel Shalem, Daphna Lifschitz, Dudu Moshe, Eitan Porat, Eitan Richardson, Guy Shiran, Itay Chachy, Jonathan Chetboun, Michael Finkelson, Michael Kupchick, Nir Zabari, Nitzan Guetta, Noa Kotler, Ofir Bibi, Ori Gordon, Poriya Panet, Roi Benita, Shahar Armon, Victor Kulikov, Yaron Inger, Yonatan Shiftan, Zeev Melumian, and Zeev Farbman. LTX-2: Efficient joint audio-visual foundation model. arXiv preprint arXiv:2601.03233, January 2026. URL https://arxiv.org/abs/2601.03233.

[15] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR. OpenReview.net, 2022. URL http://dblp.uni-trier.de/db/conf/iclr/iclr2022.html#HuSWALWWC22.

[16] Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. arXiv preprint arXiv:2506.08009, 2025.

[17] Yubo Huang, Hailong Guo, Fangtai Wu, Shifeng Zhang, Shijie Huang, Qijun Gan, Lin Liu, Sirui Zhao, Enhong Chen, Jiaming Liu, and Steven Hoi. Live Avatar: Streaming real-time audio-driven avatar generation with infinite length, 2025. URL https://arxiv.org/abs/2512.04677.

[18] Kling AI. Kling 3.0: AI video generator. https://kling.ai/feature/ai-video-generator, n.d. Accessed 16 September 2026.

[19] Jiahe Li, Jiawei Zhang, Xiao Bai, Jun Zhou, and Lin Gu. Efficient region-aware neural radiance fields for high-fidelity talking portrait synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7568–7578, 2023.

[20] Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Xin Ning, Jun Zhou, and Lin Gu. TalkingGaussian: Structure-persistent 3D talking head synthesis via gaussian splatting. In Computer Vision - ECCV 2024, pages 127–145. Springer, 2024.

[21] Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4D scans. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6):194:1–194:17, 2017. URL https://doi.org/10.1145/3130800.3130813.

[22] Yuanxun Lu, Jinxiang Chai, and Xun Cao. Live Speech Portraits: Real-time photorealistic talking-head animation. ACM Transactions on Graphics, 40(6), 2021. doi: 10.1145/3478513.3480484.

[23] MiniMax. MiniMax H3: An open model breaking the boundaries between tasks and modalities. https://www.minimax.io/blog/minimax-h3, n.d. Accessed 16 September 2026.

[24] K R Prajwal, Rudrabha Mukhopadhyay, Vinay P. Namboodiri, and C.V. Jawahar. A lip sync expert is all you need for speech to lip generation in the wild. In Proceedings of the 28th ACM International Conference on Multimedia, MM ’20, pages 484–492, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450379885. doi: 10.1145/3394171.3413532. URL https://doi.org/10.1145/3394171.3413532.

[25] Nabyl Quignon, Baptiste Chopin, Yaohui Wang, and Antitza Dantcheva. THEval. evaluation framework for talking head video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings, pages 1943–1953, June 2026.

[26] Le Shen, Qian Qiao, Tan Yu, Ke Zhou, Tianhang Yu, Yu Zhan, Zhenjie Wang, Dingcheng Zhen, Ming Tao, Shunshun Yin, and Siyuan Liu. SoulX-FlashTalk: Real-time infinite streaming of audio-driven avatars via self-correcting bidirectional distillation. arXiv preprint arXiv:2512.23379, 2025.

[27] Justus Thies, Mohamed Elgharib, Ayush Tewari, Christian Theobalt, and Matthias Nießner. Neural voice puppetry: Audio-driven facial reenactment. In Computer Vision – ECCV 2020, pages 716–731. Springer, 2020. doi: 10.1007/978-3-030-58517-4_42.

[28] Wan Team. Wan2.2: Official repository. https://github.com/Wan-Video/Wan2.2, 2025. Accessed 16 September 2026.

[29] Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In International Conference on Computer Vision (ICCV), 2023.

[30] Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, and Ping Luo. DanceGRPO: Unleashing grpo on visual generation. arXiv preprint arXiv:2505.07818, 2025.

[31] Dogucan Yaman, Fevziye Irem Eyiokur, Leonard Barmann, Seymanur Akti, Hazim Kemal Ekenel, and Alexander Waibel. Audio-visual speech representation expert for enhanced talking face video generation and evaluation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 6003–6013, 2024. doi: 10.1109/CVPRW63382.2024.00607.

[32] Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T Freeman. Improved distribution matching distillation for fast image synthesis. In NeurIPS, 2024.

[33] Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In CVPR, 2024.

[34] Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, 2025.

[35] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.

[36] Wenxuan Zhang, Xiaodong Cun, Xuan Wang, Yong Zhang, Xi Shen, Yu Guo, Ying Shan, and Fei Wang. SadTalker: Learning realistic 3D motion coefficients for stylized audio-driven single image talking face animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8652–8661, 2023.

[37] Yang Zhou, Xintong Han, Eli Shechtman, Jose Echevarria, Evangelos Kalogerakis, and Dingzeyu Li. MakeItTalk: Speaker-aware talking-head animation. ACM Transactions on Graphics, 2020. doi: 10.1145/3414685.3417774.