Phoenix-4: Real-Time Human Rendering with Emotional Intelligence
Phoenix-4 is the first real-time model to generate and control emotional states, active listening behavior, and continuous facial motion as a single, unified system. It is a real-time behavior generation engine, built from the ground up, that goes beyond photorealism to transform conversation data into emotionally responsive, context-aware facial expression and head motion with millisecond-level latency.
State Space Models, Explained Through Code
I built a minimal state space model in pure PyTorch and trained it character-by-character on tiny-shakespeare dataset to understand how SSMs and Mamba actually work. This post walks through that code and explains what each piece does, why it’s there, and how it all fits together.
AI body language: can machines read and produce nonverbal cues?
AI systems are learning to do both sides of nonverbal communication. Here's where perception stands, why production is the harder problem, and what enterprise teams should evaluate before deploying.
Interactive avatars in enterprise: how to build trust at scale
Interactive avatars for enterprise build trust through behavioral realism, precise conversational timing, and closed-loop perceptual AI. Here is how it works.
.png)
Gaussian Splatting From Scratch: Explained Through Code
I built a minimal Gaussian Splatting implementation in pure PyTorch to understand how it actually works. This post goes through that code and explains what each piece does, why it’s there, and how it all fits together.
.png)
AI faces: how they're generated, animated, and used
Modern AI faces can hold live conversations and read your expressions. Explore the rendering, turn-taking, and perception models that make it possible.
.png)
Face-to-face conversational AI vs. chatbot: understanding the full spectrum
The face-to-face conversational AI vs. chatbot comparison comes down to capability tiers. A guide to types, examples, and when to move from text to video.
Customer service training: AI video agents for realistic practice
Discover how AI video agents transform customer service training with realistic practice. Learn implementation strategies, benefits, and real-world applications.
Conversational AI for healthcare: Building HIPAA-compliant patient video conversations
Conversational AI for healthcare is evolving beyond chatbots. Learn how to build HIPAA-compliant patient video conversations with real-time AI Personas.
Learnings From Random LoRA-Finetuning Experiments On A Guided Music Synthesis Diffusion Model
A hands-on experiment fine-tuning ACE-Step v1.5, an open-source AI music generation model, to fix its garbled lyric problem using LoRA and Attention Sharpness Regularization. What started as a random Friday night side project turned into a deep dive into how a 2-billion-parameter diffusion transformer actually processes text, and what breaks when you try to change it.
.png)
Raven-1: Bringing Emotional Intelligence to Artificial Intelligence
Introducing Raven-1. A multimodal perception system that captures not just what users say, but how they say it, how they look when they say it, and what that combination actually means. It interprets tone, expression, hesitation, and context in real time, enabling AI that can truly understand intent rather than simply respond to words.
Sparrow-1: Human-Level Conversational Timing in Real-Time Voice
Sparrow-1 is a specialized, multilingual audio model for real-time conversational flow and floor transfer. It predicts when a system should listen, wait, or speak, enabling response timing that mirrors human conversation rather than simply responding as fast as possible.
From random noise to real images: Understanding loss-function formulation
From theory to code: how diffusion and flow matching models turn Gaussian probability paths into vector fields, training targets and practical loss functions.
Understanding intuition behind multi-turn LLMs through the prism of search
Discover the latest research in how LLMs use reinforcement learning to search, reason, and refine answers across multiple turns—boosting accuracy and enabling active problem-solving.
Hummingbird-0: Advancing Zero-Shot Lip Synchronization in AI-Generated Video
We made an unexpected discovery while developing our premium conversational AI technology. Components of our advanced video pipeline could be isolated and explicitly optimized for lip synchronization, with remarkable results. This serendipitous research byproduct evolved into Hummingbird, a specialized zero-shot lip-sync model that achieves state-of-the-art performance compared to other leading solutions.
Sparrow-0: Advancing Conversational Responsiveness in Video Agents with Transformer-Based Turn-Taking
In this paper, we dive into the development and research behind Sparrow-0, exploring the innovative transformer-based approach for turn-taking and its integration alongside Raven and Phoenix models within our Conversational Video Interface (CVI), an end-to-end operating system designed for building responsive video agents.
Phoenix-2: Advanced Techniques in Talking Head Generation — 3D Gaussian Splatting
This paper will cover the past, present and future of the talking-head generation research field. Specifically, we will dive deep into the trending 3D scene representations (NeRF -> 3DGS) and the benefits of employing 3DGS in avatar applications.
Phoenix-1: Realistic Avatar Generation in the Wild
This research paper, written by the Tavus team, details the development of Phoenix, a groundbreaking generative model for realistic avatar creation and text-to-video generation. Phoenix leverages audio and text-driven 3D models, integrating volumetric rendering techniques and 2D Generative Adversarial Networks (GANs) to create lifelike replicas from short video clips.