CAMEO roduces more stable and consistent human body articulation in complex motions, whereas vanilla CogVideoX-5B.
Abstract
Human video generation is becoming an increasingly important task with broad applications in graphics, entertainment, and embodied AI. Despite the rapid progress of video diffusion models (VDMs), their use for general-purpose human video generation remains underexplored, with most works constrained to image-to-video setups or narrow domains like dance videos. In this work, we propose CAMEO, a CAscaded framework for general human Motion vidEO generation. It seamlessly bridges Text-to-Motion (T2M) models and conditional VDMs, mitigating suboptimal factors that may arise in this process across both training and inference through carefully designed components. Specifically, we analyze and prepare both textual prompts and visual conditions to effectively train the VDM, ensuring robust alignment between motion descriptions, conditioning signals, and the generated videos. Furthermore, we introduce a camera-aware conditioning module that connects the two stages, automatically selecting viewpoints aligned with the input text to enhance coherence and reduce manual intervention. We demonstrate the effectiveness of our approach on both the MovieGen benchmark and a newly introduced benchmark tailored to the T2M–VDM combination, while highlighting its versatility across diverse use cases.
Overall Pipeline
Given a text prompt, we first disentangle it to separate motion-related and semantic components. The motion prompt is converted into an initial motion sequence via a text-to-motion model. The sequence is rendered as SMPL-based guidance videos, where a camera-aware conditioning module determines the viewpoints for rendering. Finally, the video diffusion model synthesizes the human video, guided by the semantic prompt and motion condition, seamlessly bridging text-to-motion and text-to-video generation.