W.A.L.T Video Diffusion

W.A.L.T is a transformer-based approach for film and video generation through diffusion modeling. It employs a causal encoder to compress images and videos into a unified latent space, along with a windowed attention architecture for joint spatial and spatiotemporal generative modeling. This design enables state-of-the-art performance on video (UCF-101 and Kinetics-600) and image (ImageNet) generation benchmarks without classifier-free guidance. Additionally, we utilize a three-stage model cascade for text-to-video generation, producing 512 x 896 resolution videos at 8 frames per second.

W.A.L.T Video Diffusion is a Transformer-based diffusion model method focused on generating realistic videos. By unifying images and videos into a latent space and leveraging a windowed attention architecture, it achieves high-quality video and image generation.