You Can't Tell What's Real Anymore. Here's What Comes Next - Cristóbal Valenzuela, Runway

Highlights

• Runway's first AI film, created in 2019, generated pixels at a 128 by 128 resolution before upscaling.

• Video generation models operate on the principle of next-frame prediction driven by compute scaling.

• Predicting successive frames enables models to inherently learn physics, gravity, and object permanence.

• Runway established a robotics lab in New York to explore video models as the brains for autonomous systems.

• Continuum connects video models to coding agents to enable visual thinking through generated simulations.

Timeline

1. 00:12 - 03:30

   Evolution of Video Generation from 2019 to Present

   Comparing early experimental clips from 2019 to modern outputs highlights the rapid technical trajectory of video generation models. Early systems operated at minimal resolutions, while contemporary models produce complex, highly detailed visual sequences that surpass human distinction between real and generated footage.

   • Runway began producing AI films in 2019 using low-resolution pixel generation models.
   • Early experimental models required forceful upscaling from 128 by 128 pixel grids.
   • Current state-of-the-art video models generate hyper-realistic characters and locations automatically.
2. 03:30 - 07:00

   Scaling Next-Frame Prediction and Physics Acquisition

   Video models function by predicting subsequent frames at scale, mirroring the fundamental mechanics of cinema. Through this continuous frame prediction, models absorb complex physical properties such as gravity, occlusion, and object interaction without requiring explicit programmed rules.

   • Cinema established 120 years ago that video is an optical illusion created by rapid frame sequencing.
   • Scaling loss drives the emergent properties and intelligence within modern video models.
   • Predicting consecutive frames allows models to acquire an implicit understanding of physical laws.
3. 07:00 - 10:04

   Interactive Worlds and Robotic Embodiment

   Transitioning from static generation to interactive systems allows users to alter generated scenes dynamically. This capability extends into robotics, where video models serve as cognitive frameworks for autonomous physical movement and spatial reasoning.

   • Future video and world models focus on real-time interactivity rather than static generation.
   • Runway maintains a robotics lab in New York housing hardware alongside creative studios.
   • World interface models process clicks and user inputs without utilizing traditional 3D software engines.
4. 10:04 - 16:43

   Continuum and Operating System Integration

   Continuum represents an operating system framework where interface elements stream directly from video models. By turning text into interactive visual portals and giving coding agents the ability to think visually, these systems change how humans and AI process spatial and structural information.

   • Continuum integrates real-time video models directly into operating system workflows.
   • Users can convert static text or website data into interactive video portals.
   • Coding agents utilize visual thinking to construct and navigate simulations of complex problems.