For the past two years, the AI industry has been obsessed with language. We’ve watched large language models grow from novelty chatbots into tools that write code, draft legal briefs, and pass medical exams. The assumption was straightforward: if a model could master the messy, ambiguous world of human language, it could eventually master everything else.
That assumption is starting to crack.
A new phrase is gaining weight in AI research labs: world models. Unlike LLMs, which operate entirely in the symbolic space of text, world models aim to build internal representations of physical reality — how objects move, how spaces relate to each other, how cause and effect unfold over time. The ambition is nothing short of teaching machines to understand the world the way humans do: not by memorizing patterns in language, but by developing an intuitive grasp of physics, space, and time.
Inside the labs building this next generation of AI, the stakes are enormous. Fei-Fei Li, the Stanford professor often called the “godmother of AI,” raised a billion dollars for her company World Labs to pursue what she calls “spatial intelligence.” Jensen Huang is building the simulation platforms and compute infrastructure behind what NVIDIA calls “physical AI.” Google DeepMind has Genie, a model that generates interactive 3D worlds from a single image. OpenAI’s Sora stunned the world not because it understood language, but because it appeared to understand how light moves through a room, how fabric drapes over a shoulder, how a camera tracks a subject through three-dimensional space.
The language era isn’t over. But the spatial era is beginning.
What Is a World Model, Exactly?
The term “world model” has been used loosely across different research communities, which has created some confusion. At its core, a world model is an AI system that builds and maintains an internal representation of an environment — physical, virtual, or conceptual — and can use that representation to reason about what will happen next.
Think about what a human does when they walk into an unfamiliar room. Within milliseconds, you understand the geometry of the space: where the walls are, which objects are on which surfaces, how you could move from the doorway to the window. You don’t consciously label every object or verbalize every spatial relationship — you just know. That’s spatial intelligence. It’s fundamental to how we navigate the world, and it’s something LLMs fundamentally lack.
A language model can tell you that “the cat is on the table.” But it has no internal concept of a table, a cat, gravity, or the spatial relationship between them. It’s manipulating symbols, not simulating reality. A world model, by contrast, would actually represent the three-dimensional scene: the cat’s position relative to the table, the fact that the cat is supported by a surface, the fact that if the table tipped, the cat would fall.
There are at least five competing approaches to building world models, according to a comprehensive analysis by The Mesis, and they differ dramatically in what they prioritize. Some focus on generating photorealistic video (OpenAI’s Sora, Runway’s Gen-3). Some focus on interactive 3D environments (World Labs, Google DeepMind’s Genie). Some focus on physical simulation for robotics (NVIDIA Cosmos). Others take a hybrid approach, combining language models with spatial reasoning modules.
The common thread: they all move beyond text.
The Key Players
World Labs and Fei-Fei Li
No conversation about world models is complete without Fei-Fei Li. After stepping back from her role as co-director of Stanford’s Human-Centered AI Institute, Li co-founded World Labs in 2024 with a mission to build “spatial intelligence” — AI systems that can perceive, generate, and interact with 3D worlds.
World Labs raised over $1 billion at a valuation north of $5 billion, making it one of the most heavily funded AI startups in history. The company released its first major product, the World API, in March 2026. It’s a public API that generates explorable 3D worlds from text, images, or video input. You describe a scene — “a sunlit apartment in Barcelona with exposed brick walls and a balcony overlooking the Gothic Quarter” — and the API returns a fully navigable 3D environment, complete with lighting, textures, and spatial coherence.
The key insight behind World Labs is that images and videos are fundamentally 2D projections of a 3D world. Traditional generative models learn to produce convincing 2D outputs, but they don’t actually understand the underlying 3D structure. World Labs’ approach starts from the opposite direction: build a rich 3D understanding, then render it back to 2D when needed. The result is spatial consistency that 2D-only models can’t match — objects don’t morph between frames, lighting stays consistent, and the geometry makes physical sense.
NVIDIA Cosmos and Physical AI
NVIDIA’s approach to world models is characteristically compute-heavy and simulation-first. At GTC 2026, Jensen Huang unveiled NVIDIA Cosmos, a platform for building “physical AI” that combines world foundation models with NVIDIA’s Omniverse simulation engine.
Cosmos isn’t one model — it’s a family of models trained on 20 million hours of video data, covering everything from highway driving to factory assembly lines to human motion. The training data includes real-world footage, synthetic data generated in Omniverse, and physics simulations. The goal is to give AI systems an understanding of physics that goes beyond what can be learned from passive video alone.
What makes Cosmos different from pure video generation models is its tight integration with simulation. A Cosmos model doesn’t just predict what a video frame will look like — it can predict what a robot’s sensors will register when it takes a specific action in a specific environment. That makes it directly useful for training autonomous systems: you can simulate thousands of edge cases, generate synthetic training data at scale, and validate control policies before deploying to physical hardware.
NVIDIA has also open-sourced portions of Cosmos under a permissive license, a strategic move that positions their hardware as the natural platform for anyone working on physical AI. If world models become the next big compute sink — and all signs suggest they will — NVIDIA’s GPUs are the obvious choice.
Google DeepMind’s Genie
Google DeepMind’s approach is perhaps the most conceptually ambitious. Genie, first demonstrated in early 2025 and expanded throughout 2026, is a “foundation world model” that can generate interactive, playable 3D environments from a single image or text prompt.
The magic of Genie isn’t just the visual quality — it’s the interactivity. You feed Genie a photo of a forest path, and it generates a fully navigable 3D environment where you can move forward, look around, and interact with objects. You show it a sketch of a game level, and it turns it into a playable game. The model has internalized enough about how the world works that it can extrapolate: it knows that paths continue, that objects have backs and sides even if they’re not visible in the input, that gravity pulls things down.
DeepMind has been characteristically tight-lipped about the technical details, but what’s publicly known suggests Genie uses a combination of video diffusion, a spatial action model, and a learned latent space that captures 3D geometry. The model was trained primarily on internet video — gameplay footage, first-person walking videos, drone flyovers — and learned to extract the underlying interactive structure without explicit 3D supervision.
OpenAI’s Sora and the Video Generation Path
When OpenAI released Sora in February 2024, the reaction was unprecedented. Here was a model that could generate minute-long videos with stunning visual coherence — reflections in puddles, dust motes in sunbeams, the precise way a woolly mammoth’s fur moves as it walks through snow. What made Sora special wasn’t just the resolution or the fidelity; it was the apparent understanding of physics.
OpenAI described Sora as a “world simulator” in their technical report, and while some researchers pushed back on that characterization, the videos spoke for themselves. Sora seemed to understand — at least approximately — object permanence, occlusion, fluid dynamics, and three-dimensional spatial relationships. It could track a subject through a scene, maintain consistent lighting across cuts, and handle complex camera movements that would break most video generation models.
The debate around Sora crystallized a key question: does generating plausible video actually require understanding the world, or is it just sophisticated pattern matching on a massive scale? The answer appears to be somewhere in between. Sora’s failures — objects spontaneously appearing or disappearing, physics breaking down in complex scenes, the “uncanny valley” of almost-but-not-quite-real motion — reveal the limits of learning world models purely from video. But its successes suggest that video-trained models do develop something resembling internal world representations, even if they’re incomplete.
Runway, Luma, and the Creative Frontier
Beyond the research labs, a wave of startups is building world model technology into tools for creative professionals. Runway’s Gen-3 Alpha, released in mid-2025, brought Sora-competitive video generation to a broader audience of filmmakers and designers. Luma AI’s Dream Machine takes a different approach, generating 3D scenes that can be explored from multiple angles rather than fixed-viewpoint video.
These tools are finding real adoption. VFX studios use them for pre-visualization and concept development. Game designers use them to rapidly prototype environments. Architects use them to generate walkthroughs from floor plans. The creative use cases are maturing faster than the robotics applications, partly because the stakes are lower — a hallucinated object in a concept video is acceptable; a hallucinated obstacle in an autonomous vehicle’s world model is not.
Why World Models Matter
The transition from language models to world models isn’t just an academic curiosity — it has profound implications for where AI creates value.
Robotics. This is the most obvious application. A robot that only understands language can follow instructions but can’t adapt to the physical world. A robot with a world model can navigate unstructured environments, manipulate unfamiliar objects, and recover from failures. Companies like Figure, Tesla, and Boston Dynamics are all investing heavily in world model approaches, and the results are starting to show: robots that can fold laundry, sort recycling, and navigate crowded warehouses without explicit programming for every scenario.
Autonomous vehicles. Self-driving cars already use primitive world models — they build 3D representations of their surroundings from sensor data and predict how other agents will move. Next-generation world models promise to make these systems more robust by handling edge cases that are too rare to appear in training data: a mattress falling off a truck on the highway, a child chasing a ball into the street, construction zones with non-standard signage.
Scientific simulation. World models trained on physics data could dramatically accelerate scientific research. Imagine a model that can simulate protein folding at atomic resolution, predict weather patterns weeks in advance, or model the structural behavior of new materials before they’re synthesized. These applications are still early-stage, but the potential impact is hard to overstate.
Creative tools and entertainment. The entertainment industry is already being reshaped by world model technology. Game engines, VFX pipelines, and virtual production workflows are integrating AI-generated 3D content. The next generation of video games may not have pre-built environments at all — they might generate worlds on the fly, uniquely tailored to each player’s actions and preferences.
The Challenges Ahead
For all the excitement, world models face significant hurdles.
Compute requirements. Training a world model requires orders of magnitude more compute than training an LLM of comparable parameter count. Video and 3D data are vastly higher-dimensional than text, and the physical simulations needed for ground-truth training data are expensive to run at scale. This creates a natural bottleneck: only a handful of companies have the resources to train frontier world models.
Data scarcity. The internet contains trillions of words of text, which is why LLMs have been able to scale so effectively. But high-quality 3D data is comparatively scarce. Most internet video is shaky, compressed, and captured from a single viewpoint. Synthetic data — generating training examples in simulation engines — helps, but it introduces its own problems: synthetic data rarely captures the full messiness of the real world.
Evaluation. How do you measure whether a world model is actually good? For language models, we have benchmarks like MMLU and HumanEval. For world models, there’s no equivalent. Physical accuracy, spatial consistency, and causal reasoning are much harder to quantify than multiple-choice test scores. The field needs better evaluation frameworks, and quickly — without them, it’s hard to know whether progress is real or just visually impressive.
Safety and misuse. A model that can generate convincing video of anything raises obvious concerns about deepfakes and misinformation. But world models introduce additional risks: a model that understands physical spaces could be used to plan real-world actions, from burglary to more serious threats. The AI safety community is only beginning to grapple with what responsible deployment of world models looks like.
What Comes Next
The trajectory is clear: AI is moving from language to space, from symbols to physics, from chatbots to world simulators. The companies and research labs that master world models won’t just build better chatbots — they’ll build systems that can operate in the physical world with something approaching human-level spatial understanding.
But it would be a mistake to frame this as LLMs versus world models. The most promising systems are hybrid: language models handle reasoning, communication, and high-level planning, while world models handle spatial understanding and physical simulation. Fei-Fei Li has described spatial intelligence as a complement to linguistic intelligence, not a replacement for it. A robot that can understand both “pick up the red cup” (language) and how to navigate around a table, grasp the cup without knocking it over, and adjust its grip if the cup starts to slip (spatial) is far more capable than one that can do either alone.
The summer of 2026 may be remembered as the moment world models broke into mainstream awareness. The technology is still raw — the demos are impressive but inconsistent, the compute costs are eye-watering, and the killer applications are still emerging. But the direction is unmistakable. After two years of teaching machines to talk, we’re finally teaching them to see, to move, and to understand the world they inhabit.
That’s a harder problem. It’s also a more important one.