However essentially a world model is something that has the understanding of 3d world and can generate novel view point given either text or image input.

The use I have seen is for robotics. You feed in the current view and describe the action you want it to do, and then it plans the arm movements. (really useful for softbody manipulation.

There are other meanings. but essentially a world model is able to reason in 3d, rather than text.

Best definition I've heard is: AI systems that can build an internal map of their surroundings to anticipate what happens next and make decisions based on their predictions about the consequences the different actions they can take would have.

There's a bunch of different approaches people are trying:

- World labs (linked in this post) is going down the route of neural 3D representation work (NeRFs, 3D Gaussian Splatting)

- Yann LeCun is pretty famously betting on JEPA architectures (check out the excellent Welch Labs videos for more)

- Google is betting on generative video

- Karl Friston was pursuing 'active interference,' which is just traditional RL techniques with different reward functions