Built a simulator of 18 Backrooms levels in PyTorch to generate the data.
Recorded 440 hours of scripted play in it.
Trained the model from scratch on that, using flow matching in a pretrained autoencoder's latent space.
The model was trained on its own imperfect frames as context so it doesn't drift right away. It recalls a few older frames by position, so a quick look back usually works.
Inspired by Oasis Minecraft.
To try it you need an NVIDIA GPU with ~4 GB.
Weights: https://huggingface.co/yelarys/noclip
Video with sound: https://x.com/yelaryss