I made an interactive write-up that walks you through the A to Z of vector embeddings, inspired by Brendan Bycroft's incredible interactive LLM visualisation guide (see here - https://bbycroft.net/llm).
Most of my work is on encoder models, and I kept noticing how few good interactive resources there are on embeddings. The ones that do exist tend to either plot some embeddings in 3D or only explain a narrow applied technique like semantic search or RAG.
So I wanted to make something that teaches how embeddings work by letting you play with them. In the write-up, there are interactive visualisations that allow you to explore the concepts of vectorisation, cosine similarity, semantic search, hierarchical retrieval, clustering, zero-shot classification and deduplication.
The centrepiece of the post is a game I've dubbed 'SemanticGeussr'. It's inspired by Wordle and GeoGuessr, and the goal is to draw a loop around a random target word by triangulating guesses on a 2D map of 43,339 common English words.
It's admittedly a bit intimidating at first, as semantic space does not perfectly match our intuition of meaning. But once you get the hang of it, the game becomes really addictive. I recently had the pleasure of demoing the game to some law students at the University of Melbourne, and we had a lot of fun throwing different vector equations at it to see where they'd land.
Some fun ones to get started with:
london - english + french
running - land + water
avg(breakfast, lunch, dinner)
0.4 * noodles + 0.6 * pasta
I've also included Python snippets alongside some of the visualisations if you want to follow along with the techniques later.
It was great fun to make and I hope you have fun playing around with it. Happy to take any questions, and if there's enough interest I might set up a dedicated games page for some of the other encoder-based games I've been working on.
Do let me know what highest score you get based on your target word and chosen path :)
raxxorraxor•32m ago