Most pixelation libraries take the lazy route where they just downscale the image and then upscale it back with nearest neighbor interpolation. It's fast but the results are usually pretty messy because the grid just cuts right through important features like faces or distinct edges. You lose a lot of the actual character of the image that way.
I tried something different where I treat pixelation as an optimization problem rather than a simple scaling operation. Instead of forcing a rigid grid onto the image it adapts the grid to match the underlying structure of the picture. The end result is pixel art that actually preserves the semantic details of the original image while still nailing that low-res aesthetic.
The secret sauce here is an edge-aware algorithm. It starts by running Sobel operators to detect edges and gradient magnitude. Once it has that data it initializes a standard grid but then iteratively moves the grid corners around to snap them to those detected edges. It uses a search-based optimization to test local positions for each corner and finds the spot that aligns best with the image boundaries.
It’s definitely heavier than the naive approach but optimizations to keep it usable. The edge detection runs on WebGL so you get a massive speedup there and it uses spatial hashing for O(1) lookups during the rendering phase. On modern hardware you are looking at maybe 100 to 500ms per image which is fast enough for most purposes. It’s pretty cool seeing the grid warp to fit the content rather than just chopping it up blindly.
yogthos•1mo ago
I tried something different where I treat pixelation as an optimization problem rather than a simple scaling operation. Instead of forcing a rigid grid onto the image it adapts the grid to match the underlying structure of the picture. The end result is pixel art that actually preserves the semantic details of the original image while still nailing that low-res aesthetic.
The secret sauce here is an edge-aware algorithm. It starts by running Sobel operators to detect edges and gradient magnitude. Once it has that data it initializes a standard grid but then iteratively moves the grid corners around to snap them to those detected edges. It uses a search-based optimization to test local positions for each corner and finds the spot that aligns best with the image boundaries.
It’s definitely heavier than the naive approach but optimizations to keep it usable. The edge detection runs on WebGL so you get a massive speedup there and it uses spatial hashing for O(1) lookups during the rendering phase. On modern hardware you are looking at maybe 100 to 500ms per image which is fast enough for most purposes. It’s pretty cool seeing the grid warp to fit the content rather than just chopping it up blindly.
https://github.com/yogthos/pixel-mosaic