Colnomic and nvidia models are great for embedding images and MUVERA can transform those to 1D vectors.
“the pipeline” - seems like this is just a personal hackathon project?
Why these models vs other multimodals? Which “nvidia models”?
"Images of german shepherds" never fails to provide some humor.
This NGA link returns over a thousand pieces by Rothko: https://www.nga.gov/artists/1839-mark-rothko/artworks
The next version can see the image and read the metadata.
A bit more context: We are include everything in the latent space (embeddings) without trying to maintain multiple indexes and hack around things. There is still a huge mountain to climb. But this one seems really promising.
Since this is a semantic search, using a vector embedding, it will handle meanings better than a text search, which would handle names better.
Would be interesting to know how relevant that approach is now.
philipkglass•4h ago
Some searches work like magic and others seem to veer off target a lot. For example, "sculpture" and "watercolor" worked just about how I'd expect. "Lamb" showed lambs and sheep. But "otter" showed a random selection of animals.
breadislove•4h ago
The search is in beta and we improving the model. Thank you for reporting the queries which are not working well.
Edit: Re the otter, I just checked and I did not found otters in the dataset. We should not return any results if the model is not sure to reduce confusion.
justincormack•4h ago
breadislove•4h ago
philipkglass•4h ago
I also expected semantic search to return similar results for "fireworks" and "pyrotechnics," since the latter is a less common synonym for the former. But I got many results for fireworks and just one result for pyrotechnics.
This is still impressive. My impulse is to poke at it with harder cases to try to reason about how it could be implemented. Thanks for your Show HN and for replying to me!
breadislove•4h ago