1. I thought the table showing a bunch of models after my laptop specs would be models I could run on my laptop.
Model Tier Example Models Params RAM Needed Your Status
Top SOTA GPT-4o, Claude 3.5 Sonnet, Gemini Ultra 1T+ Datacenter —
Large Models Llama 3 70B, Mistral Large, Command R+ 70B 64 GB Usable
Mid Models Llama 3 8B, Mistral 7B, Phi-3 Medium 7-13B 16 GB Usable
Small Models Phi-3 Mini, Llama 3 2B, Gemma 2B 1-3B 4 GB Your Tier
2. I don't know what 'usable' means. How can these models be status:usable if they are not 'my tier'? I deduce 'usable' doesn't mean usable on my laptop but I don't know what 'usable' does mean.
3. Give me links to set up the best model I can.
(Edit: now it came to my mind that Firefox has fingerprinting protection by default. But even after whitelisting this website, the result is the same)
I do appreciate that the "canirun" one
1) was also wrong 2) but let me pick my real card from the dropdown with results that seemed much more accurate.
Genuine question - is there literally any factor besides RAM and GPU (with VRAM info?)
kumarski•3h ago
Thought it was nifty so shared. Would love to see a real CS person's version of this and/or best prompt to build something like this accurately and keep it up to date as models advance and SOTA shifts.