I have a 3090 w/ 24G VRAM, it supports flash attention and bf16, but is a bit old. I dont run generative models on it as they would be slow and inaccurate, but I do run embedding models, OCR, and system one (decision) models on it.

I grabbed a minisforum before they got prohibitively expensive ( and before that a framework variant, but I got a dud from them ). If you can't do that you can estimate what you can run model wise and play with LMS ( some interesting options are now available -- in my recent tests GLM and Qwen are doing well for local stuff. )

Note.. you can always try to pool things ( not memory, that is super hard; devices as there are more small models now that you can squeeze onto things ).

I'm running an RTX 5090 (32gb VRAM) and Strix Halo (128gb unified memory). I haven't had much time to tinker with my AI setup so I'm still running Qwen3.6-27B (NVFP4 quant) on the 5090. The prices today are just insane.

I run a cluster of two Dell Pro Max GB10's 128GB unified mem per node 200G NIC ConnectX