I remember when the local AI community was just a few thousand enthusiasts who were curious about these new language models. We used to sit aside and watch OpenAI make strides with their giant models, and our wish was to bring at least some of that power to our measly small machines, locally.
Then Meta's Llama-1 leak happened and it opened the pandora's box of AI. Was it better than GPT-3.5? Not really, but it kick started the push to making small capable models. Llama.cpp was a turning point. People figured out how to run LLMs on CPU.
Then the community came up with GGML quants (later renamed to GGUF), making models even more accessible to the masses. Several companies joined the race to AGI: Mistral with their mistral-7b and mixtral models really brought more performance to small models and opened our eyes to the power of MoE.
Many models and finetunes kept popping up. TheBloke was tirelessly providing all the quants of these models. Then one day he/she went silent and we never heard from them again (hope they're ok).
You could tell this was mostly an enthusiasts hobby by looking at the names of projects! The one that was really out there was "oobabooga" The thing was actually called "Text Generation Web UI" but everyone kept calling it ooba or oobabooga (that's its creator's username).
Then came the greed... Companies figured out there was potential in this, so they worked on new language models for their own bottom-line reasons, but it didn't matter to us since we kept getting good models for free (although sometimes the licenses were restrictive and we ignored those models).
When we found out about LoRA and QLoRA, it was a game changer. So many people finetuned models for various purposes. I kept asking: do you guys really use it for role-playing? And turns out yes, many people liked the idea of talking to various AI personas. Soon people figured out how to bypass guardrails by prompt injection attacks or other techniques.
Now, 3 years later, we have tens of open-weight models. I say open-WEIGHT because I think I only saw one or two truly open-SOURCE models. I saw many open source tools developed for and around these models, so many wrappers, so many apps. Most are abandoned now. I wonder if their developers realized they were in high demand and could get paid for their hard work if they didn't just release everything out in the open.
I remember the GPT-4 era: a lot of papers and models started to appear on my feed. It was so overwhelming that I started to think: "is this was singularity feels like?" I know we're nowhere near singularity, but the pace of advancements in this field and the need to keep yourself updated at all times has truly been amazing! OpenAI used to say they didn't open-source GPT-3 because it was "too dangerous" for the society. We now have way more capable open-weight models that make GPT-3 look like a toy, and guess what, no harm happened to the society, business as usual.
A question we kept getting was: "can this 70B model run on my 3090?" Clearly, the appeal of running these LLMs locally was great, as can be seen by looking at the GPU prices. I remain hopeful that Nvidia's monopoly will collapse and we'll get more competitive prices and products from AMD, Intel, Apple, etc.
I appreciate everyone who taught me something new about LLMs and everything related to them. It's been a journey.