Here's the thing. We're very likely to continue seeing VC funding in the slop. Counterintuitive, right? Why invest in these startups that are known to fail?
...because, if you connect the dots, the same VCs investing in these are those invested in the companies selling the LLMs. So it's literally funding companies to use and promote LLMs. Keep growing the hype and bubble. Keep normalizing LLM token cost. Keep playing on the FOMO. Bring on the AI doom.
It's absolutely worth having all of these startups fail for them. They'll spend millions and maybe billions in marketing. That's what most of these startups are - marketing. Pawns.
Remember, if you want to follow the whole (what I call) "AI doomer timeline" towards the end of it you'll realize that all/most SaaS startups will have a near zero or zero value. Why? Because anyone will be able to simply prompt AI to build the software they need. Not just "one size fits all" SaaS software that comes with limitations, missing features you need (but not enough of the other users need), other multi-tenant issues, and security concerns, and of course recurring cost...but people are going to be able to build CUSTOM single-tenant software with AI without recurring costs.
SaaS was always based on the age old question of buy vs. build. The only reason it works is because it cost more to build (and maintain). On our "AI doomer timeline" this will eventually shift and no longer be the case.
Ergo, these startups are mostly doomed for failure and it's in the best interest of VCs to invest in them to accelerate the timeline. Their goal is to increase AI adoption and phase out SaaS.
resiros•7mo ago
First thing about building AI or LLM powered apps is that 90% of the time you don't need to train models from scratch. LLMs are models that have the world's knowledge, so you can "train them" by prompting them and giving them the right context.
Where to start? I think the first thing is to play around with the OpenAI playground to get an idea of what's possible with prompts.
You'll find tons of frameworks out there: LangChain, LlamaIndex, Agno, and the list goes on. But in my opinion, if you have a use case in mind, start very very simple. Almost a single prompt using the OpenAI SDK or compatible SDK like LiteLLM.
The core challenge isn't technical complexity, it's the stochastic nature of LLMs. Your prompt works perfectly on one example, breaks on another. You fix it for the second case, and it stops working for the first.
The whole trick of building an LLM powered application is finding how to deal with this in an iterative approach. You need to have a workflow that allows you to iterate quickly. For this, start very simple and then add complexity. Don't use frameworks at the beginning.
I think Hamel's approach of error analysis (https://hamel.dev/notes/llm/officehours/erroranalysis.html) is a very good starting point. The idea is that you create a prompt, run it against a test set, then identify clusters of error types and systematically fix them.
What tools do you need? In the beginning, just a playground and a Jupyter Notebook. At some point when you want to look into LLMOps tools. They provide you with observability (so if you have an agent with multiple steps you can see each step), running evals and comparing results easily, creating test sets from prod data... All the stuff that makes your life easier.
There are a lot of these, langsmith, helicone, braintrust, humanloop.. Full disclosure, I'm CEO of one of these tools (Agenta) [one of the reasons I'm answering this post :D].
But I think from your question you're not at the stage where you should look at LLMOps yet. Focus on understanding your use case first and starting a simple repeatable process.