I've been documenting ways to reduce my applications LLM spend for the last year. I've put together the first 6 parts on youtube. Its pretty easy to reduce spend by 60-80% from my eperience by just getting the basics right. I'd love to hear YOUR ideas as well, as this is an important part of launching an AI first app as a startup where saving cash is essential.
Troy.
bhagyeshsp•1h ago
Hi Troy,
I skimmed through the outline. Will take a look at the individual videos when I'm on PC.
But I have been through this cost-saving phase. I didn't see "prompt distillation" as one of the techniques in your outline.
The idea is to reduce your fixed prompt token size such as "system prompt" by removing semantic words completely and other methods. I saw a whopping 60% decrease in my fixed prompt token budget.
Pls note, my scale is small but the technique works nonetheless.
Edit: so, have you tried it? Or if you have tried it, how did it go?
troymagennis•1h ago
Troy.
bhagyeshsp•1h ago
I skimmed through the outline. Will take a look at the individual videos when I'm on PC.
But I have been through this cost-saving phase. I didn't see "prompt distillation" as one of the techniques in your outline.
The idea is to reduce your fixed prompt token size such as "system prompt" by removing semantic words completely and other methods. I saw a whopping 60% decrease in my fixed prompt token budget.
Pls note, my scale is small but the technique works nonetheless.
Edit: so, have you tried it? Or if you have tried it, how did it go?