One senior-dev (team-lead also) tried to explain to me that AI is a subfield of machine-learning, and always stochastic in nature (since ChatGPT responds differently to the same prompt).
We/they are selling tailor-made "AI-products" to other businesses, but apparently we don't know how sampling works...? Also, no one could tell me where exactly our "self-hosted" models even ran (turns out 50% of the time its just OpenAI/Anthropic), or what OCR-model our product was using.
Am I just too junior/naive to get this or am I cooked?
illwrks•3h ago
I came to the same observations; lots of experts not much expertise.
I think my wider team are on par with their ability and understanding so we now can sift through the BS a bit easier.
Nod, smile, accept that no one has a clear understanding.
illwrks•3h ago
PaulHoule•3h ago
If you want to know why Hacker News is full of people disappointed or skeptical with AI ask yourself why they put 99.9% of their effort into “zero-shot” when it is clear as day that if you get a few thousand examples and train in that you wipe the floor with “zero-shot”
randomgermanguy•3h ago
illwrks•2h ago
I can tell you how a house is built, that doesn’t make me a builder that makes me informed and opinionated. I can decorate my house however I like but im not a painter/decorator or a tradesman. I can assemble some ikea furniture, but I’m not a carpenter. I’m a consumer and I can tweak something to my liking but I can’t do anything significant.
assemblyman•1h ago
One can argue that a lot of "building with AI" is commoditized by fine-tuning and RAG libraries or even reduced to prompt engineering. A lot of it is also tricks that might work on one dataset but not others. Putting together libraries fueled by pizza and coke gives an illusion of skill and speed.
Are there grifters who are jumping onto the AI bandwagon? Of course! In spades. Are there also engineers who want to build up their skills and are failing to do so or in the process of doing so? Of course, this happens too! But there are also people who are trying to understand, debug and improve models who are not necessarily "building". After all, the scaling laws paper (the original one) was a result of pure analysis of empirical data.
randomgermanguy•3h ago
illwrks•2h ago
Agencies are like a production line, they need raw materials coming in; clients with cash, armed with opportunities, scraps of ideas or formed briefs to be worked on. They need this business so they can generate the output and keep the lights on.
AI is everywhere and everything for a lot of people now. You can be sure that Exec’s are asking their teams how are we using AI, how is it helping the business grow etc. However there’s so much AI news, it’s moving so quick and seeping into everything that difficult (from a naïeve client point of view) to know what’s fantasy and what’s reality.
So my perception is… agencies do the sifting and maintain visibility of what is real or not because they have to start drumming up future sales and business, and AI is hot right now.
Perhaps they have some training in CoPilot etc, or with some experience of creating a model, maybe they have integrated something small with something big. It may even be that being ann angency they have a more open way of working that a corporate does, and that’s the sell.
Anyway, the sales teams will proclaim themselves experts because they have to sell.