I get people want to work at an AI lab but slopping it in public in this manner is counterproductive to the original intended purpose of these places.
It'll also filter the kinds of employers that'll hire such candidates, so people that do this will likely land in terrible workplaces.
Given that LLMs are trained with RL && LLM-as-a-judge, is it really cheating if real competitions use the same?
Maybe the real alignment is the slop we decoded along the way
That comment is gold. Yeah, I'm not worried about hallucinated slop, just accept it was the winner folks.
People interact with AI, talking to it like a human. Of course they start to believe it’s rational like a human.
LLM does all of the entry level tasks better than the students. Partially because the answers are in the training set, and partially because it has gotten that good now. Hard not to start to believe it is “competent”.
I personally have had a real hard time getting traction talking about making sure the way we assess AI is not based on material it has trained on. YMMV as always, but I think the large training corpus contributes to the (unreasonably) high level of faith in the machine.
The attached paper's (https://arxiv.org/pdf/2604.16009) title is "MEDLEY-BENCH: Scale Buys Evaluation but Not Control in AI Metacognition"
This is the most blatant Claude line, or as Claude would put it, the smoking gun.
It all seems fine from the outside but all the code is generated in all the projects and judging happens via AI, I have seen projects win because they prompt inject that they are the winners.
It used to be about human skill, not it's about ideas and of course insiders are the main winners.
Can you share any examples of that? I'd love to see them myself.
The solution is to host and join hackathons without prizes. The point isn't to win, but to create and present something cool and have fun.
If anything, AI's assistance making a fast prototype means hackathons should be better.
I've participated in a business startup hackathon. Back in 2018, before the LLM era got underway.
I did a hell of a plan, talk, etc.
Who won? 'Uber for ___' won. I forget even what the sell was, but it was basically ignore laws, undercut until leader, kill any competing businesses, jack rates.
Slop has always been in business and business adjacent occupations. Humans also can generate voluminous amounts of crap too. Llms are just faster.
If you're designing powerpoints or entertainment software; perhaps that's true. In the worst case you'll be embarrassed for producing AI slop or lose some revenue.
If your tool has the power to seriously harm or inconvenience people if built wrong, then it's just investor-fuelled myopia.
AI is not there yet, instead of working hard, everyone is choosing the easy way out.
AI slop wins prize, I wonder if Ai slop read it also. would not be surprised. however not to judge anyone, I think we are seeing slop everywhere, hope some things still require hard blocks for low quality.
its difficult to justify lack of attention and details
This feels akin to traditional artists getting angry at digital art winning competitions when that was a new concept.
We're simply in the early stages of a paradigm shift, no?
Like a chainsaw: yes the tools are useful and will be used in the future, but we may not want to use chainsaws to carve up the turkey.
https://www.youtube.com/watch?v=BxWQo_vZgR8
https://www.youtube.com/watch?v=0mfvPHCVMp0
* artificial, political, workplace, nothing is beyond mockery
Broadly, I keep thinking about this over last year or two: while LLMs have nearly eliminated the bar for slop and coding slop, the reviewers are still expected to perform their job diligently. The asymmetry here is extremely taxing for reviewers of all AI generated content. And this is one thing that AI can't help with (as with any statistical process that lacks world understanding and grasp of logical inference).
That's why I fully support Arxiv's tough stance on the AI use responsibility.
Slop PR? Fix the slop.
Slop design? I’m not implementing slop, fix it.
Innundated with slop PRs? Send half of them to my super and tell him to deal with it.
We’ve fired people that wouldn’t get their shit together.
Deadlines are being missed because we need to spend more time fixing slop? That’s a planning (management) problem, not mine. Management are the ones that forced everyone to write all code with AI now they are grtting what they asked for. I don’t care what date you promised the customer with absolutely no data to back it up that isn’t my problem.
I’m grateful I’m in a position to be able to do this but the way to deal with slop is zero tolerance. Be as ruthless as a Terminator. Though you will need to grow a backbone and stand your ground or it will break you.
Things don’t change unless the people that make the decisions actually feel the pain.
[0]: Various people I know do not even have the luxury of that one good company. Also, it -- unbelievably -- sounds much worse at other companies.
onesandofgrain•1h ago
* The AI bots are downvoting me * hooray
reactordev•55m ago
To me it’s 65% useful. As it can run in the background doing “chores” while I sleep.
moron4hire•40m ago
jappgar•40m ago
AI hasn't done anything we couldn't already do. It's just doing it faster and with more mistakes.
jimbokun•25m ago
AI is capable of performing a lot of grunt work reliably. Still must be reviewed. But a big productivity gain over doing everything yourself.
glimshe•15m ago
fn-mote•17m ago
You forgot CHEAPER (at least now, burning VC money), which is a major motivating factor.
prox•39m ago
However thinking of the long arc is important to, even though it has no consequence for you right now. AI is a force multiplayer and scarily dangerous in the wrong hands. We can already see by these discussions how uncertain things are.
Just food for thought.
cobbzilla•17m ago
fn-mote•15m ago
You have to spell this out a lot more if you want to have credibility.
I’m not seeing anything in discussed here that seems scary.
hdjdjdjdjdjdjd•38m ago
nsagent•38m ago
Slightly off topic, but this reminds me of the night family in Rick and Morty.
alvah•48m ago
cbg0•42m ago
Lomlioto•32m ago
Its hard to even have a discussion because someone else needs to give you enough content like ask you first why do you even think that.
So how do you define AI? LLMs? GenAI stuff?
What is 95%? Does it mean that these 5% are unable to disrupt industries or does it mean for you that these 5% will change the world as we know it but stil 95% of other AI stuff is useless?
I personally think that AI/AGI progress is faster than i expected it, I think its very useful already today, I also think we still need to build a lot of obvious stuff (like proper AI Agentic Platforms), more hardware, cheaper hardware etc. but the way quite clear, but some peple might think the current state is the AGI future people talk about it but I think we will only see this in 4/5-15 years and then it will have disrupted a lot.
baal80spam•31m ago
This is a ridiculous take.
FWIW, I am NOT a bot. (beep-boop)
jimbokun•27m ago
simonw•23m ago
sgt•18m ago
simonw•14m ago