frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

Open in hackernews

Beyond Attention: Toward Machines with Intrinsic Higher Mental States

https://arxiv.org/abs/2505.06257
66•holografix•1d ago

Comments

quinnjh•1d ago
This is, intuitively, a really exciting title. Looking forward to reading / seeing similar work.
bwest87•1d ago
I did a chat with Gemini about the paper, and tldr is... * They introduce a loop at the beginning between Q, K, and V vectors (theoretically representing "question", "clues" and "hypothesis" of thinking) * This loop contains a non linearity (ReLU) * The loop is used to "pre select" relevant info * They then feed that into a light weight attention mechanism.

They claim OOM faster learning, and robustness acro domains. There's enough detail to probably do your own PuTorch implementation, though they haven't released code. The paper has been accepted into AMLDS2025. So peer reviewed.

At first blush, this sounds really exciting and if results hold up and are replicated, it could be huge.

saagarjha•1d ago
I don't want to dismiss this outright but I'm skimming this paper and pretty skeptical of something that's from a single guy that doesn't appear peer reviewed, spends most of its time talking about actual biology, comes up with a "RELU6" (RELU but minimum value 6), and then pushes detailed review to a future paper.
amelius•1d ago
He wrote this paper, "Cooperation is All You Need", with a group of people:

https://arxiv.org/pdf/2305.10449

And this paper in an IEEE journal:

https://arxiv.org/pdf/2211.01950

yorwba•1d ago
Figure 3 B in "Cooperation is All You Need" shows the same score curves as the top left of Figure 6 in "Beyond Attention," so it must be basically the same implementation. Yet that earlier paper is only cited once, in the Acknowledgements section. As far as I can tell, the only mathematical change in this paper is capping the ReLU at 6. But it also adds a bunch of grandiose verbiage ("triadic modulation loops", "awake thought.")

The author is clearly a crackpot. Maybe he wasn't a crackpot when he still managed to publish in peer-reviewed journals, but cognitive decline over time is not exactly unheard of.

frozenseven•1d ago
Cool insults. But perhaps you can explain why he's wrong?
anothermathbozo•23h ago
Warrantless and totally spiteful for you to make unqualified claims like “cognitive decline” from skimming two papers. This is shameful.
habinero•1d ago
I swear, most of the AI "papers" that get posted here are someone screwing around with ChatGPT on ketamine and deciding they're advancing humanity.
ivape•1d ago
You’ve just discovered the future of a jobless economy. Please write a blog post and I will surely upvote you.

Ketamine is all you need

geeunits•1d ago
Sat here vibe coding a pure assembly kernel for arm64, APL layer with conceptual memory layout. On my bed, eating a bag of chips, jobless since Jan. Everything but the Ket are mine
ivape•1d ago
You serious?
geeunits•1d ago
yasqueen ← {'yes'≡⎕C ⍵}
TeMPOraL•20h ago
Who knows, but drop the word "vibe" and this is basically the startup culture 15 years ago, so ¯\_(ツ)_/¯.

Well, okay, for better historical accuracy, replace APL with API, and the kernel for arm64 thing with Ruby on Rails on a new Macbook, but the point still stands.

ldng•1d ago
Can the anthropomorphic scam continue unchecked ? Apparently yes.
ImHereToVote•1d ago
If modeling cognitive processes is a scam, then neuroscience must be the longest-running con in history.
TeMPOraL•20h ago
Probably as long as non-anthropomorphic idiocy can.

No opinion on this submission, but a more general point. I'm not the one to jump into anthropomorphizing computers, but last year or two of LLM and adjacent research is a constant stream of papers and experiments that totally surprise everyone who refuse to even entertain comparisons between LLMs and people, while being entirely expected and completely not surprising to those who do.

mirekrusin•1d ago
Results in this paper look way too good, I guess we'll have to wait for peer reviews and replications to see if it's true.
RockyMcNuts•1d ago
When you stack transformers, don't you get meta-attention and higher mental states?
edflsafoiewq•18h ago
I don't understand the "Triadic Modulation Loop" block, does anyone else?

Also

> Competing interests: AA has a provisional patent application for the algorithm used in this paper.

Ask HN: How does Nextdoor verify phone billing address matches home address?

https://help.nextdoor.com/s/article/How-to-verify-your-address?language=en_US
1•deejaybog•58s ago•1 comments

Play Diffusion for Instant Audio Editing

https://blog.play.ai/blog/play-diffusion
1•amrrs•1m ago•0 comments

Show HN: I created a free invoice generator tool. Generate pdf in realtime

https://mvpwrappers.com/free-tools/invoice-gen
1•eashish93•3m ago•0 comments

Meta Goes Military Private FaceMash reporting for duty, sir

https://nymag.com/intelligencer/article/mark-zuckerbergs-meta-is-pivoting-to-defense-contracting.html
2•chrisdoc•3m ago•0 comments

Technical Debt Is for Everybody

https://thenewcuriosityshop.substack.com/p/technical-debt-is-for-everybody
1•gHeadphone•4m ago•0 comments

Giving a Shit as a Service

https://allenpike.com/2022/giving-a-shit
1•lopespm•7m ago•0 comments

Disaster Awaits If We Don't Secure IoT Now

https://spectrum.ieee.org/iot-security-root-of-trust
1•mdp2021•7m ago•0 comments

Nvidia to cut gaming GPU shipments by 30% by reallocating them to enterprise AI

https://overclock3d.net/news/cpu_mainboard/nvidia-reportedly-plans-to-cut-rtx-50-series-gpu-production/
1•speckx•8m ago•0 comments

Vienna sends Blue Danube into space

https://apnews.com/article/strauss-blue-danube-waltz-space-b87ac23d0060e7211fd097e631f21db2
2•sinnfeinn•8m ago•0 comments

AI Is Learning to Escape Human Control

https://www.wsj.com/opinion/ai-is-learning-to-escape-human-control-technology-model-code-programming-066b3ec5
1•lucaspauker•8m ago•0 comments

Behind the scenes of Rust string formatting and format_args ()

https://blog.m-ou.se/format-args/
1•fanf2•9m ago•0 comments

A European non-profit that investigates influential and opaque algorithms

https://aiforensics.org/
1•mooreds•12m ago•0 comments

Colt, Honeywell and Nokia join forces to trial space-based quantum-safe

https://www.nokia.com/newsroom/colt-honeywell-and-nokia-join-forces-to-trial-space-based-quantum-safe-cryptography/
1•donutloop•14m ago•0 comments

Behind the Curtain: A white-collar bloodbath

https://www.axios.com/2025/05/28/ai-jobs-white-collar-unemployment-anthropic
1•TurkishPoptart•15m ago•0 comments

Humanity May Achieve the Singularity Within the Next 6 Months, Scientists Sugges

https://www.popularmechanics.com/science/a64929206/singularity-six-months/
1•stuckinhell•16m ago•0 comments

JavaScript Imports Under the Hood (2023)

https://blog.jim-nielsen.com/2023/imports-under-the-hood/
2•pie_flavor•16m ago•0 comments

Why a Utah Chalet Is the Perfect Setting for the Ultra Wealthy in 'Mountainhead'

https://www.nytimes.com/2025/05/31/business/dealbook/mountainhead-house.html
1•mooreds•18m ago•1 comments

99 Dev Problems with Jamie Tanna [video]

https://www.youtube.com/watch?v=H9WX7OYpRRw
2•mooreds•19m ago•0 comments

Ask HN: What will humans do when AI writes and reviews code?

2•pomarie•20m ago•3 comments

Vanta bug exposed customers' data to other customers

https://techcrunch.com/2025/06/02/vanta-bug-exposed-customers-data-to-other-customers/
1•coloneltcb•22m ago•0 comments

Jemalloc Repositories Are Archived

https://github.com/jemalloc
5•zX41ZdbW•22m ago•1 comments

Decorative Text Within HTML

https://shkspr.mobi/blog/2025/05/decorative-text-within-html/
2•tobr•25m ago•0 comments

Twain Dreams

https://harpers.org/archive/2025/06/twain-dreams-samuel-clemens-john-jeremiah-sullivan/
1•samclemens•25m ago•0 comments

Designing for Neurodiversity

https://www.smashingmagazine.com/2025/06/designing-for-neurodiversity/
1•ulrischa•29m ago•0 comments

Elon and AI at Tesla

https://twitter.com/aelluswamy/status/1799646232559899098
2•bilsbie•31m ago•2 comments

Finance leaders fear destructive U.S. debt scenario

https://www.axios.com/2025/06/02/us-treasury-auction-debt-interest-rates
2•kaycebasques•32m ago•0 comments

LLMs: The Missing Compiler for Unix Tools

https://tselai.com/llms-unix-tools
1•fforflo•32m ago•0 comments

"After School" – From LAN Party (2024)

https://www.blog.radiator.debacle.us/2025/05/after-school-from-lan-party-2024.html
2•speckx•33m ago•0 comments

A Giant Plume of Saharan Dust Is Headed to Florida

https://www.nytimes.com/2025/06/02/weather/saharan-dust-florida-gulf-coast.html
3•Stratoscope•34m ago•1 comments

Bill Gates to give most of his $200B fortune to Africa

https://www.bbc.co.uk/news/articles/cn4qg5gzgzxo
7•microsoftedging•37m ago•0 comments