frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Stop Blaming Embeddings, Most RAG Failures Come from Bad Chunking

2•wehadit•57m ago
Everyone keeps arguing about embeddings, vector DBs, and model choice, but in real systems, those aren’t the things breaking retrieval. Chunking drift is. And almost nobody monitors it. A tiny formatting change in a PDF or HTML file silently shifts boundaries. Overlaps become inconsistent. Semantic units get split mid-thought. Headings flatten. Cross-format differences explode. By the time retrieval quality drops, people start tweaking the model… while the actual problem happened upstream. If you diff chunk boundaries across versions or track chunk-size variance, the drift is obvious. But most teams don’t even version their chunking logic, let alone validate segmentation or check adjacency similarity. The Industry treats chunking like a trivial preprocessing step. It’s not. It’s the single biggest source of retrieval collapse, and it’s usually invisible. Before playing with new embeddings, fix your segmentation pipeline. Chunking is repetitive, undifferentiated engineering, but if you don’t stabilize it, the rest of your RAG stack is built on sand.

Comments

billconan•46m ago
how to do chunking? I recently tried llamaindex and some other opensource solutions. the result was poor, some words or sentences were split in the middle.
popidge•27m ago
Chunking strategy is really difficult and, like you say, so important to RAG. I'm currently battling with it in a "Podcast archive -> active social trend" clip-finder app I'm working on. You have to really understand your source material and how it's formatted, consider preprocessing, consider when and where semantic breaks happen and how you can deterministically handle that in the specific domain. Adjacency similarity is a must, otherwise you leave perfectly cromulent results on the table because they didn't have the right cosine score in a vacuum.

There is some early stuff from Apple's research labs and the ColBERT team in late attention embedding (https://arxiv.org/abs/2112.01488) which looks to ease that burden, and generate compressed token-level embeddings across a document.

AI Voice Agents Can Transform a Dental Clinic

1•Olivia8•2m ago•0 comments

Fixing Intel Wi-Fi Stuttering on Latest Driver (Global BG Scan Blocking)

https://dev.moe/en/3183
2•goodburb•3m ago•0 comments

The Performance Revolution in JavaScript Tooling

https://blog.appsignal.com/2025/12/03/the-performance-revolution-in-javascript-tooling.html
1•amalinovic•3m ago•0 comments

Show HN: Diagnostic security tool for Windows endpoints and networks

https://github.com/secuditor/secuditor-free
1•mennylevinski•3m ago•0 comments

Recovering from a broken update on the Turris Omnia

https://feeding.cloud.geek.nz/posts/recovering-from-broken-update-turris-omnia/
1•speckx•4m ago•0 comments

Chinese company could become the country's first to land a reusable rocket

https://arstechnica.com/space/2025/12/this-chinese-company-could-become-the-countrys-first-to-lan...
2•palmotea•7m ago•0 comments

Show HN: Tentropy Core – open-source to run AI system code in Firecracker VMs

https://github.com/jaliil-9/tentropy-core
1•Jalil9•8m ago•1 comments

Supabase ETL – Postgres Logical Replication Framework

https://supabase.com/blog/introducing-supabase-etl
1•msdrigg•8m ago•0 comments

The State of AI: Global Survery 2025 – McKinsey

https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai
1•eibrahim•8m ago•0 comments

Nvidia Holoscan: Platform for Real-Time Edge Computing

1•loughnane•9m ago•0 comments

Ask HN: I haven't had to buy a Windows computer in 20 years

2•meifun•10m ago•0 comments

The Thermodynamics of Trading

https://signalsandthreads.com/the-thermodynamics-of-trading/
1•tosh•10m ago•0 comments

Popping the cork on new low-cost carbon capture method

https://stories.tamu.edu/news/2025/11/13/popping-the-cork-on-new-low-cost-carbon-capture-method/
2•PaulHoule•10m ago•0 comments

Microsoft lowers AI software sales quota

https://finance.yahoo.com/news/microsoft-lowers-ai-software-sales-141531121.html
6•ramoz•10m ago•0 comments

Show HN: Grapevine – Accountless API for data with built-in pricing (x402)

https://github.com/PinataCloud/grapevine
1•kyletut•10m ago•1 comments

Ever, 8 spacecraft are docked to International Space Station

https://www.space.com/space-exploration/international-space-station/for-the-1st-time-ever-8-space...
1•Brajeshwar•11m ago•0 comments

Keeping link underline styles around inline code with padding

https://chriskirknielsen.com/blog/keep-link-underline-styling-around-inline-code-padding/
1•speckx•11m ago•0 comments

Sick ants invite self-sacrifice to save colony, "Hey, come and kill me"

https://www.cbsnews.com/news/sick-ants-invite-self-sacrifice-to-save-colony-study/
1•Brajeshwar•12m ago•0 comments

The Quiet Progress of Particle Physics

https://blog.melashri.net/posts/hep-progress/
1•elashri•12m ago•0 comments

An extra solar system planet once orbited next to Earth

https://www.livescience.com/space/planets/an-extra-solar-system-planet-once-orbited-next-to-earth...
2•Brajeshwar•12m ago•0 comments

Using Floating Point Numbers as Hash Keys (2017)

https://readafterwrite.wordpress.com/2017/03/23/how-to-hash-floating-point-numbers/
1•jstrieb•13m ago•0 comments

MailGPT: A Space for Your Thoughts

https://mailg.pt/
1•zyngaro•18m ago•0 comments

Scientists discover first gene proven to directly cause mental illness

https://www.sciencedaily.com/releases/2025/12/251202052230.htm
3•Noaidi•20m ago•2 comments

Show HN: Testing hypotheses through prediction is the next step towards AGI

https://github.com/Judahmeek/Significance-Hypothesis-Based-ARC-AGI-2-puzzle-solver
3•judahmeek•21m ago•3 comments

Show HN: A $20/year invoicing tool for solo developers (simple, fast, no bloat)

https://sidepay.app/
2•mightbefun•22m ago•0 comments

DritalHub – Free Social Media Scheduling Tool for Agencies

2•dritalzentech•23m ago•0 comments

Matthew Garrett Wins £70K Libel Judgment Against Techrights Publishers

https://fossforce.com/2025/11/matthew-garrett-wins-70k-libel-judgment-against-techrights-publishers/
2•LaSombra•24m ago•0 comments

The EU is finally blacklisting Russia for money laundering

https://www.politico.eu/article/the-eu-is-finally-blacklisting-russia-for-money-laundering/
3•RockstarSprain•24m ago•0 comments

Some new features updated in FastAPI-Voyager

https://www.newsyeah.fun/voyager/?tag=demo
1•tank-34•24m ago•1 comments

Confit Byaldi

https://en.wikipedia.org/wiki/Confit_byaldi
1•tosh•25m ago•0 comments