frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Ask HN: How do you send large sets of data to an LLM

2•obayesshelton•1y ago
So, I am hitting limits with the amount of data I am sending to Claude via the API.

I am trying to send about 5000 comments but I would love to send more but I am hitting limits of the message size.

What are some good design patterns when sending large sets of data to an LLM?

I ideally need to send all the data together at it gives context to the overall prompt.

Comments

curious_curios•1y ago
Some approaches we’ve used:

- Group the comments by theme, then pass groups to the LLM to summarize/deduplicate then pass the outputs of that as context.

- RAG where only relevant parts are included in the context.

- Use an LLM with a larger context window (like Gemini pro)

obayesshelton•1y ago
Yeah, I think I need to use Gemini

Question, how if possible could you query rows in a table?

Surely the better approach would be to have some sort of connection to table rows?

What is Apple CMF(Color Matching Functions) and what's still missing

https://fullanalysis.substack.com/p/what-is-apple-cmfcolor-matching-functions
1•pierrefermat1•2m ago•0 comments

StackOverflow Won't Accept They're Dead [video]

https://www.youtube.com/watch?v=nAdxeGWVAoI
1•bmn__•4m ago•0 comments

The Horse Made the State

https://aeon.co/essays/the-origins-of-the-state-are-not-agrarian-but-pastoralist
1•ersinesen•5m ago•0 comments

Unifi Adoption for OpenWRT Devices

https://github.com/antoniocosta/unifi-openwrt
1•laurensr•6m ago•0 comments

European Company Email Isn't a Microsoft-Google Duopoly

https://ciphercue.com/blog/europe-email-microsoft-google
1•adulion•7m ago•0 comments

Science Isn't a Series of Lucky Accidents

https://worksinprogress.co/issue/accidental-discoveries/
1•ersinesen•7m ago•0 comments

Chris van Eijk: invalid input syntax for type JSON in PostgreSQL, measured

https://now-next.nl/en/insights/postgresql-invalid-input-syntax-for-type-json/
1•omegaSigma95•7m ago•0 comments

Simple Postmortem Tool

https://www.probablybrokeit.com/
1•kethchetan•9m ago•0 comments

Loadwise: Where data centers meet the grid

https://loadwise.ai
1•doener•11m ago•0 comments

Can 5k Lines of C11 Outrun ClickHouse, DuckDB, and MonetDB?

https://blog.manchon.xyz/blog/day23/
1•donperignon•13m ago•0 comments

Saying Goodbye to Lemon Squeezy

https://www.lemonsqueezy.com/blog/saying-goodbye-to-lemon-squeezy
2•FinnLobsien•13m ago•0 comments

Show HN: MeshLive – Offline Video calls and livestreams over Bluetooth mesh

https://meshlive.github.io/
1•gptgautam1•14m ago•0 comments

GitHub Redesign

https://tonsky.me/blog/github-redesign/
1•decentstates•24m ago•1 comments

Mini Organ Made from Paper [video]

https://www.youtube.com/watch?v=ujdlxAQw5oA
1•igpay•26m ago•0 comments

Ukraine critically damages Kerch bridge linking occupied Crimea with Russia

https://www.theguardian.com/world/2026/nov/17/ukraine-strikes-critical-damage-russia-kerch-bridge...
5•defly•26m ago•2 comments

An intention spec – workflow for the agentic dev cycle

https://djordjepuzic.com/blog/posts/2026-11-13-intention-spec/
1•djordjep•28m ago•0 comments

My computer misbehaved so I went down a rabbit hole (2019)

https://leonwu.tech/posts/my_computer_misbehaved_so_i_went_down_a_rabbit_hole
1•sammatilda•31m ago•0 comments

Def3r/haka: A lightweight, fast and plugable hotkey utility

https://github.com/def3r/haka
1•ankitg12•36m ago•0 comments

Beer.drv.drv.drv

https://fzakaria.com/2026/11/14/beer-drv-drv-drv
1•birdculture•36m ago•0 comments

Ops Needs Friction

https://www.jeffgeerling.com/blog/2026/ops-needs-friction/
1•dverlaeckt80•36m ago•0 comments

OpenAI sued by California over Hugging Face attack

https://www.bbc.com/news/articles/c6wy4xv2mxglo
1•vrganj•37m ago•0 comments

Brian Eno's idea of Scenius – visualised

https://www.neil-scott.com/scenius/
1•f91w•37m ago•0 comments

Quality IMAP, Part 1: Your Mailbox Is a Database

https://aerogramme.deuxfleurs.fr/blog/2026-quality-imap-part-1/
1•dverlaeckt80•37m ago•0 comments

Show HN: Call Nara. An AI voice companion built with older people in mind

https://apps.apple.com/au/app/call-nara/id6794990805
1•anthonyjg_sec•39m ago•0 comments

The founder of Teenage Engineering opens up to his creative space (2022)

https://scandinavianmind.com/human-touch-interview-jesper-kouthoofd-teenage-engineering/
1•tosh•41m ago•0 comments

Try Pragtical – Fast, extensible C and Lua code editor in the browser

https://pragtical.dev/try-it
2•rd07•42m ago•0 comments

Leak of 15M Deutsche Bank files exposes systemic failures

https://www.ft.com/content/df518845-ed97-4a38-8a38-7fa9d4d24003
2•petethomas•42m ago•0 comments

Coding Agents Underestimate the Cost of Code They Own

https://ben3d.ca/blog/coding-agents-underestimate-the-cost-of-code
1•sebg•51m ago•0 comments

A new evolution of databases: Self learning Databases

https://heatherdb.com/
1•nguthiru•52m ago•0 comments

Oxide Integrates Micron Memory and Storage into Its Cloud Computer

https://oxide.computer/blog/oxide-micron-second-generation-cloud-computer
1•tosh•56m ago•0 comments