frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Ask HN: How do you send large sets of data to an LLM

2•obayesshelton•1y ago
So, I am hitting limits with the amount of data I am sending to Claude via the API.

I am trying to send about 5000 comments but I would love to send more but I am hitting limits of the message size.

What are some good design patterns when sending large sets of data to an LLM?

I ideally need to send all the data together at it gives context to the overall prompt.

Comments

curious_curios•1y ago
Some approaches we’ve used:

- Group the comments by theme, then pass groups to the LLM to summarize/deduplicate then pass the outputs of that as context.

- RAG where only relevant parts are included in the context.

- Use an LLM with a larger context window (like Gemini pro)

obayesshelton•1y ago
Yeah, I think I need to use Gemini

Question, how if possible could you query rows in a table?

Surely the better approach would be to have some sort of connection to table rows?

/N

1•LoboxPlatform•3m ago•0 comments

Hours later, Airbyte is still unavailable

https://status.airbyte.com
1•LilBytes•5m ago•0 comments

Nvidia says its open source AI strategy remains intact after Jensen Huang's exit

https://techcrunch.com/2026/11/17/nvidia-says-its-open-source-ai-strategy-remains-intact-after-je...
1•anon373839•9m ago•0 comments

Nuke SF

https://nukesf.com/
2•iLuddite•12m ago•0 comments

Show HN: Prototype of a product management tool for AI agent companies

https://featurestream-prototype.vercel.app/
1•srinidhimurali0•13m ago•0 comments

Deutsche Bank hired a Qatari royal. Why?

https://www.ft.com/content/9edd3730-3a15-4e1a-b33a-629aff00e847
2•JumpCrisscross•14m ago•0 comments

Show HN: Neopane. Run Neovim once, show its windows in your multiplexer's panes

https://github.com/spiritledsoftware/neopane
2•ianpascoe•15m ago•1 comments

IRA Velinsky, industrial designer for Commodore and Atari

https://dfarq.homeip.net/ira-velinsky-industrial-designer-of-the-commodore-64/
2•onename•16m ago•0 comments

Voyager 1 hits one light day from Earth

https://science.nasa.gov/mission/voyager/where-are-voyager-1-and-voyager-2-now/
6•robin_reala•17m ago•1 comments

Welcome to the Era of Second-Screen Reading

https://www.theatlantic.com/books/2026/11/sarah-j-maas-acotar-second-screen-reading/689474/
2•FinnLobsien•18m ago•0 comments

Underrated HTML Attributes Worth Using

https://frontendcurated.curated.co/issues/2
2•javatuts•22m ago•0 comments

Voyager Will Carry Earth Sounds Record (1977)

https://science.nasa.gov/missions/voyager-program/voyager-will-carry-earth-sounds-record/
2•divbzero•29m ago•0 comments

Marijke Van Thielen is the first female philosopher naked in Playboy

https://playboy.nl/artikel/700346/marijke-van-thielen-is-de-eerste-filosofe-in-playboy
3•Michelangelo11•29m ago•1 comments

Reaktiv Workbook – A spreadsheet where every cell is Python

https://workbook.bui.app/
1•buibuibui•31m ago•0 comments

After 49 Years, Voyager 1 Is a Mind-Blowing One Light-Day from Earth

https://www.nytimes.com/2026/11/18/science/nasa-voyager-1-earth-light-day.html
2•quapster•33m ago•0 comments

Data centres consume 5 percent of the Netherlands' electricity

https://www.cbs.nl/en-gb/news/2026/47/data-centres-consume-5-percent-of-the-netherlands-electricity
3•HelloUsername•34m ago•0 comments

Show HN: Arbituro

https://arbituro.com/
1•arbituro•37m ago•0 comments

Why people crack games, and why I think buying them is still the better bet [video]

https://www.youtube.com/watch?v=2C1FasPUssg
1•frag•39m ago•0 comments

Security flaw found in Java, third-largest programming language

https://www.bbc.com/news/articles/cke89jevqm1vo
1•Lyngbakr•40m ago•0 comments

Show HN: Qpilot – an AI agent that runs plain-text manual test cases in Chrome

https://github.com/broxhq/qpilot
2•broxhq•40m ago•0 comments

Why didn't Apple make the Mac Apple II compatible? (2018)

https://woz.org/didnt-apple-make-mac-apple-compatible/
2•thamiltonsmith•42m ago•0 comments

Nvidia Q3 FY27 Earnings: $115.8B Revenue, $390B Supply Gamble

https://oplexa.com/insights/company-intelligence/nvidia-q3-fy27-earnings-ai-infrastructure-analysis/
2•oplexa07•43m ago•0 comments

Tired of reading changelogs? VS Code is trying something different

https://code.visualstudio.com/updates/v1_147
1•mechanicalartic•44m ago•0 comments

Ow did NASA achieve their live TV broadcast in 1969? (2013)

https://space.stackexchange.com/questions/2993
3•Skyfall106•46m ago•0 comments

Mojo: Towards One Systems Language for All Hardware, SPLASH'26 Talk [video]

https://www.youtube.com/watch?v=9-97SB5XJh4
1•pjmlp•46m ago•0 comments

Skala

https://bjango.com/mac/skala/
1•robenkleene•47m ago•0 comments

Accelerating Development for Massive Git Repositories

https://developers.googleblog.com/accelerating-development-for-massive-git-repositories/
2•kumrayu•48m ago•0 comments

After Days of Digital Outages, Americans Wonder If the Country Is Breaking

https://www.nytimes.com/2026/11/17/us/politics/outages-flights-banks-tech.html
6•edward•51m ago•2 comments

Show HN: Offline knowledge library for survival and rebuilding

https://lastbook.apposters.com/
1•ehurynovich•51m ago•0 comments

Harness adapters: Safe RL for efficient agentic development

https://huggingface.co/blog/harness-hub/harness-adapters
1•musha68k•53m ago•0 comments