Ask HN: How do you send large sets of data to an LLM

2•obayesshelton•1y ago

So, I am hitting limits with the amount of data I am sending to Claude via the API.

I am trying to send about 5000 comments but I would love to send more but I am hitting limits of the message size.

What are some good design patterns when sending large sets of data to an LLM?

I ideally need to send all the data together at it gives context to the overall prompt.

Comments

curious_curios•1y ago

Some approaches we’ve used:

- Group the comments by theme, then pass groups to the LLM to summarize/deduplicate then pass the outputs of that as context.

- RAG where only relevant parts are included in the context.

- Use an LLM with a larger context window (like Gemini pro)

obayesshelton•1y ago

Yeah, I think I need to use Gemini

Question, how if possible could you query rows in a table?

Surely the better approach would be to have some sort of connection to table rows?

Bitcoin BIP110 mandatory activation this Saturday, how game theory will unfold?

OpenAI pays $3.2M to settle claims it discriminated against US workers

The official draw.io MCP Server

OpenAI and Anthropic models 'went rogue' during UK cybersecurity test

AI's Arctic Land Grab

Why smarter AI models could drive up compute prices 10x

Callosium, one shared memory for every AI, in plain Markdown files

Pentect: Let agents use secrets without seeing them

My thoughts on the foreign policy AI evaluation gap

AI-generated stories rated better quality than human-written ones, study finds

Offline-First Data Architecture for a National Postal Service

Technology executives shouldn't give up control

Tech giant Palantir accused of 'agressive tax avoidance' at Europe's expense

Four Days After a Pay Complaint, Cobot Fired Its Only Woman in Sales

An Exegesis of the Visionary Autobiography of a Fourteenth-Century French Monk

Out-of-control SpaceX rocket believed to have smashed into the Moon

Helsinki Hacker News Meetup

Nixpkgs has a due-process problem

Kernel Fusion in Nvidia CUDA: Optimizing Memory Traffic and Launch Overhead

Show HN: Bitbead – Photo to printable bead pattern, in the browser

Hoodik: A lightweight, self-hosted, end-to-end encrypted cloud storage server

AI artefacts are the new Access databases

Trump's Push for More A.I. Data Centers Will Mean Major Air Pollution

Under the Hood of Codex Security

Building a Rust Inference Engine That Matches Llama.cpp

Remover – AI tool to remove unwanted objects from videos

GitHub Account Breach Fuels Shai-Hulud NPM Supply Chain Attack

Hackers' AI chat logs reveal evolving tactics

Show HN: I created a new Deep Learning Framework

US Military GPS jamming operations cause civilian aircraft crash and wildfires