fp.

Ask HN: How do you send large sets of data to an LLM

2•obayesshelton•6mo ago

So, I am hitting limits with the amount of data I am sending to Claude via the API.

I am trying to send about 5000 comments but I would love to send more but I am hitting limits of the message size.

What are some good design patterns when sending large sets of data to an LLM?

I ideally need to send all the data together at it gives context to the overall prompt.

curious_curios•6mo ago

Some approaches we’ve used:

- Group the comments by theme, then pass groups to the LLM to summarize/deduplicate then pass the outputs of that as context.

- RAG where only relevant parts are included in the context.

- Use an LLM with a larger context window (like Gemini pro)

obayesshelton•6mo ago

Yeah, I think I need to use Gemini

Question, how if possible could you query rows in a table?

Surely the better approach would be to have some sort of connection to table rows?