I am trying to send about 5000 comments but I would love to send more but I am hitting limits of the message size.
What are some good design patterns when sending large sets of data to an LLM?
I ideally need to send all the data together at it gives context to the overall prompt.
curious_curios•9mo ago
- Group the comments by theme, then pass groups to the LLM to summarize/deduplicate then pass the outputs of that as context.
- RAG where only relevant parts are included in the context.
- Use an LLM with a larger context window (like Gemini pro)
obayesshelton•9mo ago
Question, how if possible could you query rows in a table?
Surely the better approach would be to have some sort of connection to table rows?