Ask HN: What's a good format to submit CSV data for LLMs?

2•JimsonYang•3h ago

I need to submit like 1000 rows of data to an llm so I can ask it for trends within the data. If I use json, I check gpt tokenizer and thats like 40 tokens per row(cuz headers were being referenced everytime leading to inefficiency). Meaning 40k input, which definitely would put me in context rot(hallucination) territory. And I heard using csv was very inaccurate. Any suggetions

Comments

eimrine•3h ago

you can use good old algorythms to search your special trends. just ask LLM how to code them. any algo you might need is somewhere inside of Donald Knuth's books.

JimsonYang•2h ago

mierz00•1h ago

We analyse thousands of lines from a csv using an LLM. The only thing that worked for us was to send each individual line and analyse it one by one.

I’m not sure if that would work in your use case, but you could classify each line into a value using an LLM then hard code the trends you are looking for.

For example if you’re analysing something like support tickets. Use an LLM to classify the sentiment, and you can plot the sentiment on a graph and see if it’s trending up or down.

Ask HN: What's the current best local/open speech-to-speech setup?

Ask HN: Seeeking help to reverse engineer a PCB

Ask HN: Do you "micro-manage" your agents?

Ask HN: Why don't winter gloves have mechanical fingers?

Ask HN: Where is society heading, is there a plan for a jobless future?

Dismantling dark patterns using TDD via discovery and injunctions

Ask HN: Do B2B deals stall more from "org blindness" than product fit?

Ask HN: What's a good format to submit CSV data for LLMs?

Tell HN: 2 years building a kids audio app as a solo dev – lessons learned

Ask HN: Will agentic AI grow to handle technology leadership responsibilities?

The Cognitive Cockpit: Why I moved my data to a 1-bit e-ink display

Ask HN: Do you have any evidence that agentic coding works?

Ask HN: LLMs for new job categories?

AI hallucinate. Do you ever double check the output?

Locked out of my GCP account for 3 days, still charged, can't redirect domain

Working on reducing wasted distribution effort before publishing posts

Tell HN: Cloudflare's D1 service degraded since 2 days

Ask HN: Does DDG no longer honor "site:" prefix?

Ask HN: Room left for the "industrial" systems engineer in a post-cloud world?

Ask HN: Have you seen your Palantir file?

Ask HN: Why are so many rolling out their own AI/LLM agent sandboxing solution?

Ask HN: How are you handling non-probabilistic security for LLM agents?

Ask HN: Best practice securing secrets on local machines working with agents?

Ask HN: COBOL devs, how are AI coding affecting your work?

Ask HN: How realistically far are we from AGI?

What software businesses worked in the early Internet? Can they work again?

Ask HN: Has the macOS design become too iOS-like?

Tell HN: Cursor agent force-pushed despite explicit "ask for permission" rules

Ask HN: Is Claude Down for You?

Ask HN: How do you authorize AI agent actions in production?

Ask HN: What's a good format to submit CSV data for LLMs?

Comments

Ask HN: What's the current best local/open speech-to-speech setup?

Ask HN: Seeeking help to reverse engineer a PCB

Ask HN: Do you "micro-manage" your agents?

Ask HN: Why don't winter gloves have mechanical fingers?

Ask HN: Where is society heading, is there a plan for a jobless future?

Dismantling dark patterns using TDD via discovery and injunctions

Ask HN: Do B2B deals stall more from "org blindness" than product fit?

Ask HN: What's a good format to submit CSV data for LLMs?

Tell HN: 2 years building a kids audio app as a solo dev – lessons learned

Ask HN: Will agentic AI grow to handle technology leadership responsibilities?

The Cognitive Cockpit: Why I moved my data to a 1-bit e-ink display

Ask HN: Do you have any evidence that agentic coding works?

Ask HN: LLMs for new job categories?

AI hallucinate. Do you ever double check the output?

Locked out of my GCP account for 3 days, still charged, can't redirect domain

Working on reducing wasted distribution effort before publishing posts

Tell HN: Cloudflare's D1 service degraded since 2 days

Ask HN: Does DDG no longer honor "site:" prefix?

Ask HN: Room left for the "industrial" systems engineer in a post-cloud world?

Ask HN: Have you seen your Palantir file?

Ask HN: Why are so many rolling out their own AI/LLM agent sandboxing solution?

Ask HN: How are you handling non-probabilistic security for LLM agents?

Ask HN: Best practice securing secrets on local machines working with agents?

Ask HN: COBOL devs, how are AI coding affecting your work?

Ask HN: How realistically far are we from AGI?

What software businesses worked in the early Internet? Can they work again?

Ask HN: Has the macOS design become too iOS-like?

Tell HN: Cursor agent force-pushed despite explicit "ask for permission" rules

Ask HN: Is Claude Down for You?

Ask HN: How do you authorize AI agent actions in production?