frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

A Tale of Two AI Failures: Debugging a Simple Bug with LLMs

https://bitmovin.com/blog/hackathon-debugging-ai-tools-llms/
9•slederer•2mo ago

Comments

kichik•2mo ago
Not exactly the point of this article, but it would be cool if APIs like this can return the expected signed string for debugging. It would have to be properly limited for security. But if the API is expecting non-standard signatures, it could help developers with better debugging tools.
lillesvin•2mo ago
Given that you can't infer the error from simply looking at the signature string, I don't see how having the expected string rather than a simple "OK" or "mismatched signature" (as you get now) would make a difference?
kichik•2mo ago
You can save the expected string to a file, save your string to a file, and run diff on a hexdump of both. Even without hexdump, you should see the difference between "\n" and "\\n" in properly escaped output.
lillesvin•2mo ago
But the returned signed string will be an HMAC-SHA256 hash, won't it? Then there's not going to be any '\n' or '\\n's in there. Only thing you'll be able to tell is if it matches your hash or not, in which case 'OK' or 'not OK' will work just as well.

Or am I misunderstanding you?

kichik•2mo ago
You are indeed misunderstanding me. I am talking about returning the entire string to be signed. Not the result of the signature.
lillesvin•2mo ago
Ah, my bad. Sorry.

But couldn't you then just make the call to an echo service (like HTTPbin) or simply dump the request when you send it?

kichik•2mo ago
The echo server will have no knowledge on how to construct the string to be signed.
lillesvin•2mo ago
But neither does the actual server. HMAC only verifies that the message is from whoever it claims to be from and that it is intact. It won't know what you intended the body of the request to look like.
lillesvin•2mo ago
I know it's kinda besides the point and I don't know what language this was being done in, but I don't personally know any language where

    String signature = "POST" + "\n" + "/api/v1/..."
and

    String signature = "POST\n/api/v1/..."
don't result in identical variables, so I'm a bit puzzled why that would result in an error.

However, there's a quoting error in the failing example where the double quotes in the JSON body aren't properly escaped:

    String signature = "POST" + "\n" + "/api/v1/query" + "\n" + token + "\n" + timestamp + "\n" + "{"body":"content"}"
It may just be the example that's not correctly formatted, but the other (working) example does in fact escape the double quotes in the JSON. I guess, depending on how forgiving the used language is with quoting, that could also be the source of the error?
juancn•2mo ago
Yeah, I'm stuck here.

Another thing that's really broken is the last string with unescaped quotes.

Not sure how to interpret that unless theres a `:` (colon) operator.

saint_yossarian•2mo ago
The author mentions FoxESSCloud, which led me to https://www.foxesscloud.com/public/i18n/en/OpenApiDocument.h... with this Python example:

    signature = fr'{path}\r\n{token}\r\n{timestamp}'
So if this is indeed the API they're using it's not only literal "\\n" but also "\\r\\n", no "POST", and no body at the end.
nebster•2mo ago
I... still don't understand the issue. It looks like both examples in the table would evaluate to the same thing. Am I missing a stray "\n"?
fernly•2mo ago
agree, I feel dumb but don't see subtle issue.

Also when copy/pasting into Python to try it, I got an error because \“ is in fact U+201C not an ASCII quote. (Surely that's not the subtle issue?)

DylanSp•2mo ago
Echoing the others who say they can't understand the bug/difference; only thing I can think of is that the input string needed the escape sequence for a newline in it? So the correct code would be written as

    "POST" + "\\n" + ...
thehappypm•2mo ago
tl;dr: custom, naïve Concatenation formatting implementation can cause bugs
jojomodding•2mo ago
Perhaps this article was written by the same AI that failed to understand what it was supposed to do in the first place? The post doesn't make a lot of sense and the writing seems fishy. I still don't understand what was wrong with he first code.
snowfield•2mo ago
I often find myself clearing the context when dealing with llms to get a fresh take. Often it just has so much context reinforcing its previous decisions.

Not sure if the author tried to just start a new thread. But anyway, for now you always need to keep an eye on these things and manage it if it follows red herrings or ends up in some logical loop

Sidenote : newlines is one thing tat can be quite tricky for llms in general.

P2P crypto exchange development company

1•sonniya•4m ago•0 comments

Vocal Guide – belt sing without killing yourself

https://jesperordrup.github.io/vocal-guide/
1•jesperordrup•9m ago•0 comments

Write for Your Readers Even If They Are Agents

https://commonsware.com/blog/2026/02/06/write-for-your-readers-even-if-they-are-agents.html
1•ingve•9m ago•0 comments

Knowledge-Creating LLMs

https://tecunningham.github.io/posts/2026-01-29-knowledge-creating-llms.html
1•salkahfi•10m ago•0 comments

Maple Mono: Smooth your coding flow

https://font.subf.dev/en/
1•signa11•17m ago•0 comments

Sid Meier's System for Real-Time Music Composition and Synthesis

https://patents.google.com/patent/US5496962A/en
1•GaryBluto•24m ago•1 comments

Show HN: Slop News – HN front page now, but it's all slop

https://dosaygo-studio.github.io/hn-front-page-2035/slop-news
4•keepamovin•25m ago•2 comments

Show HN: Empusa – Visual debugger to catch and resume AI agent retry loops

https://github.com/justin55afdfdsf5ds45f4ds5f45ds4/EmpusaAI
1•justinlord•28m ago•0 comments

Show HN: Bitcoin wallet on NXP SE050 secure element, Tor-only open source

https://github.com/0xdeadbeefnetwork/sigil-web
2•sickthecat•30m ago•1 comments

White House Explores Opening Antitrust Probe on Homebuilders

https://www.bloomberg.com/news/articles/2026-02-06/white-house-explores-opening-antitrust-probe-i...
1•petethomas•30m ago•0 comments

Show HN: MindDraft – AI task app with smart actions and auto expense tracking

https://minddraft.ai
2•imthepk•35m ago•0 comments

How do you estimate AI app development costs accurately?

1•insights123•36m ago•0 comments

Going Through Snowden Documents, Part 5

https://libroot.org/posts/going-through-snowden-documents-part-5/
1•goto1•37m ago•0 comments

Show HN: MCP Server for TradeStation

https://github.com/theelderwand/tradestation-mcp
1•theelderwand•40m ago•0 comments

Canada unveils auto industry plan in latest pivot away from US

https://www.bbc.com/news/articles/cvgd2j80klmo
3•breve•41m ago•1 comments

The essential Reinhold Niebuhr: selected essays and addresses

https://archive.org/details/essentialreinhol0000nieb
1•baxtr•43m ago•0 comments

Rentahuman.ai Turns Humans into On-Demand Labor for AI Agents

https://www.forbes.com/sites/ronschmelzer/2026/02/05/when-ai-agents-start-hiring-humans-rentahuma...
1•tempodox•45m ago•0 comments

StovexGlobal – Compliance Gaps to Note

1•ReviewShield•48m ago•1 comments

Show HN: Afelyon – Turns Jira tickets into production-ready PRs (multi-repo)

https://afelyon.com/
1•AbduNebu•49m ago•0 comments

Trump says America should move on from Epstein – it may not be that easy

https://www.bbc.com/news/articles/cy4gj71z0m0o
6•tempodox•49m ago•3 comments

Tiny Clippy – A native Office Assistant built in Rust and egui

https://github.com/salva-imm/tiny-clippy
1•salvadorda656•54m ago•0 comments

LegalArgumentException: From Courtrooms to Clojure – Sen [video]

https://www.youtube.com/watch?v=cmMQbsOTX-o
1•adityaathalye•57m ago•0 comments

US moves to deport 5-year-old detained in Minnesota

https://www.reuters.com/legal/government/us-moves-deport-5-year-old-detained-minnesota-2026-02-06/
8•petethomas•1h ago•3 comments

If you lose your passport in Austria, head for McDonald's Golden Arches

https://www.cbsnews.com/news/us-embassy-mcdonalds-restaurants-austria-hotline-americans-consular-...
1•thunderbong•1h ago•0 comments

Show HN: Mermaid Formatter – CLI and library to auto-format Mermaid diagrams

https://github.com/chenyanchen/mermaid-formatter
1•astm•1h ago•0 comments

RFCs vs. READMEs: The Evolution of Protocols

https://h3manth.com/scribe/rfcs-vs-readmes/
3•init0•1h ago•1 comments

Kanchipuram Saris and Thinking Machines

https://altermag.com/articles/kanchipuram-saris-and-thinking-machines
1•trojanalert•1h ago•0 comments

Chinese chemical supplier causes global baby formula recall

https://www.reuters.com/business/healthcare-pharmaceuticals/nestle-widens-french-infant-formula-r...
2•fkdk•1h ago•0 comments

I've used AI to write 100% of my code for a year as an engineer

https://old.reddit.com/r/ClaudeCode/comments/1qxvobt/ive_used_ai_to_write_100_of_my_code_for_1_ye...
3•ukuina•1h ago•1 comments

Looking for 4 Autistic Co-Founders for AI Startup (Equity-Based)

1•au-ai-aisl•1h ago•1 comments