frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Ask HN: Not treated respectfully by colleague – advice?

107•golly_ned•3d ago•119 comments

Ask HN: Is AWS down again?

75•ajdude•21h ago•35 comments

Tell HN: macOS 26 is making me have regrets for the first time in 12yrs

14•trumbitta2•1d ago•13 comments

Tell HN: OpenAI now requires ID verification and won't refund API credits

196•retube•3d ago•117 comments

Ask HN: Dealing with "blocked" emails after DNS issue

3•nvahalik•23h ago•2 comments

Ask HN: Advice for creating a USB device linking 2 computers

20•WorldDev•2d ago•42 comments

Killer WiFi cards can block VPNs

10•willprice89•1d ago•2 comments

Ask HN: What are you doing this week?

4•SafeDusk•13h ago•7 comments

Ask HN: How should new programmers learn in the AI era?

8•loa_observer•1d ago•11 comments

Bugbunny: Securing VibeCoded Apps

5•zaddyzaddy•22h ago•0 comments

Google Demanded My Drivers Lic Before Letting Me Read an Article

77•keernan•3d ago•31 comments

Ask HN: Good LLM Observability Platforms?

5•seany62•1d ago•5 comments

What do you guys do to improve your focus?

13•pervysage•1d ago•14 comments

Ask HN: Has anyone deployed your own MCP server connector to ChatGPT?

4•QueensGambit•1d ago•3 comments

Ask HN: Do Java and .NET developers avoid learning new tech?

4•jerawaj740•14h ago•5 comments

Ask HN: Anyone else use FreePascal as their low level language?

19•rlawson•4d ago•11 comments

M5 Macs Support Memory Integrity Enforcement

14•abalone•2d ago•0 comments

Ask HN: Best open source opsgenie alternatives?

2•Poomba•1d ago•2 comments

Ask HN: What is a passkey and why is everybody asking for one lately?

22•CGMthrowaway•3d ago•6 comments

iPhone Safari Lost Bookmarks

2•vocatan•1d ago•2 comments

The Windows 7 Renaissance? StatCounter shows surge in usage

4•Stasshe•1d ago•5 comments

Ask HN: Rigorous study on what jobs are declining due to AI now?

7•AznHisoka•2d ago•6 comments

Is there an IDE that can use the local open-source model?

3•haebom•2d ago•9 comments

Ask HN: Any good books for a layman on history of quantum computing?

25•bmau5•5d ago•6 comments

Using jet engines for power generation at AI centers.

13•testrun•2d ago•5 comments

Ask HN: What's needed for a minimal production Docker deployment?

5•bcye•2d ago•4 comments

Ask HN: What do you use for focus without coffee jitters?

13•jaemo•4d ago•22 comments

Would the .NET community benefit from an open-source MassTransit fork?

9•Nakib•3d ago•7 comments

Ask HN: Where should an experienced developer start learning AI development?

15•productiveminds•4d ago•8 comments

Ask HN: Best practices for research code?

13•Eugeleo•5d ago•5 comments
Open in hackernews

Ask HN: Good LLM Observability Platforms?

5•seany62•1d ago
My company has been through 3 different "LLM Observability" vendors and they each have failed to give us the one (simple) thing we want. Willing to pay for this.

The ONLY thing we care about is the ability to: - Log an LLM completion, and be able to press a button that lets us re-run the exact same completion in a UI (industry seems to call this the "playground"). We can rerun this completion exactly how it was in production.

What we DO NOT care about: - "datasets" - "scores" - "prompt enhancers"

Comments

uaas•1d ago
I am curious, what’s the point of re-running these interactions on a UI?
muzani•3h ago
Reproduction I suppose. I would like the same things as OP too.

LLM outputs are qualitative; they can't really be automatically scored and prompt enhancements tend to multiply the bug. It can solve a problem, but introduce a new one. It's practical just to do it manually.

thiago_fm•1d ago
I'm sure if you ask Claude Code exactly that, they will develop what you want.

Tell it to create an API for the LLM data ingestion, then integrate with it on your software.

BTW, this is far from what an LLM Observability tool will offer you. You are a bit confused what O11Y is.

debadyutirc•22h ago
What entails the LLM Completion are you talking sequence of prompts with files / mcp servers. Could you share a bit more, cause I have spent some time with this and have something that might be precisely what you are asking for...
Wonnk13•16h ago
When I think of LLM / Agent observability I think of some combination of open telemetry and like Influxdb, but I don't think that's what your asking for?