frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

I built Vector. Now I'm answering the question your observability vendor won't

https://usetero.com/blog/the-question-your-observability-vendor-wont-answer
49•binarylogic•2h ago

Comments

binarylogic•2h ago
I spent a decade in observability. Built Vector, spent three years at Datadog. This is what I think is broken with observability and why.
yorwba•1h ago
I'm curious about the deep details, but the link 404s.
otterley•41m ago
And how are you solving the problem? The article does not say.

> I'm answering the question your observability vendor won't

There was no question answered here at all. It's basically a teaser designed to attract attention and stir debate. Respectfully, it's marketing, not problem solving. At least, not yet.

quadrature•8m ago
theres more information here https://docs.usetero.com/introduction/how-tero-works the link in the article is broken.

They determine what events/fields are not used and then add filters to your observability provider so you dont pay to ingest them.

matanyall•1h ago
It's so funny, I've never done a cost-benefit analysis of having "good monitoring" and then still not being able to figure out what broke and needing to pull in someone who doesn't need the monitoring at all because they built the thing.
peterldowns•1h ago
Ben, you probably don't remember me but you hired me ages ago to help out with the Python client for Timber. Just want to say thanks for that opportunity — it's been amazing to watch you guys succeed.

Also, I've ended up being responsible for infra and observability at a few startups now, and you are completely correct about the amount of waste and unnecessary cost. Looking forward to trying out Tero.

binarylogic•1h ago
Hey Peter, I absolutely remember you! Thanks for the nice comment.

And yes, data waste in this space is absurdly bad. I don't think people realize how bad it actually is. I estimate ~40% of the data (being conservative) is waste. But now we know - and knowing is half the battle :)

tot19•1h ago
Lurked on HN for years, and finally a post that made me excited enough to create an account.

First of all, thanks for you (and the team’s) work on Vector. It is one of my favorite pieces of software, and I rave about it pretty much daily.

New endeavor sounds very exciting, and I definitely can relate to the problem. Are there plans to allow Teri to be used in on-premises environments and self-hosted?

Thank you and good luck!

binarylogic•57m ago
Thank you for the nice comment. I'm glad you enjoy Vector. I poured myself into that software for many years. I'm a bit bummed with its current trajectory, though. We hope to bring the next evolution with Tero. There were many problems with Vector that I wished I could have fixed but was unable to. I hope to do those things with Tero (more to come!)

And yes, Tero is fundamentally a control plane that hooks into your data plane (whatever that is for you: OTel Collector, Datadog Agent, Vector, etc). It can run on-prem, use your own approved AI, completely within your network, and completely private.

tot19•49m ago
Appreciate the reply! Have you decided on a license yet?
stackskipton•1h ago
As Ops (DevOps/Sysadmin/SREish) person here, excellent article.

However, as always, the problem is more political than technical and those are hardest problems to solve and another service with more cost IMO won't solve it. However, there is plenty of money to be made in attempting to solve it so go get that bag. :)

At end of day, it's back to DevOps mentality and it's never caught on at most companies. Devs don't care, Project Manager wants us to stop block feature velocity and we are not properly staffed since we are "massive wasteful cost center".

binarylogic•51m ago
100% accurate. It is very much political. I'd also add that the problem is perpetuated by a disconnection between engineers who produce the data and those who are responsible for paying for it. This is somewhat intentional and exploited by vendors.

Tero doesn't just tell you how much is waste. It breaks down exactly what's wrong, attributes it to each service, and makes it possible for teams to finally own their data quality (and cost).

One thing I'm hoping catches on: now that we can put a number on waste, it can become an SLO, just like any other metric teams are responsible for. Data quality becomes something that heals itself.

mr-karan•21m ago
Just want to say thanks for creating Vector. We use it heavily at Zerodha and wrote about our setup here: https://zerodha.tech/blog/logging-at-zerodha/

It replaced both Filebeat and Logstash for us with a single binary that actually has sane resource usage (no more JVM nightmares). VRL turned out to be way more powerful than we could imagine - we do all our log parsing, metadata enrichment, and routing to different ClickHouse tables in one place. The agent/aggregator topology with disk buffering is pretty dope.

Genuinely one of my favorite pieces of infra software. Good luck with Tero.

smithclay•15m ago
Kudos to Ben for speaking to one of the elephants in the room in observability: data waste and the impact it has on your bill.

All major vendors have a nice dashboard and sometimes alerts to understand usage (broken down by signal type or tags) ... but there's clearly a need for more advanced analysis which Tero seems to be going after.

Speaking of the elephant in room in observability: why does storing data on a vendor cost so much in the first place? With most new observability startups choosing to store store data in columar formats on cheap object storage, think this is also getting challenged in 2026. The combination of cheap storage with meaningful data could breathe some new life into the space.

Excited to see what Tero builds.

jldugger•10m ago
>Turns out you can compile tens of thousands of patterns and still match at line rate.

Well, yea, sort of the magic of the regular expression <-> NFA equality theorem. Any regex can be converted to a state machine. And since you can combine regexes (and NFAs!) procedurally, this is not a surprising result.

> I ran it against the first service: ~40% waste. Another: ~60%. Another: ~30%. On average, ~40% waste.

I'm surprised it's only 40%. Observability seems to be treated like fire suppression systems: all important in a crisis, but looks like waste during normal operations.

> The AI can't find the signal because there's too much garbage in the way.

There's surprisingly simple techniques to filter out much of the garbage: compare logs from known good to known bad, and look for the stuff thats' strongly associated with bad. The precise techniques seem bayesian in nature, as the more evidence (logs) you get the more strongly associated it will appear.

More sophisticated techniques will do dimensional analysis -- are these failed requests associated with a specific pod, availability zone, locale, software version, query string, or customer? etc. But you'd have to do so much pre-analysis, prompting and tool calls that the LLM that comprise today's AI won't provide any actual value.

hinkley•8m ago
> You run observability at your company. But really, you're the cost police. You wake up to a log line in a hot path, a metric tag that exploded cardinality. You chase down the engineer. They didn't do anything wrong, they're just disconnected from what any of this costs.

Somebody didn’t math right when calculating if moving off hostedgraphite and StatsD was going to save us money or boil us alive. We moved from an inordinate number of individual stats with interpolated names to much simpler names but with cardinality and then the cardinality police showed up and kept harping on me to fix it. We were the user and customer facing portion of a SaaS company and I told them to fuck off when we were 1/7 of the overall stats traffic. I’d already reduced the cardinality by 400x and we were months past the transition date and I just wanted to work on anything that wasn’t stats for a while. Like features for the other devs or for our customers.

Very frustrating process. I suspect there’s a Missing Paper out there on how to compress stat cardinality out there somewhere. I’ve done a bit of work in that area but my efforts are in the 20% range and we need an order of magnitude. My changes were more about reducing the storage for the tags and reduced string arithmetic a bit in the process.

So, You've Hit an Age Gate. What Now?

https://www.eff.org/deeplinks/2026/01/so-youve-hit-age-gate-what-now
50•hn_acker•50m ago•22 comments

Why some clothes shrink in the wash – and how to 'unshrink' them

https://www.swinburne.edu.au/news/2025/08/why-some-clothes-shrink-in-the-wash-and-how-to-unshrink...
213•OptionOfT•3d ago•117 comments

Starlink roam 50GB is now 100GB with unlimited slow speed after that

https://starlink.com/support/article/58c9c8b7-474e-246f-7e3c-06db3221d34d
112•bahmboo•2h ago•75 comments

Dutch police have their own car stolen during car theft chase

https://dutchreview.com/news/dutch-police-car-stolen-during-chase/
19•ohjeez•34m ago•8 comments

Find a pub that needs you

https://www.ismypubfucked.com/
59•thinkingemote•2h ago•28 comments

Ford F-150 Lightning outsold the Cybertruck and was then canceled for poor sales

https://electrek.co/2026/01/13/ford-f150-lightning-outsold-tesla-cybertruck-canceled-not-selling-...
58•MBCook•57m ago•32 comments

The Unbearable Frustration of Figuring Out APIs

https://blog.ar-ms.me/thoughts/translation-cli/
33•ezekg•1h ago•15 comments

Edge of Emulation: Game Boy Sewing Machines (2020)

https://shonumi.github.io/articles/art22.html
66•mosura•3h ago•5 comments

Ask HN: Could you share your personal website here?

23•susam•1h ago•66 comments

There's a ridiculous amount of tech in a disposable vape

https://blog.jgc.org/2026/01/theres-ridiculous-amount-of-tech-in.html
664•abnercoimbre•2d ago•577 comments

I built Vector. Now I'm answering the question your observability vendor won't

https://usetero.com/blog/the-question-your-observability-vendor-wont-answer
50•binarylogic•2h ago•17 comments

Show HN: HyTags – HTML as a Programming Language

https://hytags.org
18•lassejansen•1d ago•4 comments

I’m leaving Redis for SolidQueue

https://www.simplethread.com/redis-solidqueue/
244•amalinovic•8h ago•98 comments

Xoscript

https://xoscript.com/history.xo
28•gabordemooij•2h ago•17 comments

Show HN: A 10KiB kernel for cloud apps

https://github.com/ReturnInfinity/BareMetal-Cloud
25•ianseyler•2h ago•1 comments

Government drops plans for mandatory digital ID to work in UK

https://www.bbc.com/news/articles/c3385zrrx73o
88•FridayoLeary•2h ago•27 comments

4k tons of potatoes to be given away for free in Berlin

https://www.the-berliner.com/english-news-berlin/4000-tons-of-potatoes-to-be-given-away-for-free/
90•mrzool•1h ago•68 comments

Lago (Open-Source Billing) is hiring across teams and geos

1•Rafsark•5h ago

Virginia Faulkner: Writer, Editor and Ghostwriter?

https://lithub.com/virginia-faulkner-writer-editor-and-ghostwriter/
5•samclemens•5d ago•1 comments

How have prices changed in a year? NPR checked 114 items at Walmart

https://www.npr.org/2026/01/14/nx-s1-5638908/walmart-prices-inflation-affordability-shrinkflation
63•srameshc•1h ago•30 comments

A Brief Introduction to the Basics of Game Theory

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1968579
41•7777777phil•2d ago•5 comments

I Hate GitHub Actions with Passion

https://xlii.space/eng/i-hate-github-actions-with-passion/
264•xlii•7h ago•214 comments

Show HN: Tiny FOSS Compass and Navigation App (<2MB)

https://github.com/CompassMB/MBCompass
102•nativeforks•7h ago•30 comments

1000 Blank White Cards

https://en.wikipedia.org/wiki/1000_Blank_White_Cards
324•eieio•15h ago•58 comments

System Programming in Linux: A Hands-On Introduction "Demo" Programs

https://github.com/stewartweiss/intro-linux-sys-prog
66•teleforce•8h ago•3 comments

FBI raids Washington Post reporter's home

https://www.theguardian.com/us-news/2026/jan/14/fbi-raid-washington-post-hannah-natanson
655•echelon_musk•3h ago•387 comments

Never-before-seen Linux malware is "more advanced than typical"

https://arstechnica.com/security/2026/01/never-before-seen-linux-malware-is-far-more-advanced-tha...
83•Brajeshwar•3h ago•20 comments

A 40-line fix eliminated a 400x performance gap

https://questdb.com/blog/jvm-current-thread-user-time/
341•bluestreak•19h ago•71 comments

Every GitHub object has two IDs

https://www.greptile.com/blog/github-ids
306•dakshgupta•1d ago•67 comments

ASCII Clouds

https://caidan.dev/portfolio/ascii_clouds/
309•majkinetor•15h ago•55 comments