frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Show HN: Ex-Deloitte auditor open-sourced the whole SOC 2 method for your AI

https://github.com/Chiaro-HQ/methodology
28•yylyyl•1h ago

Comments

yylyyl•1h ago
I'm the author. CPA, spent 5 years doing SOC 2 fieldwork at Deloitte, recently started my own audit firm.

This repo is the methodology we actually audit against, not a summary of it: 86 controls, 355 test attributes with pass criteria, evidence standards, and the Type II testing method including how we call deviations, with worked examples. It's generated from the same JSON that drives our audit tooling, so it can't drift from what we actually do.

What "for your AI" means concretely: the framework is JSON rather than prose, so the controls, the attributes, their pass criteria and the evidence map are all machine-readable, and there's a file listing every tool our server exposes. It's CC BY 4.0 — point your own model at it and run your own readiness against the same bar the examination applies. The part that makes that work is 498 calibration examples: each one records a judgment call, the verdict an AI reached, the verdict that was correct, and why. Without those a model grades itself generously.

Why publish it: the audit criteria (AICPA's Trust Services Criteria) are public, but every firm's actual testing layer is a black box. A buyer holding two SOC 2 reports can't tell whether one auditor inspected evidence and the other just collected screenshots — the reports look identical. This year's Delve episode (hundreds of near-identical reports, procedures allegedly drafted before client evidence arrived — allegations Delve disputes) made that opacity harder to defend. Standards bodies publish their standards; we think the testing layer should be public too.

The thing that surprised us building it: sampling mostly shouldn't exist at small-company scale. Sampling exists in audit because looking at everything used to be expensive. At the company sizes we serve, populations are tiny — for most of our 79 sample-typed attributes, a proper sample would have been most of the population anyway — and AI collapsed the cost of looking at the rest. So the method defaults to testing complete populations, and sampling survives only as a disclosed fallback with hash-seeded selection that nobody (client, us, or either side's software) can steer.

Limitations, stated plainly: this is a methodology, not a track record. The firm is new — one Type I issued, no Type II yet. We published the method before the first Type II run on purpose, so it can't be quietly fitted to results afterward. Peer review: enrolled, first review due 2027.

Happy to answer anything about how SOC 2 audits actually work from the inside.

devy•54m ago
Is this how every auditor does though or just you/Deloitte? SOC 2 is a set of guidelines, not mandates, there is one size fits all and every org may design their own security controls based on their unique systems. Is that the same philosophy on the auditing side as well?
yylyyl•47m ago
The method is mine. How I test, what evidence I accept, how I call a deviation. Every firm has their own methodology but none of them publish it.
bookmon•52m ago
Don't most people get the SOC 2 audit to also have credibility from a reputable firm? How valid would an AI SOC 2 audit be for marketing purposes
yylyyl•45m ago
Your AI can read the open source methodology to get audit ready for you without relying on a compliance platform. The actual audit is done by a licensed firm.
bijowo1676•50m ago
Thank you for sharing this, I have a question: can you tell us some lore behind SOC 2 certification, why is this certification is most frequently pursued by startups/tech companies?

Do all SOC 2 audit certifications worth the same, or some worth more, (big four vs smaller firm?)

Are there other/better alternative certifications that provide higher level of assurance to clients?

yylyyl
jpitz•40m ago
This is an incredible resource for someone trying to prep for an audit. Thank you!
yylyyl•37m ago
Appreciate it! Happy to answer any questions.
•
38m ago
SOC 2 is almost a must-have for startups if they want to pursue enterprise deals. Not all soc 2 audits worth the same. People usually trust 2 types of audits: 1. a reputable name or 2. a transparent report that can show all the details like what evidence the auditor checks, how the auditor checks it, and what the auditor finds.

And soc 2 currently is the mostly widely accepted one. There are other more specialized ones for sure but not as universally accepted as soc2.

Show HN: Simple algorithm and color space to generate diverse skin tones

https://toneyalexander.github.io/inclusive-color-space/
193•automatoney•2h ago•47 comments

Ray Bradbury's "There Will Come Soft Rains" is set today (2026-08-04)

https://short-stories.co/@raybradbury/there-will-come-soft-rains-6k8vr4xxlnmj
470•askvictor•7h ago•211 comments

When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation

https://arxiv.org/abs/2602.16763
30•doppp•1h ago•31 comments

DeepSeek V4 Flash on a Single AMD MI300X

https://github.com/ryanzhou/deepseek-v4-flash-mi300x
283•zhoutong•7h ago•62 comments

Germany Records Historic 12B KWh Solar Feed-In in July 2026

https://solarquarter.com/2026/08/03/germany-records-historic-12-billion-kwh-solar-feed-in-in-july...
103•johnbarron•3h ago•102 comments

Keyv and friends compromised in active Shai-Hulud supply chain attack

https://www.aikido.dev/blog/keyv-and-friends-compromised-in-npm-supply-chain-attack
150•cimi_•6h ago•72 comments

Truemetrics (YC S23) Is Hiring in Berlin – GTM Lead

https://www.ycombinator.com/companies/truemetrics/jobs/bIQQ7tP-founding-gtm-lead
1•truemetricsIngo•34m ago

LLMs reward expertise

https://www.seangoedecke.com/llms-reward-expertise/
1258•MaxMussio•20h ago•518 comments

Online ad giant Adform was hacked, proving once again why ad blockers are needed

https://this.weekinsecurity.com/online-advertising-giant-adform-was-hacked-proving-once-again-why...
104•speckx•2h ago•26 comments

Buckminster Fuller: everything I know

https://www.bfi.org/about-fuller/everything-i-know/
92•simonebrunozzi•6h ago•29 comments

Apple says more ex-employees may have taken confidential data to OpenAI

https://techcrunch.com/2026/08/04/apple-says-more-ex-employees-may-have-taken-confidential-data-t...
110•thewebguyd•1h ago•74 comments

Dates That Don't Exist (2015)

https://blog.yossarian.net/2015/06/09/Dates-That-Dont-Exist
70•EndXA•3d ago•47 comments

First Came the DOGE Cuts, Then Came the Wildfires

https://www.outdoorlife.com/conservation/doge-cuts-forest-service-wildfires/
9•hn_acker•27m ago•2 comments

Harness Engineering for Self-Improvement

https://lilianweng.github.io/posts/2026-07-04-harness/
228•tosh•11h ago•44 comments

Xbox goes down. You can't play games you own on disc

https://birchtree.me/blog/xbox-goes-down-you-cant-play-games-you-own-on-disc/
407•surprisetalk•5h ago•473 comments

Looking inside a 1970s PROM chip that stores data in microscopic fuses (2019)

https://www.righto.com/2019/07/looking-inside-1970s-prom-chip-that.html
26•Jimmc414•3d ago•4 comments

The AI Demand Bubble

https://www.wheresyoured.at/the-ai-demand-bubble/
61•7777777phil•1h ago•16 comments

Why Large Language Models Fail at Tabular Prediction

https://arxiv.org/abs/2608.02412
76•sbulaev•7h ago•30 comments

Show HN: Fine-tune an 8B model on a 4 GB laptop GPU

https://github.com/MakazhanAlpamys/Soup
92•MakazhanAlpamys•6h ago•14 comments

RCade: The Arcade Cabinet with CI/CD Deployment, Custom Graphics Card for CRT [video]

https://www.youtube.com/watch?v=W-OpIbLUOU0
28•evakhoury•5d ago•7 comments

Why etymologies matter: How tracing words can illuminate history (2024)

https://resobscura.substack.com/p/why-i-love-etymologies
73•benbreen•3d ago•16 comments

Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone

https://github.com/leonickson1/Swiftlet
278•leonickson•1d ago•128 comments

Rebuilding and analysing 4 years of Wordle stats from WhatsApp chat logs

https://blog.omgmog.net/post/rebuilding-wordle-stats-from-whatsapp/
17•surprisetalk•1d ago•4 comments

Ten advances in mathematics and theoretical computer science

https://openai.com/index/ten-advances-in-mathematics/
605•milkshakes•1d ago•888 comments

Devtools must be open source

https://blog.exe.dev/devtools-must-be-open-source
694•bryanmikaelian•1d ago•227 comments

Amazonian civilization had estimated 3M people in 3% of forest area

https://www.science.org/content/article/odd-shapes-hidden-dense-amazon-rainforest-reveal-sprawlin...
249•marojejian•6d ago•181 comments

Homebench – Benchmark local LLMs for speed, memory, and quality

https://github.com/david-g-3654/homebench
48•davai-g•7h ago•3 comments

There Will Come Soft Rains (1950) [pdf]

https://users.wpi.edu/~zrbutzke/Docs/BradburyStories(1).pdf
291•pmg101•18h ago•106 comments

Webb telescope finds signs of ancient disaster for Neptune's moons

https://www.reuters.com/science/webb-telescope-finds-signs-ancient-disaster-neptunes-moons-2026-0...
17•Teever•1h ago•3 comments

Learning-Rust.Github.io: Rust Programming Language Tutorials for Everyone

https://learning-rust.github.io
37•dumindunuwan•7h ago•7 comments