frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Falsify: Hypothesis-Inspired Shrinking for Haskell (2023)

https://www.well-typed.com/blog/2023/04/falsify/
90•birdculture•1y ago

Comments

sshine•1y ago
How does Hedgehog and Hypothesis differ in their shrinking strategies?

The article uses the words "integrated" vs. "internal" shrinking.

> the raison d’être of internal shrinking: it doesn’t matter that we cannot shrink the two generators independently, because we are not shrinking generators! Instead, we just shrink the samples that feed into those generators.

Besides that it seems like falsify has many of the same features like choice of ranges and distributions.

_jackdk_•1y ago
This is the key sentence:

> The key insight of the Hypothesis library is that instead of shrinking generated values, we instead shrink the samples produced by the PRNG.

Hedgehog loses shrink information when you do a monadic bind (Gen a -> (a -> Gen b) -> Gen b). Hypothesis parses values out of the stream of data generated by the PRNG, so when it "binds", you are still just consuming off that stream of random numbers, and you can shrink the stream to shrink the generated values.

Here is a talk that applies the Hypothesis idea to test C++: https://www.youtube.com/watch?v=C6joICx1XMY . Discussion of PBT implementation approaches begins at 6:30.

thesz•1y ago
This is fascinating!

If I understand correctly, they approximate language of inputs of a function to discover minimal (in some sense, like "shortest description length") inputs that violate relations between inputs and outputs of a function under scrutiny.

evertedsphere•1y ago

    newtype Parser a = Parser ([Word] -> (a, [Word])
missing a paren here
moomin•1y ago
I’m honestly completely failing to understand the basic idea here. What does this look like for generating and shrinking random strings,
chriswarbo•1y ago
One straightforward approach would be:

- Generate a random number N for the size (maybe restricted to some Range)

- Generate N `Char` values, by using a random number for each code point.

- Combine those Chars into a string

falsify runs a generator by applying it to an infinite binary tree, with random numbers in the nodes. A generator can either consume a single number (taken from the root node of a tree), or it can run two other generators (one gets run on the left child, the other gets run on the right). Hence the above generator would use the value in the left child as N, then run the "generate N Chars" generator on the right child. The latter generator would run a Char generator on its left child, and an 'N-1 Chars' generator on its right child; and so on.

To shrink, we just run the generator on a tree with smaller numbers. In this case, a smaller number in the left child will cause fewer Chars to be generated; and smaller numbers in the right tree will cause lower code-points to be generated. falsify's tree representation also has a special case for the smallest tree (which returns 0 for its root, and itself for each child).

mjw1007•1y ago
I've found in practice that shrinking to get the "smallest amount of detail" is often unhelpful.

Suppose I have a function which takes four string parameters, and I have a bug which means it crashes if the third is empty.

I'd rather see this in the failure report:

("ldiuhuh!skdfh", "nd#lkgjdflkgdfg", "", "dc9ofugdl ifugidlugfoidufog")

than this:

("", "", "", "")

gwern•1y ago
Really? Your examples seem the opposite. I am left immediately thinking, "hm, is it failing on a '!', some sort of shell issue? Or is it truncating the string on '#', maybe? Or wait, there's a space in the third one, that looks pretty dangerous, as well as noticeably longer so there could be a length issue..." As opposed to the shrunk version where I immediately think, "uh oh: one of them is not handling an empty input correctly." Also, way easier to read, copy-paste, and type.
dullcrisp•1y ago
Their point is that in the unshrunk example the “special” value stands out.

I guess if we were even more clever we could get to something more like (…, …, "", …).

gwern•1y ago
The special value doesn't stand out, though. All three examples I gave were what I thought skimming his comment before my brain caught up to his caveat about an empty third argument. The empty string looked like it was by far the most harmless part... Whereas if they are all empty strings, then by definition the empty string stands out as the most suspicious possible part.
tybug•1y ago
shae•1y ago
I care about the edge between "this value fails, one value over succeeds". I wish shrinking were fast enough to tell me if there are multiple edges between those values.
The Hypothesis explain phase [1][2] does this!

  fails_on_empty_third_arg(
      a = "",  # or any other generated value
      b = "",  # or any other generated value
      c = "",  
      d = "",  # or any other generated value
  )
[1] https://hypothesis.readthedocs.io/en/latest/reference/api.ht...

[2] https://github.com/HypothesisWorks/hypothesis/pull/3555

chriswarbo•1y ago
> As opposed to the shrunk version where I immediately think, "uh oh: one of them is not handling an empty input correctly."

I agree that non-empty strings are worse, but unfortunately `("", "", "", "")` wouldn't only make me think of empty strings; e.g. I'd wonder whether duplicate/equal values are the problem.

chriswarbo•1y ago
> I'd rather see this in the failure report:

> ("ldiuhuh!skdfh", "nd#lkgjdflkgdfg", "", "dc9ofugdl ifugidlugfoidufog")

I would prefer LazySmallcheck's result, which would be the following:

    (_, _, "", _)
Where `_` indicates that part of the input wasn't evaluated.
yorwba•1y ago
A minimal reproducing example cannot guarantee that you'll correctly diagnose a bug just by looking at the example (because multiple potential bugs could cause the same example to fail) but it can guarantee that when you step through the code to understand what's happening, you won't have to deal with huge amounts of irrelevant data.

Maybe an alternative shrinking procedure could directly minimize the number of instructions that need to be executed to hit a failure...

edsko•1y ago
(Author of falsify here.) You are absolutely correct that the empty string isn't always the best counter-example. The goal of shrinking is to shrink to the _simplest_ possible value (this is true for all approaches to shrinking). What constitutes "simple" is very much domain specific. It would certainly be possible to write a generator that would shrink to, say, "foo", as the canonical "simplest" example of a simple string. Indeed, since we are working in a lazy language, you could (with a bit of effort) shrink to `undefined` if the other arguments are not used at all.
mjw1007•1y ago
I agree it can be domain-specific, but I think it's more common than not that empty containers, and the number zero, are corner cases rather than typical values.

So I think it would be a decent quality-of-life improvement to make generators of the sort you suggest easily available, and have the tutorial docs use them from the start.

A 3rd World Embedded Engineer Responds to "RISC-V They Should Have Known Better"

https://rvembedded.com/blog_post/12/
218•Narishma•4h ago•117 comments

Claude: System Prompts

https://platform.claude.com/docs/en/release-notes/system-prompts
434•tosh•8h ago•190 comments

The Case Against Formal Verification, 50 Years Later

https://ivan-gavran.github.io/0-social-processes-paper
13•ghuntley•26m ago•4 comments

Protobuf has LSP support. You're welcome

https://buf.build/blog/protobuf-lsp
47•theanonymousone•2h ago•18 comments

SIMD in the 90s: Programming Intel's Pentium MMX

https://pikuma.com/blog/programming-intel-pentium-mmx-simd
16•ibobev•3d ago•1 comments

Models Are Getting Dumber on Purpose

https://w4g1.dev/blog/models-are-getting-dumber-on-purpose
142•hruvhwe•2h ago•89 comments

The AI Credit Resale Economy

https://vectoral.com/blog/who-are-the-token-brokers
187•mlenhard•6h ago•69 comments

MathCode, Mathematical Coding Agent

https://math-ai-org.github.io/mathcode/
30•homarp•2h ago•9 comments

St Lucie Nuclear Reactor Unit 1 manually shutdown, 3 control rods drop into core

https://www.wptv.com/news/treasure-coast/region-st-lucie-county/saint-lucie-nuclear-power-plant-u...
125•toomuchtodo•5h ago•89 comments

Low-Tech Ceramic Water Filter

https://wiki.lowtechlab.org/wiki/Filtre_%C3%A0_eau_c%C3%A9ramique/en
20•Bluestein•5d ago•7 comments

NIH is ending a key grant for budding clinical researchers

https://www.science.org/content/article/nih-ending-key-grant-budding-clinical-researchers
109•brandonb•4h ago•53 comments

Anton Chekhov played at love most of his life

https://commonreader.wustl.edu/winning-and-losing-at-the-great-game-of-intimacy/
44•lermontov•1d ago•6 comments

Clamiga: Common Lisp for the Amiga

https://nnamgreb.de/blog/Clamiga+-+Common+Lisp+for+the+Amiga
60•emptybits•3d ago•4 comments

Firefox for iOS now has a native adblocker

https://support.mozilla.org/en-US/kb/block-ads-firefox-ios
440•pentagrama•8h ago•189 comments

Plastic mechanical computer from 1963: The Digi-Comp 1 [video]

https://www.youtube.com/watch?v=-y8bGBE71yw
26•tobr•1d ago•6 comments

Tell HN: Cloudflare silently injects its analytics when you switch nameservers

142•stagas•3h ago•38 comments

Before Rightmove, there was the Cosmorama

https://www.ianvisits.co.uk/articles/before-rightmove-there-was-the-cosmorama-londons-forgotten-p...
18•brod_ie•5d ago•2 comments

Archie G. Norcross' Maine Forest Fire Maps (1918–22)

https://publicdomainreview.org/collection/maine-forest-fire-maps/
20•samclemens•3d ago•3 comments

The weekend is 100 years old

https://www.theguardian.com/money/2026/aug/16/the-weekend-is-100-years-old-skiveday-fridays-and-h...
147•lentil_soup•5h ago•82 comments

Tasklet (YC P26) Is Hiring a Head of Design Engineering

https://tasklet.ai/careers/head-of-design-engineering
1•mayop100•7h ago

ICE Shot a Journalist and Threw Him in Detention. He's Approaching 300 Days

https://theintercept.com/2026/08/16/ricardo-parias-ice-detention-journalist-los-angeles/
42•Jimmc414•1h ago•12 comments

A True Telnet BBS on a Casio Calculator

https://ei3lh.eu/2026/08/16/a-true-telnet-bbs-on-a-casio-calculator/
71•austinallegro•8h ago•9 comments

Stripe Clinches over $7B Deal to Buy AI Firm OpenRouter

https://www.bloomberg.com/news/articles/2026-08-16/stripe-nears-deal-to-buy-ai-firm-openrouter-fo...
14•zacharyozer•34m ago•6 comments

Chestnut – eGPU dock with open-source firmware

https://hwbusters.com/news/comma-ai-egpu-dock-runs-open-source-firmware-249-bare-799-with-an-rx-9...
121•txrx0000•2d ago•35 comments

A SAT Attack on Tarski's High School Algebra Problem

https://arxiv.org/abs/2608.08421
72•matt_d•4d ago•28 comments

Asus Bike Booster

https://www.asus.com/accessories/bike-booster/asus-oxiis/oxiis-intelligent-bike-booster/
585•wiradikusuma•4d ago•410 comments

The deep history behind the Road to Nowhere inside the Great Smoky Mountains

https://www.wunc.org/environment/2026-08-10/road-to-nowhere-great-smoky-mountains
15•yareally•3d ago•5 comments

Research papers using "kidney disappointment" instead of "kidney failure"

https://scholar.google.com/scholar?q=%22kidney+disappointment%22
318•Alifatisk•8h ago•115 comments

The Trumps' Crypto Project Just Got One Step Closer to Becoming a Bank

https://www.motherjones.com/politics/2026/08/donald-trump-world-liberty-regulatory-approval/
20•rib3ye•1h ago•15 comments

GPS and the Lost Art of Getting Lost

https://www.newyorker.com/news/annals-of-inquiry/gps-and-the-lost-art-of-getting-lost
61•karakoram•6h ago•51 comments