https://github.com/filiph/unsure/blob/master/lib/src/calcula...
I assume this is a montecarlo approach? (Not to start a flamewar, at least for us data scientists :) ).
> The computation is quite slow. In order to stay as flexible as possible, I'm using the Monte Carlo method. Which means the calculator is running about 250K AST-based computations for every calculation you put forth.
So therefore I conclude Monte Carlo is being used.
import numpy as np
def monte_carlo(formula, iterations=100000):
res = [formula() for _ in range(iterations)]
return np.percentile(res, [0, 2.5, \*range(10, 100, 10),
97.5, 100])
def uncertain_division():
return 100 / np.random.uniform(2, 4)
monte_carlo(uncertain_division, iterations=100000)Someone also turned it into the https://github.com/rethinkpriorities/squigglepy python library
Thanks for the link!
Want to use it every 3 months or so to pretend that we know what we can squeeze in the roadmap for the quarter.
> 0.4.0
> BRAKING: x~y (read: range from x to y) now means "flat distribution from x to y". Every value between x and y is as likely to be emitted.
> For normal distribution, you can now use x+-d, which puts the mean at x, and the 95% (2 sigma) bounds at distance d from x.
https://github.com/filiph/unsure/blob/master/CHANGELOG.md#04...
The reason I'm asking: unsure also has a CLI version (which is leaps and bounds faster and in some ways easier to use) but I rarely find myself using it. (Nowadays, I use https://filiph.github.io/napkin/, anyway, but it's still a web app rather than a CLI tool.)
>there is no way of knowing why the range is between 4 and 6
??? There is. It is the ~ symbol.
It would especially outperform the Monte-Carlo approach drastically.
Although fuzzy-number can be used to model many different kinds of uncertainties.
If you want to do arithmetic and not a black box approach you just have to realize that you can perform them on the alpha-cuts with ordinary interval arithmetic. Then you can evaluate arbitrary expressions involving fuzzy numbers, keeping the strengths and weaknesses of interval arithmetic.
The sampling based approach is very similar to Monte-Carlo, but you sample at certain well defined points.
> Range is always a normal distribution, with the lower number being two standard deviations below the mean, and the upper number two standard deviations above. Nothing fancier is possible, in terms of input probability distributions.
Consider https://f-droid.org/en/packages/com.nunosempere.distribution...
Note: If you're curious why there is a negative number (-5) in the histogram, that's just an inevitable downside of the simplicity of the Unsure Calculator. Without further knowledge, the calculator cannot know that a negative number is impossible
But if I just ask the calculator "1~2" (i.e. just a range without any operators), the histogram shows what looks like a normal distribution centered around 1.5[2].
Shouldn't the histogram be flat if the distribution is uniform?
[1] https://github.com/filiph/unsure/blob/123712482b7053974cbef9...
> Range is always a normal distribution, with the lower number being two standard deviations below the mean, and the upper number two standard deviations above. Nothing fancier is possible, in terms of input probability distributions.
The HN mods gave me an opportunity to resubmit the link, so I did. If I had more time, I'd have also upgraded the tool to the latest version and fix the wording. But unfortunately, I didn't find the time to do this.
Apologies for the confusion!
I really don't known how good it is.
If you're treating -1~1 as a normal distribution, then it's centered on 0. If you're working out the answer using a Monte Carlo simulation, then you're going to be testing out different values from that distribution, right? And aren't you going to be more likely to test values closer to 0? So surely the most likely outputs should be far from 0, right?
When I look at the histogram it creates, it varies by run, but the most common output seems generally closest to zero (and sometimes is exactly zero). Wouldn't that mean that it's most frequently picking values closest to -1 or 1 denoninator?
For normal it is higher but maybe not much more so.
Now I'm realizing that this doesn't actually work, and even in more typical calculations the input values that produce the central 95% of the output are not necessarily drawn from the 95% CIs of the inputs. Which is fine and makes sense, but this example makes it very obvious how arbitrary it is to just drop the lowermost and uppermost 2.5%s rather than choosing any other 95/5 partition of the probability mass.
- for command line, fermi: https://git.nunosempere.com/NunoSempere/fermi
- for android, a distribution calculator: https://f-droid.org/en/packages/com.nunosempere.distribution...
People might also be interested in https://www.squiggle-language.com/, which is a more complex version (or possibly <https://git.nunosempere.com/personal/squiggle.c>, which is a faster but much more verbose version in C)
```
5M 12M # number of people living in Chicago
beta 1 200 # fraction of people that have a piano
30 180 # minutes it takes to tune a piano, including travel time
/ 48 52 # weeks a year that piano tuners work for
/ 5 6 # days a week in which piano tuners work
/ 6 8 # hours a day in which piano tuners work
/ 60 # minutes to an hour
```
multiplication is implied as the default operation, fits are lognormal.
900K 1.5M # tonnes of rice per year NK gets from Russia
* 1K # kg in a tone
* 1.2K 1.4K # calories per kg of rice
/ 1.9K 2.5K # daily caloric intake
/ 25M 28M # population of NK
/ 365 # years of food this buys
/ 1% # as a percentage
I actually quite like it. Really clean, easy to see all the important elements. Lovely clear legible monospace serif font.
In the grand HN tradition of being triggered by a word in the post and going off on a not-quite-but-basically-totally-tangential rant:
There’s (at least) three areas here that are footguns with these kinds of calculations:
1) 95% is usually a lot wider than people think - people take 95% as “I’m pretty sure it’s this,” whereas it’s really closer to “it’d be really surprising if it were not this” - by and large people keep their mental error bars too close.
2) probability is rarely truly uncorrelated - call this the “Mortgage Derivatives” maxim. In the family example, rent is very likely to be correlated with food costs - so, if rent is high, food costs are also likely to be high. This skews the distribution - modeling with an unweighted uniform distribution will lead to you being surprised at how improbable the actual outcome was.
3) In general normal distributions are rarer than people think - they tend to require some kind of constraining factor on the values to enforce. We see them a bunch in nature because there tends to be negative feedback loops all over the place, but once you leave the relatively tidy garden of Mother Nature for the chaos of human affairs, normal distributions get pretty abnormal.
I like this as a tool, and I like the implementation, I’ve just seen a lot of people pick up statistics for the first time and lose a finger.
I love this. I've never though of statistics like a power tool or firearm, but the analogy fits really well.
You can test yourself at https://blog.codinghorror.com/how-good-an-estimator-are-you/.
> Heaviest blue whale ever recorded
I don't think estimation errors regarding things outside of someone's area of familiarity say much.
You could ask a much "easier"" question from the same topic area and still get terrible answers: "What percentage of blue whales are blue?" Or just "Are blue whales blue?"
Estimating something often encountered but uncounted seems like a better test. Like how many cars pass in front of my house every day. I could apply arithmetic, soft logic and intuition to that. But that would be a difficult question to grade, given it has no universal answer.
The question asks for the heaviest, which I think cannot be more than three times the normal weight, and probably no less than 1.3. That lands me at 15--600 tonnes using primitive arithmetic. The calculator in OP suggests 40--320.
The real value is apparently 170, but that doesn't really matter. The process of arriving at an interval that is as wide as necessary but no wider is the point.
Estimation is a skill that can be trained. It is a generic skill that does not rely on domain knowledge beyond some common sense.
For some of these, your answer could span orders of magnitude. E.g. my answer for the heaviest blue whale would probably be 5-500 tons because I don't have a good concept of things that weigh 500 tons. The important point is that I'm right around 9 times in 10, not that I had a precise estimate.
To continue your example of 5-7 days, it would turn into an estimate of 5-700 days. So somewhere between a week or two years. And fair enough, whatever you're estimating will land somewhere in between. But how do I proceed from there with actual planning or budget?
Edit: And by the way given a large enough view, estimates like this can still be valuable, because when you add these estimates together the resulting probability distribution narrows considerably. e.g. at just 10 tasks of this size, you get a 95% CI of 245~460 per task. At 20, 225~430 per task.
Note that this is obviously reductive as there's no way an estimate of 5-700 would imply a normal distribution centred at 352.5, it would be more like a logarithmic distribution where the mean is around 10 days. And additionally, this treats each task as independent...i.e. one estimate being at the high end wouldn't mean another one would be as well.
You make up the number you wanted to hear in the first place that ostensibly works with the rest of the schedule. That’s why engineering estimates are so useless - it’s not that they’re inaccurate or unrealistic - it’s that if we insisted on giving them realistic estimates we’d get fired and replaced by someone else who is willing to appease management and just kick the can down the road a few more weeks.
Your budget should be allocated for say 80% confidence (which the tool helpfully provides behind a switch) and your stakeholders must be on board with this. It shouldn’t be too hard to do since everyone has some experience with missed engineering deadlines. (Bezos would probably say 70% or even less.)
Stakeholders really liked specifying work times as t_i ~ PERT(min, mode, max) because it mimics their thinking and handles typical real-world asymmetrical distributions.
[Background: PERT is just a re-parameterized beta distribution that's more user-friendly and intuitive https://rpubs.com/Kraj86186/985700]
This is the reason Fermi estimation works. You can test people on it, and almost universally they get more accurate with this method.
If you got less certain of the result in the example, that's probably a good thing. People are default overconfident with their estimated error bars.
You say this but yet roughly in a top level comment mentions people keep their error bars too close.
Being overconfident with error bars means placing them too close to the point estimation, i.e. the error bars are too narrow.
By contrast, precise calculations can be extremely complex but with the expectation that the answer they produce is correct. The far larger number of factors and operations involved can obscure a very significant error, either in mathematical process or in the assumptions the equation is based on, but the result may still be assumed to be right because it has been derived from a precise formula that is expected to yield good results.
So the strength of it is in keeping it simple and not trying to get too fancy, with the understanding that it's just a ballpark/sanity check. I still feel like the Drake equation in particular has too many terms for which we don't have enough sample data to produce a reasonable guess. But I think this is generally understood and it's seen as more of a thought experiment.
...if the only things you know about an uncertain value are its expectation and variance, yes.
Often you know other things. Often you don't know expectation and variance with any certainty.
Without having fully digested how the Unsure Calculator computes, it seems to me you could perhaps "weight" the ranges you pass to the calculator. Rather than a standard bell curve the Calculator could apply a more tightly focused — or perhaps skewed curve for that term.
If you think your salary will be in the range of 10 to 20, but more likely closer to 10 you could:
10<~20 (not to be confused with less-than)
or: 10!~20 (not to be confused with factorial)
or even: 10~12~20 to indicate a range of 10 to 20 ... leaning toward 12.
The problem with similar tools is that of the very high barrier to entry. This is what my project was trying to address, though imperfectly (the user still needs to understand, at the very least, the concept of probability distributions).
Any reason why we kept it 250k and now a lower number like 10k
The histogram is neat, I don't think qalc has that. On the other hand, it took 8 seconds to calculate the default (exceedingly trivial) example. Is that JavaScript, or is the server currently very busy?
I really like that more people are thinking in these terms. Reasoning about sources of variation is a capability not all people are trained in or develop, but it is increasingly important.[3]
[1]: https://git.sr.ht/~kqr/precel
[2]: https://entropicthoughts.com/precel-like-excel-for-uncertain...
It seems to break for ranges including 0 though
100 / -1~1 = -3550~3500
I think the most correct answer here is -inf~inf
It's hard for me to imagine _dividing_ by -1~1 in a real-world scenario, but let's say we divide by 0~10, which also includes zero. For example, we are dividing the income between 0 to 10 shareholders (still forced, but ok).
Clearly, it's possible to have a division by zero here, so "0 sharehodlers would each get infinity". And in fact, if you try to compute 500 / 0, or even 500~1000 / 0, it will correctly show infinity.
But if you divide by a range that merely _includes_ zero, I don't think it should give you infinity. Ask yourself this: does 95% of results of 500 / 0~10 become infinity?
https://en.wikipedia.org/wiki/Interval_arithmetic
I think arbitrary distribution choice is dangerous. You're bound to end up using lots of quantities that are integers, or positive only (for example). "Confidence" will be very difficult to interpret.
Does it support constraints on solutions? E.g. A = 3~10, B = 4 - A, B > 0
[0] https://dmos62.github.io/personal-financial-growth-simulator...
I want to ask about adjacent projects - user interface libraries that provide input elements for providing ranges and approximate values. I'm starting my search around https://www.inkandswitch.com/ and https://malleable.systems/catalog/ but I think our collective memory has seen more examples.
Not true. If there are no negative terms, the equation cannot have negative values.
> Note: If you're curious why there is a negative number (-5) in the histogram, that's just an inevitable downside of the simplicity of the Unsure Calculator. Without further knowledge, the calculator cannot know that a negative number is impossible (in other words, you can't have -5 civilizations, for example).
The input to this was "1.5~3 x 0.9~1.0 x 0.1~0.4 x 0.1~1.0 x 0.1~1.0 x 0.1~0.2 x 304~10000" - every single range was positive, so regardless of what this represents, it should be impossible to get a negative result.
I guess this is a consequence of "I am not sure about the exact number here, but I am 95% sure it's somewhere in this range" so it's actually considering values outside of the specified range. In this case, 10% either side of all the ranges is positive except the large "304~10000".
Trying with a simpler example: "1~2 x 1~2" produces "1.3~3.4" as a result, even though "1~4" seems more intuitive. I assume this is because the confidence of 1 or 4 is now only 90% if 1~2 was at 95%, but it still feels off.
I wonder if the 95% thing actually makes sense, but I'm not especially good at stats, certainly not enough to be sure how viable this kind of calculator is with a tighter range. But just personally, I'd expect "1~2" to mean "I'm obviously not 100% sure, or else I wouldn't be using this calculator, but for this experiment assume that the range is definitely within 1~2, I just don't know where exactly".
It also has support for "error forms" which is close to what the calculator in OP uses. That takes a little more sophistication than just performing operations on the lower and upper number in parallel. In particular, the given points don't represent actual endpoints on a distribution, but rather low and high probability events. Things more or less likely than those can happen, it's just rare.
> I'm not especially good at stats
It shows! All the things you complain about make perfect sense given a little more background knowledge.
They explain that the range you give as input is seen as only being 95% correct, so the calculator adds low-probability values outside of the ranges you specified.
I can see how that surprises you, but it's also a defensible design choice.
Every single range here includes positive and negative numbers. To get the correct resulting distribution you have to take into account the entire input distribution. All normal distributions have a non-zero possibility to be negative.
If you want to consider only the numbers inside the range you can look at interval arithmetic, but that does not give you a resulting distribution.
Without default confidence: 0~9
With confidence: 0%100~9%95
We are sure it is 0 or more and we are %95 certain it is 9 or less.
Would that work?
Consider maybe 1 + 1 ~ +-2 like Q factor, if you know what I mean.
That would help to filter out more probabilistic noise in using it to help reason with.
persons = 10~15 // → 10~15
budget = persons * 1~2 // → 12~27
Should it not say 10-30?
rogueptr•3d ago
thih9•3d ago
BrandoElFollito•3d ago
notfed•3d ago
Means "100 divided by some number between 4 and 6"
throwanem•3d ago
BrandoElFollito•3d ago