Certainly a univarient model in the type system could be useful, but it would be extra powerful (and more correct) if it could handle covariance.
Otherwise, it feels to me that it'd be consistently wrong to model the variables as independent. And any program of notable size is gonna be far too big to consider correlations between all the variables.
As for how one might do the learning, I don't know yet!
It's not part of the type system, it's just the giry monad as a library.
As presented in the article, it is indeed just a library.
Bayes is mentioned on page 46.
> And why does it need to be part of the type system? It could be just a library.
It is a library that defines a type.
It is not a new type system, or an extension to any particularly complicated type system.
> Am I missing something?
Did you read it?
https://www.microsoft.com/en-us/research/wp-content/uploads/...
Bayes isn't mentioned in the linked article. But thanks for the links.
If you go down this road far enough you eventually end up reinventing particle filters and similar.
If you have an HD map you can solve for it using building shapes or by looking at the street with cameras. WiFi seems like it would help, but the locations of the WiFi terminals are themselves based on crowdsourced GPS.
... until you use a ferry.
Sounds like a classic case of programmers ignoring corner cases: Towing, ferries, car trains, pushing the car because it broke down...
It's when you find messages in the log like "this should never happen".
It appears that a similar approach is implemented in some modern Fortran libraries.
Or specify they're paying X per day, but want hourly itemized billing...but it should definitely come out to X per day (this was one employer which meant I invoiced them with like 8 digits of precision due to how it divided, and they refused to accept a line item for mathematical uncertainty aggregates).
I wonder what it'd look like to propagate this kind of uncertainty around. You might want to check the user's input against a representative distribution to see if it's unusual and, depending on the cost of an error vs the friction of asking, double-check the input.
Floats try to keep the relative error at bay, so their absolute precision varies greatly. You need to sum them starting with the smallest magnitude, and do many other subtle tricks, to limit rounding errors.
That's one way to look at it.
Another is that Money is certain only at the point of exchange.
> It appears that a similar approach is implemented in some modern Fortran libraries.
I'd be curious about that. Do you have a link?
eg. 10cm +8mm/-3mm
for what the acceptable range is, both bigger and smaller.
id expect something like "are we there yet" referencing GPS should understand the direction of the error and what directions of uncertainty are better or worse
It's not statistical. If the machinist makes a part that's not within the +/- bounds, they throw it away and start again. If you tried to fit multiple parts, all with only statistical respect for tolerances, you would run into trouble almost 100% of the time with just a few pieces.
Probability distributions (even very simple ones) provide a much clearer view across any domain where there’s inherent uncertainty.
It usually takes some "finesse" to get your data / measurements into territory where the errors are even small in the first place. So, I think it is probably better to do things like this Uncertain<T> for the kinds of long/fat/heavy tailed and oddly shaped distributions that occur in real world data { IF the expense doesn't get in your way some other way, that is, as per Senior Engineer in the article }.
It’s so cool!
I always start my introductory course on Haskell with a demo of the Monty Hall problem with the probability monad and using rationals to get the exact probability of winning using the two strategies as a fraction.
Does Anglican kind of do this?
What is really curious is why, after being reinvented so many times, it is not more mainstream. I would love to talk to people who have tried using it in production and then decided it was a bad idea (if they exist).
[1]: https://www.boost.org/doc/libs/1_89_0/libs/numeric/interval/... [2]: https://arblib.org/
A computation graph which gets sampled like here is much slower but can be accurate. You don't need an abstract domain which loses precision at every step.
On the CPU you probably get implicit parallel execution with pipelines and re-ordering etc, and on the GPU you can set up something similar.
> Under the hood, Uncertain<T> models GPS uncertainty using a Rayleigh distribution.
And the Rayleigh distribution is clearly not just an interval with a uniformly random distribution in between. Normal interval arithmetic isn't useful because that uniform random distribution isn't at all a good model for the real world.
Take for example that Boost library you linked. Ask it to compute (-2,2)*(-2,2). It will give (-4,4). A more sensible result might be something like (-2.35, 2.35). The -4 lower bound is only attainable when you have -2 and 2 as the multiplicands which are at the extremes of the interval; probabilistically if we assume these are independent random variables then two of them achieving this extreme value simultaneously should have an even lower probability.
In noisy sensors case there's some arbitrary low probability of them being actually super wrong, if you go by true 10^-10 outlier bounds they will be useless for any practical use, while the 99% confidence range is a relatively small rent.
More often you want some other distribution and say (-2, 2) and those are the 90th percentile interval not the absolute bounds, 0 is more likely than -2 and -3 is possible but rare. It's not bounds, you can ask you model for your 99th or 99.9th percentile value or whatever tolerance you want and get something outside of (-2,2).
Doesn't addition as well? Like if you roll d6+d6, the output range is 2-12, but it's not nearly the same as if you rolled d11+1.
But I'm not so sure in your conclusion: a good enough model could be universally useful. See how everyone uses IEEE 754 floats, despite them giving effectively one very specific model of uncertainty. Most of the time this just works, and sometimes people have to explicitly work around floats' weirdnesses (whether that's because they carefully planned ahead because they know what they are doing, or whether they got a nasty surprise first). But overall they are still useful enough to be used almost universally.
If you make some assumptions about your error (a popular one is to assume Gaussian distribution) then you can calculate the error of the result quite elegantly.
It's a nice excercise to write some custom C++ types that have (value, error) and automatically propagate them as you perform mathematical operations on them.
Unfortunately, in the real world only very few measurements have a gaussian error distribution, and the problem are systematic (non-random) errors, and reasoning about them is a lot harder.
So this means that automatically handling error propagation is in most cases pointless, since you need to manually analyze the situation anyway.
Aside from the original research paper needing to be included in the repo, it definitely does not need anything more than what's already there. It all builds and compiles without errors, only 2 warnings for the library proper and 6 warnings for the test project. Oh and it comes with a unit testing project: 59 tests written that covers about 73% of the library code. Only 2 tests failed.
Even having a unit testing library means it beats out like 50% of all repos you see on GitHub.
When you see y = m * x + b, your recollections of math class may note that you can easily solve for "m" or find a regression for "m" and "b" given various data points. But from a programming perspective, if these are all literal values, all this is is a "render" function. How can you reverse an arbitrary render function?
There are various approaches, depending on how Bayesian you want to be, but they boil down to: if your language supports redefining operators based on the types of the variables, and you have your variables contain a full specification of the subgraphs of computations that lead to them... you can create systems that can simultaneously do "forward passes" by rendering the relationships, and "backward passes" where the system can automatically calculate a gradient/derivative and thus allow a training system to "nudge" the likeliest values of variables in the right direction. By sampling these outputs, in a mathematically sound way, you get the weights that form a model.
Every layer in a deep neural network is specified in this way. Because of the composability of these operations, systems like PyTorch can compile incredibly optimal instructions for any combination of layers you can think of, just by specifying the forward-pass relationships.
So Uncertain<T> is just the tip of the iceberg. I'd recommend that everyone experiment with the idea that a numeric variable might be defined by metadata about its potential values at any given time, and that you can manipulate that metadata as easily as adding `a + b` in your favorite programming language.
Are there PLs that support this kind of thing at the language level as you are describing?
https://news.ycombinator.com/item?id=28941145 has some discussion here as well, though it’s a few years old.
Pyro and NumPyro seem to be popular at the moment!
If I have a library, though, that lets me add and multiply not just floats but entire computation subgraphs with the same exact + and * operators, though, I can have the library reverse that function automatically, and say: “optimize the parameters to minimize the difference between the curve and the data points.”
LLMs and other ML systems, to paint with a very broad stroke, solve that problem with billions of parameters in a million-dimensional space. Developing intuition for those high dimensions is hard! But the code is simple because once you’ve done the math for the forward pass, you can go straight from chalkboard to Python code, and the libraries largely assist with reversing and building a GPU-accelerated training process automatically!
The fact that one can play with complicated nested probability distributions that unify these concepts, as one would play with dolls in a dollhouse, is the point!
With the eventual goal of running various simulations over different randomly generated outcomes based on those probability distributions.
A spin-off, Signaloid, is taking this technology to market. I'm also researching using this in state estimation (e.g., particle filters).
Another place where I think this would be neat would be in CAD. Imagine if you are trying to create a model of an existing workpiece or of a room, and your measurements don't exactly add up. It's really frustrating and you have to go back and measure again, and you usually end up idealizing the model and putting in rounder numbers to make it fit, but it is less true to reality. It would be cool if you could put in uncertainties for all lengths and angles, and it would run a solver to minimize the total error.
mackross•5mo ago