In my own experience with data science work:

On multiple occasions I’ve seen noticeable accumulated rounding error crop up by the end of a cumprod over very long timeseries of floats.

But in my cases it has just been a minor nuisance that I have to double check whether a mismatch came from float-related quirkiness or a code mistake.

I got into the habit of just using integers where applicable for cases that need to match down to the last bit, and numpy.isclose() for inherently floaty cases where it’s anticipated and reasonable.

Only speaking from my own experience, which doesn’t involve chaotic systems or anything like that. I’m curious to hear from people in other domains who may have more exciting stories.

For physics simulations, or even in RF work, the tolerances you need are quite a bit more than say, AI work. For example, for one of our RF algorithms, we "have" to use doubles (I actually can think of an optimization that can let us use floats, but it's quite involved to fix it) to get the required precision. That precision requirement is 10^-15. For physics simulations, I've heard of accuracies of up to 10^-20.

I will say, generally, it depends on what you mean. There's like, numerical drift, and then there's converting to the discrete part. Generally, I'd say that "getting into the discrete part" of an algorithm, unless you're trying to match behavior of another piece of software, is fine. Like, if you're playing a game, people are not going to care if the shot just grazed them and missed, but the "Actual True Value" is like 10^-9 off or something. They will just chalk it up to "close, unlucky". If you have one canonical place for your floating point errors from which your "discrete" parts derive, it's no biggie.

But at work, I recently debugged an algorithm in which the author, effectively wanted to compute the integer and fractional parts of a double type. Their reasoning was that, for the fractional part, there can be imprecision, so you should do some fancy math to recover the fractional part.

This is giga wrong for the application we were building. You never want to try computing the "same double" in two different ways and compose the results like this. Intuitively, if one path "thinks" your double is 15.00001, and the other path "thinks" its 14.99999, you're now saying that the double is 15 (integer) + .99999 (fractional) = 15.99999: this is way off now!

You should just use one double, and extract out the integer and fractional parts. It doesn't matter if the double is imprecise - it's authoritatively imprecise. Much easier to reason about.

This is the first time I've debugged an edge case like this since I wouldn't implement something like this, but it's an example of how subtle floating point errors can be.