The AMM article is fine and the video embedded in the article is pretty good too.

Exactly! You got it 100% right.

> A method for constructing an infinite sequence of approximations to the exact solution.

They just rename it as a "exact" method and rename the old "radical" as inexact. In math definitions change from time to time, but in any case they didn't solve the old problem.

Floating numbers with a fixed precision based on a fixed base, are really limited on what rational numbers can be represented. For example, the standard floating point formats cannot exactly represent the fraction 1/3. One would expect that 3 times 1/3 would result in 1, but it does not. This is based on an implementation decision on how to represent rational numbers. There are computer algebraic systems that represent fractions like 1/3 exactly, such that 1 times 1/3 is equal to 1. In a similar way it is possible to represent certain real numbers like the square root of 2, such that the square of such a number is exactly equal to 2. To being able to print such a number in a certain base (10 for example) does not mean it is not possible to represent such numbers in a way that it is possible to perform calculations with these.

One should realize that the implementation choice to represent floating numbers in the way it is done in processors is related to performance and memory usage.