Hi, everyone!
I have a question on numbers in JS. JS is using IEEE 754 format to represent and operate with numbers. This means that representation of almost all real numbers is approximate. For example, there is no way to represent 0.3 in binary format (radix 2) with finite number of digits after the point. Thus, the result of 0.3.toFixed(20) is 0.29999999999999998890. But then, the result of 3/10 is 0.3. It seems that JS is able to calculate the exact result in this case. But my feeling is that JS shows a rounded value in this case, as the result of +“0.29999999999999998890” (i.e. Number(0.29999999999999998890)) is 0.3.
So can anybody explain the exact rules that JS applies when rounding numbers?
2 Likes
TL;DR:
JS shows a rounded number according to a rounding error that it expects. Since performing arithmetic operations on floats adds up to the rounding error, it gets higher than expected, and as a result the number is printed as a long inaccurate decimal number.
Long Story:
Hi!
First of all this is one of the greatest questions I have seen in this forum and it actually made me look a few things up and learn some new things.
Good thing is that you already know about IEEE 754 which is used by JS and many other languages to represent floating point numbers. but this standard contains more than just a way to store floating point numbers. Since these rounding errors are quite common there is an error measuring metric called ulp (units in last place) to define the precision of the floating point and show the expected value, that’s why JS shows (3 / 10) as 0.3 although it’s not what it has stored. Since these rounding errors tend to build up each time you do an arithmetic operation on floating point numbers, IEEE has also introduced algorithms for each operation (addition, subtraction, …) to reduce the rounding errors.
But all of that is still not enough. There is no guarantee when it comes to performing binary operations on floating point numbers and as a result you should be careful and try to avoid floats when accuracy matters.
If you want to know the exact math and more information on this topic take a look at this old but useful article by David Goldberg which is recommended by IEEE: What Every Computer Scientist Should Know About Floating-Point Arithmetic
There is also a very useful article on “Bigger On The Inside” blog: What Every JavaScript Developer Should Know About Floating Point Numbers
3 Likes
@tuxitop, thank you very much for your reply. I have read the provided materials and here is my conclusion. JS is storing value of 0.3 in exactly rounded binary form. And the concept of keeping rounding errors within 0.5 ulp does not imply showing the rounded value of 0.3 in binary format as 0.3 in decimal format. So my idea is that JS is applying some extra rules of its own(on top of IEEE) in order to show rounded decimal values. Would like to hear your thoughts on this. Your materials were very helpful. Thank you.