Working with numbers in javascript is there a easier way i missed

1

Hi all after working on my calculator project one area that i didnt expect to find problems with was the actual representation of the numbers after the calculation … turns out i got a lot of unexpected outcomes … others i did expect eg 7 divided by 3 would give 2.3333333333333335 so i googled how to sort this … first i got toPrecision() method then toFixed()

Using toFixed(2) i could output 2.33

But that didnt sort all problems if i had 0.0001 + 0.0001 i wanted 0.0002 but my output would be 0.00

so i set about creating a function to sort these problems out

then discovered more lol

so i have a function now created and it dose what i want it to do … but i would like some input on whether i missed an obvious easier solution.

so if you could look at the repl sheet i did and your solution solves all my test cases i would love to know
Here is my repl … if you want to try other number combinations you think i missed open up another repl and copy and paste mine into it

2

Hey, you can use this:

function simpleRounding(x){
  var acc = 1000000000000;
  return Math.round(x*acc)/acc;
}

This should remove all the rounding errors. It still returns 2.333333333333 (which is correct).

1 Like
3

Dam was just about to change to yours as i could live with the 7/3 … i would have to take a couple of 0s of as my display is for 8 digits only

but i decided to test a couple of numbers with .7 /3 to be safe eg 777.7 / 3 … 2592.3566666666666
way to big for my calc screen … where as my bloated function returns 2592.23

Thanks for that though … numbers are strange in programming lol

updated my repl with your contribution so its easier to see

4

Found a solution to that:

function simpleRounding(x){
    var max = 12;
    var acc = Math.pow(10,(max - Math.round(x).toString().length));
    return Math.round(x*acc)/acc;
}

This will always return a number with length 12 (max) or less;

5

nice nearly there added a change to yours

    var max = 12;
    var acc = Math.pow(10,(max - Math.round(x).toString().length));
    var ans =Math.round(x*acc)/acc;
    return ans.toPrecision(8);
	 	}```
using toPrecision(8) makes sure i dont go over .... it dosent seem to recognise decmial points
so i get 8 numbers and a decimal in the middle but i can fit that
only one now i discovered is 0.0001*0.0001 which technically should return a error on my calc which it used to display as 1E-8 now comes out as 1.0000000E-8 but that overflows big time for me .... will use updated code from you with my toPrecision(8) add on .. and create a function to shorten that dodgy result 
So thanks for that very happy with the result
6

It displays all the zeroes because they are all significant. Wouldn’t setting max to 8 work?

7

@BenGitter unfortunately need max 8 to work so i changed yours again lol as i found another test case which mine failed on but yours with a change worked … so updated my repl if you care to look

will leave this for now … and will see can i find more test cases rather than chopping and changing … will also look at a couple of calculators and see how they handle things (a bit obsessive i am i think lol)
thanks for the help and input

8

So wouldn’t it work to use var max = 8? I think that should work (at least for numbers below 100000000 (8 zeros))

9

@BenGitter dam getting tired now writing wrong … need max 9 i should have said … but with max 9 one test case had me overflowing on display.
then i found 0.001+0.0006 … looks a obvious =0.0016 but no gives 0.0015999999999
so i adjusted your code and got 0.0016 and other one that overflowed is sorted so for the time being im happy …
will leave it for now and see can i find more crazy numbers before i start changing things again

1 Like