# Calculator off... is yours too?

Discussion in 'Mac Apps and Mac App Store' started by LeeTom, Apr 22, 2005.

1. ### LeeTom macrumors 68000

Joined:
May 31, 2004
#1
Okay, Calculator says that
9,077.38 - 8,980.75 = 96.629999999999

Huh?????
Can anyone duplicate this?

Joined:
Jan 6, 2004
3. ### Mitthrawnuruodo Moderator emeritus

Joined:
Mar 10, 2004
Location:
Bergen, Norway
#3
I get that to (using Calculator 3.2.1 under 10.3.9)... I thought they fixed that float inaccuarcy in Calculator a while ago...

...good thing I soon get Tiger (hopefully) with a new Calc in Dashboard...

4. ### tobefirst macrumors 68040

Joined:
Jan 24, 2005
Location:
St. Louis, MO
#4
if i recall correctly from high school...

If we assume that the 9 is repeating (and Calculator just didn't have enough spots...as opposed to the number ending after the tenth nine) then 96.629999999999 is actually the same as 96.63, since there are no numbers in between. Assuming that the 9 is repeating, this is actually a correct answer.

Again, that's if I remember correctly from a class I took 6 years ago.

5. ### LeeTom thread starter macrumors 68000

Joined:
May 31, 2004
#5
96.629999 repeating is not equivalent to 96.63. Time to take that class again!

6. ### Mechcozmo macrumors 603

Joined:
Jul 17, 2004
#6
Run Windows Update, it does the same thing. Kinda. I'll look for some screenshots... anyways....

7. ### Thom_Edwards macrumors regular

Joined:
Apr 11, 2003
#7
i have to concur. his original problem is subtraction with 2 decimal places. you don't get repeating decimal places in addition/subtraction EVER. repeating decimals only happen in division (if you constrain this topic to basic math like +, -, /, and *)

does 1.5 - 1.4 = .1111111111111111111111111111... ? no, it doesn't

8. ### Mitthrawnuruodo Moderator emeritus

Joined:
Mar 10, 2004
Location:
Bergen, Norway
#8

9. ### camomac macrumors 6502a

Joined:
Jan 26, 2005
Location:
Left Coast
#9
okay... interesting. i'm at work right now (on a windows xp machine) and sure enought it doesn't have the same problem. it says 96.63.

i completely agree with Thom_Edwards post. this is strange??

10. ### tobefirst macrumors 68040

Joined:
Jan 24, 2005
Location:
St. Louis, MO
#10
Like I said, it's been awhile, but I'm still thinking I'm right. You are right in saying that 1.5 - 1.4 does not equal .11111... because there are numbers between .1 (the logical answer) and the repeating answer: .11111....

What I'm saying is that I was taught that .1 is the same as .0999999....

What number comes between .1 and .0999999....? There isn't one. Therefore, as I was taught, they are the same number.

And I know this sounds like total BS. And that is what I thought when I heard it, too...but after thinking about it, I accepted it, and thought it was pretty cool. I'll see if I can find it online somewhere.

11. ### tobefirst macrumors 68040

Joined:
Jan 24, 2005
Location:
St. Louis, MO
#11
12. ### Loge macrumors 68020

Joined:
Jun 24, 2004
Location:
England
#12
You are correct. As college analysis courses will teach, every real number has a unique non-terminating decimal expansion. As 1/3 can be written 0.333333... , so can 1 be written 0.9999999999...

13. ### khammack macrumors regular

Joined:
Sep 28, 2004
Location:
Portland, OR
#13

That's not entirely accurate. Do keep in mind that the computer has no concept of "decimal" places. All arithmetic is done in binary.

For reference, I tried the same calculation on linux (the system I happen to be in front of) using python and scheme:

Code:
```Python 2.4.1 (#2, Mar 30 2005, 21:51:10)
[GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 9077.38 - 8980.75
96.6299999999992

guile> (- 9077.38 8980.75)
96.6299999999992
```
To understand this you need to convert the numbers into binary (perhaps even ieee floating point notation) and then do the arithmetic in binary. I think what you'll probably find is that at least one of the fractional values .38 or .75 are infinitely repeating patterns when expressed as a binary number.

(I'd do the conversion for you, but I'm at work; if nobody has posted the solution later I'll check back and do it).

-kev

14. ### sonictruth macrumors member

Joined:
Nov 18, 2004
Location:
Pennsylvania
#14
Happens in Quicken too

I've had the same issue with Quicken 2004's portfolio... for instance, a cost basis indicated as \$2,999.9999 (reality is a purchase of \$3K), and other columns calculated based on the 'wrong' value. So, since I am assuming Quicken does not use the OS X calculator, is this a difference in floating point calcs on the processor???

15. ### Mitthrawnuruodo Moderator emeritus

Joined:
Mar 10, 2004
Location:
Bergen, Norway
#15
The problem is a small error in how Calc is handeling some Floats... and it should have been corrected...
Just because there's a way to prove (mathematicly) that 1 = .999999... that has NO relevance here, whatsoever....

16. ### tobefirst macrumors 68040

Joined:
Jan 24, 2005
Location:
St. Louis, MO
#16
I agree that it's trivial, but I wouldn't say that it has no relevance. Haha...the title of the thread was Calculator off... is yours too?, but it's not "off" by definition.

And after all, isn't Calculator an application to "solve" mathematical formulas?

Of course, I'm just playing around and don't mean to offend anyone.

17. ### Gizmotoy macrumors 65816

Joined:
Nov 6, 2003
#17
Yea, mine does the same thing... and by buddy's version in Tiger does it too. Oddly his dashboard calculator gives the right answer.

18. ### Thom_Edwards macrumors regular

Joined:
Apr 11, 2003
#18

i quite understand what you are saying (and the others that want to say that Calculator is "correct"), but this isn't a class in how computers work and theoretical mathematics.

if i were writing a program that was teaching kids how to subtract decimal numbers and i gave them that crazy answer, it would be reported as a bug from EVERY user. i guarantee it.

also, i appreciate your condescending attitude on my understanding of binary and floating points, but just because you can recreate it on linux doesn't make it *right*.

19. ### eva01 macrumors 601

Joined:
Feb 22, 2005
Location:
Gah! Plymouth
#19
well to try and put it to rest i believe it ONLY applies to .9999 repeating nothing else with a .9 repeating like .099 repeating, i believe the calculator is just off

20. ### Tortellino macrumors newbie

Joined:
Apr 22, 2005
#20
How have you set the precision in your calculator? Since you supplied numbers with only 2 decimal digits, you need to round off your answer to two decimals as well. Once you do it you will get the answer you expect. I admit it would be thoughtful to detect these things automatically, but if you Calculator view precision is set to 16 digits, and you only supply 6, then the computer will display the closest real number with 16 digits to the binary result it obtained.

That being said, apart from setting appropriate precision for the Calculator display, there are programs like Mathematica that can work with infinite precision by performing operations on non-binary representation of numbers.

I did not find that explanation condescending, but rather thorough, tested on an independent platform. If it hurt your feelings then you probably should work on turning your sensitivity down a notch.

21. ### realityisterror macrumors 65816

Joined:
Aug 30, 2003
Location:
Snellville, GA
#21
uhh...

2/3 - 1/3 ???
or maybe you consider that division as it is 2/3...

reality

22. ### eva01 macrumors 601

Joined:
Feb 22, 2005
Location:
Gah! Plymouth
#22
1/3 and 2/3 are not two decimal spots they are repeating

23. ### clayj macrumors 604

Joined:
Jan 14, 2005
Location:
visiting from downstream
#23
This occurs in a lot of different programs, including Excel, when you do math that involves decimals. Since all math in a computer is ultimately performed in binary, decimals that don't translate well to binary (for example, 0.1 cannot be perfectly converted to binary and back again) sometimes are mishandled.

Ideally, the program should know to correct this, but sometimes, you have to force it to give you the correct answer. For example, in Excel 2003 for Windows, the formula "=1-0.9-0.09-0.01" should return 0, but it actually returns -1.9082E-17 (or -0.0000000000000000191). Changing the formula to "=ROUND(1-0.9-0.09-0.01,8)" (which rounds the result to 8 decimal places, more than we need considering our numbers have no more than 2 decimal places each) gives the correct result of 0.

24. ### eva01 macrumors 601

Joined:
Feb 22, 2005
Location:
Gah! Plymouth
#24
exact same thing happens when finding the root on a TI-83 and TI-86 Graphing calculator (well not that problem, but with finding root in graphs), it is because of the algorithm the calculator uses (not sure if it occurs with the TI-90 haven't used it)

25. ### telecomm macrumors 65816

Joined:
Nov 30, 2003
Location:
Rome
#25
This topic has been discussed about a million times (well, that might be off, I'm using Calculator)

I'm too lazy to do a thread search (looks like I'm not the only one), or I would provide links.