Calculator off... is yours too?

LeeTom

macrumors 68000
Original poster
May 31, 2004
1,528
72
Okay, Calculator says that
9,077.38 - 8,980.75 = 96.629999999999

Huh?????
Can anyone duplicate this?
 

Mitthrawnuruodo

Moderator emeritus
Mar 10, 2004
13,594
141
Bergen, Norway
I get that to (using Calculator 3.2.1 under 10.3.9)... I thought they fixed that float inaccuarcy in Calculator a while ago... :confused:

...good thing I soon get Tiger (hopefully) with a new Calc in Dashboard... :rolleyes:
 

tobefirst

macrumors 601
Jan 24, 2005
4,029
1,139
St. Louis, MO
if i recall correctly from high school...

If we assume that the 9 is repeating (and Calculator just didn't have enough spots...as opposed to the number ending after the tenth nine) then 96.629999999999 is actually the same as 96.63, since there are no numbers in between. Assuming that the 9 is repeating, this is actually a correct answer.

Again, that's if I remember correctly from a class I took 6 years ago. :)
 

LeeTom

macrumors 68000
Original poster
May 31, 2004
1,528
72
tobefirst said:
if i recall correctly from high school...

If we assume that the 9 is repeating (and Calculator just didn't have enough spots...as opposed to the number ending after the tenth nine) then 96.629999999999 is actually the same as 96.63, since there are no numbers in between. Assuming that the 9 is repeating, this is actually a correct answer.

Again, that's if I remember correctly from a class I took 6 years ago. :)
96.629999 repeating is not equivalent to 96.63. Time to take that class again!
 

Mechcozmo

macrumors 603
Jul 17, 2004
5,215
2
Run Windows Update, it does the same thing. Kinda. I'll look for some screenshots... anyways....
 

Thom_Edwards

macrumors regular
Apr 11, 2003
240
0
LeeTom said:
96.629999 repeating is not equivalent to 96.63. Time to take that class again!
i have to concur. his original problem is subtraction with 2 decimal places. you don't get repeating decimal places in addition/subtraction EVER. repeating decimals only happen in division (if you constrain this topic to basic math like +, -, /, and *)

does 1.5 - 1.4 = .1111111111111111111111111111... ? no, it doesn't
 

camomac

macrumors 6502a
Jan 26, 2005
765
170
Left Coast
okay... interesting. i'm at work right now (on a windows xp machine) and sure enought it doesn't have the same problem. it says 96.63.

i completely agree with Thom_Edwards post. this is strange??
 

tobefirst

macrumors 601
Jan 24, 2005
4,029
1,139
St. Louis, MO
Thom_Edwards said:
i have to concur. his original problem is subtraction with 2 decimal places. you don't get repeating decimal places in addition/subtraction EVER. repeating decimals only happen in division (if you constrain this topic to basic math like +, -, /, and *)

does 1.5 - 1.4 = .1111111111111111111111111111... ? no, it doesn't
Like I said, it's been awhile, but I'm still thinking I'm right. You are right in saying that 1.5 - 1.4 does not equal .11111... because there are numbers between .1 (the logical answer) and the repeating answer: .11111....

What I'm saying is that I was taught that .1 is the same as .0999999....

What number comes between .1 and .0999999....? There isn't one. Therefore, as I was taught, they are the same number.

And I know this sounds like total BS. And that is what I thought when I heard it, too...but after thinking about it, I accepted it, and thought it was pretty cool. I'll see if I can find it online somewhere.
 

Loge

macrumors 68030
Jun 24, 2004
2,678
1,147
England
tobefirst said:
Like I said, it's been awhile, but I'm still thinking I'm right. You are right in saying that 1.5 - 1.4 does not equal .11111... because there are numbers between .1 (the logical answer) and the repeating answer: .11111....

What I'm saying is that I was taught that .1 is the same as .0999999....

What number comes between .1 and .0999999....? There isn't one. Therefore, as I was taught, they are the same number.

And I know this sounds like total BS. And that is what I thought when I heard it, too...but after thinking about it, I accepted it, and thought it was pretty cool. I'll see if I can find it online somewhere.
You are correct. As college analysis courses will teach, every real number has a unique non-terminating decimal expansion. As 1/3 can be written 0.333333... , so can 1 be written 0.9999999999...
 

khammack

macrumors regular
Sep 28, 2004
166
0
Portland, OR
Thom_Edwards said:
i have to concur. his original problem is subtraction with 2 decimal places. you don't get repeating decimal places in addition/subtraction EVER. repeating decimals only happen in division (if you constrain this topic to basic math like +, -, /, and *)

does 1.5 - 1.4 = .1111111111111111111111111111... ? no, it doesn't

That's not entirely accurate. Do keep in mind that the computer has no concept of "decimal" places. All arithmetic is done in binary.

For reference, I tried the same calculation on linux (the system I happen to be in front of) using python and scheme:

Code:
Python 2.4.1 (#2, Mar 30 2005, 21:51:10)
[GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 9077.38 - 8980.75
96.6299999999992

guile> (- 9077.38 8980.75)
96.6299999999992
To understand this you need to convert the numbers into binary (perhaps even ieee floating point notation) and then do the arithmetic in binary. I think what you'll probably find is that at least one of the fractional values .38 or .75 are infinitely repeating patterns when expressed as a binary number.

(I'd do the conversion for you, but I'm at work; if nobody has posted the solution later I'll check back and do it).

-kev
 

sonictruth

macrumors member
Nov 18, 2004
60
0
Pennsylvania
Happens in Quicken too

I've had the same issue with Quicken 2004's portfolio... for instance, a cost basis indicated as $2,999.9999 (reality is a purchase of $3K), and other columns calculated based on the 'wrong' value. So, since I am assuming Quicken does not use the OS X calculator, is this a difference in floating point calcs on the processor???
 

Mitthrawnuruodo

Moderator emeritus
Mar 10, 2004
13,594
141
Bergen, Norway
The problem is a small error in how Calc is handeling some Floats... and it should have been corrected...
Just because there's a way to prove (mathematicly) that 1 = .999999... that has NO relevance here, whatsoever....
 

tobefirst

macrumors 601
Jan 24, 2005
4,029
1,139
St. Louis, MO
Mitthrawnuruodo said:
The problem is a small error in how Calc is handeling some Floats... and it should have been corrected...
Just because there's a way to prove (mathematicly) that 1 = .999999... that has NO relevance here, whatsoever....
I agree that it's trivial, but I wouldn't say that it has no relevance. :) Haha...the title of the thread was Calculator off... is yours too?, but it's not "off" by definition.

And after all, isn't Calculator an application to "solve" mathematical formulas? :)

Of course, I'm just playing around and don't mean to offend anyone.
 

Gizmotoy

macrumors 65816
Nov 6, 2003
1,081
118
Yea, mine does the same thing... and by buddy's version in Tiger does it too. Oddly his dashboard calculator gives the right answer.
 

Thom_Edwards

macrumors regular
Apr 11, 2003
240
0
khammack said:
That's not entirely accurate. Do keep in mind that the computer has no concept of "decimal" places. All arithmetic is done in binary.

For reference, I tried the same calculation on linux (the system I happen to be in front of) using python and scheme:

Code:
Python 2.4.1 (#2, Mar 30 2005, 21:51:10)
[GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 9077.38 - 8980.75
96.6299999999992

guile> (- 9077.38 8980.75)
96.6299999999992
To understand this you need to convert the numbers into binary (perhaps even ieee floating point notation) and then do the arithmetic in binary. I think what you'll probably find is that at least one of the fractional values .38 or .75 are infinitely repeating patterns when expressed as a binary number.

(I'd do the conversion for you, but I'm at work; if nobody has posted the solution later I'll check back and do it).

-kev

i quite understand what you are saying (and the others that want to say that Calculator is "correct"), but this isn't a class in how computers work and theoretical mathematics.

if i were writing a program that was teaching kids how to subtract decimal numbers and i gave them that crazy answer, it would be reported as a bug from EVERY user. i guarantee it.

also, i appreciate your condescending attitude on my understanding of binary and floating points, but just because you can recreate it on linux doesn't make it *right*.
 

eva01

macrumors 601
Feb 22, 2005
4,714
0
Gah! Plymouth
well to try and put it to rest i believe it ONLY applies to .9999 repeating nothing else with a .9 repeating like .099 repeating, i believe the calculator is just off
 

Tortellino

macrumors newbie
Apr 22, 2005
26
0
Thom_Edwards said:
if i were writing a program that was teaching kids how to subtract decimal numbers and i gave them that crazy answer, it would be reported as a bug from EVERY user. i guarantee it.
How have you set the precision in your calculator? Since you supplied numbers with only 2 decimal digits, you need to round off your answer to two decimals as well. Once you do it you will get the answer you expect. I admit it would be thoughtful to detect these things automatically, but if you Calculator view precision is set to 16 digits, and you only supply 6, then the computer will display the closest real number with 16 digits to the binary result it obtained.

That being said, apart from setting appropriate precision for the Calculator display, there are programs like Mathematica that can work with infinite precision by performing operations on non-binary representation of numbers.

Thom_Edwards said:
also, i appreciate your condescending attitude on my understanding of binary and floating points, but just because you can recreate it on linux doesn't make it *right*.
I did not find that explanation condescending, but rather thorough, tested on an independent platform. If it hurt your feelings then you probably should work on turning your sensitivity down a notch.
 

realityisterror

macrumors 65816
Aug 30, 2003
1,354
0
Snellville, GA
Thom_Edwards said:
i have to concur. his original problem is subtraction with 2 decimal places. you don't get repeating decimal places in addition/subtraction EVER.
uhh...

2/3 - 1/3 ???
or maybe you consider that division as it is 2/3...

reality
 

clayj

macrumors 604
Jan 14, 2005
7,473
180
visiting from downstream
This occurs in a lot of different programs, including Excel, when you do math that involves decimals. Since all math in a computer is ultimately performed in binary, decimals that don't translate well to binary (for example, 0.1 cannot be perfectly converted to binary and back again) sometimes are mishandled.

Ideally, the program should know to correct this, but sometimes, you have to force it to give you the correct answer. For example, in Excel 2003 for Windows, the formula "=1-0.9-0.09-0.01" should return 0, but it actually returns -1.9082E-17 (or -0.0000000000000000191). Changing the formula to "=ROUND(1-0.9-0.09-0.01,8)" (which rounds the result to 8 decimal places, more than we need considering our numbers have no more than 2 decimal places each) gives the correct result of 0.
 

eva01

macrumors 601
Feb 22, 2005
4,714
0
Gah! Plymouth
clayjohanson said:
This occurs in a lot of different programs, including Excel, when you do math that involves decimals. Since all math in a computer is ultimately performed in binary, decimals that don't translate well to binary (for example, 0.1 cannot be perfectly converted to binary and back again) sometimes are mishandled.

Ideally, the program should know to correct this, but sometimes, you have to force it to give you the correct answer. For example, in Excel 2003 for Windows, the formula "=1-0.9-0.09-0.01" should return 0, but it actually returns -1.9082E-17 (or -0.0000000000000000191). Changing the formula to "=ROUND(1-0.9-0.09-0.01,8)" (which rounds the result to 8 decimal places, more than we need considering our numbers have no more than 2 decimal places each) gives the correct result of 0.
exact same thing happens when finding the root on a TI-83 and TI-86 Graphing calculator (well not that problem, but with finding root in graphs), it is because of the algorithm the calculator uses (not sure if it occurs with the TI-90 haven't used it)
 

telecomm

macrumors 65816
Nov 30, 2003
1,371
8
Rome
This topic has been discussed about a million times (well, that might be off, I'm using Calculator) ;)

I'm too lazy to do a thread search (looks like I'm not the only one), or I would provide links.