Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Calculator off... is yours too?

LeeTom

LeeTom

macrumors 68000
Original poster
May 31, 2004
1,587
301
Okay, Calculator says that
9,077.38 - 8,980.75 = 96.629999999999

Huh?????
Can anyone duplicate this?
 
I get that to (using Calculator 3.2.1 under 10.3.9)... I thought they fixed that float inaccuarcy in Calculator a while ago... :confused:

...good thing I soon get Tiger (hopefully) with a new Calc in Dashboard... :rolleyes:
 
if i recall correctly from high school...

If we assume that the 9 is repeating (and Calculator just didn't have enough spots...as opposed to the number ending after the tenth nine) then 96.629999999999 is actually the same as 96.63, since there are no numbers in between. Assuming that the 9 is repeating, this is actually a correct answer.

Again, that's if I remember correctly from a class I took 6 years ago. :)
 
tobefirst said:
if i recall correctly from high school...

If we assume that the 9 is repeating (and Calculator just didn't have enough spots...as opposed to the number ending after the tenth nine) then 96.629999999999 is actually the same as 96.63, since there are no numbers in between. Assuming that the 9 is repeating, this is actually a correct answer.

Again, that's if I remember correctly from a class I took 6 years ago. :)
Click to expand...

96.629999 repeating is not equivalent to 96.63. Time to take that class again!
 
Run Windows Update, it does the same thing. Kinda. I'll look for some screenshots... anyways....
 
LeeTom said:
96.629999 repeating is not equivalent to 96.63. Time to take that class again!
Click to expand...

i have to concur. his original problem is subtraction with 2 decimal places. you don't get repeating decimal places in addition/subtraction EVER. repeating decimals only happen in division (if you constrain this topic to basic math like +, -, /, and *)

does 1.5 - 1.4 = .1111111111111111111111111111... ? no, it doesn't
 
okay... interesting. i'm at work right now (on a windows xp machine) and sure enought it doesn't have the same problem. it says 96.63.

i completely agree with Thom_Edwards post. this is strange??
 
Thom_Edwards said:
i have to concur. his original problem is subtraction with 2 decimal places. you don't get repeating decimal places in addition/subtraction EVER. repeating decimals only happen in division (if you constrain this topic to basic math like +, -, /, and *)

does 1.5 - 1.4 = .1111111111111111111111111111... ? no, it doesn't
Click to expand...

Like I said, it's been awhile, but I'm still thinking I'm right. You are right in saying that 1.5 - 1.4 does not equal .11111... because there are numbers between .1 (the logical answer) and the repeating answer: .11111....

What I'm saying is that I was taught that .1 is the same as .0999999....

What number comes between .1 and .0999999....? There isn't one. Therefore, as I was taught, they are the same number.

And I know this sounds like total BS. And that is what I thought when I heard it, too...but after thinking about it, I accepted it, and thought it was pretty cool. I'll see if I can find it online somewhere.
 
tobefirst said:
Like I said, it's been awhile, but I'm still thinking I'm right. You are right in saying that 1.5 - 1.4 does not equal .11111... because there are numbers between .1 (the logical answer) and the repeating answer: .11111....

What I'm saying is that I was taught that .1 is the same as .0999999....

What number comes between .1 and .0999999....? There isn't one. Therefore, as I was taught, they are the same number.

And I know this sounds like total BS. And that is what I thought when I heard it, too...but after thinking about it, I accepted it, and thought it was pretty cool. I'll see if I can find it online somewhere.
Click to expand...

You are correct. As college analysis courses will teach, every real number has a unique non-terminating decimal expansion. As 1/3 can be written 0.333333... , so can 1 be written 0.9999999999...
 
Thom_Edwards said:
i have to concur. his original problem is subtraction with 2 decimal places. you don't get repeating decimal places in addition/subtraction EVER. repeating decimals only happen in division (if you constrain this topic to basic math like +, -, /, and *)

does 1.5 - 1.4 = .1111111111111111111111111111... ? no, it doesn't
Click to expand...


That's not entirely accurate. Do keep in mind that the computer has no concept of "decimal" places. All arithmetic is done in binary.

For reference, I tried the same calculation on linux (the system I happen to be in front of) using python and scheme:

Code: 
Python 2.4.1 (#2, Mar 30 2005, 21:51:10)
[GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 9077.38 - 8980.75
96.6299999999992

guile> (- 9077.38 8980.75)
96.6299999999992

To understand this you need to convert the numbers into binary (perhaps even ieee floating point notation) and then do the arithmetic in binary. I think what you'll probably find is that at least one of the fractional values .38 or .75 are infinitely repeating patterns when expressed as a binary number.

(I'd do the conversion for you, but I'm at work; if nobody has posted the solution later I'll check back and do it).

-kev
 
Happens in Quicken too

I've had the same issue with Quicken 2004's portfolio... for instance, a cost basis indicated as $2,999.9999 (reality is a purchase of $3K), and other columns calculated based on the 'wrong' value. So, since I am assuming Quicken does not use the OS X calculator, is this a difference in floating point calcs on the processor???
 
The problem is a small error in how Calc is handeling some Floats... and it should have been corrected...
Just because there's a way to prove (mathematicly) that 1 = .999999... that has NO relevance here, whatsoever....
 
Mitthrawnuruodo said:
The problem is a small error in how Calc is handeling some Floats... and it should have been corrected...
Just because there's a way to prove (mathematicly) that 1 = .999999... that has NO relevance here, whatsoever....
Click to expand...

I agree that it's trivial, but I wouldn't say that it has no relevance. :) Haha...the title of the thread was Calculator off... is yours too?, but it's not "off" by definition.

And after all, isn't Calculator an application to "solve" mathematical formulas? :)

Of course, I'm just playing around and don't mean to offend anyone.
 
Yea, mine does the same thing... and by buddy's version in Tiger does it too. Oddly his dashboard calculator gives the right answer.
 
khammack said:
That's not entirely accurate. Do keep in mind that the computer has no concept of "decimal" places. All arithmetic is done in binary.

For reference, I tried the same calculation on linux (the system I happen to be in front of) using python and scheme:

Code: 
Python 2.4.1 (#2, Mar 30 2005, 21:51:10)
[GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 9077.38 - 8980.75
96.6299999999992

guile> (- 9077.38 8980.75)
96.6299999999992

To understand this you need to convert the numbers into binary (perhaps even ieee floating point notation) and then do the arithmetic in binary. I think what you'll probably find is that at least one of the fractional values .38 or .75 are infinitely repeating patterns when expressed as a binary number.

(I'd do the conversion for you, but I'm at work; if nobody has posted the solution later I'll check back and do it).

-kev
Click to expand...


i quite understand what you are saying (and the others that want to say that Calculator is "correct"), but this isn't a class in how computers work and theoretical mathematics.

if i were writing a program that was teaching kids how to subtract decimal numbers and i gave them that crazy answer, it would be reported as a bug from EVERY user. i guarantee it.

also, i appreciate your condescending attitude on my understanding of binary and floating points, but just because you can recreate it on linux doesn't make it *right*.
 
well to try and put it to rest i believe it ONLY applies to .9999 repeating nothing else with a .9 repeating like .099 repeating, i believe the calculator is just off
 
Thom_Edwards said:
if i were writing a program that was teaching kids how to subtract decimal numbers and i gave them that crazy answer, it would be reported as a bug from EVERY user. i guarantee it.
Click to expand...

How have you set the precision in your calculator? Since you supplied numbers with only 2 decimal digits, you need to round off your answer to two decimals as well. Once you do it you will get the answer you expect. I admit it would be thoughtful to detect these things automatically, but if you Calculator view precision is set to 16 digits, and you only supply 6, then the computer will display the closest real number with 16 digits to the binary result it obtained.

That being said, apart from setting appropriate precision for the Calculator display, there are programs like Mathematica that can work with infinite precision by performing operations on non-binary representation of numbers.

Thom_Edwards said:
also, i appreciate your condescending attitude on my understanding of binary and floating points, but just because you can recreate it on linux doesn't make it *right*.
Click to expand...

I did not find that explanation condescending, but rather thorough, tested on an independent platform. If it hurt your feelings then you probably should work on turning your sensitivity down a notch.
 
Thom_Edwards said:
i have to concur. his original problem is subtraction with 2 decimal places. you don't get repeating decimal places in addition/subtraction EVER.
Click to expand...

uhh...

2/3 - 1/3 ???
or maybe you consider that division as it is 2/3...

reality
 
This occurs in a lot of different programs, including Excel, when you do math that involves decimals. Since all math in a computer is ultimately performed in binary, decimals that don't translate well to binary (for example, 0.1 cannot be perfectly converted to binary and back again) sometimes are mishandled.

Ideally, the program should know to correct this, but sometimes, you have to force it to give you the correct answer. For example, in Excel 2003 for Windows, the formula "=1-0.9-0.09-0.01" should return 0, but it actually returns -1.9082E-17 (or -0.0000000000000000191). Changing the formula to "=ROUND(1-0.9-0.09-0.01,8)" (which rounds the result to 8 decimal places, more than we need considering our numbers have no more than 2 decimal places each) gives the correct result of 0.
 
clayjohanson said:
This occurs in a lot of different programs, including Excel, when you do math that involves decimals. Since all math in a computer is ultimately performed in binary, decimals that don't translate well to binary (for example, 0.1 cannot be perfectly converted to binary and back again) sometimes are mishandled.

Ideally, the program should know to correct this, but sometimes, you have to force it to give you the correct answer. For example, in Excel 2003 for Windows, the formula "=1-0.9-0.09-0.01" should return 0, but it actually returns -1.9082E-17 (or -0.0000000000000000191). Changing the formula to "=ROUND(1-0.9-0.09-0.01,8)" (which rounds the result to 8 decimal places, more than we need considering our numbers have no more than 2 decimal places each) gives the correct result of 0.
Click to expand...

exact same thing happens when finding the root on a TI-83 and TI-86 Graphing calculator (well not that problem, but with finding root in graphs), it is because of the algorithm the calculator uses (not sure if it occurs with the TI-90 haven't used it)
 
This topic has been discussed about a million times (well, that might be off, I'm using Calculator) ;)

I'm too lazy to do a thread search (looks like I'm not the only one), or I would provide links.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back