JavaScript Rounding Question...

simX

macrumors 6502a
Original poster
May 28, 2002
755
0
Bay Area, CA
I have a JavaScript program that does some rounding of numbers so that they don't get too long when presented in text fields.

However, it seems that Internet Explorer (conveniently) likes to ignore these things and continues to display the unrounded numbers (for example, it displays "8.019600000000000000001" instead of the "8.0196" that the JavaScript code should create).

Here's the snippit of code:

kval = (Math.round(tempK3*1000))/1000
document.show.kvalue.value = kval
How come that doesn't work in Internet Explorer, but it seems to work fine in Mozilla? (Note: "tempK3" is a previous variable.)
 

cleo

macrumors 65816
Jan 21, 2002
1,186
0
Tampa Bay Area, FL, USA
I don't know the answer to your specific question or if there's a workaround for IE, but in my experience it's always 1000% safer to use server-side scripting language for such things... there's just too many quirks from browser to browser.
 

simX

macrumors 6502a
Original poster
May 28, 2002
755
0
Bay Area, CA
Heh, yeah, but what would you suggest in lieu of JavaScript? Basically what I'm trying to do is on-the-fly calculating of the x and y values of the point where a user clicks on the graph. It seems JavaScript is the best way to do this, even if I do have to compensate for some browser quirks.
 

Choppaface

macrumors 65816
Jan 22, 2002
1,187
0
SFBA
isn't there a truncate function?

you could try just grabbing a substring of the first x chars in the number (or you could split it into an array on the period, replace element 1 of that array (the nums after the decimal) with it's first few digits, then join using ".".......)
 

backspinner

macrumors 6502a
Apr 29, 2002
547
0
Eindhoven
this snippet handles cases like this, including leading zero and decimal separators:

if (b.value.charAt(0) == '.') b.value = "0" + b.value;
if (b.value.indexOf('.') == -1) b.value = b.value + ".00";
while (b.value.charAt(b.value.length-3) != '.') b.value = b.value + '0';
 

oldMac

macrumors 6502a
Oct 25, 2001
522
1
Strange Problem = Strange Solution

Hi SimX,

Strange issue you've come across.
The error isn't happening in the round. The error is happening in the division portion of your number. Apparently, IE's division method isn't very precise.

Oddly, but fortunately, if you convert the number to a string, IE seems to recognize that it's not smart enough to hold a number of that precision and will happily truncate it for you. (This works fine in Netscape 4.7 and 6, too.)

-------------- snip ----------------

var tempK3 = 8.019600000000000000001;
var kval = Math.round(tempK3*10000)/10000;

document.show.kvalue.value = String(kval);

--------------- end snip -----------
 

bikebuddy

macrumors newbie
Jul 18, 2002
1
0
I think OldMac is right. I don't program in JavaScript, but I suspect your problem is in the decimal to binary conversion when dividing by 1000. In particular, while 10^-3 has a finite decimal representation, it does not have a finite binary representation and so the division is not performed exactly.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.