macOS Downside to NSDecimal and NSDecimalNumber?

[perrien]

I'm working on a program that does quite a bit of 3D math with planes, points, lines, etc. The most base class has 3 floats for x, y and z coordinates and I find I'm doing a fair bit of fudging and working around rounding issues with floats.

I was thinking of moving it over to either NSDecimal or NSDecimalNumber and am wondering 1) will this probably work around those issues and 2) aside from more complicated code, what are the drawbacks?

I'm assuming that NSDecimalNumber is slower and uses more memory than NSDecimal which is slower and uses more memory than a float but the final model I'm working with is probably only going to have around 4-600 points so I don't think speed or memory are going to be huge concerns. So that just leaves more complicated code. I'm okay with that if it does away with the "if (abs(a-b)<0.001)" and all the other massaging that needs to be done.

Advice? Suggestions?

 

[chown33]

You should consider the double type before using NSDecimalNumber. At least double will have FPU hardware support, and be a relatively simple source change (you could use a C typedef).

Also please explain exactly what kind of issues you're having with floats. You say rounding problems, but is it really rounding or is it a precision problem? They are two different issues: you can have loss-of-precision problems that are unrelated to rounding, and you can have rounding problems unrelated to precision. For example, a true rounding problem would not be remedied by switching to the double type. It's also possible that switching to NSDecimalNumber wouldn't fix a rounding problem, either.

How familiar are you with floating-point representations?

I'm okay with that if it does away with the "if (abs(a-b)<0.001)" and all the other massaging that needs to be done.

Learn to use C macros. Example:

#define EQ(a,b)  (abs((a)-(b))<0.001)

if ( EQ(foo,bar) )  ...blah blah blah

You can even define macros that take the precision-limit as an arg:

#define NEAR(a,b,c)  (abs((a)-(b))<(c))

if ( NEAR(foo,bar,0.001) )  ...blah blah blah

 

[perrien]

You should consider the double type before using NSDecimalNumber.

That would be a fair bit easier, I may try that also.

Also please explain exactly what kind of issues you're having with floats. You say rounding problems, but is it really rounding or is it a precision problem?

I'm not entirely sure, or exactly how to tell the difference. All points are going to be calculated vs entered so are likely have long decimal places. All the problems seem to boil down to Point1 never equals Point2 because 152.13...6 != 152.13...7. I would guess it may be more of a precision than rounding problem and doubles may take care of that but I'd still need to use some arbitrary precision level to decide when 2 things are equal.

How familiar are you with floating-point representations?

Marginally. From my understanding since we're converting base 10 to base 2 back and forth, errors arise there as one number can't be exactly described in the other base.

 

[chown33]

I'm not entirely sure, or exactly how to tell the difference. All points are going to be calculated vs entered so are likely have long decimal places. All the problems seem to boil down to Point1 never equals Point2 because 152.13...6 != 152.13...7. I would guess it may be more of a precision than rounding problem and doubles may take care of that but I'd still need to use some arbitrary precision level to decide when 2 things are equal.

I think you should do some test cases and experiments.

Take one of your known problematic calculations with all floats. Code it using the exact data representations that cause the problem; take no shortcuts, no optimizations. It should exhibit exactly the same issues as when used in your 3D code.

Next, change the type to double and gather some experimental data. Compare the all-double results to the all-float results. If the precision difference is in the least digit(s), then you can probably use double for all your calculations EXCEPT comparison. For comparison (and ONLY comparison), cast the double to float (which rounds to float precision) and then compare for equality (e.g. macrofy this). What this does is give you the extra precision of double for all calculations and stored representations, but it rounds to only the 6-7 significant decimal digits of a float for comparisons. I assume the dynamic range of float (1E+/-38) is adequate.

Next, take the same problematic calculation and convert to the NSDecimal type. Perform all the same calculations and comparisons. If NSDecimal solves the problems, then it's mainly a question of speed and notational difficulty. By the latter, I mean you can't write a=b+c/d; you have to write a bunch of function calls or macros or whatever that cause the actual NSDecimal calculations. Personally, I don't see how that kind of expanded code would be more readable than the abs(a-b) thing you've got now. If the only issue you have now is comparison for equality, macros (or doubles cast to floats) can solve that. Replacing every calculation like a=b+c/d with a bunch of functions seems like insane overkill.

And that's not even taking speed into account. A slowdown of 3 orders of magnitude (microseconds to milliseconds, or milliseconds to seconds) wouldn't much surprise me. I'm not saying that's what you'll get, you need to run some experiments to find out what you'll really get. I'm just saying that a huge slowdown wouldn't surprise me.

Marginally. From my understanding since we're converting base 10 to base 2 back and forth, errors arise there as one number can't be exactly described in the other base.

If that were the only source of errors, you'd have a point. But it's not. Subtraction, division, and other operations can lose huge amounts of precision.

At the very least, read the wikipedia article. Also see "What Every Computer Scientist Should Know About Floating-Point Arithmetic" linked to from the wikipedia article.

 

[gnasher729]

I'm working on a program that does quite a bit of 3D math with planes, points, lines, etc. The most base class has 3 floats for x, y and z coordinates and I find I'm doing a fair bit of fudging and working around rounding issues with floats.

I was thinking of moving it over to either NSDecimal or NSDecimalNumber and am wondering 1) will this probably work around those issues and 2) aside from more complicated code, what are the drawbacks?

I'm assuming that NSDecimalNumber is slower and uses more memory than NSDecimal which is slower and uses more memory than a float but the final model I'm working with is probably only going to have around 4-600 points so I don't think speed or memory are going to be huge concerns. So that just leaves more complicated code. I'm okay with that if it does away with the "if (abs(a-b)<0.001)" and all the other massaging that needs to be done.

Advice? Suggestions?

Use double instead of float. There will be situations where float doesn't give enough precision; double will give enough precision in most situations even with slightly bad code. If double is not precise enough then you can bet that using NSDecimal isn't going to help you either. And using NSDecimal has the huge disadvantage that the number of people with any experience able to help just shrinks by a factor 1000.

Precision is one problem with floating point arithmetic; the other is that any floating point arithmetic, whether binary or decimal, will involve rounding error. Using NSDecimalNumber is not going to help with this. However, having done quite a lot of 3d graphics at some point in my life, I never found any need to test numbers for equality as you seem to do. You may want to post the code that causes you problems.

I'm not entirely sure, or exactly how to tell the difference. All points are going to be calculated vs entered so are likely have long decimal places. All the problems seem to boil down to Point1 never equals Point2 because 152.13...6 != 152.13...7. I would guess it may be more of a precision than rounding problem and doubles may take care of that but I'd still need to use some arbitrary precision level to decide when 2 things are equal.

Question: Why do you need to know whether two points are equal? If you find two points using two different calculations, chances that they are equal exactly zero. So you are trying to solve some unusual problem here.

 

[perrien]

Question: Why do you need to know whether two points are equal? If you find two points using two different calculations, chances that they are equal exactly zero. So you are trying to solve some unusual problem here.

So here's basically what I'm doing. Start with a cube (six individual faces), now slice off one corner. You're replacing 1 point with three and now have seven faces. The way things are currently stored, each face has it's own set of points so on each face, one point becomes two. After cutting the three faces you have six points but only three are actually unique.

I'm sure there's a dozen different ways to model this and I wouldn't be surprised if I've chosen one of the least efficient but it's one where I can understand the math required and can visualize what's going on in my head.

 

[perrien]

I think you should do some test cases and experiments.

Take one of your known problematic calculations with all floats....

Next, change the type to double and gather some experimental data. Compare the all-double results to the all-float results.

I already had a few test cases where things went pear shaped so I just did a simple find and replace, float to double on the whole project, ran them again and all came out well. That simple fix seems to have done the job.

Many thanks to all and I'll definitely take a look at the linked articles.

 

[chown33]

The way things are currently stored, each face has it's own set of points so on each face, one point becomes two.

A conventional way of doing this is to define each vertex only once, e.g. in an array. Each face then consists of the vertex-array indexes that make up that face. That is, a face is a vertex-index list instead of a vertex list, where each vertex consists of 3 coordinate values.

Some consequences:
1. Vertex coordinate equality isn't a problem, because no vertex coordinate is represented more than once.
2. Vertex equality is simplified: you don't compare 3 values for equality, you only compare one integer vertex-index for equality with another vertex-index.

Another consequence is the indirection through the array to retrieve actual vertex coordinate values. In C, this is unlikely to be a significant speed loss, especially if you use pointers instead of vertex-indices (not worth doing unless and until speed issues are measured and significant). The same principles apply with pointers: comparison is a single pointer compare, not a tuple, and each vertex-pointer is unique and distinct.