Fun with floating point

WPF uses double-precision floating point numbers (double in C#) in much of its public API and it uses single-precision floating point for much of its internal rendering.  So floating point math is something we deal with constantly.  Oddly enough, I actually knew very little about the gory details until I recently tried to write a container that had to use doubles as a key, which required working around the problems with precision in floating point math.  I found the whole exercise to be very interesting, and so I’ll present what I learned here.

The Problem
Consider this code:

if (0.8 - 0.7 == 0.1)
{
   Console.WriteLine("Math makes sense.");
}
Believe it or not, the comparison will fail.  This bothers me, because I don’t know how to write code that will work as expected.  (In this simple case, the compiler will actually generate a warning the the code in unreachable.)

Binging around the internet, I found a great article, which I recommend as a good read:
http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm

A common approach is to never compare floating point numbers directly, but instead just check that the absolute value of the difference is small enough.  The rub, of course, is that it is very hard to pick the right value for “small enough”.  Especially since floating point numbers cover a huge range – from very, very small to very, very large.  For example, +/- 0.0001 is probably fine for some values, but is probably not great for very, very large numbers and is certainly a bad choice for very, very small numbers where this tolerance is actually larger than the numbers being worked with.

Read more: Presentation Source

Posted via email from jasper22's posterous