Everything2
Near Matches
Ignore Exact
Full Text
Everything2

floating-point comparison

created by Potatoswatter

(idea) by Potatoswatter (2.1 mon) (print)   ?   (I like it!) Wed Jun 26 2002 at 4:15:38

Although C provides the == operator for use with floating-point variables (float and double), and most processor ISAs have instructions to support it, it is not the best way to compare two numbers.

Real numbers are represented in processor registers by a finite number of bits. If you perform an operation that requires more bits in the answer than the processor has, you get an imprecise answer. This is normally alright, because the error appears in the least-significant bits.

However, for the operation of comparison, all bits are significant. If two numbers differ in the slightest, they appear unequal. Thus 1.0000001 - 1 will not reliably equal .0000001. Unless both operands are obvious sums of powers of two, the test will almost always fail in cases where promotion is required to compare a single-precision float to a double-precision double.

It would be nice if C did provide a useful comparison operator, but it doesn't. You just have to do it as subtraction and an absolute value.

if ( abs( a - b ) < .000001 ) SpinWheels();

Use your programming karma and do test runs to invent the proper margin of error.


printable version
chaos

1+1=1.99999999486 Object oriented vs. procedural programming The karma that makes you spend all the money you've got in any given moment karma
The Art of Computer Programming FPU Floating point
Y'know, if you log in, you can write something here, or contact authors directly on the site. Create a New User if you don't already have an account.
  Epicenter
Login
Password

password reminder
register

Everything2 Help

Cool Staff Picks
Just another sprinkling of indeterminacy
Beef noodle soup
The stars stood still and naked
Sensei, we've got another lesbian stuck in the goddamn shredder
car chase
Tricks of the Propagandist
American flag etiquette
Der Ring des Nibelungen
Real war: Civilization threatened!
Big O
Carl Jung
Elie Wiesel
Robert Crumb
bow
New Writeups
Aerobe
Watch out for falling meat(poetry)
C-Dawg
Beelzebub has a devil put aside for me(fiction)
Pavlovna
My Better Half(fiction)
kanoodle
Molson muscle(essay)
aneurin
You pays your money and you takes your choice(idea)
shaogo
July 20, 2008(log)
Glowing Fish
Tualatin River(place)
The Jacket
Words of Advice(idea)
John_Fox
Good Intentions Gone Wrong(person)
Heitah
Posthumous Oscar(thing)
ignis_glaciesque
University of South Florida(place)
ignis_glaciesque
Flogstaskriket(idea)
liveforever
Caesar's last breath(idea)
dagnyswaggart
she wants to believe(personal)
antigravpussy
he doesn't know, but her eyes widen too far(thing)
E2 is a by-product of the existence of The Everything Development Company