Everything2
Near Matches
Ignore Exact
Full Text
Everything2

Rounding of Numbers to n Significant Figures

created by namtap

(idea) by namtap (6.3 y) (print)   ?   (I like it!) Fri May 25 2001 at 16:19:41

PREMISE

Rounding numbers will introduce a bias if done incorrectly. Round off error is inevitable, but bias is not.

DISCUSSION

The following algorithm has been proposed for the rounding of numbers to n significant figures. This method has the advantage of reducing the bias that may accumulate when the discarded portion of the number is exactly one-half a unit in the nth place. The method is useful in that it increases the accuracy of scientific research data.

The algorithm would also be of use in financial transactions, where the accumulation of error could potentially add up to big bucks. Imagine; a penny here, a penny there, soon you have two cents.

The resulting number can be said to be well rounded.

ALGORITHM

   To round a number to n significant figures, discard all
   digits to the right of the nth place. If the discarded
   number is less than one-half a unit in the nth place, leave
   the nth digit unchanged. If the discarded number is greater
   than one-half a unit in the nth place, increase the
   nth digit by 1. If the discarded number is exactly one-half
   a unit in the nth place, leave the nth digit
   unchanged if it is an even number and add 1 to it if it is odd.

   -- Measurement Systems: Application and Design, Doebelin, Ernest O., page 61.
     Copyright 1975 McGraw-Hill, Inc.

EXAMPLES

The following examples illustrate the major points of the algorithm.

  1. Round 1.23456 to 3 significant digits.

    Discard all digits to the right of the 3rd place. Since 0.00456 is less than half of 0.01, the 3rd digit remains unchanged.

    Result: 1.23.

  2. Round 1.23678 to 3 significant figures.

    Discard all digits to the right of the 3rd place. Since 0.00678 is greater than half of 0.01, the 3rd digit is increased by 1.

    Result: 1.24.

  3. Round 1.245 to 3 significant figures.

    Discard all digits to the right of the 3rd place. Since 0.005 is exactly half of 0.01, and since the 3rd digit is even, the 3rd digit remains unchanged.

    Result: 1.24

  4. Round 1.275 to 3 significant figures.

    Discard all digits to the right of the 3rd place. Since 0.005 is exactly half of 0.01, and since the 3rd digit is odd, the 3rd digit is increased by 1.

    Result: 1.28

CONCLUSION

I hope this is useful and not pedantic.


printable version
chaos

significant digits Accumulate Microsoft transaction server well rounded
algorithm Drugs That May Increase Male Libido M&M's data
premise pedantic Rounding discard
right example significant figures application
statistics Dread Zeppelin Odd Even
penny imagine discussion Introduce
Y'know, if you log in, you can write something here, or contact authors directly on the site. Create a New User if you don't already have an account.
  Epicenter
Login
Password

password reminder
register

Everything2 Help

Cool Staff Picks
Little presents from the Node Fairy:
How to Solve an Academic Problem
Prostitution
NetHack
America
When being chased by CIA trainees, don't mention Belgium to the waffle house physicist
My Lai Massacre
Arizona-Sonora Desert Museum
Discovery
Freeman and me and the rest of the world
The Feynman Lectures on Physics
sandfly
Madeleine L'Engle
Dulce et Decorum Est
New Writeups
TheOrientalAfrican
ToTheGuyWithAidsSittingN... (I Am Really Really Sorry)(person)
Heitah
Why I love Everything2(person)
trixingee
Dungeon Mastering for the first time(idea)
Netrat0
It's Called Subtext, Honey(person)
eyeofthebeholder
The Dragon(idea)
Heitah
consist, comprise, constitute, or compose(idea)
Meezzio
Gotlandssnus(thing)
argv
Astral Plane(idea)
Madara
One Winged Angel(fiction)
Tom Rook
Talk is cheap(poetry)
shaogo
Adelle Davis(person)
Aerobe
race car g sfjsgsd(poetry)
Binah
Dream Log: July 5, 2008(dream)
StrawberryFrog
Forgotten things in space(idea)
antigravpussy
velvet revolution fairy tale(idea)
This affordable entertainment brought to you by The Everything Development Company