View Single Post
red-beard red-beard is offline
canna change law physics
 
red-beard's Avatar
 
Join Date: Jul 2000
Location: Houston, Tejas
Posts: 43,411
Garage
Quote:
Originally Posted by DavidB911 View Post
We were always told to convert it to a number between 0 and 10. Then you can multiply the number of decimals needed. Say if its "1234" being squared it would be:

(1234)^2 = 1522756

Divide 1234 by 1000, then square 1.234. Multiply the answer by 1e6 to get 1522756.
You get serious rounding errors that way. That works in engineering, but it won't work in banking software.

A lot the answer depends on the use.
__________________
James
The pessimist complains about the wind; the optimist expects it to change; the engineer adjusts the sails.- William Arthur Ward (1921-1994)
Red-beard for President, 2020
Old 10-14-2010, 05:27 AM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #5 (permalink)