ACCURACY :
accuracy | base | buttons | constants | contact | detail view | legal | ops | precedence | prefs | terms | usage
This calculator is not responsible for anything, ever.
Internal calculations are based on 64bit floating point numbers, there are limits in accuracy and size of numbers defined. The normal Calc Window may display 32bit floating point numbers to hide rounding error ugliness which can occur in the engine. Detailed View will show greater accuracy. Also, the engine is based in java, for what that's worth. Natively the calculator thinks in terms of radians, and in base 10, so conversion to degrees and alternate bases may introduce itsy bitsy errors of its own. All of these errors are really small. If I thought they were a problem, I'd fix them. Beware of rounding error in other bases, it may not be immediately obvious that 3.7777777 in base 8 should really be 4. I'll try to improve rounding inteligence and configurability in future realeases.
Numbers are less accurate as they become greater in magnitude, by the very nature of floating point math on a computer. base 10 numbers are limited to magnitude less than ~10^308. If this isn't good enough for you, find a variable precision calculator which does it's own mathematical operations.
Numbers in other bases are limited in magnitude to less than or equal to 2^63, ~ 9e18 or 1y2p0ij32e8e7_base 36. Entering numbers (or expressions which evaluate to numbers) larger than this should yield an explicit error, but could theoretically just yield incorrect results.
As of version 0.8, setting the Significant Digits slider has some effect on the number of decimal places retained in base conversion. Lower bases keep more decimals, which is reasonable, but the relationship between Significant Digits and Base Precision is fuzzy at best. Suffice it to say, more Significant Digits means more precision.
accuracy | base | buttons | constants | contact | detail view | legal | ops | precedence | prefs | terms | usage