Kbase 21678: Incorrect rounding with Double and long double Variables
| Autor |
Progress Software Corporation - Progress |
| Acesso |
Público |
| Publicação |
10/29/2008 |
|
Status: Verified
FACT(s) (Environment):
Actuate 4.x
SYMPTOM(s):
Reports in Actuate may experience decimal "rounding" problems.
Only for Double variables, long double variables and the Floating Point Control.
Value rounded down instead of being rounded up.
CAUSE:
The problem is caused by the way that Actuate interacts with some C runtime libraries.
If the variable is declared as double or long double, it will fail to round correctly. For some cases this is due to the limitation of C runtime libraries.
FIX:
The solution is to use the Currency format.
Currency variables are stored as 96-bit numbers (12 bytes) in a two's complement integer format, and are scaled by 1,000,000,000 to give a fixed point number with 20 digits to the left of the decimal point and 9 digits to the right.
This representation provides a range of :-
-39,614,081,257,132,168,796.771975167 to
+39,614,081,257,132,168,796.771975168.
The Currency data type is useful for calculations involving money and for fixed-point calculations in which accuracy is particularly important.