Kbase P92707: Decimal values passed to the AppServer from a .Net or Java front end have very small rounding errors
| Autor |
Progress Software Corporation - Progress |
| Acesso |
Público |
| Publicação |
12/23/2008 |
|
Status: Verified
SYMPTOM(s):
Decimal values passed to the AppServer from a .Net or Java front end have very small rounding errors
A decimal value in a temp-table that is passed to the AppServer and written to a database will not compare properly (i.e. comparing a literal value of .50 to a database field which appears to have a value of 0.50 will not result in the comparison being equal)
FACT(s) (Environment):
OpenEdge 10.x
All Supported Operating Systems
CAUSE:
Due to floating point rounding and conversion issues decimal values will always have small rounding errors when passed from a .Net or Java front end to the AppServer
FIX:
There are three possible workarounds for this issue. The workarounds are as follows:
1) Convert the decimal values to strings before passing to the AppServer.
2) Convert the decimal values to integers (using an implied decimal place) before passing to the AppServer.
3) Pass the values to the AppServer as decimals then on the AppServer side convert the decimal to a string (i.e. STRING(DecimalField,ProperFormat)) then convert the string back to a decimal value using the DECIMAL() function.