Most of us decided to use (ASP) VBScript to quickly write a script or build a simple website, writing code in a ‘higher-level’ has the advantage to keep our mind on the functionality of the script or website and forget about the low-level technical stuff.
Unfortunately, we have to worry about the following issue (example is using Classis ASP/VBScript)
Response.Write CDbl(112.51)<>CDbl(94.55+17.96)
'Answer: TRUE -> WRONG!!
Response.Write CCur(112.51)<>CCur(94.55+17.96)
'Answer: False
The correct answer is FALSE.
The problem is in fact that a CDbl (Convert to Double) will fill 64bits with ones and zeros and it will not be 112.51 but 112.5099999999999999999 (or something similiar).
Another example:
Response.Write CSng(0.1)=CDbl(0.1)
'Answer: FALSE -> WRONG!!
Explanation
0.1 as Single (in Binary) is: 0.00011001100110011001100.
0.1 as Double (in Binary) is: 0.000110011001100110011001100110011001100110011001100110.
Also see a related MSDN Blog article.
(Note: This was tested on my Intel Xeon X64 with Windows 2003 Server X64, running IIS6 in 32bit mode)
One Comment