When 38.3 - 38 equals 0.299999999999997... and what to do about it Added by ~Hank Nonhipilygon on May 17, 2010 | Version 1
|
|
If you need the digits behind the decimal seperator of a number, you might subtract the integer of the number from the number itself: 1.3 - int(1.3) e.g. would result in 0.3. But there is a pitfall... try this with 38.3 - int(38.3), and LotusScript returns: 0.299999999999997. According to a colleague (yes, I asked ;-)), the reason is that some decimal numbers cannot be described exactly in binary format, or to be more exact, they are irrational in binary format. This leads to the inexact result above. To be on the safe side, you might have to do a string operation (ouch...): Dim nb as Double
nb = 38.3
Dim varTmp As Variant
varTmp = Split(Cstr(nb), ".") ' be aware of location dependend settings here ("," as decimal seperator)
dim afterDecTxt as String
If Ubound(varTmp) = 0 Then
afterDecTxt = ""
Else
afterDecTxt = varTmp(1)
End If
About the AuthorMarcus Foerster works
as Systems Architect for Pentos AG in Munich, Germany. His focus lies on
the application side, creating collaboration systems for users, groups
and enterprises to get their daily work done. This approach includes
integrating complex workflows with intuitive user interfaces, using
Lotus Notes/Domino with Adobe Flex and other Web technologies. Read
more in Marcus' blog: http://marcus.foerster.com/blog |