sum function 9-10 decimal places? - InfoPath Dev
in

InfoPath Dev

sum function 9-10 decimal places?

Last post 08-01-2012 12:12 PM by ebonymb. 3 replies.
Page 1 of 1 (4 items)
Sort Posts: Previous Next
  • 04-02-2009 07:45 AM

    sum function 9-10 decimal places?

    i have a form that sum(amts) on a text box.  i tell it to force 2 decimal places and it does for maybe 90% of the forms submitted, but the other 10% it does 9-10 decimal places.

    for example it might add:

    $152.42 + $32.01.  that equals 184.43.  and when filling out the form, that text box shows that.  but when i submit the form it changes it to "184.42999999999998"

    I can't understand why though.


  • 04-02-2009 08:31 AM In reply to

    Re: sum function 9-10 decimal places?

    Floating point arithmetic in XPath is computed according to the IEEE 754 standard, which allocates a fixed number of bits for the computation.  I don't know a whole lot about floating point arithmetic, but I do know that it is not perfectly precise, and as you have experienced, tiny fractional parts can be lost or gained along the way.

    I think the following formula might do the trick to get the result to have just one or two decimal places all the time:

    round(sum(contents of sum) * 100) div 100

    Please give it a try.

    Jimmy Rishe / Software Developer / Microsoft MVP
    Qdabra Software
  • 04-02-2009 09:23 AM In reply to

    Re: sum function 9-10 decimal places?

    cool, i'll give this a try and keep an eye on it for a few days.  thanks for the quick reply and explanation on the calculations!

  • 08-01-2012 12:12 PM In reply to

    Re: sum function 9-10 decimal places?

    i had this exact same problem.  I used your suggestion and it resolved my issue temporarily only to occur again today.  My thought is I might be able to use the round(sum()*100) div 100 formula in a hidden field and display the result as text in a visible field.  Can that be done?

Page 1 of 1 (4 items)
Copyright © 2003-2019 Qdabra Software. All rights reserved.
View our Terms of Use.