Alaska Software Inc. - Problem with round
Username: Password:
AuthorTopic: Problem with round
Adrian WykrotaProblem with round
on Fri, 26 Jun 2015 11:49:02 +0200
Version 1.90.355

PROCEDURE Main
LOCAL uVal := 0
PRIVATE cFormula := "(14.086600-14.286100)*1000"
? "Formula : " + cFormula
uVal := &cFormula
? "Value :" + var2char(uVal)
uVal := ROUND(uVal, 0)
? "Round :" + var2char(uVal)
INKEY(0)
RETURN

Any workaround this ?
Andreas Gehrs-Pahl
Re: Problem with round
on Sat, 27 Jun 2015 03:52:02 -0400
Adrian,

If you use str() instead of Var2Char() and specify enough decimals, you will 
see that the display value of -199.5000 is already rounded, as the actual 
result of the calculation is something closer to this: -199.4999999999987, 
and the correct result of Round(-199.4999999999987, 0) is just -199.0!

? "Value :" + str(uVal, 20, 15) ==> -199.499999999998700

>Any workaround this ?

As a workaround, you could do simple two-step rounding, like this:

Round(Round(uVal, 6), 0) -- or more logically:

uVal := Round(&cFormula, 6) ==> -199.500000
uVal := Round(uVal,      0) ==> -200.00

Of course the value of 6 (decimals) for rounding the result is just an 
arbitrary value suggested by me, as you use operands with six decimals, and 
you could as well use 5, 2, or even 1 decimal (in this particular case). 
But using the same precision as the operands have is always a good starting 
point.

This issue is caused by the way decimal floating point numbers are 
represented and manipulated in binary format, using the IEEE 754 standard. 
And as Xbase++ doesn't have BCD (Binary-Coded Decimal) math routines, the 
limitation of the IEEE 754 standard will be what you have to live with.

Andreas

Andreas Gehrs-Pahl
Absolute Software, LLC

phone: (989) 723-9927
email: Andreas.GP@Charter.net
web:   http://www.Aerospace-History.net
Adrian WykrotaRe: Problem with round
on Mon, 29 Jun 2015 14:04:01 +0200
I will try.Thank you.



Użytkownik "Andreas Gehrs-Pahl" napisał w wiadomości grup 
dyskusyjnych:1qtzzb5wraagp$.cjkaszrgu42o.dlg@40tude.net...

Adrian,

If you use str() instead of Var2Char() and specify enough decimals, you will
see that the display value of -199.5000 is already rounded, as the actual
result of the calculation is something closer to this: -199.4999999999987,
and the correct result of Round(-199.4999999999987, 0) is just -199.0!

? "Value :" + str(uVal, 20, 15) ==> -199.499999999998700

>Any workaround this ?

As a workaround, you could do simple two-step rounding, like this:

Round(Round(uVal, 6), 0) -- or more logically:

uVal := Round(&cFormula, 6) ==> -199.500000
uVal := Round(uVal,      0) ==> -200.00

Of course the value of 6 (decimals) for rounding the result is just an
arbitrary value suggested by me, as you use operands with six decimals, and
you could as well use 5, 2, or even 1 decimal (in this particular case).
But using the same precision as the operands have is always a good starting
point.

This issue is caused by the way decimal floating point numbers are
represented and manipulated in binary format, using the IEEE 754 standard.
And as Xbase++ doesn't have BCD (Binary-Coded Decimal) math routines, the
limitation of the IEEE 754 standard will be what you have to live with.

Andreas

Andreas Gehrs-Pahl
Absolute Software, LLC

phone: (989) 723-9927
email: Andreas.GP@Charter.net
web:   http://www.Aerospace-History.net
Adrian WykrotaRe: Problem with round
on Mon, 29 Jun 2015 14:07:12 +0200
It's working

PROCEDURE Main
LOCAL uVal := 0
PRIVATE cFormula := "(14.086600-14.286100)*1000"
? "Formula : " + cFormula
uVal := &cFormula
? "Value :" + var2char(uVal)
uVal := ROUND(uVal, 6)
? "Round #1 :" + var2char(uVal)
uVal := ROUND(uVal, 0)
? "Round #2 :" + var2char(uVal)
INKEY(0)
RETURN


Użytkownik "Andreas Gehrs-Pahl" napisał w wiadomości grup 
dyskusyjnych:1qtzzb5wraagp$.cjkaszrgu42o.dlg@40tude.net...

Adrian,

If you use str() instead of Var2Char() and specify enough decimals, you will
see that the display value of -199.5000 is already rounded, as the actual
result of the calculation is something closer to this: -199.4999999999987,
and the correct result of Round(-199.4999999999987, 0) is just -199.0!

? "Value :" + str(uVal, 20, 15) ==> -199.499999999998700

>Any workaround this ?

As a workaround, you could do simple two-step rounding, like this:

Round(Round(uVal, 6), 0) -- or more logically:

uVal := Round(&cFormula, 6) ==> -199.500000
uVal := Round(uVal,      0) ==> -200.00

Of course the value of 6 (decimals) for rounding the result is just an
arbitrary value suggested by me, as you use operands with six decimals, and
you could as well use 5, 2, or even 1 decimal (in this particular case).
But using the same precision as the operands have is always a good starting
point.

This issue is caused by the way decimal floating point numbers are
represented and manipulated in binary format, using the IEEE 754 standard.
And as Xbase++ doesn't have BCD (Binary-Coded Decimal) math routines, the
limitation of the IEEE 754 standard will be what you have to live with.

Andreas

Andreas Gehrs-Pahl
Absolute Software, LLC

phone: (989) 723-9927
email: Andreas.GP@Charter.net
web:   http://www.Aerospace-History.net