January 25, 2011 at 8:10 am
hi all,
I try to give a expression to round some filed like below,
Round(Fields!unit_price.Value, Fields!Dom_Price_Places.Value)
---- Fields!unit_price.Value(decimal(18,8))
----Fields!Dom_Price_Places.Value(tinyint)
now I run into a one Fields!unit_price.Value=2.34500000, Fields!Dom_Price_Places.Value=2
What I get is
Round(CDec("2.34500000"), 2) return 2.34.
Round(2.34500000, 2) return 2.35
What's the difference here?
Is this a bug?
And help would be appreciated. Thanks,
January 25, 2011 at 6:43 pm
Any Help? thanks
January 25, 2011 at 6:54 pm
ada.xi (1/25/2011)
hi all,I try to give a expression to round some filed like below,
Round(Fields!unit_price.Value, Fields!Dom_Price_Places.Value)
---- Fields!unit_price.Value(decimal(18,8))
----Fields!Dom_Price_Places.Value(tinyint)
now I run into a one Fields!unit_price.Value=2.34500000, Fields!Dom_Price_Places.Value=2
What I get is
Round(CDec("2.34500000"), 2) return 2.34.
Round(2.34500000, 2) return 2.35
What's the difference here?
Is this a bug?
And help would be appreciated. Thanks,
What is CDec? I can't find it in BOL anywhere.
My guess, CDec is converting "2.34500000" to 2.34, not 2.345.
January 25, 2011 at 7:08 pm
Cdec ---Convert to Decimal.
And (CDec("2.34500000")=2.34500000) return true.
January 25, 2011 at 7:10 pm
ada.xi (1/25/2011)
Cdec ---Convert to Decimal.
In what? As I said, I can't find it anywhere in Books Online.
January 25, 2011 at 7:23 pm
Lynn Pettis (1/25/2011)
ada.xi (1/25/2011)
Cdec ---Convert to Decimal.In what? As I said, I can't find it anywhere in Books Online.
Nevermind, I found it in the Visual Studio Help System, it is a VB function.
January 27, 2011 at 11:08 am
You say that
Round(CDec("2.34500000"), 2) return 2.34.
but it should be
Round(2.34500000, 2) return 2.35
What happens when the number is 2.35500000 does it return
2.35 or 2.36?
If it returns 2.36 you are encountering something called banker's rounding where a 5 will round up to an even number, but a 5 will round down if the round up number is odd. In other words the 5 will alway round in the direction of the even number whether that is up or down.
Without the long explanation of why banker's rounding exists, most people just code around it by testing the digit they are rounding and if it is a 5 they will add a value of 1 to that digit to make sure it rounds up.
Hope this helps.
January 31, 2011 at 8:32 am
Daniel Bowlin (1/27/2011)
If it returns 2.36 you are encountering something called banker's rounding where a 5 will round up to an even number, but a 5 will round down if the round up number is odd. In other words the 5 will alway round in the direction of the even number whether that is up or down.
I think the use of CDec is invoking the bankers rounding somehow. Try dropping that function if you can; maybe put the rounding in the query if that's possible.
For what it's worth, here's something more from MSDN:
When the fractional part is exactly 0.5, CInt and CLng always round it to the nearest even number. For example, 0.5 rounds to 0 and 1.5 rounds to 2. CInt and CLng differ from the Fix and Int functions, which truncate, rather than round, the fractional part of a number. Also, Fix and Int always return a value of the same type as is passed in.
Though it doesn't include CDec in that list, I wouldn't be surprised if it behaves like CInt and CLng.
Viewing 8 posts - 1 through 7 (of 7 total)
You must be logged in to reply to this topic. Login to reply