April 24, 2008 at 8:16 pm
Hi,
Recently i tried the two blocks of code.
select round(0.994,3), round(0.996,2)
resulted in the error below "An error occurred while executing batch. Error message is: Arithmetic Overflow."
later like the below
DECLARE @RoundedVar decimal(10,2)
set @RoundedVar = 0.994;
SELECT @RoundedVar
set @RoundedVar = 0.996;
SELECT round(@RoundedVar,2)
and got the results as 0.99 and 1.00
How the constants in the first block are treated intermediately and what went wrong with them. What made the second block to give the result.
Cany any one clarify my doubts on this.
Thanks and Rgds,
PL.Seenivasan
April 25, 2008 at 12:29 am
[font="Verdana"]
set @RoundedVar = 0.996;
SELECT round(@RoundedVar,2)
Even if you would have been tried this code, you will get the o/p which you did get earlier. That is 1.00
DECLARE @RoundedVar decimal(10,2)
set @RoundedVar = 0.996;
SELECT @RoundedVar
In your case, it checks the 3rd value, as you have mentioned 2 as scale. So it will skeep that much of, 2, digits from the right side of decimal point and will check whether the 3rd value is less than 5, equal to 5 or greater than 5. In you case, for the value 0.994, 3rd values is 4 and it is less than 5 so it returned 0.99. and in case of 0.996, 3rd values is 6 and greater than 5, so it has added 1 into 0.99, which is 0.99 + 1. so the o/p is 1.00
Hopes I have explained it well.
Mahesh[/font]
MH-09-AM-8694
Viewing 2 posts - 1 through 1 (of 1 total)
You must be logged in to reply to this topic. Login to reply