October 21, 2014 at 7:47 am
Hello,
why are there more decimal positions here
PRINT cast(111 as decimal(38,35)) / 23
--> 4.82608695652173913043478260869565217
then here
DECLARE @Zähler decimal(38,35)
, @Nenner decimal(38,35)
SET @Zähler = 111.0
SET @Nenner = 23.0
PRINT cast(@Zähler as decimal(38,35)) / @Nenner
--> 4.826086 :w00t:
Of course, in the upper part 23 is implicitly an integer, in the lower example it is declared as decimal. But what if I need to devide by 23.5? Why is deviding by an decimal reducing the results decimal positions?
Thanks in advance
Michael
October 21, 2014 at 9:26 am
Interesting. I dont know the answer, but it could be to do with having the values set in the variables, as being explicit with the decimals and cast gives the correct precision, which you would have though should give the same result as when using decimal variables:
PRINT CAST(111.0 AS DECIMAL(38,35))/ 23.0
= 4.8260869565217391304347826086956521
October 21, 2014 at 9:41 am
Good discussion of this weirdness found here: http://blogs.msdn.com/b/sqlprogrammability/archive/2006/03/29/564110.aspx
Basically, your first division results in an effective precision of (38,35) and the second one effectively (38,6)
Gerald Britton, Pluralsight courses
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply