Post reply

Need Help

  • January 20, 2011 at 5:17 am

    #243985

    Hi,

    I have a strange problem while writing one SP. Could you please help me to find out why the results are not same for the below queries and let me know the solution to get the same results for both the queries.

    DECLARE @Q1 NUMERIC(28,8) = 3110.49650652;

    DECLARE @Q2 NUMERIC(28,8) = 3110.48997448;

    --Query 1

    SELECT CONVERT(NVARCHAR, (3110.49650652 / (3110.49650652 + 3110.48997448) * 100.0))

    --Query2

    SELECT CONVERT(NVARCHAR, (@Q1 / (@Q1 + @Q2) * 100.0))

    Result1: 50.0000525000337161157000

    Result2: 50.000053

    second result is getting round off for 6 decimals. But the values in the variables are same as first query.

    Thanks,

    Ravi.

  • January 20, 2011 at 7:18 am

    #1275911

    That is probably because the numbers you typed are using another data type behind the scenes (I guess float?) than the variables, which are of the numeric data type.

    When using the numeric data type in a calculation (such as division), the precision of the resulting number is the result of a formula.

    Paul explains it rather well in this topic:

    http://www.sqlservercentral.com/Forums/Topic870098-338-1.aspx

    Need an answer? No, you need a question
    My blog at https://sqlkover.com.
    MCSE Business Intelligence - Microsoft Data Platform MVP

Viewing 2 posts - 1 through 1 (of 1 total)

You must be logged in to reply to this topic. Login to reply