Post reply

float values - arithmetic overflow. Help!

  • January 10, 2012 at 8:39 am

    #391276

    Hi all,

    The following two values represent values found in variables that are meant to be datatype float. I've pasted the actual values here in this select.

    I am receiving the following error:

    select (5.80978185621035E+306 * (1 + 12035932061.6487))

    produces Arithmetic overflow error converting expression to data type float.

    I’ve tried a few things below to overcome, but with no success. Do you have any ideas on how to allow for the value to be displayed (and stored in a variable datatype float)?

    Thanks in advance,

    Sharon

    select (CONVERT(FLOAT(53),5.80978185621035E+306) * (1 + CONVERT(FLOAT(53),12035932061.6487)))

    select CONVERT(FLOAT,(CAST(5.80978185621035E+306 AS FLOAT) * (1 + CAST(12035932061.6487 AS FLOAT))))

    select (CAST(5.80978185621035E+306 AS FLOAT) * (1 + CAST(12035932061.6487 AS DEC(38,8))))

  • January 10, 2012 at 8:53 am

    #1431229

    Float only goes up to 1.79E+308 IIRC.

    What are you doing to need to go higher than that?

    Forever trying to learn
    My blog - http://www.cadavre.co.uk/
    For better, quicker answers on T-SQL questions, click on the following...http://www.sqlservercentral.com/articles/Best+Practices/61537/
    For better, quicker answers on SQL Server performance related questions, click on the following...http://www.sqlservercentral.com/articles/SQLServerCentral/66909/

  • January 10, 2012 at 8:57 am

    #1431234

    Thanks, Cadavre, for your response.

    I'm not sure what 1.79E+308 represents or means. Which one of the two exceeded this value (and how)?

    To answer your question, there are large sums of very large numbers that need to be calc'd in the manner you see in the example. They are aggregates of financial data...

  • January 10, 2012 at 9:13 am

    #1431246

    Sha_ (1/10/2012)

    Thanks, Cadavre, for your response.

    I'm not sure what 1.79E+308 represents or means. Which one of the two exceeded this value (and how)?

    To answer your question, there are large sums of very large numbers that need to be calc'd in the manner you see in the example. They are aggregates of financial data...

    Umm, well, the sum you showed: -

    (5.80978185621035E+306 * (1 + 12035932061.6487))

    Is greater than the maximum value allowed for float datatype (which I think it 1.79E+308).

    Forever trying to learn
    My blog - http://www.cadavre.co.uk/
    For better, quicker answers on T-SQL questions, click on the following...http://www.sqlservercentral.com/articles/Best+Practices/61537/
    For better, quicker answers on SQL Server performance related questions, click on the following...http://www.sqlservercentral.com/articles/SQLServerCentral/66909/

  • January 10, 2012 at 9:22 am

    #1431254

    The calc is not the question. 1.79E+308 is. I don't know what 1.79E+308 is or how to interpret it.

  • January 10, 2012 at 9:33 am

    #1431262

    Sha_ (1/10/2012)

    The calc is not the question. 1.79E+308 is. I don't know what 1.79E+308 is or how to interpret it.

    I guess I don't understand why you're having difficulty in understanding. Your original sum has the number "5.80978185621035E+306" in it, so if you understand that then why don't you understand 1.79E+308 ?

    1.79E+308 is 1.79*10^(308)

    5.80978185621035E+306 is 5.80978185621035*10^(306)

    What I'm saying is that your sum, exceeds the maximum size of the float datatype.

    Forever trying to learn
    My blog - http://www.cadavre.co.uk/
    For better, quicker answers on T-SQL questions, click on the following...http://www.sqlservercentral.com/articles/Best+Practices/61537/
    For better, quicker answers on SQL Server performance related questions, click on the following...http://www.sqlservercentral.com/articles/SQLServerCentral/66909/

  • January 10, 2012 at 10:13 am

    #1431293

    Thank you. This explains it: 1.79E+308 is 1.79*10^(308)

    I wasn't aware of that. Sorry, I'm not a math guy 🙂

    Thank you again.

Viewing 7 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply