December 15, 2008 at 8:33 am
Hi,
When you try this code:
DECLARE @a BIGINT
SET @a = POWER(2,30)
PRINT @a
you get:
1073741824
When you try this code:
DECLARE @a BIGINT
SET @a = POWER(2,31)
PRINT @a
you get:
Msg 232, Level 16, State 3, Line 2
Arithmetic overflow error for type int, value = 2147483648.000000.
Why is this? I mean, @a should be able to hold all (positive) values up to 2^63 -1
Thanks,
Ray
December 15, 2008 at 8:45 am
I think you might be confused
a bigint takes this
SET @a = 9223372036854775807
PRINT @a
SET @a = POWER(2,31-1)
PRINT @a
AND NOT THIS
SET @a = POWER(2,31)-1
PRINT @a
----------------------------------------------
Try to learn something about everything and everything about something. - Thomas Henry Huxley
:w00t:
Posting Best Practices[/url]
Numbers / Tally Tables[/url]
December 15, 2008 at 8:47 am
actually maybe I am confused I will look further
----------------------------------------------
Try to learn something about everything and everything about something. - Thomas Henry Huxley
:w00t:
Posting Best Practices[/url]
Numbers / Tally Tables[/url]
December 15, 2008 at 8:53 am
yeah that was my next solution was to perhaps cast the vaules as it would prob return the same type as what is passed in
----------------------------------------------
Try to learn something about everything and everything about something. - Thomas Henry Huxley
:w00t:
Posting Best Practices[/url]
Numbers / Tally Tables[/url]
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply
This website stores cookies on your computer.
These cookies are used to improve your website experience and provide more personalized services to you, both on this website and through other media.
To find out more about the cookies we use, see our Privacy Policy