December 15, 2008 at 8:33 am
Hi,
When you try this code:
DECLARE @a BIGINT
SET @a = POWER(2,30)
PRINT @a
you get:
1073741824
When you try this code:
DECLARE @a BIGINT
SET @a = POWER(2,31)
PRINT @a
you get:
Msg 232, Level 16, State 3, Line 2
Arithmetic overflow error for type int, value = 2147483648.000000.
Why is this? I mean, @a should be able to hold all (positive) values up to 2^63 -1
Thanks,
Ray
December 15, 2008 at 8:45 am
I think you might be confused
a bigint takes this
SET @a = 9223372036854775807
PRINT @a
SET @a = POWER(2,31-1)
PRINT @a
AND NOT THIS
SET @a = POWER(2,31)-1
PRINT @a
----------------------------------------------
Try to learn something about everything and everything about something. - Thomas Henry Huxley
:w00t:
Posting Best Practices[/url]
Numbers / Tally Tables[/url]
December 15, 2008 at 8:47 am
actually maybe I am confused I will look further
----------------------------------------------
Try to learn something about everything and everything about something. - Thomas Henry Huxley
:w00t:
Posting Best Practices[/url]
Numbers / Tally Tables[/url]
December 15, 2008 at 8:53 am
yeah that was my next solution was to perhaps cast the vaules as it would prob return the same type as what is passed in 🙂
----------------------------------------------
Try to learn something about everything and everything about something. - Thomas Henry Huxley
:w00t:
Posting Best Practices[/url]
Numbers / Tally Tables[/url]
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply