March 23, 2015 at 7:42 am
For whatever reason i'm unable to cast anything more thtan e-4 to a float which makes no sence. Am i missing something?
select cast( '1.550e-6' as float)
?????????
returns 1.55E-06
????????
select cast( '1.550e-5' as float)
?????????
returns 1.55E-05
????????
select cast( '1.550e-4' as float)
returns 0.000155
select cast( '1.550e10' as float)
returns 15500000000
March 23, 2015 at 8:10 am
Snargables (3/23/2015)
For whatever reason i'm unable to cast anything more thtan e-4 to a float which makes no sence. Am i missing something?select cast( '1.550e-6' as float)
?????????
returns 1.55E-06
????????
select cast( '1.550e-5' as float)
?????????
returns 1.55E-05
????????
select cast( '1.550e-4' as float)
returns 0.000155
select cast( '1.550e10' as float)
returns 15500000000
Looks fine to me. What's the problem?
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin
March 23, 2015 at 8:11 am
select cast( '1.550e-6' as float)
?????????
returns 1.55E-06
????????
shouldnt it return .00000155 ?
March 23, 2015 at 8:18 am
it's not actuallt converting anything when it anything over e-5. or under however u want to look at it.
March 23, 2015 at 8:19 am
I'm not sure why it is displayed in exponential notation, but it is still a float and appears to have the right value.
If you use decimal instead, I think that it will be displayed as you want.
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin
March 23, 2015 at 8:20 am
Snargables (3/23/2015)
it's not actuallt converting anything when it anything over e-5. or under however u want to look at it.
It is converting from varchar to float.
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin
March 23, 2015 at 8:22 am
Snargables (3/23/2015)
it's not actuallt converting anything when it anything over e-5. or under however u want to look at it.
Try these
select cast( '1.550e-6' as float) * 100000
select '1.550e-6' * 1000
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin
March 23, 2015 at 9:07 am
I'm sure i could multiply it by something to get it to work however that doesnt explain why it will convert 1.35e6, 1.35e5, 1.35e4, 1.35e3, 1.35e2, 1.35e1, 1.35e-1, 1.35e-2, 1.35e-3, and 1.35e-4 properly however will not do any conversion when its 1.35e-5, 1.35e-6, 1.35e-7 etc. Unless i'm missing something which i'm sure i am it appears to be a bug of sorts.
when i do the cast for 1.35e-5 it equals 1.35e-5
when i do the cast for 1.35e-4 it equlas .000135
i need it to evaluate 1.35e-5 to .0000135 and the cast('1.35e-5' as float) aint cuttin it hwever it works for cast('1.35e-4' as float) and so on.
The only reason i'm aware of this is because we have a process that consumes some 3rd part data and out of 8 million recordfs over time there are three of them that were e-5 and because it wasnt converting them to .00005 it's breaking the reports.
March 23, 2015 at 9:37 am
Snargables (3/23/2015)
I'm sure i could multiply it by something to get it to work however that doesnt explain why it will convert 1.35e6, 1.35e5, 1.35e4, 1.35e3, 1.35e2, 1.35e1, 1.35e-1, 1.35e-2, 1.35e-3, and 1.35e-4 properly however will not do any conversion when its 1.35e-5, 1.35e-6, 1.35e-7 etc. Unless i'm missing something which i'm sure i am it appears to be a bug of sorts.when i do the cast for 1.35e-5 it equals 1.35e-5
when i do the cast for 1.35e-4 it equlas .000135
i need it to evaluate 1.35e-5 to .0000135 and the cast('1.35e-5' as float) aint cuttin it hwever it works for cast('1.35e-4' as float) and so on.
The only reason i'm aware of this is because we have a process that consumes some 3rd part data and out of 8 million recordfs over time there are three of them that were e-5 and because it wasnt converting them to .00005 it's breaking the reports.
You seem to be confusing the words 'data conversion' with 'display format'. The conversion happens properly, but the converted value is not displayed the way you want. As I suggested, you can cast to decimal(p,n) to change that behaviour.
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin
March 23, 2015 at 11:08 am
i got u now.
select cast(cast('1.55E-05' as float)as decimal(25,20))
Viewing 10 posts - 1 through 9 (of 9 total)
You must be logged in to reply to this topic. Login to reply