float problems (with front-end Java app.)
-
Hi there,
We are using the float datatype in MSSQL 2000 as the datatype for storing a variety of weights.
As far as we know this is our only option, as we are working in the front-end of our web application with Java, but we are getting some really weird things happening.. does anyone know what other options beyond float that we can use?
This might be the wrong place to ask the above questions, but we are also getting some weird displaying of values when verifying our test data..
I.E. ItemWeight comes back as: 5.3999999999999999E-2 - any ideas how to fix this weird output? It might be ultimately related to our front-end issues.. so any help is appreciated!
TIA.
Michelle
"Work like you don't need the money;
dance like no one is watching;
sing like no one is listening;
love like you've never been hurt;
and live every day as if it were your last."
~ an old Irish proverb -
Hi Michelle,
quote:
We are using the float datatype in MSSQL 2000 as the datatype for storing a variety of weights.As far as we know this is our only option, as we are working in the front-end of our web application with Java, but we are getting some really weird things happening.. does anyone know what other options beyond float that we can use?
This might be the wrong place to ask the above questions, but we are also getting some weird displaying of values when verifying our test data..
I.E. ItemWeight comes back as: 5.3999999999999999E-2 - any ideas how to fix this weird output? It might be ultimately related to our front-end issues.. so any help is appreciated!
that is the way float and real data types are designed to work. They are imprecise. Only a more or less good approximation! I guess they are relicts from the times when storage space was a huge problem.
Try to use the decimal data type, this is an exact one. Although I have no experience with Java, there should be something similar
This one might also help
http://www.sqlservercentral.com/forum/link.asp?TOPIC_ID=13161
Cheers,
Frank
--
Frank Kalis
Microsoft SQL Server MVP
Webmaster: http://www.insidesql.org/blogs
My blog: http://www.insidesql.org/blogs/frankkalis/[/url] -
Hi there Frank...
Thanks for your quick response!
quote:
that is the way float and real data types are designed to work. They are imprecise. Only a more or less good approximation! I guess they are relicts from the times when storage space was a huge problem.Well.. we knew that they were less precise but didn't think it would be an issue, as the precision needed is only to 2-3 decimal places.
However, our test data was inserted for that example as: 0.054, but when a quick SELECT * FROM tblName is run.. we got the "funky" output 5.3999999999999999E-2 🙂
Another insert into the same field, different value: 0.106, come out just as it went in.. go figure ;>
But granted, we were having problems in the original DB design on matching datatypes (Java again), so float seemed to be the "easy" answer. I guess it goes to show that easy doesn't usually mean right.
quote:
Try to use the decimal data type, this is an exact one. Although I have no experience with Java, there should be something similarWe will have to investigate this a bit to see what we have in Java that will work out & up match for us. I have emailed my team the links to this thread & to the link below you included and will let you know what we end up trying out!
quote:
This one might also helphttp://www.sqlservercentral.com/forum/link.asp?TOPIC_ID=13161
Cheers,
Frank
Thanks for the other link.. there might be some "trial & error" in our future, but a senior Java person we have access to might be able to shed some light on it.. although he knows MySQL more than SQL.
Cheers.
Michelle.
"Work like you don't need the money;
dance like no one is watching;
sing like no one is listening;
love like you've never been hurt;
and live every day as if it were your last."
~ an old Irish proverb -
Some fractions end as a repetitive binary fraction (mantissa/exponent stuff).
If my memory serve my right .1 an example.
-
Hi Michelle,
sorry, just came across your response.
quote:
However, our test data was inserted for that example as: 0.054, but when a quick SELECT * FROM tblName is run.. we got the "funky" output 5.3999999999999999E-2 🙂Another insert into the same field, different value: 0.106, come out just as it went in.. go figure ;>
you can't tell for sure you get the same results again. It's nondeterministic!
quote:
We will have to investigate this a bit to see what we have in Java that will work out & up match for us. I have emailed my team the links to this thread & to the link below you included and will let you know what we end up trying out!looking forward to hear what happens
quote:
Thanks for the other link.. there might be some "trial & error" in our future, but a senior Java person we have access to might be able to shed some light on it.. although he knows MySQL more than SQL.that makes no difference, because the data type 'decimal' is a standard numeric SQL 92 data type, and as such also implemented in MySQL
For further information see this one
http://www.mysql.com/doc/en/Numeric_types.html
HTH
Cheers,
Frank
--
Frank Kalis
Microsoft SQL Server MVP
Webmaster: http://www.insidesql.org/blogs
My blog: http://www.insidesql.org/blogs/frankkalis/[/url] -
Hi Michelle,
quote:
We will have to investigate this a bit to see what we have in Java that will work out & up match for us. I have emailed my team the links to this thread & to the link below you included and will let you know what we end up trying out!here's a tip from our web developers
http://java.sun.com/j2se/1.4.1/docs/guide/jdbc/getstart/mapping.html
Chapter 8.3.11 deals with decimal and numeric
Cheers,
Frank
--
Frank Kalis
Microsoft SQL Server MVP
Webmaster: http://www.insidesql.org/blogs
My blog: http://www.insidesql.org/blogs/frankkalis/[/url]
Viewing 6 posts - 1 through 5 (of 5 total)
You must be logged in to reply to this topic. Login to reply