Rcently, I was asked to pull some statistics from a SQL database. Only minimal information was available on the data itself, and the task required determining percentages and creating trends from the data. It was relatively easy to pull out the data and perform calculations, but the results looked funny when I had finished. So I dug further in.

Some of the numbers were integers from counts of row occurrences. A simple Example would be when I had to find the answer to this: "How many times did customer A order during a year, and what percent of the companies total orders comprised customer A.?" Giving some numbers to this example, if Customer A had 16 orders out of 140 total orders for all customers, then to find the percentage of Customer A's orders it would be (16/140)*100=11.43%. If we promised to give all salespersons who had customers ordering at least 10% of our total business a special bonus, this salesperson would get zip with my initial calculations. That's because in SQL Server, select (16/140)*100”=0.

However since our actual equation was much more complex, we did not see that SQL returned 0 after truncating some of our calculations. The calculations using integers, decimal, etc. do not always return the expected result. This is especially true with complex calculations or functions where the error can remain unseen.

Try the following example in query analyzer:

select 6/9
select (6 + .0)/9
select (6 - .0)/9
select (6 * 1.0)/9

The first equation result is truncated to the nearest integer, which is 0. The last three equations return the correct amount. You can see that if this calculation is part of a large complex equation you can produce major errors. And you might not easily find them.

The second example even has a float declaration and a decimal but gives two different results. Try this second example in query analyzer:

DECLARE @angle1 float
SET @angle1 = (.222456 + 6/9)
SELECT 'The SINE of the angle1 is: ' + CONVERT(varchar,SIN(@angle1))
GO
DECLARE @angle2 float
SET @angle2 = (.222456 + (6 + .0)/9)
SELECT 'The SINE of the angle2 is: ' + CONVERT(varchar,SIN(@angle2))
GO

When the equations are complex it is very easy to miss this conversion error. There are notations in SQL arithmetic stating that conversion to decimal may not occur.

Great! We could convert all of our table data to decimals and all numbers to decimal.

The Easier Way

Have a good idea what of type is your data: integer, decimal, float....whatever. And most importantly, make sure your data is returning the results you expect. It was very easy to add or subtract (.0) or multiply by (1.0) to the equation at the point of error to force a decimal, which is easier than changing all the data.

6/9=0 is ok only when you are truncating to the nearest integer. But never assume the result of an equation is correct without testing it for truncation or rounding results that could cause errors.

Roy Carlson

Illinois Student Assistance Commission