A Fix() Function in T-SQL
-
Comments posted to this topic are about the content posted at http://www.sqlservercentral.com/columnists/rfarley/afixfunctionintsql.asp
Rob Farley
LobsterPot Solutions & Adelaide SQL Server User Group
Company: http://www.lobsterpot.com.au
Blog: http://blogs.lobsterpot.com.au -
Solution for constipated mathematicians, work it out with paper and pencil, if that doesn't work use LOGs
-
Nice function.
Tip 1: Instead of the CASE to get rid of negative values, use ABS.
Tip 2: Add a CASE to work around the domain error you'd get from LOG10(0).
Tip 3: Stop being lazy and use decimal instead of float. Because float can't represent many values exactly, you run the risk of errors. For instance:
SELECT dbo.fix(1.15) -- Returns 1.1 instead of 1.2
alter
function dbo.fix(@num numeric(36,18), @digits int) returns numeric(36,18) as
begin
declare @res float
select @res = case when @num = 0 then 0
else round(@num,@digits-1-floor(log10(abs(@num)))) end
return (@res)
end
Hugo Kornelis, SQL Server/Data Platform MVP (2006-2016)
Visit my SQL Server blog: https://sqlserverfast.com/blog/
SQL Server Execution Plan Reference: https://sqlserverfast.com/epr/ -
This article is quite a coincidence, I was just thinking this morning, how could I use log? Now if I could just remember how they worked, it's been 14 years since I've last used them.
-
Having completed IT + Maths degrees at university it is nice to see someone using some maths in the real world - I reckon 99/100 SQL developers would've used a varchar conversion with something to cater for the possible decimal point and minus signs and lived with the performance loss. Nice article and good thinking!
-
Very sweet!
But why the extra overhead of declaring and setting a variable? Why not just return the calc'd value?
...
BEGIN
RETURN CASE WHEN ...
END
SQL DBA,SQL Server MVP(07, 08, 09) "It's a dog-eat-dog world, and I'm wearing Milk-Bone underwear." "Norm", on "Cheers". Also from "Cheers", from "Carla": "You need to know 3 things about Tortelli men: Tortelli men draw women like flies; Tortelli men treat women like flies; Tortelli men's brains are in their flies".
-
Nice to see someone using LOGs. But I would trust your results more if you displayed them, instead of using the WHERE expected_result <> actual_result.
The problem I found was that you don't get exact representation when using float. Frankly, I have never used float before, so maybe I am doing something incorrectly, but note the example below.
declare @num float
, @res float
, @digits int
SET @num = 1.23456678
SET @digits = 4
select @res = case when @num > 0
then round(@num,@digits-1-floor(log10(@num)))
else round(@num,@digits-1-floor(log10(-@num)))
end
select @num as ORIGINAL_NUMBER, @res AS RESULT
, @digits as NBR_SIG_DIGITS
RESULT IS:
ORIGINAL_NUMBER RESULT NBR_SIG_DIGITS
1.23456678 1.2349999999999999 4
Probably not what you really want (I assume you want 1.235 ??)
-
Hi Skip,
>>The problem I found was that you don't get exact representation when using float<<
Of course you don't - check Books Online. Float is an approximate-number data type. For most numbers, it can only store a (close) approximization.
Internally, float uses base-2 representation. As a result, only numbers that can be written as x + (y / EXP(2, z)) where x and y are integers and z is a non-negative integer can be represented exactly in that representation. Just like our common base-10 representation can only represent an approximation of 1/3, float can only represent an approximation of 1.235.
That's one of the reasons why I wrote that the function should use decimal instead of float in my previous reply (though I now see that I forgot to change the datatype for the internal variable).
Hugo Kornelis, SQL Server/Data Platform MVP (2006-2016)
Visit my SQL Server blog: https://sqlserverfast.com/blog/
SQL Server Execution Plan Reference: https://sqlserverfast.com/epr/ -
True, I should've used numeric. I said in the article it was out of laziness.
I chose float originally because that's the type that the trigonometry functions use. Ideally I'd have it return the same type as is passed in (the way that the others work). But I knew numeric would be better... I was just being lazy.
As for using abs... yes - that would've been better. And I can't believe I didn't see that it doesn't work for 0. That's awful of me.
Thanks Hugo...
Rob
Rob Farley
LobsterPot Solutions & Adelaide SQL Server User Group
Company: http://www.lobsterpot.com.au
Blog: http://blogs.lobsterpot.com.au
Viewing 9 posts - 1 through 8 (of 8 total)
You must be logged in to reply to this topic. Login to reply