November 26, 2009 at 10:01 pm
please check this ,
DECLARE @srtWord VARCHAR(8000)
SELECT
@srtWord = '1'+SPACE(7998)+'2'
select @srtWord
select @srtWord = REPLACE(REPLACE(REPLACE(CAST(@srtWord AS VARCHAR(MAX)),' ','$#$'),'$$#',''),'$#$',' ')
select @srtWord
Regards,
Mitesh OSwal
+918698619998
November 27, 2009 at 12:52 am
Mitesh, it was mentioned earlier that using varchar(max) solves the problem of data loss.
November 27, 2009 at 10:39 am
Mitesh Oswal (11/26/2009)
please check this ,DECLARE @srtWord VARCHAR(8000)
SELECT
@srtWord = '1'+SPACE(7998)+'2'
select @srtWord
select @srtWord = REPLACE(REPLACE(REPLACE(CAST(@srtWord AS VARCHAR(MAX)),' ','$#$'),'$$#',''),'$#$',' ')
select @srtWord
Yes, Mitesh. It does the job correctly. But, as we've already seen in this thread, there are two things that make it very, very slow...
1. Expansion of the data... each space is converted to 3 characters.
2. Use of VARCHAR(MAX)... the mere use of this datatype slows things down substantially.
Check it out...
--===== Create and populate a test table
IF OBJECT_ID('TempDB..#TestTable','U') IS NOT NULL
DROP TABLE #TestTable
SELECT TOP 1000
CAST('1'+SPACE(7998)+'2' AS VARCHAR(8000)) AS SrtWord
INTO #TestTable
FROM Master.sys.All_Columns ac1
CROSS JOIN Master.sys.All_Columns ac2
PRINT REPLICATE('=',100)
PRINT '========== Mitesh''s method =========='
SET STATISTICS TIME ON
SELECT REPLACE(REPLACE(REPLACE(CAST(SrtWord AS VARCHAR(MAX)),' ','$#$'),'$$#',''),'$#$',' ')
FROM #TestTable
WHERE CHARINDEX(' ',SrtWord) > 0
SET STATISTICS TIME OFF
PRINT REPLICATE('=',100)
PRINT '========== Method from the article =========='
SET STATISTICS TIME ON
SELECT REPLACE(REPLACE(REPLACE(SrtWord,' ',' '+CHAR(7)),CHAR(7)+' ',''),CHAR(7),'')
FROM #TestTable
WHERE CHARINDEX(' ',SrtWord) > 0
SET STATISTICS TIME OFF
PRINT REPLICATE('=',100)
PRINT '========== Michael''s method =========='
SET STATISTICS TIME ON
SELECT REPLACE(
REPLACE(
REPLACE(
REPLACE(
REPLACE(
REPLACE(
REPLACE(
SrtWord
,' ',' ') --32+1 Spaces
,' ',' ') --16+1 Spaces
,' ',' ') -- 8+1 Spaces
,' ',' ') -- 4+1 Spaces
,' ',' ') -- 2+1 Spaces
,' ',' ') -- 1+1 Spaces
,' ',' ') -- 1+1 Spaces
FROM #TestTable
WHERE CHARINDEX(' ',SrtWord) > 0
SET STATISTICS TIME OFF
Results...
(1000 row(s) affected)
====================================================================================================
========== Mitesh's method ==========
(1000 row(s) affected)
SQL Server Execution Times:
CPU time = 49047 ms, elapsed time = 53405 ms.
====================================================================================================
========== Method from the article ==========
(1000 row(s) affected)
SQL Server Execution Times:
CPU time = 2328 ms, elapsed time = 2574 ms.
====================================================================================================
========== Michael's method ==========
(1000 row(s) affected)
SQL Server Execution Times:
CPU time = 297 ms, elapsed time = 355 ms.
--Jeff Moden
Change is inevitable... Change for the better is not.
November 27, 2009 at 4:07 pm
Must be some hardware having influence on this, or is this being affected by collation..
Pro Duo, 2gb ram, 700MB free, SQL 2005
(1000 row(s) affected)
====================================================================================================
========== Mitesh's method ==========
(1000 row(s) affected)
SQL Server Execution Times:
CPU time = 47940 ms, elapsed time = 37283 ms.
====================================================================================================
========== Method from the article ==========
(1000 row(s) affected)
SQL Server Execution Times:
CPU time = 60107 ms, elapsed time = 48444 ms.
====================================================================================================
========== Michael's method ==========
(1000 row(s) affected)
SQL Server Execution Times:
CPU time = 4586 ms, elapsed time = 3263 ms
C# Gnu
____________________________________________________
November 27, 2009 at 11:30 pm
Wow, over 300 posts. Great Job, Jeff and thanks for the citation! 🙂
[font="Times New Roman"]-- RBarryYoung[/font], [font="Times New Roman"] (302)375-0451[/font] blog: MovingSQL.com, Twitter: @RBarryYoung[font="Arial Black"]
Proactive Performance Solutions, Inc. [/font][font="Verdana"] "Performance is our middle name."[/font]
November 28, 2009 at 11:37 am
It has been fun to say the least, Barry. And, I'll always refer to you whenever I hear the term "Pseudo-Cursor" because you're the first one I heard it from. It's incredibly descriptive so I've been using it wherever I used to say "set based loop" which didn't make a lot of sense to many people. I site the articles you wrote on things like this because they pretty much say it all, as well. 😛
--Jeff Moden
Change is inevitable... Change for the better is not.
November 28, 2009 at 11:51 am
C# Screw (11/25/2009)
Hello Jeffwould you think the HasSpace is finally getting rid of the RBAR? - I am thinking this because this is the only solution where SQL can use an index to eliminate a set of rows without 'touching them'?
I don't believe so... you can get a similar problem doing INDEX SEEKs by seeking the same info over and over as it would in a triangular join.
I lost the test due to an unexpected reboot (rookie mistake on my part), but I didn't find a speed advantage nor a read advantage to using a calculated column like HasSpace over just using a good WHERE clause. If I have the time, I'll rebuild the test to show what I mean.
It has been very interesting reading your artical on using Tally instead of while loops - I will try and think of solutions using Tally in future.
Thanks for the feedback on that. I've got one code boo-boo at the end that I need to fix. Hope to get to that soon, as well.
And one day I also hope to understand the Pork Chop thing as well!:-)
There was a forum post where some manager was whining about not being able to control one of his developers. Basically, the developer was riding rough-shod over him no matter what he said or did including refusal to follow company standards, etc, etc. I gave him some suggestions and he kept making excuses as to why not to use those suggestions. After about 4 more posts of his whining, I suggested he take the developer out to a nice pork chop dinner to get his attention... tie him to the chair and hand feed him the pork chops.... with a slingshot... at point blank range.
It's become a bit of a metaphor for some on this forum for "telling someone like it is" in a somewhat brutally direct and less than tactful fashion. Of course, there are two ways to sling such high velocity pork and folks normally try to take the high road using code examples. That's a good thing.
--Jeff Moden
Change is inevitable... Change for the better is not.
November 28, 2009 at 5:01 pm
Thanks for comments Jeff..
I am having a play with trying to write a solution using Jeff's Tally technique - rather than looping.
I have managed the following - but this is just cleaning up a variable :
--
-- I have tried to create a solution learning Jeff's Tally technique instead of looping
--
DECLARE @MyString VARCHAR(8000), @NewString VARCHAR(8000)
SET @MyString = '1' + SPACE(1) + '2' + SPACE(2) + '3' + SPACE(3) + '4' + SPACE(4)
SET @NewString = ''
-- So far I have things working to remove double spaces from a variable @MyString
--
SELECT @NewString = @NewString + SUBSTRING(@MyString,N,1)
FROM dbo.Tally t
WHERE t.N <= DATALENGTH(@MyString)
--Only append this character if this character and the next character are not both spaces
AND (SUBSTRING(@MyString,N,1)+ ISNULL(SUBSTRING(@MyString,N+1,1),'')) <> SPACE(2)
SELECT @NewString
I am wondering how to then progress the code so it could be used to update a column rather than a variable.
Just a for fun exercise / challenge !
C# Gnu
____________________________________________________
November 28, 2009 at 5:23 pm
I should add I wanted to try and do it without creating a UDF !
(Just in case any pork lovers listening ... :-))
C# Gnu
____________________________________________________
November 28, 2009 at 5:58 pm
Havn't achieved in-line SQL yet but performance as Tally-UDF is not too bad!
[using modified Jeff's script - hope that is ok]
====================================================================================================
========== Method from the article ==========
SQL Server Execution Times:
CPU time = 48517 ms, elapsed time = 48426 ms.
====================================================================================================
========== Michael's method ==========
SQL Server Execution Times:
CPU time = 3385 ms, elapsed time = 3270 ms.
====================================================================================================
========== Tally UDF method ==========
SQL Server Execution Times:
CPU time = 9126 ms, elapsed time = 9438 ms.
====================================================================================================
========== Looping UDF method ==========
SQL Server Execution Times:
CPU time = 17846 ms, elapsed time = 17481 ms.
Pretty much half the SQL looping time ..
SET NOCOUNT ON
go
--===== Create a UDF using Tally Technique
CREATE FUNCTION dbo.CleanUsingTally(@MyString VarChar(8000))
RETURNS VarChar(8000)
AS
BEGIN
-- So far I have things working to remove double spaces from a variable @MyString
--
DECLARE @NewString VARCHAR(8000)
SET @NewString = ''
SELECT @NewString = @NewString + SUBSTRING(@MyString,N,1)
FROM dbo.Tally t
WHERE t.N <= DATALENGTH(@MyString)
--Only append this character if this character and the next character are not both spaces
AND (SUBSTRING(@MyString,N,1)+ ISNULL(SUBSTRING(@MyString,N+1,1),'')) <> ' '
RETURN @NewString
END
GO
--===== Create a UDF Looping Technique
CREATE FUNCTION dbo.fn_CleanUpUsingLooping(@S VARCHAR(max))
RETURNS VARCHAR(max)
BEGIN
WHILE CHARINDEX(' ',@S) > 0
SELECT @s-2 = REPLACE(@S,' ',' ')
RETURN @s-2
END
go
--===== Create and populate a test table
IF OBJECT_ID('TempDB..#TestTable','U') IS NOT NULL
DROP TABLE #TestTable
SELECT TOP 1000
CAST('1'+SPACE(7998)+'2' AS VARCHAR(8000)) AS SrtWord
INTO #TestTable
FROM Master.sys.All_Columns ac1
CROSS JOIN Master.sys.All_Columns ac2
PRINT REPLICATE('=',100)
PRINT '========== Mitesh''s method =========='
SET STATISTICS TIME ON
SELECT REPLACE(REPLACE(REPLACE(CAST(SrtWord AS VARCHAR(MAX)),' ','$#$'),'$$#',''),'$#$',' ')
FROM #TestTable
WHERE CHARINDEX(' ',SrtWord) > 0
SET STATISTICS TIME OFF
PRINT REPLICATE('=',100)
PRINT '========== Method from the article =========='
SET STATISTICS TIME ON
SELECT REPLACE(REPLACE(REPLACE(SrtWord,' ',' '+CHAR(7)),CHAR(7)+' ',''),CHAR(7),'')
FROM #TestTable
WHERE CHARINDEX(' ',SrtWord) > 0
SET STATISTICS TIME OFF
PRINT REPLICATE('=',100)
PRINT '========== Michael''s method =========='
SET STATISTICS TIME ON
SELECT REPLACE(
REPLACE(
REPLACE(
REPLACE(
REPLACE(
REPLACE(
REPLACE(
SrtWord
,' ',' ') --32+1 Spaces
,' ',' ') --16+1 Spaces
,' ',' ') -- 8+1 Spaces
,' ',' ') -- 4+1 Spaces
,' ',' ') -- 2+1 Spaces
,' ',' ') -- 1+1 Spaces
,' ',' ') -- 1+1 Spaces
FROM #TestTable
WHERE CHARINDEX(' ',SrtWord) > 0
SET STATISTICS TIME OFF
PRINT REPLICATE('=',100)
PRINT '========== Tally UDF method =========='
SET STATISTICS TIME ON
SELECT dbo.CleanUsingTally(SrtWord)
FROM #TestTable
SET STATISTICS TIME OFF
PRINT REPLICATE('=',100)
PRINT '========== Looping UDF method =========='
SET STATISTICS TIME ON
SELECT dbo.fn_CleanUpUsingLooping(SrtWord)
FROM #TestTable
SET STATISTICS TIME OFF
go
DROP TABLE #TestTable
DROP FUNCTION dbo.CleanUsingTally
DROP FUNCTION dbo.fn_CleanUpUsingLooping
C# Gnu
____________________________________________________
November 29, 2009 at 6:22 am
Oh but if I switch current database from Master to 'Another' database I get very differerent results:
====================================================================================================
========== Mitesh's method ==========
SQL Server Execution Times:
CPU time = 26052 ms, elapsed time = 28192 ms.
====================================================================================================
========== Method from the article ==========
SQL Server Execution Times:
CPU time = 1201 ms, elapsed time = 1242 ms.
====================================================================================================
========== Michael's method ==========
SQL Server Execution Times:
CPU time = 141 ms, elapsed time = 139 ms.
====================================================================================================
========== Tally UDF method ==========
SQL Server Execution Times:
CPU time = 7160 ms, elapsed time = 7733 ms.
====================================================================================================
========== Looping UDF method ==========
SQL Server Execution Times:
CPU time = 11435 ms, elapsed time = 12161 ms.
C# Gnu
____________________________________________________
November 29, 2009 at 11:08 am
C# Screw (11/27/2009)
Must be some hardware having influence on this, or is this being affected by collation..Pro Duo, 2gb ram, 700MB free, SQL 2005
Looks like I didn't lose the code after all. I've been sick and under the influence of some pretty good cough syrup... I thought I lost that code. Heh... I'm having trouble keeping up with most postings, as well. My mind's eye is enjoying the colors more than the code. 😛
Yeah, there's something wierd going on there... the method from the article performed a whole lot better on my machine than yours. I'm using the SQL_Latin1_General_CP1_CI_AS collation (default from installation) on 2k5 sp3 on a single 1.8GHz CPU desktop box with 1GB ram over Windows XP sp3. Positive proof of what we've said all along... "It depends" and you have to test even the best written solutions.
This has been a great discussion and I'm proud of everyone that has participated because there were no food fights as what typically happens on such a long thread with so many personalities involved. Very well done, one and all. I really appreciate the tests that you've run.
Heh... Lesson learned for me... Unfortunately, I violated my own rule of testing before I published. I made the rather silly assumption that 3 nested REPLACEs would be faster than 7 or 8 like in Michael's good code especially since I didn't account for effeciency of reducing 33 spaces to 1 on the first swipe. I'm also amazed at how much a difference COLLATION made a difference and I'm really happy that Paul brought it up. Looking back on it, I can see why it makes such a difference (background search for like values) and why the Binary collations are so speedy. The good thing about Michael's is that it seems to be rather tolerant of things like collation (not too many translations of the space character in any collation) and there's no chance of using an incorrect "unlikely" character because it doesn't use any.
Let me "read down" and see what else you've posted. I see that you're starting to play with the Tally table... Just a bit of a warning, though... although you can write some very short, very fast code with it aqnd it can do a lot of wonderful things, it's not a panacea. I don't believe we'll be able to come up with anything using a Tally table that will touch what Micheal's T-SQL code does. Heh... but I've also been wrong before. 😛
--Jeff Moden
Change is inevitable... Change for the better is not.
November 29, 2009 at 11:14 am
C# Screw (11/29/2009)
Oh but if I switch current database from Master to 'Another' database I get very differerent results:
Ah ha... I wonder how many other folks have the default DB as MASTER instead of TempDB? That's one of the first things I change on a new or a legacy system that I've inherited so that people's "stuff" don't end up littering the MASTER DB.
Any idea what the differences are between your MASTER DB and the "Another" DB are on your box? It could be a very important performance tip for other things and not just this bit of code.
--Jeff Moden
Change is inevitable... Change for the better is not.
November 29, 2009 at 5:07 pm
Hi there
I thought I would have a go writing some SQL to list database properties:
here is the result (I have underlined the differences I have spotted):
Property Master ChallinorsDev_CARE II tempdb
Collation Latin1_General_CI_AS SQL_Latin1_General_CP1_CI_AS Latin1_General_CI_AS
ComparisonStyle 196609 196609 196609
IsAnsiNullDefault 0 0 0
IsAnsiNullsEnabled 0 0 0
IsAnsiPaddingEnabled 0 0 0
IsAnsiWarningsEnabled 0 0 0
IsArithmeticAbortEnabled 0 0 0
IsAutoClose 0 0 0
IsAutoCreateStatistics 1 1 1
IsAutoShrink 0 0 0
IsAutoUpdateStatistics 1 1 1
IsCloseCursorsOnCommitEnabled 0 0 0
IsFulltextEnabled 0 1 0
IsInStandBy 0 0 0
IsLocalCursorsDefault 0 0 0
IsMergePublished 0 0 0
IsNullConcat 0 0 0
IsNumericRoundAbortEnabled 0 0 0
IsParameterizationForced 0 0 0
IsPublished 0 0 0
IsQuotedIdentifiersEnabled 0 0 0
IsRecursiveTriggersEnabled 0 0 0
IsSubscribed 0 0 0
IsSyncWithBackup 0 0 0
IsTornPageDetectionEnabled 0 0 0
LCID 1033 1033 1033
Recovery SIMPLE FULL SIMPLE
SQLSortOrder 0 52 0
Status ONLINE ONLINE ONLINE
Updateability READ_WRITE READ_WRITE READ_WRITE
UserAccess MULTI_USER MULTI_USER MULTI_USER
Version 611 611 611
Here is the T-SQL F.Y.I
SET NOCOUNT ON
Go
CREATE FUNCTION [dbo].[fn_Split] (@SplitString varChar(8000), @SplitBy VarChar(30))
RETURNS @tParts TABLE (PartOrder Int, Part VarChar(8000))
AS
--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Description:Split string, split by the @SplitBy seperator character(s)
--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Change History
-- WhenWhoWhat
-- 17/10/2008R.BrigzyInitial
--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BEGIN
DECLARE @part VARCHAR(8000), @i Int, @PartOrder Int
--CREATE TABLE #tResult (PartOrder Int, Part VarChar(8000))
-- if split by is multi char : replace each occurance with tilde ~
SET @SplitString = REPLACE(@SplitString, @SplitBy, '~')
-- Spin through string - insert into result set when hitting tilde
SET @i = 1
SET @Part = ''
SET @PartOrder = 1
WHILE @i <= LEN(@SplitString)
BEGIN
IF SUBSTRING(@SplitString,@i,1) <> '~'
SET @Part = @Part + SUBSTRING(@SplitString,@i,1)
ELSE
BEGIN
INSERT INTO @tParts (partorder, part) VALUES (@PartOrder, @Part)
SET @Part = ''
SET @PartOrder = @PartOrder +1
END
SET @i = @i + 1
END
--Get last part
INSERT INTO @tParts (partorder, part) VALUES (@PartOrder, @Part)
-- All done
RETURN
END
Go
--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Generic code to compare database peoperties
-- RBrigzy 29/11/09
-- Define the list of databases we wish to compare
DECLARE @DatabaseList VARCHAR(8000)
SET @DatabaseList = 'master, ChallinorsDev_CARE II, tempdb'
-- Define the database properties we wish compare
DECLARE @PropertyList VARCHAR(8000)
SET @PropertyList = 'Collation, ComparisonStyle, IsAnsiNullDefault, IsAnsiNullsEnabled, IsAnsiPaddingEnabled, IsAnsiWarningsEnabled,'+
'IsArithmeticAbortEnabled, IsAutoClose, IsAutoCreateStatistics,IsAutoShrink,IsAutoUpdateStatistics, IsCloseCursorsOnCommitEnabled,'+
'IsFulltextEnabled, IsInStandBy, IsLocalCursorsDefault, IsMergePublished,IsNullConcat,IsNumericRoundAbortEnabled,IsParameterizationForced,' +
'IsQuotedIdentifiersEnabled,IsPublished,IsRecursiveTriggersEnabled,IsSubscribed,IsSyncWithBackup,IsTornPageDetectionEnabled,LCID,'+
'Recovery, SQLSortOrder, Status, Updateability, UserAccess, Version'
-- Get property and databse names into temp tables
SELECT PartOrder DBOrder, LTRIM(Part) DBName
INTO #tDatabases
FROM dbo.fn_Split(@DatabaseList,',')
SELECT PartOrder PropertyOrder, LTRIM(Part) [Property]
INTO #tProperties
FROM dbo.fn_Split(@PropertyList,',')
-- Get all property values for all databases
SELECT *,DATABASEPROPERTYEX(DBName,[Property]) PropertyValue
INTO #tDatabaseProperties
FROM #tDatabases, #tProperties
SELECT LEFT(Property+SPACE(30),30) Property,
LEFT(CAST([Master] AS VARCHAR(100))+ SPACE(30),30) [Master],
LEFT(CAST([ChallinorsDev_CARE II]AS VARCHAR(100))+ SPACE(30),30) [ChallinorsDev_CARE II],
LEFT(CAST([tempdb]AS VARCHAR(100))+ SPACE(30),30) [tempdb]
FROM
(SELECT DBName, Property, PropertyValue
FROM #tDatabaseProperties ) pv
PIVOT
(
MAX(PropertyValue)
FOR DBName IN
([Master], [ChallinorsDev_CARE II], [tempdb])
) AS pvt
-- Get Matches
go
-- Cleanup
DROP TABLE #tDatabases
DROP TABLE #tProperties
DROP TABLE #tDatabaseProperties
DROP FUNCTION dbo.[fn_Split]
C# Gnu
____________________________________________________
November 29, 2009 at 5:40 pm
The collation and sort order would certainly explain the performance differences. All of mine (including Master and TempDB) are setup using the installation default of SQL_Latin1_General_CP1_CI_AS. You might want to change yours especially on TempDB... could make a world of difference in overall performance.
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 15 posts - 301 through 315 (of 425 total)
You must be logged in to reply to this topic. Login to reply