July 17, 2011 at 6:26 am
Hi Bob (Dixie etc)
Re:
If anyone would post up some examples where a cursor solution outperforms a set-based solution, I will look at them respectfully.
Here's an interesting discussion I encountered recently about the appropriateness of cursors compared to set based approaches for a problem that involved removing all the non alpha characters in each row of a given column.
http://ask.sqlservercentral.com/questions/75404/strip-all-but-alpha-chars-out-of-a-string
One poster was adamant that looping was the way to go. Another (Magnus Ahlkvist) came up with an elegant non-loop solution but in the end, being an in-line function, it was arguably still rbar and evidently didn't perform much better than a loop. In the end I didn't pursue it because I realised that I didn't need to do what I had originally asked for help on. It does though, remain a nagging instance of where our loopy friends seem to have put up a pretty good fight.
Cheers
...One of the symptoms of an approaching nervous breakdown is the belief that ones work is terribly important.... Bertrand Russell
July 17, 2011 at 11:37 am
GPO (7/17/2011)
Hi Bob (Dixie etc)Re:
If anyone would post up some examples where a cursor solution outperforms a set-based solution, I will look at them respectfully.
Here's an interesting discussion I encountered recently about the appropriateness of cursors compared to set based approaches for a problem that involved removing all the non alpha characters in each row of a given column.
http://ask.sqlservercentral.com/questions/75404/strip-all-but-alpha-chars-out-of-a-string
One poster was adamant that looping was the way to go. Another (Magnus Ahlkvist) came up with an elegant non-loop solution but in the end, being an in-line function, it was arguably still rbar and evidently didn't perform much better than a loop. In the end I didn't pursue it because I realised that I didn't need to do what I had originally asked for help on. It does though, remain a nagging instance of where our loopy friends seem to have put up a pretty good fight.
Cheers
Heh... Looks like I'm going to have to "politely" make a couple of suggestions on that particular link my own self. People keep using the Tally Table the same cruddy way they would use an actual loop. They're just not thinking right. Just because someone has used a Tally Table, it doesn't mean they're using it correctly. As I've said many times, "Before you can think outside the box, you must FIRST realize... YOU'RE IN A FREAKIN' BOX!!!" 😀
It'll take me a couple of hours but... one lesson in "set based" thinking coming right up. Now... where did I put my favorite Pork Chop Launcher? 😉
--Jeff Moden
Change is inevitable... Change for the better is not.
July 17, 2011 at 11:45 am
Go Jeff... Go Jeff...
Incidentally, I don't get the pork chop reference.
...One of the symptoms of an approaching nervous breakdown is the belief that ones work is terribly important.... Bertrand Russell
July 17, 2011 at 4:54 pm
The Dixie Flatline (7/11/2011)
If anyone would post up some examples where a cursor solution outperforms a set-based solution, I will look at them respectfully.
Bob
This brought back memories of a thread a while back where two WHILE loop based solutions were ahead of the pack....
I think SwePeso's ended up being the fastest (as usual 😀 ) using a WHILE loop.
http://www.sqlservercentral.com/Forums/Topic1049892-392-2.aspx
I don't think that much effort was put into a set based solution, and in the end the OP went with CLR, but certainly the loops were doing well.
MM
select geometry::STGeomFromWKB(0x0106000000020000000103000000010000000B0000001000000000000840000000000000003DD8CCCCCCCCCC0840000000000000003DD8CCCCCCCCCC08408014AE47E17AFC3F040000000000104000CDCCCCCCCCEC3F9C999999999913408014AE47E17AFC3F9C99999999991340000000000000003D0000000000001440000000000000003D000000000000144000000000000000400400000000001040000000000000F03F100000000000084000000000000000401000000000000840000000000000003D0103000000010000000B000000000000000000143D000000000000003D009E99999999B93F000000000000003D009E99999999B93F8014AE47E17AFC3F400000000000F03F00CDCCCCCCCCEC3FA06666666666FE3F8014AE47E17AFC3FA06666666666FE3F000000000000003D1800000000000040000000000000003D18000000000000400000000000000040400000000000F03F000000000000F03F000000000000143D0000000000000040000000000000143D000000000000003D, 0);
July 17, 2011 at 6:57 pm
mister.magoo (7/17/2011)
The Dixie Flatline (7/11/2011)
If anyone would post up some examples where a cursor solution outperforms a set-based solution, I will look at them respectfully.
Bob
This brought back memories of a thread a while back where two WHILE loop based solutions were ahead of the pack....
I think SwePeso's ended up being the fastest (as usual 😀 ) using a WHILE loop.
http://www.sqlservercentral.com/Forums/Topic1049892-392-2.aspx
I don't think that much effort was put into a set based solution, and in the end the OP went with CLR, but certainly the loops were doing well.
I agree. There are instances where a While Loop will blow the doors off of set based solutions but such instances are a bit like finding hen's teeth... unless you still think that Yield signs are Yellow. 😉
--Jeff Moden
Change is inevitable... Change for the better is not.
July 17, 2011 at 7:10 pm
GPO (7/17/2011)
Go Jeff... Go Jeff...Incidentally, I don't get the pork chop reference.
Ok... here's my rather long winded, high velocity Pork Chop reply...
The "Pork Chop" reference comes from a much older post that I've actually lost track of. Some manager was whining about how to control one of his "developers" who was riding rough-shod over company programming standards and violating most good practices, as well. To make a really long story shorter, I tried coaching the manager and every suggestion I offered was met with the equivalent of "I can't... I don't actually have a spine" type of reply. I got fed up and ended the thread with a suggestion something like the following...
You need to talk with the "developer" one-on-one. Take him out to dinner to get his attention... a nice "Pork Chop" dinner. Tie him to the chair and feed him the Pork Chops... at point-blank range with a "Wrist Rocket".
In case you don't get that reference, a "Wrist Rocket" is a type of very, very powerful sling shot. 😉
--Jeff Moden
Change is inevitable... Change for the better is not.
July 17, 2011 at 7:15 pm
LutzM (7/11/2011)
It's the same like arguing if a horse will be the better choice than a mule in terms of average speed over a long distance when there's a motorbike available. 😉
BWAA-HAAA!!!! I didn't see THAT before. Now THAT's some funny stuff! Well done, Lutz! 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
July 18, 2011 at 5:28 am
Jeff Moden (7/17/2011)
It'll take me a couple of hours but...one lesson in "set based" thinking coming right up.
looking forward to that already Jeff
Jeff Moden (7/17/2011)
Now... where did I put my favorite Pork Chop Launcher? 😉
Uh oh, duck everybody :w00t:
-----------------------------------------------------------------------------------------------------------
"Ya can't make an omelette without breaking just a few eggs" 😉
July 18, 2011 at 5:56 am
USE tempdb
GO
Declare @TotalRows int, @minRow int
CREATE TABLE #Temp
( Id int identity(1,1),
Queue_stub uniqueidentifier
)
INSERT INTO #temp
SELECT Queue_Stub
FROM <table_name> WHERE condition
SET @TotalRows=@@Rowcount
SET @minRow=1
--Using loops
DECLARE @queue_stub uniqueidentifier
SET @queue_stub = ''
WHILE @minRow <=@TotalRows
BEGIN
SET @queue_stub = ''
SELECT @queue_stub = queue_Stub FROM #temp with (NOLOCK) WHERE id = @minRow
SELECT @message_body = queue_message
FROM <table_name>
WHERE queue_stub = @queue_stub
IF (@queue_stub is not null and @should_playback = 1 )
BEGIN
EXEC function
@queue_stub = @queue_stub,
@command = @message_body,
@isDebug = 0
END
SET @minRow= @minRow +1
END
July 18, 2011 at 6:29 am
Perry Whittle (7/18/2011)
Jeff Moden (7/17/2011)
It'll take me a couple of hours but...one lesson in "set based" thinking coming right up.looking forward to that already Jeff
Jeff Moden (7/17/2011)
Now... where did I put my favorite Pork Chop Launcher? 😉Uh oh, duck everybody :w00t:
I did it yesterday, Perry. See the following...
You'll be able to tell, right off, that I was a little ticked at the prevailing attitude of a supposedly intelligent poster. I just couldn't believe the words coming out of his "mouth" especially since he forgot to include a Clustered Index on the Tally Table and his test rig is, ummm... let's say, politely say "Wholly less than adequate".
--Jeff Moden
Change is inevitable... Change for the better is not.
July 18, 2011 at 6:33 am
kritika (7/18/2011)
USE tempdbGO
Declare @TotalRows int, @minRow int
CREATE TABLE #Temp
( Id int identity(1,1),
Queue_stub uniqueidentifier
)
INSERT INTO #temp
SELECT Queue_Stub
FROM <table_name> WHERE condition
SET @TotalRows=@@Rowcount
SET @minRow=1
--Using loops
DECLARE @queue_stub uniqueidentifier
SET @queue_stub = ''
WHILE @minRow <=@TotalRows
BEGIN
SET @queue_stub = ''
SELECT @queue_stub = queue_Stub FROM #temp with (NOLOCK) WHERE id = @minRow
SELECT @message_body = queue_message
FROM <table_name>
WHERE queue_stub = @queue_stub
IF (@queue_stub is not null and @should_playback = 1 )
BEGIN
EXEC function
@queue_stub = @queue_stub,
@command = @message_body,
@isDebug = 0
END
SET @minRow= @minRow +1
END
Thanks for posting your solution but we still don't know what your function does nor why it requires RBAR. Any chance of you posting the code for the function?
--Jeff Moden
Change is inevitable... Change for the better is not.
July 18, 2011 at 7:05 am
well thats a pretty long SP for this....
it basically processes the stuck elements in the queue.
confidential types soorrryyyyy 🙁
July 18, 2011 at 8:41 am
I understand the relucatance to use loops in order to do set-based processing, but lets say you had a table which had a million records in it, and which was currently being used in production. Let's say you were investigating a bug which had been lurking about for months, and you finally found the solution.
Now, let's say the solution required you to update a batch of records from the table, based on the results of a query.
You could write the query as an UPDATE
SET [FIELDS] = [VALUES] WHERE [ROWS] = [FILTERS]. Correct me if I'm wrong though, but doing this would cause the rows being updated to all be locked until the update was finished.
Any other updates that are being done on those records would be locked out until the whole transaction was finished. Same with any SELECT statements that were not written with NOLOCK hints.
If this update was somewhat intensive, and took an hour or so to process, lets say because the query required to find those records was an expensive one, would you not agree that in this case a row-by-row update would be superior, as, despite the fact that it may take much longer to complete, it only locks one record at a time?
July 18, 2011 at 12:23 pm
kramaswamy (7/18/2011)
@Jeff,I understand the relucatance to use loops in order to do set-based processing, but lets say you had a table which had a million records in it, and which was currently being used in production. Let's say you were investigating a bug which had been lurking about for months, and you finally found the solution.
Now, let's say the solution required you to update a batch of records from the table, based on the results of a query.
You could write the query as an UPDATE
SET [FIELDS] = [VALUES] WHERE [ROWS] = [FILTERS]. Correct me if I'm wrong though, but doing this would cause the rows being updated to all be locked until the update was finished.
Any other updates that are being done on those records would be locked out until the whole transaction was finished. Same with any SELECT statements that were not written with NOLOCK hints.
If this update was somewhat intensive, and took an hour or so to process, lets say because the query required to find those records was an expensive one, would you not agree that in this case a row-by-row update would be superior, as, despite the fact that it may take much longer to complete, it only locks one record at a time?
Although I don't speak for Jeff in this matter, I would say "it depends" (to coin a cliche`). If the query to identify rows to be updated was really expensive, it might be a good idea to use that query to populate a temp table with the keys of the rows in question. Then the update itself would be independent of the expensive query. It might also be a good idea to run the update during a maintenance window (if available) when concurrent activity would be minimized.
But even if we all agree that the superior solution in your example is to minimize locking, rather than to minimize query speed, that isn't the point. Nobody is arguing that procedural solutions are always evil, just that set-based solutions are generally faster and usually preferable. Do exceptions exist? Of course they do. The problem is when someone looks at an exception and tries to turn it into the general rule. Lots of procedural-thinking developers choose to use a while loop because it is a familiar construct, and because they don't understand the performance implications.
Yes, there are cases where a developer might be doing a little ad-hoc exploration and the runtime of a while loop doesn't justify the additional development time it would take to come up with a set based solution. But if that developer never gets in the habit of non-procedural thinking, he or she is more likely to use procedural solutions when they are inappropriate.
One of the great things about SSC is that you often see competing solutions being tested, until one proves itself to be faster than the others. Set based solutions are usually the winners.
__________________________________________________
Against stupidity the gods themselves contend in vain. -- Friedrich Schiller
Stop, children, what's that sound? Everybody look what's going down. -- Stephen Stills
July 18, 2011 at 10:44 pm
kramaswamy (7/18/2011)
If this update was somewhat intensive, and took an hour or so to process, lets say because the query required to find those records was an expensive one, would you not agree that in this case a row-by-row update would be superior, as, despite the fact that it may take much longer to complete, it only locks one record at a time?
Probably not and, no, it wouldn't be a problem. If it takes a while to find the rows to update, I darned sure wouldn't accomplish that task during the update. And, no... even though I've had to update many table with millions of rows, it's never taken an hour.
You are correct... well, partially. Every system has a "tipping point" for updates. For the larger systems, it's usually more than a couple of million rows before a system loses its mind. So, yes, I'll sometimes use a loop to do updates... 500,000 or a million rows at a time. 😉 I'll also tell you that, for such voluminous updates, I normally don't have to worry about any contention because I'll normally snap a TABLOCKX on the table for the second or two I'm in it. 😉 It may take 30 minutes waiting for the TABLOCKX to take effect, but it's just sitting there... waiting for the right moment to pounce and then WHAM! Lock the table, do my thing, and get out... usually in less than a second depending on what needs to be done.
I've also a similar thing for large volume deletes...
And, now that I look back, I see Dixie responded in a similar fashion. 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 15 posts - 16 through 29 (of 29 total)
You must be logged in to reply to this topic. Login to reply