loops

  • Hi Bob (Dixie etc)

    Re:

    If anyone would post up some examples where a cursor solution outperforms a set-based solution, I will look at them respectfully.

    Here's an interesting discussion I encountered recently about the appropriateness of cursors compared to set based approaches for a problem that involved removing all the non alpha characters in each row of a given column.

    http://ask.sqlservercentral.com/questions/75404/strip-all-but-alpha-chars-out-of-a-string

    One poster was adamant that looping was the way to go. Another (Magnus Ahlkvist) came up with an elegant non-loop solution but in the end, being an in-line function, it was arguably still rbar and evidently didn't perform much better than a loop. In the end I didn't pursue it because I realised that I didn't need to do what I had originally asked for help on. It does though, remain a nagging instance of where our loopy friends seem to have put up a pretty good fight.

    Cheers

    ...One of the symptoms of an approaching nervous breakdown is the belief that ones work is terribly important.... Bertrand Russell

  • GPO (7/17/2011)


    Hi Bob (Dixie etc)

    Re:

    If anyone would post up some examples where a cursor solution outperforms a set-based solution, I will look at them respectfully.

    Here's an interesting discussion I encountered recently about the appropriateness of cursors compared to set based approaches for a problem that involved removing all the non alpha characters in each row of a given column.

    http://ask.sqlservercentral.com/questions/75404/strip-all-but-alpha-chars-out-of-a-string

    One poster was adamant that looping was the way to go. Another (Magnus Ahlkvist) came up with an elegant non-loop solution but in the end, being an in-line function, it was arguably still rbar and evidently didn't perform much better than a loop. In the end I didn't pursue it because I realised that I didn't need to do what I had originally asked for help on. It does though, remain a nagging instance of where our loopy friends seem to have put up a pretty good fight.

    Cheers

    Heh... Looks like I'm going to have to "politely" make a couple of suggestions on that particular link my own self. People keep using the Tally Table the same cruddy way they would use an actual loop. They're just not thinking right. Just because someone has used a Tally Table, it doesn't mean they're using it correctly. As I've said many times, "Before you can think outside the box, you must FIRST realize... YOU'RE IN A FREAKIN' BOX!!!" 😀

    It'll take me a couple of hours but... one lesson in "set based" thinking coming right up. Now... where did I put my favorite Pork Chop Launcher? 😉

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Go Jeff... Go Jeff...

    Incidentally, I don't get the pork chop reference.

    ...One of the symptoms of an approaching nervous breakdown is the belief that ones work is terribly important.... Bertrand Russell

  • The Dixie Flatline (7/11/2011)


    If anyone would post up some examples where a cursor solution outperforms a set-based solution, I will look at them respectfully.

    Bob

    This brought back memories of a thread a while back where two WHILE loop based solutions were ahead of the pack....

    I think SwePeso's ended up being the fastest (as usual 😀 ) using a WHILE loop.

    http://www.sqlservercentral.com/Forums/Topic1049892-392-2.aspx

    I don't think that much effort was put into a set based solution, and in the end the OP went with CLR, but certainly the loops were doing well.

    MM



    select geometry::STGeomFromWKB(0x0106000000020000000103000000010000000B0000001000000000000840000000000000003DD8CCCCCCCCCC0840000000000000003DD8CCCCCCCCCC08408014AE47E17AFC3F040000000000104000CDCCCCCCCCEC3F9C999999999913408014AE47E17AFC3F9C99999999991340000000000000003D0000000000001440000000000000003D000000000000144000000000000000400400000000001040000000000000F03F100000000000084000000000000000401000000000000840000000000000003D0103000000010000000B000000000000000000143D000000000000003D009E99999999B93F000000000000003D009E99999999B93F8014AE47E17AFC3F400000000000F03F00CDCCCCCCCCEC3FA06666666666FE3F8014AE47E17AFC3FA06666666666FE3F000000000000003D1800000000000040000000000000003D18000000000000400000000000000040400000000000F03F000000000000F03F000000000000143D0000000000000040000000000000143D000000000000003D, 0);

  • Forum Etiquette: How to post Reporting Services problems
  • [/url]
  • Forum Etiquette: How to post data/code on a forum to get the best help - by Jeff Moden
  • [/url]
  • How to Post Performance Problems - by Gail Shaw
  • [/url]

  • mister.magoo (7/17/2011)


    The Dixie Flatline (7/11/2011)


    If anyone would post up some examples where a cursor solution outperforms a set-based solution, I will look at them respectfully.

    Bob

    This brought back memories of a thread a while back where two WHILE loop based solutions were ahead of the pack....

    I think SwePeso's ended up being the fastest (as usual 😀 ) using a WHILE loop.

    http://www.sqlservercentral.com/Forums/Topic1049892-392-2.aspx

    I don't think that much effort was put into a set based solution, and in the end the OP went with CLR, but certainly the loops were doing well.

    I agree. There are instances where a While Loop will blow the doors off of set based solutions but such instances are a bit like finding hen's teeth... unless you still think that Yield signs are Yellow. 😉

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • GPO (7/17/2011)


    Go Jeff... Go Jeff...

    Incidentally, I don't get the pork chop reference.

    Ok... here's my rather long winded, high velocity Pork Chop reply...

    http://ask.sqlservercentral.com/questions/75404/strip-all-but-alpha-chars-out-of-a-string?page=1#76191

    The "Pork Chop" reference comes from a much older post that I've actually lost track of. Some manager was whining about how to control one of his "developers" who was riding rough-shod over company programming standards and violating most good practices, as well. To make a really long story shorter, I tried coaching the manager and every suggestion I offered was met with the equivalent of "I can't... I don't actually have a spine" type of reply. I got fed up and ended the thread with a suggestion something like the following...

    You need to talk with the "developer" one-on-one. Take him out to dinner to get his attention... a nice "Pork Chop" dinner. Tie him to the chair and feed him the Pork Chops... at point-blank range with a "Wrist Rocket".

    In case you don't get that reference, a "Wrist Rocket" is a type of very, very powerful sling shot. 😉

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • LutzM (7/11/2011)


    It's the same like arguing if a horse will be the better choice than a mule in terms of average speed over a long distance when there's a motorbike available. 😉

    BWAA-HAAA!!!! I didn't see THAT before. Now THAT's some funny stuff! Well done, Lutz! 😀

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Jeff Moden (7/17/2011)


    It'll take me a couple of hours but...one lesson in "set based" thinking coming right up.

    looking forward to that already Jeff

    Jeff Moden (7/17/2011)


    Now... where did I put my favorite Pork Chop Launcher? 😉

    Uh oh, duck everybody :w00t:

    -----------------------------------------------------------------------------------------------------------

    "Ya can't make an omelette without breaking just a few eggs" 😉

  • USE tempdb

    GO

    Declare @TotalRows int, @minRow int

    CREATE TABLE #Temp

    ( Id int identity(1,1),

    Queue_stub uniqueidentifier

    )

    INSERT INTO #temp

    SELECT Queue_Stub

    FROM <table_name> WHERE condition

    SET @TotalRows=@@Rowcount

    SET @minRow=1

    --Using loops

    DECLARE @queue_stub uniqueidentifier

    SET @queue_stub = ''

    WHILE @minRow <=@TotalRows

    BEGIN

    SET @queue_stub = ''

    SELECT @queue_stub = queue_Stub FROM #temp with (NOLOCK) WHERE id = @minRow

    SELECT @message_body = queue_message

    FROM <table_name>

    WHERE queue_stub = @queue_stub

    IF (@queue_stub is not null and @should_playback = 1 )

    BEGIN

    EXEC function

    @queue_stub = @queue_stub,

    @command = @message_body,

    @isDebug = 0

    END

    SET @minRow= @minRow +1

    END

  • Perry Whittle (7/18/2011)


    Jeff Moden (7/17/2011)


    It'll take me a couple of hours but...one lesson in "set based" thinking coming right up.

    looking forward to that already Jeff

    Jeff Moden (7/17/2011)


    Now... where did I put my favorite Pork Chop Launcher? 😉

    Uh oh, duck everybody :w00t:

    I did it yesterday, Perry. See the following...

    http://ask.sqlservercentral.com/questions/75404/strip-all-but-alpha-chars-out-of-a-string?page=1#76191

    You'll be able to tell, right off, that I was a little ticked at the prevailing attitude of a supposedly intelligent poster. I just couldn't believe the words coming out of his "mouth" especially since he forgot to include a Clustered Index on the Tally Table and his test rig is, ummm... let's say, politely say "Wholly less than adequate".

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • kritika (7/18/2011)


    USE tempdb

    GO

    Declare @TotalRows int, @minRow int

    CREATE TABLE #Temp

    ( Id int identity(1,1),

    Queue_stub uniqueidentifier

    )

    INSERT INTO #temp

    SELECT Queue_Stub

    FROM <table_name> WHERE condition

    SET @TotalRows=@@Rowcount

    SET @minRow=1

    --Using loops

    DECLARE @queue_stub uniqueidentifier

    SET @queue_stub = ''

    WHILE @minRow <=@TotalRows

    BEGIN

    SET @queue_stub = ''

    SELECT @queue_stub = queue_Stub FROM #temp with (NOLOCK) WHERE id = @minRow

    SELECT @message_body = queue_message

    FROM <table_name>

    WHERE queue_stub = @queue_stub

    IF (@queue_stub is not null and @should_playback = 1 )

    BEGIN

    EXEC function

    @queue_stub = @queue_stub,

    @command = @message_body,

    @isDebug = 0

    END

    SET @minRow= @minRow +1

    END

    Thanks for posting your solution but we still don't know what your function does nor why it requires RBAR. Any chance of you posting the code for the function?

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • well thats a pretty long SP for this....

    it basically processes the stuck elements in the queue.

    confidential types soorrryyyyy 🙁

  • @jeff,

    I understand the relucatance to use loops in order to do set-based processing, but lets say you had a table which had a million records in it, and which was currently being used in production. Let's say you were investigating a bug which had been lurking about for months, and you finally found the solution.

    Now, let's say the solution required you to update a batch of records from the table, based on the results of a query.

    You could write the query as an UPDATE

    SET [FIELDS] = [VALUES] WHERE [ROWS] = [FILTERS]. Correct me if I'm wrong though, but doing this would cause the rows being updated to all be locked until the update was finished.

    Any other updates that are being done on those records would be locked out until the whole transaction was finished. Same with any SELECT statements that were not written with NOLOCK hints.

    If this update was somewhat intensive, and took an hour or so to process, lets say because the query required to find those records was an expensive one, would you not agree that in this case a row-by-row update would be superior, as, despite the fact that it may take much longer to complete, it only locks one record at a time?

  • kramaswamy (7/18/2011)


    @Jeff,

    I understand the relucatance to use loops in order to do set-based processing, but lets say you had a table which had a million records in it, and which was currently being used in production. Let's say you were investigating a bug which had been lurking about for months, and you finally found the solution.

    Now, let's say the solution required you to update a batch of records from the table, based on the results of a query.

    You could write the query as an UPDATE

    SET [FIELDS] = [VALUES] WHERE [ROWS] = [FILTERS]. Correct me if I'm wrong though, but doing this would cause the rows being updated to all be locked until the update was finished.

    Any other updates that are being done on those records would be locked out until the whole transaction was finished. Same with any SELECT statements that were not written with NOLOCK hints.

    If this update was somewhat intensive, and took an hour or so to process, lets say because the query required to find those records was an expensive one, would you not agree that in this case a row-by-row update would be superior, as, despite the fact that it may take much longer to complete, it only locks one record at a time?

    Although I don't speak for Jeff in this matter, I would say "it depends" (to coin a cliche`). If the query to identify rows to be updated was really expensive, it might be a good idea to use that query to populate a temp table with the keys of the rows in question. Then the update itself would be independent of the expensive query. It might also be a good idea to run the update during a maintenance window (if available) when concurrent activity would be minimized.

    But even if we all agree that the superior solution in your example is to minimize locking, rather than to minimize query speed, that isn't the point. Nobody is arguing that procedural solutions are always evil, just that set-based solutions are generally faster and usually preferable. Do exceptions exist? Of course they do. The problem is when someone looks at an exception and tries to turn it into the general rule. Lots of procedural-thinking developers choose to use a while loop because it is a familiar construct, and because they don't understand the performance implications.

    Yes, there are cases where a developer might be doing a little ad-hoc exploration and the runtime of a while loop doesn't justify the additional development time it would take to come up with a set based solution. But if that developer never gets in the habit of non-procedural thinking, he or she is more likely to use procedural solutions when they are inappropriate.

    One of the great things about SSC is that you often see competing solutions being tested, until one proves itself to be faster than the others. Set based solutions are usually the winners.

    __________________________________________________

    Against stupidity the gods themselves contend in vain. -- Friedrich Schiller
    Stop, children, what's that sound? Everybody look what's going down. -- Stephen Stills

  • kramaswamy (7/18/2011)


    If this update was somewhat intensive, and took an hour or so to process, lets say because the query required to find those records was an expensive one, would you not agree that in this case a row-by-row update would be superior, as, despite the fact that it may take much longer to complete, it only locks one record at a time?

    Probably not and, no, it wouldn't be a problem. If it takes a while to find the rows to update, I darned sure wouldn't accomplish that task during the update. And, no... even though I've had to update many table with millions of rows, it's never taken an hour.

    You are correct... well, partially. Every system has a "tipping point" for updates. For the larger systems, it's usually more than a couple of million rows before a system loses its mind. So, yes, I'll sometimes use a loop to do updates... 500,000 or a million rows at a time. 😉 I'll also tell you that, for such voluminous updates, I normally don't have to worry about any contention because I'll normally snap a TABLOCKX on the table for the second or two I'm in it. 😉 It may take 30 minutes waiting for the TABLOCKX to take effect, but it's just sitting there... waiting for the right moment to pounce and then WHAM! Lock the table, do my thing, and get out... usually in less than a second depending on what needs to be done.

    I've also a similar thing for large volume deletes...

    And, now that I look back, I see Dixie responded in a similar fashion. 😀

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Viewing 15 posts - 16 through 29 (of 29 total)

    You must be logged in to reply to this topic. Login to reply